Image processing method, device, electronic device and storage medium

Through the target CNN model, sensitive images are identified and processed, heat maps are generated and desensitized, which solves the problem of sensitive images recognition and processing in image sharing, and improves the quality and integrity of information sharing.

CN113554059BActive Publication Date: 2024-11-29BEIJING DAJIA INTERNET INFORMATION TECH CO LTD

Patent Information

Application Number
CN202110701731.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-06-23
Publication Date
2024-11-29
Estimated Expiration
2041-06-23

AI Technical Summary

Technical Problem

During the image sharing process, it is difficult for the existing technology to effectively identify and process sensitive images, resulting in low quality of information sharing, and directly removing sensitive images will lead to information loss.

Method used

The target convolutional neural network (CNN) model is used to identify sensitive images through the training set, and generate heat maps, desensitize sensitive image areas, and retain information of non-sensitive areas.

Benefits of technology

It improves the quality and integrity of information sharing, avoids information loss, and improves the efficiency of identifying and processing sensitive images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113554059B_ABST
    Figure CN113554059B_ABST
Patent Text Reader

Abstract

The present disclosure discloses a method, device, electronic device and storage medium for image processing, which belongs to the field of image recognition technology, including: when a picture to be recognized is obtained, the picture to be recognized is input into a target convolutional neural network (CNN) model, the target CNN network model is obtained by training a first training set, the target CNN network model is used to identify whether the input picture is a sensitive picture, and the first training set includes picture samples marked with whether it is a sensitive picture; when the target CNN network model identifies the picture to be recognized as a sensitive picture, a heat map of the picture to be recognized generated by the target CNN network model is obtained, and the heat map of the picture to be recognized includes at least one sensitive image area; the image in at least one sensitive image area is desensitized to obtain an output picture. The image processing method, device, electronic device and storage medium provided by the present disclosure are adopted to at least solve the problem of low quality of information sharing in the existing picture sharing process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of image recognition, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of the Internet and wireless communication technologies, people have more and more ways to share information. As a new information sharing function, picture sharing allows users to show their daily lives to others through pictures (such as photos or frames in videos), which improves the convenience and fun of information sharing.

[0003] Among the pictures that people share with each other, there may be some illegal pictures that violate laws or social ethics, such as pictures containing pornographic content (such as images of breasts, buttocks, or triangles) or violent content. In order to prevent the spread of illegal pictures from bringing adverse effects to society, it is usually necessary to manually identify illegal pictures in the pictures being spread and remove the identified illegal pictures. However, the removal of illegal pictures will lead to information loss during the information sharing process, thereby reducing the quality of information sharing between people. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a picture processing method, device, electronic device and storage medium to solve the problem of low quality of information sharing in current picture sharing.

[0005] The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present application, a method for processing an image is provided, including:

[0007] When a picture to be identified is obtained, the picture to be identified is input into a target convolutional neural network (CNN) model, wherein the target CNN network model is obtained by training with a first training set, the target CNN network model is used to identify whether the input picture is a sensitive picture, and the first training set includes picture samples marked with whether they are sensitive pictures;

[0008] In the case where the target CNN network model identifies the image to be identified as a sensitive image, obtaining a heat map of the image to be identified generated by the target CNN network model, wherein the heat map of the image to be identified includes at least one sensitive image area;

[0009] Desensitization processing is performed on the image in the at least one sensitive image area to obtain an output image.

[0010] In one embodiment, when the image to be identified is obtained, before the image to be identified is input into the target CNN model, the method further includes:

[0011] Training the initial CNN model based on the first training set to obtain a first CNN model;

[0012] Inputting a picture of a first picture sample into the first CNN model, and obtaining a first result output by the first CNN model at a target layer, wherein the first picture sample is any picture sample in the first training set, and the target layer includes at least one of an intermediate layer and an output layer;

[0013] The first result is used as a soft target of the second result output by the second CNN model at the target layer, and the model parameters of the second CNN model are iteratively updated to obtain the target CNN model, wherein the second result is the result obtained by inputting the image of the first image sample into the second CNN model.

[0014] In one embodiment, when the image to be identified is obtained, before the image to be identified is input into the target convolutional neural network (CNN) model, the method further includes:

[0015] Acquire a first sample set and a second sample set, wherein the first sample set includes labeled picture samples, and the second sample set includes unlabeled picture samples;

[0016] Inputting the first sample set into the M-1th CNN model, and training to obtain the Mth CNN model, wherein M is a positive integer;

[0017] Inputting the picture samples in the second sample set into the Mth CNN model to annotate the input picture samples, and adding the annotated picture samples to the first sample set;

[0018] In the case that the M CNN models do not meet the training stop condition, using the M CNN model as the M-1 CNN model, and re-implementing the first sample set into the M-1 CNN model;

[0019] When the M CNN models meet the training stop condition, the sample set is used as the first training set.

[0020] In one embodiment, after using the sample set as the first training set, the method further includes:

[0021] Performing image data enhancement processing on the image samples in the first training set to obtain a third sample set, wherein the image data enhancement processing includes performing at least one of rotation, flipping, scaling, translation, scale transformation, noise perturbation, color transformation, and occlusion on the image;

[0022] The third sample set is added to the first training set to update the first training set.

[0023] In one embodiment, performing desensitization processing on the image in the at least one sensitive image area to obtain an output image includes:

[0024] Determine a target desensitization processing method for the image to be identified among N preset desensitization processing methods, where N is an integer greater than 1;

[0025] Based on the target desensitization processing method, the image in the at least one sensitive image area is desensitized to obtain an output image.

[0026] In one of the embodiments, the N preset desensitization processing methods include setting mosaic, blurring and replacing with a target image, wherein the target image is an image associated with the image in the picture to be identified except for the at least one sensitive image area.

[0027] In one embodiment, determining a target desensitization processing method for the image to be identified among N preset desensitization processing methods includes:

[0028] Obtaining a sensitivity level of at least one sensitive image area in the thermal map;

[0029] Among the N preset desensitization processing methods, determine a target desensitization processing method corresponding to the highest sensitivity level among the sensitivity levels of the at least one sensitive image area.

[0030] According to a second aspect of an embodiment of the present application, there is provided a picture processing device, including:

[0031] The picture input module is configured to input the picture to be identified into a target CNN model when the picture to be identified is obtained, wherein the target CNN network model is trained by a first training set, the target CNN network model is used to identify whether the input picture is a sensitive picture, and the first training set includes picture samples marked as sensitive pictures;

[0032] a heat map acquisition module, configured to acquire a heat map of the image to be identified generated by the target CNN network model when the target CNN network model identifies the image to be identified as a sensitive image, wherein the heat map of the image to be identified includes at least one sensitive image area;

[0033] The image output module is configured to perform desensitization processing on the image in the at least one sensitive image area to obtain an output image.

[0034] In one embodiment, the device further comprises:

[0035] A first training module is configured to train the initial CNN model based on the first training set to obtain a first CNN model;

[0036] A result output module is configured to input a picture of a first picture sample into the first CNN model to obtain a first result output by the first CNN model at a target layer, wherein the first picture sample is any picture sample in the first training set, and the target layer includes at least one of an intermediate layer and an output layer;

[0037] The parameter updating module is configured to use the first result as a soft target of the second result output by the second CNN model at the target layer, iteratively update the model parameters of the second CNN model, and obtain the target CNN model, wherein the second result is the result obtained by inputting the image of the first image sample into the second CNN model.

[0038] In one embodiment, the device further comprises:

[0039] A sample set acquisition module, configured to acquire a first sample set and a second sample set, wherein the first sample set includes labeled picture samples, and the second sample set includes unlabeled picture samples;

[0040] A second training module is configured to input the first sample set into the M-1th CNN model to train and obtain the Mth CNN model, wherein M is a positive integer;

[0041] a labeling module, configured to input the picture samples in the second sample set into the Mth CNN model to label the input picture samples, and add the labeled picture samples to the first sample set;

[0042] A reset module is configured to use the Mth CNN model as the M-1th CNN model and re-execute inputting the first sample set into the M-1th CNN model when the M CNN models do not meet the training stop condition;

[0043] The training set generation module is configured to use the sample set as the first training set when the M CNN models meet the training stop condition.

[0044] In one embodiment, the device further comprises:

[0045] a data enhancement module, configured to perform image data enhancement processing on the image samples in the first training set to obtain a third sample set, wherein the image data enhancement processing includes performing at least one of rotation, flipping transformation, scaling transformation, translation transformation, scale transformation, noise perturbation, color transformation and occlusion on the image;

[0046] The training set updating module is configured to add the third sample set to the first training set to update the first training set.

[0047] In one embodiment, the picture output module includes:

[0048] A processing mode determining unit is configured to determine a target desensitization processing mode of the image to be identified from N preset desensitization processing modes, where N is an integer greater than 1;

[0049] The image output unit is configured to perform desensitization processing on the image in the at least one sensitive image area based on the target desensitization processing method to obtain an output image.

[0050] In one of the embodiments, the N preset desensitization processing methods include setting mosaic, blurring and replacing with a target image, wherein the target image is an image associated with the image in the picture to be identified except for the at least one sensitive image area.

[0051] In one embodiment, the processing mode determination unit includes:

[0052] A sensitivity level acquisition subunit, configured to acquire a sensitivity level of at least one sensitive image area in the thermal map;

[0053] The processing mode subunit is configured to determine, among the N preset desensitization processing modes, a target desensitization processing mode corresponding to the highest sensitivity level among the sensitivity levels of the at least one sensitive image area.

[0054] According to a third aspect of an embodiment of the present application, an electronic device is provided, which may include:

[0055] processor;

[0056] a memory for storing instructions executable by the processor;

[0057] The processor is configured to execute the instructions to implement the image processing method as shown in any one of the embodiments of the first aspect.

[0058] According to the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an image processing device or an electronic device, the image processing device or the electronic device is enabled to implement the image processing method as shown in any one of the embodiments of the first aspect.

[0059] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, including a computer program or instructions, and when the computer program / instructions are executed by a processor, the image processing method shown in any one of the embodiments of the first aspect is implemented.

[0060] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0061] The image to be identified is input into the target CNN model trained by the first training set, the image to be identified is identified by the target CNN model, and a heat map of the image to be identified is generated when the image to be identified is a sensitive image, and finally the image of the sensitive image area in the generated heat map is desensitized, thereby achieving desensitization of part of the image to be identified. In this way, through the embodiment of the present application, compared with directly removing sensitive images, at least the information of the image outside the sensitive image area can be retained, thereby improving the integrity of information in the information sharing process, and then improving the quality of information sharing between people.

[0062] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0064] Figure 1 is a flowchart of a method for processing an image according to an exemplary embodiment;

[0065] Figure 2 is a schematic structural diagram of an initial CNN model according to an exemplary embodiment;

[0066] Figure 3 is a schematic diagram of training a CNN model using distillation technology according to an exemplary embodiment;

[0067] Figure 4is a flowchart of semi-supervised training according to an exemplary embodiment;

[0068] Figure 5 is a schematic diagram of a heat map according to an exemplary embodiment;

[0069] Figure 6 is a structural block diagram of a picture processing device according to an exemplary embodiment;

[0070] Figure 7 It is a structural block diagram of a computing device according to an exemplary embodiment. DETAILED DESCRIPTION

[0071] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0072] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0073] See also Figure 1 , is a flowchart of a method for image processing according to an exemplary embodiment. The method for image processing is applied to an electronic device, such as a computer terminal or a server. For ease of understanding, this solution is described using a server as an example. Figure 1 As shown, the above-mentioned image processing method includes the following steps 101-104.

[0074] Step 101: When a picture to be identified is obtained, the picture to be identified is input into a target convolutional neural network (CNN) model, wherein the target CNN network model is trained by a first training set, and the first training set includes sensitive picture samples marked with sensitive labels.

[0075] It should be noted that the above-mentioned target CNN model can be any deep neural network model that can identify whether the picture to be identified is a sensitive picture, and the above-mentioned target CNN model can generate a heat map of the picture to be identified and mark the picture to be identified with a sensitive label when the picture is identified as a sensitive picture.

[0076] In one embodiment of the present application, the above-mentioned target CNN model can be obtained by training a first training set including image samples marked with labels, and the label is used to indicate whether the image sample is a sensitive image, that is, before inputting the image to be identified into the target convolutional neural network CNN model, the electronic device can also obtain the above-mentioned target CNN model based on the above-mentioned first training set training.

[0077] The target CNN model is obtained by training the first training set, and the target CNN model can be obtained by training a preset initial CNN model with the first training set, and the training process can be as follows:

[0078] The images in the image samples of the first training set are used as the input of the initial CNN model. The loss of the trained CNN model is calculated through the output results of the trained CNN model and the label results of the image samples. The model parameters (such as weights, etc.) of the CNN model are adjusted according to the calculated loss to achieve model iteration. The preset CNN model is iterated repeatedly until the iteration stop condition is reached, such as the loss of the CNN model tends to be unchanged. The CNN model obtained by the final iteration is used as the above-mentioned target CNN model.

[0079] It should be noted that, in order to ensure the recognition accuracy of the target CNN model obtained by training, the number of network layers of the initial CNN model backbone network can be set to be large. In addition, the backbone network can include a network of the Big-transfer or efficientnet series.

[0080] Of course, the target CNN model may be obtained by training based on the first training set, or may be obtained by training in other ways. Specifically, before step 101, the following steps may be further included:

[0081] The initial CNN model is trained based on the first training set to obtain a first CNN model;

[0082] Inputting a picture of a first picture sample into a first CNN model, and obtaining a first result output by the first CNN model in a target layer, wherein the first picture sample is any picture sample in a first training set, and the target layer includes at least one of an intermediate layer and an output layer;

[0083] The first result is used as a soft target of the second result output by the second CNN model in the target layer, and the model parameters of the second CNN model are iteratively updated to obtain a target CNN model, wherein the second result is a result obtained by inputting the image of the first image sample into the second CNN model.

[0084] Based on this, by taking the trained first CNN model as the soft target of the output result of the second CNN model, the knowledge learned by the first CNN model is transferred to the second CNN model, and the above-mentioned target CNN model is trained, thereby making the training method of the target CNN model more flexible and ensuring the recognition accuracy of the target CNN model.

[0085] Among them, the above-mentioned initial CNN model is trained based on the first training set to obtain the first CNN model, that is, the picture of the first training picture sample is used as the input of the initial CNN model, and the model parameters are adjusted according to the output results of the model and the label results of the picture samples, so as to realize repeated training iterations of the initial CNN model and obtain the above-mentioned first CNN model.

[0086] For example, one can construct Figure 2 The figure shows an initial CNN model 21 including a backbone network, and the initial CNN model is trained by a first training set to obtain a first CNN model, which can identify the image type of the input image, such as sensitive (e.g., exposed breasts, exposed buttocks, exposed triangle area) and normal (i.e., insensitive), and mark the input image with corresponding labels.

[0087] In addition, the first CNN model and the second CNN model can be any CNN model that can identify whether a picture is a sensitive picture. The first CNN model and the second CNN model can also generate a heat map of the picture when the picture is identified as a sensitive picture.

[0088] It should be noted that the recognition accuracy of the above-mentioned first CNN model can be higher than that of the second CNN model. In this way, since the output result of the first CNN model is used as a soft target for the output result of the second CNN model, the model parameters of the second CNN model are iteratively updated, so that the output result of the second CNN model after the parameter update gradually approaches the output result of the first CNN model or is consistent with the output result of the first CNN model, thereby realizing the transfer of the knowledge learned by the first CNN model to the second CNN model to obtain the target CNN model.

[0089] In addition, the first CNN model and the second CNN model can be CNN models with the same network architecture; or, the first CNN model and the second CNN model can also be CNN models with different network architectures. Specifically, the number of network layers of the first CNN model can be greater than the number of network layers of the second CNN model, that is, the second CNN model has a faster recognition speed than the first CNN model. By migrating the recognition accuracy of the first CNN model to the second CNN model, when using the target CNN model (that is, the second CNN model finally obtained after iterative updating) for recognition, not only the accuracy of the target CNN model can be guaranteed, but also the recognition speed of the CNN recognition model can be improved.

[0090] For example, assuming that Figure 2 The initial CNN model 21 shown is a CNN model with a large number of network layers. Then, after the initial CNN model 21 is trained by the first training set to obtain a first CNN model, the first CNN model can be used as a teacher model.

[0091] At this time, the CNN model with fewer network layers is used as the student model (i.e., the second CNN model), such as Figure 3 As shown, the image samples are input into the teacher model and the student model respectively, and the output result of the teacher model is used as the soft target of the output result of the student model, so that the student model fits the output result of the teacher model. For example, it can be the result of fitting the middle layer of the teacher model through Hint-learning, that is, the result of the middle layer of the teacher model is used as the result of the middle layer of the student model (that is, the target layer includes the middle layer); and / or, the result of fitting the output layer of the teacher model through knowledge distillation, that is, the result of the output layer of the teacher model is used as the result of the output layer of the student model (that is, the target layer includes the output layer);

[0092] The loss function is used to calculate the simulation loss between the output of the student model and the output of the teacher model. When the simulation loss does not meet the preset conditions (such as the simulation loss is greater than the preset loss), the network parameters of the student model are updated, and the image samples are input into the teacher model and the student model respectively. The output of the teacher model is used as the soft target of the output of the student model, and the student model is used to fit the output of the teacher model until the simulation loss meets the preset conditions and the student model converges. At this time, the output of the student model is close to the output of the teacher model, and the student model is saved (that is, the target CNN model is obtained).

[0093] In one embodiment of the present application, the first training set may include sensitive image samples marked with sensitive labels, and the sensitive labels are used to indicate that the images of the image samples are sensitive images. Of course, to ensure the robustness of the training, the first training set may also include non-sensitive image samples marked with normal labels, and the normal labels are used to indicate that the image samples are not sensitive images, which is not limited here.

[0094] It should be noted that the labeling of each image sample in the above-mentioned first training set can be achieved through manual operation; or, in order to reduce the workload of manual operation in obtaining the first training set, the labeling of some image samples in the above-mentioned first training set can also be achieved through machine labeling.

[0095] Specifically, in the case where the image to be identified is obtained, before the image to be identified is input into the target convolutional neural network CNN model, the following may also be included:

[0096] Acquire a first sample set and a second sample set, wherein the first sample set includes labeled picture samples, and the second sample set includes unlabeled picture samples;

[0097] Input the first sample set into the M-1th CNN model and train the Mth CNN model, where M is a positive integer;

[0098] Input the image samples in the second sample set into the Mth CNN model to annotate the input image samples, and add the annotated image samples to the first sample set;

[0099] When the M CNN models do not meet the training stop condition, the Mth CNN model is used as the M-1th CNN model, and the first sample set is input into the M-1th CNN model again;

[0100] When the M CNN models meet the training stop condition, the first sample set is used as the first training set.

[0101] Based on this, in the process of training the CNN model, the CNN model to be trained is trained by a first sample set including labeled image samples, and the unlabeled image samples are labeled by the trained CNN model, and the labeled samples are added to the first sample set until the trained CNN model meets the training stop condition. The first sample set is used as the above-mentioned training sample set to obtain the training sample set in a semi-supervised manner, thereby reducing the workload of obtaining the training sample set and improving the efficiency of obtaining the training sample set.

[0102] In one embodiment of the present application, the image samples in the first sample set are marked with labels, and the image samples in the second sample set are all marked with labels, and the sample images in the first sample set and the second sample set may both include sensitive images and non-sensitive images.

[0103] For example, among video and picture resources, a batch of suspected vulgar pictures and their video frames (i.e., sensitive pictures) are extracted based on historical review records or user complaint and report records; at the same time, a batch of normal video and picture resource data (i.e., non-sensitive pictures) are extracted.

[0104] It should be noted that the labels of the above-mentioned image samples can be marked according to pre-set image classification standards, and the image classification standards include judgment criteria for dividing images into sensitive images and non-sensitive images. For example, sensitive images can be images that include at least one sensitive content including exposed breasts, exposed buttocks, exposed triangle area, underwear, close-ups of feet, close-ups of crotch, and close-ups of buttocks, while non-sensitive images are images that do not include the above-mentioned sensitive content.

[0105] In addition, the M-1th CNN model can be an initial CNN model with model parameters pre-set (in this case, M=1, that is, the 0th CNN model represents the initial CNN model), or it can be a CNN model obtained after any iteration in the training process of the initial CNN model (that is, M is greater than 1), and the M-1th CNN model can include one CNN model or multiple CNN models, and the multiple CNN models can be CNN models used to identify whether a picture is a sensitive picture according to different image features (such as chest, buttocks or triangle area, etc.).

[0106] In one embodiment of the present application, the first sample set is input into the M-1th CNN model and the M-1th CNN model is trained. The method may be to input the picture sample of the first sample set into the M-1th CNN model, and the M-1th CNN model outputs the recognition result; calculate the loss between the recognition result and the marking result of the label in the picture sample by a preset loss function; determine whether the calculated loss meets the preset condition, and if the calculated loss does not meet the preset condition, update the network parameters in the CNN model to obtain the M-th CNN model, and the M-th CNN model does not meet the training stop condition.

[0107] Among them, the above-mentioned judgment of whether the calculated loss meets the preset condition can be to judge whether the difference between the calculated loss and the loss calculated in the previous training is less than or equal to the preset difference (such as tending to be unchanged), or, it can also be to judge whether the calculated loss is less than or equal to the preset loss, and so on.

[0108] In addition, the above-mentioned updating of the network parameters in the CNN model can be to adjust the network parameters of the CNN model according to a preset network parameter updating method, such as a gradient descent method.

[0109] It should be noted that after the Mth CNN model is trained, the electronic device can input the unlabeled samples in the second sample set into the Mth CNN model to obtain image samples labeled by the machine, and add the labeled samples to the first sample set. In this way, before the training stops, the CNN model obtained through training is continuously used to label the unlabeled samples and update the first sample set until the above training meets the stopping condition, and the updated first sample set is used as the training sample set.

[0110] Taking the above M=3 as an example for explanation, that is, the above M-1th model is the second CNN model obtained after training the initial CNN model twice. At this time, the second CNN model is used as the CNN model to be trained, and the electronic device inputs the first sample set into the CNN model to be trained to obtain the third CNN model; then, the electronic device inputs the picture samples in the second sample set into the third CNN model, and annotates the unlabeled picture samples in the second sample set through the third CNN model, that is, annotates whether the picture of each picture sample in the second sample set is a sensitive picture, and adds the labeled picture samples to the first sample set to update the first sample set; the loss of the third CNN model is calculated by a preset loss function. If the loss of the third CNN model does not meet the preset condition (that is, the third CNN model does not meet the training stop condition), the network parameters of the third CNN model are updated, and the third CNN model is used as the CNN model to be trained, and the first sample set is input into the CNN model to be trained again; if the loss of the third CNN model meets the preset condition, the updated first sample set is used as the above training sample set.

[0111] Of course, in the process of obtaining the training sample set mentioned above, the electronic device may simultaneously train multiple CNN models to be trained through the first sample set to obtain multiple trained CNN models, and the multiple CNN models to be trained may be CNN models that identify whether a picture is a sensitive picture according to different image features (such as chest, buttocks or triangle area, etc.); the multiple trained CNN models jointly mark whether the picture samples in the second sample set are sensitive images, and add the marked picture samples to the first sample set.

[0112] In another example, Figure 4As shown, the electronic device can use the labeled pictures of the labeled picture set (i.e., the first sample set) to train multiple initial CNN models (including multiple CNN models respectively used to identify whether the images in at least two image areas such as the buttocks, chest and triangle area are sensitive), and use the trained multiple CNN models as multiple Teacher models; combine multiple teacher models to annotate unlabeled pictures (i.e., picture samples of the second sample set), for example, if at least one Teacher model recognizes that the image in its corresponding image area is sensitive, then the unlabeled picture is labeled as a sensitive image; if multiple Teacher models all recognize that the image in its corresponding image area is insensitive, then the unlabeled picture is labeled as a normal image; add the labeled pictures annotated by the multiple Teacher models to the labeled picture set; if any one of the multiple Teacher models does not meet the training stop condition, use the Teacher model as a student model, train the Student model through the updated labeled picture set, that is, update the network parameters of the Student model, use the trained Student model as a new Teacher model, and re-execute the combination of multiple Teacher models. The model annotates unlabeled images; when multiple Teacher models meet the training stop conditions, the final updated annotated image set is used as the first training set.

[0113] In one embodiment of the present application, the electronic device may form the first training set through the labeled images obtained by the semi-supervised processing. Of course, the electronic device may also expand the formed first training set to further enrich the image samples in the first training set.

[0114] Specifically, after taking the sample set as the first training set, the method further includes:

[0115] Performing image data enhancement processing on the image samples in the first training set to obtain a third sample set, wherein the image data enhancement processing includes performing at least one of rotation, flipping transformation, scaling transformation, translation transformation, scale transformation, noise perturbation, color transformation, and occlusion on the image;

[0116] The third sample set is added to the first training set to update the first training set.

[0117] Based on this, by performing the above-mentioned image data enhancement processing on the image samples in the first training set, and updating the first training set with the sample set obtained after the image data enhancement processing, the obtained first training set is enriched and the acquisition efficiency of the first training set is further improved.

[0118] In the embodiment of the present application, the above-mentioned image data enhancement processing on the image samples in the first training set may be performing image data enhancement processing on part of the image samples in the first training set, or may be performing image data enhancement processing on all the image samples in the first training set.

[0119] In the process of performing image data enhancement processing on the image samples of the first training set, the labels of the image samples obtained after the enhancement processing continue to use the labels of the image samples before the processing.

[0120] In addition, the above-mentioned picture data enhancement processing may include at least one of rotating, flipping, scaling, translating, scaling, noise perturbation, color transformation and occlusion of the picture. For example, the above-mentioned first training set includes a picture of a groundhog, and the picture of the groundhog can be subjected to picture data enhancement processing such as rotation, flipping, scaling, translating, scaling, noise perturbation, color transformation and occlusion to obtain a rotated groundhog picture, a flipped groundhog picture, a scaled groundhog picture, a translational groundhog picture, a scaled groundhog picture, a noise perturbed groundhog picture, a color transformed groundhog picture and an occluded groundhog picture.

[0121] Step 102: When the target CNN network model identifies the image to be identified as a sensitive image, obtain a heat map of the image to be identified generated by the target CNN network model, wherein the heat map of the image to be identified includes at least one sensitive image area.

[0122] In one embodiment of the present application, since the CNN model has Class Activation Mapping (CAM) technology, CAM can calculate the basis for the model to determine that a picture is a sensitive picture, that is, when the picture is a sensitive picture, calculate the weight of each image feature (including buttocks, chest, and triangle area, etc.) in determining that the picture is a sensitive picture; according to the weight of each image feature, determine whether the area where the image feature is located is a sensitive image area, if the weight of the image feature is greater than or equal to the preset weight, then determine that the area where the image feature is located is a sensitive image area, otherwise, it is not a sensitive image area; and mark the image of the sensitive image area with preset colors (such as red, grayscale) of different depths to obtain a heat map, and the darker the preset color, the greater the weight.

[0123] For example, the basis for classifying the image to be identified as a sensitive image can be obtained through the weight calculation of the CNN model, such as Figure 5As shown in the heat map in , the dotted area (the dotted area is actually red or gray, and the darker the red or gray, the greater the weight) is the basis for determining that the image to be identified is exposed chest, that is, the weight of the chest image feature is greater than or equal to the preset weight, and the dotted area is the above-mentioned sensitive image area.

[0124] Step 103: Desensitize the image in at least one sensitive image area to obtain an output image.

[0125] In one embodiment of the present application, the above-mentioned electronic device performs desensitization processing on the image area within the sensitive image area, which may be achieved by adopting a predetermined desensitization processing method; or, the electronic device may also select different desensitization processing methods according to actual scene requirements to perform desensitization processing on the image area within the sensitive image area.

[0126] Specifically, in the above step 103,

[0127] Perform desensitization processing on at least one sensitive image area in the thermal map to obtain an output image, including:

[0128] Determine a target desensitization processing method for the image to be identified among N preset desensitization processing methods, where N is an integer greater than 1;

[0129] Based on the target desensitization processing method, the image of at least one sensitive image area in the thermal map is desensitized to obtain an output image.

[0130] Based on this, by determining the target desensitization processing method among N preset desensitization processing methods according to actual scene requirements, the image of the above-mentioned at least one sensitive image area is desensitized to further improve the quality of the desensitization processing.

[0131] Among them, any one of the above-mentioned N preset desensitization methods may be a processing method that can realize setting the sensitive information in the sensitive image area to an invisible state. For example, the preset desensitization method may be to delete the image in the desensitized image area from the picture to be identified, so that the desensitized image area in the picture to be identified is updated to a blank image area, and so on.

[0132] Specifically, the above-mentioned N preset desensitization processing methods include at least one of the following: setting mosaic, blurring and replacing with a target image, wherein the target image is an image associated with an image other than at least one sensitive image area in the picture to be identified.

[0133] Based on this, by setting mosaics, blurring or replacing images in sensitive image areas with target images, not only can the image content in the sensitive image areas in the image to be identified be prevented from being viewed, but the display quality of the image to be identified can also be guaranteed.

[0134] It should be noted that the target image may be any pre-set image, or may be an image associated with an image other than the image in the sensitive image area in the image to be identified, and so on.

[0135] In an embodiment of the present application, among the N preset desensitization processing methods, determining the target desensitization processing method for the image to be identified can be based on at least one scene information such as the sensitivity of the image to be identified and the current policy requirements for sensitive images, and determining that the preset desensitization processing method corresponding to the scene information is the above-mentioned target desensitization processing method.

[0136] For example, in one case, a desensitization processing method of replacing the target image may be used as the target desensitization processing method; and in another case, a desensitization processing method of setting a mosaic may be used as the target desensitization processing method.

[0137] Alternatively, determining a target desensitization processing method for the image to be identified among the N preset desensitization processing methods includes:

[0138] Obtaining a sensitivity level of at least one sensitive image area in the thermal map;

[0139] Among N preset desensitization processing methods, determine a target desensitization processing method corresponding to a highest sensitivity level among the sensitivity levels of at least one sensitive image area.

[0140] Based on this, the corresponding desensitization method is determined as the target desensitization method through the highest sensitivity level among the sensitivity levels of at least one sensitive image area in the above-mentioned picture to be identified, so as to make the desensitization method more suitable and further improve the desensitization effect.

[0141] Among them, the above-mentioned sensitivity level can be determined according to the image content in the sensitive image area. For example, if the image content exposes the buttocks, the sensitivity level is low, if the image content exposes the chest, the sensitivity level is moderate, if the image content exposes the triangle area, the sensitivity level is high, and so on; or, it can also be determined according to the color depth of each sensitive image area in the heat map, the darker the color, the higher the sensitivity level.

[0142] In addition, among the N preset desensitization processing methods mentioned above, the target desensitization processing method corresponding to the highest sensitivity level among the sensitivity levels of at least one sensitive image area is determined. The electronic device may have a preset correspondence between different sensitivity levels and preset desensitization processing methods, and the electronic device may determine the preset sensitive processing method that has a corresponding relationship with the above-mentioned highest sensitivity level as the target desensitization method.

[0143] Exemplarily, a high sensitivity level corresponds to setting a mosaic, a moderate sensitivity level corresponds to blurring, and a low sensitivity level corresponds to replacing with a target image. If the above-mentioned highest sensitivity level is a moderate sensitivity level, blurring is determined as the above-mentioned target desensitization method, and so on.

[0144] Based on this, the image to be identified is input into the target CNN model trained by the first training set, the image to be identified is identified by the target CNN model, and a heat map of the image to be identified is generated when the image to be identified is identified as a sensitive image, and finally the image of the sensitive image area in the generated heat map is desensitized, thereby achieving desensitization of part of the image to be identified. In this way, through the embodiment of the present application, compared with directly removing sensitive images, at least the information of images outside the sensitive image area can be retained, thereby improving the integrity of information in the information sharing process, and then improving the quality of information sharing between people.

[0145] In order to facilitate the understanding of the image desensitization processing process of the image processing method of the present application, Example 1 and Example 2 are provided here for illustration, as follows:

[0146] Example 1

[0147] a. Input the static cover image (i.e. the image to be identified) into the trained Student model (i.e. the target CNN model). The model will predict the sensitive label of the image and calculate the heat map.

[0148] b. If the corresponding sensitive label and actual scene requirements are met, the pixels in the heat map that represent sensitivity can be replaced, such as darker red pixels, such as sensitive pixels in the original image. There are multiple replacement methods, such as blurring, mosaics, "one-click dressing" (i.e. replacing with non-sensitive images in the image), etc.

[0149] c. Different strategies can be applied to the pixels of sensitive parts (i.e., the target desensitization treatment method can be determined). For example, if the sensitive parts hit are not serious, blurring can be performed, if they are serious, mosaics can be directly applied, and some minor ones can be operated by "one-click dressing";

[0150] d. In actual usage scenarios, the types of sensitive information that need to be processed and whether mosaics or blurs are used can be adjusted according to business or policy needs.

[0151] Example 2

[0152] a. Extract frames from the original video in real time (i.e., the images to be recognized are video frames) and feed the extracted frames into the Student model;

[0153] b. The frame data is input into the Student model. The model will identify the sensitive labels in the image, calculate the heat map in the original frame data, and perform desensitization processing based on the heat map.

[0154] c. If the corresponding sensitive labels and actual scene requirements are met, the sensitive parts of the frame data are directly desensitized according to the heat map obtained by the model. You can choose to set mosaics, blur processing, "one-click dressing", etc.

[0155] d. Restore the desensitized video frame data to video data, form desensitized video data, and finally display it to the user. If you choose "One-click dressing", the final effect presented will be imperceptible to the user, that is, the user will not be able to find that the image has been processed.

[0156] Figure 6 The figure is a schematic diagram showing the structure of a picture processing device according to an exemplary embodiment.

[0157] like Figure 6 As shown, the image processing device 600 may specifically include:

[0158] The picture input module 601 is configured to input the picture to be identified into the target CNN model when the picture to be identified is obtained, wherein the target CNN network model is trained by the first training set, the target CNN network model is used to identify whether the input picture is a sensitive picture, and the first training set includes picture samples marked as sensitive pictures;

[0159] The heat map acquisition module 602 is configured to acquire a heat map of the image to be identified generated by the target CNN network model when the target CNN network model identifies the image to be identified as a sensitive image, wherein the heat map of the image to be identified includes at least one sensitive image area;

[0160] The image output module 603 is configured to perform desensitization processing on the image in the at least one sensitive image area to obtain an output image.

[0161] Based on this, the image to be identified is input into the target CNN model trained by the first training set, the image to be identified is identified by the target CNN model, and a heat map of the image to be identified is generated when the image to be identified is identified as a sensitive image, and finally the image of the sensitive image area in the generated heat map is desensitized, thereby achieving desensitization of part of the image to be identified. In this way, through the embodiment of the present application, compared with directly removing sensitive images, at least the information of images outside the sensitive image area can be retained, thereby improving the integrity of information in the information sharing process, and then improving the quality of information sharing between people.

[0162] In one embodiment, the device 600 further includes:

[0163] A first training module is configured to train the initial CNN model based on the first training set to obtain a first CNN model;

[0164] A result output module is configured to input a picture of a first picture sample into the first CNN model to obtain a first result output by the first CNN model at a target layer, wherein the first picture sample is any picture sample in the first training set, and the target layer includes at least one of an intermediate layer and an output layer;

[0165] The parameter updating module is configured to use the first result as a soft target of the second result output by the second CNN model at the target layer, iteratively update the model parameters of the second CNN model, and obtain the target CNN model, wherein the second result is the result obtained by inputting the image of the first image sample into the second CNN model.

[0166] Based on this, by taking the trained first CNN model as the soft target of the output result of the second CNN model, the knowledge learned by the first CNN model is transferred to the second CNN model, and the above-mentioned target CNN model is trained, thereby making the training method of the target CNN model more flexible and ensuring the recognition accuracy of the target CNN model.

[0167] In one embodiment, the device 600 further includes:

[0168] A sample set acquisition module, configured to acquire a first sample set and a second sample set, wherein the first sample set includes labeled picture samples, and the second sample set includes unlabeled picture samples;

[0169] A second training module is configured to input the first sample set into the M-1th CNN model to train and obtain the Mth CNN model, wherein M is a positive integer;

[0170] a labeling module, configured to input the picture samples in the second sample set into the Mth CNN model to label the input picture samples, and add the labeled picture samples to the first sample set;

[0171] A reset module is configured to use the Mth CNN model as the M-1th CNN model and re-execute inputting the first sample set into the M-1th CNN model when the M CNN models do not meet the training stop condition;

[0172] The training set generation module is configured to use the sample set as the first training set when the M CNN models meet the training stop condition.

[0173] Based on this, in the process of training the CNN model, the CNN model to be trained is trained by a first sample set including labeled image samples, and the unlabeled image samples are labeled by the trained CNN model, and the labeled samples are added to the first sample set until the trained CNN model meets the training stop condition. The first sample set is used as the above-mentioned training sample set to obtain the training sample set in a semi-supervised manner, thereby reducing the workload of obtaining the training sample set and improving the efficiency of obtaining the training sample set.

[0174] In one embodiment, the device 600 further includes:

[0175] a data enhancement module, configured to perform image data enhancement processing on the image samples in the first training set to obtain a third sample set, wherein the image data enhancement processing includes performing at least one of rotation, flipping transformation, scaling transformation, translation transformation, scale transformation, noise perturbation, color transformation and occlusion on the image;

[0176] The training set updating module is configured to add the third sample set to the first training set to update the first training set.

[0177] Based on this, by performing the above-mentioned image data enhancement processing on the image samples in the first training set, and updating the first training set with the sample set obtained after the image data enhancement processing, the obtained first training set is enriched and the acquisition efficiency of the first training set is further improved.

[0178] In one embodiment, the picture output module 603 includes:

[0179] A processing mode determining unit is configured to determine a target desensitization processing mode of the image to be identified from N preset desensitization processing modes, where N is an integer greater than 1;

[0180] The image output unit is configured to perform desensitization processing on the image in the at least one sensitive image area based on the target desensitization processing method to obtain an output image.

[0181] Based on this, by determining the target desensitization processing method among N preset desensitization processing methods according to actual scene requirements, the image of the above-mentioned at least one sensitive image area is desensitized to further improve the quality of the desensitization processing.

[0182] In one of the embodiments, the N preset desensitization processing methods include setting mosaic, blurring and replacing with a target image, wherein the target image is an image associated with the image in the picture to be identified except for the at least one sensitive image area.

[0183] Based on this, by setting mosaics, blurring or replacing images in sensitive image areas with target images, not only can the image content in the sensitive image areas in the image to be identified be prevented from being viewed, but the display quality of the image to be identified can also be guaranteed.

[0184] In one embodiment, the processing mode determination unit includes:

[0185] A sensitivity level acquisition subunit, configured to acquire a sensitivity level of at least one sensitive image area in the thermal map;

[0186] The processing mode subunit is configured to determine, among the N preset desensitization processing modes, a target desensitization processing mode corresponding to the highest sensitivity level among the sensitivity levels of the at least one sensitive image area.

[0187] Based on this, the corresponding desensitization method is determined as the target desensitization method through the highest sensitivity level among the sensitivity levels of at least one sensitive image area in the above-mentioned picture to be identified, so as to make the target desensitization method more suitable and further improve the desensitization effect.

[0188] Based on the same inventive concept, the present application embodiment also provides a computing device, specifically combined with Figure 7 Provide detailed explanation.

[0189] Figure 7 It is a structural block diagram of a computing device according to an exemplary embodiment.

[0190] like Figure 7 As shown, the computing device 700 is a structural diagram of an exemplary hardware architecture of a computing device that can implement the image processing method and image processing apparatus in the embodiments of the present application. The computing device may refer to the electronic device in the embodiments of the present application.

[0191] The computing device 700 may include a processor 701 and a memory 702 storing computer program instructions.

[0192] Specifically, the processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0193] The memory 702 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 702 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 702 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 702 is a non-volatile solid-state memory. In a particular embodiment, the memory 702 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0194] The processor 701 reads and executes the computer program instructions stored in the memory 702 to perform the following steps:

[0195] Processor 701 executes, when a picture to be identified is obtained, inputting the picture to be identified into a target CNN model, wherein the target CNN network model is obtained by training with a first training set, the target CNN network model is used to identify whether the input picture is a sensitive picture, and the first training set includes picture samples marked with whether they are sensitive pictures;

[0196] In the case where the target CNN network model identifies the image to be identified as a sensitive image, obtaining a heat map of the image to be identified generated by the target CNN network model, wherein the heat map of the image to be identified includes at least one sensitive image area;

[0197] Desensitization processing is performed on the image in the at least one sensitive image area to obtain an output image.

[0198] Based on this, the image to be identified is input into the target CNN model trained by the first training set, the image to be identified is identified by the target CNN model, and a heat map of the image to be identified is generated when the image to be identified is identified as a sensitive image, and finally the image of the sensitive image area in the generated heat map is desensitized, thereby achieving desensitization of part of the image to be identified. In this way, through the embodiment of the present application, compared with directly removing sensitive images, at least the information of images outside the sensitive image area can be retained, thereby improving the integrity of information in the information sharing process, and then improving the quality of information sharing between people.

[0199] In one embodiment, the processor 701 executes training of the initial CNN model based on the first training set to obtain a first CNN model;

[0200] Inputting a picture of a first picture sample into the first CNN model, and obtaining a first result output by the first CNN model at a target layer, wherein the first picture sample is any picture sample in the first training set, and the target layer includes at least one of an intermediate layer and an output layer;

[0201] The first result is used as a soft target of the second result output by the second CNN model at the target layer, and the model parameters of the second CNN model are iteratively updated to obtain the target CNN model, wherein the second result is the result obtained by inputting the image of the first image sample into the second CNN model.

[0202] Based on this, by taking the trained first CNN model as the soft target of the output result of the second CNN model, the knowledge learned by the first CNN model is transferred to the second CNN model, and the above-mentioned target CNN model is trained, thereby making the training method of the target CNN model more flexible and ensuring the recognition accuracy of the target CNN model.

[0203] In one embodiment, the processor 701 executes obtaining a first sample set and a second sample set, wherein the first sample set includes labeled picture samples, and the second sample set includes unlabeled picture samples;

[0204] Inputting the first sample set into the M-1th CNN model, and training to obtain the Mth CNN model, wherein M is a positive integer;

[0205] Inputting the picture samples in the second sample set into the Mth CNN model to annotate the input picture samples, and adding the annotated picture samples to the first sample set;

[0206] In the case that the M CNN models do not meet the training stop condition, using the M CNN model as the M-1 CNN model, and re-implementing the first sample set into the M-1 CNN model;

[0207] When the M CNN models meet the training stop condition, the sample set is used as the first training set.

[0208] Based on this, in the process of training the CNN model, the CNN model to be trained is trained by a first sample set including labeled image samples, and the unlabeled image samples are labeled by the trained CNN model, and the labeled samples are added to the first sample set until the trained CNN model meets the training stop condition. The first sample set is used as the above-mentioned training sample set to obtain the training sample set in a semi-supervised manner, thereby reducing the workload of obtaining the training sample set and improving the efficiency of obtaining the training sample set.

[0209] In one embodiment, the processor 701 performs image data enhancement processing on the image samples in the first training set to obtain a third sample set, wherein the image data enhancement processing includes performing at least one of rotation, flip transformation, scaling transformation, translation transformation, scale transformation, noise disturbance, color transformation, and occlusion on the image;

[0210] The third sample set is added to the first training set to update the first training set.

[0211] Based on this, by performing the above-mentioned image data enhancement processing on the image samples in the first training set, and updating the first training set with the sample set obtained after the image data enhancement processing, the obtained first training set is enriched and the acquisition efficiency of the first training set is further improved.

[0212] In one embodiment, the processor 701 determines a target desensitization processing method for the image to be identified among N preset desensitization processing methods, where N is an integer greater than 1;

[0213] Based on the target desensitization processing method, the image in the at least one sensitive image area is desensitized to obtain an output image.

[0214] Based on this, by determining the target desensitization processing method among N preset desensitization processing methods according to actual scene requirements, the image of the above-mentioned at least one sensitive image area is desensitized to further improve the quality of the desensitization processing.

[0215] In one of the embodiments, the N preset desensitization processing methods include setting mosaic, blurring and replacing with a target image, wherein the target image is an image associated with the image in the picture to be identified except for the at least one sensitive image area.

[0216] Based on this, by setting mosaics, blurring or replacing images in sensitive image areas with target images, not only can the image content in the sensitive image areas in the image to be identified be prevented from being viewed, but the display quality of the image to be identified can also be guaranteed.

[0217] In one of the embodiments, the processor 701 executes obtaining a sensitivity level of at least one sensitive image area in the thermal map;

[0218] Among the N preset desensitization processing methods, determine a target desensitization processing method corresponding to the highest sensitivity level among the sensitivity levels of the at least one sensitive image area.

[0219] Based on this, the corresponding desensitization method is determined as the target desensitization method through the highest sensitivity level among the sensitivity levels of at least one sensitive image area in the above-mentioned picture to be identified, so as to make the target desensitization method more suitable and further improve the desensitization effect.

[0220] In one example, the computing device 700 may further include a transceiver 703 and a bus 704. Figure 5 As shown, the processor 701, the memory 702 and the transceiver 703 are connected via a bus 704 and communicate with each other.

[0221] Bus 704 includes hardware, software or both.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral control interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 704 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0222] The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used to implement the image processing method described in the embodiment of the present application.

[0223] In some possible implementations, various aspects of the method provided by the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to various exemplary embodiments of the present disclosure described above in this specification. For example, the computer device may execute the image processing method described in the embodiments of the present application.

[0224] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0225] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable image transmission device to produce a machine, so that the instructions executed by the processor of the computer or other programmable image transmission device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0226] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable image transmission device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0227] These computer program instructions may also be loaded onto a computer or other programmable image transmission device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0228] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for processing an image, characterized in that: include: When a picture to be identified is obtained, the picture to be identified is input into a target CNN network model, wherein the target CNN network model is obtained by training with a first training set, the target CNN network model is used to identify whether the input picture is a sensitive picture, and the first training set includes picture samples marked with whether they are sensitive pictures; In the case where the target CNN network model identifies the image to be identified as a sensitive image, obtaining a heat map of the image to be identified generated by the target CNN network model, wherein the heat map of the image to be identified includes at least one sensitive image area; Performing desensitization processing on the image in the at least one sensitive image area to obtain an output image; In the case where the image to be identified is obtained, before the image to be identified is input into the target CNN network model, the method further includes: Acquire a first sample set and a second sample set, wherein the first sample set includes labeled picture samples, and the second sample set includes unlabeled picture samples; Inputting the first sample set into the M-1th CNN model, and training to obtain the Mth CNN model, wherein M is a positive integer; Inputting the picture samples in the second sample set into the Mth CNN model to annotate the input picture samples, and adding the annotated picture samples to the first sample set; In the case that the M CNN models do not meet the training stop condition, using the M CNN model as the M-1 CNN model, and re-implementing the first sample set into the M-1 CNN model; When the M CNN models meet the training stop condition, the first sample set is used as the first training set.

2. The method according to claim 1, characterized in that In the case where the image to be identified is obtained, before the image to be identified is input into the target CNN network model, the method further includes: Training the initial CNN model based on the first training set to obtain a first CNN model; Inputting a picture of a first picture sample into the first CNN model, and obtaining a first result output by the first CNN model at a target layer, wherein the first picture sample is any picture sample in the first training set, and the target layer includes at least one of an intermediate layer and an output layer; The first result is used as a soft target of the second result output by the second CNN model at the target layer, and the model parameters of the second CNN model are iteratively updated to obtain the target CNN network model, wherein the second result is the result obtained by inputting the image of the first image sample into the second CNN model.

3. The method according to claim 1, characterized in that After taking the first sample set as the first training set, the method further includes: Performing image data enhancement processing on the image samples in the first training set to obtain a third sample set, wherein the image data enhancement processing includes performing at least one of rotation, flipping, scaling, translation, scale transformation, noise perturbation, color transformation, and occlusion on the image; The third sample set is added to the first training set to update the first training set.

4. The method according to claim 1, characterized in that: The desensitizing the image in the at least one sensitive image area to obtain an output image includes: Determine a target desensitization processing method for the image to be identified among N preset desensitization processing methods, where N is an integer greater than 1; Based on the target desensitization processing method, the image in the at least one sensitive image area is desensitized to obtain an output image.

5. According to the method of claim 4, the N preset desensitization processing methods include setting mosaic, blurring and replacing with target image, wherein: The target image is an image associated with an image other than the at least one sensitive image area in the picture to be identified.

6. The method according to claim 4, wherein determining a target desensitization processing method for the image to be identified among N preset desensitization processing methods comprises: Obtaining a sensitivity level of at least one sensitive image area in the thermal map; Among the N preset desensitization processing methods, determine a target desensitization processing method corresponding to the highest sensitivity level among the sensitivity levels of the at least one sensitive image area.

7. A picture processing device, characterized in that: include: The picture input module is configured to input the picture to be identified into a target CNN network model when the picture to be identified is obtained, wherein the target CNN network model is trained by a first training set, the target CNN network model is used to identify whether the input picture is a sensitive picture, and the first training set includes picture samples marked as sensitive pictures; a heat map acquisition module, configured to acquire a heat map of the image to be identified generated by the target CNN network model when the target CNN network model identifies the image to be identified as a sensitive image, wherein the heat map of the image to be identified includes at least one sensitive image area; An image output module is configured to perform desensitization processing on the image in the at least one sensitive image area to obtain an output image; The device further comprises: A sample set acquisition module, configured to acquire a first sample set and a second sample set, wherein the first sample set includes labeled picture samples, and the second sample set includes unlabeled picture samples; A second training module is configured to input the first sample set into the M-1th CNN model to train and obtain the Mth CNN model, wherein M is a positive integer; a labeling module, configured to input the picture samples in the second sample set into the Mth CNN model to label the input picture samples, and add the labeled picture samples to the first sample set; A reset module is configured to use the Mth CNN model as the M-1th CNN model and re-execute inputting the first sample set into the M-1th CNN model when the M CNN models do not meet the training stop condition; The training set generation module is configured to use the first sample set as the first training set when the M CNN models meet the training stop condition.

8. The device according to claim 7, characterized in that The device further comprises: A first training module is configured to train the initial CNN model based on the first training set to obtain a first CNN model; A result output module is configured to input a picture of a first picture sample into the first CNN model to obtain a first result output by the first CNN model at a target layer, wherein the first picture sample is any picture sample in the first training set, and the target layer includes at least one of an intermediate layer and an output layer; The parameter updating module is configured to use the first result as a soft target of the second result output by the second CNN model at the target layer, iteratively update the model parameters of the second CNN model, and obtain the target CNN network model, wherein the second result is the result obtained by inputting the image of the first image sample into the second CNN model.

9. The device according to claim 7, characterized in that The device further comprises: a data enhancement module, configured to perform image data enhancement processing on the image samples in the first training set to obtain a third sample set, wherein the image data enhancement processing includes performing at least one of rotation, flipping transformation, scaling transformation, translation transformation, scale transformation, noise perturbation, color transformation and occlusion on the image; The training set updating module is configured to add the third sample set to the first training set to update the first training set.

10. The device according to claim 7, characterized in that The picture output module includes: A processing mode determining unit is configured to determine a target desensitization processing mode of the image to be identified from N preset desensitization processing modes, where N is an integer greater than 1; The image output unit is configured to perform desensitization processing on the image in the at least one sensitive image area based on the target desensitization processing method to obtain an output image.

11. The device according to claim 10, characterized in that The N preset desensitization processing methods include setting mosaic, blurring and replacing with a target image, wherein the target image is an image associated with the image in the picture to be identified except for the at least one sensitive image area.

12. The device according to claim 10, wherein the processing mode determination unit comprises: A sensitivity level acquisition subunit, configured to acquire a sensitivity level of at least one sensitive image area in the thermal map; The processing mode subunit is configured to determine, among the N preset desensitization processing modes, a target desensitization processing mode corresponding to the highest sensitivity level among the sensitivity levels of the at least one sensitive image area.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an image processing device or an electronic device, the image processing device or the electronic device implements the image processing method as described in any one of claims 1 to 6.

15. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or instruction is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Image recognition model training method, device and equipment and storage medium

    CN111639710A

  • Anti-candid image processing method and device, terminal and storage medium

    CN111711794A

  • Image data detection method, device and equipment and computer readable storage medium

    CN111726648A

  • Popular picture automatic identification and content filtering method on mobile internet

    CN112395442A

Cited By

  • Knowledge distillation from non-streaming to streaming encoder

    US20240290332A1