Customized sensitive image detection

Abstract

The invention provides a sensitive image detection method and device and a computer readable medium. A sample image containing a sensitive object may be received. The sample image may be pre-processed to generate a target image. The target image may be encoded into a target vector. An image to be detected may be received. The to-be-detected image may be encoded into a to-be-detected vector. The probability that the to-be-detected image contains the sensitive object and / or the position of the sensitive object in the to-be-detected image can be calculated by performing correlation calculation on the target vector and the to-be-detected vector.

Description

Background Art

[0001] With the development and popularization of computer technology and network technology, the Internet has become an important channel for information dissemination. People can publish various types of information through the Internet, such as news, self-created novels, commercial advertisements, etc. These information may contain some sensitive images that violate laws and regulations or make people uncomfortable. Therefore, before releasing information to the public or its target audience, it is necessary to detect the images in the information to identify sensitive images and block them. Summary of the invention

[0002] This Summary is provided to introduce a group of concepts that will be further described in the following Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0003] Embodiments of the present disclosure propose methods, devices, and computer-readable media for sensitive image detection. A sample image containing a sensitive object may be received. The sample image may be preprocessed to generate a target image. The target image may be encoded into a target vector. An image to be detected may be received. The image to be detected may be encoded into a vector to be detected. The probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected may be calculated by performing a correlation calculation on the target vector and the vector to be detected.

[0004] It should be noted that one or more of the above aspects include features described in detail below and specifically pointed out in the claims. The following description and drawings set forth in detail certain illustrative features of the one or more aspects. These features are merely indicative of various ways in which the principles of the various aspects may be employed, and the present disclosure is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The disclosed aspects will hereinafter be described in conjunction with the accompanying drawings, which are provided to illustrate rather than limit the disclosed aspects.

[0006] Figure 1 An exemplary process for sensitive image detection according to an embodiment of the present disclosure is shown.

[0007] Figure 2 An exemplary process for preprocessing a sample image according to an embodiment of the present disclosure is shown.

[0008] Figure 3A An exemplary process for performing subject target discrimination according to an embodiment of the present disclosure is shown.

[0009] Figure 3BAnother exemplary process for performing subject target identification according to an embodiment of the present disclosure is shown.

[0010] Figure 4 An example of identifying a region in a sample image according to an embodiment of the present disclosure is shown.

[0011] Figure 5 An exemplary process for performing rotational invariance detection according to an embodiment of the present disclosure is shown.

[0012] Figure 6 An exemplary process for performing color invariance detection according to an embodiment of the present disclosure is shown.

[0013] Figure 7 An exemplary process for calculating the probability that an image to be detected contains a sensitive object and the position of the sensitive object in the image to be detected according to an embodiment of the present disclosure is shown.

[0014] Figure 8 is a flow chart of an exemplary method for sensitive image detection according to an embodiment of the present disclosure.

[0015] Fig. 9 An exemplary device for sensitive image detection according to an embodiment of the present disclosure is shown.

[0016] Fig.10 Another exemplary device for sensitive image detection according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0017] The present disclosure will now be discussed with reference to several exemplary embodiments. It should be understood that the discussion of these embodiments is only used to enable those skilled in the art to better understand and thereby implement the embodiments of the present disclosure, and does not teach any limitation on the scope of the present disclosure.

[0018] Sensitive image detection can be performed by an image recognition model. The image recognition model can extract and analyze the features of the input image, and then identify the category of the object in the image. A set of sensitive objects can be predefined. The set of sensitive objects can include objects that violate laws and regulations, such as violence, pornography, etc. For a certain image, when the object contained in the image recognition model is identified as one of the predefined sensitive objects in the set, that is, when the image recognition model identifies that the image contains a sensitive object, then the image can be determined to be a sensitive image. In order to enable the image recognition model to identify sensitive objects, it is necessary to train the image recognition model using training data generated based on sensitive objects. A large number of images containing various types of sensitive objects can be collected, and these images can be manually annotated to create training data for training the image recognition model. This process requires a lot of computing resources and human resources, so this method is suitable for image detection for general sensitive objects, that is, image detection for objects included in a predefined set of sensitive objects.

[0019] The embodiments of the present disclosure propose customized sensitive image detection. Customized sensitive image detection is intended to perform image detection for any sensitive object customized by the user. The object may be an object that cannot be directly recognized by the existing image recognition model. In this article, the user refers to a person with sensitive image detection needs, such as a website manager, a staff member of a relevant functional agency, etc. The embodiments of the present disclosure propose to perform customized sensitive image detection through a target tracking model. The target tracking model is a computer vision model that can locate the target object in a continuous video frame. The user can provide an image containing the sensitive object that he wants to shield. In this article, the image containing the sensitive object provided by the user is referred to as a sample image. When receiving an image to be detected on which sensitive image detection needs to be performed, the target tracking model can calculate the probability that the image to be detected contains the sensitive object in the sample image and the position of the sensitive object in the image to be detected as the detection result for the image to be detected. The above process can detect the customized sensitive object in the image to be detected without creating training data and training the model for the user-defined sensitive object. In this way, customized sensitive image detection can be achieved without consuming a large amount of computing resources and human resources.

[0020] In order to improve the accuracy and robustness of the target tracking model when performing detection on objects in the sample image, the sample image can be preprocessed to generate a target image. The target image can be encoded into a target vector. The target vector can be stored and then used to perform correlation calculation with the to-be-detected vector of the to-be-detected image, so as to calculate the probability that the to-be-detected image contains the sensitive object in the sample image and the position of the sensitive object in the to-be-detected image. The embodiments of the present disclosure propose a variety of preprocessing operations, such as subject target discrimination, rotation invariance detection, color invariance detection, etc.

[0021] The subject target discrimination is intended to make the detection result output by the target tracking model meet the detection intention of the user. The sample image provided by the user may be of unclear subject, that is, it may contain multiple objects, but only one of these multiple objects may be the sensitive object that the user really wants to block. If such an image is provided to the target tracking model, it may cause noise interference to the target tracking model, resulting in the sensitive object that the user really wants to block being difficult to detect. A set of regions in the sample image can be identified by performing region proposal on the sample image, each region can contain an object and is associated with a probability indicating that the region contains an object. Considering that the sample image provided by the user is an image containing multiple objects with unclear subject, but the sensitive object that it wants to block may be clearer and clearer than other objects in the image, the probability that the area where the sensitive object is located contains an object should be higher. Based on this, the embodiment of the present disclosure proposes to extract an area with a probability greater than a predetermined threshold, and to crop a sub-image corresponding to the extracted area from the sample image. The cropped sub-image can be used as a target image corresponding to the sample image. The target tracking model can then perform sensitive image detection based on these target images and return the detection results to the user. If the user finds that the detection result contains an object that he does not want to shield, he can delete the previously provided sample image including the object that he does not want to shield, and re-provide a sample image with a clearer theme. This method can prompt the target tracking model to output a detection result that meets the user's detection intention. Alternatively, after a group of areas in the sample image are identified by performing region recommendation on the sample image, and an area with a probability greater than a predetermined threshold is extracted from the group of areas, a sensitive area containing sensitive objects can be determined, and a sub-image corresponding to the sensitive area can be cut from the sample image as a target image. For example, the extracted area can be presented to the user for selection, and the area selected by the user is determined as a sensitive area. The number of target images generated by this method is less than the number of target images generated by the previous method. Accordingly, the memory resources used to store the target image and the computing resources used to perform correlation calculations can be reduced. Therefore, this method can prompt the target tracking model to output a detection result that meets the user's detection intention while using fewer memory resources and computing resources.

[0022] Rotation invariance detection is intended to improve the robustness of the target tracking model against object rotation. At present, when the object in the image to be detected has an angle difference or flipping relative to the object in the sample image, the detection rate of the target tracking model is low and cannot meet the sensitive image detection requirements. The embodiment of the present disclosure proposes to rotate the current image by a predetermined angle to generate a rotated image, and input the current image and the rotated image to the target tracking model. If the target tracking model can detect the object in the current image in the rotated image, the rotated image is discarded; and if the target tracking model cannot detect the object in the current image in the rotated image, the rotated image is used as the target image. The above method can improve the robustness of the target tracking model against object rotation without requiring the user to provide multiple sample images of the same object at different angles. In addition, only when the target tracking model cannot detect the object in the current image in the rotated image, the rotated image is used as the target image, which can reduce the memory resource usage for storing the target image and the computing resource usage for performing correlation calculations.

[0023] Color invariance detection aims to improve the robustness of the target tracking model for the case where the color of the object changes. At present, when there is a color difference between the object in the image to be detected and the object in the sample image, the detection rate of the target tracking model is low and cannot meet the requirements of sensitive object detection. The embodiment of the present disclosure proposes to perform color conversion on the image to generate a color-converted image as the target image when it is determined that the color of the image is allowed to be ignored when encoding the image. It can be determined whether the color of the image is allowed to be ignored when encoding the image according to the parameters associated with the image or the category of the object contained in the image. The color conversion can be performed by performing a three-channel color random swap on the image or converting the image into a grayscale image. The above method can improve the robustness of the target tracking model for the case where the object color changes without requiring the user to provide multiple sample images of different colors of the same object. In addition, performing color conversion on the image only when the color of the image is allowed to be ignored when encoding the image can reduce the memory resource usage for storing the target image and the computing resource usage for performing color conversion and correlation calculation.

[0024] Various embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 An exemplary process 100 for sensitive image detection according to an embodiment of the present disclosure is shown. The process 100 may be performed by a sensitive image detection application according to an embodiment of the present disclosure.

[0026] At 102, a sample image containing a sensitive object may be received. The sample image may be provided by a user via a user interface of a sensitive image detection application. The user interface may be an interface presented on a computing device of a user when the sensitive image detection application is enabled. The sensitive object may be any object that the user customizes to be shielded.

[0027] At 104, the sample image may be preprocessed to generate a target image. The purpose of preprocessing the sample image is to improve the accuracy and robustness of the target tracking model when performing detection on the object in the sample image. The target image may be at least one of the following: a sub-image in the sample image, a rotated image of the sample image, a rotated image of the sub-image, a color-converted image of the sample image, a color-converted image of the sub-image, etc. Figures 2 to 6 hereinafter is used to illustrate an exemplary process for preprocessing a sample image.

[0028] At 106, an image to be detected may be received. The image to be detected may be an image on which sensitive image detection needs to be performed.

[0029] At 108, the probability that the image to be detected contains the sensitive object in the sample image and / or the position of the sensitive object in the image to be detected can be calculated by the target tracking model. The target tracking model can be a unified target tracking (Unicorn) model, a nano tracking (Nano Tracking) model, etc. For example, through the target tracking model, the target image can be encoded into a target vector, the image to be detected can be encoded into a vector to be detected, and the probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected can be calculated by performing a correlation calculation on the target vector and the vector to be detected. Figure 7 To illustrate an exemplary process for calculating the probability that an image to be detected contains a sensitive object and the position of the sensitive object in the image to be detected.

[0030] At 110 , it may be determined whether the probability calculated at 108 is greater than a predetermined threshold.

[0031] If it is determined at 110 that the probability is greater than a predetermined threshold, process 100 proceeds to 112. At 112, the image to be detected can be presented. For example, the image to be detected can be presented in the user interface of the sensitive image detection application according to the embodiment of the present disclosure. Preferably, the probability that the image to be detected contains the sensitive object in the sample image and / or the bounding box corresponding to the position of the sensitive object in the image to be detected can be displayed in the image to be detected. In this way, the user can know which sensitive object the image to be detected contains, and then know why the image to be detected is presented on the user interface. Alternatively or additionally, at 114, a sample image can be presented. For example, the sample image can be presented in the user interface of the sensitive image detection application in association with the image to be detected. In this way, the user can know which sample image uploaded by him is used as the basis for detecting the sensitive object.

[0032] A user may upload multiple sample images containing similar sensitive objects. In addition, preprocessing the sample image may generate multiple target images corresponding to the sample image. Accordingly, multiple highly overlapping bounding boxes for the same sensitive object may be detected in the same image to be detected. Techniques such as non-maximum suppression (NMS) can be used to merge these multiple bounding boxes to generate a unique bounding box. The unique bounding box can be displayed in the image to be detected. In this case, the sample image presented in association with the image to be detected can be the sample image with the highest detection probability among the multiple sample images uploaded by the user.

[0033] If it is determined at 110 that the probability is not greater than the predetermined threshold, process 100 proceeds to 116. At 116, the image to be detected and the sample image are not presented.

[0034] In process 100, the user-defined sensitive object can be detected in the image to be detected without creating training data and training a model for the user-defined sensitive object. Through process 100, customized sensitive image detection can be achieved without consuming a large amount of computing resources and human resources.

[0035] It should be understood that the above Figure 1The process described for sensitive image detection is merely exemplary. Depending on the actual application requirements, the steps in the process for sensitive image detection may be replaced or modified in any manner, and the process may include more or fewer steps. For example, in process 100, a target tracking model is used to perform sensitive image detection, but the embodiments of the present disclosure are not limited to this. In some embodiments, a target detection model such as a YOLO (You Only Look Once) model may be used to perform sensitive image detection. In addition, the specific order or hierarchy of the steps in process 100 is merely exemplary, and the process for sensitive image detection may be performed in an order different from that described.

[0036] Figure 2 An exemplary process 200 for preprocessing a sample image according to an embodiment of the present disclosure is shown. The process 200 may correspond to Figure 1 The process 200 includes three types of operations, such as subject target discrimination 210, rotation invariance detection 220, and color invariance detection 230. Through the process 200, one or more target images corresponding to the sample image 202 can be generated.

[0037] The subject target discrimination 210 can be performed on the sample image 202. The sample image provided by the user may have an unclear subject, that is, it may contain multiple objects, but only one of these multiple objects may be a sensitive object that the user really wants to block. If such an image is provided to the target tracking model, it may cause noise interference to the target tracking model, making it difficult to detect the sensitive object that the user really wants to block. The subject target discrimination 210 aims to ensure that the detection results output by the target tracking model meet the user's detection intention. The subject target discrimination 210 can crop a portion of the sample image 202 to generate one or more sub-images 212. The one or more sub-images 212 can be the target image 214. The subject target discrimination 210 can be performed in a variety of implementations. This will be discussed later in conjunction with Figure 3A and Figure 3B To illustrate an exemplary process of performing subject target discrimination.

[0038] The rotation invariance detection 220 can be performed on the sample image 202 or the sub-image 212 obtained by the subject target discrimination 210. At present, when the object in the image to be detected has an angle difference or flips relative to the object in the sample image, the detection rate of the target tracking model is low and cannot meet the sensitive image detection requirements. The rotation invariance detection 220 aims to improve the robustness of the target tracking model against object rotation. The rotation invariance detection 220 can rotate the sample image 202 or the sub-image 212 by a predetermined angle to generate a rotated image 222. The predetermined angle is, for example, 180 degrees. The rotated image 222 can be the target image 224. This will be discussed later in conjunction with Figure 5 To illustrate an exemplary process of performing rotational invariance detection.

[0039] The color invariance detection 230 can be performed on the sample image 202, the sub-image 212 obtained by the subject target discrimination 210, or the rotated image 222 obtained by the rotation invariance detection 220. At present, when there is a color difference between the object in the image to be detected and the object in the sample image, the detection rate of the target tracking model is low and cannot meet the sensitive object detection requirements. The color invariance detection 230 is intended to improve the robustness of the target tracking model against the situation where the color of the object changes. The color invariance detection 230 can perform color conversion on the image to generate a color converted image 232. The color converted image 232 can be the target image 234. Figure 6 An exemplary process of performing color invariance detection is described below.

[0040] It should be understood that Figure 2 The three types of preprocessing operations shown are only exemplary. According to actual application requirements, the sample image can be preprocessed in other ways. In addition, these three types of operations can be performed independently. For any sample image, only any one or two of the three types of operations can be performed. In addition, the specific order or hierarchy of the steps in process 200 is only exemplary, and the process for preprocessing the sample image can be performed in an order different from the described order.

[0041] Figure 3A An exemplary process 300a for performing subject target discrimination according to an embodiment of the present disclosure is shown. The process 300a may correspond to Figure 2 An implementation of subject target identification 210 in FIG.

[0042] At 302, a sample image containing a sensitive object may be obtained.

[0043] At 304, a set of regions in the sample image may be identified by performing region recommendation on the sample image. Each region may contain an object and is associated with a probability indicating that the region contains the object. The region recommendation operation may be performed by a region recommendation model. The region recommendation model is, for example, a Region Convolutional Neural Network (R-CNN) model, a Fast R-CNN model, a Faster R-CNN model, and the like. Figure 4 An example of identifying regions in a sample image according to an embodiment of the present disclosure is shown. Assume that image 400a is a sample image uploaded by a user. Image 400a contains three objects, namely dog ​​402, building 404, and cloud 406. One of the three objects is a sensitive object that the user wishes to shield. A set of regions obtained by performing region recommendation on image 400a is shown in image 400b. Each region contains one object. For example, region 408 contains dog 402, region 410 contains building 404, and region 412 contains cloud 406.

[0044] At 306, a region with a probability greater than a predetermined threshold may be extracted from the group of regions. Considering that even if the sample image provided by the user is an image containing multiple objects with unclear subject matter, the sensitive object that the user wishes to shield may be clearer and more specific than other objects in the image, the probability that the region where the sensitive object is located contains an object should be higher.

[0045] At 308 , it may be determined whether the number of extracted regions is greater than one.

[0046] If it is determined at 308 that the number of extracted regions is not greater than one, then this indicates that the sample image contains only a single distinct object. In this case, process 300a ends at 310.

[0047] If it is determined at 308 that the number of extracted regions is greater than one, it indicates that the sample image contains a plurality of distinct objects. In this case, process 300a proceeds to 312. At 312, a sub-image corresponding to the extracted region may be cropped from the sample image. The cropped sub-image may be used as a target image corresponding to the sample image. Figure 4 Taking the sample image 400b in FIG. 4 as an example, it is assumed that the extracted regions with probabilities greater than a predetermined threshold are regions 410 and 412. In this case, two sub-images corresponding to regions 410 and 412 can be cut out from the sample image 400b as target images.

[0048] The target tracking model can then perform sensitive image detection based on these target images and return the detection results to the user. If the user finds that the detection results contain objects that he does not want to block, he can delete the previously provided sample images including the objects that he does not want to block, and re-provide sample images with clearer themes. This approach can prompt the target tracking model to output detection results that meet the user's detection intentions.

[0049] Figure 3B Another exemplary process 300b for performing subject target discrimination according to an embodiment of the present disclosure is shown. The process 300b may correspond to Figure 2 Another implementation of the subject target identification 210 in FIG.

[0050] Steps 352 to 360 may correspond to Figure 3A Steps 302 to 310 in .

[0051] Unlike process 300a, in process 300b, when it is determined that the number of extracted regions is greater than one, a sensitive region containing a sensitive object among the extracted regions may be determined. At 362, the extracted regions may be presented to the user. For example, the extracted regions may be presented by displaying a bounding box corresponding to the extracted regions in the sample image. Each region can be selected. For example, referring to Figure 4 , in image 400b, bounding boxes corresponding to region 408, region 410, and region 412 may be displayed.

[0052] At 364, a selection of the extracted area may be received. For example, the user may select the area by clicking on the area he / she wants to select. The area selected by the user may be an area that he / she believes contains a sensitive object.

[0053] At 366 , the selected area may be determined to be a sensitive area containing a sensitive object.

[0054] At 368, a sub-image corresponding to the sensitive area may be cropped from the sample image. The cropped sub-image may be used as a target image corresponding to the sample image. Figure 4 For example, assume that the area selected by the user is 410. That is, the user considers the building 404 included in the area 410 to be a sensitive object. In this case, a sub-image corresponding to the area 410 can be cropped from the image 400b as the target image.

[0055] Compared with the number of target images generated by process 300a, the number of target images generated by process 300b is less. Accordingly, the memory resources used to store the target images and the computing resources used to perform the correlation calculation can be reduced. Therefore, process 300b can cause the target tracking model to output a detection result that meets the user's detection intention while using less memory resources and computing resources.

[0056] It should be understood that the above Figure 3A and Figure 3B as well as Figure 4 The process described for performing thematic target discrimination is merely exemplary. The steps in the process for performing thematic target discrimination may be replaced or modified in any manner, and the process may include more or fewer steps, depending on the actual application requirements. In addition, the specific order or hierarchy of steps in process 300a and process 300b is merely exemplary, and the process for performing thematic target discrimination may be performed in an order different from that described.

[0057] Figure 5 An exemplary process 500 for performing rotational invariance detection according to an embodiment of the present disclosure is shown. The process 500 may correspond to Figure 2 Step 220 in .

[0058] At 502, a current image may be obtained. The current image may be a sample image or a sub-image in the sample image.

[0059] At 504, a rotated image may be generated by rotating the current image by a predetermined angle, such as 180 degrees.

[0060] At 506, the current image and the rotated image may be input to a target tracking model. The target tracking model may be a model for calculating the probability that the image to be detected contains a sensitive object, for example Figure 1 The target tracking model mentioned in step 108 in FIG.

[0061] At 508 , a probability that the rotated image contains the object in the current image may be calculated by the target tracking model.

[0062] At 510 , it may be determined whether the calculated probability is less than a predetermined threshold.

[0063] If it is determined at 510 that the calculated probability is not less than a predetermined threshold, it indicates that the target tracking model is able to detect the object contained in the current image in the rotated image. In other words, the target tracking model is resistant to the rotation of the current image. In this case, process 500 proceeds to 512. At 512, the rotated image can be discarded. As an example, if the shape of the object contained in the current image is relatively simple, such as circular, symmetrical, etc., the target tracking model is generally able to detect the object contained in the current image in the rotated image. In this case, it is not necessary to use the rotated image as the target image.

[0064] If it is determined at 510 that the calculated probability is less than a predetermined threshold, it indicates that the target tracking model cannot detect the object contained in the current image in the rotated image. In other words, the target tracking model is not resistant to the rotation of the current image. In this case, process 500 proceeds to 514. At 514, the rotated image can be used as the target image.

[0065] In process 500, if the target tracking model can detect the object in the current image in the rotated image, the rotated image is discarded; and if the target tracking model cannot detect the object in the current image in the rotated image, the rotated image is used as the target image. Through process 500, the robustness of the target tracking model for object rotation situations can be improved without requiring the user to provide multiple sample images of the same object at different angles. In addition, using the rotated image as the target image only when the target tracking model cannot detect the object in the current image in the rotated image can reduce the memory resource usage for storing the target image and the computing resource usage for performing correlation calculations.

[0066] It should be understood that the above Figure 5 The process described for performing rotational invariance detection is merely exemplary. The steps in the process for performing rotational invariance detection may be replaced or modified in any manner, and the process may include more or fewer steps, depending on the actual application requirements. In addition, the specific order or hierarchy of the steps in process 500 is merely exemplary, and the process for rotational invariance detection may be performed in an order different from the order described.

[0067] Figure 6 An exemplary process 600 for performing color invariance detection according to an embodiment of the present disclosure is shown. The process 600 may correspond to Figure 2 Color invariance detection 230 in.

[0068] At 602, a current image may be obtained. The current image may be at least one of: a sample image, a sub-image in the sample image, a rotated image for the sample image, and a rotated image for the sub-image in the sample image.

[0069] At 604, it can be determined whether the color of the current image is allowed to be ignored when encoding the current image. Whether the color of the current image is allowed to be ignored when encoding the current image can be determined in a variety of ways. In one embodiment, a parameter associated with the current image for indicating whether the color of the current image is allowed to be ignored when encoding the current image can be read to determine. The parameter can be provided synchronously by the user when providing a sample image corresponding to the current image. In another embodiment, a group of objects that are not allowed to be ignored in color can be predefined. The group of objects can include, for example, blood, traffic lights, etc. The category of the object contained in the current image can be identified. If the object contained in the current image belongs to the group of predefined objects that are not allowed to be ignored in color, it indicates that the color of the current image is not allowed to be ignored when encoding the current image.

[0070] If it is determined at 604 that ignoring the colors of the current image is not permitted when encoding the current image, process 600 ends at 606 .

[0071] If it is determined at 604 that the color of the current image is allowed to be ignored when encoding the current image, color conversion can be performed on the current image to generate a color-converted image as the target image. The color conversion can include performing a three-channel color random swap on the current image or converting the current image into a grayscale image.

[0072] Process 600 may proceed to 608. At 608, for each pixel in the current image, the color of the pixel may be obtained. At 610, a set of color channels for representing the color may be determined. The set of color channels may be, for example, a red (R) channel, a green (G) channel, and a blue (B) channel. At 612, two channels in the set of color channels may be randomly selected. At 614, two values ​​corresponding to the two channels may be interchanged. This color conversion method may generate more color variants to improve the detection rate of the target tracking model.

[0073] Alternatively or additionally, process 600 may proceed to step 616. At step 616, the current image may be converted to a grayscale image. This color conversion may cause the target tracking model to focus on recognizing the shape of the object rather than the color.

[0074] In process 600, the robustness of the target tracking model against the object color change situation can be improved without requiring the user to provide multiple sample images of the same object in different colors. In addition, performing color conversion on the image only when the color of the image is allowed to be ignored when encoding the image can reduce the memory resource usage for storing the target image and the computing resource usage for performing color conversion and correlation calculation.

[0075] It should be understood that the above Figure 6 The process described for performing color invariance detection is exemplary only. The steps in the process for performing color invariance detection may be replaced or modified in any manner, and the process may include more or fewer steps, depending on actual application requirements. In addition, the specific order or hierarchy of the steps in process 600 is exemplary only, and the process for performing color invariance detection may be performed in an order different from the order described.

[0076] Figure 7 An exemplary process 700 for calculating the probability that an image to be detected contains a sensitive object and the position of the sensitive object in the image to be detected according to an embodiment of the present disclosure is shown. The process 700 may correspond to Figure 1 Step 108 in .

[0077] The target image 702 may be obtained by preprocessing a sample image containing a sensitive object. The image to be detected 704 may be an image on which sensitive image detection needs to be performed.

[0078] The target image 702 may be encoded into a target vector 722. For example, the target image 702 may be provided to an encoder 720 in the target tracking model 710. The encoder 720 may encode the target image 702 to generate the target vector 722. The process of encoding the target image 702 into the target vector 722 may be performed offline. The target vector 722 may be stored in a designated memory. The designated memory may be a memory accessible to the target tracking model 710.

[0079] The image to be detected 704 may be encoded into a vector to be detected 724. For example, the image to be detected 704 may be provided to an encoder 720 in the target tracking model 710. The encoder 720 may encode the image to be detected 704, thereby generating a vector to be detected 724.

[0080] Subsequently, a detection result 732 for the image to be detected 704 can be obtained. The detection result 732 may include, for example, a probability 734 that the image to be detected 704 contains a sensitive object, a position 736 of the sensitive object in the image to be detected 704, and the like. This operation may be performed by a correlation calculation unit 730 in the target tracking model 710. The correlation calculation unit 730 may read a previously stored target vector 722, and calculate a probability 734 that the image to be detected 704 contains a sensitive object and / or a position 736 of the sensitive object in the image to be detected 704 by performing a correlation calculation on the target vector 722 and the vector to be detected 724. It should be understood that the user may provide multiple sample images, and each sample image may have one or more target images, so there may be multiple target vectors. Each target vector will contain features of the corresponding sensitive object. Accordingly, when receiving the image to be detected 704, the correlation calculation unit 730 will recall all target vectors and perform correlation calculation on each target vector and the vector to be detected to calculate the probability that the image to be detected contains the corresponding sensitive object and / or the position of the sensitive object in the image to be detected.

[0081] After receiving the sample image provided by the user, the sample image can be stored in a designated memory. The memory can be the same as or different from the memory used to store the target vector. The user may delete the sample image previously provided. For example, if the user finds that the detection result contains an object that the user does not want to shield, the user can delete the previously provided sample image including the object that the user does not want to shield, and re-provide a sample image with a clearer theme. When an indication is received that the sample image has been deleted, the target vector corresponding to the sample image can be deleted accordingly, such as the target vector of the target image generated by preprocessing the sample image. In this way, it is possible to avoid subsequent sensitive image detection for objects that the user does not want to shield.

[0082] It should be understood that the above Figure 7 The described process for calculating the probability that the image to be detected contains a sensitive object and the position of the sensitive object in the image to be detected is merely exemplary. According to actual application requirements, the steps in the process for calculating the probability that the image to be detected contains a sensitive object and the position of the sensitive object in the image to be detected can be replaced or modified in any way, and the process can include more or fewer steps. For example, in addition to the target image 702, a sample image provided by the user can also be provided to the encoder 720 to generate a vector of the sample image. The vector can also be stored and used to perform sensitive image detection. In addition, Figure 7 The target tracking model 710 shown is only an example of a target tracking model. According to actual application requirements, the target tracking model may have any other structure and may include more or fewer modules.

[0083] Figure 8 is a flow chart of an exemplary method 800 for sensitive image detection according to an embodiment of the present disclosure.

[0084] At 810, a sample image containing a sensitive object may be received.

[0085] At 820 , the sample image may be pre-processed to generate a target image.

[0086] At 830, the target image may be encoded into a target vector.

[0087] At 840 , an image to be inspected may be received.

[0088] At 850 , the image to be detected may be encoded into a vector to be detected.

[0089] At 860 , the probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected may be calculated by performing a correlation calculation on the target vector and the vector to be detected.

[0090] In one embodiment, the preprocessing of the sample image may include: identifying a group of regions in the sample image by performing region recommendation on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting regions having a probability greater than a predetermined threshold from the group of regions; determining whether the number of extracted regions is greater than one; and in response to determining that the number of extracted regions is greater than one, cropping a sub-image corresponding to the extracted region from the sample image as a target image.

[0091] In one embodiment, the preprocessing of the sample image may include: identifying a group of regions in the sample image by performing region recommendation on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting regions with a probability greater than a predetermined threshold from the group of regions; determining whether the number of extracted regions is greater than one; in response to determining that the number of extracted regions is greater than one, determining a sensitive region in the extracted regions that contains the sensitive object; and cropping a sub-image corresponding to the sensitive region from the sample image as a target image.

[0092] In one embodiment, determining the sensitive area in the extracted area containing the sensitive object may include: presenting the extracted area by displaying a bounding box corresponding to the extracted area in the sample image, each area being selectable; receiving a selection of the extracted area; and determining the selected area as the sensitive area.

[0093] In one embodiment, the preprocessing of the sample image may include: generating a rotated image by rotating a current image by a predetermined angle, the current image being the sample image or a sub-image in the sample image; inputting the current image and the rotated image into a target tracking model, the target tracking model being a model for calculating the probability that the image to be detected contains the sensitive object; calculating, through the target tracking model, the probability that the rotated image contains the object in the current image; determining whether the calculated probability is less than a predetermined threshold; and in response to determining that the calculated probability is less than the predetermined threshold, using the rotated image as a target image.

[0094] In one embodiment, the preprocessing of the sample image may include: determining whether it is allowed to ignore the color of the current image when encoding the current image, the current image being at least one of: the sample image, a sub-image in the sample image, a rotated image for the sample image, and a rotated image for the sub-image; and in response to determining that it is allowed to ignore the color of the current image when encoding the current image, performing color conversion on the current image to generate a color-converted image as a target image.

[0095] The performing color conversion on the current image may include, for each pixel in the current image: obtaining the color of the pixel; determining a set of color channels for representing the color; randomly selecting two channels from the set of color channels; and swapping two values ​​corresponding to the two channels.

[0096] The performing color conversion on the current image may include converting the current image into a grayscale image.

[0097] In one embodiment, method 800 may also include: determining whether the probability is greater than a predetermined threshold; and in response to determining that the probability is greater than the predetermined threshold: presenting the image to be detected, and displaying the probability and / or a bounding box corresponding to the position in the image to be detected; and / or presenting the sample image.

[0098] In one embodiment, method 800 may also include: storing the sample image in a first memory; storing the target vector in a second memory, which is the same as or different from the first memory; receiving an indication that the sample image is deleted; and deleting the target vector in response to receiving the indication that the sample image is deleted.

[0099] It should be understood that the method 800 may also include any other steps / processes for sensitive image detection according to the embodiments of the present disclosure as described above.

[0100] Fig. 9 An exemplary apparatus 900 for sensitive image detection according to an embodiment of the present disclosure is shown.

[0101] The device 900 may include: a sample image receiving module 910 for receiving a sample image containing a sensitive object; a preprocessing module 920 for preprocessing the sample image to generate a target image; a target image encoding module 930 for encoding the target image into a target vector; a to-be-detected image receiving module 940 for receiving an image to be detected; an to-be-detected image encoding module 950 for encoding the image to be detected into a to-be-detected vector; and a calculation module 960 for calculating the probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected by performing correlation calculation on the target vector and the to-be-detected vector. In addition, the device 900 may also include any other module configured for sensitive image detection according to the embodiments of the present disclosure as described above.

[0102] Fig.10 Another exemplary apparatus 1000 for sensitive image detection according to an embodiment of the present disclosure is shown.

[0103] The apparatus 1000 may include: a processor 1010; and a memory 1020 storing computer executable instructions. When the computer executable instructions are executed, the processor 1010 may: receive a sample image containing a sensitive object, pre-process the sample image to generate a target image, encode the target image into a target vector, receive an image to be detected, encode the image to be detected into a vector to be detected, and calculate the probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected by performing a correlation calculation on the target vector and the vector to be detected.

[0104] In one embodiment, the preprocessing of the sample image may include: identifying a group of regions in the sample image by performing region recommendation on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting regions having a probability greater than a predetermined threshold from the group of regions; determining whether the number of extracted regions is greater than one; and in response to determining that the number of extracted regions is greater than one, cropping a sub-image corresponding to the extracted region from the sample image as a target image.

[0105] In one embodiment, the preprocessing of the sample image may include: identifying a group of regions in the sample image by performing region recommendation on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting regions with a probability greater than a predetermined threshold from the group of regions; determining whether the number of extracted regions is greater than one; in response to determining that the number of extracted regions is greater than one, determining a sensitive region in the extracted regions that contains the sensitive object; and cropping a sub-image corresponding to the sensitive region from the sample image as a target image.

[0106] Determining the sensitive area in the extracted area containing the sensitive object may include: presenting the extracted area by displaying a bounding box corresponding to the extracted area in the sample image, each area being selectable; receiving a selection of the extracted area; and determining the selected area as the sensitive area.

[0107] In one embodiment, the preprocessing of the sample image may include: generating a rotated image by rotating a current image by a predetermined angle, the current image being the sample image or a sub-image in the sample image; inputting the current image and the rotated image into a target tracking model, the target tracking model being a model for calculating the probability that the image to be detected contains the sensitive object; calculating, through the target tracking model, the probability that the rotated image contains the object in the current image; determining whether the calculated probability is less than a predetermined threshold; and in response to determining that the calculated probability is less than the predetermined threshold, using the rotated image as a target image.

[0108] In one embodiment, the preprocessing of the sample image may include: determining whether it is allowed to ignore the color of the current image when encoding the current image, the current image being at least one of: the sample image, a sub-image in the sample image, a rotated image for the sample image, and a rotated image for the sub-image; and in response to determining that it is allowed to ignore the color of the current image when encoding the current image, performing color conversion on the current image to generate a color-converted image as a target image.

[0109] The performing color conversion on the current image may include, for each pixel in the current image: obtaining the color of the pixel; determining a set of color channels for representing the color; randomly selecting two channels from the set of color channels; and swapping two values ​​corresponding to the two channels.

[0110] In one embodiment, when the computer executable instructions are executed, they may also cause the processor 1010 to: determine whether the probability is greater than a predetermined threshold; and in response to determining that the probability is greater than the predetermined threshold: present the image to be detected, and display the probability and / or the bounding box corresponding to the position in the image to be detected; and / or present the sample image.

[0111] In one embodiment, the computer executable instructions, when executed, may also cause the processor 1010 to: store the sample image in a first memory; store the target vector in a second memory, which is the same as or different from the first memory; receive an indication that the sample image has been deleted; and in response to receiving the indication that the sample image has been deleted, delete the target vector.

[0112] It should be understood that the processor 1010 may also execute any other steps / processes of the method for sensitive image detection according to the embodiments of the present disclosure as described above.

[0113] The embodiments of the present disclosure propose a computer program product for sensitive image detection, including a computer program, which is executed by a processor to: receive a sample image containing a sensitive object; preprocess the sample image to generate a target image; encode the target image into a target vector; receive an image to be detected; encode the image to be detected into a vector to be detected; and calculate the probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected by performing a correlation calculation on the target vector and the vector to be detected. In addition, the computer program can also be executed to implement any other steps / processes of the method for sensitive image detection according to the embodiments of the present disclosure as described above.

[0114] Embodiments of the present disclosure may be embodied in a computer-readable medium. The computer-readable medium may include instructions, which, when executed, cause the processor to: receive a sample image containing a sensitive object; preprocess the sample image to generate a target image; encode the target image into a target vector; receive an image to be detected; encode the image to be detected into a vector to be detected; and calculate the probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected by performing a correlation calculation on the target vector and the vector to be detected. In addition, when executed, the instructions may also cause the processor to perform any other steps / processes of the method for sensitive image detection according to the embodiments of the present disclosure as described above.

[0115] It should be understood that all operations in the above-described methods are merely exemplary, and the present disclosure is not limited to any operations in the methods or the order of these operations, but should cover all other equivalent transformations under the same or similar concepts. In addition, unless otherwise specified or it is clear from the context that it is directed to a singular form, the articles "a" and "an" as used in this specification and the appended claims should generally be interpreted as meaning "one" or "one or more".

[0116] It should also be understood that all modules in the above described device can be implemented in various ways. These modules can be implemented as hardware, software, or a combination thereof. In addition, any module in these modules can be further divided into submodules or combined together functionally.

[0117] Processors have been described in conjunction with various devices and methods. These processors can be implemented using electronic hardware, computer software or any combination thereof. Whether these processors are implemented as hardware or software will depend on specific application and the overall design constraints imposed on the system. As an example, the processor provided in the present disclosure, any part of the processor or any combination of processors can be implemented using a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a state machine, a gated logic unit, a discrete hardware circuit, and other suitable processing components configured to perform the various functions described in the present disclosure. The function of the processor provided in the present disclosure, any part of the processor or any combination of processors can be implemented using software executed by a microprocessor, a microcontroller, a DSP or other suitable platforms.

[0118] Software should be broadly considered to mean instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, functions, etc. Software can reside in a computer-readable medium. A computer-readable medium can include, for example, a memory, which can be, for example, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic stripe), an optical disk, a smart card, a flash memory device, a random access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a register, or a removable disk. Although the memory is shown as being separated from the processor in the multiple aspects provided in the present disclosure, the memory can also be located inside the processor, such as a cache or a register.

[0119] The above description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents to the elements of the various aspects described in this disclosure that are known or to be known to those of ordinary skill in the art are expressly incorporated herein and covered by the claims.

Claims

1. A method for sensitive image detection, comprising: receiving a sample image containing a sensitive object; Preprocessing the sample image to generate a target image; encoding the target image into a target vector; Receiving an image to be detected; Encoding the image to be detected into a vector to be detected; and The probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected is calculated by performing correlation calculation on the target vector and the vector to be detected.

2. The method according to claim 1, wherein: The preprocessing of the sample image comprises: identifying a set of regions in the sample image by performing region proposal on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting, from the set of regions, regions with probabilities greater than a predetermined threshold; determining whether the number of extracted regions is greater than one; and In response to determining that the number of the extracted regions is greater than one, a sub-image corresponding to the extracted regions is cropped from the sample image as a target image.

3. The method according to claim 1, wherein: The preprocessing of the sample image comprises: identifying a set of regions in the sample image by performing region proposal on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting, from the set of regions, regions with probabilities greater than a predetermined threshold; determining whether the number of extracted regions is greater than one; In response to determining that the number of the extracted regions is greater than one, determining a sensitive region among the extracted regions that includes the sensitive object; and A sub-image corresponding to the sensitive area is cut out from the sample image as a target image.

4. The method according to claim 3, wherein: The determining of the sensitive area containing the sensitive object in the extracted area comprises: presenting the extracted regions by displaying bounding boxes corresponding to the extracted regions in the sample image, each region being selectable; receiving a selection of an extracted region; and The selected area is determined as the sensitive area.

5. The method according to claim 1, wherein: The preprocessing of the sample image comprises: generating a rotated image by rotating a current image by a predetermined angle, the current image being the sample image or a sub-image in the sample image; Inputting the current image and the rotated image into a target tracking model, wherein the target tracking model is a model for calculating the probability that the image to be detected contains the sensitive object; Calculating, by means of the target tracking model, a probability that the rotated image contains the object in the current image; determining whether the calculated probability is less than a predetermined threshold; and In response to determining that the calculated probability is less than the predetermined threshold, the rotated image is used as a target image.

6. The method according to claim 1, wherein: The preprocessing of the sample image comprises: determining whether to allow ignoring the color of a current image when encoding the current image, the current image being at least one of: the sample image, a sub-image in the sample image, a rotated image for the sample image, and a rotated image for the sub-image; and In response to determining that it is allowed to ignore the color of the current image when encoding the current image, color conversion is performed on the current image to generate a color-converted image as a target image.

7. The method according to claim 6, wherein: The performing color conversion on the current image comprises, for each pixel in the current image: Obtain the color of the pixel; determining a set of color channels for representing the color; randomly selecting two channels from the set of color channels; and The two values ​​corresponding to the two channels are swapped.

8. The method according to claim 6, wherein: The performing color conversion on the current image comprises: Convert the current image into a grayscale image.

9. The method according to claim 1, further comprising: determining whether the probability is greater than a predetermined threshold; as well as In response to determining that the probability is greater than the predetermined threshold: presenting the image to be detected, and displaying the probability and / or the bounding box corresponding to the position in the image to be detected; and / or The sample image is presented.

10. The method according to claim 1, further comprising: storing the sample image in a first memory; storing the target vector in a second memory, the second memory being the same as or different from the first memory; receiving an indication that the sample image is deleted; as well as In response to receiving an indication that the sample image is deleted, deleting the target vector.

11. A device for sensitive image detection, comprising: processor; as well as a memory storing computer executable instructions that, when executed, cause the processor to: receiving a sample image containing a sensitive object, Preprocessing the sample image to generate a target image, Encode the target image into a target vector, Receive the image to be detected, Encoding the image to be detected into a vector to be detected, and The probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected is calculated by performing correlation calculation on the target vector and the vector to be detected.

12. The device according to claim 11, wherein: The preprocessing of the sample image comprises: identifying a set of regions in the sample image by performing region proposal on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting, from the set of regions, regions with probabilities greater than a predetermined threshold; determining whether the number of extracted regions is greater than one; and In response to determining that the number of the extracted regions is greater than one, a sub-image corresponding to the extracted regions is cropped from the sample image as a target image.

13. The device according to claim 11, wherein: The preprocessing of the sample image comprises: identifying a set of regions in the sample image by performing region proposal on the sample image, each region containing an object and associated with a probability indicating that the region contains the object; extracting, from the set of regions, regions with probabilities greater than a predetermined threshold; determining whether the number of extracted regions is greater than one; In response to determining that the number of the extracted regions is greater than one, determining a sensitive region among the extracted regions that includes the sensitive object; and A sub-image corresponding to the sensitive area is cut out from the sample image as a target image.

14. The device according to claim 13, wherein: The determining of the sensitive area containing the sensitive object in the extracted area comprises: presenting the extracted regions by displaying bounding boxes corresponding to the extracted regions in the sample image, each region being selectable; receiving a selection of an extracted region; and The selected area is determined as the sensitive area.

15. The device according to claim 11, wherein The preprocessing of the sample image comprises: generating a rotated image by rotating a current image by a predetermined angle, the current image being the sample image or a sub-image in the sample image; Inputting the current image and the rotated image into a target tracking model, wherein the target tracking model is a model for calculating the probability that the image to be detected contains the sensitive object; Calculating, by means of the target tracking model, a probability that the rotated image contains the object in the current image; determining whether the calculated probability is less than a predetermined threshold; and In response to determining that the calculated probability is less than the predetermined threshold, the rotated image is used as a target image.

16. The device according to claim 11, wherein The preprocessing of the sample image comprises: determining whether to allow ignoring the color of a current image when encoding the current image, the current image being at least one of: the sample image, a sub-image in the sample image, a rotated image for the sample image, and a rotated image for the sub-image; and In response to determining that it is allowed to ignore the color of the current image when encoding the current image, color conversion is performed on the current image to generate a color-converted image as a target image.

17. The device according to claim 16, wherein: The performing color conversion on the current image comprises, for each pixel in the current image: Obtain the color of the pixel; determining a set of color channels for representing the color; randomly selecting two channels from the set of color channels; and The two values ​​corresponding to the two channels are swapped.

18. The device according to claim 11, wherein The computer executable instructions, when executed, further cause the processor to: determining whether the probability is greater than a predetermined threshold; and In response to determining that the probability is greater than the predetermined threshold: presenting the image to be detected, and displaying the probability and / or the bounding box corresponding to the position in the image to be detected; and / or The sample image is presented.

19. The device according to claim 11, wherein: The computer executable instructions, when executed, further cause the processor to: storing the sample image in a first memory; storing the target vector in a second memory, the second memory being the same as or different from the first memory; receiving an indication that the sample image is deleted; as well as In response to receiving an indication that the sample image is deleted, deleting the target vector.

20. A computer readable medium for sensitive image detection, comprising instructions which, when executed, cause a processor to: receiving a sample image containing a sensitive object; Preprocessing the sample image to generate a target image; encoding the target image into a target vector; Receiving an image to be detected; Encoding the image to be detected into a vector to be detected; and The probability that the image to be detected contains the sensitive object and / or the position of the sensitive object in the image to be detected is calculated by performing correlation calculation on the target vector and the vector to be detected.

Images