An image detection method, device, equipment and computer readable storage medium
By introducing convolutional and residual modules into the YOLOv3 model for feature extraction and fusion, the problem of insufficient feature map extraction in small target image detection is solved, thus improving the detection accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2022-07-29
- Publication Date
- 2026-07-21
AI Technical Summary
The existing YOLOv3 model suffers from insufficient feature map extraction when detecting small target images, resulting in low detection accuracy.
By employing the convolutional and residual modules in the initial image detection model to extract features from the sample images, combining feature maps at different levels for feature fusion, and training the model based on the sample images and target feature maps, a target image detection model is generated.
It improves the detection accuracy of small target images, and the generated target image detection model contains more shallow and deep feature information, resulting in more accurate detection results.
Smart Images
Figure CN116824315B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and more particularly to an image detection method, apparatus, device, and computer-readable storage medium. Background Technology
[0002] With the continuous development of deep learning, image detection technology based on deep learning has also made great progress. Currently, single-stage detection models (You Look Only Once v3, YOLOv3) are commonly used for image detection. The YOLOv3 model extracts feature maps from different levels of the image and constructs feature vectors, then trains the model based on these feature vectors. However, when using the YOLOv3 model to detect small objects, there is a problem of insufficient feature map extraction, resulting in low detection accuracy for small objects. Summary of the Invention
[0003] To address the aforementioned technical problems, this application aims to provide an image detection method, apparatus, device, and computer-readable storage medium that can solve the problem of insufficient feature map extraction when detecting small target images in related technologies, thereby improving the detection accuracy of small target images.
[0004] The technical solution of this application is implemented as follows:
[0005] An image detection method, the method comprising:
[0006] Obtain a sample image of the target object; wherein the sample image contains sensitive information about the target object;
[0007] The sample image is used to extract features using the convolution module in the initial image detection model to obtain a first feature map, and the first feature map is used to extract features using the residual module in the initial image detection model to obtain a second feature map;
[0008] Based on the first feature map and the second feature map, feature fusion is performed to obtain the target feature map;
[0009] The initial image detection model is trained based on the sample image and the target feature map to obtain the target image detection model;
[0010] The target image detection model is used to detect the image to be detected to determine whether the image to be detected contains sensitive information.
[0011] In the above scheme, the step of using the convolution module in the initial image detection model to extract features from the sample image to obtain the first feature map includes:
[0012] The sample image is processed by convolutional layer of the convolution module to obtain the first sub-feature map;
[0013] The first sub-feature map is standardized using the normalization function of the convolution module to obtain the second sub-feature map;
[0014] The activation function of the convolution module is used to perform nonlinear processing on the second sub-feature map to obtain the first feature map.
[0015] In the above scheme, the step of using the residual module in the initial image detection model to extract features from the first feature map to obtain the second feature map includes:
[0016] The first feature map is processed using the first residual module to obtain the first residual feature map;
[0017] The first residual feature map is processed using the second residual module to obtain the second residual feature map;
[0018] The first feature map and the second residual feature map are fused along the channel dimension to obtain the second feature map.
[0019] In the above scheme, obtaining the target feature map based on the first feature map and the second feature map includes:
[0020] The size of the first feature map is processed, and the size-processed first feature map is standardized and nonlinearized to obtain the third feature map;
[0021] The second feature map is standardized and nonlinearized to obtain the fourth feature map;
[0022] The third feature map and the fourth feature map are fused along the channel dimension to obtain the target feature map.
[0023] In the above scheme, processing the first feature map to obtain the third feature map includes:
[0024] The first feature map is resized to obtain the third sub-feature map;
[0025] The third sub-feature map is standardized using a normalization function to obtain the fourth sub-feature map;
[0026] The fourth sub-feature map is nonlinearly processed using an activation function to obtain the third feature map.
[0027] In the above scheme, the process of processing the second feature map to obtain the fourth feature map includes:
[0028] The second feature map is standardized using a normalization function to obtain the fifth sub-feature map;
[0029] The fifth sub-feature map is nonlinearly processed using an activation function to obtain the fourth feature map.
[0030] In the above scheme, training the initial image detection model based on the sample image and the target feature map to obtain the target image detection model includes:
[0031] The target feature map is processed by the first, second and third processes respectively to obtain the fifth, sixth and seventh feature maps;
[0032] The feature maps at different levels of the fifth, sixth, and seventh feature maps are determined respectively;
[0033] The initial image detection model is trained based on the sample image and the feature maps at different levels to obtain the target image detection model.
[0034] In the above scheme, the feature maps at different levels that determine the fifth, sixth, and seventh feature maps respectively include:
[0035] The fifth feature map is processed using the first multi-scale filter in the initial image detection model to obtain the sixth sub-feature map; wherein, the first multi-scale filter includes M convolutional layers; M is a positive integer greater than 1;
[0036] The sixth feature map and the seventh feature map are processed by the second multi-scale filter in the initial image detection model to obtain the seventh sub-feature map and the eighth sub-feature map; wherein, the second multi-scale filter includes N convolutional layers; N is a positive integer greater than 1.
[0037] In the above scheme, the step of training the initial image detection model based on the sample image and the feature maps at different levels to obtain the target image detection model includes:
[0038] The initial image detection model is trained based on the sample image, the sixth sub-feature map, the seventh sub-feature map, and the eighth sub-feature map to obtain the target image detection model.
[0039] An image detection device, the device comprising:
[0040] An acquisition unit is used to acquire a sample image of a target object; wherein the sample image contains sensitive information about the target object;
[0041] The processing unit is configured to use the convolution module in the initial image detection model to extract features from the sample image to obtain a first feature map, and use the residual module in the initial image detection model to extract features from the first feature map to obtain a second feature map;
[0042] A determining unit is configured to perform feature fusion based on the first feature map and the second feature map to obtain a target feature map;
[0043] The processing unit is further configured to train the initial image detection model based on the sample image and the target feature map to obtain a target image detection model;
[0044] The detection unit is used to use the target image detection model to detect the image to be detected and determine whether the image to be detected contains sensitive information.
[0045] An image detection device, the device comprising: a processor, a memory, and a communication bus;
[0046] The communication bus is used to realize the communication connection between the processor and the memory;
[0047] The processor is used to execute the image detection program stored in the memory to implement the steps of the image detection method described above.
[0048] A computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the image detection method described above.
[0049] The image detection method, apparatus, device, and computer-readable storage medium provided in this application can acquire sample images of target objects containing sensitive information. A first feature map is obtained by using a convolutional module in an initial image detection model to extract features from the sample image. A second feature map is obtained by using a residual module in the initial image detection model to extract features from the first feature map. Then, a target feature map is obtained based on the first and second feature maps and fused. The initial image detection model is trained based on the sample image and the target feature map to obtain a target image detection model. Finally, the target image detection model is used to detect the image to be detected to determine whether the image contains sensitive information. The target image detection model is trained based on the target feature map determined by the first feature map extracted by the convolution module and the second feature map extracted by the residual module. Since the convolution module extracts shallow feature information and the residual module extracts deep feature information, the generated target image detection model contains both shallow and deep feature information. Therefore, when using the generated target image detection model to detect sensitive information in an image, the detection result is more accurate. This solves the problem of insufficient feature map extraction when detecting small target images in related technologies, which leads to low detection accuracy for small target images. Attached Figure Description
[0050] Figure 1 A schematic flowchart of an image detection method provided in an embodiment of this application;
[0051] Figure 2 This is a schematic diagram of a convolution module in an image detection method provided in an embodiment of this application;
[0052] Figure 3 A flowchart illustrating another image detection method provided in an embodiment of this application;
[0053] Figure 4 A schematic diagram of the first residual module in an image detection method provided in an embodiment of this application;
[0054] Figure 5 This is a schematic diagram illustrating the processing of a target feature map in an image detection method provided in an embodiment of this application;
[0055] Figure 6 A schematic diagram illustrating the determination of different or branched feature maps in an image detection method provided in an embodiment of this application;
[0056] Figure 7 A schematic diagram of a first multi-scale filter in an image detection method provided in an embodiment of this application;
[0057] Figure 8A schematic diagram of the backbone network in an image detection method provided in an embodiment of this application;
[0058] Figure 9 A schematic diagram of a second multi-scale filter in an image detection method provided in an embodiment of this application;
[0059] Figure 10 This is a schematic diagram illustrating the training of a model in an image detection method provided in an embodiment of this application;
[0060] Figure 11 This is a schematic diagram of the structure of an image detection device provided in an embodiment of this application;
[0061] Figure 12 This is a schematic diagram of the structure of an image detection device provided in an embodiment of this application. Detailed Implementation
[0062] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.
[0063] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.
[0064] This application provides an image detection method, which can be applied to an image detection device. (Refer to...) Figure 1 As shown, the method includes the following steps:
[0065] Step 101: Obtain a sample image of the target object.
[0066] The sample images contain sensitive information about the target object.
[0067] In the embodiments of this application, the sample image may refer to an image in which the sensitive parts of the target object are directly exposed; in one feasible implementation, the target object may be a person.
[0068] It should be noted that the sample images can be obtained from websites using web crawling algorithms. Specifically, web crawling algorithms can be used to analyze the information on a website to obtain the web address where the sample images are located. Then, the web page corresponding to the web address is accessed, and a large number of images in which the sensitive parts of the person are directly exposed are downloaded from the corresponding web page to obtain the sample images.
[0069] Step 102: Use the convolution module in the initial image detection model to extract features from the sample image to obtain the first feature map, and use the residual module in the initial image detection model to extract features from the first feature map to obtain the second feature map.
[0070] The first feature map can refer to an image capable of representing first-level feature information of the sample image; the second feature map can refer to an image capable of representing second-level feature information of the sample image. In this embodiment, a convolution module can be used to operate on the sample image to obtain the first feature map, and similarly, a residual module can be used to operate on the first feature map to obtain the second feature map. In one feasible implementation, the convolution module can be represented by DBL, referring to... Figure 2 As shown, the convolutional module (DBL) includes a convolutional layer (Conv), a batch normalization layer (BN), and an activation layer (RELU).
[0071] Step 103: Perform feature fusion based on the first feature map and the second feature map to obtain the target feature map.
[0072] In this embodiment of the application, the first feature map and the second feature map can be processed, and then the processed first feature map and the second feature map can be fused to obtain the target feature map.
[0073] Step 104: Train the initial image detection model based on the sample image and the target feature map to obtain the target image detection model.
[0074] In this embodiment, the target image detection model can be obtained by training an initial image detection model using a neural network algorithm based on sample images and target feature maps. In one feasible implementation, the initial image detection model can be pre-trained on the ImageNet dataset using transfer learning. After pre-training, the initial image detection model can be trained again using sample images and target feature maps. It should be noted that when training the initial image detection model, the initial learning rate can be set to 0.001, the batch size to 8, the weights to 0.0001, the momentum to 0.9, and the number of training iterations to 50,000. After the 30,000th training iteration, the learning rate can be set to 0.0001, and after the 40,000th training iteration, the learning rate can be set to 0.00001. When the loss curve of the initial image detection model stabilizes, the training of the initial image detection model is complete, and the target image detection model is obtained.
[0075] It should be noted that before training the initial image detection model, a training environment for training the initial image detection model needs to be set up. In one feasible implementation, the operating system in the training environment can be Ubuntu 18.04.1 LTS, the central processing unit (CPU) can be an Intel Core i7 8750H, the memory can be 16G, the graphics card can be an NVIDIA GeForce GTX2080Ti, and the deep learning framework can be the Python machine learning library (PyTorch) framework.
[0076] Step 105: Use a target image detection model to detect the image to be detected and determine whether there is sensitive information in the image to be detected.
[0077] In this embodiment of the application, the image to be detected can be input into the target image detection model for testing, thereby determining whether the image to be detected contains information that directly exposes the sensitive parts of the target object.
[0078] The image detection method provided in this application is based on training a target image detection model using a target feature map determined by a first feature map extracted by a convolution module and a second feature map extracted by a residual module. Because the convolution module extracts shallow feature information and the residual module extracts deep feature information, the generated target image detection model contains both more shallow and deep feature information. Therefore, when using the generated target image detection model to detect sensitive information in an image, the detection result is more accurate. This solves the problem in related technologies where feature map extraction is insufficient when detecting small target images, resulting in low detection accuracy for small target images.
[0079] Based on the foregoing embodiments, this application provides another image detection method, referring to... Figure 3 As shown, the method includes the following steps:
[0080] Step 201: The image detection device acquires a sample image of the target object.
[0081] The sample images contain sensitive information about the target object.
[0082] Step 202: The image detection device uses the convolutional layer of the convolutional module to perform convolution processing on the sample image to obtain the first sub-feature image.
[0083] The purpose of convolution processing is to extract different features from different input images.
[0084] In this embodiment, the first sub-feature map can be obtained by performing logical operations between the pixels in the convolution kernel corresponding to the convolutional layer and the pixels at the corresponding positions in the sample image; in one feasible implementation, the size of the first sub-feature map can be 208*208*64.
[0085] Step 203: The image detection device uses the normalization function of the convolution module to standardize the first sub-feature map to obtain the second sub-feature map.
[0086] Wherein, the normalization function can refer to a function that summarizes the statistical distribution of the sample images; the second sub-feature map can refer to the one obtained after standardizing the first sub-feature map. In one feasible implementation, the size of the second sub-feature map can be 208*208*64, and the first sub-feature map can be standardized using the batch normalization (BN) function as shown in the following expression (1):
[0087]
[0088] It should be noted that μ B and x represents the mean and variance of the batch data. i This represents the input feature vector. This represents the output feature vector, and ε can be set to 0.0001.
[0089] Step 204: The image detection device uses the activation function of the convolution module to perform nonlinear processing on the second sub-feature map to obtain the first feature map.
[0090] The activation function can refer to a function that incorporates nonlinear factors that enhance the network model.
[0091] In this embodiment, the second sub-feature map can be nonlinearly processed using the Sigmoid activation function, the Tanh activation function, or the Rectified Linear Unit (RELU). In one feasible implementation, the RELU activation function as shown in expression (2) below can be used to nonlinearly process the second sub-feature map.
[0092]
[0093] Step 205: The image detection device uses the first residual module of the residual module to process the first feature map to obtain the first residual feature map.
[0094] In the embodiments of this application, reference is made to Figure 4 As shown, the first residual module includes two convolutional modules. It can fuse the first feature map and the first feature map processed by the first residual module in the channel dimension to obtain the first residual feature map. That is, the first feature map is processed by the two convolutional modules in sequence to obtain the feature map after two convolutions. Then, the feature map after two convolutions and the first feature map are added in the channel dimension to obtain the first residual feature map.
[0095] Step 206: The image detection device uses the second residual module of the residual module to process the first residual feature map to obtain the second residual feature map.
[0096] In this embodiment of the application, the second residual module includes two first residual modules; the first residual feature map can be input into one first residual module to obtain a residual module feature map, and then the obtained residual module feature map can be input into another first residual module to obtain the second residual feature map.
[0097] Step 207: The image detection device fuses the first feature map and the second residual feature map in the channel dimension to obtain the second feature map.
[0098] Specifically, the first feature map and the second residual feature map can be added together along the channel dimension to obtain the second feature map; in one feasible implementation, the number of channels in the second feature map can be 128; it should be noted that the size of the first feature map is the same as the size of the second feature map.
[0099] Step 208: The image detection device processes the size of the first feature map, and then standardizes and nonlinearizes the size-processed first feature map to obtain the third feature map.
[0100] In this embodiment, the size parameter may refer to data characterizing the size of the first feature map; the size of the first feature map may be processed first, and then the processed first feature map may be standardized and nonlinearized in sequence to obtain the third feature map.
[0101] It should be noted that step 208 can be achieved in the following way:
[0102] Step 208a1: The image detection device performs a size transformation on the first feature map to obtain the third sub-feature map.
[0103] In this embodiment of the application, the first feature map can be downsampled to obtain the third sub-feature map; that is, the third sub-feature map is obtained by reducing the size of the first feature map; in one feasible implementation, the size of the third sub-feature map can be 208*208*64, and the size transformation can be represented by DS.
[0104] Step 208a2: The image detection device uses a normalization function to standardize the third sub-feature map to obtain the fourth sub-feature map.
[0105] It should be noted that the normalization function used when processing the third sub-feature map can be the same as the normalization function used when standardizing the first sub-feature map; in one feasible implementation, the size of the fourth sub-feature map can be 208*208*64, and the normalization function can be represented by Nor.
[0106] Step 208a3: The image detection device uses an activation function to perform nonlinear processing on the fourth sub-feature map to obtain the third feature map.
[0107] It should be noted that the activation function used when processing the fourth sub-feature map can be the same as the activation function used when processing the second sub-feature map; in one feasible implementation, the activation function can be represented by Act.
[0108] Step 209: The image detection device performs standardization and nonlinearization processing on the second feature map to obtain the fourth feature map.
[0109] Here, the size parameter can refer to data that characterizes the size of the second feature map.
[0110] It should be noted that step 209 can be achieved in the following way:
[0111] Step 209b1: The image detection device uses a normalization function to standardize the second feature map to obtain the fifth sub-feature map.
[0112] It should be noted that the normalization function used when processing the second feature map can be the same as the normalization function used when standardizing the third sub-feature map; in one feasible implementation, the size of the fifth sub-feature map can be 104*104*128.
[0113] Step 209b2: The image detection device uses an activation function to perform nonlinear processing on the fifth sub-feature map to obtain the fourth feature map.
[0114] It should be noted that the activation function used when processing the fifth sub-feature map can be the same as the activation function used when processing the fourth sub-feature map.
[0115] Step 210: The image detection device fuses the third feature map and the fourth feature map in the channel dimension to obtain the target feature map.
[0116] The target feature map represents the feature map obtained by adding the third feature map and the fourth feature map in the channel dimension; the target feature map can refer to the fused feature map.
[0117] In this embodiment of the application, the second feature map can be obtained by adding the number of channels of the first feature map to the number of channels of the second residual feature map; in one feasible implementation, the size of the target feature map can be 104*104*192 (that is, the number of channels of the second feature map can be 192), and the fusion can be represented by Con.
[0118] In this embodiment of the application, the entire process of fusing the third feature map and the fourth feature map in the channel dimension can be represented by the following expression (3):
[0119] f fused =Con(DS(Act(Nor(Cov1_1))),Act(Nor(Res2_2))) Expression (3)
[0120] Where Con represents fusion, DS represents size transformation, Act represents activation function, and Nor represents normalization function.
[0121] It should be noted that after obtaining the target feature map, it is necessary to reduce its dimensionality (i.e., dimensionality reduction). Convolution operations can be used to reduce the dimensionality of the target feature map. In one feasible implementation, the size of the target feature map is 104*104*192, the size of the convolution kernel is 1*1*128, and the convolution stride is 2. This results in a target feature map with a size of 104*104*128 after dimensionality reduction.
[0122] In one feasible implementation, refer to Figure 5 As shown, with a first feature map size of 208*208*64 and a second feature map size of 104*104*128, the first feature map is sequentially downsampled, standardized, and activated to obtain a processed first feature map with a size of 104*104*64. The second feature map is sequentially standardized and activated to obtain a processed second feature map with a size of 104*104*128. The feature map with a size of 104*104*64 and the feature map with a size of 104*104*128 are added together along the channel dimension to obtain a fused feature map with a size of 104*104*192. After obtaining the fused feature map, a 1*1*128 convolutional kernel can be used to reduce the dimensionality of the fused feature map to obtain a dimensionality-reduced feature map with a size of 104*104*128.
[0123] Step 211: The image detection device performs first processing, second processing and third processing on the target feature map respectively to obtain the fifth feature map, the sixth feature map and the seventh feature map.
[0124] In this embodiment of the application, the target feature map can be first processed to obtain the fifth feature map, then processed to obtain the sixth feature map, and then processed to obtain the seventh feature map.
[0125] In one feasible implementation, refer to Figure 6 As shown in the dashed box, the process of obtaining the fifth feature map is as follows: First, the target feature map is processed using 8 first residual modules (Res*8) to obtain the processed feature map, which is denoted as the first residual feature map; then, the first residual feature map is processed using 8 first residual modules (Res*8) to obtain the processed feature map, which is denoted as the second residual feature map; then, the second residual feature map is processed using 4 first residual modules (Res*4) to obtain the processed feature map, which is denoted as the third residual feature map; finally, the third residual feature map is processed using 5 first residual modules (Res*4) to obtain the fifth feature map.
[0126] The process of obtaining the sixth feature map is as follows: the fifth feature map mentioned above is input into the convolutional module (DBL) to obtain the first convolutional feature map. The first convolutional feature map is then upsampled to obtain the upsampled feature map, which is denoted as the first upsampled feature map. The second residual feature map mentioned above is then concatenated with the first upsampled feature map to obtain the concatenated feature map, which is denoted as the first concatenated feature map. The first concatenated feature map is then sequentially input into 5 convolutional modules (DBL*5) to obtain the sixth feature map.
[0127] The process of obtaining the seventh feature map is as follows: the sixth feature map obtained above is input into the convolution module to obtain the second convolution feature map, and then the second convolution feature map is upsampled to obtain the upsampled feature map, which is denoted as the second upsampled feature map; then the first residual feature map and the second upsampled feature map are concatenated to obtain the concatenated feature map, which is denoted as the second concatenated feature map, and then the second concatenated feature map is input into 5 convolution modules (DBL*5) in sequence to obtain the seventh feature map.
[0128] Step 212: The image detection device determines the feature maps at different levels of the fifth feature map, the sixth feature map, and the seventh feature map.
[0129] In the embodiments of this application, features can be extracted from the fifth feature map using convolutional kernels of different levels to obtain feature maps of different levels of the fifth feature. Similarly, features can be extracted from the sixth feature map using convolutional kernels of different levels to obtain feature maps of different levels of the sixth feature. Likewise, features can be extracted from the seventh feature map using convolutional kernels of different levels to obtain feature maps of different levels of the seventh feature. It should be noted that the levels of the convolutional kernels used to process the fifth, sixth, and seventh feature maps can be the same or different.
[0130] It should be noted that step 212 can be achieved in the following way:
[0131] Step 212c1: The image detection device uses the first multi-scale filter in the initial image detection model to process the fifth feature map to obtain the sixth sub-feature map.
[0132] The first multi-scale filter includes M convolutional layers, where M is a positive integer greater than 1; and the feature map output from the previous layer can be directly input into the first multi-scale filter for convolution processing.
[0133] In one feasible implementation, let's take M=3 as an example: (Refer to...) Figure 7 As shown, the kernel sizes used in the three convolutional layers of the first multi-scale filter are 1*1, 3*3, and 5*5, respectively. It should be noted that the 5*5 kernel can be replaced by two cascaded 3×3 kernels, and the parameters of the two cascaded 3*3 kernels only account for 18 / 25 of the parameters of a single 5*5 kernel, which can reduce the number of parameters in the network. In this way, by activating the multi-scale receptive field of the deep feature map, the semantic information of the deep features is enhanced, and the target background of different sizes can be captured in the feature map, adding contextual information of various scales.
[0134] In this embodiment of the application, the process of processing the feature map using the first multi-scale filter can be represented by the following expression (4):
[0135]
[0136] in, Let f represent the first multi-scale filter. j Let m represent the activation function, Ψ represent the high layers in the backbone network, and m represent the high layers in the backbone network. j This represents the feature map output by the j-th layer.
[0137] In one feasible implementation, refer to Figure 8As shown, the backbone network can be an improved DarkNet53; DarkNet53 has one convolutional module (DBL) and five residual groups, namely Res1, Res2, Res3, Res4 and Res5, each residual group containing the same or different number of residual modules; and feature fusion is added to the backbone network. Feature fusion can receive feature maps output from DBL and Res2 and fuse the two feature maps. The fused feature map will be input to Res3. The fused feature map has more shallow detail information, which is beneficial for the detection of small object images.
[0138] In one feasible implementation, the number of channels for 1*1 convolutional kernels, 3*3 convolutional kernels, and 5*5 convolutional kernels can all be 256.
[0139] Step 212c2: The image detection device uses the second multi-scale filter in the initial image detection model to process the sixth feature map and the seventh feature map respectively, to obtain the seventh sub-feature map and the eighth sub-feature map.
[0140] The second multi-scale filter includes N convolutional layers, where N is a positive integer greater than 1, and the feature map output from the previous layer can be directly input into the second multi-scale filter for convolution processing.
[0141] In one feasible implementation, let's take N including 4 as an example: (Refer to...) Figure 9 As shown, the convolutional kernel sizes used in the four convolutional layers of the second multi-scale filter are 1*1, 3*3, one 5*5, and 7*7, respectively. It should be noted that the 5*5 convolutional kernel can be replaced by two cascaded 3×3 convolutional kernels, and the 7*7 convolutional kernel can be replaced by three cascaded 3*3 convolutional kernels. In this way, by activating the multi-scale receptive field of the deep feature map, the semantic information of the deep features is enhanced, and the target background of different sizes can be captured in the feature map, adding contextual information of various scales.
[0142] In this embodiment of the application, the process of processing the feature map using a second multi-scale filter can be represented by the following expression (5):
[0143]
[0144] in, G represents the second multi-scale filter. j Let m represent the activation function, Ψ represent the high layers in the backbone network, and m represent the high layers in the backbone network. j This represents the feature map output by the j-th layer.
[0145] In one feasible implementation, the number of channels for 1*1 convolutional kernels, 3*3 convolutional kernels, 5*5 convolutional kernels, and 7*7 convolutional kernels can all be 200.
[0146] It should be noted that the embodiments in this application are only illustrated by the example that the number of convolutional layers in the second multi-scale filter is greater than the number of convolutional layers in the first multi-scale filter. The specific settings can be adjusted according to the actual situation.
[0147] Step 213: The image detection device trains the initial image detection model based on the sample image and feature maps at different levels to obtain the target image detection model.
[0148] The feature information in the feature maps at different levels is different.
[0149] It should be noted that when step 212 is implemented through steps 202c1 and 202c2, step 213 can be implemented through step 213d:
[0150] Step 213d: The image detection device trains the initial image detection model based on the sample image, the sixth sub-feature map, the seventh sub-feature map, and the eighth sub-feature map to obtain the target image detection model.
[0151] In this embodiment of the application, the target image detection model can be obtained by training an initial image detection model using a neural network algorithm based on sample images and the sixth, seventh, and eighth sub-feature maps.
[0152] In the embodiments of this application, reference is made to Figure 10 As shown, the acquired dataset can be divided into training and test sets according to a certain ratio. Then, the training and test sets are preprocessed to obtain preprocessed training and test sets. The initial image detection model is then trained using the training set. After training is completed, the target image detection model can be obtained.
[0153] It should be noted that the target image detection model can be evaluated by its detection accuracy. In one feasible implementation, the detection accuracy of the target image detection model can be measured by calculating evaluation criteria such as average precision (AP), mean average precision (mAP), and detection speed. Among them, detection speed can refer to the average detection time of an image. AP can be calculated by the following expression (6), and mAP can be calculated by the following expression (7):
[0154] AP(c)=∫Precision(c)dRecall(c) Expression (6)
[0155]
[0156] In the above expression (6), Precise represents precision, Recall represents recall, and K in the above expression (7) represents the number of image categories. It should be noted that when there is only one type of image, AP and mAP are equal.
[0157] In one feasible implementation, the target image detection model can be the YOLOv3-I detection model. Comparing the detection results of the YOLOv3-I model with those of the unmodified YOLOv3 model, the average detection time for YOLOv3-I is 0.063 seconds, and its average accuracy is 77.1%. In contrast, the average detection time for YOLOv3 is 0.035 seconds, and its average accuracy is 76.86%. The results show that the average accuracy of YOLOv3-I is higher than that of YOLOv3, indicating better detection performance. Although the average detection time of YOLOv3-I is longer than that of YOLOv3, it still meets the real-time requirements.
[0158] Step 214: The image detection device uses a target image detection model to detect the image to be detected and determine whether there is sensitive information in the image to be detected.
[0159] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.
[0160] The image detection method provided in this application is based on training a target image detection model using a target feature map determined by a first feature map extracted by a convolution module and a second feature map extracted by a residual module. Because the convolution module extracts shallow feature information and the residual module extracts deep feature information, the generated target image detection model contains both more shallow and deep feature information. Therefore, when using the generated target image detection model to detect sensitive information in an image, the detection result is more accurate. This solves the problem in related technologies where feature map extraction is insufficient when detecting small target images, resulting in low detection accuracy for small target images.
[0161] Based on the foregoing embodiments, embodiments of this application provide an image detection device, which can be applied to... Figure 1 and 3 In the image detection method provided in the corresponding embodiment, refer to Figure 11 As shown, the device 3 may include: an acquisition unit 31, a determination unit 32, and a processing unit 33, wherein:
[0162] The acquisition unit 31 is used to acquire a sample image of the target object; wherein the sample image contains sensitive information of the target object;
[0163] The processing unit 32 is used to extract features from the sample image using the convolution module in the initial image detection model to obtain a first feature map, and to extract features from the first feature map using the residual module in the initial image detection model to obtain a second feature map;
[0164] The determining unit 33 is used to perform feature fusion based on the first feature map and the second feature map to obtain the target feature map;
[0165] Processing unit 33 is also used to train the initial image detection model based on the sample image and the target feature map to obtain the target image detection model;
[0166] The detection unit 33 is also used to use a target image detection model to detect the image to be detected and determine whether there is sensitive information in the image to be detected.
[0167] In this embodiment of the application, the processing unit 32 is further configured to perform the following steps:
[0168] The sample image is processed by convolutional layers of the convolution module to obtain the first sub-feature map;
[0169] The first sub-feature map is standardized using the normalization function of the convolution module to obtain the second sub-feature map;
[0170] The activation function of the convolution module is used to perform non-linear processing on the second sub-feature map to obtain the first feature map.
[0171] In this embodiment of the application, the processing unit 32 is further configured to perform the following steps:
[0172] The first feature map is processed using the first residual module to obtain the first residual feature map;
[0173] The second residual module is used to process the first residual feature map to obtain the second residual feature map;
[0174] The first feature map and the second residual feature map are fused along the channel dimension to obtain the second feature map.
[0175] In this embodiment of the application, the determining unit 33 is further configured to perform the following steps:
[0176] The size of the first feature map is processed, and the size-processed first feature map is standardized and nonlinearized to obtain the third feature map;
[0177] The second feature map is standardized and nonlinearized to obtain the fourth feature map.
[0178] The third and fourth feature maps are fused along the channel dimension to obtain the target feature map.
[0179] In this embodiment of the application, the determining unit 33 is further configured to perform the following steps:
[0180] The first feature map is resized to obtain the third sub-feature map;
[0181] The third sub-feature map is standardized using a normalization function to obtain the fourth sub-feature map;
[0182] The fourth sub-feature map is nonlinearly processed using an activation function to obtain the third feature map.
[0183] In this embodiment of the application, the determining unit 33 is further configured to perform the following steps:
[0184] The second feature map is standardized using a normalization function to obtain the fifth sub-feature map;
[0185] The fifth sub-feature map is nonlinearly processed using an activation function to obtain the fourth feature map.
[0186] In this embodiment of the application, the processing unit 33 is further configured to perform the following steps:
[0187] The target feature map is processed by the first, second and third processes respectively to obtain the fifth, sixth and seventh feature maps;
[0188] The feature maps at different levels of the fifth, sixth, and seventh feature maps are determined respectively;
[0189] The initial image detection model is trained based on sample images and feature maps at different levels to obtain the target image detection model.
[0190] In this embodiment of the application, the processing unit 33 is further configured to perform the following steps:
[0191] The fifth feature map is processed using the first multi-scale filter in the initial image detection model to obtain the sixth sub-feature map; wherein, the first multi-scale filter includes M convolutional layers; M is a positive integer greater than 1;
[0192] The sixth and seventh feature maps are processed by the second multi-scale filter in the initial image detection model to obtain the seventh and eighth sub-feature maps. The second multi-scale filter consists of N convolutional layers, where N is a positive integer greater than 1.
[0193] In this embodiment of the application, the detection unit 33 is further configured to perform the following steps:
[0194] The initial image detection model is trained based on the sample image, the sixth sub-feature map, the seventh sub-feature map, and the eighth sub-feature map to obtain the target image detection model.
[0195] It should be noted that the specific implementation process of the steps performed by each unit in this embodiment can be referred to Figure 1 and 3 The implementation process of the image detection method provided in the corresponding embodiments will not be described in detail here.
[0196] The image detection apparatus provided in this application obtains a target image detection model based on a target feature map determined by a first feature map extracted by a convolution module and a second feature map extracted by a residual module. Because the convolution module extracts shallow feature information and the residual module extracts deep feature information, the generated target image detection model contains both more shallow and deep feature information. Therefore, when using the generated target image detection model to detect sensitive information in an image, the detection result is more accurate. This solves the problem in related technologies where feature map extraction is insufficient when detecting small target images, resulting in low detection accuracy for small target images.
[0197] Based on the foregoing embodiments, embodiments of this application provide an image detection device that can be applied to... Figure 1 and 3 The image detection method provided in the corresponding embodiment is as follows: Figure 12 As shown, the device 4 may include a processor 41, a memory 42, and a communication bus 43;
[0198] Communication bus 43 is used to realize the communication connection between processor 41 and memory 42;
[0199] The processor 41 is used to execute the image detection program stored in the memory 42 to perform the following steps:
[0200] Obtain a sample image of the target object; wherein the sample image contains sensitive information about the target object;
[0201] The first feature map is obtained by using the convolution module in the initial image detection model to extract features from the sample image, and the second feature map is obtained by using the residual module in the initial image detection model to extract features from the first feature map.
[0202] The target feature map is obtained by fusing features based on the first and second feature maps.
[0203] The initial image detection model is trained based on the sample images and target feature maps to obtain the target image detection model;
[0204] A target image detection model is used to detect the image to determine whether it contains sensitive information.
[0205] In other embodiments of this application, the processor 41 is used to execute the convolution module in the initial image detection model stored in the memory 42 to extract features from the sample image to obtain a first feature map, in order to implement the following steps:
[0206] The sample image is processed by convolutional layers of the convolution module to obtain the first sub-feature map;
[0207] The first sub-feature map is standardized using the normalization function of the convolution module to obtain the second sub-feature map;
[0208] The activation function of the convolution module is used to perform non-linear processing on the second sub-feature map to obtain the first feature map.
[0209] In other embodiments of this application, the processor 41 is used to execute the residual module in the memory 42 to extract features from the first feature map to obtain a second feature map, in order to perform the following steps:
[0210] The first feature map is processed using the first residual module to obtain the first residual feature map;
[0211] The second residual module is used to process the first residual feature map to obtain the second residual feature map;
[0212] The first feature map and the second residual feature map are fused along the channel dimension to obtain the second feature map.
[0213] In other embodiments of this application, the processor 41 is used to execute the memory 42 to obtain a target feature map based on the first feature map and the second feature map, in order to implement the following steps:
[0214] The size of the first feature map is processed, and the size-processed first feature map is standardized and nonlinearized to obtain the third feature map;
[0215] The second feature map is standardized and nonlinearized to obtain the fourth feature map.
[0216] The third and fourth feature maps are fused along the channel dimension to obtain the target feature map.
[0217] In other embodiments of this application, the processor 41 is used to process the first feature map stored in the memory 42 to obtain a third feature map, in order to implement the following steps:
[0218] The first feature map is resized to obtain the third sub-feature map;
[0219] The third sub-feature map is standardized using a normalization function to obtain the fourth sub-feature map;
[0220] The fourth sub-feature map is nonlinearly processed using an activation function to obtain the third feature map.
[0221] In other embodiments of this application, the processor 41 is used to process the second feature map stored in the memory 42 to obtain a fourth feature map, in order to perform the following steps:
[0222] The second feature map is standardized using a normalization function to obtain the fifth sub-feature map;
[0223] The fifth sub-feature map is nonlinearly processed using an activation function to obtain the fourth feature map.
[0224] In other embodiments of this application, processor 41 is used to train an initial image detection model based on sample images and target feature maps stored in memory 42 to obtain a target image detection model, thereby implementing the following steps:
[0225] The target feature map is processed by the first, second and third processes respectively to obtain the fifth, sixth and seventh feature maps;
[0226] The feature maps at different levels of the fifth, sixth, and seventh feature maps are determined respectively;
[0227] The initial image detection model is trained based on sample images and feature maps at different levels to obtain the target image detection model.
[0228] In other embodiments of this application, the processor 41 is used to execute feature maps of different levels that are stored in the memory 42 to determine the fifth feature map, the sixth feature map, and the seventh feature map, respectively, to perform the following steps:
[0229] The fifth feature map is processed using the first multi-scale filter in the initial image detection model to obtain the sixth sub-feature map; wherein, the first multi-scale filter includes M convolutional layers; M is a positive integer greater than 1;
[0230] The sixth and seventh feature maps are processed by the second multi-scale filter in the initial image detection model to obtain the seventh and eighth sub-feature maps. The second multi-scale filter consists of N convolutional layers, where N is a positive integer greater than 1.
[0231] In other embodiments of this application, the processor 41 is used to execute feature maps at different levels in the memory 42 to train the initial image detection model and obtain the target image detection model to perform the following steps:
[0232] The initial image detection model is trained based on the sample image, the sixth sub-feature map, the seventh sub-feature map, and the eighth sub-feature map to obtain the target image detection model.
[0233] It should be noted that the specific implementation process of the steps executed by the processor in this embodiment can be referred to Figure 1 and 3 The implementation process of the image detection method provided in the corresponding embodiments will not be described in detail here.
[0234] The image detection device provided in this application embodiment obtains a target image detection model based on a target feature map determined by a first feature map extracted by a convolution module and a second feature map extracted by a residual module. Because the convolution module extracts shallow feature information and the residual module extracts deep feature information, the generated target image detection model contains both more shallow and deep feature information. Therefore, when using the generated target image detection model to detect sensitive information in an image, the detection result is more accurate. This solves the problem in related technologies where feature map extraction is insufficient when detecting small target images, resulting in low detection accuracy for small target images.
[0235] Based on the foregoing embodiments, embodiments of this application provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement... Figure 1 and 3 The corresponding embodiment provides the steps of the image detection method.
[0236] It should be noted that the aforementioned computer-readable storage media can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various electronic devices that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.
[0237] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0238] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0239] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0240] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0241] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0242] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0243] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. An image detection method, characterized in that, The method includes: Obtain a sample image of the target object; wherein the sample image contains sensitive information about the target object; The initial image detection model uses a convolutional module to extract features from the sample image to obtain a first feature map, and then uses a residual module in the initial image detection model to extract features from the first feature map to obtain a second feature map. The convolutional module includes a convolutional layer, a normalization function, and an activation function, used to sequentially perform convolution processing, normalization processing, and nonlinearization processing on the sample image. The residual module includes a first residual module and a second residual module; the first residual module includes two convolutional modules, and the second residual module includes two first residual modules. Based on the first feature map and the second feature map, feature fusion is performed to obtain a target feature map; wherein, the target feature map is the feature map obtained by adding the processed first feature map and the processed second feature map in the channel dimension; The initial image detection model is trained based on the sample image and the target feature map to obtain the target image detection model; The target image detection model is used to detect the image to be detected to determine whether the image to be detected contains sensitive information; The step of training the initial image detection model based on the sample image and the target feature map to obtain the target image detection model includes: The target feature map is processed by the first, second and third processes respectively to obtain the fifth, sixth and seventh feature maps; The fifth feature map is processed using the first multi-scale filter in the initial image detection model to obtain the sixth sub-feature map; wherein, the first multi-scale filter includes M convolutional layers; M is a positive integer greater than 1; The sixth feature map and the seventh feature map are processed by the second multi-scale filter in the initial image detection model to obtain the seventh sub-feature map and the eighth sub-feature map; wherein, the second multi-scale filter includes N convolutional layers; N is a positive integer greater than 1; The initial image detection model is trained based on the sample image, the sixth sub-feature map, the seventh sub-feature map, and the eighth sub-feature map to obtain the target image detection model.
2. The method according to claim 1, characterized in that, The step of using the convolution module in the initial image detection model to extract features from the sample image to obtain a first feature map includes: The sample image is processed by convolutional layer of the convolution module to obtain the first sub-feature map; The first sub-feature map is standardized using the normalization function of the convolution module to obtain the second sub-feature map; The activation function of the convolution module is used to perform nonlinear processing on the second sub-feature map to obtain the first feature map.
3. The method according to claim 1, characterized in that, The step of using the residual module in the initial image detection model to extract features from the first feature map to obtain the second feature map includes: The first feature map is processed using the first residual module to obtain the first residual feature map; The first residual feature map is processed using the second residual module to obtain the second residual feature map; The first feature map and the second residual feature map are fused along the channel dimension to obtain the second feature map.
4. The method according to claim 1, characterized in that, The step of obtaining the target feature map based on the first feature map and the second feature map includes: The size of the first feature map is processed, and the size-processed first feature map is standardized and nonlinearized to obtain the third feature map; The second feature map is standardized and nonlinearized to obtain the fourth feature map; The third feature map and the fourth feature map are fused along the channel dimension to obtain the target feature map.
5. The method according to claim 4, characterized in that, The process of processing the first feature map to obtain the third feature map includes: The first feature map is resized to obtain the third sub-feature map; The third sub-feature map is standardized using a normalization function to obtain the fourth sub-feature map; The fourth sub-feature map is nonlinearly processed using an activation function to obtain the third feature map.
6. The method according to claim 4, characterized in that, The process of processing the second feature map to obtain the fourth feature map includes: The second feature map is standardized using a normalization function to obtain the fifth sub-feature map; The fifth sub-feature map is nonlinearly processed using an activation function to obtain the fourth feature map.
7. An image detection device, characterized in that, The device includes: An acquisition unit is used to acquire a sample image of a target object; wherein the sample image contains sensitive information about the target object; The processing unit is configured to extract features from the sample image using the convolution module in the initial image detection model to obtain a first feature map, and to extract features from the first feature map using the residual module in the initial image detection model to obtain a second feature map; wherein, the convolution module includes a convolutional layer, a normalization function, and an activation function, and is used to sequentially perform convolution processing, normalization processing, and nonlinearization processing on the sample image; the residual module includes a first residual module and a second residual module, the first residual module including two convolutional modules, and the second residual module including two first residual modules; A determining unit is configured to perform feature fusion based on the first feature map and the second feature map to obtain a target feature map; wherein, the target feature map is a feature map obtained by adding the processed first feature map and the processed second feature map in the channel dimension; The processing unit is further configured to train the initial image detection model based on the sample image and the target feature map to obtain a target image detection model; The detection unit is used to use the target image detection model to detect the image to be detected and determine whether the image to be detected contains sensitive information. The processing unit is further configured to perform first, second, and third processing on the target feature map respectively to obtain a fifth feature map, a sixth feature map, and a seventh feature map; process the fifth feature map using a first multi-scale filter in the initial image detection model to obtain a sixth sub-feature map; the first multi-scale filter includes M convolutional layers; M is a positive integer greater than 1; process the sixth feature map and the seventh feature map using a second multi-scale filter in the initial image detection model respectively to obtain a seventh sub-feature map and an eighth sub-feature map; the second multi-scale filter includes N convolutional layers; N is a positive integer greater than 1; and train the initial image detection model based on the sample image, the sixth sub-feature map, the seventh sub-feature map, and the eighth sub-feature map to obtain a target image detection model.
8. An image detection device, characterized in that, The device includes: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the image detection program stored in the memory to implement the steps of the image detection method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the image detection method as described in any one of claims 1 to 6.