Image detection method and device, electronic equipment and readable storage medium

Abstract

This application discloses an image detection method, apparatus, electronic device, and readable storage medium, applied in the field of target detection technology. The image detection method includes: acquiring an image to be processed and determining all candidate boxes corresponding to the image to be processed; traversing all the candidate boxes and determining a first output feature map corresponding to the traversed candidate box, and adjacent output feature maps adjacent to the first output feature map; calculating a first feature vector corresponding to the first output feature map and adjacent feature vectors corresponding to the adjacent output feature maps based on the traversed candidate boxes; performing feature vector fusion on the first feature vector and the adjacent feature vectors to obtain a target feature vector; and performing image detection based on the target feature vector corresponding to each candidate box to obtain an image detection result. This application solves the technical problem of poor image detection performance in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to an image detection method and device, electronic equipment and a readable storage medium. BACKGROUND

[0002] With the rapid development of deep learning related technologies, various tasks in the field of computer vision have widely used deep learning technologies, and a target detection algorithm based on deep learning has been the most mainstream target detection algorithm, which has far exceeded traditional target detection algorithms in accuracy. At present, a target detection algorithm based on deep learning technology can be divided into two types of models, i.e., One-Stage and Two-Stage, according to the characteristics of the model network structure and the calculation process. The main representatives of the Two-Stage model are Faster R-CNN, Mask R-CNN and Cascade R-CNN, and the Two-Stage model is usually characterized by high accuracy but a slightly complex calculation process, which is a choice for pursuing accuracy. However, the Two-Stage model usually only uses a single scale output feature map, and the information of the target sample obtained is very limited, so that a large amount of information contained in the multi-scale output feature map cannot be reasonably used, resulting in poor detection effect of the image target. SUMMARY

[0003] The main purpose of the present application is to provide an image detection method, device, electronic equipment and computer readable storage medium, which aims to solve the technical problem of poor image detection effect in the prior art.

[0004] To achieve the above purpose, the present application provides an image detection method applied to an image detection device, which comprises the following steps:

[0005] An image to be processed is obtained, and all candidate boxes corresponding to the image to be processed are determined;

[0006] All the candidate boxes are traversed, a first output feature map corresponding to the traversed candidate box is determined, and an adjacent output feature map adjacent to the first output feature map is determined;

[0007] A first feature vector corresponding to the first output feature map and an adjacent feature vector corresponding to the adjacent output feature map are calculated according to the traversed candidate box;

[0008] The first feature vector and the adjacent feature vector are fused to obtain a target feature vector;

[0009] An image detection result is obtained by performing image detection according to the target feature vector corresponding to each candidate box.

[0010] Optionally, the step of traversing all the candidate boxes and determining the first output feature map corresponding to the traversed candidate box, and the adjacent output feature maps adjacent to the first output feature map, includes:

[0011] Obtain all scale feature maps corresponding to the image to be processed;

[0012] Traverse all the candidate boxes, determine the candidate box size of the traversed candidate boxes, input the candidate box size and each scale feature map into the second stage of the preset two-stage model for output feature map selection, and output the first output feature map corresponding to the traversed candidate boxes.

[0013] In each of the scale feature maps, neighboring output feature maps that are adjacent to the first output feature map are selected.

[0014] Optionally, the step of filtering out neighboring output feature maps adjacent to the first output feature map from each of the scale feature maps includes:

[0015] Determine the position of the first output feature map in each of the scale feature maps;

[0016] The feature maps in each scale feature map that are adjacent to the first output feature map are taken as adjacent output feature maps.

[0017] Optionally, the step of taking the feature maps in each of the scale feature maps that are adjacent to the first output feature map as adjacent output feature maps includes:

[0018] If the position of the first output feature map is the first position preceding each of the scale feature maps, then the second position preceding each of the scale feature maps is taken as the adjacent output feature map.

[0019] If the position of the first output feature map is the first position after each of the scale feature maps, then the second position after each of the scale feature maps is taken as the adjacent output feature map.

[0020] If the position of the first output feature map is located in the middle two positions of each of the scale feature maps, then the position before and the position after the first output feature map in each of the scale feature maps are taken as adjacent output feature maps.

[0021] Optionally, the step of calculating the first feature vector corresponding to the first output feature map and the neighboring feature vectors corresponding to the neighboring output feature maps based on the traversed candidate boxes includes:

[0022] Based on the traversed candidate boxes, the corresponding first feature region is cropped in the first output feature map, and the corresponding adjacent feature region is cropped in the adjacent output feature map;

[0023] The first feature region and the adjacent feature regions are input into the second stage of the preset two-stage model for feature extraction, and the first feature vector and the adjacent feature vector are obtained respectively.

[0024] Optionally, the step of fusing the first feature vector and the adjacent feature vectors to obtain the target feature vector includes:

[0025] Obtain the first vector weight of the first feature vector and the adjacent vector weights of the adjacent feature vectors;

[0026] Based on the first vector weight and the neighboring vector weight, the first feature vector and the neighboring feature vectors are fused to obtain the target feature vector.

[0027] Optionally, the step of determining all candidate boxes corresponding to the image to be processed includes:

[0028] The image to be processed is input into the first stage of a preset two-stage model for model training to generate a prediction box for the image to be processed.

[0029] If an object is detected in the prediction box, the prediction box is used as the foreground box;

[0030] Calculate the probability value of the object in each of the foreground boxes being the target object, and sort all the foreground boxes according to the probability value;

[0031] The foreground boxes at the top of the sorted list are used as candidate boxes for the image to be processed.

[0032] To achieve the above objectives, this application also provides an image detection apparatus, which is applied to an image detection device, and the image detection apparatus includes:

[0033] The candidate box module is used to acquire the image to be processed and determine all candidate boxes corresponding to the image to be processed.

[0034] The feature map module is used to traverse all the candidate boxes, determine the first output feature map corresponding to the traversed candidate box, and the adjacent output feature map adjacent to the first output feature map.

[0035] The vector module is used to calculate the first feature vector corresponding to the first output feature map and the adjacent feature vector corresponding to the adjacent output feature maps based on the traversed candidate boxes;

[0036] The fusion module is used to fuse the first feature vector and the adjacent feature vectors to obtain the target feature vector;

[0037] The detection module is used to perform image detection based on the target feature vector corresponding to each candidate box, and obtain the image detection result.

[0038] This application also provides an electronic device, the electronic device comprising: a memory, a processor, and a program of the image detection method stored in the memory and executable on the processor, wherein when the program of the image detection method is executed by the processor, it can implement the steps of the image detection method as described above.

[0039] This application also provides a readable storage medium, which is a computer-readable storage medium, on which a program implementing an image detection method is stored. When the program implementing the image detection method is executed by a processor, it implements the steps of the image detection method as described above.

[0040] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image detection method described above.

[0041] This application provides an image detection method, apparatus, electronic device, and computer-readable storage medium. The method involves acquiring an image to be processed and determining all candidate boxes corresponding to the image; traversing all candidate boxes to determine a first output feature map corresponding to each traversed candidate box, as well as adjacent output feature maps adjacent to the first output feature map. This overcomes the limitation of traditional deep learning-based two-stage models that extract features from a single-scale output feature map, fully utilizing the rich positional and semantic information contained in feature maps of different scales; calculating a first feature vector corresponding to the first output feature map and adjacent feature vectors corresponding to the adjacent output feature maps based on the traversed candidate boxes; fusing the first feature vector and the adjacent feature vectors to obtain a target feature vector; and performing image detection based on the target feature vector corresponding to each candidate box to obtain an image detection result. By fully utilizing the positional and semantic information contained in feature maps of multiple scales, this method can more accurately predict the category and location of target samples in an image, thereby further improving the object detection effect. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the first embodiment of the image detection method of this application;

[0045] Figure 2 This is a flowchart illustrating the second embodiment of the image detection method of this application;

[0046] Figure 3 This is a schematic diagram of the structure of the multi-scale output feature map in this application;

[0047] Figure 4 This is a flowchart illustrating the third embodiment of the image detection method of this application;

[0048] Figure 5 This is a schematic diagram of the structural composition of the image detection device of this application;

[0049] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the image detection method in the embodiments of this application.

[0050] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0051] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Example 1

[0053] This application provides an image detection method. In the first embodiment of the image detection method of this application, referring to... Figure 1 The image detection method includes:

[0054] Step S10: Obtain the image to be processed and determine all candidate boxes corresponding to the image to be processed;

[0055] Step S20: Traverse all the candidate boxes and determine the first output feature map corresponding to the traversed candidate box, as well as the adjacent output feature maps adjacent to the first output feature map.

[0056] Step S30: Calculate the first feature vector corresponding to the first output feature map and the adjacent feature vector corresponding to the adjacent output feature maps based on the traversed candidate boxes;

[0057] Step S40: Perform feature vector fusion on the first feature vector and the adjacent feature vectors to obtain the target feature vector;

[0058] Step S50: Perform image detection based on the target feature vector corresponding to each candidate box to obtain the image detection result.

[0059] In this embodiment, it should be noted that the preset model in this application is a two-stage model, mainly represented by Faster R-CNN, Mask R-CNN, and Cascade R-CNN. These models are characterized by generally high accuracy, but their computational process is slightly more complex, making them a choice for those seeking higher precision. The Faster R-CNN model is the most important prototype model among two-stage models, serving as the foundational prototype for all other currently mainstream deep learning-based two-stage models. In other words, other mainstream two-stage models are all improvements and developments based on the network structure of Faster R-CNN.

[0060] In commonly used two-stage models for object detection, an image passes through N convolutional layers (conv layers) to generate a large number of candidate bounding boxes (proposals). This is the first stage, called the Region Proposal Network (RPN). Then, in the second stage, the Region of Interest (ROI) pooling layer uses the candidate bounding boxes generated by the RPN to crop RoIs (Regions of Interest) from the feature maps output by the convolutional layers. These RoIs are then input into the classifier in the second stage for classification and bounding box regression. Depending on the backbone structure, the convolutional layers typically generate one or more (scale-wise) output feature maps. The ROI pooling layer then selects a RoI from one of the output feature maps generated by the convolutional layers according to explicit rules, cropping the corresponding region (patch) for use in the second stage of model training or inference.

[0061] The image to be processed is an image that requires object detection. The candidate box is a foreground box with high confidence selected from the candidate boxes generated by the one-stage RPN network. The first output feature map is an output feature map corresponding to the traversed candidate box. The adjacent output feature maps are the previous and next feature maps of the first output feature map in all output feature map sequences sorted by scale. There is at least one adjacent output feature map. The first feature vector is a vector calculated from the first output feature map corresponding to the traversed candidate box. The adjacent feature vector is a vector calculated from the adjacent feature vectors corresponding to the traversed candidate box. The target feature vector is a target feature vector obtained by fusing the first feature vector and adjacent feature vectors corresponding to the same candidate box. The image detection result is the final object detection result.

[0062] After the image is processed by the RPN network in the first stage, a large number of candidate boxes (proposals) are generated. Only foreground boxes are retained, while background boxes are discarded. After a series of further processing, only the N foreground boxes with the highest confidence scores remain. At the beginning of the second stage, the ROI Pooling layer divides the N foreground candidate boxes into positive and negative samples (positive and negative sample division only occurs during the model training phase and is skipped during the testing and inference phase).

[0063] For example, steps S10 to S40 include:

[0064] The process involves acquiring the image to be processed and obtaining the predicted bounding boxes generated by the first-stage RPN network. Background bounding boxes are discarded, and foreground bounding boxes are retained. The foreground bounding boxes with higher confidence scores from the remaining predicted bounding boxes are selected as the final candidate bounding boxes. All candidate bounding boxes are iterated through, and the first output feature map and adjacent output feature maps corresponding to each candidate bounding box are determined. Based on the first feature vectors corresponding to the traversed candidate bounding boxes and the first and adjacent feature vectors corresponding to the extreme first output feature maps, the first and adjacent feature vectors corresponding to the adjacent output feature maps are fused to obtain the target feature vectors corresponding to each candidate bounding box. The target feature vectors corresponding to each candidate bounding box are then input into the subsequent network of the preset model for image detection to obtain the image detection results.

[0065] In existing two-stage models, during the second stage of processing, only one of the four scale output feature maps generated by the backbone network combined with FPN is selected. The corresponding region patches (RoIs) are cropped from the proposals generated in the first stage and processed by the ROI Pooling layer to obtain a feature vector of a specific size. This vector is then input into the subsequent network for computation to obtain the final prediction result. However, the implementation increases the number of actual applications of the output feature maps and makes full use of multi-scale feature maps to obtain more accurate detection results.

[0066] In step S10, the step of determining all candidate boxes corresponding to the image to be processed includes:

[0067] Step A10: Input the image to be processed into the first stage of the preset two-stage model for model training to generate the prediction box of the image to be processed;

[0068] Step A20: If an object is detected in the prediction box, then the prediction box is used as the foreground box;

[0069] Step A30: Calculate the probability value of the object in each of the foreground boxes being the target object, and sort all the foreground boxes according to the probability value;

[0070] Step A40: Select the foreground boxes at the top of the sorted list as candidate boxes corresponding to the image to be processed.

[0071] In this embodiment, it should be noted that the predicted bounding box refers to all candidate bounding boxes generated after the RPN network processing in the first stage of object detection. The foreground bounding box is the predicted bounding box that confirms the presence of an object in the bounding box, while the predicted bounding box that does not contain an object in the bounding box is the background bounding box. The probability value is the probability that the detected object is the target object. For example, in face detection, the probability value is the probability that the detected object is a face. A custom probability value threshold can be defined. If the preset probability value threshold is set to 0.8, then candidate bounding boxes with a probability value lower than 0.8 will not appear.

[0072] For example, steps A10 to A40 include:

[0073] The image to be processed is input into a preset two-stage model for model training. Multiple prediction boxes for the image to be processed are generated through the RPN network. If an object is detected in the prediction box, the prediction box is determined to be a foreground box. If no object is detected in the prediction box, the prediction box is determined to be a background box. All background boxes are removed. The probability value of the object in each foreground box being the target object is calculated by the preset two-stage model. All foreground boxes are sorted according to the probability value, and the foreground boxes in the top of the sorted list are used as candidate boxes for the image to be processed.

[0074] It's important to note that in the first stage, after the image is processed by the RPN network to generate multiple predicted bounding boxes, only the foreground boxes are retained, while all background boxes are discarded. The n foreground boxes with the highest confidence scores are then kept. At the start of the second stage, if it's determined that the current process is model training, the remaining foreground boxes are divided into positive and negative samples. If it's determined that the current process is object detection, the positive and negative sample division process is skipped.

[0075] In step S30, the step of calculating the first feature vector corresponding to the first output feature map and the adjacent feature vectors corresponding to the adjacent output feature maps based on the traversed candidate boxes includes:

[0076] Step S31: Based on the traversed candidate boxes, crop out the corresponding first feature region in the first output feature map, and crop out the corresponding adjacent feature region in the adjacent output feature map;

[0077] Step S32: Input the first feature region and the adjacent feature regions into the second stage of the preset two-stage model for feature extraction, and obtain the first feature vector and the adjacent feature vector respectively.

[0078] In this embodiment, it should be noted that, based on the position of the candidate box, the first feature region and the adjacent feature region at the corresponding position can be obtained in the first output feature map and the adjacent output feature map. The first feature region and the adjacent feature region are part of the image region containing the target object. The preset two-stage model is a two-stage model that has been trained or is being trained. After inputting the first feature region and the adjacent feature region containing the target detection object into the preset two-stage model, the corresponding first feature vector and adjacent feature vector can be obtained.

[0079] For example, steps S31 to S32 include:

[0080] Based on the position of the candidate boxes traversed, the corresponding first feature region is cropped from the first output feature map, and the corresponding adjacent feature regions are also cropped from the adjacent output feature maps. The first feature region and the adjacent feature regions are input into the second stage of the preset two-stage model for feature extraction, and the first feature vector and the adjacent feature vector are obtained respectively.

[0081] This application provides an image detection method that acquires an image to be processed and determines all candidate boxes corresponding to the image; it traverses all candidate boxes to determine a first output feature map corresponding to the traversed candidate box and adjacent output feature maps adjacent to the first output feature map. This method overcomes the limitation of the original two-stage deep learning model, which most often extracts features from a single-scale output feature map, and fully utilizes the rich positional and semantic information contained in feature maps of different scales; it calculates a first feature vector corresponding to the first output feature map and adjacent feature vectors corresponding to the adjacent output feature maps based on the traversed candidate boxes; it performs feature vector fusion on the first feature vector and the adjacent feature vectors to obtain a target feature vector; and it performs image detection based on the target feature vector corresponding to each candidate box to obtain an image detection result. By fully utilizing the positional and semantic information contained in feature maps of multiple scales, this method can more accurately predict the category and position of target samples in the image, thereby further improving the target detection effect.

[0082] Example 2

[0083] Furthermore, referring to Figure 2 Based on the first embodiment of this application, in another embodiment of this application, the same or similar content as in the first embodiment can be referred to the above description and will not be repeated hereafter. Based on this, in step S20, the step of traversing all the candidate boxes and determining the first output feature map corresponding to the traversed candidate box, and the adjacent output feature maps adjacent to the first output feature map, includes:

[0084] Step S21: Obtain all scale feature maps corresponding to the image to be processed;

[0085] Step S22: Traverse all the candidate boxes, determine the candidate box size of the traversed candidate boxes, input the candidate box size and each scale feature map into the second stage of the preset two-stage model for output feature map selection, and output the first output feature map corresponding to the traversed candidate boxes.

[0086] Step S23: Select the adjacent output feature maps that are adjacent to the first output feature map from each of the scale feature maps.

[0087] In this embodiment, it should be noted that the feature map is a feature map of multiple scales output by the preset model based on the image to be processed, and the candidate box size is the candidate box length and candidate box width. Based on the candidate box size, a corresponding first output feature map can be obtained through a two-stage model.

[0088] For example, steps S21 to S23 include:

[0089] All feature maps of the image to be processed are extracted by a preset two-stage model. The size of the candidate boxes to be traversed is determined. The size of the candidate boxes and each feature map are input into the second stage of the preset two-stage model for output feature map selection. The first output feature map corresponding to the traversed candidate boxes is determined by preset calculation. The adjacent output feature maps corresponding to the first output feature map are selected from each candidate box.

[0090] Specifically, such as Figure 3 As shown, the output feature maps are designated as (C2, C3, C4, C5) in order of their scales based on the four output feature maps. The preset calculation formula for the feature map selection parameters is as follows: Where k is the feature map selection parameter, w is the candidate box width, and h is the candidate box height, typically... =4, and the output feature map corresponding to the feature map selection parameter calculated according to the above formula is used as the first output feature map. For example, if k=3, the selected output feature map is C3.

[0091] In step S23, the step of filtering out adjacent output feature maps that are adjacent to the first output feature map from each of the scale feature maps includes:

[0092] Step B10: Determine the position of the first output feature map in each of the scale feature maps;

[0093] Step B20: The feature maps in each scale feature map that are adjacent to the first output feature map are taken as adjacent output feature maps.

[0094] In this embodiment, it should be noted that, typically, the feature maps are four multi-scale output feature maps generated by the backbone network of the two-stage model combined with the FPN network. The four feature maps are as follows: Figure 2 As shown, the positions of the feature maps can be determined sequentially as (C2, C3, C4, C5) according to their size from largest to smallest.

[0095] For example, steps B10 to B20 include:

[0096] Determine the position of the first output feature map in each feature map, specifically the order of the first output feature map in the feature map sequence, and take all feature maps whose positions are adjacent to the first output feature map as adjacent output feature maps.

[0097] Sort all output feature maps corresponding to the candidate boxes and their scales from largest to smallest to obtain an output feature map sequence. In the output feature map sequence, the output feature map that precedes the first output feature map and the output feature map that follows the first output feature map are taken as adjacent output feature maps.

[0098] In step B20, the step of taking the feature maps whose positions are adjacent to the first output feature map in each of the scale feature maps as adjacent output feature maps includes:

[0099] Step B21: If the position of the first output feature map is the first position preceding each of the scale feature maps, then the second position preceding each of the scale feature maps is taken as the adjacent output feature map.

[0100] Step B22: If the position of the first output feature map is the first position after each of the scale feature maps, then the second position after each of the scale feature maps is taken as the adjacent output feature map.

[0101] Step B23: If the position of the first output feature map is located in the middle two positions of each of the scale feature maps, then the position before and the position after the first output feature map in each of the scale feature maps are taken as adjacent output feature maps.

[0102] In this embodiment, specifically, the positions of the feature maps can be determined sequentially from largest to smallest as (C2, C3, C4, C5). If the first output feature map is C2, then the adjacent output feature map is C3; if the first output feature map is C3, then the adjacent output feature maps are C2 and C4; if the first output feature map is C4, then the adjacent output feature maps are C3 and C5; if the first output feature map is C5, then the adjacent output feature map is C4. In summary, when the selected first output feature map is C3 or C4, a total of 3 feature maps need to be selected; otherwise, a total of 2 feature maps are selected.

[0103] This application provides a feature map selection method, which involves obtaining all scale feature maps corresponding to the image to be processed; determining the candidate box size of the traversed candidate boxes; inputting the candidate box size and each scale feature map into the second stage of a preset two-stage model for output feature map selection; and outputting the first output feature map corresponding to the traversed candidate boxes; and selecting adjacent output feature maps that are adjacent to the first output feature map from each scale feature map. This fully utilizes the positional and semantic information contained in the multi-scale feature maps, enabling the model to more accurately predict the category and position of target samples in the image, thereby further improving the performance of the target detection model.

[0104] Example 3

[0105] Furthermore, referring to Figure 4 Based on the first embodiment of this application, in another embodiment of this application, the same or similar content as in the first embodiment can be referred to the above description, and will not be repeated hereafter. Based on this, in step S40, the step of fusing the first feature vector and the adjacent feature vectors to obtain the target feature vector includes:

[0106] Step S41: Obtain the first vector weight of the first feature vector and the adjacent vector weights of the adjacent feature vectors;

[0107] Step S42: Based on the first vector weight and the neighboring vector weight, the first feature vector and the neighboring feature vectors are fused to obtain the target feature vector.

[0108] In this embodiment, it should be noted that the weights of the first vector and the adjacent vectors differ depending on the position of the first output feature map, representing the fusion coefficients of the first feature vector and the adjacent feature vectors in calculating the target feature vector.

[0109] For example, steps S41 to S42 include:

[0110] The first vector weight of the first feature vector and the adjacent vector weights of the adjacent feature vectors are obtained respectively. Based on the first vector weight of the first feature vector and the adjacent vector weights of the adjacent feature vectors, the first vector weight and the adjacent vector weights are fused to obtain the target feature vector.

[0111] Specifically, feature vectors are calculated using (F2, F3, F4, F5) corresponding to the output feature maps (C2, C3, C4, C5), respectively. When the first output feature map is C2, the adjacent output feature map is C3. Feature fusion is performed on C2 and C3, and the expression for calculating the feature vector fusion of the candidate box is: Where k corresponds to the number of the first output feature map C2, and the first feature fusion coefficient is... =0.75, adjacent feature fusion coefficient =0.25, C2 corresponds to the first eigenvector. For C3, the adjacent feature vectors are C3 and C4; when the first output feature map is C3 or C4, the feature vector fusion calculation expression for the candidate box is: Where k corresponds to the number of the first output feature map C3 or C4, and the adjacent feature fusion coefficients = =0.25, first feature fusion coefficient =0.5, Let C3 or C4 be the first eigenvector. Let C3 or C4 be the next adjacent feature vector; when the first output feature map is C5, the adjacent output feature map is C4. Then, C5 and C4 are fused. The expression for calculating the feature vector fusion of the candidate box is: Where k corresponds to the number of the first output feature map C5, and the first feature fusion coefficient is... =0.75, adjacent feature fusion coefficient =0.25, C5 corresponds to the first eigenvector. The adjacent feature vectors of C4.

[0112] This application provides a feature vector fusion method, which obtains the first vector weight of the first feature vector and the adjacent vector weight of the adjacent feature vector; based on the first vector weight and the adjacent vector weight, the first feature vector and the adjacent feature vector are fused to obtain the target feature vector. By multi-scale vector fusion, more vector information can be obtained, thereby improving the target detection effect.

[0113] Example 4

[0114] This application also provides an image detection device, such as... Figure 5 As shown, the image detection device is applied to an image detection equipment, and the image detection device includes:

[0115] The candidate box module C10 is used to acquire the image to be processed and determine all candidate boxes corresponding to the image to be processed.

[0116] The feature map module C20 is used to traverse all the candidate boxes, determine the first output feature map corresponding to the traversed candidate box, and the adjacent output feature map adjacent to the first output feature map.

[0117] Vector module C30 is used to calculate the first feature vector corresponding to the first output feature map and the adjacent feature vector corresponding to the adjacent output feature map based on the traversed candidate boxes;

[0118] The fusion module C40 is used to fuse the first feature vector and the adjacent feature vectors to obtain the target feature vector.

[0119] The detection module C50 is used to perform image detection based on the target feature vector corresponding to each candidate box to obtain the image detection result.

[0120] Optionally, the feature map module C20 is further configured to:

[0121] Obtain all scale feature maps corresponding to the image to be processed;

[0122] Traverse all the candidate boxes, determine the candidate box size of the traversed candidate boxes, input the candidate box size and each scale feature map into the second stage of the preset two-stage model for output feature map selection, and output the first output feature map corresponding to the traversed candidate boxes.

[0123] In each of the scale feature maps, neighboring output feature maps that are adjacent to the first output feature map are selected.

[0124] Optionally, the feature map module C20 is further configured to:

[0125] Obtain all scale feature maps corresponding to the image to be processed;

[0126] Determine the candidate box size of the candidate boxes to be traversed, input the candidate box size and each scale feature map into the second stage of the preset two-stage model to select the output feature map, and output the first output feature map corresponding to the traversed candidate boxes.

[0127] In each of the scale feature maps, neighboring output feature maps that are adjacent to the first output feature map are selected.

[0128] Optionally, the feature map module C20 is further configured to:

[0129] Determine the position of the first output feature map in each of the scale feature maps;

[0130] The feature maps in each scale feature map that are adjacent to the first output feature map are taken as adjacent output feature maps.

[0131] Optionally, the feature map module C20 is further configured to:

[0132] If the position of the first output feature map is the first position preceding each of the scale feature maps, then the second position preceding each of the scale feature maps is taken as the adjacent output feature map.

[0133] If the position of the first output feature map is the first position after each of the scale feature maps, then the second position after each of the scale feature maps is taken as the adjacent output feature map.

[0134] If the position of the first output feature map is located in the middle two positions of each of the scale feature maps, then the position before and the position after the first output feature map in each of the scale feature maps are taken as adjacent output feature maps.

[0135] Optionally, the vector module C30 is further configured to:

[0136] Based on the traversed candidate boxes, the corresponding first feature region is cropped in the first output feature map, and the corresponding adjacent feature region is cropped in the adjacent output feature map;

[0137] The first feature region and the adjacent feature regions are input into the second stage of the preset two-stage model for feature extraction, and the first feature vector and the adjacent feature vector are obtained respectively.

[0138] Optionally, the fusion module C40 is further configured to:

[0139] Obtain the first vector weight of the first feature vector and the adjacent vector weights of the adjacent feature vectors;

[0140] Based on the first vector weight and the neighboring vector weight, the first feature vector and the neighboring feature vectors are fused to obtain the target feature vector.

[0141] Optionally, the candidate box module C10 is further configured to:

[0142] The image to be processed is input into the first stage of a preset two-stage model for model training to generate a prediction box for the image to be processed.

[0143] If an object is detected in the prediction box, the prediction box is used as the foreground box;

[0144] Calculate the probability value of the object in each of the foreground boxes being the target object, and sort all the foreground boxes according to the probability value;

[0145] The foreground boxes at the top of the sorted list are used as candidate boxes for the image to be processed.

[0146] The image detection device provided by this invention employs the image detection methods described in Embodiment 1, Embodiment 2, or Embodiment 3 above, solving the technical problem of poor image detection performance. Compared with the prior art, the beneficial effects of the image detection device provided by the embodiments of this invention are the same as those of the image detection methods provided in the above embodiments, and other technical features in this image detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0147] Example 5

[0148] This invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the image detection method described in Embodiment 1 above.

[0149] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0150] like Figure 6 As shown, electronic device D90 may include processing unit D10 (e.g., central processing unit, graphics processor, etc.), which can perform various appropriate actions and processes according to programs stored in read-only memory D20 (ROM) or programs loaded from storage devices into random access memory D30 (RAM). RAMD30 also stores various programs and data required for the operation of the electronic device. Processing unit D10, ROMD20, and RAMD30 are interconnected via a bus. Input / output (I / O) interface D40 is also connected to the bus.

[0151] Typically, the following systems can be connected to the I / O interface: input devices D50, including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices D60, including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices D70, including, for example, magnetic tapes, hard disks, etc.; and communication devices D80. Communication device D80 allows electronic device D90 to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0152] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device D70, or installed from a ROM D20. When the computer program is executed by a processing device, it performs the functions defined in the methods of embodiments of this disclosure.

[0153] The electronic device provided by this invention employs the image detection method described in Embodiment 1, Embodiment 2, or Embodiment 3 above, thus solving the technical problem of poor image detection performance. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiments of this invention are the same as those of the image detection method provided in Embodiment 1 above, and other technical features of this electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0154] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0155] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0156] Example 6

[0157] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the image detection method in the first embodiment described above.

[0158] The computer-readable storage medium provided in this embodiment of the invention may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0159] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0160] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire an image to be processed and determine all candidate boxes corresponding to the image to be processed; traverse all the candidate boxes and determine a first output feature map corresponding to the traversed candidate box, and adjacent output feature maps adjacent to the first output feature map; calculate a first feature vector corresponding to the first output feature map and adjacent feature vectors corresponding to the adjacent output feature maps based on the traversed candidate boxes; perform feature vector fusion on the first feature vector and the adjacent feature vectors to obtain a target feature vector; and perform image detection based on the target feature vector corresponding to each candidate box to obtain an image detection result.

[0161] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0163] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0164] The computer-readable storage medium provided by this invention stores computer-readable program instructions for executing the above-described image detection method, thus solving the technical problem of poor image detection results. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiments of this invention are the same as the beneficial effects of the image detection methods provided in Embodiments 1, 2, or 3 above, and will not be repeated here.

[0165] Example 7

[0166] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image detection method described above.

[0167] The computer program product provided in this application solves the technical problem of poor image detection performance. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this invention are the same as the beneficial effects of the image detection methods provided in Embodiment 1, Embodiment 2 or Embodiment 3 above, and will not be repeated here.

[0168] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. An image detection method, characterized in that, The image detection method includes: Obtain the image to be processed and determine all candidate boxes corresponding to the image to be processed; Traverse all the candidate boxes and determine the first output feature map corresponding to the traversed candidate box, as well as the adjacent output feature maps adjacent to the first output feature map. Calculate the first feature vector corresponding to the first output feature map and the adjacent feature vectors corresponding to the adjacent output feature maps based on the traversed candidate boxes; The first feature vector and the adjacent feature vectors are fused to obtain the target feature vector. Image detection is performed based on the target feature vector corresponding to each candidate box to obtain the image detection result; The step of traversing all the candidate boxes and determining the first output feature map corresponding to the traversed candidate box, and the adjacent output feature maps adjacent to the first output feature map, includes: Obtain all scale feature maps corresponding to the image to be processed; Traverse all the candidate boxes, determine the candidate box size of the traversed candidate boxes, input the candidate box size and each scale feature map into the second stage of the preset two-stage model for output feature map selection, and output the first output feature map corresponding to the traversed candidate boxes. Determine the position of the first output feature map in each of the scale feature maps; If the position of the first output feature map is the first position preceding each of the scale feature maps, then the second position preceding each of the scale feature maps is taken as the adjacent output feature map. If the position of the first output feature map is the first position after each of the scale feature maps, then the second position after each of the scale feature maps is taken as the adjacent output feature map. If the position of the first output feature map is located in the middle two positions of each of the scale feature maps, then the position before and the position after the first output feature map in each of the scale feature maps are taken as adjacent output feature maps.

2. The image detection method as described in claim 1, characterized in that, The step of calculating the first feature vector corresponding to the first output feature map and the neighboring feature vectors corresponding to the neighboring output feature maps based on the traversed candidate boxes includes: Based on the traversed candidate boxes, the corresponding first feature region is cropped in the first output feature map, and the corresponding adjacent feature region is cropped in the adjacent output feature map; The first feature region and the adjacent feature regions are input into the second stage of the preset two-stage model for feature extraction, and the first feature vector and the adjacent feature vector are obtained respectively.

3. The image detection method as described in claim 1, characterized in that, The step of fusing the first feature vector and the adjacent feature vectors to obtain the target feature vector includes: Obtain the first vector weight of the first feature vector and the adjacent vector weights of the adjacent feature vectors; Based on the first vector weight and the neighboring vector weight, the first feature vector and the neighboring feature vectors are fused to obtain the target feature vector.

4. The image detection method as described in claim 1, characterized in that, The step of determining all candidate boxes corresponding to the image to be processed includes: The image to be processed is input into the first stage of a preset two-stage model for model training to generate a prediction box for the image to be processed. If an object is detected in the prediction box, the prediction box is used as the foreground box; Calculate the probability value of the object in each of the foreground boxes being the target object, and sort all the foreground boxes according to the probability value; The foreground boxes at the top of the sorted list are used as candidate boxes for the image to be processed.

5. An image detection device, characterized in that, The image detection device includes: The candidate box module is used to acquire the image to be processed and determine all candidate boxes corresponding to the image to be processed. The feature map module is used to traverse all the candidate boxes, determine the first output feature map corresponding to the traversed candidate box, and the adjacent output feature maps adjacent to the first output feature map. The feature map module is also used to obtain all scale feature maps corresponding to the image to be processed; traverse all the candidate boxes, determine the candidate box size of the traversed candidate boxes, input the candidate box size and each scale feature map into the second stage of a preset two-stage model for output feature map selection, and output the first output feature map corresponding to the traversed candidate boxes; determine the position of the first output feature map in each scale feature map; if the position of the first output feature map is the first position before each scale feature map, then the second position before each scale feature map is taken as the adjacent output feature map; if the position of the first output feature map is the first position after each scale feature map, then the second position after each scale feature map is taken as the adjacent output feature map; if the position of the first output feature map is the middle two positions of each scale feature map, then the position before and the position after the first output feature map in each scale feature map are taken as the adjacent output feature maps. The vector module is used to calculate the first feature vector corresponding to the first output feature map and the adjacent feature vector corresponding to the adjacent output feature maps based on the traversed candidate boxes; The fusion module is used to fuse the first feature vector and the adjacent feature vectors to obtain the target feature vector; The detection module is used to perform image detection based on the target feature vector corresponding to each candidate box, and obtain the image detection result.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the image detection method according to any one of claims 1 to 4.

7. A readable storage medium, characterized in that, The readable storage medium stores a program for implementing an image detection method, which is executed by a processor to implement the steps of the image detection method as described in any one of claims 1 to 4.

Images