Target detection method, device, electronic equipment and medium

Abstract

In the target detection method, device, electronic device and medium provided by the present application, multiple fused feature maps including the topmost feature map and the fusion result of the topmost feature map and the feature map set are obtained based on the topmost feature map and the feature map set of at least one level except the topmost feature map, each fused feature map is input into the region proposal network to obtain multiple target detection frames, and then the targets under the multiple target detection frames are identified according to the target recognition network to obtain the target recognition result. This solution can improve the accuracy of target detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target detection method, device, electronic device and medium. Background Art

[0002] Object detection is an important research direction of image processing technology and is widely used in robot navigation, intelligent monitoring, and other fields.

[0003] The deep learning model is the mainstream algorithm of target detection technology. In related technologies, the image to be identified is input into the convolutional neural network, multi-level feature extraction is performed on the image to be identified, and the target detection recognition frame is obtained based on the top-level feature map, and then the target under the target detection recognition frame is identified.

[0004] However, the top-level feature map will lose the pixel features of small targets during the feature extraction process, so that small targets in the image cannot be detected, thereby reducing the accuracy of target detection. Summary of the invention

[0005] The present application provides a target detection method, device, electronic device and medium to solve the technical problem that small targets in an image cannot be detected, thereby reducing the accuracy of target detection.

[0006] In a first aspect, the present application provides a target detection method, comprising: performing feature extraction on an input image to obtain feature maps at each level; wherein the level of the feature map is positively correlated with the semantic information of the feature map; obtaining multiple fused feature maps based on the top-level feature map and the feature map set; the multiple fused feature maps include the top-level feature map and the fusion result of the top-level feature map and the feature map set; each feature map set includes feature maps of at least one level other than the top-level; inputting each fused feature map and the input image into a region proposal network, so that the region proposal network performs target detection based on the fused feature map and the input image, and obtains a target detection box corresponding to each fused feature map; mapping the target detection box to the fused feature map corresponding to the target detection box, and obtaining a region feature map corresponding to each fused feature map; the region feature map includes the region where the target detection box is mapped in the fused feature map; inputting the input image and the region feature map corresponding to each fused feature map into a target recognition network, and obtaining multiple target recognition results.

[0007] Optionally, there are multiple feature map sets, and the feature maps in each feature map set are not completely the same.

[0008] Optionally, the method obtains multiple fused feature maps based on the feature map of the top layer and the feature map set, including: for the top layer, obtaining a fused feature map based on the feature map of the top layer; for predetermined levels other than the top layer, fusing the feature map of the top layer with the feature map set corresponding to the predetermined level, and using the fusion result as the fused feature map, until all the predetermined levels are fused; wherein the feature map set corresponding to each predetermined level includes feature maps of all levels except the top layer and not lower than the predetermined level.

[0009] Optionally, the predetermined level is one of the multiple consecutive adjacent levels of the next level of the topmost layer; for the predetermined levels other than the topmost layer, the feature map of the topmost layer is fused with the feature map set corresponding to the predetermined level, and the fusion result is used as the fused feature map until all the predetermined levels are fused, including: performing convolution processing on the feature maps corresponding to the topmost layer and the predetermined level so that the number of channels of the feature maps corresponding to the topmost layer and the predetermined level after processing are consistent; using the feature map of the topmost layer as the fused feature map of the topmost layer, and performing the following processing on each predetermined level from top to bottom: upsampling the fused feature map of the previous level of the predetermined level; the size of the feature map obtained by the upsampling is consistent with that of the feature map at the predetermined level; fusing the feature map at the predetermined level with the feature map obtained by the upsampling to obtain a fused feature map corresponding to the predetermined level.

[0010] Optionally, the convolution processing of the feature maps corresponding to the top layer and the predetermined layers includes: respectively convolving the feature maps corresponding to the top layer and each of the predetermined layers with a first number of 1x1 convolution kernels; the first number is the number of channels of the feature map after the convolution processing.

[0011] Optionally, the feature extraction of the input image to obtain the feature maps at each level includes: inputting the input image into a residual network to obtain the feature maps at each level output by the residual network; the residual network includes residual blocks of multiple consecutive stages, the output of the residual block of the current stage is the input of the residual block of the next stage, and the feature maps output by the residual blocks of each stage are used as the feature maps at each level; the residual network is trained based on the difference between the output and input of each residual block approaching zero.

[0012] Optionally, the method of enabling the region proposal network to perform target detection according to the fused feature map and the input image to obtain the target detection frames corresponding to each of the multiple fused feature maps includes: for the input fused feature map, setting multiple candidate frames with predetermined scales and different proportions at positions corresponding to each pixel point of the fused feature map in the input image; wherein different fused feature maps correspond to different scales, and the size of the fused feature map is negatively correlated with the scale; determining the region types corresponding to the multiple candidate frames, and deleting the candidate frames whose region types are background types to obtain target candidate frames; wherein the region types include foreground types and background types; and performing correction filtering on the target candidate frames to obtain the target detection frames corresponding to the fused feature map.

[0013] Optionally, the target detection frame corresponding to each fused feature map is mapped to the fused feature map to obtain regional feature maps corresponding to the multiple fused feature maps, including: inputting the target detection frame corresponding to each fused feature map and the fused feature map into the ROI Align network to obtain the regional feature map corresponding to the fused feature map output by the ROI Align network; wherein the ROI Align network is used to: adjust the size of the target detection frame corresponding to the fused feature map according to the ratio of the size of the fused feature map to the size of the input image; map the adjusted target detection frame to the fused feature map, and extract the regional feature map corresponding to the target detection frame; based on the bilinear interpolation method, adjust the regional feature map to a predetermined size.

[0014] In a second aspect, the present application provides a target detection device, comprising: a feature fusion module, used to obtain multiple fused feature maps based on the top-level feature map and the feature map set; the multiple fused feature maps include the top-level feature map and the fusion result of the top-level feature map and the feature map set; each feature map set includes at least one level of feature maps except the top-level; a target detection frame generation module, used to input each fused feature map and the input image into a region proposal network, so that the region proposal network performs target detection based on the fused feature map and the input image, and obtains the target detection frame corresponding to each fused feature map; a target detection module, used to map the target detection frame to the fused feature map corresponding to the target detection frame, and obtain the region feature map corresponding to each fused feature map; the region feature map includes the region where the target detection frame is mapped in the fused feature map; a target recognition module, used to input the input image and the region feature map corresponding to each fused feature map into a target recognition network, and obtain multiple target recognition results.

[0015] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described above.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the method as described above when executed by a processor.

[0017] In the target detection method, device, electronic device and medium provided by the present application, multiple fused feature maps including the top-level feature map and the fusion result of the top-level feature map and the feature map set are obtained based on the feature map of the top-level and the feature map set including at least one level of feature maps except the top-level, each fused feature map is input into the region proposal network to obtain multiple target detection frames, and then the targets under the multiple target detection frames are identified according to the target recognition network to obtain the target recognition result. The fused feature map in this scheme combines the semantic information of the top-level feature map and the position information of the bottom-level feature map, so when the target recognition result is obtained based on the fused feature map, the detection of small targets can be achieved, thereby improving the accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the embodiments of the present application, and together with the description are used to explain the principles of the embodiments of the present application.

[0019] The above drawings have shown clear embodiments of the embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the embodiments of the present application in any way, but to illustrate the concepts of the embodiments of the present application for those skilled in the art by referring to specific embodiments.

[0020] Figure 1 A schematic diagram of a process flow of a target detection method provided in Example 1 of the present application;

[0021] Figure 2 A flowchart of another target detection method provided in Example 1 of the present application;

[0022] Figure 3 Schematic diagram of the process of generating a fusion feature map in an example;

[0023] Figure 4 is a flowchart of the fusion feature map generation process in another example;

[0024] Figure 5 A schematic diagram of the process of setting a candidate box in an example;

[0025] Figure 6 A flowchart of another target detection method provided in Example 1 of the present application;

[0026] Figure 7 A schematic diagram of the structure of the target detection device provided in Example 2 of the present application;

[0027] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Example 3 of the present application.

[0028] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] Object detection is an important research direction of image processing technology and is widely used in robot navigation, intelligent monitoring, and other fields.

[0031] The deep learning model is the mainstream algorithm of target detection technology. In related technologies, the image to be identified is input into the convolutional neural network, multi-level feature extraction is performed on the image to be identified, and the target detection recognition frame is obtained based on the top-level feature map, and then the target under the target detection recognition frame is identified.

[0032] However, in the process of extracting features through convolution operations layer by layer, the semantic information of the feature map is gradually enhanced, and the local position information is gradually weakened, that is to say, the pixel features of small targets will be gradually lost. In this way, in some scenarios such as target detection in aerial images, the shooting distance of aerial images is high, which makes the targets in the image smaller. During detection, it is necessary to select the area of ​​small targets for subsequent recognition of the small target category. However, in the related art, the position information of small targets is lost in the feature map of the highest layer, so it is impossible to detect small targets in the image, thereby reducing the accuracy of target detection.

[0033] The technical solutions of the present application and the technical solutions of the present application are described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In the description of the present application, unless otherwise clearly specified and limited, each term should be understood in a broad sense in the art. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0034] Embodiment 1

[0035] Figure 1 A schematic diagram of a target detection process provided in Example 1 of the present application, such as Figure 1 As shown, the target detection method provided in this embodiment includes:

[0036] S101: extracting features from an input image to obtain feature maps at each level; wherein the level of the feature map is positively correlated with the semantic information of the feature map;

[0037] S102: obtaining a plurality of fused feature maps according to the top-most feature map and the feature map set; the plurality of fused feature maps include the top-most feature map and a fusion result of the top-most feature map and the feature map set; each feature map set includes a feature map of at least one level except the top-most level;

[0038] S103: Input each fused feature map and the input image into a region proposal network, so that the region proposal network performs target detection according to the fused feature map and the input image, and obtains a target detection frame corresponding to each fused feature map;

[0039] S104: Mapping the target detection frame to a fused feature map corresponding to the target detection frame to obtain a regional feature map corresponding to each fused feature map; the regional feature map includes a region where the target detection frame is mapped in the fused feature map;

[0040] S105: Input the input image and the regional feature map corresponding to each fusion feature map into a target recognition network to obtain multiple target recognition results.

[0041] In this embodiment, the input image is an image to be detected. In practical applications, the input image can be input in a predetermined size to facilitate subsequent processing. Feature extraction is performed on the input image to obtain feature maps at each level. The level of the feature map is positively correlated with the semantic information of the feature map. The semantic information is information provided by meaningful features. It can be understood that the higher the level of the feature map, the stronger the semantic information, but the lower the resolution.

[0042] In one example, S101 includes: inputting the input image into a residual network to obtain the feature maps at each level output by the residual network; the residual network includes residual blocks of multiple consecutive stages, the output of the residual block of the current stage is the input of the residual block of the next stage, and the feature maps output by the residual blocks of each stage are used as the feature maps at each level. The residual network is trained based on the difference between the output and input of each residual block approaching zero.

[0043] In this example, the residual network includes multiple residual blocks, and the feature maps output by each residual block are used as feature maps at each level. As an example, the residual network can be Resnet-50, which has 5 stages of residual blocks, and each stage of residual blocks includes 10 layers of network.

[0044] The residual network is trained based on the difference between the output and input of the residual block approaching zero. The difference between the output and input of the residual block is the residual function. Exemplarily, the input of the residual block is x, the output is F(x), the residual function is H(x)=F(x)-x, and the residual network is trained based on H(x) approaching 0. It can be seen from practical applications that optimizing the residual function is much easier than optimizing the objective function F(x). For details, please refer to the relevant technology. Therefore, the residual network is easier to train than the commonly used VGG network based on objective function training. This also ensures the identity mapping between the input and output of the residual block. Therefore, using the residual network to obtain the feature map can avoid the problem of deep network degradation, and thus can improve the accuracy of the target network model in detecting the input image.

[0045] After obtaining the feature maps of each level, S102 is executed to obtain multiple fused feature maps according to the feature map of the top layer and the feature map set. The multiple fused feature maps include the feature map of the top layer and the fusion result of the feature map of the top layer and the feature map set; the multiple fused feature maps have different sizes, and each feature map set includes at least one feature map of the level other than the top layer.

[0046] It is worth noting that, for the feature map fusion method, the feature values ​​of corresponding bits in the feature map can be added to obtain the corresponding fused feature map.

[0047] Based on the above scheme, in one example, the number of the feature map sets is multiple, and the feature maps in each feature map set are not completely the same. That is to say, the fused feature map includes multiple fusion results, and the number of fused feature maps exceeds 2. Therefore, through this example, multiple fused feature maps can be obtained, and the input image can be detected from multiple levels, thereby improving the accuracy of target detection.

[0048] S102 will be described exemplarily below. Figure 2A flowchart of another target detection method provided in Example 1 of the present application is shown as follows: Figure 2 As shown, the S102 includes:

[0049] S201: For the top layer, a fused feature map is obtained based on the feature map of the top layer;

[0050] S202: For predetermined levels other than the topmost level, fuse the feature map of the topmost level with a set of feature maps corresponding to the predetermined levels, and use the fusion result as a fused feature map until all the predetermined levels are fused.

[0051] In practical applications, all levels corresponding to the output feature maps can be set as preset levels, or some levels corresponding to the output feature maps can be set as preset levels. The feature map set corresponding to each preset level includes feature maps at all levels except the top level, which are not lower than the preset level. For example, the top-level feature map C5, the preset level is C3, then the feature map set corresponding to C3 includes C4 and C3.

[0052] In practical applications, the predetermined levels may be continuous and adjacent levels. For example, the feature graph includes 5 levels {C1, C2, C3, C4, C5}, C5 is the top-level feature graph, and the predetermined levels may include {C2, C3, C4}. Of course, the predetermined levels may also be discontinuous levels. In combination with the above example, the predetermined levels may include {C1, C2, C4}.

[0053] The following will take the scenario where the predetermined level is one of a plurality of consecutive adjacent levels of the next level of the top level as an example to illustrate S202. In one example, S202 includes:

[0054] Performing convolution processing on the feature maps corresponding to the top layer and the predetermined layer, so that the number of channels of the feature maps corresponding to the top layer and the predetermined layer after processing is consistent;

[0055] The top-most feature map is used as the top-most fused feature map, and the following processing is performed for each predetermined level from top to bottom:

[0056] Upsampling the fused feature map of the previous level of the predetermined level;

[0057] The feature map at this level is fused with the feature map obtained by upsampling to obtain a fused feature map at this level.

[0058] In practical applications, the size of the output feature map is usually halved layer by layer as the level increases. To ensure that the information gap between feature maps at adjacent levels is not too large, the number of channels of the output feature map is doubled layer by layer. Therefore, in this example, the feature maps corresponding to the topmost layer and the predetermined layer are processed so that the number of channels of the feature maps corresponding to the topmost layer and the predetermined layer after processing are consistent, so that a fused feature map can be generated based on the feature maps corresponding to the topmost layer and the predetermined layer.

[0059] For the processing process, as an example, the feature maps corresponding to the top layer and each of the predetermined layers are convolved with a first number of 1x1 convolution kernels, where the first number is the number of channels of the feature map after the convolution process.

[0060] In this example, the top-most feature map is convolved with the first number of 1x1 convolution kernels, and the features at each level are convolved with the first number of 1x1 convolution kernels. It can be understood that the 1x1 convolution kernel will not affect the feature values ​​of the feature maps at each level, and the number of channels of the feature maps corresponding to the top-most layer and the predetermined level after the operation is the first number. As an example, the first number can be 256.

[0061] In this example, the feature maps corresponding to the top layer and the predetermined layer are processed with a unified number of channels through the first number of 1x1 convolution kernels, which can avoid affecting the feature values ​​in the feature maps at each layer, thereby further ensuring the accuracy of the target recognition results.

[0062] In S202, after unifying the number of channels, the top-most feature map is used as the top-most fused feature map. The fused feature map of the previous layer of each predetermined layer is upsampled from top to bottom, and the size of the upsampled fused feature map is consistent with that of the feature map at the predetermined layer. The feature map at this layer is then fused with the upsampled feature map to obtain a fused feature map at this layer.

[0063] It is worth noting that there is no limitation on the downsampling method in this example, and it may be, for example, an anti-pooling method or a nearest neighbor difference method.

[0064] The following is an example of this solution combined with actual scenarios: Figure 3 This is a flowchart of the fusion feature map generation process in an example. Figure 3As shown in the figure, ResNet-50 is used to extract features from the input image of HxW size, and the output feature maps of 5 levels are obtained, namely {C1, C2, C3, C4, C5}, and the levels are gradually increased. When obtaining the feature map of each level, it is necessary to downsample on the basis of the feature map of the previous level, so the size of {C1, C2, C3, C4, C5} decreases layer by layer. For example, the size of C4 is 1 / 16 of the input image, and the size of C5 is 1 / 32 of the input image. The predetermined level is {C2, C3, C4}. C5 is convolved with 256 1x1 convolution kernels to obtain the fused feature map M5 corresponding to C5. C4 is convolved with 256 1x1 convolution kernels, and the operation result is fused with the upsampled M5 to obtain the fused feature map M4 with 256 channels corresponding to C4. By analogy, the fused feature maps M3 and M2 corresponding to C3 and C2 are obtained. In practical applications, the fused feature maps {M2, M3, M4, M5} are convolved with 256 3x3 convolution kernels respectively to obtain the final fused feature maps {P2, P3, P4, P5}. This is to reduce the aliasing effect caused by the above upsampling. In practical applications, P5 can also be upsampled to obtain the corresponding P6. The size of the feature map corresponding to P6 is smaller. P6 is to prevent the target from being too large. When the target detection frame is generated later, the candidate frame cannot frame the target. P6 corresponds to a larger candidate frame to frame the larger target.

[0065] In this example, the feature map at a predetermined level is fused with the fused feature map corresponding to the previous level, and the fusion result of the previous level is utilized, thereby reducing the amount of calculation in the target detection process.

[0066] In this example, the feature atlas includes all levels except the top level that are not lower than the predetermined level, which enriches the semantic information and local position information in the fused feature map after fusion, and also improves the transition between features in the fused feature map, thereby improving the accuracy of target recognition detection.

[0067] In another example, the S102 includes:

[0068] For the top layer, a fused feature map is obtained based on the feature map of the top layer;

[0069] For each second predetermined level except the top level, feature fusion is performed on the feature map of the top level with the feature maps under the second predetermined level to obtain fused feature maps of multiple levels.

[0070] In this example, for the topmost layer, a fused feature map is obtained based on the feature map of the topmost layer; for each second predetermined layer except the topmost layer, the topmost layer is fused with the feature map under the second predetermined layer. The feature map set corresponding to each second predetermined layer includes the feature map corresponding to the second predetermined layer.

[0071] In one example, the second predetermined level is a plurality of continuous adjacent levels including a level below the topmost level; for each second predetermined level except the topmost level, the feature map of the topmost level is respectively fused with the feature map under the second predetermined level to obtain fused feature maps of multiple levels, including:

[0072] Performing convolution processing on the feature maps corresponding to the topmost layer and the second predetermined layer, so that the number of channels of the feature maps corresponding to the topmost layer and the second predetermined layer after processing is consistent;

[0073] The top-most feature map is used as the top-most fused feature map, and the following processing is performed for each second predetermined level from top to bottom: the top-most fused feature map is upsampled; the feature map obtained by upsampling has the same size as the feature map at the second predetermined level;

[0074] The feature map at the second predetermined level is fused with the feature map obtained by upsampling to obtain a fused feature map corresponding to the second predetermined level.

[0075] In this example, the number of channels of the feature maps at all levels is also unified first. For specific methods, please refer to the above example. Then, the top-level fused feature map is upsampled, and the feature map obtained by upsampling is consistent with the size of the feature map at the second predetermined level. It can be understood that for different second predetermined levels, the top-level fused feature map is upsampled different times. The upsampled fused feature map is fused with the feature map at the predetermined level after the unified channels to obtain a fused feature map at the predetermined level.

[0076] The following is an explanation of this example in conjunction with specific scenarios: Figure 4 This is a flowchart of the fusion feature map generation process in another example. Figure 4As shown, the feature map includes 5 levels of feature maps, {C1, C2, C3, C4, C5}. The size of {C1, C2, C3, C4, C5} is halved layer by layer, and the second predetermined level is {C2, C3, C4}. C5 is convolved with 256 1x1 convolution kernels to obtain the fused feature map M5 corresponding to C5. C4 is convolved with 256 1x1 convolution kernels, and the operation result is fused with M5 to obtain the fused feature map M4 corresponding to C4. Similarly, C3 and C2 are convolved with 256 1x1 convolution kernels respectively, and the operation results are fused with M5 respectively to obtain the fused feature map M3 corresponding to C3 and the fused feature map M2 corresponding to C2.

[0077] In this example, the top-level fused feature map is directly fused with the feature map under the second predetermined level to obtain a fused feature map of the second predetermined level, which reduces the number of feature maps in the feature map set, thereby reducing the computational burden of the model and improving the speed of target detection.

[0078] After obtaining multiple fused feature maps through S102, S103 is executed: each fused feature map and the input image are input into the region proposal network to obtain target detection frames corresponding to multiple fused feature maps. The output target detection frame is generated based on the input image, that is, the output target detection frame is marked on the input image. The region proposal network is used for target detection.

[0079] The following is an illustrative description of the region proposal network.

[0080] In one example, S103 includes:

[0081] For the input fusion feature map, a plurality of candidate frames having a predetermined scale and different proportions are set at a position corresponding to each pixel point of the fusion feature map in the input image; wherein different fusion feature maps correspond to different scales, and the size of the fusion feature map is negatively correlated with the scale;

[0082] Determine the region types corresponding to the multiple candidate frames, and delete the candidate frames whose region types are background types, to obtain a target candidate frame; wherein the region types include foreground types and background types;

[0083] The target candidate frame is corrected and filtered to obtain the target detection frame corresponding to the fused feature map. In practical applications, the fused feature map is generated based on the feature maps at each level of the input image, so each pixel in the fused feature map corresponds to a pixel in the input image. In this embodiment, multiple candidate frames are set at positions corresponding to each pixel of the fused feature map in the input image, and the multiple candidate frames are all of the same predetermined scale and different proportions. The scale of the candidate frame is the area of ​​the candidate frame. In addition, different fused feature maps correspond to different scales, and the size of the fused feature map is negatively correlated with the scale.

[0084] That is to say, each input fusion feature map corresponds to a candidate box of a certain scale, and the ratio of the candidate box is varied, so that objects of different shapes can be selected more accurately. In addition, the larger the size of the fusion feature map, the smaller the scale of the corresponding candidate box. The larger the scale of the fusion feature map, the higher the resolution of the fusion feature map, the clearer the local position information, and the more features of the corresponding small objects. Therefore, in the fusion feature map corresponding to the lower level, a relatively small scale can be used to select small objects. It can be understood that candidate boxes of different scales are used to select objects of different sizes in the input image.

[0085] Through the above scheme, a large number of candidate boxes marked on the input image are obtained. The areas under these candidate boxes may be foreground or background, where the foreground is the target to be identified, while the background does not need to be identified. Therefore, in this example, the Softmax algorithm can be used to determine the area types corresponding to multiple candidate boxes, and the area types include foreground type and background type. By deleting the candidate boxes with the background type, the target candidate boxes with the foreground type are obtained.

[0086] Then, the target candidate frame is corrected according to the regression loss function. Among the corrected target candidate frames, there may be some target candidate frames with high overlap. It can be understood that these target candidate frames with high overlap correspond to the same target. Therefore, in this example, the corrected target candidate frames are filtered by the non-maximum suppression method to filter out the target candidate frames with high overlap, and obtain the target detection frame corresponding to the fusion feature map.

[0087] The following is an illustrative example of this solution combined with specific scenarios: Figure 3 Examples in Figure 3 As shown, the fused feature map includes {P2, P3, P4, P5, P6}, whose sizes decrease in sequence, and the corresponding candidate box scale is {16 2 ,32 2 ,64 2 ,128 2 ,256 2}, each scale corresponds to candidate boxes of three ratios {1:1, 1:2, 2:1}. Figure 5 This is a flowchart of setting a candidate box in an example. Figure 5 As shown, the fused feature map 52 is P2, P2 includes pixel point A1, A1 corresponds to point A0 in the input image 51, and the scales of three ratios {1:1, 1:2, 2:1} are set with point A0 as the center, and the scales are 16 2 Similarly, for each pixel in the input image 51 corresponding to a pixel in C2, the scales of the three ratios of 1:1, 1:2, and 2:1 are set to 16 2 Candidate box 53.

[0088] Then, the candidate boxes 53 in the input image are classified by the Softmax algorithm to obtain target candidate boxes with promising frames. The target candidate boxes are then corrected by the regression loss function. The corrected target candidate boxes are filtered based on the non-maximum suppression method to filter out target candidate boxes with large repetitions to obtain the final target detection box. The target detection box can be output in the form of parameters. For example, the parameters of the target detection box include (x, y, w, h). Among them, x and y are the center coordinates of the target detection box relative to the input image, and w and h are the width and height of the target detection box.

[0089] In this example, a target candidate box is directly generated on the input image based on the pixel points of the fused feature map. Compared with the method of generating a target detection box by sliding the candidate box in the prior art, the generation speed of the target detection box is improved.

[0090] S104 will be further described below.

[0091] After obtaining the target detection frames corresponding to multiple fused feature maps through S103, the target detection frames corresponding to the multiple fused feature maps are respectively mapped to the fused feature map to obtain the regional feature maps corresponding to the multiple fused feature maps. The regional feature map corresponding to each fused feature map includes the region in the fused feature map where the target detection frame corresponding to the fused feature map is mapped; it can be understood that the present embodiment obtains the target detection frame through S101-S103, and the next step is to identify the target under the target detection frame. The target detection frame is marked on the input image, and the features under the target detection frame in the input image are not obvious enough to realize the recognition of the target under the target detection frame. Therefore, the target detection frame is mapped to the corresponding fused feature map, and the features of the area corresponding to the target detection frame in the fused feature map are used to identify the target recognition result.

[0092] In one example, S104 includes: inputting the target detection box corresponding to each of the fused feature maps and the fused feature map into the ROI Align network, and obtaining the regional feature map corresponding to the fused feature map output by the ROI Align network; wherein the ROI Align network is used to:

[0093] According to the ratio of the size of the fused feature map to the size of the input image, adjusting the size of the target detection frame corresponding to the fused feature map;

[0094] Mapping the adjusted target detection frame to the fused feature map, and extracting the regional feature map corresponding to the target detection frame;

[0095] Based on a bilinear interpolation method, the regional feature map is adjusted to a predetermined size.

[0096] In this example, the target detection box corresponding to the fused feature map and the fused feature map are input into the ROI Align (Region of Interest Align) network to obtain the regional feature map corresponding to the fused feature map. For example, the target detection box T2 corresponding to the fused feature map C2 is input into the ROI Align network, and the regional feature map corresponding to T2 mapped in C2 is output.

[0097] In this example, based on the target detection frame, ROI Align is used to extract the features of the area corresponding to the target detection frame from the corresponding fusion feature map. Since the sizes of target detection frames are different, the corresponding area sizes are also different. In order to obtain target recognition results later, ROI Align is used to output the area feature map of a predetermined size.

[0098] ROI Align is based on the bilinear interpolation method to adjust the regional feature map to a predetermined size. Compared with the ROI Pooling (Region of Interest Pooling) network used in related technologies, it avoids the deviation problem of the position of the regional feature map caused by quantization rounding, thereby improving the accuracy of target detection.

[0099] In S105, the input image and the regional feature map corresponding to each fused feature map are respectively input into the target recognition network to obtain multiple target recognition results output by the target recognition network. In practical applications, the target recognition result includes the input image marked with a target detection frame and target information. The target information includes the target category.

[0100] In one example, the target recognition network includes a tiled layer and a fully connected layer; the tiled layer is used to convert the number of channels of the input regional feature map to one; and the fully connected layer is used to perform convolution calculation on the output of the tiled layer.

[0101] In this example, the tiled layer is used to transform the output three-dimensional region feature map into a two-dimensional region feature map so that the region feature map can be convolved with the fully connected layer. In practical applications, two fully connected layers can be used to classify the objects under the region feature map.

[0102] The following is an illustrative description of the target detection process in combination with the above examples: Figure 6 A flowchart of another target detection method provided in Example 1 of the present application is shown as follows: Figure 6 As shown, the input image 61 is input into the ResNet-50 network 62 to obtain the output feature maps at each level, the feature maps at each level are fused to obtain multiple fused feature maps 63, the multiple fused feature maps 63 are input into the region proposal network 64 to obtain the target detection frame 65, the input image 61, the target detection frame 65 and the fused feature map 63 corresponding to the target detection frame 65 are input into the ROI Align network 66 to obtain the regional feature map, the regional feature map and the input image are input into the target recognition network, and the target recognition result 69 is obtained through the tiled layer 67 and the fully connected layer 68 in the target recognition network.

[0103] In the target detection method provided by this embodiment, based on the top-level feature map and the feature map set including the feature map of at least one level except the top-level, multiple fused feature maps including the top-level feature map and the fusion result of the top-level feature map and the feature map set are obtained, each fused feature map is input into the region proposal network to obtain multiple target detection frames, and then the targets under the multiple target detection frames are identified according to the target recognition network to obtain the target recognition result. The fused feature map in this solution combines the semantic information of the top-level feature map and the position information of the bottom-level feature map, so when the target recognition result is obtained based on the fused feature map, the detection of small targets can be realized, thereby improving the accuracy of target detection.

[0104] Embodiment 2

[0105] Figure 7 This is a schematic diagram of the structure of the target detection device provided in Example 2 of the present application, such as Figure 7 As shown, the target detection method provided in this embodiment includes:

[0106] A feature extraction module 71 is used to extract features from an input image to obtain feature maps at each level; wherein the level of the feature map is positively correlated with the semantic information of the feature map;

[0107] A feature fusion module 72, configured to obtain a plurality of fused feature maps according to the top-level feature map and the feature map set; the plurality of fused feature maps include the top-level feature map and the fusion result of the top-level feature map and the feature map set; each feature map set includes at least one level of feature maps except the top-level;

[0108] The target detection frame generation module 73 is used to input each fused feature map and the input image into a region proposal network, so that the region proposal network performs target detection according to the fused feature map and the input image, and obtains a target detection frame corresponding to each fused feature map;

[0109] The target detection module 74 is used to map the target detection frame to the fused feature map corresponding to the target detection frame to obtain a regional feature map corresponding to each fused feature map; the regional feature map includes the region where the target detection frame is mapped in the fused feature map;

[0110] The target recognition module 75 is used to input the input image and the regional feature map corresponding to each fusion feature map into the target recognition network to obtain multiple target recognition results.

[0111] In this embodiment, the input image is an image to be detected. In practical applications, the input image can be input in a predetermined size to facilitate subsequent processing. Feature extraction is performed on the input image to obtain feature maps at each level. The level of the feature map is positively correlated with the semantic information of the feature map. The semantic information is information provided by meaningful features. It can be understood that the higher the level of the feature map, the stronger the semantic information, but the lower the resolution.

[0112] In one example, the feature extraction module 71 is specifically used to input the input image into the residual network to obtain the feature maps at each level output by the residual network; the residual network includes residual blocks of multiple consecutive stages, the output of the residual block of the current stage is the input of the residual block of the next stage, and the feature maps output by the residual blocks of each stage are used as the feature maps at each level; the residual network is trained based on the difference between the output and input of each residual block approaching zero.

[0113] In this example, the residual network includes multiple residual blocks, and the feature map output by each residual block is used as the feature map at each level. The residual network is trained based on the difference between the output and input of the residual block approaching zero. The difference between the output and input of the residual block is the residual function. Exemplarily, the input of the residual block is x, the output is F(x), the residual function is H(x)=F(x)-x, and the residual network is trained based on H(x) approaching 0. It can be seen from practical applications that optimizing the residual function is much easier than optimizing the objective function F(x). For details, see the relevant technology. Therefore, the residual network is easier to train than the commonly used VGG network based on objective function training. This also ensures the identity mapping between the input and output of the residual block. Therefore, using the residual network to obtain the feature map can avoid the problem of deep network degradation, and thus can improve the accuracy of the target network model in detecting the input image.

[0114] After the feature extraction module 71 obtains the feature maps of each level, the feature fusion module 72 obtains multiple fused feature maps based on the feature map of the top layer and the feature map set. The multiple fused feature maps include the feature map of the top layer and the fusion result of the feature map of the top layer and the feature map set; the sizes of the multiple fused feature maps are different, and each feature map set includes at least one level of feature maps except the top layer.

[0115] Based on the above scheme, in one example, the number of feature map sets is multiple, and the feature maps in different feature map sets are different. That is to say, the fused feature map includes multiple fusion results, and the number of fused feature maps exceeds 2. Therefore, through this example, multiple fused feature maps can be obtained, and the input image can be detected from multiple levels, thereby improving the accuracy of target detection.

[0116] The feature fusion module 72 is exemplarily described below.

[0117] A feature fusion module 72 is specifically used to obtain a fused feature map based on the feature map of the top layer;

[0118] The feature fusion module 72 is specifically used to fuse the feature map of the topmost layer with the feature map set corresponding to the predetermined layer except the topmost layer, and use the fusion result as the fused feature map until all the predetermined layers are fused.

[0119] In practical applications, all levels corresponding to the output feature maps can be set as preset levels, or some levels corresponding to the output feature maps can be set as preset levels. The feature map set corresponding to each preset level includes feature maps at all levels not lower than the preset level except the topmost level.

[0120] In practical applications, the predetermined levels may be continuous adjacent levels, or of course, the predetermined levels may also be discontinuous levels.

[0121] The following will take the scenario where the predetermined level is one of the multiple consecutive adjacent levels of the next level of the top level as an example to illustrate the feature fusion module 72.

[0122] A feature fusion module 72, specifically configured to perform convolution processing on the feature maps corresponding to the top layer and the predetermined layer, so that the number of channels of the feature maps corresponding to the top layer and the predetermined layer after processing are consistent;

[0123] The feature fusion module 72 is further configured to use the top-most feature map as the top-most fused feature map, and perform the following processing for each predetermined level in order from top to bottom:

[0124] The feature fusion module 72 is further configured to upsample the fused feature map of the previous level of the predetermined level;

[0125] The feature fusion module 72 is specifically used to fuse the feature map at this level with the feature map obtained by upsampling to obtain a fused feature map at this level.

[0126] In practical applications, the size of the output feature map is usually halved layer by layer as the level increases. To ensure that the information gap between feature maps at adjacent levels is not too large, the number of channels of the output feature map is doubled layer by layer. Therefore, in this example, the feature maps corresponding to the topmost layer and the predetermined layer are processed so that the number of channels of the feature maps corresponding to the topmost layer and the predetermined layer after processing are consistent, so that a fused feature map can be generated based on the feature maps corresponding to the topmost layer and the predetermined layer.

[0127] For the processing process, as an example, the feature fusion module 72 is specifically used to perform convolution operations on the feature maps corresponding to the top layer and each of the predetermined layers with a first number of 1x1 convolution kernels, where the first number is the number of channels of the feature map after convolution processing.

[0128] In this example, the feature fusion module 72 performs convolution operation on the topmost feature map with the first number of 1x1 convolution kernels, and similarly performs convolution operation on the features at each level with the first number of 1x1 convolution kernels. It can be understood that the 1x1 convolution kernel will not affect the feature values ​​of the feature maps at each level, and the number of channels of the feature maps corresponding to the topmost layer and the predetermined level after the operation are both the first number.

[0129] In this example, the feature maps corresponding to the top layer and the predetermined layer are processed with a unified number of channels through the first number of 1x1 convolution kernels, which can avoid affecting the feature values ​​in the feature maps at each layer, thereby further ensuring the accuracy of the target recognition results.

[0130] After unifying the number of channels, the top-level feature map is used as the top-level fused feature map. The fused feature map of the previous level of each predetermined level is upsampled from top to bottom, and the size of the upsampled fused feature map is consistent with that of the feature map at the predetermined level. The feature map at this level is then fused with the upsampled feature map to obtain the fused feature map at this level.

[0131] In this example, the feature map at the predetermined level is fused with the fused feature map corresponding to the previous level, and the fusion result of the previous level is used, thereby reducing the amount of calculation in the target detection process. In this example, the feature map set includes all levels except the top level that are not lower than the predetermined level, enriching the semantic information and local position information in the fused fused feature map, and also improving the transition between features in the fused feature map, thereby improving the accuracy of target recognition detection.

[0132] After obtaining multiple fused feature maps, each fused feature map and the input image are input into a region proposal network to obtain target detection frames corresponding to multiple fused feature maps. The output target detection frame is generated based on the input image, that is, the output target detection frame is marked on the input image. The region proposal network is used for target detection.

[0133] The target detection frame generation module 73 is exemplarily described below.

[0134] In one example, the target detection frame generation module 73 is specifically used to set a plurality of candidate frames with predetermined scales and different proportions at positions corresponding to each pixel point of the fused feature map in the input image for the input fused feature map; wherein different fused feature maps correspond to different scales, and the size of the fused feature map is negatively correlated with the scale;

[0135] The target detection frame generation module 73 is further configured to determine the region types corresponding to the plurality of candidate frames, and delete the candidate frames whose region types are background types, to obtain the target candidate frame; wherein the region types include foreground types and background types;

[0136] The target detection frame generation module 73 is further specifically used to correct and filter the target candidate frame to obtain the target detection frame corresponding to the fused feature map.

[0137] In practical applications, the fused feature map is generated based on the feature maps at each level of the input image, so each pixel in the fused feature map corresponds to a pixel in the input image. In this embodiment, multiple candidate boxes are set at positions corresponding to each pixel in the fused feature map in the input image, and the multiple candidate boxes are all of the same predetermined scale and different proportions. The scale of the candidate box is the area of ​​the candidate box. In addition, different fused feature maps correspond to different scales, and the size of the fused feature map is negatively correlated with the scale.

[0138] That is to say, each input fusion feature map corresponds to a candidate box of a certain scale, and the ratio of the candidate box is varied, so that objects of different shapes can be selected more accurately. In addition, the larger the size of the fusion feature map, the smaller the scale of the corresponding candidate box. The larger the scale of the fusion feature map, the higher the resolution of the fusion feature map, the clearer the local position information, and the more features of the corresponding small objects. Therefore, in the fusion feature map corresponding to the lower level, a relatively small scale can be used to select small objects. It can be understood that candidate boxes of different scales are used to select objects of different sizes in the input image.

[0139] Through the above scheme, a large number of candidate boxes marked on the input image are obtained. The areas under these candidate boxes may be foreground or background, where the foreground is the target to be identified, while the background does not need to be identified. Therefore, in this example, the Softmax algorithm is used to determine the area types corresponding to multiple candidate boxes, and the area types include foreground type and background type. The candidate boxes with the area type of background type are deleted, and the target candidate boxes with the area type of foreground type are obtained.

[0140] Then, the target candidate frame is corrected according to the regression loss function. Among the corrected target candidate frames, there may be some target candidate frames with high overlap. It can be understood that these target candidate frames with high overlap correspond to the same target. Therefore, in this example, the corrected target candidate frames are filtered by the non-maximum suppression method to filter out the target candidate frames with high overlap, and obtain the target detection frame corresponding to the fusion feature map.

[0141] Then, the candidate boxes in the input image are classified by the Softmax algorithm to obtain the target candidate boxes with promising prospects. The target candidate boxes are then corrected by the regression loss function. The corrected target candidate boxes are filtered based on the non-maximum suppression method to filter out the target candidate boxes with large duplication to obtain the final target detection box. The target detection box can be output in the form of parameters.

[0142] In this example, a target candidate box is directly generated on the input image based on the pixel points of the fused feature map. Compared with the method of generating a target detection box by sliding the candidate box in the prior art, the generation speed of the target detection box is improved.

[0143] The target detection module 74 will be further described below.

[0144] After obtaining the target detection frames corresponding to multiple fused feature maps through the target detection module 74, the target detection frames corresponding to the multiple fused feature maps are respectively mapped to the fused feature map to obtain the regional feature maps corresponding to the multiple fused feature maps. The regional feature map corresponding to each fused feature map includes the region where the target detection frame corresponding to the fused feature map is mapped in the fused feature map; it can be understood that the target detection frame is obtained in this embodiment, and the next step is to identify the target under the target detection frame. The target detection frame is marked on the input image, and the features under the target detection frame in the input image are not obvious enough to realize the recognition of the target under the target detection frame. Therefore, the target detection frame is mapped to the corresponding fused feature map, and the features of the area corresponding to the target detection frame in the fused feature map are used to identify the target recognition result.

[0145] In one example, the target detection module 74 is specifically used to input the target detection box corresponding to each of the fused feature maps and the fused feature map into the ROI Align network to obtain the regional feature map corresponding to the fused feature map output by the ROI Align network; wherein the ROI Align network is used to:

[0146] According to the ratio of the size of the fused feature map to the size of the input image, adjusting the size of the target detection frame corresponding to the fused feature map;

[0147] Mapping the adjusted target detection frame to the fused feature map, and extracting the regional feature map corresponding to the target detection frame;

[0148] Based on a bilinear interpolation method, the regional feature map is adjusted to a predetermined size.

[0149] In this example, the target detection box and the fused feature map corresponding to the fused feature map are input into the ROI Align (Region of Interest Align, region of interest quantization) network to obtain the regional feature map corresponding to the fused feature map.

[0150] In this example, based on the target detection frame, ROI Align is used to extract the features of the area corresponding to the target detection frame from the corresponding fusion feature map. Since the sizes of target detection frames are different, the corresponding area sizes are also different. In order to obtain target recognition results later, ROI Align is used to output the area feature map of a predetermined size.

[0151] ROI Align is based on the bilinear interpolation method to adjust the regional feature map to a predetermined size. Compared with the ROI Pooling (Region of Interest Pooling) network used in related technologies, it avoids the deviation problem of the position of the regional feature map caused by quantization rounding, thereby improving the accuracy of target detection.

[0152] The input image and the regional feature map corresponding to each fusion feature map are respectively input into the target recognition network through the target recognition module 75 to obtain multiple target recognition results output by the target recognition network. In practical applications, the target recognition result includes the input image marked with the target detection frame and target information. The target information includes the target category.

[0153] In one example, the target recognition network includes a tiled layer and a fully connected layer; the tiled layer is used to convert the number of channels of the input regional feature map to one; and the fully connected layer is used to perform convolution calculation on the output of the tiled layer.

[0154] In this example, the tiled layer is used to transform the output three-dimensional region feature map into a two-dimensional region feature map so that the region feature map can be convolved with the fully connected layer. In practical applications, two fully connected layers can be used to classify the objects under the region feature map.

[0155] In the target detection device provided by this embodiment, the feature fusion module obtains multiple fused feature maps including the top-level feature map and the fusion result of the top-level feature map and the feature map set based on the top-level feature map and the feature map set including at least one level of feature maps except the top-level feature map. The target detection frame generation module inputs each fused feature map into the region proposal network to obtain multiple target detection frames. The target recognition module recognizes the targets under the multiple target detection frames based on the target recognition network to obtain the target recognition result. The fused feature map in this solution combines the semantic information of the top-level feature map and the position information of the bottom-level feature map. Therefore, when obtaining the target recognition result based on the fused feature map, it is possible to detect small targets, thereby improving the accuracy of target detection.

[0156] Embodiment 3

[0157] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Example 3 of the present application, such as Figure 8 As shown, the electronic device includes:

[0158] The electronic device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, the memory 292, and the communication interface 293 may communicate with each other through the bus 294. The communication interface 293 may be used for information transmission. The processor 291 may call the logic instructions in the memory 292 to execute the method of the above embodiment.

[0159] In addition, the logic instructions in the above-mentioned memory 292 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0160] The memory 292 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present application. The processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 292, that is, implementing the methods in the above method embodiments.

[0161] The memory 292 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 292 may include a high-speed random access memory and may also include a non-volatile memory.

[0162] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in any embodiment.

[0163] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0164] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A target detection method, characterized in that: include: Extracting features from the input image to obtain feature maps at each level; wherein the level of the feature map is positively correlated with the semantic information of the feature map; According to the feature map of the top layer and the feature map set, a plurality of fused feature maps are obtained; the plurality of fused feature maps include the feature map of the top layer and the fusion result of the feature map of the top layer and the feature map set; each feature map set includes the feature map of at least one level except the top layer, wherein the plurality of fused feature maps are obtained according to the feature map of the top layer and the feature map set, including: for the top layer, obtaining a fused feature map based on the feature map of the top layer; for each second predetermined level except the top layer, the feature map of the top layer is respectively fused with the feature map under the second predetermined level to obtain fused feature maps of multiple levels; Input each fused feature map and the input image into a region proposal network, so that the region proposal network performs target detection according to the fused feature map and the input image, and obtains a target detection frame corresponding to each fused feature map; wherein the output target detection frame is marked on the input image; Mapping the target detection frame to the fused feature map corresponding to the target detection frame to obtain a regional feature map corresponding to each fused feature map; the regional feature map includes the region where the target detection frame is mapped in the fused feature map; wherein mapping the target detection frame corresponding to each fused feature map to the fused feature map to obtain the regional feature maps corresponding to the multiple fused feature maps comprises: inputting the target detection frame corresponding to each fused feature map and the fused feature map into an ROI Align network to obtain a regional feature map corresponding to the fused feature map output by the ROI Align network; wherein the ROI Align network is used to: adjust the size of the target detection frame corresponding to the fused feature map according to the ratio of the size of the fused feature map to the size of the input image; mapping the adjusted target detection frame to the fused feature map to extract the regional feature map corresponding to the target detection frame; adjusting the regional feature map to a predetermined size based on a bilinear interpolation method; The input image and the regional feature map corresponding to each fusion feature map are input into the target recognition network to obtain multiple target recognition results.

2. The method according to claim 1, characterized in that: There are multiple feature map sets, and the feature maps in each feature map set are not completely the same.

3. The method according to claim 2, characterized in that The method of obtaining multiple fused feature maps based on the top-level feature map and the feature map set includes: For the top layer, a fused feature map is obtained based on the feature map of the top layer; For predetermined levels other than the topmost layer, the feature map of the topmost layer is fused with the feature map set corresponding to the predetermined level, and the fusion result is used as the fused feature map until all the predetermined levels are fused; wherein the feature map set corresponding to each predetermined level includes feature maps of all levels not lower than the predetermined level except the topmost layer.

4. The method according to claim 3, characterized in that The predetermined level is one of a plurality of consecutive adjacent levels of a level below the topmost level; for the predetermined levels other than the topmost level, the feature map of the topmost level is fused with a set of feature maps corresponding to the predetermined level, and the fusion result is used as the fused feature map, until all the predetermined levels are fused, including: Performing convolution processing on the feature maps corresponding to the top layer and the predetermined layer, so that the number of channels of the feature maps corresponding to the top layer and the predetermined layer after processing is consistent; The feature map of the top layer is used as the fused feature map of the top layer, and the following processing is performed for each predetermined level from top to bottom: the fused feature map of the previous level of the predetermined level is upsampled; the feature map obtained by the upsampling has the same size as the feature map at the predetermined level; the feature map at the predetermined level is fused with the feature map obtained by the upsampling to obtain the fused feature map corresponding to the predetermined level.

5. The method according to claim 4, characterized in that The performing convolution processing on the feature maps corresponding to the topmost layer and the predetermined layer includes: The feature maps corresponding to the top layer and each of the predetermined layers are respectively convolved with a first number of 1x1 convolution kernels; the first number is the number of channels of the feature map after the convolution processing.

6. The method according to any one of claims 1 to 5, characterized in that: The feature extraction of the input image to obtain feature maps at each level includes: The input image is input into the residual network to obtain the feature maps at each level output by the residual network; the residual network includes residual blocks of multiple consecutive stages, the output of the residual block of the current stage is the input of the residual block of the next stage, and the feature maps output by the residual blocks of each stage are used as the feature maps at each level; the residual network is trained based on the difference between the output and input of the residual block approaching zero.

7. The method according to any one of claims 1 to 5, characterized in that: The step of enabling the region proposal network to perform target detection according to the fused feature map and the input image to obtain target detection frames corresponding to each of the plurality of fused feature maps includes: For the input fusion feature map, a plurality of candidate frames having a predetermined scale and different proportions are set at a position corresponding to each pixel point of the fusion feature map in the input image; wherein different fusion feature maps correspond to different scales, and the size of the fusion feature map is negatively correlated with the scale; Determine the region types corresponding to the multiple candidate frames, and delete the candidate frames whose region types are background types, to obtain a target candidate frame; wherein the region types include foreground types and background types; The target candidate frame is corrected and filtered to obtain the target detection frame corresponding to the fused feature map.

8. A target detection device, characterized in that: include: A feature extraction module, used to extract features from an input image to obtain feature maps at each level; wherein the level of the feature map is positively correlated with the semantic information of the feature map; A feature fusion module, configured to obtain a plurality of fused feature maps according to the top-level feature map and the feature map set; the plurality of fused feature maps include the top-level feature map and a fusion result of the top-level feature map and the feature map set; each feature map set includes a feature map of at least one level except the top-level; The target detection frame generation module is used to input each fused feature map and the input image into a region proposal network, so that the region proposal network performs target detection according to the fused feature map and the input image, and obtains a target detection frame corresponding to each fused feature map; wherein the output target detection frame is marked on the input image; A target detection module, used for mapping the target detection frame to the fused feature map corresponding to the target detection frame, and obtaining a regional feature map corresponding to each fused feature map; the regional feature map includes the region in which the target detection frame is mapped in the fused feature map; wherein, mapping the target detection frame corresponding to each fused feature map to the fused feature map, and obtaining the regional feature maps corresponding to the multiple fused feature maps, comprises: inputting the target detection frame corresponding to each fused feature map and the fused feature map into the ROIAlign network, and obtaining the regional feature map corresponding to the fused feature map output by the ROIAlign network; wherein, the ROIAlign network is used for: adjusting the size of the target detection frame corresponding to the fused feature map according to the ratio of the size of the fused feature map to the size of the input image; mapping the adjusted target detection frame to the fused feature map, and extracting the regional feature map corresponding to the target detection frame; adjusting the regional feature map to a predetermined size based on a bilinear interpolation method; A target recognition module, used for inputting the input image and the regional feature map corresponding to each fusion feature map into a target recognition network to obtain multiple target recognition results; Among them, the feature fusion module is specifically used to obtain a fused feature map based on the feature map of the top layer for the top layer; for each second predetermined level except the top layer, the feature map of the top layer is respectively fused with the feature map under the second predetermined level to obtain fused feature maps of multiple levels.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images