Target detection method and device, medium and electronic equipment

Abstract

The invention discloses a target detection method and device, a medium and electronic equipment, and relates to the field of computer vision. The accuracy of multi-scale target detection can be improved. The method comprises the following steps: acquiring a to-be-detected input image; the input image is input into a feature extraction model, feature extraction is carried out on the input image, and the feature extraction model comprises a first feature extraction module, a first feature fusion module and a first feature weighting module; the first feature extraction module extracts pyramid features from the input image, wherein the pyramid features comprise a plurality of levels of first feature maps; the first feature fusion module fuses the first feature maps of each level in the pyramid features to obtain a second feature map of each level after fusion; the first feature weighting module is used for weighting the second feature map of each level to obtain detection pyramid features; and performing target detection on the detection pyramid features to obtain a detection result.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a target detection method, device, medium and electronic device. Background Art

[0002] As an important task in computer vision, object detection has a wide range of applications, including intelligent security, autonomous driving, robot navigation, medical diagnosis and other fields.

[0003] The main purpose of target detection is to accurately identify the category of objects and locate the position of specific targets in complex visual scenes. With the development of technology and the diversification of data sources, traditional single-scale features may not be able to effectively capture the features of targets of different sizes and resolutions. To solve this problem, modern target detection algorithms introduce pyramid structures or feature pyramid networks (FPNs) to organize feature images from different scales in a hierarchical manner. This strategy enables the algorithm to handle both small and large targets at the same time, thereby improving the accuracy and generalization of detection.

[0004] However, when existing target detection algorithms extract feature information, information between different levels of the feature pyramid is transmitted separately, resulting in that features of different scales are not effectively utilized, making it difficult to improve detection accuracy. Summary of the invention

[0005] The present application provides a target detection method, device, medium and electronic device, which can enhance the fusion between features at different levels of a feature pyramid, thereby improving detection accuracy.

[0006] In a first aspect, the present application provides a target detection method, comprising:

[0007] Get the input image to be detected;

[0008] Inputting the input image into a feature extraction model to extract features from the input image, wherein the feature extraction model includes a first feature extraction module, a first feature fusion module, and a first feature weighting module;

[0009] The first feature extraction module extracts a pyramid feature from the input image, wherein the pyramid feature includes a first feature map of multiple levels;

[0010] The first feature fusion module fuses the first feature maps of each level in the pyramid feature to obtain a fused second feature map of each level;

[0011] The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature;

[0012] Performing target detection on the detection pyramid features to obtain a detection result.

[0013] According to the target detection method in this embodiment, the feature extraction model is used to extract features from the input image. The feature extraction model includes a first feature extraction module, a first feature fusion module and a first feature weighting module. The first feature extraction module extracts the pyramid features of the input image. The first feature fusion module fuses the feature maps of different levels of the pyramid features, which can enhance the fusion between features of different scales, thereby balancing the importance of low-level and high-level information, avoiding the loss of local or global information, and facilitating the improvement of detection accuracy during detection. In addition, the first feature weighting module is used to weight the features of each level in the pyramid features, thereby enhancing the attention and utilization of important features in the feature extraction process, accelerating the convergence of the model, and improving the feature extraction capability of the model.

[0014] In an exemplary embodiment, the step of fusing the first feature maps of each level in the pyramid features to obtain the fused second feature maps of each level includes:

[0015] The first feature map of the target layer in the pyramid feature is merged with the first feature map of the adjacent upper layer of the target layer to obtain the second feature map of the target layer;

[0016] The target layer is a layer other than the top layer in the pyramid feature; the second feature map of the top layer of the pyramid feature is the same as the first feature map.

[0017] In an exemplary embodiment, the feature extraction model further includes a second feature fusion module,

[0018] The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature, including:

[0019] The second feature fusion module fuses the second feature map of the target layer in the pyramid feature with the second feature maps of each upper layer above the target layer to obtain a third feature map of the target layer;

[0020] The target layer is a layer other than the top layer in the pyramid feature; the third feature map of the top layer of the pyramid feature is the same as the second feature map;

[0021] The first feature weighting module weights the third feature maps of each level to obtain detection pyramid features.

[0022] In an exemplary embodiment, the feature extraction model further includes a third feature fusion module, and the levels of the pyramid features are top layer, middle layer, and bottom layer from top to bottom.

[0023] The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature, including:

[0024] The third feature fusion module fuses the third feature map of the top layer with the third feature map of the bottom layer in the pyramid feature to obtain a fourth feature map of the bottom layer; and

[0025] The third feature map of the top layer and the third feature map of the middle layer in the pyramid feature are merged to obtain a fourth feature map of the middle layer;

[0026] The fourth feature map of the top layer in the pyramid feature is the same as the third feature map;

[0027] The first feature weighting module weights the fourth feature map of each level to obtain a detection pyramid feature.

[0028] In an exemplary embodiment, the first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature, including:

[0029] The first feature weighting module performs maximum pooling on the second feature map of each level to obtain a first pooled feature map, and performs average pooling on the second feature map of each level to obtain a second pooled feature map;

[0030] The first feature weighting module combines the first pooled feature map and the second pooled feature map of each level to obtain a feature map to be screened at each level;

[0031] The first feature weighting module determines the weight of the feature map to be screened through an activation function, and weights the feature map to be screened based on the weight to obtain a detection pyramid feature.

[0032] In an exemplary embodiment, the step of fusing the first feature maps of each level in the pyramid features to obtain the fused second feature map of each level includes:

[0033] Upsampling the first feature map of the upper layer of the target layer in the pyramid feature to obtain a first feature map to be fused in the upper layer, wherein the first feature map to be fused has the same size as the first feature map of the target layer;

[0034] Fusing the first feature map to be fused with the first feature map of the target layer to obtain a second feature map of the target layer;

[0035] The target layer is a layer other than the uppermost layer in the pyramid feature.

[0036] In an exemplary embodiment, the feature extraction model further includes a second feature weighting module for weighting the fourth feature map of the top layer and the fourth feature map of the bottom layer, and fusing the weighted feature maps to obtain detection pyramid features.

[0037] In a second aspect, the present application provides a target detection device, comprising:

[0038] An image acquisition module, used to acquire an input image to be detected;

[0039] An image input module, used for inputting the input image into a feature extraction model to perform feature extraction on the input image, wherein the feature extraction model includes a first feature extraction module, a first feature fusion module and a first feature weighting module;

[0040] The first feature extraction module extracts a pyramid feature from the input image, wherein the pyramid feature includes a first feature map of multiple levels;

[0041] The first feature fusion module fuses the first feature maps of each level in the pyramid feature to obtain a fused second feature map of each level;

[0042] The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature;

[0043] The detection output module is used to perform target detection on the detection pyramid features to obtain a detection result.

[0044] In a third aspect, the present application provides an electronic device, the electronic device comprising a memory and one or more processors. The memory stores one or more computer programs, the computer programs comprising instructions, and when the instructions are executed by the processor, the electronic device can perform the target detection method as in the first aspect.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the target detection method as in the first aspect.

[0046] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the target detection method as described in the first aspect.

[0047] It can be understood that the beneficial effects that can be achieved by the target detection device, electronic device, computer-readable storage medium, and computer program product provided above can refer to the beneficial effects in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a target detection method provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the difference. It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way. In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more.

[0052] It should be noted that the “at…” in the embodiments of the present application may be the instant when a certain situation occurs, or may be a period of time after the occurrence of a certain situation, and the embodiments of the present application do not specifically limit this.

[0053] The implementation of this embodiment will be described in detail below with reference to the accompanying drawings.

[0054] This embodiment provides a target detection method. By way of example, the target detection method can be applied to various electronic devices such as computers (PCs), tablet computers, virtual reality / augmented reality devices, wearable devices, industrial computers, and vehicle computers; it can also be applied to servers, cloud computing, server clusters, etc., and this embodiment does not make any special limitations on this.

[0055] Figure 1A schematic flow chart of a target detection method provided in an embodiment of the present application is shown.

[0056] like Figure 1 As shown, the target detection method may include the following steps:

[0057] Step 101: Obtain an input image to be detected.

[0058] The input image may be an image obtained by photographing the object to be detected, or an image obtained by preprocessing the image obtained by photographing. The object to be detected may be a scene, an object, a human body, an animal, or a face, etc. The preprocessing may include cropping, grayscale processing, slicing processing, etc., which are not particularly limited in this embodiment.

[0059] Step 102: Input the input image into a feature extraction model to perform feature extraction on the input image. The feature extraction model includes a first feature extraction module, a first feature fusion module and a first feature weighting module.

[0060] Exemplarily, the input image may be represented as image I, and a slice operation may be performed on image I to obtain an image S after the slice operation, and then image S is input into a feature extraction model for feature extraction processing.

[0061] The slicing operation specifically includes: dividing the image in a specific way, reducing the height and width of the input image, and using multiple convolution kernels to increase the number of channels of the input image. After the slicing operation, the spatial information of the input image is retained, and the number of channels is increased to form a new feature map. For example, if the input image is 3*256*256, after the slicing operation, a 32*128*128 feature map can be generated.

[0062] The feature extraction model in this embodiment may include a first feature extraction module, a first feature fusion module and a first feature weighting module, through which the input image is subjected to feature extraction to obtain the detection pyramid feature of the input image. Specifically:

[0063] Step 103: The first feature extraction module uses a convolutional neural network to extract pyramid features from the input image, where the pyramid features include first feature maps of multiple levels.

[0064] Step 104: The first feature fusion module fuses the first feature maps of each level in the pyramid feature to obtain a fused second feature map of each level.

[0065] Step 105: The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature.

[0066] Step 106: Perform target detection on the detection pyramid features to obtain a detection result.

[0067] Among them, the first feature extraction module, the first feature fusion module and the first feature weighting module are all implemented by convolutional layers, and these convolutional layers process the input image in sequence to obtain the detection pyramid features of the input image.

[0068] The first feature extraction module uses a convolutional neural network to extract features, and the convolutional neural network can be a feature pyramid network. The input image is input into the feature pyramid network to extract the pyramid features of the input image. The pyramid feature refers to a plurality of images with decreasing sizes, and each layer of the image is reduced based on the previous layer of the image.

[0069] Exemplarily, the feature pyramid network can be composed of a three-layer structure, each layer includes: two convolution layers, each convolution layer can be a 3*3 convolution kernel, padding (padding) is 1, each convolution layer can be followed by a batch normalization layer and a ReLU activation function, and finally a maximum pooling layer is connected, the pooling window size is 2*2, and the stride is 2. The input image is first sliced ​​to generate an image S, and then the image S is subjected to a feature extraction operation. After each layer of processing, a new feature map with a size reduced by one time is obtained. After the input image is processed by three layers, three feature maps C1, C2, and C3 of different scales are obtained, namely, the first feature map. The scales of the first feature maps at different levels are different. From top to bottom, the scales are from small to large. The scale of the bottom layer is the largest, and the scale of the top layer is the smallest. The feature map at the bottom layer contains more local information, while the feature map at the top layer contains global information.

[0070] The feature pyramid network may also include a structure with more layers, such as 5 layers, 6 layers, etc. Correspondingly, the pyramid features extracted by the feature pyramid network may also include first feature maps of more levels, such as 5 levels, 6 levels, etc.

[0071] After the feature pyramid network extracts pyramid features of multiple levels, the pyramid features can be used as input to the first feature fusion module, which fuses the first feature maps of each level to obtain the second feature map of each level after fusion.

[0072] The first fusion module fuses different first feature maps by adding or multiplying corresponding elements in different first feature maps. Specifically, the first fusion module fuses the first feature map of the target layer in the pyramid feature with the first feature map of the adjacent upper layer of the target layer to obtain the second feature map of the target layer. The target layer is the layer other than the top layer in the pyramid feature, and the second feature map of the top layer of the pyramid feature is the same as the first feature map.

[0073] Taking the pyramid feature including three levels as an example, the three levels are called the top level, the middle level, and the bottom level from top to bottom, then the top level is the top level, and the target level includes the middle level and the bottom level.

[0074] When the target layer is the middle layer, its adjacent upper layer is the top layer, and when the target layer is the bottom layer, its adjacent upper layer is the middle layer. The first fusion module can fuse the first feature map of the top layer and the first feature map of the middle layer, and the fused feature map is used as the second feature map of the middle layer. The first fusion module can fuse the first feature map of the middle layer and the first feature map of the bottom layer, and the fused feature map is used as the second feature map of the bottom layer. The first feature map of the upper layer is directly passed as the second feature map to the next module, that is, the first feature weighting module for processing. After fusion, the second feature maps of each layer can form a new pyramid feature and pass it to the first feature weighting module.

[0075] In this implementation, feature maps of adjacent scales are fused, and the model can learn feature expressions of different scales, thereby being more suitable for the detection of objects to be detected at different scales, which is beneficial to improving the accuracy of multi-scale target detection.

[0076] Exemplarily, the feature extraction model also includes a second feature fusion module, which fuses the second feature map of the target layer in the pyramid feature with the second feature maps of each upper layer above the target layer to obtain a third feature map of the target layer; wherein the target layer is the level in the pyramid feature except the top layer; the third feature map of the top layer of the pyramid feature is the same as the second feature map; and the first feature weighting module weights the third feature maps of each level to obtain a detection pyramid feature.

[0077] Specifically, when the target layer is the bottom layer, the second feature fusion module can fuse the second feature map of the top layer, the second feature map of the middle layer and the second feature map of the bottom layer, and use the fused feature map as the third feature map of the bottom layer. When the target layer is the middle layer, the second feature fusion module fuses the second feature map of the top layer and the second feature map of the middle layer, and uses the fused feature map as the third feature map of the middle layer. The second feature map of the top layer is directly used as the third feature map of the top layer, and the third feature maps of each layer constitute a new pyramid feature and are passed to the next module, namely the first feature weighting module, for processing.

[0078] In this implementation, the feature maps of the bottom layer can be fused with the feature maps of all the upper layers, so as to learn the global information extracted by the upper layer and enhance the global information of the lower layer, so that the information contained in the features is richer and the representation ability of the features is enhanced.

[0079] In an exemplary embodiment, the feature extraction model further includes a third feature fusion module, and the levels of the pyramid feature are top layer, middle layer, and bottom layer from top to bottom. The third feature fusion module fuses the third feature map of the top layer in the pyramid feature with the third feature map of the bottom layer to obtain the fourth feature map of the bottom layer; and fuses the third feature map of the top layer in the pyramid feature with the third feature map of the middle layer to obtain the fourth feature map of the middle layer; the fourth feature map of the top layer in the pyramid feature is the same as the third feature map; the first feature weighting module weights the fourth feature map of each level to obtain the detection pyramid feature.

[0080] For example, the pyramid features include feature map P1, feature map P2, and feature map P3 from bottom to top. For the high-level feature map P3, the third feature fusion module first uses a transposed convolution with a step size of 2 and a convolution kernel of 3*3 to expand the high-level feature map to increase the size of the feature map P3, and then uses bilinear interpolation for upsampling to make the sampled feature map P3 the same size as the bottom-level feature map P1, and then fuse the two.

[0081] In this implementation, the top layer is fused with the bottom layer, and the feature maps of non-adjacent layers are fused, which can increase the scale difference between different layers, thereby learning the extraction of features under large-scale differences, making the model more suitable for feature extraction of different scales, and is conducive to improving the accuracy of multi-scale target detection. The fused fourth feature map constitutes a new pyramid feature, which is input into the first feature weighting module for weighting.

[0082] In an exemplary embodiment, when the feature maps of different levels are fused, the feature map of the upper layer can be upsampled first so that the size of the feature map of the upper layer is the same as the size of the feature map of the lower layer, so as to add and fuse the corresponding elements to obtain a fused feature map. In general, the first feature map of the upper layer of the target layer in the pyramid feature is upsampled to obtain the first feature map to be fused of the upper layer, and the first feature map to be fused has the same size as the first feature map of the target layer; the first feature map to be fused is fused with the first feature map of the target layer to obtain the second feature map of the target layer.

[0083] Still taking the pyramid feature as an example with three levels. In this embodiment, three feature fusion modules are provided, and each module adopts the above-mentioned method when performing the fusion processing. For two or three feature maps of different levels to be fused, the feature map of the upper layer is upsampled, the size is increased, and it is the same size as the feature map of the lower layer, and then fused. For example, when the target layer is the bottom layer, when the bottom layer is fused with the first feature map of the middle layer, the first feature map of the middle layer can be upsampled so that the first feature map to be fused after sampling is the same size as the first feature map of the bottom layer, and then the first feature map to be fused is fused with the first feature map of the bottom layer, and the fused feature map is used as the second feature map of the bottom layer. When the bottom layer is fused with the second feature map of the middle layer and the top layer, the second feature map of the middle layer can be upsampled so that the upsampled feature map is the same size as the second feature map of the bottom layer, and the second feature map of the top layer is upsampled so that the upsampled feature map is also the same size as the second feature map of the bottom layer, and then the three feature maps of the same size are added and fused by corresponding elements to obtain a fused feature map, which is used as the third feature map of the bottom layer. When fusing the third feature maps of the bottom layer and the top layer, first upsample the third feature map of the top layer so that the adopted feature map has the same size as the third feature map of the bottom layer, and then fuse the sampled feature map with the third feature map of the bottom layer, and the fused feature map is used as the fourth feature map of the bottom layer.

[0084] The sampling multiple of each upsampling may be different. The upsampling may be implemented by a neighbor interpolation algorithm, a bilinear interpolation algorithm, a bicubic interpolation algorithm, etc., but the present embodiment is not limited thereto.

[0085] After feature fusion, the new pyramid features obtained after fusion can be input into the first feature weighting module. The first feature weighting module can increase the weight of important features and reduce the weight of unimportant features, thereby achieving the effect of feature screening and improving the training efficiency of the model.

[0086] Specifically, the first feature weighting module is implemented using a channel attention mechanism. The pyramid feature is input into the channel attention mechanism module to obtain the weight of each channel and output the weighted detection pyramid feature. The first feature weighting module can also first input the top, middle, and bottom feature maps of the pyramid feature into the channel attention mechanism module for processing to obtain the weight of each channel, and then fuse them with the three feature maps before processing, and then obtain the feature maps of the top, middle, and bottom layers after feature screening through a 1*1 convolution operation.

[0087] In an exemplary implementation, the feature extraction model includes a first feature extraction module, a first feature fusion module, and a first feature weighting module. The first feature weighting module is connected after the first feature fusion module, and is used to weight the second feature map of each level to obtain a detection pyramid feature, including: the first feature weighting module performs maximum pooling on the second feature map of each level to obtain a first pooled feature map, and average pooling on the second feature map of each level to obtain a second pooled feature map; the first feature weighting module combines the first pooled feature map of each level with the second pooled feature map to obtain a feature map to be screened at each level; the first feature weighting module determines the weight of the feature map to be screened through an activation function, and weights the feature map to be screened based on the weight to obtain a detection pyramid feature.

[0088] The first feature weighting module first performs maximum pooling and average pooling on the second feature map of each level in the input pyramid feature, and then combines the two feature maps of each layer after different pooling processing, and then uses the sigmoid activation function to determine the weight of each channel. Among them, the purpose of maximum pooling is to extract the most relevant data from each channel, and the purpose of average pooling is to evenly obtain all data from the feature map. The combination of these pooling methods is conducive to extracting the most representative information from each channel while minimizing information loss.

[0089] Exemplarily, the feature extraction model in this embodiment may include a first feature extraction module, a first feature fusion module, a second feature fusion module, a third feature fusion module, and a first feature weighting module in sequence. The first feature weighting module is connected after the third feature fusion module, and performs weighted processing on the pyramid features output by the third feature fusion module to obtain the final detection pyramid features.

[0090] In an exemplary embodiment, the feature extraction model may further include, in sequence, a first feature extraction module, a first feature fusion module, a second feature fusion module, a first feature weighting module, a third feature fusion module, and a second feature weighting module.

[0091] The first feature weighting module receives the pyramid features output by the second feature fusion module, performs weighted processing on them, and inputs the processed pyramid features into the third feature fusion module for fusion. The third feature fusion module can output the fused pyramid features as the input of the second feature weighting module for weighted processing again.

[0092] The second feature weighting module weights the high-level feature maps and the bottom-level feature maps in the pyramid feature respectively through the channel attention mechanism, and then fuses the weighted feature maps to enhance the feature representation of the model. Specifically, the second feature weighting module can weight the fourth feature map of the top layer, and then weight the fourth feature map of the bottom layer, and then fuse the weighted top-level features with the bottom-level features as the final detection pyramid feature.

[0093] The detection pyramid features are detected by the target detection module, and the detection results are output. The target detection module can be a yolov8 model, specifically a detection head (yolo head) of yolov8. The detection result is used to indicate the object to be detected included in the input image. First, the yolov8 model can be trained first, and the detection pyramid features are extracted from the sample image using the above steps during training, and then the detection pyramid features of the sample image are predicted using the yolov8 model, and the area with the maximum probability of the object to be detected in the sample image is framed, and then compared with the real frame where the object to be detected is located, the loss function is calculated, and then the gradient is calculated, and each parameter in the network model is updated by gradient back propagation, and then repeated, and the network model is continuously optimized so that the loss value continues to decrease, and finally the network model with the minimum loss value and the best effect is obtained. The network can divide the input image into n*n grids, and each grid point presets three anchor boxes of different sizes. If the center of the object is in the grid, the grid is responsible for the object, and each grid predicts three bounding boxes, and each bounding box contains 5 parameters, which are x coordinate, y coordinate, width, height and center point confidence. The network iteratively calculates the loss value through back propagation, continuously adjusts the anchor box, and then uses the trained network model to predict the detection pyramid features of the input image to obtain the final detection result.

[0094] It is understandable that the above-mentioned target detection module can be used as a part of the feature extraction model, so that the feature extraction model can directly output the detection result.

[0095] Furthermore, this embodiment also provides a target detection device, which can be used to perform the above target detection method. Figure 2As shown, the target detection device 200 may specifically include: an image acquisition module 201, used to acquire an input image to be detected; an image input module 202, used to input the input image into a feature extraction model to perform feature extraction on the input image, wherein the feature extraction model includes a first feature extraction module 203, a first feature fusion module 204 and a first feature weighting module 205; the first feature extraction module 203 extracts pyramid features from the input image, wherein the pyramid features include first feature maps of multiple levels; the first feature fusion module 204 fuses the first feature maps of each level in the pyramid features to obtain a second feature map of each level after fusion; the first feature weighting module 205 weights the second feature map of each level to obtain a detection pyramid feature; and a detection output module 206, used to perform target detection on the detection pyramid feature to obtain a detection result.

[0096] In an exemplary embodiment, the first feature fusion module 204 is specifically used to fuse the first feature map of the target layer in the pyramid feature with the first feature map of the adjacent upper layer of the target layer to obtain the second feature map of the target layer; wherein the target layer is the level in the pyramid feature except the top layer; the second feature map of the top layer of the pyramid feature is the same as the first feature map.

[0097] In an exemplary embodiment, the feature extraction model also includes a second feature fusion module, which fuses the second feature map of the target layer in the pyramid feature with the second feature maps of each upper layer above the target layer to obtain a third feature map of the target layer; wherein the target layer is the level other than the top layer in the pyramid feature; the third feature map of the top layer of the pyramid feature is the same as the second feature map; and the first feature weighting module weights the third feature maps of each level to obtain a detection pyramid feature.

[0098] In an exemplary embodiment, the feature extraction model also includes a third feature fusion module, and the levels of the pyramid feature are top layer, middle layer, and bottom layer from top to bottom. The third feature fusion module fuses the third feature map of the top layer in the pyramid feature with the third feature map of the bottom layer to obtain a fourth feature map of the bottom layer; and fuses the third feature map of the top layer in the pyramid feature with the third feature map of the middle layer to obtain a fourth feature map of the middle layer; the fourth feature map of the top layer in the pyramid feature is the same as the third feature map; the first feature weighting module weights the fourth feature map of each level to obtain a detection pyramid feature.

[0099] In an exemplary embodiment, the detection output module 206 is specifically used for: the first feature weighting module performs maximum pooling on the second feature map of each level to obtain a first pooled feature map, and performs average pooling on the second feature map of each level to obtain a second pooled feature map; the first feature weighting module combines the first pooled feature map of each level with the second pooled feature map to obtain a feature map to be screened at each level; the first feature weighting module determines the weight of the feature map to be screened through an activation function, and weights the feature map to be screened based on the weight to obtain a detection pyramid feature.

[0100] In an exemplary embodiment, the first feature fusion module 204 is specifically used to: upsample the first feature map of the upper layer of the target layer in the pyramid feature to obtain the first feature map to be fused of the upper layer, and the first feature map to be fused has the same size as the first feature map of the target layer; fuse the first feature map to be fused with the first feature map of the target layer to obtain the second feature map of the target layer; the target layer is a layer other than the uppermost layer in the pyramid feature.

[0101] In an exemplary embodiment, a second feature weighting module is further included, which is used to weight the fourth feature map of the top layer and the fourth feature map of the bottom layer, and fuse the weighted feature maps to obtain the detection pyramid feature.

[0102] The specific details of each module or unit in the above target detection device have been described in detail in the corresponding target detection method, so they will not be repeated here.

[0103] The embodiment of the present application also provides an electronic device, Figure 3 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. Figure 3 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0104] like Figure 3 As shown, electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In RAM 603, various programs and data required for system operation are also stored. CPU 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0105] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0106] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the embodiment of the present application are executed.

[0107] For example, when the computer program is executed by the central processing unit (CPU) 601, it can be executed as follows: obtaining an input image to be detected; inputting the input image into a feature extraction model to perform feature extraction on the input image, wherein the feature extraction model includes a first feature extraction module, a first feature fusion module, and a first feature weighting module; the first feature extraction module extracts pyramid features from the input image, wherein the pyramid features include first feature maps of multiple levels; the first feature fusion module fuses the first feature maps of each level in the pyramid features to obtain a second feature map of each level after fusion; the first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature; and performing target detection on the detection pyramid feature to obtain a detection result.

[0108] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0109] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0111] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and the one or more programs include instructions, and when the instructions are executed by the electronic device, the electronic device implements the method described in the above embodiment.

[0112] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0113] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A target detection method, characterized in that: include: Get the input image to be detected; Inputting the input image into a feature extraction model to extract features from the input image, wherein the feature extraction model includes a first feature extraction module, a first feature fusion module, and a first feature weighting module; The first feature extraction module extracts a pyramid feature from the input image, wherein the pyramid feature includes a first feature map of multiple levels; The first feature fusion module fuses the first feature maps of each level in the pyramid feature to obtain a fused second feature map of each level; The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature; Performing target detection on the detection pyramid features to obtain a detection result.

2. The target detection method according to claim 1, characterized in that: The step of fusing the first feature maps of each level in the pyramid features to obtain the fused second feature maps of each level includes: The first feature map of the target layer in the pyramid feature is merged with the first feature map of the adjacent upper layer of the target layer to obtain the second feature map of the target layer; i The target layer is a layer other than the top layer in the pyramid feature; the second feature map of the top layer of the pyramid feature is the same as the first feature map.

3. The target detection method according to claim 2, characterized in that: The feature extraction model also includes a second feature fusion module, The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature, including: The second feature fusion module fuses the second feature map of the target layer in the pyramid feature with the second feature maps of each upper layer above the target layer to obtain a third feature map of the target layer; The target layer is a layer other than the top layer in the pyramid feature; the third feature map of the top layer of the pyramid feature is the same as the second feature map; The first feature weighting module weights the third feature maps of each level to obtain detection pyramid features.

4. The target detection method according to claim 3, characterized in that: The feature extraction model also includes a third feature fusion module, and the levels of the pyramid features are top layer, middle layer, and bottom layer from top to bottom. The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature, including: The third feature fusion module fuses the third feature map of the top layer with the third feature map of the bottom layer in the pyramid feature to obtain a fourth feature map of the bottom layer; and The third feature map of the top layer and the third feature map of the middle layer in the pyramid feature are merged to obtain a fourth feature map of the middle layer; The fourth feature map of the top layer in the pyramid feature is the same as the third feature map; The first feature weighting module weights the fourth feature map of each level to obtain a detection pyramid feature.

5. The target detection method according to claim 1, characterized in that: The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature, including: The first feature weighting module performs maximum pooling on the second feature map of each level to obtain a first pooled feature map, and performs average pooling on the second feature map of each level to obtain a second pooled feature map; The first feature weighting module combines the first pooled feature map and the second pooled feature map of each level to obtain a feature map to be screened at each level; The first feature weighting module determines the weight of the feature map to be screened through an activation function, and weights the feature map to be screened based on the weight to obtain a detection pyramid feature.

6. The target detection method according to claim 1, characterized in that: The step of fusing the first feature maps of each level in the pyramid features to obtain the fused second feature maps of each level includes: Upsampling the first feature map of the upper layer of the target layer in the pyramid feature to obtain a first feature map to be fused in the upper layer, wherein the first feature map to be fused has the same size as the first feature map of the target layer; Fusing the first feature map to be fused with the first feature map of the target layer to obtain a second feature map of the target layer; The target layer is a layer other than the uppermost layer in the pyramid feature.

7. The target detection method according to claim 4, characterized in that: The feature extraction model also includes a second feature weighting module, which is used to weight the fourth feature map of the top layer and the fourth feature map of the bottom layer, and fuse the weighted feature maps to obtain the detection pyramid feature.

8. A target detection device, characterized in that: include: An image acquisition module, used to acquire an input image to be detected; An image input module, used for inputting the input image into a feature extraction model to perform feature extraction on the input image, wherein the feature extraction model includes a first feature extraction module, a first feature fusion module and a first feature weighting module; The first feature extraction module extracts a pyramid feature from the input image, wherein the pyramid feature includes a first feature map of multiple levels; The first feature fusion module fuses the first feature maps of each level in the pyramid feature to obtain a fused second feature map of each level; The first feature weighting module weights the second feature map of each level to obtain a detection pyramid feature; The detection output module is used to perform target detection on the detection pyramid features to obtain a detection result.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the target detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor and a memory, wherein one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the target detection method described in any one of claims 1 to 7.

Images