Object detection methods, training methods for object detection models, and electronic devices

Abstract

This invention relates to the field of target detection technology, specifically to a target detection method, a training method for a target detection model, and an electronic device. The detection method includes acquiring an image to be processed; inputting the image to be processed into a first feature extraction unit and a second feature extraction unit sequentially connected in a target detection model for feature extraction, obtaining a first feature and a second feature; extracting local and global features from the first feature, and fusing the extracted features to obtain a first fused feature; fusing the second feature with the first fused feature to obtain a second fused feature; and inputting the second fused feature into a detection module in the target detection model to determine the target in the image to be processed. By extracting local and global features separately during feature extraction, the target detection model can not only focus on overall features but also on local details, thereby accurately detecting small targets in the image to be processed.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to target detection methods, target detection model training methods, and electronic devices. Background Technology

[0002] With the development of artificial intelligence technology, an increasing number of object detection algorithms have emerged. In the past two years, anchor-free algorithms have made remarkable progress. These algorithms avoid the pre-setting of anchor boxes, reducing a large amount of related computation, resulting in significantly improved memory usage and speed. This has led to more widespread application of object detection algorithms.

[0003] There is an increasing number of cameras being installed at high altitudes in squares, overpasses, and highways, leading to a rise in target detection tasks in high-resolution scenes. However, in these scenarios, targets are relatively small, and many algorithms compress small targets, weakening their feature information. This results in lower accuracy for small target detection in ultra-high resolution scenes, and consequently, lower accuracy for target detection in ultra-high resolution scenes. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a target detection method, a target detection model training method, and an electronic device to solve the problem of low accuracy in target detection in ultra-high resolution scenes.

[0005] According to a first aspect, embodiments of the present invention provide a target detection method, comprising:

[0006] Obtain the image to be processed;

[0007] The image to be processed is input into the first feature extraction unit and the second feature extraction unit connected sequentially in the target detection model to extract features, thereby obtaining the first feature and the second feature.

[0008] Local and global features are extracted from the first feature, and the extracted features are fused to obtain the first fused feature;

[0009] The second feature is fused with the first fusion feature to obtain the second fusion feature;

[0010] The second fused feature is input into the detection module of the target detection model to determine the target in the image to be processed.

[0011] The target detection method provided in this embodiment of the invention extracts local features and global features respectively during feature extraction, so that the target detection model can not only pay attention to the overall features but also to the local details, thereby accurately detecting small targets in the image to be processed.

[0012] In conjunction with the first aspect, in the first embodiment of the first aspect, the step of extracting local and global features from the first feature and fusing the extracted features to obtain a first fused feature includes:

[0013] The first feature is convolved to determine the local feature;

[0014] The first feature is subjected to global pooling and convolution processing to determine the global feature;

[0015] The local features are added to the global features to determine the first fused feature.

[0016] The target detection method provided in this invention integrates global features through global pooling, extracts global information through convolution, and adds local features to global features to add an attention mechanism to the target detection model, thereby improving the accuracy of small target detection.

[0017] In conjunction with the first aspect, in the second embodiment of the first aspect, the detection module includes a first feature extraction module, which includes a third feature extraction unit and a fourth feature extraction unit connected in sequence. The step of inputting the second fused features into the detection module of the target detection model to determine the target in the image to be processed includes:

[0018] The second fused feature is input into the first feature extraction module to obtain the first-level feature output by the third feature extraction unit and the second-level feature output by the fourth feature extraction unit;

[0019] The first-level features and the second-level features are fused together and target detection is performed to determine the first target in the image to be processed.

[0020] The target detection method provided in this embodiment of the invention utilizes a third feature extraction unit and a fourth feature extraction unit connected in sequence to achieve the fusion of deep and shallow features, thereby improving the accuracy of the detected first target.

[0021] In conjunction with the second embodiment of the first aspect, in the third embodiment of the first aspect, the step of fusing the first-level features and the second-level features and performing target detection to determine the first target in the image to be processed includes:

[0022] The first-level features are subjected to dilated convolution to obtain the first feature.

[0023] The second-level features are deconvolutionally processed to obtain the second feature;

[0024] The first feature and the second feature are fused together and target detection is performed to determine the first target.

[0025] The target detection method provided in this embodiment of the invention selects dilated convolution as the connection layer for the first-level features, i.e., shallow features. Dilated convolution has the characteristic of expanding the receptive field. Using dilated convolution in the shallow feature layer is beneficial to retain more information and improve the network's sensitivity to small targets. For the second-level features, i.e., deep network, deconvolution is used to adjust the feature map size to the size of the shallow features. After the deep and shallow features are fused, the feature information is enriched, and the accuracy of the first target is improved.

[0026] In conjunction with the second embodiment of the first aspect, in the fourth embodiment of the first aspect, the detection module further includes a second feature extraction module connected to the first feature extraction module. The second feature extraction module includes a fifth feature extraction unit and a sixth feature extraction unit connected in sequence. The step of inputting the second fused features into the detection module of the target detection model to determine the target in the image to be processed further includes:

[0027] The second-level features are input into the second feature extraction module to obtain the third-level features output by the fifth feature extraction unit and the fourth-level features output by the sixth feature extraction unit.

[0028] The third-level features are fused with the fourth-level features and target detection is performed to determine the second target in the image to be processed.

[0029] The target detection method provided in this embodiment of the invention has a first feature extraction module corresponding to the first target, i.e. the detection of small targets. Its fused feature retains more information, so this module is mainly responsible for the detection of small targets. The second feature extraction module corresponds to the second target, i.e. the detection of large targets. It uses two feature extraction modules connected in sequence. Its fused feature layer loses small target features due to multiple compressions, so this module is mainly responsible for the detection of large targets.

[0030] In conjunction with the fourth embodiment of the first aspect, in the fifth embodiment of the first aspect, the detection module further includes a seventh feature extraction unit connected between the first feature extraction module and the second feature extraction module. The step of fusing the third-level features and the fourth-level features and performing target detection to determine the second target in the image to be processed includes:

[0031] The third-level feature is subjected to dilated convolution to obtain the third feature;

[0032] The fourth-level feature is deconvolved to obtain the fourth feature;

[0033] The third feature and the fourth feature are fused to obtain the second fused feature;

[0034] The fifth feature is obtained by performing dilated convolution based on the second fusion feature.

[0035] The fifth-level features output by the seventh feature extraction unit are deconvolved to obtain the sixth feature;

[0036] The fifth feature and the sixth feature are fused together and target detection is performed to determine the second target.

[0037] The target detection method provided in this invention performs dilated convolution or deconvolution processing on features at different levels, which not only helps to retain a wider range of information but also enriches the feature information, further improving the accuracy of the second target detection.

[0038] In conjunction with the third or fifth embodiment of the first aspect, in the sixth embodiment of the first aspect, the step of fusing the first feature and the second feature and performing target detection to determine the first target includes:

[0039] The first feature and the second feature are fused together, and the fused feature is then convolved to obtain the third fused feature.

[0040] Target detection is performed on the third fusion feature to determine the first target;

[0041] or,

[0042] The step of fusing the fifth feature and the sixth feature and performing target detection to determine the second target includes:

[0043] The fifth feature is fused with the sixth feature, and the fused feature is then convolved to obtain the fourth fused feature.

[0044] The fourth fusion feature is used for target detection to determine the second target.

[0045] The target detection method provided in this embodiment of the invention performs convolution processing after feature fusion, which can reduce the aliasing effect caused by image fusion.

[0046] According to a second aspect, embodiments of the present invention provide a method for training an object detection model, comprising:

[0047] Obtain sample images and their labels;

[0048] The sample image is input into the first feature extraction unit and the second feature extraction unit connected sequentially in the preset detection model to extract features, thereby obtaining the first feature and the second feature.

[0049] Local and global features are extracted from the first feature, and the extracted features are fused to obtain the first fused feature;

[0050] The second feature is fused with the first fusion feature to obtain the second fusion feature;

[0051] The second fused feature is input into the detection module of the preset detection model to determine the predicted target in the sample image;

[0052] The parameters of the preset detection model are updated based on the difference between the predicted target and the label to determine the target detection model.

[0053] According to a third aspect, embodiments of the present invention provide an electronic device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the target detection method described in the first aspect or any embodiment of the first aspect.

[0054] According to a fourth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer instructions for causing the computer to perform the target detection method described in the first aspect or any embodiment of the first aspect.

[0055] It should be noted that the beneficial effects of the target detection model training method, electronic device, and computer-readable storage medium provided in the embodiments of the present invention can be found in the description of the corresponding beneficial effects of the target detection method above, and will not be repeated here. Attached Figure Description

[0056] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 This is a flowchart of a target detection method according to an embodiment of the present invention;

[0058] Figure 2 This is a flowchart of a target detection method according to an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the structure for extracting local and global features according to an embodiment of the present invention;

[0060] Figure 4 This is a flowchart of a target detection method according to an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the target detection model according to an embodiment of the present invention;

[0062] Figure 6 This is a flowchart of a training method for an object detection model according to an embodiment of the present invention;

[0063] Figure 7 This is a structural block diagram of a target detection device according to an embodiment of the present invention;

[0064] Figure 8 This is a schematic diagram of the structure of a training device for a target detection model according to an embodiment of the present invention;

[0065] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The target detection method provided in this invention can be applied to scenarios such as elevated roads and highways to detect targets of different scales in these scenarios. For example, a camera can be mounted on the roof of a building near an elevated road, facing the elevated road or highway, to detect different types of motor vehicles in these scenarios, enabling the detection of targets such as cars, vans, buses, trucks, and tricycles at ultra-high resolution.

[0068] In this embodiment of the invention, small targets include distant targets and small targets in the foreground of an image. A small target can be defined as one whose area in the image is less than a preset threshold, and all targets in the image other than small targets can be referred to as large targets.

[0069] According to embodiments of the present invention, an object detection method or an embodiment of an object detection model training method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0070] This embodiment provides a target detection method that can be used in electronic devices such as computers, servers, mobile phones, and tablets. Figure 1 This is a flowchart of a target detection method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0071] S11, Obtain the image to be processed.

[0072] As mentioned above, the image to be processed can be an image acquisition device set up in the corresponding traffic scene, or it can be an image captured by a mobile terminal, etc. There are no restrictions on the source of the image to be processed; it depends on the application scenario of the target detection method.

[0073] S12, the image to be processed is input into the first feature extraction unit and the second feature extraction unit connected sequentially in the target detection model to extract features, and the first feature and the second feature are obtained.

[0074] The object detection model is trained using sample images, and the specific training process will be described below. The backbone network of this object detection model is used for feature extraction from the image to be processed; that is, the object detection model includes a first feature extraction unit and a second feature extraction unit connected in sequence.

[0075] It should be noted that the structures of the first feature extraction unit and the second feature extraction unit can be the same or different. For example, both the first feature extraction unit and the second feature extraction unit can be residual units. Both the first feature extraction unit and the second feature extraction unit include convolutional layers, or may also include other layers adapted to convolutional layers; no limitations are placed on them here.

[0076] The first feature extraction unit and the second feature extraction unit are connected sequentially; that is, the first feature extraction unit is positioned closer to the input layer of the object detection model than the second feature extraction unit. Specifically, after the image to be processed is input into the object detection model, it first passes through the first feature extraction unit and then through the second feature extraction unit. The output of the first feature extraction unit is called the first feature, and the output of the second feature extraction unit is called the second feature. Based on the order of the first and second feature extraction units in the object detection module, the first feature can be called the shallow feature, and the second feature can be called the deep feature.

[0077] S13, extract local and global features from the first feature, and fuse the extracted features to obtain the first fused feature.

[0078] This object detection model also includes local and global branches, which form the attention mechanism module. The local branch extracts local features from the first feature, while the global branch extracts global features from the first feature. Local feature extraction can be achieved by directly convolving the first feature; global feature extraction can be achieved by first performing global convolution on the first feature, followed by convolution, and so on. The specific structural details of the local and global branches are not limited here, as long as they can extract both local and global features. The placement of the local and global branches within the object detection model is also not limited, as long as they are placed before feature fusion.

[0079] After extracting local and global features, the two are fused to obtain the first fused feature. During the fusion process, corresponding parts of the local and global features are added together, or the two are directly concatenated.

[0080] The specifics of this step will be described in detail below.

[0081] S14, the second feature is fused with the first fusion feature to obtain the second fusion feature.

[0082] After processing the first feature using S13, a first fused feature is obtained. This first fused feature is then fused again with the second feature output by the second feature extraction unit to obtain a second fused feature. Therefore, it can be seen that the first feature is processed in parallel by three branches: the first branch is the second feature extraction unit, the second branch is a local branch, and the third branch is a global branch. The results of the second and third branches are first fused, and then fused with the result of the first branch to obtain the second fused feature.

[0083] The fusion of the second feature and the first fusion feature is obtained by multiplying the second feature and the first fusion feature, which yields the second fusion feature.

[0084] S15, input the second fused feature into the detection module in the target detection model to determine the target in the image to be processed.

[0085] The object detection model also includes a detection module, which performs object detection based on previously extracted features to obtain the objects in the image to be processed. These objects include, but are not limited to, the small objects mentioned above; they can also include large objects. The specific type of object to be detected in the image depends on the actual scene requirements. If scene A requires small objects, the detection module will output the small objects; if scene B requires both small and large objects, the detection module will output both.

[0086] For a detection module that can detect both small and large targets, the module includes two detection branches: one for detecting small targets and the other for detecting large targets.

[0087] In addition to the detection branch, the detection module may also include other feature extraction modules for feature extraction or feature fusion of the second fused features, etc. The specific processing is set according to actual needs.

[0088] The specifics of this step will be described in detail below.

[0089] The target detection method provided in this embodiment extracts local and global features during feature extraction, enabling the target detection model to not only focus on overall features but also on local details, thereby accurately detecting small targets in the image to be processed.

[0090] This embodiment provides a target detection method that can be used in electronic devices such as computers, servers, mobile phones, and tablets. Figure 2 This is a flowchart of a target detection method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0091] S21, Obtain the image to be processed.

[0092] Please see details Figure 1 S11 of the illustrated embodiment will not be described again here.

[0093] S22, the image to be processed is input into the first feature extraction unit and the second feature extraction unit connected sequentially in the target detection model to extract features, and the first feature and the second feature are obtained.

[0094] Please see details Figure 1 S12 of the illustrated embodiment will not be described again here.

[0095] S23, extract local and global features from the first feature respectively, and fuse the extracted features to obtain the first fused feature.

[0096] Local and global branches are established in the shallow layers of the object detection model network to extract local and global features, respectively. Local branches focus more on local image details, while global branches focus more on overall image information. The combination of these two branches enhances the network's attention to these two aspects, allowing it to focus on the relevant parts. Specifically, S23 includes:

[0097] S231, perform convolution processing on the first feature to determine local features.

[0098] Local features are obtained by convolutional processing of the first feature. It should be noted that this convolutional processing is not limited to convolution alone; it can also include other adaptive processing, such as activation. Taking ResNet18 as the backbone of this object detection model as an example, ... Figure 3 as well as Figure 5 As shown, an attention mechanism structure is added after the feature layer of the residual network res2a in the backbone network. Figure 5 The specific structure of the attention mechanism described in the text is as follows: Figure 3 As shown.

[0099] Specifically, such as Figure 5 As shown, res2a corresponds to the first feature extraction unit mentioned above, and res2b is... Figure 3 The `Residual` in this context corresponds to the second feature extraction unit described above. The first feature is input into the second feature extraction unit, the local branch, and the global branch, respectively. For example, ... Figure 3 As shown, the middle branch is a local branch, which is used to perform feature extraction and activation on the first feature to obtain local features. That is, the local branch uses two 1x1 convolutional blocks Conv 1x1 with a stride of 1 to extract local information.

[0100] S232, perform global pooling and convolution on the first feature to determine the global feature.

[0101] Global branch is Figure 3 The branch far from Residual first performs global pooling on the first feature and then performs convolution and other processing. First, a global average pooling layer is introduced to integrate global information, and then two 1x1 convolutional blocks Conv 1x1 with stride of 1 are used to extract global information and obtain global features.

[0102] S233, add the local features and the global features to determine the first fused feature.

[0103] The local features are added to the global features, and then activated by an activation layer, such as a sigmoid layer, to obtain the first fused feature.

[0104] S24, the second feature is fused with the first fusion feature to obtain the second fusion feature.

[0105] After obtaining the first fused feature, it is multiplied with the second feature output by the res2b layer to obtain the second fused feature, thereby adding local and global attention to the network.

[0106] Please see details Figure 1 S14 of the illustrated embodiment will not be described again here.

[0107] S25, input the second fused feature into the detection module in the target detection model to determine the target in the image to be processed.

[0108] Please see details Figure 1 S15 of the illustrated embodiment will not be described again here.

[0109] The target detection method provided in this embodiment integrates global features through global pooling, extracts global information through convolution, and adds local features to global features to add an attention mechanism to the target detection model, thereby improving the accuracy of small target detection.

[0110] This embodiment provides a target detection method that can be used in electronic devices such as computers, servers, mobile phones, and tablets. Figure 4 This is a flowchart of a target detection method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0111] S31, Obtain the image to be processed.

[0112] Please see details Figure 1 S11 of the illustrated embodiment will not be described again here.

[0113] S32, the image to be processed is input into the first feature extraction unit and the second feature extraction unit connected sequentially in the target detection model to extract features, and the first feature and the second feature are obtained.

[0114] Please see details Figure 1 S12 of the illustrated embodiment will not be described again here.

[0115] S33, Local and global features are extracted from the first feature, and the extracted features are fused to obtain the first fused feature.

[0116] Please see details Figure 2 S23 of the illustrated embodiment will not be described again here.

[0117] S34, the second feature is fused with the first fusion feature to obtain the second fusion feature.

[0118] Please see details Figure 1 S14 of the illustrated embodiment will not be described again here.

[0119] S35, input the second fused feature into the detection module in the target detection model to determine the target in the image to be processed.

[0120] The detection module includes a first feature extraction module and a second feature extraction module connected to the first feature extraction module. The first feature extraction module includes a third feature extraction unit and a fourth feature extraction unit connected in sequence. The second feature extraction module includes a fifth feature extraction unit and a sixth feature extraction unit connected in sequence.

[0121] Specifically, the feature extraction module includes at least one feature extraction unit. The detection module includes a first feature extraction module and a second feature extraction module connected sequentially. The first feature extraction module extracts shallow features, and the second feature extraction module extracts deep features. It should be noted that shallow and deep features are relative concepts. The first feature extraction module extracts shallow features relative to the second feature extraction module, but it extracts deep features relative to the first feature extraction unit. Based on the hierarchical relationship of feature extraction, from shallow to deep, the modules are: first feature extraction unit, second feature extraction unit, first feature extraction module, and second feature extraction module.

[0122] Furthermore, the first feature extraction module includes a third and a fourth feature extraction unit connected in sequence, and the second feature extraction module includes a fifth and a sixth feature extraction unit connected in sequence. Therefore, based on the hierarchical relationship of feature extraction, and following a progression from shallow to deep, these are respectively the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, the fourth feature extraction unit, the fifth feature extraction unit, and the sixth feature extraction unit.

[0123] It should be noted that the structure of each feature extraction unit can be the same or different; no restrictions are placed on this. For example, corresponding to Figure 5 res3a-res3b are referred to as the first feature extraction module, res3a as the third feature extraction unit, and res3b as the fourth feature extraction unit; res5a-res5b and res6 are referred to as the second feature extraction module, res5a-res5b as the fifth feature extraction unit, and res6 as the sixth feature extraction unit.

[0124] Specifically, S35 includes:

[0125] S351, the second fused feature is input into the first feature extraction module to obtain the first-level feature output by the third feature extraction unit and the second-level feature output by the fourth feature extraction unit.

[0126] like Figure 5 As shown, the output features of the third feature extraction unit res3a are referred to as the first-level features, and the output features of the fourth feature extraction unit res3b are referred to as the second-level features.

[0127] S352, fuse the first-level features with the second-level features and perform target detection to determine the first target in the image to be processed.

[0128] Since the first-level features represent shallow features and the second-level features represent deep features, fusing these two sets of features achieves the fusion of shallow and deep features. Based on the fused result, target detection is then performed to obtain the first target in the image to be processed. This first target can be defined as a small target. Because the first feature extraction module extracts shallow features compared to the second feature extraction module, shallow features retain more feature details. Using these features for small target detection has high accuracy.

[0129] It's important to note that the first-level and second-level features don't necessarily refer to adjacent layers; they can also be non-adjacent layers. Deep-shallow fusion mechanisms typically employ two or more consecutive layers. When using multiple layers, the deepest feature with the lowest resolution is first fused with the adjacent, larger, shallower layer, and the result is then fused with an even larger shallower feature. This step-by-step fusion of adjacent layers results in a feature image with less aliasing, better fusion performance, and richer features. There is no limit to the number of layers fused. Here, using ResNet18 as an example, two sets of feature layers are selected for fusion to obtain the prediction branch.

[0130] In some alternative implementations, S352 includes:

[0131] (1) Perform dilated convolution on the first-level features to obtain the first features.

[0132] (2) Perform deconvolution on the second-level features to obtain the second features.

[0133] (3) The first feature and the second feature are fused together and target detection is performed to determine the first target.

[0134] Regarding the fusion of features from two different levels, dilated convolution is introduced for shallow features, while deconvolution is introduced for deep features. Specifically, for the first level of features, i.e., shallow features, dilated convolution is used as the connecting layer. Dilated convolution has the characteristic of expanding the receptive field; using dilated convolution in the shallow feature layer helps retain a larger range of information, improving the network's sensitivity to small targets. For the second level of features, i.e., the deep network, deconvolution is used to adjust the feature map size to the size of the shallow features, thus fusing the deep and shallow features, enriching the feature information, and improving the accuracy of the identified first target.

[0135] like Figure 5 As shown, dilated convolution corresponds to module T2, and deconvolution corresponds to module P3_UP. Module T2 includes a dilated convolution layer (Dilated conv), a normalization layer (BN), and a scaling layer (Scale); module P3_UP includes a deconvolution layer (deconv), a normalization layer (BN), and a scaling layer (Scale).

[0136] To ensure consistency in explanation, the T2 module will be referred to as the Tx module, and the P3_UP module as the Px_UP module. Subsequent modules involving aliasing reduction will also be referred to as the Px module. This Px module includes convolutional layers (conv), normalization layers (BN), scaling layers (scale), and activation layers (ReLU). Of course, using ReLU as the activation layer is merely an example; other activation functions can also be used.

[0137] Specifically, the first-level features are processed by dilated convolution using the P3_UP module to obtain the first feature; the second-level features are processed by deconvolution using the T2 module to obtain the second feature. The first and second features are then fused by addition, and target detection is performed on this fusion to obtain the first target.

[0138] In some alternative implementations, step (3) of S352 above may include:

[0139] 3.1) The first feature and the second feature are fused together, and the fused feature is convolved to obtain the third fused feature.

[0140] 3.2) Perform target detection on the third fusion feature to determine the first target.

[0141] After feature fusion, the Px module is introduced for convolution processing, for example, Figure 5 The P3 module described above obtains the third fusion feature. Then, target detection is performed based on the third fusion feature to determine the first target.

[0142] The output for the first target can be its location information, such as target center point prediction, target width and height regression, and human keypoint prediction. The specific output method for the first target, or which features of the first target are output, is determined based on the actual application requirements.

[0143] S353, input the second-level features into the second feature extraction module to obtain the third-level features output by the fifth feature extraction unit and the fourth-level features output by the sixth feature extraction unit.

[0144] The second-level features are output by the fourth feature extraction unit in the first feature extraction module. The second-level features are input into the fifth feature extraction unit to obtain the third-level features. The third-level features are then processed by the sixth feature extraction unit to obtain the fourth-level features.

[0145] like Figure 5 As shown, the fifth feature extraction unit includes res5a-res5b connected sequentially, and the sixth feature extraction unit is res6. The fifth feature extraction unit is used to extract features from the second-level features to obtain the third-level features; the sixth feature extraction unit is used to extract features from the third-level features to obtain the fourth-level features.

[0146] S354, fuses the third-level features with the fourth-level features and performs target detection to determine the second target in the image to be processed.

[0147] The fusion method of the third-level features and the fourth-level features is similar to the fusion method of the first-level features and the second-level features described above. In this case, the third-level features are shallow features relative to the fourth-level features, and the fourth-level features are deep features relative to the third-level features.

[0148] In some alternative implementations, S354 includes:

[0149] (1) Perform dilated convolution on the third-level features to obtain the third features.

[0150] (2) Perform deconvolution on the fourth-level features to obtain the fourth feature.

[0151] (3) The third feature and the fourth feature are fused to obtain the second fused feature.

[0152] (4) Based on the second fusion feature, dilated convolution is performed to obtain the fifth feature.

[0153] (5) Perform deconvolution processing on the fifth-level features output by the seventh feature extraction unit to obtain the sixth feature.

[0154] (6) The fifth and sixth features are fused and the target is detected to determine the second target.

[0155] like Figure 5 The detection module further includes a seventh feature extraction unit, res4a-res4b, which extracts the fifth-level features and performs deconvolution processing (T4 module processing) on ​​these features to obtain the sixth feature. When fusing the third and fourth features, the sum is input into the P5 module for processing to obtain the second fused feature. The second fused feature is then subjected to dilated convolution processing (P5_UP module processing) to obtain the fifth feature.

[0156] The fifth and sixth features are fused by addition. The fused features are then processed by the P4 module and then used for target detection. The second target obtained here is the large target.

[0157] Applying dilated convolution or deconvolution to features at different levels not only helps retain a wider range of information but also enriches the feature information, further improving the accuracy of second target detection.

[0158] In some alternative embodiments, step (6) of S354 above may include:

[0159] 6.1) The fifth feature and the sixth feature are fused together, and the fused feature is convolved to obtain the fourth fused feature.

[0160] 6.2) Perform target detection on the fourth fusion feature to determine the second target.

[0161] The convolutional processing here is the same as that of the P4 module mentioned above, which yields the fourth fused feature. This fourth fused feature is then used for target detection to obtain the second target. Performing convolutional processing after feature fusion reduces the aliasing effect caused by image fusion.

[0162] Alternatively, after obtaining the small target and the large target through the two detection branches respectively, duplicate targets are removed, thus deleting redundant targets. For example, using the NMS principle, redundant targets with an IOU greater than 0.35 are deleted to obtain the final detected targets.

[0163] The target detection method provided in this embodiment utilizes a third and fourth feature extraction unit connected in sequence to fuse shallow and deep features, thereby improving the accuracy of the detected first target. The first feature extraction module corresponds to the first target, i.e., the detection of small targets. Its fused features retain more information, so this module is mainly responsible for the detection of small targets. The second feature extraction module corresponds to the second target, i.e., the detection of large targets. It uses two feature extraction modules connected in sequence. Due to multiple compressions in its fused feature layer, small target features are lost, so this module is mainly responsible for the detection of large targets.

[0164] As a specific application example, such as Figure 5 As shown, this object detection method introduces an attention mechanism structure after res2a, which consists of two parts: a local branch and a global branch. The local branch uses two 1x1 convolutional blocks with a stride of 1 to extract local information; the global branch first introduces a global average pooling layer to integrate global information, and then uses two 1x1 convolutional blocks with a stride of 1 to extract global information. The convolutional blocks are composed of convolutional layers combined with Batch Normalization (BN) layers and HardSwish activation functions. Then, the local and global features are added together, followed by a sigmoid layer for activation, and finally multiplied with the res2b feature layer. This allows the feature layer to focus on both large global object information and small local object feature information.

[0165] After the attention mechanism structure, feature fusion is performed on res2b and res3b. First, the shallower res2b layer is connected to module T2, such as... Figure 5 The connection module shown consists of dilated convolutions, batch normalization (BN) layers, and scale layers. Dilated convolutions help retain more information over a wider range, improving the network's sensitivity to small targets. The deeper res3b layers are then subjected to an up-applying operation, P2_UP, specifically as follows... Figure 5 The algorithm consists of deconvolution, batch normalization (BN) layers, and scale layers. After resizing the feature map to match the size of the shallow feature layers, the shallow and deep feature layers are added together to complete feature fusion. This fusion is then connected to the P3 module to obtain a prediction branch rich in feature information. Since this prediction branch is obtained from two shallower feature layers, res2b and res3b, it contains more information and is therefore primarily responsible for detecting small targets.

[0166] The same method is used for feature fusion in layers res4b, res5b, and res6, which are then connected to the P4 module to obtain the fused feature layer. Therefore, these three layers are quite deep, and after multiple compressions, most of the features of small targets are lost. Thus, this branch is mainly responsible for predicting large targets.

[0167] This embodiment provides a target detection method that can be used in electronic devices such as computers, servers, mobile phones, and tablets. Figure 6 This is a flowchart of a target detection method according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps:

[0168] S41, Obtain the sample image and its label.

[0169] S42, input the sample image into the first feature extraction unit and the second feature extraction unit connected sequentially in the preset detection model to extract features, and obtain the first feature and the second feature.

[0170] S43, extract local and global features from the first feature, and fuse the extracted features to obtain the first fused feature.

[0171] S44, the second feature is fused with the first fused feature to obtain the second fused feature.

[0172] S45, input the second fusion feature into the detection module in the preset detection model to determine the predicted target in the sample image.

[0173] S46, Update the parameters of the preset detection model based on the difference between the predicted target and the label to determine the target detection model.

[0174] The processing of S41-S45 is similar to the target detection method described above. The difference is that the input in the target detection method is the image to be detected, while the input here is the sample image.

[0175] The sample image is input into a pre-defined detection model for target prediction to obtain the predicted target. Loss is calculated based on the prediction module and the label, thereby updating the parameters of the pre-defined detection model. After multiple iterations, the target detection model is determined.

[0176] The labels include the center point and width / height of the target; correspondingly, the predicted target includes the predicted center point and width / height of the target. For the detection of the target center point and the regression of width / height, since there are many types of targets in real-world applications, it is difficult to achieve a balance in the number of targets among them. Therefore, the focal loss method is used for the center point prediction branch, with weight adjustments, and the L1 loss algorithm is used for the position offset branch and the target width / height regression branch.

[0177] The attention mechanism in the target detection model of this method enables the network to pay attention to both local and global information at the same time. Furthermore, the feature fusion design enriches the information of the prediction feature layer, and the use of multi-branch prediction intuitively extracts shallow fusion feature information, making the network more accurate in detecting small targets and achieving efficient detection of small targets in ultra-high resolution scenes.

[0178] As a specific application scenario in this embodiment, it is applied to wide-open scenes such as squares to detect targets of different scales in outdoor settings. In this example, the method is applied to a gun-ball linkage camera system, with a camera mounted on the roof of a factory building to detect 12 types of targets within the factory grounds, including faces, heads and shoulders, whole people, riders, non-motorized vehicles, cars, vans, and buses. In this scenario, a sample set is used to train a preset detection model. Using input images of a fixed resolution size of M×N, the preset detection model is trained, resulting in a model capable of detecting targets of different scales, especially small targets, in very large scenes.

[0179] As another specific application scenario of this embodiment, it is applied to scenarios such as elevated roads and highways to detect targets of different scales in these scenarios. In this example, a camera is mounted on the roof of a building near the elevated road, facing the elevated road, to detect different types of motor vehicles in the elevated road and highway scenarios. In this scenario, a sample set is used to train a preset detection model. Input images of a fixed resolution size of M×N are fed into the preset detection model for training, resulting in a model that can detect five types of targets—cars, vans, buses, trucks, and tricycles—at ultra-high resolution.

[0180] This embodiment also provides a target detection device or a target detection model training device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0181] This embodiment provides a target detection device, such as Figure 7 As shown, it includes:

[0182] The first acquisition module 51 is used to acquire the image to be processed;

[0183] The first input module 52 is used to input the image to be processed into the first feature extraction unit and the second feature extraction unit connected sequentially in the target detection model for feature extraction, so as to obtain the first feature and the second feature.

[0184] The first extraction module 53 is used to extract local features and global features from the first feature, and to fuse the extracted features to obtain the first fused feature;

[0185] The first fusion module 54 is used to fuse the second feature with the first fusion feature to obtain the second fusion feature;

[0186] The detection module 55 is used to input the second fused features into the detection module of the target detection model to determine the target in the image to be processed.

[0187] This embodiment also provides a training device for an object detection model, such as... Figure 8 As shown, it includes:

[0188] The second acquisition module 61 is used to acquire sample images and their labels;

[0189] The second input module 62 is used to input the sample image into the first feature extraction unit and the second feature extraction unit connected sequentially in the preset detection model for feature extraction, so as to obtain the first feature and the second feature.

[0190] The second extraction module 63 is used to extract local features and global features from the first feature, and to fuse the extracted features to obtain the first fused feature;

[0191] The second fusion module 64 is used to fuse the second feature with the first fusion feature to obtain the second fusion feature;

[0192] Prediction module 65 is used to input the second fused feature into the detection module in the preset detection model to determine the predicted target in the sample image;

[0193] The update module 66 is used to update the parameters of the preset detection model based on the difference between the predicted target and the label, so as to determine the target detection model.

[0194] In this embodiment, the target detection device or the target detection model training device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0195] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0196] This invention also provides an electronic device having the above-described features. Figure 7 The target detection device shown, or Figure 8 The training device for the target detection model shown.

[0197] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of the present invention, such as... Figure 9As shown, the electronic device may include: at least one processor 71, such as a CPU (Central Processing Unit), at least one communication interface 73, memory 74, and at least one communication bus 72. The communication bus 72 is used to enable communication between these components. The communication interface 73 may include a display screen or a keyboard; optionally, the communication interface 73 may also include a standard wired interface or a wireless interface. The memory 74 may be high-speed RAM (Random Access Memory) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 74 may also be at least one storage device located remotely from the aforementioned processor 71. The processor 71 may be combined with... Figure 7 The apparatus described in 8 has an application program stored in memory 74, and the processor 71 calls the program code stored in memory 74 to perform any of the above method steps.

[0198] The communication bus 72 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 72 can be divided into an address bus, a data bus, and a control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0199] The memory 74 may include volatile memory, such as random-access memory (RAM); the memory may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 74 may also include a combination of the above types of memory.

[0200] The processor 71 can be a central processing unit (CPU), a network processor (NP), or a combination of CPU and NP.

[0201] The processor 71 may further include a hardware chip. This hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0202] Optionally, the memory 74 is also used to store program instructions. The processor 71 can invoke the program instructions to implement the target detection method or the target detection model training method as shown in any embodiment of this application.

[0203] This invention also provides a non-transitory computer storage medium storing computer-executable instructions that can execute the target detection method or the target detection model training method in any of the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0204] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A target detection method, characterized in that, Applications include small target detection in ultra-high resolution scenes, including: Obtain the image to be processed; The image to be processed is input into the first feature extraction unit and the second feature extraction unit connected sequentially in the target detection model to extract features, thereby obtaining the first feature and the second feature. Local and global features are extracted from the first feature, and the extracted features are fused to obtain the first fused feature; The second feature is fused with the first fusion feature to obtain the second fusion feature; The second fused feature is input into the detection module of the target detection model to determine the target in the image to be processed; wherein, the first-level feature obtained by inputting the second fused feature into the detection module is subjected to dilated convolution processing to obtain a first feature, and the second-level feature obtained by inputting the second fused feature into the detection module is subjected to deconvolution processing to obtain a second feature; the first feature and the second feature obtained by the processing are fused and target detection is performed to determine a first target, which is a small target; In this context, the first-level features represent shallow features, and the second-level features represent deep features.

2. The method according to claim 1, characterized in that, The step of extracting local and global features from the first feature and fusing the extracted features to obtain a first fused feature includes: The first feature is convolved to determine the local feature; The first feature is subjected to global pooling and convolution processing to determine the global feature; The local features are added to the global features to determine the first fused feature.

3. The method according to claim 1, characterized in that, The detection module includes a first feature extraction module, which includes a third feature extraction unit and a fourth feature extraction unit connected in sequence. The step of inputting the second fused features into the detection module of the target detection model to determine the target in the image to be processed includes: The second fused feature is input into the first feature extraction module to obtain the first-level feature output by the third feature extraction unit and the second-level feature output by the fourth feature extraction unit; The first-level features and the second-level features are fused together and target detection is performed to determine the first target in the image to be processed.

4. The method according to claim 3, characterized in that, The detection module further includes a second feature extraction module connected to the first feature extraction module. The second feature extraction module includes a fifth feature extraction unit and a sixth feature extraction unit connected in sequence. The step of inputting the second fused features into the detection module of the target detection model to determine the target in the image to be processed further includes: The second-level feature is input into the second feature extraction module to obtain the third-level feature output by the fifth feature extraction unit and the fourth-level feature output by the sixth feature extraction unit. The third-level feature is a shallow feature relative to the fourth-level feature, and the fourth-level feature is a deep feature relative to the third-level feature. The third-level features are fused with the fourth-level features and target detection is performed to determine the second target in the image to be processed.

5. The method according to claim 4, characterized in that, The detection module further includes a seventh feature extraction unit connected between the first feature extraction module and the second feature extraction module. The step of fusing the third-level features and the fourth-level features and performing target detection to determine the second target in the image to be processed includes: The third-level feature is subjected to dilated convolution to obtain the third feature; The fourth-level feature is deconvolved to obtain the fourth feature; The third feature and the fourth feature are fused to obtain the second fused feature; The fifth feature is obtained by performing dilated convolution based on the second fusion feature. The fifth-level features output by the seventh feature extraction unit are deconvolved to obtain the sixth feature; The fifth feature and the sixth feature are fused together and target detection is performed to determine the second target.

6. The method according to claim 1 or 5, characterized in that, The step of fusing the first feature and the second feature and performing target detection to determine the first target includes: The first feature and the second feature are fused together, and the fused feature is then convolved to obtain the third fused feature. Target detection is performed on the third fusion feature to determine the first target; or, The fifth and sixth features are fused and used for target detection to determine the second target, including: The fifth feature is fused with the sixth feature, and the fused feature is then convolved to obtain the fourth fused feature. The fourth fusion feature is used for target detection to determine the second target.

7. A training method for an object detection model, characterized in that, include: Obtain sample images and their labels; The sample image is input into the first feature extraction unit and the second feature extraction unit connected sequentially in the preset detection model to extract features, thereby obtaining the first feature and the second feature. Local and global features are extracted from the first feature, and the extracted features are fused to obtain the first fused feature; The second feature is fused with the first fusion feature to obtain the second fusion feature; The second fused feature is input into the detection module of the preset detection model to determine the predicted target in the sample image; wherein, the first-level feature obtained by inputting the second fused feature into the preset detection module is subjected to dilated convolution processing to obtain the first feature, and the second-level feature obtained by inputting the second fused feature into the preset detection module is subjected to deconvolution processing to obtain the second feature; the processed first feature and the second feature are fused and target detection is performed to determine the predicted target, wherein the predicted target is a small target; wherein, the first-level feature represents shallow features, and the second-level feature represents deep features; The parameters of the preset detection model are updated based on the difference between the predicted target and the label to determine the target detection model.

8. An electronic device, characterized in that, include: The device includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the target detection method of any one of claims 1-6, or to perform the training method of the target detection model of claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the target detection method of any one of claims 1-6, or to perform the training method of the target detection model of claim 7.

Images