Detection device for target detection and target detection method

Abstract

The invention relates to a detection device for target detection and a target detection method, and belongs to the technical field of artificial intelligence, the detection device for target detection comprises a trunk feature extraction module, a feature fusion module and a detection head module which are connected in sequence; the trunk feature extraction module is used for performing feature extraction on a target detection image to obtain a pyramid feature map, and the target detection image is a parking lot image; the feature fusion module is used for performing feature fusion on the pyramid feature map to obtain a feature fusion map, and the feature fusion module comprises a multi-scale feature fusion sub-module and a cross-scale feature enhancement sub-module; the detection head module is used for detecting the feature fusion image and then outputting a detection result, and a prediction frame loss function in the detection head module is an SIoU loss function. The target detection precision is improved by improving a network structure in the feature fusion module and a loss function in the detection head module.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a detection device and a target detection method for target detection. Background Art

[0002] The current target detection technology for on-street parking scenes is mainly divided into two categories: traditional machine learning and deep learning. Traditional methods rely on manual feature design and have poor generalization ability. Thanks to the development of hardware, deep learning has become mainstream, and its powerful feature learning ability is superior to traditional methods. Deep learning algorithms are divided into two types: dual-stage and single-stage. Although the dual-stage algorithm improves detection performance, it is limited by the anchor box and is difficult to adapt to complex and diverse target sizes and shapes, and the inference time is long. The single-stage algorithm considers different scales and shapes, is simple and fast, and meets the requirements of real-time detection. However, in small target detection and occlusion detection, there will be problems of insufficient accuracy leading to missed detection or false detection, and real-time processing may still be difficult in high-resolution images or complex scenes.

[0003] The YOLOv8 algorithm can improve the multi-scale target detection capability in complex scenes by strengthening the target contour and texture features. However, in the on-street parking scene, the high-position camera is fixed in position, resulting in obstacles such as roadside trees and electric poles blocking the field of view, forming difficult samples with only partial features. The lack of detailed features not only increases the risk of missed detection of small targets, but may also cause missed detection of targets with obvious features. It is more convenient and practical to use contextual information to assist in the detection of occluded targets. However, in complex parking scenes, contextual information is often insufficient. Therefore, it is necessary to make full use of the existing multi-level semantic information of occluded targets in the image for detection. In addition, during peak traffic hours, different types of vehicles and pedestrians appear densely in the picture, and windy and snowy weather will also reduce the image quality and cause roadside vegetation to appear in different postures. These factors make the on-street parking scene complex and changeable, resulting in inaccurate positioning of detection results.

[0004] Therefore, how to achieve high-precision target detection is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, it is necessary to provide a detection device for target detection to improve the target detection accuracy.

[0006] In order to solve the above problems, in a first aspect, the present invention provides a detection device for target detection, comprising: a backbone feature extraction module, a feature fusion module and a detection head module connected in sequence; The backbone feature extraction module is used to extract features from a target detection image to obtain a pyramid feature map, wherein the target detection image is a parking lot image; The feature fusion module is used to perform feature fusion on the pyramid feature map to obtain a feature fusion map, wherein the feature fusion module includes a multi-scale feature fusion submodule and a cross-scale feature enhancement submodule, the multi-scale feature fusion submodule is used to fuse first feature information of different scales in the pyramid feature map, and the cross-scale feature enhancement submodule is used to fuse second feature information of different scales in the pyramid feature map based on the first feature information to obtain the feature fusion map; The detection head module is used to output a detection result after detecting the feature fusion map, wherein the prediction box loss function in the detection head module is a SIoU loss function.

[0007] In a possible implementation manner, the first feature information includes scale information; The second feature information includes semantic information.

[0008] In a possible implementation, the multi-scale feature fusion submodule includes a multi-scale collaboration unit; The multi-scale collaborative unit is used to adjust the channel sizes of the feature maps of different scales in the pyramid feature map, and perform feature extraction on the feature maps after the channel sizes are adjusted.

[0009] In a possible implementation, the multi-scale collaborative unit includes a CBS convolution layer, a pooling layer, a splicing layer, an interpolation layer, and a C2f extraction layer, which are specifically used for: For the first scale feature map, the CBS convolution layer is used to adjust the channel size, and the pooling layer is used for downsampling so that the channel size of the first scale feature map is the same as that of the second scale feature map; For the third-scale feature map, the CBS convolution layer is used to adjust the channel size, and the interpolation layer is used for downsampling so that the channel size of the third-scale feature map is the same as that of the second-scale feature map; The first, second and third feature maps with the same channel size are spliced, and the C2f extraction layer is used to extract features from the spliced ​​first feature map, wherein the scales from the first scale to the third scale decrease in sequence.

[0010] In a possible implementation, the multi-scale feature fusion submodule includes a scale sequence feature fusion unit; The scale sequence feature fusion unit is used to increase the dimensions of feature maps of different scales in the pyramid feature map, and perform feature extraction on the feature maps after the dimensions are increased.

[0011] In a possible implementation, the scale sequence feature fusion unit includes a 1×1 convolution layer, an interpolation layer, a concatenation layer, a dimensionality increase layer, a 3D convolution layer, a BN layer, a SiLU activation function, a 3D maximum pooling layer and a dimensionality reduction layer, which are specifically used for: Use a 1×1 convolutional layer to adjust the channel size of the first and third scale feature maps, and use an interpolation layer to downsample them so that the first, second, and third scale feature maps have the same size; Use a dimensionality increase layer to increase the depth dimension of each scale feature map, and splice each scale feature map based on the depth dimension, and use a 3D convolution layer, a BN layer and a SiLU activation function to extract features from the spliced ​​second feature map to obtain a third feature map; A 3D maximum pooling layer and a dimensionality reduction layer are used to perform dimensionality reduction processing on the third feature map.

[0012] In a possible implementation, the cross-scale feature enhancement submodule includes a scale sequence feature fusion unit and a cross-scale sequence detection layer; The cross-scale sequence detection layer includes an upsampling layer, a splicing layer and a C2f extraction layer.

[0013] In a possible implementation, the cross-scale feature enhancement submodule is specifically used to: Use an upsampling layer to upsample the fourth-scale feature map, and splice the upsampled fourth-scale feature map with the fifth-scale feature map, and use a C2f extraction layer to perform feature extraction on the spliced ​​fourth feature map to obtain a fourth feature map; Use a scale sequence feature fusion unit to fuse the fifth scale feature map, the fourth scale feature map, and the sixth scale feature map to obtain a fifth layer feature map; The fourth feature map and the fifth feature map are added together.

[0014] In a possible implementation, the SIoU loss function includes angle loss, distance loss, shape loss and IoU loss, wherein the angle loss is associated with the distance loss; The calculation formula of the SIoU loss function is:

[0015] in, is the IoU loss, is the distance loss, is the shape loss.

[0016] In a second aspect, the present invention further provides a target detection method, which is implemented based on the above detection device, and the target detection method includes: Acquire an image data set consisting of a plurality of target detection images, wherein the target detection images are parking lot images; Detecting the target detection image in the image data set using the detection device, wherein the detection device includes a backbone feature extraction module, a feature fusion module, and a detection head module, and detecting the target detection image in the image data set using the detection device includes: Using the backbone feature extraction module to extract features from the target detection image to obtain a pyramid feature map; Using the feature fusion module to perform feature fusion on the pyramid feature map to obtain a feature fusion map, wherein the feature fusion module includes a multi-scale feature fusion submodule and a cross-scale feature enhancement submodule, the multi-scale feature fusion submodule is used to fuse first feature information of different scales in the pyramid feature map, and the cross-scale feature enhancement submodule is used to fuse second feature information of different scales in the pyramid feature map based on the first feature information to obtain the feature fusion map; The detection head module is used to detect the feature fusion map and then output the detection result, wherein the prediction box loss function in the detection head module is the SIoU loss function.

[0017] The present invention firstly extracts features from the target detection image through the backbone feature extraction module to obtain pyramid feature maps of different scales; then uses the feature fusion module to perform feature fusion on the pyramid feature map to obtain a feature fusion map, specifically obtains the first feature information through the multi-scale feature fusion submodule in the feature fusion module to improve the accuracy of detecting targets of different scales, obtains the feature fusion map based on the first feature information through the second feature information in the cross-scale feature enhancement submodule in the feature fusion module to improve the detection accuracy of difficult samples; finally, the SIoU loss function in the detection head module is used to more accurately locate the target. Thus, the problems of missed detection and wrong detection in target detection are reduced, and the detection accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic structural diagram of an embodiment of a detection device for target detection provided by the present invention; Figure 2 An overall framework diagram of an MSS-YOLO algorithm proposed in one embodiment of the present invention; Figure 3 An original detection result diagram proposed by an embodiment of the present invention; Figure 4 A schematic diagram of the position relationship between a target prediction frame and a real frame proposed in one embodiment of the present invention; Figure 5 A method flow chart of an embodiment of a target detection method provided by the present invention; Figure 6 A visual graph of the MF-EFFN heat map provided by an embodiment of the present invention; Figure 7 A visualization result diagram of SSFF-LFE effect provided by an embodiment of the present invention; Figure 8 A comparison diagram of various errors before and after the loss function is improved according to an embodiment of the present invention; Fig. 9 This is a comparison diagram of mAP of various categories in the front and back paths of an algorithm provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0020] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0021] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0022] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present invention may be combined with other embodiments.

[0023] A specific embodiment of the present invention, as Figure 1 As shown, Figure 1 A schematic diagram of the structure of an embodiment of a detection device 100 for target detection provided by the present invention, comprising: a backbone feature extraction module 101, a feature fusion module 102 and a detection head module 103 connected in sequence; The backbone feature extraction module 101 is used to extract features from the target detection image to obtain a pyramid feature map, wherein the target detection image is a parking lot image; The feature fusion module 102 is used to perform feature fusion on the pyramid feature map to obtain a feature fusion map, wherein the feature fusion module includes a multi-scale feature fusion submodule and a cross-scale feature enhancement submodule, the multi-scale feature fusion submodule is used to fuse the first feature information of different scales in the pyramid feature map, and the cross-scale feature enhancement submodule is used to fuse the second feature information of different scales in the pyramid feature map based on the first feature information to obtain the feature fusion map; The detection head module 103 is used to detect the feature fusion map and then output the detection result, wherein the prediction box loss function in the detection head module is the SIoU loss function.

[0024] Among them, the network structure of the backbone feature extraction module is the CSPDarkNet network structure. It is understandable that YOLOv8 may have insufficient accuracy in small target detection and occlusion detection, resulting in missed detection or false detection. Although the YOLOv8 series model is superior in speed, real-time processing may still be difficult in high-resolution images or complex scenes. Therefore, the detection device for target detection in the present invention uses an improved YOLOv8 algorithm. In order to adapt to these characteristics of on-street parking scenes and improve the detection accuracy, Figure 2 As shown, Figure 2 This is an overall framework diagram of the MSS-YOLO algorithm proposed in one embodiment of the present invention. The multi-scale feature fusion submodule, cross-scale feature enhancement submodule and loss function in the YOLOv8 algorithm are mainly improved.

[0025] The present invention firstly extracts features from the target detection image through the backbone feature extraction module to obtain pyramid feature maps of different scales; then uses the feature fusion module to perform feature fusion on the pyramid feature map to obtain a feature fusion map, specifically obtains the first feature information through the multi-scale feature fusion submodule in the feature fusion module to improve the accuracy of detecting targets of different scales, obtains the feature fusion map based on the first feature information through the second feature information in the cross-scale feature enhancement submodule in the feature fusion module to improve the detection accuracy of difficult samples; finally, the SIoU loss function in the detection head module is used to more accurately locate the target. Thus, the problems of missed detection and wrong detection in target detection are reduced, and the detection accuracy is improved.

[0026] In one embodiment of the present invention, the first feature information includes scale information; The second feature information includes semantic information.

[0027] It is understandable that in a natural traffic environment, when a target such as a vehicle is projected onto the image plane by a high-position camera, its size is inversely proportional to the actual distance, resulting in frequent instances of multi-scale targets in side road parking detection. Since features in an image vary significantly with scale, it is easy to cause false detection problems. YOLOv8 only uses summation or concatenation to process pyramid features during feature fusion, and fails to fully utilize the correlation between feature maps of each pyramid.

[0028] Therefore, the multi-scale feature fusion submodule MF-EFFN in the present invention can integrate the first feature information of more scales, that is, scale information, and effectively combine the high-dimensional information of the high-level feature map with the detailed information of the low-level feature map to reduce the false detection caused by improper scale selection and improve the accuracy of detecting objects of different scales. The cross-scale feature enhancement submodule SSFF-LFE can fuse the second feature information, that is, fuse the high-level semantic information, increase the richness of the underlying semantic information, and improve the detection accuracy of difficult samples.

[0029] In one embodiment of the present invention, the multi-scale feature fusion submodule includes a multi-scale collaboration unit; The multi-scale collaborative unit is used to adjust the channel sizes of feature maps of different scales in the pyramid feature map, and perform feature extraction on the feature maps after the channel sizes are adjusted.

[0030] The multi-scale collaborative unit includes CBS convolution layer, pooling layer, interpolation layer and C2f extraction layer, which are used for: For the first scale feature map, the CBS convolution layer is used to adjust the channel size, and the pooling layer is used for downsampling so that the channel size of the first scale feature map is the same as that of the second scale feature map; For the third-scale feature map, the CBS convolution layer is used to adjust the channel size, and the interpolation layer is used for downsampling so that the channel size of the third-scale feature map is the same as that of the second-scale feature map; The first, second and third feature maps with the same channel size are spliced, and the C2f extraction layer is used to extract features from the spliced ​​first feature map, wherein the scales from the first scale to the third scale decrease in sequence.

[0031] It is understandable that YOLOv8 uses the feature fusion network of PAN-FPN for feature fusion, but in the feature fusion process, only the summation or splicing method is used to fuse the pyramid features, and the correlation between all pyramid feature maps cannot be effectively utilized. The present invention uses the scale sequence feature fusion unit SSFF to perform multi-scale feature fusion, and designs a multi-scale collaborative method MSC to fuse feature information of more scales. In order to better combine the high-dimensional information of the high-level feature map with the detailed information of the low-level feature map, the target features obtained at different scales are complemented, the false detection and missed detection problems caused by improper scale selection are reduced, and the accuracy of targets of different scales is improved.

[0032] Among them, the first scale feature map is a large-size feature map, that is, Figure 2 In B5, the second scale feature map is the medium-sized feature map, that is, Figure 2 In B4, the third scale feature map is a small-size feature map, that is, Figure 2 In B3, the first, second and third characteristic diagrams are process characteristic diagrams, which are not shown in the figure.

[0033] For large-size feature maps, the CBS convolution layer is used to process them, and the number of channels is adjusted to 1C. Then, the pooling method is used for downsampling to maintain the effectiveness and diversity of the high-resolution features of the on-road targets, so that their size is the same as that of the medium-scale feature map. Among them, when fusing higher-dimensional features, the adaptive average pooling method is used to make the network pay more attention to the average information, which helps to retain the overall information of the high-scale features; when fusing lower-dimensional features, the maximum pooling method is used to retain the most significant features in the feature map, making the network more robust to small position changes; for small-size feature maps, the CBS convolution layer is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling to maintain the richness of local features of low-resolution images, prevent the loss of small target feature information, and make its size match that of the medium-scale feature map. Finally, the feature maps of the same size, large, medium, and small, are spliced ​​on the channel size, and finally the spliced ​​features are extracted through the C2f extraction layer.

[0034] In one embodiment of the present invention, the multi-scale feature fusion submodule includes a scale sequence feature fusion unit; The scale sequence feature fusion unit is used to increase the dimensions of feature maps of different scales in the pyramid feature map, and perform feature extraction on the feature maps after the dimensions are increased.

[0035] The scale sequence feature fusion unit includes 1×1 convolution layer, interpolation layer, dimension enhancement layer, 3D convolution layer, BN layer, SiLU activation function, 3D maximum pooling layer and dimension reduction layer, which are specifically used for: Use a 1×1 convolutional layer to adjust the channel size of the first and third scale feature maps, and use an interpolation layer to downsample them so that the first, second, and third scale feature maps have the same size; Use the dimension-raising layer to increase the depth dimension of each scale feature map, and splice each scale feature map based on the depth dimension, and use the 3D convolution layer, BN layer and SiLU activation function to extract features from the spliced ​​second feature map to obtain the third feature map; The third feature map is reduced in dimension using a 3D maximum pooling layer and a dimensionality reduction layer.

[0036] To exploit the correlation between all pyramid feature maps, such as Figure 2 As shown in the figure, the scale sequence feature fusion unit SSFF is used to add the fusion of the feature maps of the B3, B4 and B5 layers. The B3, B4 and B5 feature maps are normalized to the same size and then stacked together as the input of the 3D convolution to combine multi-scale features. Specifically, the number of channels of the B4 and B5 feature levels is first changed to 256 using 1×1 convolution, and the nearest neighbor interpolation method is used to resize them to the size of the B3 level. The unsqueeze method is used to increase the dimension of each feature map, changing it from a three-dimensional tensor [height, width, channel] to a four-dimensional tensor [depth, height, width, channel]. Then the 4D feature maps are connected along the depth dimension to form a 3D feature map for subsequent convolution. Finally, the multi-scale sequence feature extraction is completed using three-dimensional convolution, batch normalization BN and SiLU activation function, and the extracted features are reduced in dimension using 3D maximum pooling and squeeze methods.

[0037] In one embodiment of the present invention, the cross-scale feature enhancement submodule includes a scale sequence feature fusion unit and a cross-scale sequence detection layer; The cross-scale sequence detection layer includes upsampling layer, concatenation layer and C2f extraction layer.

[0038] The cross-scale feature enhancement submodule is specifically used for: Use an upsampling layer to upsample the fourth-scale feature map, and splice the upsampled fourth-scale feature map with the fifth-scale feature map, and use a C2f extraction layer to perform feature extraction on the spliced ​​fourth feature map to obtain a fourth feature map; Use a scale sequence feature fusion unit to fuse the fifth scale feature map, the fourth scale feature map, and the sixth scale feature map to obtain a fifth layer feature map; The fourth feature map and the fifth feature map are added together.

[0039] Among them, the fourth scale feature map is a feature map generated by multiple downsampling of the third scale feature map, that is, Figure 2In P3, the fifth scale feature map is a smaller size feature map in the pyramid feature map, that is, Figure 2 In B2, the sixth scale feature map is a feature map generated by multiple downsampling of the second scale feature map, that is, Figure 2 P4, the fourth and fifth characteristic diagrams are process characteristic diagrams, which are not shown in the figure.

[0040] It is understandable that MF-EFFN can fuse more feature maps and improve the algorithm's utilization of scale. However, it still outputs P3, P4 and P5 feature maps of different scales from the B3, B4 and B5 layers through multiple downsampling, and some detailed target features in the image will be lost. The loss of these detailed features has a great impact on the algorithm's detection of low-feature targets such as small targets and heavy occlusions. The P2 layer, due to its higher resolution, can capture more image detail information. This means that even when the target is partially occluded or the target scale is small, the P2 layer may capture some useful detail features. The traditional P2 layer increases upsampling in the FPN part, amplifies the low-level features, and outputs the features of this layer, but does not fully integrate the high-level semantic information and cannot complement the target features.

[0041] Therefore, to address the problem of missed detection of low-feature targets due to the loss of target detail features in on-street parking scenarios, the present invention proposes an SSFF-LFE network, and adds a cross-scale sequence low-feature target detection layer CSS-P2. By fusing high-level semantic information, the richness of the underlying semantic information is increased, thereby improving the detection accuracy of low-feature targets.

[0042] Specifically, the P3 layer is first upsampled to enlarge the low-level features and image resolution, and then spliced ​​with the semantic information of the B2 layer to supplement the semantic information lost in the upsampling process, and the C2f convolution module is used to extract the spliced ​​features. Secondly, considering that the size of the shallow feature map will change during the image downsampling process, but the scale will not change, the scale sequence feature fusion unit SSFF is used to fuse the shallow feature map B2 and the output of the P3 and P4 layers. As the three feature maps of the SSFF input have the same resolution and different scales, the feature maps of different sizes can be regarded as scale space, and the effective feature maps of different resolutions are adjusted to the same resolution for splicing, and the scale sequence features of the horizontally stacked feature maps are extracted in the SSFF using three-dimensional convolution. Finally, the feature maps obtained in the first two steps are stacked along the new dimension using the Add method to integrate features of different scales, abstraction levels or types, such as edges, textures, etc. Comprehensively enhance the overall feature expression ability.

[0043] In one embodiment of the present invention, the SIoU loss function includes angle loss, distance loss, shape loss and IoU loss, wherein the angle loss is associated with the distance loss; The calculation formula of SIoU loss function is:

[0044] in, is the IoU loss, which is the intersection-over-union ratio between the real box and the predicted box. is the distance loss, is the shape loss.

[0045] It is understandable that since the high-position camera that collects video data in the on-street parking scene is fixed in position, when there are many parked vehicles, dense occlusion often occurs, resulting in the problem of missed detection of targets in the picture, such as Figure 3 As shown, Figure 3 This is an original detection result diagram proposed by an embodiment of the present invention. Figure 3 In the image shown, a significant detection error can be observed. The black and white cars pointed to by arrow A are mistakenly marked as a single entity by the same bounding box. Although the confidence level reaches 0.93, this is still an incorrect labeling. This high-confidence mislabeling causes the algorithm to discard the low-confidence box of the black car in post-processing, resulting in the black car not being identified separately, resulting in missed detection. In addition, in the image area pointed to by arrow B, the black car is only partially framed out of the target area due to the occlusion of leaves. Both of these situations indicate the problem of insufficient positioning accuracy.

[0046] It can be understood that the bounding box regression loss depends on the relative position of the target prediction box and the real box. The YOLOv8 bounding box regression loss adopts the CIoU loss function. The CIoU loss function does not consider the situation of inconsistent directions between the two boxes. The slight deficiencies in the convergence direction may also accumulate into large errors during the training process, making it difficult for the model to accurately locate the bounding box of the target in the road scene. The present invention adopts the SIoU loss function to use the vector angle in the regression to match the problem of inconsistent directions between the prediction box and the real box. SIoU includes four parts: angle loss, distance loss, shape loss, and IoU loss. Adding angle loss can minimize the number of variables related to distance.

[0047] like Figure 4 As shown, Figure 4 A schematic diagram of the position relationship between a target prediction box and a real box proposed in one embodiment of the present invention. Where T is the minimum bounding rectangle of the real box and the prediction box, and are the width and height of the minimum circumscribed rectangle respectively, and c is the length of its diagonal. is the prediction box, is the real frame, is the distance between the center point of the predicted box and the real box, called the true value distance, It is the angle between the true value distance and the X-axis direction. It is the angle between the true value distance and the Y-axis direction. is the horizontal distance between the center point of the real box and the center point of the predicted box, It is the vertical distance between the center point of the real box and the center point of the predicted box.

[0048] Therefore, the formula for the angle loss in SIoU is defined as:

[0049] According to the angle loss formula, during the training process, the network model compares the coordinate parameters of the center point of the predicted box and the real box to calculate the angles 𝛼 and 𝛽 between the two. If the 𝛼 value is less than 45 degrees, the model training will focus on reducing 𝛼, which means that the predicted box will gradually align with the real box in the X-axis direction. Similarly, when the angle 𝛽 is less than 45 degrees, the model training will focus on reducing 𝛽, which means that the predicted box will tend to align with the real box in the Y-axis direction, thereby speeding up the network training.

[0050] The distance loss function is related to the minimum outer rectangle, so the distance loss formula in SIoU is defined as:

[0051]

[0052]

[0053]

[0054] Due to different Angle will affect the distance loss, so we introduce The factor relates the distance loss to the angle loss. As it approaches 0, the contribution of the distance loss function should decrease. When the angle approaches 45 degrees, the contribution value of the distance loss function becomes relatively larger. δx and δy represent the absolute values ​​of the difference between the width and height of the two frames, respectively.

[0055] The shape loss formula in SIoU is defined as:

[0056]

[0057]

[0058] in, is a hyperparameter that is different for each dataset and represents the weight factor of the shape loss in that dataset. and Indicates the width and height of the real box, and Indicates the width and height of the prediction box.

[0059] The IoU loss formula in SIoU is defined as:

[0060] It is the intersection-over-union ratio of the true box and the predicted box.

[0061] The calculation formula of SIoU loss function is:

[0062] In order to better implement the detection device for target detection in the embodiment of the present invention, based on the detection device for target detection, correspondingly, Figure 5 As shown, the embodiment of the present invention also provides a method flow chart of a target detection method, including: S501: Acquire an image data set consisting of a plurality of target detection images, wherein the target detection image is a parking lot image; S502: extracting features from the target detection image using a backbone feature extraction module to obtain a pyramid feature map; S503: using a feature fusion module to perform feature fusion on the pyramid feature map to obtain a feature fusion map, wherein the feature fusion module includes a multi-scale feature fusion submodule and a cross-scale feature enhancement submodule, the multi-scale feature fusion submodule is used to fuse first feature information of different scales in the pyramid feature map, and the cross-scale feature enhancement submodule is used to fuse second feature information of different scales in the pyramid feature map based on the first feature information to obtain the feature fusion map; S504: Utilize the detection head module to detect the feature fusion image and then output the detection result, wherein the prediction box loss function in the detection head module is the SIoU loss function.

[0063] It is understandable that in order to achieve the detection of the desired target, the detection device for target detection needs to be trained and trained to the target state. The target detection training process in the present invention is described below.

[0064] In one embodiment, the training and testing environment of the present invention is Python 3.8, PyTorch1.12.1, and the hardware environment is CPU- Intel(R) i5-13600k, GPU- NVIDIA GeForce RTX 4070 Ti SUPER. During the training process, the input image is preprocessed so that the input image is 640×640 pixels. The batch size is set to 32, and the iterative training is performed 200 times. SGD is used to update the gradient in the training, the initial learning rate is 0.01, and the weight decay coefficient is 0.0005. mAP is used as the accuracy evaluation index, APs is used to measure small targets, APm is used to measure medium targets, and AP1 is used to measure large targets. FPS is used as the algorithm real-time indicator, Parameters measures the model parameter quantity, and size measures the model size.

[0065] At present, there are various data sets for on-road scenes. The KITTI data set uses a car-mounted camera to collect images from a driving perspective, which is more suitable for research in the direction of unmanned driving. The CADP data set collects data from a high-position perspective on the road, but only focuses on vehicles running on the road, excluding on-road parking spaces. Although the VisDrone data set includes on-road parking spaces, the size of the vehicles in the image is relatively small due to the use of drones for collection. The UA-DETRAC data set collects data from a monitoring perspective, and some of the data include parking areas. On this basis, the present invention also integrates the all-day high-position monitoring data of different on-road parking locations provided by the traffic management department. The final data set contains 7093 sample images. While ensuring independent distribution, the data set is divided into a training set, a validation set, and a test set according to a ratio of 7:1:2.

[0066] The experimental results using the above dataset are as follows: In order to verify the effectiveness of MSC, SSFF and finally MF-EFFN, the present invention sets up an ablation experiment.

[0067] The experimental results of the sequence feature fusion network ablation experiment under multi-scale collaboration are shown in Table 1.

[0068] Table 1 Experimental results of MF-EFFN network (%)

[0069] As can be seen from Table 1, after using MSC and SSFF alone, Aps, APm and AP1 are all improved to a certain extent, especially MSC, the large, medium and small detection accuracy is improved by an average of 1.1%. It can be seen that both methods can effectively fuse multi-scale feature information and improve the detection effect of targets of different scales. The multi-scale feature fusion submodule MF-EFFN after the two work together can further improve the detection effect of multi-scale targets. Compared with the benchmark algorithm, Aps, APm and AP1 are improved by 1.7%, 1.9% and 2.4% respectively. This shows that the MF-EFFN proposed in the present invention effectively utilizes the correlation between pyramid feature maps to improve the accuracy of the algorithm for detecting targets of multiple scales, thereby solving the problem of large scale spans between distant large targets and near small targets in this scenario. In order to verify whether the improvement can extract more significant features of the target, the present invention visualizes the extracted features in the form of a heat map. Problems such as missed detection and false detection before the application of the strategy are marked with orange boxes. The effect is as follows. Figure 6 shown. Figure 6 This is a visualization of the MF-EFFN heat map provided by an embodiment of the present invention. The heat map uses the intensity change of color to represent the response degree of the model to the target detection effect in different areas of the image, reflecting the ability of the model to identify the target. The hotter areas in the heat map, that is, the red areas, indicate that the model predicts that these areas are more likely to be targets, while the colder areas, that is, the blue areas, indicate that the prediction probability is lower. Figure 6 It can be seen that after applying the MF-EFFN proposed in this invention, the consistency between the red area of ​​the heat map corresponding to each type of image and the actual target position is higher than before the improvement. It proves that after using the MF-EFFN strategy, more significant features can be extracted for multi-scale targets, and the phenomenon of missed detection and false detection of targets of different scales can be significantly improved.

[0070] The comparison of algorithm performance before and after using the SSFF-LFE strategy is shown in Table 2.

[0071] Table 2 Experimental results of SSFF-LFE improvement strategy (%)

[0072] As shown in Table 2, after using the SSFF-LFE strategy, APs increased from 50.4% to 58.7%, indicating that the detection effect of small targets has been significantly improved. APm and APl increased by 2.1% and 0.9% respectively, indicating that after integrating higher-level features, there is still a certain effect on medium and large-scale targets. It can be seen that this strategy can effectively improve the detection effect of difficult samples such as distant small targets. In the visualization results before and after applying the SSFF-LFE improvement strategy, the problem solving situation is marked with orange circles, such as Figure 7 shown. Figure 7A visualization result diagram of SSFF-LFE effect provided by an embodiment of the present invention. Figure 7 It can be seen that compared with the baseline algorithm, after using the SSFF-LFE strategy, more difficult samples such as distant small targets at the edge of the field of view and occluded targets in this scene are detected, proving that SSFF-LFE effectively solves the problem of missed detection of difficult samples in the original algorithm.

[0073] The comparison results before and after using the improved strategy for on-road target positioning based on SIoU are shown in Table 3.

[0074] Table 3 Experimental results of improved strategy for on-road target positioning based on SIoU (%)

[0075] As can be seen from Table 3, after using the SIoU loss function, the mAP value of the algorithm is increased by 1.3%, which proves that the invention can improve the accuracy of the model after using the SIoU loss function instead of the original loss function. Specifically, after using the SIoU loss function, APs and AP1 are improved to a certain extent, AP1 is improved by 1.5%, APs is improved by 0.4%, but the value of APm is slightly reduced by 1%. We speculate that this is because for larger-scale targets, the regression of their boxes needs to consider the impact of angles more, while for smaller-scale targets, the impact of angles is smaller.

[0076] In order to accurately evaluate the impact of the improved on-road target localization strategy based on the SIoU loss function on the algorithm accuracy and specifically analyze its effect in reducing detection errors, the present invention additionally uses the TIDE tool proposed by Bolyaetal for in-depth research. The experimental results are as follows: Figure 8 shown. Figure 8 A comparison chart of various types of errors before and after the loss function is improved according to an embodiment of the present invention. TIDE is specifically used to identify and quantify different types of errors in detection algorithms. It divides detection errors into six categories, including classification errors (Cls), positioning errors (Loc), classification and positioning errors (Both), duplicate detection errors (Dupe), background errors (Bkg) and undetected GT errors (Miss). The values ​​of the six indicators are positively correlated with the number of times the errors occur. In addition, another classification method is used to divide the errors into two categories: False Positive (FP) and False Negative (FN). Figure 8In the figure, the pie chart shows the relative contribution of each error, while the bar chart shows their absolute contribution. After using the SIoU loss function, all types of errors are reduced by 0.72% overall, of which the errors caused by inaccurate positioning include Loc and Both, with a reduction of 59.7%, indicating that the increase in model accuracy after using SIoU is mainly caused by more accurate positioning, which verifies the effectiveness of this strategy for positioning errors. The changes in the remaining categories of errors are not large, and have little impact on the improvement of model accuracy. In the end, the total error FP+FN is reduced by 0.91%, and the quality of target positioning is improved.

[0077] In order to verify the effectiveness of the three strategies proposed in this chapter working together on the benchmark algorithm, the present invention also sets up another ablation experiment. The experimental results are shown in Table 4.

[0078] Table 4 Ablation experiment results of improved strategy (%)

[0079] It can be seen from Table 4 that when the three strategies act alone, the model accuracy is improved to varying degrees. Among them, the mAP is improved the most when MF-EFFN acts alone, reaching 2.1%, which shows that the problem of large target scale span in on-street parking scenarios is more prominent. And the improvements based on any two strategies are better than the improvements of any single strategy. When the three improved strategies proposed in the present invention act together, the algorithm achieves the highest accuracy, with an mAP of 74.5% and an AP50 of 89.2%, which are 3.1% and 1.9% higher than the baseline model, respectively, indicating that the three improved strategies proposed in the present invention can be effectively combined. In order to further verify its detection accuracy mAP for each category, the present invention counted the mAP of different categories on the road before and after the algorithm improvement, and the results are as follows. Fig. 9 shown. Fig. 9 This is a comparison chart of mAP of various categories on the road before and after an algorithm improvement provided in one embodiment of the present invention. It can be seen that the MSS-YOLO proposed in the present invention has improved the mAP values ​​of various categories of targets to varying degrees. The category with the highest accuracy is car, with an mAP of 91.9%, an increase of 1.4%, which can effectively assist in parking management of road traffic, such as time-based charging, illegal parking identification, etc. The category with the highest accuracy improvement is person, which has increased by 7.2%, which can better ensure the safety of pedestrians in actual on-street parking scenarios. The overall mAP has increased by 3.1% to 74.5%, which can meet the accuracy requirements of on-street parking detection.

[0080] The present invention also compares and analyzes the proposed MSS-YOLO with the existing typical target detection algorithms YOLOv5, YOLOv6, YOLOX and RTDETR, and the input images are uniformly adjusted to 640×640. The experimental results are shown in Table 5.

[0081] Table 5 Comparative experimental results of improved algorithms

[0082] As can be seen from Table 5, the algorithm accuracy of the improved algorithm MSS-YOLO proposed in the present invention in the on-street parking scenario is significantly better than the existing algorithms. The mAP, AP50 and AP75 accuracy indicators all achieved the best results. Compared with the poor YOLOX algorithm, the mAP increased by 9.0%. Compared with the YOLOv5 and YOLOv6 algorithms of the same level, the mAP values ​​increased by 4.2% and 3.9% respectively. Compared with the baseline YOLOv8 algorithm, the number of channels of the multi-scale feature maps P3, P4, and P5 was reduced from the original 256, 512, and 1024 to 128, 256, and 256. The number of MSS-YOLO parameters decreased significantly, with a decrease of 18.6%. The inference speed was also increased from 79.6FPS to 97.5FPS. Although it is slightly lower than YOLOX, it is enough to meet the real-time requirement of 30FPS. Even so, it still has obvious advantages over RTDETR, an object detection algorithm that uses a real-time transform architecture. Not only is the mAP accuracy 5.0% higher than that of the RTDETR algorithm, but it is also significantly better than RTDETR in terms of computing resource consumption, with the computing cost indicator Param being 45.5% of RTDETR.

[0083] In summary, when applied to on-street parking scenarios, the MSS-YOLO proposed in the present invention has the highest accuracy, a low false detection rate, and a medium level of computing resources, and has achieved a preliminary balance between accuracy and computing cost.

[0084] The MSS-YOLO algorithm proposed in this paper combines the above three strategies. With a small amount of computational effort, the mAP reaches 74.5%, which is 3.1% higher than the baseline. In the comparative experiment, it is also verified that it is superior to mainstream detection models such as RTDETR, and the detection accuracy of the main target car in the on-street parking scene reaches 91.9%, which promotes the efficient management of road traffic.

[0085] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0086] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A detection device for target detection, characterized in that: include: A backbone feature extraction module, a feature fusion module and a detection head module connected in sequence; The backbone feature extraction module is used to extract features from a target detection image to obtain a pyramid feature map, wherein the target detection image is a parking lot image; The feature fusion module is used to perform feature fusion on the pyramid feature map to obtain a feature fusion map, wherein the feature fusion module includes a multi-scale feature fusion submodule and a cross-scale feature enhancement submodule, the multi-scale feature fusion submodule is used to fuse first feature information of different scales in the pyramid feature map, and the cross-scale feature enhancement submodule is used to fuse second feature information of different scales in the pyramid feature map based on the first feature information to obtain the feature fusion map; The detection head module is used to output a detection result after detecting the feature fusion map, wherein the prediction box loss function in the detection head module is a SIoU loss function.

2. The detection device for target detection according to claim 1, characterized in that: The first feature information includes scale information; The second feature information includes semantic information.

3. The detection device for target detection according to claim 2, characterized in that: The multi-scale feature fusion submodule includes a multi-scale collaboration unit; The multi-scale collaborative unit is used to adjust the channel sizes of the feature maps of different scales in the pyramid feature map, and perform feature extraction on the feature maps after the channel sizes are adjusted.

4. The detection device for target detection according to claim 3, characterized in that: The multi-scale collaborative unit includes a CBS convolution layer, a pooling layer, a splicing layer, an interpolation layer and a C2f extraction layer, which are specifically used for: For the first scale feature map, the CBS convolution layer is used to adjust the channel size, and the pooling layer is used for downsampling so that the channel size of the first scale feature map is the same as that of the second scale feature map; For the third-scale feature map, the CBS convolution layer is used to adjust the channel size, and the interpolation layer is used for downsampling so that the channel size of the third-scale feature map is the same as that of the second-scale feature map; The first, second and third feature maps with the same channel size are spliced, and the C2f extraction layer is used to extract features from the spliced ​​first feature map, wherein the scales from the first scale to the third scale decrease in sequence.

5. The detection device for target detection according to claim 2, characterized in that: The multi-scale feature fusion submodule includes a scale sequence feature fusion unit; The scale sequence feature fusion unit is used to increase the dimensions of feature maps of different scales in the pyramid feature map, and perform feature extraction on the feature maps after the dimensions are increased.

6. The detection device for target detection according to claim 5, characterized in that: The scale sequence feature fusion unit includes a 1×1 convolution layer, an interpolation layer, a splicing layer, a dimension increase layer, a 3D convolution layer, a BN layer, a SiLU activation function, a 3D maximum pooling layer and a dimension reduction layer, which are specifically used for: Use a 1×1 convolutional layer to adjust the channel size of the first and third scale feature maps, and use an interpolation layer to downsample them so that the first, second, and third scale feature maps have the same size; Use a dimensionality increase layer to increase the depth dimension of each scale feature map, and splice each scale feature map based on the depth dimension, and use a 3D convolution layer, a BN layer and a SiLU activation function to extract features from the spliced ​​second feature map to obtain a third feature map; A 3D maximum pooling layer and a dimensionality reduction layer are used to perform dimensionality reduction processing on the third feature map.

7. The detection device for target detection according to claim 2, characterized in that: The cross-scale feature enhancement submodule includes a scale sequence feature fusion unit and a cross-scale sequence detection layer; The cross-scale sequence detection layer includes an upsampling layer, a splicing layer and a C2f extraction layer.

8. The detection device for target detection according to claim 7, characterized in that: The cross-scale feature enhancement submodule is specifically used for: Use an upsampling layer to upsample the fourth-scale feature map, and splice the upsampled fourth-scale feature map with the fifth-scale feature map, and use a C2f extraction layer to perform feature extraction on the spliced ​​fourth feature map to obtain a fourth feature map; Use a scale sequence feature fusion unit to fuse the fifth scale feature map, the fourth scale feature map, and the sixth scale feature map to obtain a fifth layer feature map; The fourth feature map and the fifth feature map are added together.

9. The detection device for target detection according to claim 1, characterized in that: The SIoU loss function includes angle loss, distance loss, shape loss and IoU loss, wherein the angle loss is associated with the distance loss; The calculation formula of the SIoU loss function is: in, is the IoU loss, is the distance loss, is the shape loss.

10. A target detection method, characterized in that: The target detection method is implemented based on the detection device for target detection according to any one of claims 1 to 9, and the target detection method includes: Acquire an image data set consisting of a plurality of target detection images, wherein the target detection images are parking lot images; Detecting the target detection image in the image data set using the detection device, wherein the detection device includes a backbone feature extraction module, a feature fusion module, and a detection head module, and detecting the target detection image in the image data set using the detection device includes: Using the backbone feature extraction module to extract features from the target detection image to obtain a pyramid feature map; Using the feature fusion module to perform feature fusion on the pyramid feature map to obtain a feature fusion map, wherein the feature fusion module includes a multi-scale feature fusion submodule and a cross-scale feature enhancement submodule, the multi-scale feature fusion submodule is used to fuse first feature information of different scales in the pyramid feature map, and the cross-scale feature enhancement submodule is used to fuse second feature information of different scales in the pyramid feature map based on the first feature information to obtain the feature fusion map; The detection head module is used to detect the feature fusion map and then output the detection result, wherein the prediction box loss function in the detection head module is the SIoU loss function.

Images