Single-layer feature map and explicit object based anchor-free traffic sign detection method

Abstract

The application discloses an anchor frame-free traffic sign detection method based on a single-layer feature map and an explicit object, and comprises the following steps: performing image enhancement on an input original image; inputting the enhanced image into a single-layer feature extractor and a receptive field expander to obtain a fused single-layer feature map; obtaining three detection head branches, including a classification branch, a positioning branch and an explicit object branch, from the single-layer feature map through convolution; respectively calculating loss functions of the classification branch, the positioning branch and the explicit object branch, and training and optimizing the classification branch, the positioning branch and the explicit object branch through the loss functions; and obtaining a prediction target through the optimized classification branch, the positioning branch and the explicit object branch for prediction. The application solves the problems that the detection accuracy needs to be improved due to the great influence of light factors on traffic signs and the great change of detection scales, and does not have the problems of great data labeling cost, great network training cost, great memory burden, complex structure or insufficient feature utilization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to an anchor-free traffic sign detection method based on a single-layer feature map and an explicit object. BACKGROUND

[0002] In recent years, with the application and development of front-line advanced technologies such as Internet of Vehicles, artificial intelligence and 5G communication network in intelligent transportation system (ITS), the intelligent transportation system plays an irreplaceable role in the transportation industry. As an important function of the intelligent transportation system and a key link of the driverless, we should also consider how to improve the performance of traffic sign detection when the intelligent transportation system has great potential. As one of the three basic tasks of computer vision, the classification task only identifies the object class existing in the image, and focuses on the overall content description of the image. The target detection needs to judge whether a certain class of object appears in the image first, and if it appears, the target is positioned, and finally the class information and the position information of the bounding box of each target are output, which pays more attention to specific targets. Compared with general target detection, the main difficulties of traffic sign detection lie in the large scale variation of the target, the complex real road environment, the problems such as weather changes, object occlusion and similar background color interference. Traffic sign detection has experienced years of development, and the current mainstream algorithms are divided into: traditional hand-crafted feature-based detection algorithms and deep learning-based detection algorithms. The traditional method mainly detects based on the physical characteristics of traffic signs, and the common methods are shape-based method and color-based method, but the detection accuracy and speed cannot meet the ideal effect, and unnecessary computational overhead will be generated. With the first application of convolutional neural network in the field of computer vision in 2012, a series of deep learning methods have emerged in the field of target detection, and the common methods are R-CNN series, SSD series and YOLO series detection algorithms, and these methods are improved for the characteristics of the target and applied to traffic sign detection, but there are still many deficiencies.

[0003] Firstly, traffic signs are exposed to the outdoors, and will be affected by the light factor when taking pictures, and the light condition changes greatly with time, season and weather, and the detection difficulty is particularly obvious in the low light night and the overexposed sunny day. The former is not conducive to the detection of traffic signs due to low visibility, and the latter is also affected by the detection due to the strong light irradiation of the over-bright scene, and these problems can be alleviated to a certain extent by data enhancement processing on the pictures. However, the existing methods will cause some problems, some of which have too high data labeling cost and are only suitable for specific data sets, and some of which increase the additional network training cost by relying on priori enhancement.

[0004] Secondly, in the field of general object detection, the feature pyramid network, which has always been a focus, provides multi-scale target information for the detection head by encoding the multi-layer feature maps output by the backbone network, and manually specifies the detection layer where the prediction box of different sizes falls. The divide-and-conquer strategy undoubtedly greatly reduces the overlap and is beneficial to detecting targets with large scale changes. However, it also brings a relatively large memory burden, reduces the speed of the detector, and makes the structure of the single-stage detector relatively complex. Chen et al. found through decoupling experiments that the last layer of feature maps of the backbone network already contains sufficient context information for detecting targets of various scales, and then proposed an anchor-based detection network that only uses single-layer feature maps without a complex pyramid. In subsequent development, single-feature-map detection began to shine in the anchor-free field, and algorithms such as CornerNet, CenterNet, and ObjectBox based on key point detection abandoned the predefined anchor box that would introduce additional hyperparameters and used single feature maps to predict targets. However, algorithms based on dense point detection, such as FCOS, still mostly use pyramid networks for hierarchical detection, and the redundant prediction boxes generated by the large number of sample points can only be removed by post-processing. Meanwhile, feature layer allocation for targets also leads to insufficient feature utilization. SUMMARY

[0005] (I) Technical problems to be solved

[0006] Based on the above problems, the present application provides an anchor-free traffic sign detection method based on single-layer feature maps and explicit objects, which solves the problems of large detection scale changes and low detection accuracy of traffic signs due to the influence of light factors, and the problems of large data labeling cost, network training cost, memory burden, complex structure or insufficient feature utilization after improving detection accuracy.

[0007] (II) Technical solutions

[0008] Based on the above technical problems, the present application provides an anchor-free traffic sign detection method based on single-layer feature maps and explicit objects, comprising the following steps:

[0009] S1, image enhancement is performed on the input original image;

[0010] S2, the enhanced image is input into a single-layer feature extractor and a receptive field expander to obtain a fused single-layer feature map;

[0011] S3, three detection head branches, including a classification branch, a positioning branch, and an explicit object branch, are obtained by convolution from the single-layer feature map;

[0012] S4, loss functions of the classification branch, the positioning branch, and the explicit object branch are calculated respectively, and the classification branch, the positioning branch, and the explicit object branch are trained and optimized through the loss functions.

[0013] S5, obtaining a prediction target by prediction through the optimized classification branch, the positioning branch and the explicit object branch.

[0014] Further, the step S1 comprises:

[0015] S11, scaling an input original image to a uniform size;

[0016] S12, inputting the scaled image into a parameter training subnetwork to train parameters of a filter;

[0017] S13, loading the trained parameters of the filter into the filter respectively, and performing filter enhancement processing on the original image through the filter.

[0018] Further, the parameter training subnetwork comprises five convolutional layers and two fully connected layers; and the filter comprises an exposure filter, a tone filter and a brightness filter.

[0019] Exposure filter: I o =2 E I i ;

[0020] Tone filter:

[0021] Brightness filter: I o =(1-param)*I i +param*Lum(I i );

[0022] wherein E is a parameter of the exposure filter, t k ∈{t0,t1,…,t L-1}, t is a parameter of the tone filter, L is a total number of the parameters of the tone filter, k and j are both numbers of the parameters of the tone filter, i is a channel number, the function clip(x,min,max) represents limiting a value in a range of (min,max), param is a parameter of the brightness filter, Lum(I i ) represents multiplying values of pixels on R, G and B components by corresponding brightness coefficients, I i represents an input pixel value of the filter, and I o represents an output pixel value of the filter.

[0023] Further, the step S2 comprises:

[0024] S21, inputting the enhanced image into a single-layer feature extractor to obtain C1-C5 feature maps through continuous down-sampling;

[0025] S22, the P3 feature map is obtained by performing a channel shift operation on the C3 feature map, the P4 feature map is obtained by performing a channel shift operation on the P3 feature map and the C4 feature map after fusion by a feature aggregator FA, and the P5 feature map is obtained by performing a channel shift operation and ASPP on the P4 feature map and the C5 feature map after fusion by the feature aggregator FA;

[0026] S23, inputting the P5 feature map into a receptive field expander to output a fused single-layer feature map t5.

[0027] Further, the step S21 comprises: using maximum pooling and convolution to change the size and channel number of the enhanced image to obtain C1-C3 feature maps in sequence; then inputting the C3 feature map into an MLP block to perform a channel shift operation to obtain a C4 feature map with the same change in size and channel, and inputting the C4 into the MLP block to obtain a C5 feature map;

[0028] In the step S22, before fusion by the feature aggregator FA, further comprising: convolving two feature maps fused by the feature aggregator FA into feature maps with the same size and channel number;

[0029] In the step S23, the receptive field expander comprises a plurality of residual dilation blocks with different hole rates.

[0030] Further, the step S3 comprises: obtaining a network output in the format of (batch, 3, H, W, C+5) by performing two 1x1 convolutions and one 5x5 convolution on the single-layer feature map t5, wherein the channel number C+5 comprises a channel number of four positioning branches, a channel number of one explicit object branch, and a channel number of C classification branches, the positioning branch predicts relative position information (x c , y c , w, h) of a detection frame, (x c , y c ) is the center point coordinate, w and h are the width and height of the detection frame respectively, the explicit object branch predicts an objectness value, and the classification branch predicts C class probabilities of each pixel on the feature map.

[0031] Further, the step S4 comprises:

[0032] S41, performing positive and negative sample matching

[0033] S42, calculating loss functions of the classification branch, the positioning branch, and the explicit object branch according to the samples respectively, and the loss functions are respectively:

[0034]

[0035]

[0036]

[0037] Wherein, N represents the number of all samples, N pos represents the number of positive samples, respectively represent the predicted values of the category score, coordinate value and objectness value obtained through the network, respectively represent the true values of the category score, coordinate value and objectness value obtained through the network, u1 and u2 are weighting parameters, and CE, SmoothL1 and GIoU are loss functions;

[0038] S43, calculating a total loss function according to the loss functions of the classification branch, the positioning branch and the explicit object branch:

[0039] L 总 =λ1L cls +λ2L reg +λ3L obj ,

[0040] Wherein, λ1, λ2 and λ3 are coefficients;

[0041] S44, training the classification branch, the positioning branch and the explicit object branch according to the total loss function, and obtaining the optimized classification branch, the positioning branch and the explicit object branch.

[0042] Further, the step S41 comprises: traversing each GT true value, calculating the GIoU value of all candidate positions, and taking the average value as the adaptive threshold of the GT; for each GT, selecting the position with the GIoU greater than the adaptive threshold in the candidate position as a positive sample, and the rest as a negative sample;

[0043] In the step S44, the explicit object branch and the positioning branch are trained separately.

[0044] Further, the step S5 further comprises: calculating the classification confidence of the predicted target, and eliminating the predicted target with a classification confidence lower than a threshold, wherein the classification confidence of the predicted target is:

[0045] P cls_end =P cls ×Q obj ,

[0046] Wherein, P cls is the predicted category score, and Q obj is the predicted objectness value.

[0047] The application also discloses a single-layer feature map and explicit object-based anchor-free traffic sign detection system, comprising:

[0048] at least one processor; and at least one memory connected with the processor, wherein:

[0049] The memory stores program instructions executable by the processor, and the processor invoking the program instructions can execute the detection method.

[0050] (Three) beneficial effects

[0051] The above technical solutions of the present application have the following advantages:

[0052] (1) The present application adopts adaptive enhancement processing of images containing exposure, tone and brightness three filters, and adopts a small convolution network to predict the parameters of the filter, which realizes the self-adaptive enhancement of the image while effectively eliminating the influence of the weather light brought by the external environment, and reduces the influence of light on the detection accuracy.

[0053] (2) The present application introduces a channel shift operation MLP block in the top-down path through a single layer feature extractor to learn more effective feature expression, enhance scale correlation, reduce spatial information loss in the down-sampling process, and fully fuse low-resolution feature maps containing rich semantic information and high-resolution feature maps containing rich spatial information through a feature aggregator FA, then through a receptive field expander to adapt to object detection of different scales, make up for the shortcomings of single layer feature maps, generate output features with various receptive field sizes, weight the importance of each channel, learn the correlation between channels, significantly improve the feature extraction and strengthening ability of the model, and reduce the parameter amount, use the attention mechanism to adaptively calibrate the importance of each channel through explicit modeling, and solve the problem of detection scale change while having small memory burden and simple structure.

[0054] (3) The present application improves the conventional detection head, proposes an explicit object branch and separately trains it, supervises the learning of background samples, improves the classification accuracy, and in the prediction reasoning, the product of the prediction values of the explicit object branch and the classification object branch is taken as the classification confidence, then compared with a threshold to screen out the target frame, thereby effectively suppressing negative samples with high confidence, alleviating the influence of sample imbalance on target detection, fully utilizing the features, and improving the detection accuracy.

[0055] (4) The present application proposes a new positioning loss by weighting GIoU and SmoothL1 as loss terms, which not only uses the scale-insensitive IoU loss to regress the target as a whole, considers the correlation between coordinate points, but also adds the SmoothL1 loss to optimize each value individually, which is beneficial to improve the accuracy of the detection method based on dense points.

[0056] (5) The anchor box free traffic sign detection method of the present application does not need anchor box labeling, reduces data labeling cost, and has the input image subjected to scaling processing, which is beneficial to saving the cost of network training. BRIEF DESCRIPTION OF DRAWINGS

[0057] The features and advantages of the present application will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, which are schematic and should not be interpreted in a limiting sense, in which:

[0058] Figure 1 The overall structure diagram of the anchor box free traffic sign detection method based on single layer feature map and explicit object of the embodiment of the present application is shown in the figure.

[0059] Figure 2 The structure diagram of the parameter training subnetwork of the embodiment of the present application is shown in the figure.

[0060] Figure 3 The structure diagram of the single layer feature extractor of the embodiment of the present application is shown in the figure.

[0061] Figure 4 The structure diagram of the feature aggregator FA of the embodiment of the present application is shown in the figure.

[0062] Figure 5 The structure diagram of the MLP block of the embodiment of the present application is shown in the figure.

[0063] Figure 6 The structure diagram of the receptive field expander of the embodiment of the present application is shown in the figure.

[0064] Figure 7 The structure diagram of the detection head branch prediction and loss function optimization of the embodiment of the present application is shown in the figure.

[0065] Figure 8 The flow chart of the dynamic positive and negative sample distribution of the embodiment of the present application is shown in the figure.

[0066] Figure 9 The traffic sign detection effect diagram of the detection method of the embodiment of the present application on the GTSDB dataset is shown in the figure.

[0067] Figure 10 The traffic sign detection effect diagram of the detection method of the embodiment of the present application on the TT100K dataset is shown in the figure.

[0068] Figure 11 The traffic sign detection effect diagram of the detection method of the embodiment of the present application on the CCTSDB dataset is shown in the figure. DETAILED DESCRIPTION

[0069] The specific embodiments of the present application will be further described in detail below in conjunction with the drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0070] Embodiment 1 of this invention is an anchorless traffic sign detection method based on a single-layer feature map and explicit objects, the overall structure of which is as follows: Figure 1 As shown, it includes the following steps:

[0071] S1. Enhance the input original image using the image enhancement module:

[0072] S11. Scale the input original image to a uniform size: 256×256 pixels;

[0073] Scaling the input image can help save on network training costs.

[0074] S12. The scaled image is fed into the parameter training sub-network to train the filter parameters so that the filter is suitable for the enhancement processing of the original image.

[0075] The parameter training subnetwork is as follows: Figure 2 As shown, it includes 5 3×3 convolutional layers and 2 fully connected layers, outputting L+2 filter parameters: one exposure filter parameter E, and L tone filter parameters {t0, t1, ..., t2}. L-1}, the parameter param of a brightness filter.

[0076] S13. Load the parameters of the trained filter into the filter respectively, and perform filtering enhancement processing on the original image through the filter;

[0077] Since traffic sign detection is conducted outdoors, it is significantly affected by lighting conditions. Therefore, the filters selected primarily focus on adjusting the image's color and brightness: exposure, hue, and luminance filters. These filters are differentiable, allowing for adaptive adjustment of hyperparameters to sequentially enhance the image. The exposure and hue filters are pixel-level filters, operating on each individual pixel. The input pixel value I of the filter is set. i =(R i G i B i Output pixel value I o =(R o G o B o (R, G, B) represents the values ​​of the three color channels: red, green, and blue. The filter includes three filters: an exposure filter, a tone filter, and a luminance filter, as detailed below:

[0078] Exposure filter: I o =2 E I i ;

[0079] Wherein, E is a hyper parameter, used to adjust the light and dark degree of the image, when the target in the image is not too bright or too dark, it is moderate exposure, the light of outdoor will affect the exposure degree of the image at any time, therefore, through the adaptive adjustment of the filter, the overexposure problem caused by the non-ideal exposure can be alleviated.

[0080] Tone filter: the tone adjustment of the image is for the lightness of the image, the tone filter is designed as a monotonic and piecewise linear function, containing L parameters {t0, t1, …, t L-1}, in the embodiment, L = 8; in addition, each point on the curve is represented as (k / L, T k / T L ), the prefix and T k in the formula represents the prefix sum of the first k points, t k is the kth tone filter parameter, k and j are both the number of tone filter parameters, i is the channel number, and the mapping function is as follows, wherein the clip(x, min, max) function limits the value in the (min, max) range.

[0081]

[0082] Luminance filter: the brightness of the image refers to the brightness of the picture, first, the value of the pixel on the R, G and B components is multiplied by the corresponding luminance coefficient (here, the default value is used) Lum(Ii) = 0.27Ri + 0.67Gi + 0.06Bi, and then the mapping function is used to obtain the output value.

[0083] I o = (1-param) * I i + param * Lum(I i )

[0084] The original high-resolution image is filtered and enhanced by the filter, transmitted to the subsequent detection network for joint learning, and the parameters are optimized through back propagation. The filter is independent of the resolution, and the training result of the filter parameters is not affected by the low-resolution image.

[0085] S2, input the enhanced image into a feature encoder, the feature encoder includes a single-layer feature extractor and a receptive field expander, to obtain a fused single-layer feature map t5; as shown in Figure 1 ;

[0086] The single-layer feature extractor is used to fully fuse the low-resolution feature map containing rich semantic information and the high-resolution feature map containing rich spatial information, and a residual dilation block is used to generate output features with various receptive field sizes;

[0087] S21, input the enhanced image into a single-layer feature extractor, and continuously down-sample to obtain C1-C5 feature maps;

[0088] The down-sampling process includes: first, using maxpool and 3x3 convolution (stride = 2) to change the size and channel number of the enhanced image of HxWx32, sequentially obtaining C1-C3 feature maps of ; then inputting C3 into an MLP block to perform channel shift operation and learning more rich local features, obtaining C4 feature maps of , and inputting C4 into an MLP block to obtain C5 feature maps of ; C4 and C5 feature maps are low-resolution feature maps containing rich semantic information;

[0089] S22, obtaining P3 feature maps by performing channel shift operation on C3 feature maps, fusing the P3 feature maps and C4 feature maps through a feature aggregator FA, and then performing channel shift operation to obtain P4 feature maps, fusing the P4 feature maps and C5 feature maps through a feature aggregator FA, and then performing channel shift operation and ASPP to obtain P5 feature maps; steps S21 and S22 are as shown in Figure 3 ;

[0090] The structure of the feature aggregator FA (Feature Aggregator) is as shown in Figure 4 . First, the two input feature maps with the same size and channel number are spliced in the channel dimension through 3x3 convolution, second, the average of all pixel values in each channel is calculated using global pooling operation, and the spatial dimension is compressed to obtain a new 1x1 channel map, then two fully connected layers FC are used, one for dimension reduction and one for dimension increase, which reduces the network parameters and adds more nonlinear features; finally, the weight vector obtained by the sigmoid function activation is multiplied with the spliced feature map, and a 1x1 convolution is used to transform back to the original channel number, and the output feature map is obtained. P3, P4 feature maps are high-resolution feature maps containing rich spatial information, and C4 and C5 feature maps are low-resolution feature maps containing rich semantic information, and the P5 feature maps containing the most rich semantic information are obtained by fusing through the feature aggregator FA.

[0091] Since the two feature maps of the feature aggregator FA need to have the same size and channel number, before fusion, P3 and P4 are respectively subjected to 1x1 convolution and 3x3 convolution to make the size and channel number the same as C4 and C5, so as to perform feature map fusion;

[0092] ASPP (atrous spatial pyramid pooling) represents atrous spatial pyramid pooling.

[0093] The structure of the MLP block (Multilayer Perceptron Block) is shown in Figure 5 The input vector F is first mapped to a plurality of tokens through 7x7 convolution (stride = 4), and then the features are moved along the width direction, like a global model to add more locality; then the features are transmitted using a depth separable convolution, and then shifted along the height direction, and finally a residual connection is used to reduce the model complexity to reduce overfitting. The reason for using depth separable convolution is that it not only helps to encode the position information of the features, but also uses fewer parameters, greatly improving the efficiency. Through the setting of the parameters of the depth separable convolution, the MLP block has different processing effects. C3 to C4, C4 to C5 are down-sampled through the MLP block, while C3 to P3, C4 to P4, C5 to P5 are shifted through the MLP block, but the size and the number of channels do not change, and the setting parameters of the two MLPs are inconsistent.

[0094] S23, input the P5 feature map into the receptive field expander, and output the fused single-layer feature map t5; as shown in Figure 6 ;

[0095] In single feature map detection, the single receptive field problem will be encountered, which will cause the model to be more biased towards large or small targets, which is not conducive to detecting datasets with large scale changes. Therefore, we use residual connection and dilated convolution to expand the receptive field to obtain a series of feature maps with rich receptive fields. The receptive field expander includes four residual dilated blocks with hole rates r of 2, 4, 6, and 8, respectively. The structure of the residual dilated block is shown in Figure 6 , which first reduces the dimension through 1x1 convolution to change the number of channels of the feature map, then passes through 3x3 dilated convolution, and finally increases the dimension through 1x1 convolution to ensure that the size of the output feature map remains unchanged, while the receptive field of the convolution kernel is increased through 3x3 dilated convolution, so that the output of each layer contains more range of feature information. The design of the subsequent multiple detection heads is based on the idea of divide and conquer. The high-resolution output head is used for detecting small targets, and the low-resolution output head is used for detecting large targets to adapt to different scale object detection. The fundamental reason for this is that different feature layers correspond to different receptive fields, and different scale targets require different receptive fields. Therefore, the receptive field expander is used to solve this problem, and the feature maps with different receptive fields are fused to enable single-layer feature maps to detect targets of different scales. Through multiple residual dilated blocks with different hole rates, output features with different receptive field sizes are obtained, i.e. the fused single-layer feature map t5.

[0096] Each convolution is composed of a convolution layer, a batch normalization layer, and an activation function.

[0097] S3, three detection head branches are obtained by convolution of the single-layer feature map t5, including a classification branch, a positioning branch and an explicit object branch; as shown in Figure 7

[0098] The positioning branch is used for predicting coordinate information of the detection frame; the classification branch is used for predicting class probability of each pixel on the feature map, and C binary classifiers are trained, wherein C is the number of classes of the data set, and generally 3 classes are divided for the detection data set; the objectness explicit object branch is used for supervised learning of background samples, and is used for predicting the degree of prediction accuracy of each sample, that is, the degree of overlap between the predicted detection frame and the real frame, which is represented by the intersection over union IoU between the predicted frame and the real frame in the positioning branch, and the positive sample is predicted as positive and the negative sample is predicted as negative.

[0099] The network output in the format of (batch, 3, H, W, C+5) is obtained by two 1x1 convolution and one 5x5 convolution of the single-layer feature map t5, wherein the channel number C+5 includes the channel number of the four positioning branches, the channel number of the one explicit object branch and the channel number of the C classification branches, 4 represents the relative position information (x c , y c , w, h) of the detection frame predicted by the positioning branch, (x c , y c ) is the center point coordinate, w and h are the width and height of the boundary frame respectively, 1 represents the objectness value predicted by the explicit object branch, and C represents the C class probabilities of each pixel on the feature map predicted by the classification branch, and in the embodiment, the detection data set is generally divided into 3 classes, C=3, and the network output in the format of (batch, 3, H, W, 8) is obtained.

[0100] S4, loss functions of the classification branch, the positioning branch and the explicit object branch are calculated respectively, and the classification branch, the positioning branch and the explicit object branch are trained and optimized through the loss functions; as shown in Figure 7

[0101] S41, positive and negative sample matching is performed

[0102] As shown in Figure 8 , the GIoU (Generalized Intersection over Union) value of all candidate positions is calculated for each GT (ground truth), and the average value thereof is taken as the adaptive threshold of the GT; for each GT, the position with the GIoU greater than the adaptive threshold in the candidate position is selected as the positive sample for predicting the GT, and the rest is taken as the negative sample.

[0103] ​​S42, calculate the loss functions of the classification branch, the positioning branch and the explicit object branch according to the samples respectively;

[0104] The loss is divided into three parts, which are explicit object loss, target positioning loss and target classification loss. The classification loss and the positioning loss are only calculated for positive samples, and the explicit object loss is calculated for all samples.

[0105] Classification loss: cross-CE loss is used to calculate the cross-entropy loss between the predicted probability of the target belonging to each category and the true target category.

[0106]

[0107] Positioning loss: SmoothL1 and IoU loss are combined and weighted, and the regression of a single value is considered on the basis of the conventional IoU loss to obtain a new positioning loss. The IoU loss used here is GIoU loss. Through experiments, the embodiment sets the weighting parameters u1=5 and u2=2:

[0108]

[0109] Objectness loss: SmoothL1 loss is used when calculating the loss, which is not sensitive to outliers and anomalies, and the gradient change is relatively small, which is better than the ordinary L1 loss in convergence effect. Among them, is the IoU value calculated for the predicted frame and the real frame in the positioning branch.

[0110]

[0111] Among them, N represents the number of all samples, N pos represents the number of positive samples, V pred , represent the predicted values of the category score, coordinate value and objectness value obtained through the network, V gt , represent the true values of the category score, coordinate value and objectness value obtained through the network.

[0112] S43, calculate the total loss function according to the loss functions of the classification branch, the positioning branch and the explicit object branch;

[0113] The total loss L 总 of the network is the sum of the three branch losses. Through experiments, the embodiment sets the coefficients λ1=1, λ2=8 and λ3=16.

[0114] L 总 = λ1L cls + λ2Lreg +λ3L obj

[0115] S44. Train the classification branch, localization branch, and explicit object branch according to the total loss function to obtain the optimized classification branch, localization branch, and explicit object branch;

[0116] The explicit object branch and the localization branch can be trained separately or jointly. However, in the scenario of this embodiment, the model performs better when the explicit object branch and the localization branch are trained separately, so the explicit object branch and the localization branch are trained separately.

[0117] S5. The predicted target is obtained by predicting through the optimized classification branch, localization branch and explicit object branch. The classification confidence of the predicted target is calculated by multiplying the predicted value of the explicit object branch and the predicted value of the localization branch. The predicted targets with a classification confidence of less than the threshold are removed.

[0118] An objectness branch is introduced to supervise learning of foreground and background samples. Its output value represents the accuracy of prediction for each sample, with positive samples predicted as positive and negative samples predicted as negative. Figure 7 As shown, the final classification confidence score is defined as: P cls_end It is the classification confidence score obtained after post-processing, P cls It is the predicted classification score, Q obj It is the predicted objectness value.

[0119] P cls_end =P cls ×Q obj

[0120] During inference, the classification confidence of the final target is equal to the predicted classification score multiplied by the predicted objectness value. This can suppress negative samples with high classification scores and alleviate the problem of sample imbalance.

[0121] To verify the traffic sign detection performance, the proposed method was compared with other existing methods on three representative traffic sign datasets: GTSDB, TT100K, and CCTSDB2021, as shown in Tables 1-3. The detection results of the proposed method on the GTSDB, TT100K, and CCTSDB2021 traffic sign datasets are also shown in the figures. Figures 9-11 As shown, the effectiveness of the proposed method has been verified, and good results have been achieved. In terms of accuracy, it is comparable to most mainstream target detection algorithms, while also having a lower computational load and parameter count.

[0122] Table 1 Performance comparison on GTSDB

[0123]

[0124] Table 2 Performance comparison on TT100K

[0125]

[0126] Table 3 Performance comparison on CCTSDB2021

[0127]

[0128] Finally, it should be noted that the above detection method can be converted into software program instructions, which can be implemented by using a detection system including a processor and a memory, or by computer instructions stored in a non-transitory computer readable storage medium. The integrated unit implemented in the form of software function unit described above can be stored in a computer readable storage medium. The software function unit described above is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0129] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the embodiments of the present application are described in combination with the drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An anchor-free traffic sign detection method based on single-layer feature map and explicit object, characterized in that, The method comprises the following steps: S1, image enhancement is performed on the input original image; S2, the enhanced image is input into a single-layer feature extractor and a receptive field expander to obtain a fused single-layer feature map; The step S2 comprises: S21, the enhanced image is input into the single-layer feature extractor to obtain C1-C5 feature maps through continuous downsampling; The step S21 comprises: the enhanced image is changed in size and channel number through maximum pooling and convolution to obtain C1-C3 feature maps in sequence; then the C3 feature map is input into an MLP block to perform a channel shift operation to obtain a C4 feature map which also changes in size and channel number, and similarly, the C4 feature map is input into the MLP block to obtain a C5 feature map; S22, the C3 feature map is subjected to a channel shift operation to obtain a P3 feature map, the P3 feature map and the C4 feature map are fused through a feature aggregator FA, and then subjected to a channel shift operation to obtain a P4 feature map, the P4 feature map and the C5 feature map are fused through the feature aggregator FA, and then subjected to a channel shift operation and ASPP to obtain a P5 feature map; In the step S22, before being fused through the feature aggregator FA, it further comprises: convolving the two feature maps fused through the feature aggregator FA into a feature map which is the same in size and channel number; S23, the P5 feature map is input into the receptive field expander to output the fused single-layer feature map t5; In the step S23, the receptive field expander comprises a plurality of residual dilation blocks with different hole rates; S3, three detection head branches including a classification branch, a positioning branch and an explicit object branch are obtained through convolution from the single-layer feature map; Step S3 includes: processing the single-layer feature map t5 through two 1×1 convolutions and one 5×5 convolution to obtain a network output in the format (batch, 3, H, W, C+5), where the number of channels C+5 includes the number of channels for 4 localization branches, the number of channels for 1 explicit object branch, and the number of channels for C classification branches. The relative position information (x, y, y) of the detection boxes predicted by the localization branches is also included. c y c ,w,h),(x c y c ) represents the center point coordinates, w and h represent the width and height of the detection box, respectively, the objectness value predicted by the explicit object branch, and the C class probabilities of each pixel on the feature map predicted by the classification branch. S4, loss functions of the classification branch, the positioning branch and the explicit object branch are calculated respectively, and the classification branch, the positioning branch and the explicit object branch are trained and optimized through the loss functions; S5, a prediction target is obtained through prediction by the optimized classification branch, positioning branch and explicit object branch.

2. The single-layer feature map and explicit object based anchor-free traffic sign detection method according to claim 1, characterized in that, The step S1 comprises: S11, the input original image is scaled to a uniform size; S12, the scaled image is transmitted into a parameter training subnetwork to train parameters of a filter; S13, the trained parameters of the filter are loaded into the filter respectively, and the original image is filtered and enhanced through the filter.

3. The single-layer feature map and explicit object based anchor-free traffic sign detection method according to claim 2, characterized in that, The parameter training subnetwork comprises 5 convolution layers and 2 fully connected layers; the filter comprises an exposure filter, a tone filter and a brightness filter: Exposure filter: I o = 2 E I i ; Tone filter: Luma filter: I o = (1 - param) * I i + param * Lum(I i ); wherein E is a parameter of the exposure filter, t k ∈ {t0,t1,…,t L-1} is a parameter of the tone filter, L is a total number of parameters of the tone filter, k and j are both numbers of parameters of the tone filter, i is a channel number, the clip(x,min,max) function represents limiting a value in a range of (min,max), param is a parameter of the luminance filter, Lum(I i ) represents multiplying values of a pixel on R, G, B three components by corresponding luminance coefficients, I i represents an input pixel value of the filter, I o represents an output pixel value of the filter.

4. The single-layer feature map and explicit object based anchor-free traffic sign detection method according to claim 1, characterized in that, The step S4 comprises: S41, positive and negative sample matching is performed S42, loss functions of the classification branch, the positioning branch and the explicit object branch are calculated according to the samples respectively, and are respectively: wherein N represents the number of all samples, N pos represents the number of positive samples, Vpred、 respectively represent the predicted values of the category score, coordinate value and objectness value obtained through the network, V gt 、 respectively represent the true values of the category score, coordinate value and objectness value obtained through the network, u1 and u2 are weighting parameters, and CE, SmoothL1 and GIoU are loss functions. S43, a total loss function is calculated according to the loss functions of the classification branch, the positioning branch and the explicit object branch: L 总 = λ1L cls + λ2L reg + λ3L obj , Wherein, λ1, λ2, λ3 are coefficients; S44, the classification branch, the positioning branch and the explicit object branch are trained according to the total loss function to obtain the optimized classification branch, positioning branch and explicit object branch.

5. The single-layer feature map and explicit object based anchor-free traffic sign detection method according to claim 4, characterized in that, The step S41 comprises: traversing each GT ground truth value, calculating the GIoU value of all candidate positions, and taking the average value as the adaptive threshold of the GT; for each GT, selecting the position with a GIoU greater than the adaptive threshold from the candidate positions as a positive sample, and the rest as negative samples. In the step S44, the explicit object branch and the positioning branch are trained separately.

6. The single-layer feature map and explicit object based anchor-free traffic sign detection method according to claim 1, characterized in that, The step S5 further comprises: calculating the classification confidence of the prediction target, and eliminating the prediction target with a classification confidence lower than a threshold; the classification confidence of the prediction target: P cls_end = P cls × Q obj , where P cls is the predicted classification score, Q obj is the predicted objectness value.

7. An anchor box-free traffic sign detection system based on single-layer feature maps and explicit objects, characterized in that, comprises: at least one processor; and at least one memory connected to the processor in communication, wherein: the memory stores program instructions executable by the processor, and the processor invoking the program instructions can execute the detection method according to any one of claims 1 to 6.

Images