Traffic sign detection method and apparatus, and storage medium

Abstract

Disclosed in the present invention is a traffic sign detection method, comprising: acquiring a traffic road image to be detected; preprocessing the traffic road image; inputting the preprocessed traffic road image into a pre-trained traffic sign detection model to obtain a classification result output by the model; marking a detected traffic sign on the basis of the classification result from the traffic sign detection model, and outputting an image of the marked traffic sign. The traffic sign detection model uses an improved RT-DETR model in which on the basis of an RT-DETR model, layers 5-7 are replaced with three downsampling feature extraction layers of a feature learning fusion module DualBlocks, wherein each replaced layer consists of a DualConv module, an average pooling module and an ReLu linear activation function. Layers 11 and 16 of an RT-DETR network model are separately replaced with a dynamic upsampling layer of a dynamic upsampling operator Dysample, wherein the dynamic upsampling layer consists of a sampling point generator, a sampling apparatus and an interpolation function. The present invention can reduce the number of parameters while improving the detection accuracy.

Description

A traffic sign detection method, device, and storage medium Technical Field

[0001] This invention belongs to the field of traffic scenarios, and specifically relates to a traffic sign detection method, device and storage medium. Background Technology

[0002] Traffic sign detection is a crucial computer vision task designed to automatically identify and locate traffic signs in images or videos to improve traffic safety and intelligent traffic management. This technology is widely used in autonomous driving and advanced driver assistance systems (ADAS) to detect potential risks and is an important component of intelligent transportation. In real-world environments, traffic signs are often small, have complex backgrounds, and vary in size, making identification difficult. Drivers may miss or misjudge signs, seriously impacting vehicle safety. Therefore, rapid and accurate traffic sign identification has become one of the challenges and key focuses in traffic management tasks.

[0003] Current target detection algorithms for traffic scenarios are mainly divided into two stages. The first stage is based on traditional manual feature detection algorithms, which mainly distinguish traffic signs by their shape and color. The second stage is based on deep learning detection algorithms, which learn features through convolutional neural networks. These algorithms can automatically learn features and have strong feature representation capabilities and generalization, making them applicable to various scenarios. They are also faster and more accurate.

[0004] Deep learning-based object detection algorithms are mainly divided into two categories. The first category generates candidate regions and then performs classification, such as YOLO, SSD, and CNN. The second category achieves object detection in an end-to-end manner, such as the DETR series. DETR introduces a Transformer structure, eliminating manual prior knowledge and simplifying the network structure. By incorporating a self-attention mechanism during the encoding process, global semantics are enhanced, resulting in good performance for large object detection. However, DETR still faces challenges such as poor detection performance for small objects, long training time, and difficulty in convergence. Summary of the Invention

[0005] The purpose of this invention is to provide a traffic sign detection method. Based on the RT-DETR network model, a traffic sign detection model is constructed to recognize traffic signs in different traffic scenarios, thereby improving the accuracy of the recognition results. This invention is achieved through the following technical solutions.

[0006] In a first aspect, the present invention provides a traffic sign detection method, comprising:

[0007] Acquire images of the traffic roads to be detected;

[0008] Preprocess traffic and road images;

[0009] The preprocessed traffic road image is input into a pre-trained traffic sign detection model to obtain the classification result output by the model;

[0010] The detected traffic signs are labeled based on the classification results of the traffic sign detection model, and the labeled traffic sign images are output.

[0011] The traffic sign detection model adopts an improved RT-DETR network model. The improved RT-DETR network model is based on the RT-DETR network model, replacing layers 5-7 with downsampling feature extraction layers of the three-layer feature learning fusion module DualBlocks. Each replacement layer consists of a DualConv module, an average pooling module, and a ReLU linear activation function. Layers 11 and 16 of the RT-DETR network model are replaced with a dynamic upsampling layer of the dynamic upsampling operator Dysample, which consists of a sampling point generator, a sampling device, and an interpolation function.

[0012] Optionally, the feature learning fusion module DualBlocks includes a first branch and a second branch. The first branch includes multiple serially connected DualConv modules, and the second branch includes a serially connected DualConv module and an average pooling module.

[0013] The feature learning fusion module DualBlocks performs the following specific operations:

[0014] In the first branch, the feature map with input height H, width W, and number of channels C is subjected to multiple DualConv convolutions;

[0015] In the second branch, the feature map with input height H, width W, and number of channels C is sequentially subjected to DualConv convolution and average pooling;

[0016] The feature maps output from the first and second branches are concatenated together and then processed using the ReLU linear activation function to obtain a feature map with a height of H / 2, a width of W / 2, and 2C channels.

[0017] Optionally, the DualConv module performs the following specific operations:

[0018] The input features are divided into G groups, the total number of convolutional filters is N, the total number of input feature channels is M, the input feature mapping channels are divided into G M / G groups, and the convolutional filters are divided into G N / G groups.

[0019] Each N / G filter contains G M / G input feature map channels. Sequentially, 3×3 and 1×1 convolution operations are performed in parallel on the input feature maps of one M / G input feature map channel under each N / G filter. Simultaneously, the input feature maps in other channels undergo only 1×1 convolution operations. Finally, the outputs of the resulting grouping scheme are superimposed to obtain the final convolution result. DualConv is a dual convolution scheme that combines the advantages of grouped convolution and non-uniform convolution.

[0020] Optionally, the dynamic upsampling operator Dysample performs the following specific operations:

[0021] For the input feature map,

[0022] In the first branch, the input feature map is first processed by a sampling point generator to select a suitable sampling point generation method, and then the sampling device controls the specific size of the feature map to be generated to determine the number of sampling points to be inserted.

[0023] In the second branch, the size of the input feature map is enlarged, the features of the original image are placed in the corresponding positions, and the positions of the sampling points to be inserted are left empty;

[0024] Finally, the generated sampling points from these two branches are inserted into their corresponding positions using an interpolation function to generate the upsampled features of the image. The dynamic upsampling operator Dysample customizes upsampling from the perspective of point sampling, which is more resource-efficient and easier to implement.

[0025] Optionally, the sampling point generator includes a static sampling point generator and a dynamic sampling point generator;

[0026] The static sampling point generator controls the generation of static range factors. The static range factors are generated by first processing the input features through C input and 2s output channels. 2 The linear layer generation size is 2s 2 The offset O is ×H×W, where H and W are the height and width of the input feature map, respectively, and s represents the size change factor in the dynamic upsampling operator Dysample. The offset O is multiplied by a specific parameter, and then its size is reshaped to 2×sH×sW through pixel shuffling. Finally, the reshaped offset O and the original raster G are combined to generate the sampling set δ.

[0027] The dynamic sampling point generator controls the generation of the dynamic range factor. The dynamic range factor first passes through the first branch with input and output channels numbered C and 2s, respectively. 2 Linear layers are used to generate a size of 2s 2The offset O of ×H×W is then used to generate the offset within the theoretical boundary conditions through the activation function Sigmoid and the coefficient value of 0.5. After being combined with the offset O of the second branch, its size is reshaped to 2×sH×sW through pixel shuffling. Finally, the reshaped offset O and the original raster G are combined to generate the sampling set δ.

[0028] Optionally, the dynamic upsampling operator Dysample achieves point sampling by controlling the initial sampling position, adjusting the offset, and grouping the upsampling process.

[0029] Optionally, the dynamic upsampling operator Dysample controls the initial sampling position and adjusts the offset by controlling the static initial sampling position and adjusting the offset, and by controlling the dynamic initial sampling position and adjusting the offset.

[0030] The process of controlling the static initial sampling position and adjusting the offset includes separating the sampling points and generating sampling points using a fixed offset distance;

[0031] The process of controlling the dynamic initial sampling position and adjusting the offset includes generating sampling points by using dynamic offset distance and local offset, selecting to evenly distribute multiple sampling points in a grid, and setting the offset of the sampling points to a uniform standard.

[0032] The upsampling process is grouped into multiple groups along the channel dimension. Each group performs the calculation of controlling the initial sampling position and adjusting the offset to obtain the sampling points of that group. The sampling points of each group are then combined to obtain the point sampling result.

[0033] Optionally, the training method for the traffic sign detection model includes,

[0034] Obtain traffic sign image samples and annotation information from different traffic environments;

[0035] The image samples are divided into training set samples and test set samples;

[0036] Image samples are preprocessed to determine the annotation information for traffic sign images;

[0037] The preprocessed training set samples and test machine samples are input into the pre-built RT-DETR network to train the RT-DETR network;

[0038] The classification accuracy of the trained RT-DETR network is tested using test set samples. If the classification accuracy reaches the preset accuracy requirement, training stops; otherwise, the RT-DETR network continues to be trained using training set samples.

[0039] In a second aspect, the present invention provides a traffic sign detection device, comprising:

[0040] The image acquisition module is configured to acquire images of the traffic roads to be detected.

[0041] The preprocessing module is configured to preprocess traffic road images;

[0042] The image detection module is configured to input preprocessed traffic road images into a pre-trained traffic sign detection model to obtain the classification results output by the model.

[0043] In addition, the detection result output module is configured to label the detected traffic signs according to the classification results of the traffic sign detection model and output the labeled traffic sign images;

[0044] The traffic sign detection model adopts an improved RT-DETR network model. The improved RT-DETR network model is based on the RT-DETR network model, replacing layers 5-7 with downsampling feature extraction layers of the three-layer feature learning fusion module DualBlocks. Each replacement layer consists of a DualConv module, an average pooling module, and a ReLU linear activation function. Layers 11 and 16 of the RT-DETR network model are replaced with a dynamic upsampling layer of the dynamic upsampling operator Dysample, which consists of a sampling point generator, a sampling device, and an interpolation function.

[0045] Thirdly, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, having a computer-executable instruction stored thereon, which, when executed by a processor, implements the steps of the pass flag detection method as described in the first aspect. Beneficial effects

[0046] This invention is based on the RT-DETR network model. First, it introduces a feature learning fusion module, DualBlocks, which combines multiple small convolutional kernels and grouped convolutions to enable the extraction network to help deeper convolutional layers effectively extract information from shallow layers, while reducing the number of network parameters and improving accuracy. Second, it introduces a dynamic upsampling operator, Dysample, to recover upsampled features from the perspective of point sampling, which saves more resources. Finally, it constructs a traffic sign detection model based on the improved RT-DETR network model, which has fewer network parameters and less computation compared to the original RT-DETR network model, while maintaining higher accuracy and detection speed. Attached Figure Description

[0047] Figure 1 is a flowchart of the traffic sign detection method of this invention;

[0048] Figure 2 is a structural diagram of the feature learning fusion module DualBlocks used in the method of the present invention;

[0049] Figure 3 is a schematic diagram of the principle of DualConv in the feature learning fusion module DualBlocks;

[0050] Figure 4 shows the structure of the dynamic upsampling operator Dysample.

[0051] Figure 5 is a schematic diagram of the sampling point generator in the dynamic upsampling operator;

[0052] Figure 6 is a schematic diagram of the static initial sampling point settings and offset distance;

[0053] Figure 7 shows a schematic diagram of dynamic initial sampling point settings and offset distance.

[0054] Figure 8 is a schematic diagram of the improved RT-DETR network model structure of the present invention;

[0055] Figure 9 shows the detection effect of the improved RT-DETR network model of the present invention. Detailed Implementation

[0056] The following description, in conjunction with the accompanying drawings and specific embodiments, provides further details. In this description, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature.

[0057] Example 1: This example describes a method for detecting traffic signs, including:

[0058] Acquire the image of the traffic sign to be detected;

[0059] Preprocess the traffic sign images;

[0060] The preprocessed traffic sign image is input into a pre-trained traffic sign detection model to obtain the classification result output by the model.

[0061] The detected traffic signs are labeled based on the classification results of the traffic sign detection model, and the labeled traffic sign images are output.

[0062] The traffic sign detection model adopts an improved RT-DETR network model. The improved RT-DETR network model is based on the RT-DETR network model, replacing layers 5-7 with downsampling feature extraction layers of the three-layer feature learning fusion module DualBlocks. Each replacement layer consists of a DualConv module, an average pooling module, and a ReLU linear activation function. Layers 11 and 16 of the RT-DETR network model are replaced with a dynamic upsampling layer of the dynamic upsampling operator Dysample, which consists of a sampling point generator, a sampling device, and an interpolation function.

[0063] In practical applications, the overall process is shown in Figure 1. First, a traffic sign dataset is selected. The dataset is cleaned to address the uneven distribution of samples. Then, a baseline network model is selected and improved. The improved network model is used to train and evaluate the preprocessed dataset. The optimal network model is determined based on the loss function value and the accuracy of recognition. Finally, the results are evaluated to obtain the final result.

[0064] This embodiment mainly involves the following contents.

[0065] I. Data Acquisition and Processing

[0066] The dataset selected for this invention is the TT100K traffic sign dataset, a public dataset containing 221 traffic sign categories. However, due to the uneven distribution of signs, many categories have a low number of instances. This invention selects traffic sign categories with more than 50 instances per category and cleans the dataset to address the uneven distribution. Finally, 7963 images from 45 categories were selected and split into training and testing sets in a 4:1 ratio. Model training and testing were then conducted based on these sets.

[0067] II. Model Construction and Training

[0068] 2.1 Construction of Traffic Sign Detection Model

[0069] In this embodiment, the traffic sign detection model adopts an improved RT-DETR network model, as shown in Figure 8. Based on the RT-DETR network model, a three-layer feature learning and fusion module, DualBlocks, is added between layers 4 and 8 to perform downsampling feature extraction. Each layer consists of a DualConv module, an average pooling module, and a ReLU linear activation function. A dynamic upsampling operator, Dysample, is added between layers 10 and 11 of the RT-DETR network model. This layer consists of a sampling point generator, a sampling device, and an interpolation function. A dynamic upsampling operator, Dysample, is also added between layers 15 and 17 of the RT-DETR network model.

[0070] As shown in Figure 2, the feature learning fusion module DualBlocks includes a first branch and a second branch. The first branch includes multiple serially connected DualConv modules, and the second branch includes a serially connected DualConv module and an average pooling module. The feature learning fusion module DualBlocks performs the following specific operations: In the first branch, multiple DualConv convolutions are performed on the feature map with input height H, width W, and number of channels C; In the second branch, DualConv convolutions and average pooling are performed sequentially on the feature map with input height H, width W, and number of channels C; Finally, the feature maps output from these two branches are concatenated together and the ReLU linear activation function is used to obtain a feature map with height H / 2, width W / 2, and number of channels 2C.

[0071] Replacing the strided convolution in the original RT-DETR network model with DualConv convolution is a dual convolution scheme that combines the advantages of grouped convolution and non-uniform convolution. DualConv convolution performs the following specific operations, as shown in Figure 3: the input features are divided into G groups, the total number of convolutional filters is N, and the total number of input feature channels is M. The input feature mapping channels are divided into G M / G groups, and the convolutional filters are divided into G N / G groups. Each N / G filter contains G M / G input feature mapping channels. Sequentially, a 3×3 convolution kernel and a 1×1 convolution kernel are performed in parallel on the input feature maps in one M / G input feature mapping channel under each N / G filter. Simultaneously, the input feature maps in other channels only undergo a 1×1 convolution kernel. This preserves the original information of the feature maps, helping deeper convolutional layers extract more effective features. Through this grouped convolution, each convolutional filter can extract information from only 1 / G input feature channels. Finally, the outputs of the grouped scheme are sequentially superimposed to obtain the final convolution result.

[0072] The original network model used the nearest neighbor interpolation algorithm for upsampling. Nearest neighbor interpolation simply copies pixel values, which can lead to jagged edges or distortion in the image and easily result in information loss during upsampling. This invention uses the dynamic upsampling operator Dysample for upsampling feature recovery, customizing the upsampling process from a point sampling perspective, which is more resource-efficient and easier to implement. The dynamic upsampling operator Dysample upsamples the encoded image features, changing the image size from H in height and W in width to 2H in height and 2W in width, while maintaining the same number of channels. During this size change, the generated sampling points are more correlated with surrounding nodes. As shown in Figure 4, the dynamic upsampling operator Dysample performs the following specific operations: For the input feature map x, in the first branch, the input feature map is first processed by a sampling point generator to select a suitable sampling point generation method, determine the values ​​of the generated sampling points, and then the sampling device controls the specific size of the feature map to be generated to ensure the expected image size and determine the number of sampling points to be inserted; in the second branch, the size of the input feature map is enlarged, the features of the original image are placed in the corresponding positions to ensure that they are not distorted, and the positions of the sampling points to be inserted are left empty; finally, the generated sampling points are inserted into the corresponding positions through the interpolation function, generating the upsampled feature x of the image. ’ In Figure 4, s represents the size variation factor in the dynamic upsampling operator Dysample, and in this embodiment, s is set to 2.

[0073] The sampling point generator in Figure 4 includes a static sampling point generator and a dynamic sampling point generator, as shown in Figure 5.

[0074] The static sampling point generator controls the generation of static range factors. The static range factors are generated as input features, first processed by C input and 2s output channels respectively. 2 The linear layer generation size is 2s 2 The offset O is ×H×W, where H and W are the height and width of the input feature map, respectively, and s represents the size change factor in the dynamic upsampling operator Dysample. In this embodiment, s is set to 2. The offset O is multiplied by a specific parameter 0.25, which exactly satisfies the theoretical boundary condition between overlap and non-overlap. Then, its size is reshaped to 2×sH×sW through pixel shuffling. Finally, the reshaped offset O and the original raster G are combined to generate the sampling set δ.

[0075] The dynamic sampling point generator controls the generation of the dynamic range factor. The dynamic range factor generation first passes through the first branch with input and output channels of C and 2s respectively. 2 Linear layers are used to generate a size of 2s 2The offset O of ×H×W is then used to generate the offset within the theoretical boundary conditions through the activation function Sigmoid and the coefficient value of 0.5. In Figure 5, б is the Sigmoid function. After being combined with the offset O of the second branch, its size is reshaped to 2×sH×sW through pixel shuffling. Finally, the reshaped offset O and the original raster G are combined to generate the sampling set δ.

[0076] The sampling point generator prioritizes geometrically isolated points, making the algorithm particularly suitable for non-uniformly distributed data features. In this invention, the sampling point generator can make the sampling points on the image connect more smoothly, reducing jagged edges and noise amplification.

[0077] The dynamic upsampling operator Dysample achieves point sampling by controlling the initial sampling position, adjusting the offset, and grouping the upsampling process. The initial sampling position involves setting the sampling points at the same location. Adjusting the offset range results in different offsets for the upsampled values. This ignores the positional relationships of neighboring points, leading to uneven distribution of the initial sampling positions. If the offset is zero within a certain range, it can cause tonal distortion in the sampled image. The static range factor determines the initial sampling point settings and offset distance, as shown in Figure 6. It separates the sampling point positions and uses a fixed offset distance to generate sampling points. This allows the generated sampling points to have a larger receptive field, acquiring more information, and is suitable for local large feature restoration. The dynamic range factor determines the initial sampling point settings and offset distance, as shown in Figure 7. It uses dynamic offset distance and local bias to generate sampling points. Four sampling points are evenly distributed in a grid, and the offsets of the sampling points are set to a uniform standard to prevent overlapping and cross-interference. This reduces the noise impact from a larger receptive field and is suitable for local small feature restoration. The upsampling process involves grouping the feature map into g groups along the channel dimension and generating g group offsets. Each group of channel features undergoes the steps shown in Figure 4 to obtain the upsampled image of that channel feature. Finally, the feature images generated from each group are combined sequentially to obtain the final upsampled feature map.

[0078] 2.2 Training of the traffic sign detection model

[0079] The pre-defined training set is input into the improved RT-DETR network model for training. After each training round, the test set is input into the model for evaluation. This process is repeated to continuously optimize the parameters of the improved RT-DETR network model, ultimately resulting in the fully trained model.

[0080] 2.3 Experiment

[0081] The training model in this embodiment is based on the Linux operating system, uses Python 3.8 as the programming language, PyTorch 2.0.0 as the deep learning framework, and CUDA 11.8. The system is trained on a 4090 GPU with 24GB of memory, with 200 training epochs, a batch size of 24, using the Adam optimizer, and an initial learning rate of 1×10⁻⁶. -4 .

[0082] The average accuracy, number of parameters, computational cost, and detection speed of the statistically improved model on the test set are compared with those of the basic RT-DETR network model. The results are shown in Table 1 below.

[0083] Where mAP refers to the average precision under different recall rates, here it is mAP@0.5. mAp(samll) refers to the average precision under different recall rates generated by the COCO tool for small targets. The number of parameters refers to the total number of parameters generated by the overall network model through convolutional layers, pooling layers, etc. FLOPS refers to the number of parameters calculated by the overall network model through convolutional layers, pooling layers, etc. Here, G in GFLOPS means that the number of parameters is in GB. FPS refers to the frame rate of image processing, which is calculated based on the speed of processing each image.

[0084] Table 1 Comparison results of adding evaluation indicators for different modules

[0085] As shown in Table 1, compared with the original RT-DETR network, the improved network of this invention, by adding the DySample module, only slightly increases the number of model parameters, while improving the recognition accuracy and detection speed by 10 frames. After adding DualConv convolution, the number of model parameters decreases by 4M, and the computational cost also decreases by 7.3, while ensuring that the recognition speed is not reduced. The accuracy of mAp on small target objects was calculated using the COCO toolkit. The improved modules effectively improve the detection accuracy of small targets, demonstrating the superiority of the improved model.

[0086] III. Practical Applications

[0087] The traffic road image to be identified is input into the trained model to identify traffic signs in the image, thus verifying the effectiveness of the traffic sign detection model. Figure 9 shows the traffic sign marking results identified by the improved model on the traffic road image. If the model detects traffic signs in the traffic road image, it marks the traffic signs with rectangular boxes and saves the output image.

[0088] Example 2, based on the same inventive concept as Example 1, introduces a traffic sign detection device, comprising:

[0089] The image acquisition module is configured to acquire images of the traffic roads to be detected.

[0090] The preprocessing module is configured to preprocess traffic road images;

[0091] The image detection module is configured to input preprocessed traffic road images into a pre-trained traffic sign detection model to obtain the classification results output by the model.

[0092] In addition, the detection result output module is configured to output the labeled traffic sign image based on the classification result of the traffic sign detection model;

[0093] The traffic sign detection model adopts an improved RT-DETR network model. The improved RT-DETR network model is based on the RT-DETR network model. Between layers 4 and 8, a three-layer feature learning and fusion module, DualBlocks, is added as a downsampling feature extraction layer. Each layer consists of a DualConv module, an average pooling module, and a ReLU linear activation function. Between layers 10 and 11 of the RT-DETR network model, a dynamic upsampling operator, Dysample, is added as an upsampling layer. This layer consists of a sampling point generator, a sampling device, and an interpolation function. Between layers 15 and 17 of the RT-DETR network model, a dynamic upsampling operator, Dysample, is also added as an upsampling layer.

[0094] Example 3, based on the same inventive concept as Examples 1 and 2, introduces a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the traffic sign detection method as described in Example 1 or 2.

[0095] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0099] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A traffic sign detection method characterized by, The method comprises the following steps: acquiring a traffic road image to be detected; preprocessing the traffic road image; inputting the preprocessed traffic road image into a pre-trained traffic sign detection model to obtain a classification result output by the model; labeling the detected traffic sign according to the classification result of the traffic sign detection model, and outputting a labeled traffic sign image; wherein the traffic sign detection model adopts an improved RT-DETR network model, the improved RT-DETR network model is based on the RT-DETR network model, the 5th-7th layers are replaced with a down-sampling feature extraction layer of a three-layer feature learning fusion module DualBlocks, each layer of the replacement is composed of a DualConv module, an average pooling module and a ReLu linear activation function, the 11th layer and the 16th layer of the RT-DETR network model are replaced with a dynamic up-sampling layer of a dynamic up-sampling operator Dysample, the layer is composed of a sampling point generator, a sampling device and an interpolation function.

2. The method of claim 1, wherein, The feature learning fusion module DualBlocks comprises a first branch and a second branch, the first branch comprises a plurality of serially connected DualConv modules, and the second branch comprises a serially connected DualConv module and an average pooling module; The feature learning fusion module DualBlocks performs the following specific operations: in the first branch, a plurality of DualConv convolutions are performed on the input feature map with a height of H, a width of W and a channel number of C; in the second branch, DualConv convolution and average pooling are sequentially performed on the input feature map with a height of H, a width of W and a channel number of C; the feature maps output by the first branch and the second branch are spliced together and subjected to ReLu linear activation function to obtain a feature map with a height of H / 2, a width of W / 2 and a channel number of 2C.

3. The method of claim 2, wherein, The DualConv module performs the following specific operations: input features are divided into G groups, the total number of convolution filters is N, the total number of input feature mapping channels is M, the input feature mapping channels are divided into G M / G, and the convolution filters are divided into G N / G; each N / G filter contains G M / G input feature mapping channels, 3*3 convolution kernel convolution operation and 1*1 convolution kernel convolution operation are performed on the input feature map in each M / G input feature mapping channel under each N / G filter in sequence, while the input feature map in other channels only performs 1*1 convolution kernel convolution operation; finally, the output results of the grouping scheme are sequentially superimposed to obtain the final convolution result.

4. The method of claim 1 wherein, The dynamic up-sampling operator Dysample performs the following specific operations: for the input feature map, in the first branch, the input feature map is first subjected to the sampling point generator to select a suitable sampling point generation method, and then subjected to the sampling device to control the specific size of the feature map to be generated, to determine the number of sampling points to be inserted; in the second branch, the size of the input feature map is enlarged, the features of the original picture are placed in the corresponding position, and the sampling point position to be inserted is left empty; Finally, the two branches insert the generated sampling points into the corresponding positions through an interpolation function to generate the up-sampled features of the picture.

5. The method of claim 4, wherein, The sampling point generator comprises a static sampling point generator and a dynamic sampling point generator. The static sampling point generator controls the generation of static range factors. The static range factors are generated by first processing the input features through C input and 2s output channels. 2 The linear layer generation size is 2s 2 The offset O is ×H×W, where H and W are the height and width of the input feature map, respectively, and s represents the size change factor in the dynamic upsampling operator Dysample. The offset O is multiplied by a specific parameter, and then its size is reshaped to 2×sH×sW through pixel shuffling. Finally, the reshaped offset O and the original raster G are combined to generate the sampling set δ. The dynamic state sampling point generator controls the generation of a dynamic range factor, which is first generated by a linear layer with input and output channel numbers of C and 2s 2 respectively, to generate an offset O with a size of 2s 2 ×H×W, then pass through an activation function Sigmoid and a coefficient value of 0.5 to generate an offset within the theoretical boundary condition, and then combine the offset O of the second branch after passing through the pixel shuffle to reshape its size to 2×sH×sW, and finally combine the offset O after size reshaping and the original grid G to generate a sampling set δ.

6. The method of claim 4 wherein, The dynamic up-sampling operator Dysample realizes point sampling by controlling initial sampling positions, adjusting offset distances, and grouping up-sampling processes.

7. The method of claim 6, wherein, The dynamic up-sampling operator Dysample realizes point sampling by controlling initial sampling positions, adjusting offset distances, and grouping up-sampling processes. The controlling of the static initial sampling positions and the adjusting of the offset distances comprise separating the positions of the sampling points and generating the sampling points by using fixed-size offset distances. The controlling of the dynamic initial sampling positions and the adjusting of the offset distances comprise generating the sampling points by using dynamic offset distances and local bias, and selecting to uniformly distribute multiple sampling points in a grid, and setting the offset distances of the sampling points to a uniform standard. The grouping of the up-sampling processes comprises dividing feature maps along a channel dimension into multiple groups, performing the controlling of the initial sampling positions and the adjusting of the offset distances on each group respectively, obtaining sampling points of the group, and combining the sampling points of the groups to obtain a point sampling result.

8. The method of claim 7, wherein, The traffic sign detection model training method comprises, acquiring traffic sign image samples and annotation information from different traffic environments; dividing the image samples into training set samples and test set samples; preprocessing the image samples to determine traffic sign image annotation information; inputting the preprocessed training set samples and test set samples into a pre-constructed RT-DETR network to train the RT-DETR network; stopping the training if the classification accuracy of the trained RT-DETR network reaches the preset accuracy requirement, otherwise continuing to train the RT-DETR network by using the training set samples.

9. A traffic sign detection apparatus characterized by comprising: comprises: an image acquisition module configured to acquire a traffic road image to be detected; a preprocessing module configured to preprocess the traffic road image; an image detection module configured to input the preprocessed traffic road image into a pre-trained traffic sign detection model to obtain a classification result output by the model; and a detection result output module configured to mark the detected traffic sign according to the traffic sign detection model classification result and output the marked traffic sign image; wherein the traffic sign detection model adopts an improved RT-DETR network model, the improved RT-DETR network model is based on the RT-DETR network model, the 5th-7th layers are replaced with a down-sampling feature extraction layer of a three-layer feature learning fusion module DualBlocks, each layer of the replacement is composed of a DualConv module, an average pooling module and a ReLu linear activation function, the 11th layer and the 16th layer of the RT-DETR network model are replaced with a dynamic up-sampling layer of a dynamic up-sampling operator Dysample, the layer is composed of a sampling point generator, a sampling device and an interpolation function.

10. A computer readable storage medium having stored thereon computer programs / instructions, having stored thereon computer executable instructions which, when executed by a processor, implement the steps of the landmark detection method of any one of claims 1 to 8.

Images