Railway annunciator dynamic adjustment real-time detection method based on improved RT-DETR network

Abstract

The invention discloses a railway signal machine dynamic adjustment real-time detection method based on an improved RT-DETR network, and relates to the technical field of computer vision, and the method comprises the following steps: 1, an unmanned plane collects a railway signal machine data set and carries out the preprocessing; 2, a railway signal machine detection model is constructed, an RT-DETR network is used as a basic network, the RT-DETR network comprises a ResNet50 backbone network and an Effect Hybrid Encode efficient hybrid encoder, and a part of 3 * 3 convolution in the backbone network ResNet50 is replaced with a deformable dynamic quantitative sampling attention mechanism of DQSATT (DQSATT) dynamic sampling points; an up-sampling operator in the efficient hybrid encoder is replaced by a dynamic grouping up-sampler DyGSample; carrying out bounding box regression adjustment by adopting a bounding box regression loss function LRDIOU based on RIoU; 3, selecting the optimal weight to predict the original aerial photography signal machine image data set, and taking the signal machine data set to be detected as an input and output detection result; 4, applying the trained railway annunciator detection model, and performing ablation experiment analysis; 5, performing evaluation index comparison on the detection result, and performing visual analysis on a loss function; as an ultra-fast, high-real-time and high-precision lightweight annunciator detection model, the improved real-time dynamic adjustment RT-DETR model has excellent performance and effectiveness in railway annunciator extraction.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and to technologies such as target detection, deep learning and image recognition, and is specifically manifested in a real-time detection method for dynamic adjustment of railway signal machines based on an improved RT-DETR network. Background Art

[0002] Railway signal detection plays an important role in improving the operational safety performance of the railway system. Railway signal detection can ensure the normal operation of signal equipment and avoid safety accidents caused by equipment failure or abnormality, thereby ensuring the safety of trains and passengers. Dynamic adjustment and real-time detection of the operating status of signal machines can help the railway dispatching system to more accurately grasp the train operation status, optimize signal control strategies, reduce train waiting time, and improve railway transportation efficiency.

[0003] In practical applications, there are some limitations and challenges in artificial target detection. First, artificial target detection is easily affected by human factors, such as fatigue and distraction, which may lead to inaccurate or missing detection results. Second, for a large amount of image or video data, manual detection is not only inefficient but also prone to errors, especially when dealing with complex scenes or targets. Several other mainstream models, such as Faster R-CNN, YOLOv5 and classic backbone networks DetNet and FishNet, simulate the human visual system and use deep learning and other algorithms to automatically analyze and process images or videos, thereby realizing automatic detection and recognition of targets. However, they still need to be pre-trained and fine-tuned on ImageNet, and the detection accuracy of small targets is not high enough, the time cost is high, and the computational cost is high. After DETR was proposed, a series of works such as Deformable DETR, Conditional DETR, and DINO were proposed one after another, constantly solving the problems exposed by the original DETR, and constantly approaching the optimal performance of this framework. The advent of DETR has revealed a new research route for researchers. Summary of the invention

[0004] Deformable DETR proposed Deformable Attention to solve the problem of Attention 0 in the standard Transformer (the weights of some data points may be very close to 0, which is numerically close to being ignored), speed up the convergence of the model and reduce the complexity of the algorithm. At the same time, it introduced multi-scale features to solve the problem of insufficient performance in small target detection. Group DETR introduced multiple object queries, which not only retained the end-to-end reasoning advantage of DETR, but also took advantage of the one-to-many advantage in training to improve performance and speed up the convergence of the model. DAB-DETR accelerated the convergence of DETR by introducing the idea of ​​"denoising learning", and the subsequent DINO thoroughly improved this framework for a real-time detection method for dynamic adjustment of railway signal machines based on an improved RT-DETR network. At present, the traditional Transformer uses a standard self-attention mechanism, which processes all pixels in the image, resulting in a large amount of calculation. Some upsamplers introduce a lot of workload due to time-consuming dynamic convolution and additional subnets used to generate dynamic kernels. The existing loss function of bounding box regression has the same value under different prediction results, which reduces the convergence speed and accuracy of bounding box regression.

[0005] Based on the above problems, the contributions made by the present invention are as follows:

[0006] The object of the present invention is to provide a real-time detection method for dynamic adjustment of railway signal lights based on an improved RT-DETR network, in view of the existing technical defects, comprising the following steps:

[0007] Step 1: Collect railway signal machine dataset using drone aerial photography:

[0008] The railway signal data set used in the present invention is derived from railway drone aerial images of stations in multiple cities in China; these images not only contain information about railway tracks, but also capture a large number of railway signal machines; each original aerial image has a size of 4864×3648 pixels, with high resolution and clear details, providing rich visual information for subsequent target detection.

[0009] Step 2: Preprocess the data set:

[0010] Perform operations such as contrast enhancement, noise filtering, and image sharpening on the image; perform necessary cropping and splicing on the image, use the Labelimg tool to manually annotate the pre-processed image, generate the corresponding label image, and produce a dataset dedicated to railway signal recognition;

[0011] The dataset includes two subfolders, Images and Labels. Images stores the training images, and Labels stores the xml files corresponding to the training images. The prepared dataset is divided into training set, test set, and validation set, and the corresponding label boxes are adjusted using the code. The training set image path is saved in train.txt, the test set image path is saved in test.txt, and the validation set image path is saved in val.txt.

[0012] Step 3: Improve the original RT-DETR network:

[0013] The railway signal detection model is based on the RT-DETR network, which includes a ResNet50 backbone network and an efficient hybrid encoder Efficient Hybrid Encode. Part of the 3×3 convolution in the backbone network ResNet50 is replaced by a deformable dynamic quantization sampling attention mechanism DQSAT of dynamic sampling points; the upsampling operator in the efficient hybrid encoder Efficient Hybrid Encode is replaced by a dynamic grouping upsampler DyGSample; the loss function of the BoundingBox in the Decoder network of RT-DETR adopts a new bounding box regression loss function L based on the minimum point distance RIoU Perform bounding box regression adjustment;

[0014] The backbone network ResNet50 is mainly composed of a plurality of bottleneck layers composed of 1×1 and 3×3 convolutions alternatingly. On this basis, the introduction of the dynamic quantization sampling attention mechanism DQSAT can enhance the representation ability and feature extraction ability of the model; the main idea of ​​ResNet50 is to use bottleneck layer residual learning to solve the gradient vanishing problem in deep convolutional neural networks; in the 1×1 convolution kernel of the bottleneck layer, since the number of channels has been reduced, the dynamic quantization sampling attention mechanism DQSAT added after the convolution brings relatively less calculation amount. The present invention increases the complexity of the backbone network while ensuring the backbone performance, and improves the calculation accuracy and expressiveness of the backbone network;

[0015] The backbone network ResNet50 includes five layers of benchmark modules, namely Conv1, Conv2_x, Conv3_x, Conv4_x and Conv5_x. Each layer of the benchmark module includes three convolutional layers. The second 3×3 convolution in each benchmark module is replaced with a deformable dynamic quantization sampling attention mechanism DQSAT module of dynamic sampling points.

[0016] The deformable dynamic quantization sampling attention mechanism DQSAT module of the dynamic sampling point can dynamically select sampling points instead of processing the entire image in a fixed manner, and focus on those areas that are most important to the current task; in the upper part, a set of reference points are evenly placed on the feature map, and the offsets of these points are learned by the query through the offset network; as shown in the lower side, the deformed keys and values ​​are projected from the sampled features according to the deformation points; the relative position deviation is also calculated by the deformation point, which enhances the multi-head attention of the output transformation feature; the size of the input and output feature maps of each layer are marked, and this Offsetnetwork needs to be controlled in the network code to be added or not; the deformable dynamic quantization sampling attention mechanism DQSAT module of the dynamic sampling point has a computational cost similar to that of the PVT or Swin transformer; the only additional overhead comes from the subnetwork used to generate the offset, and the complexity of the entire module can be summarized as: Ω(DQSAT)=2HWN s C+2HWC 2 +2N s C 2 +(k 2 +2)N s C; In the dynamic quantization sampling attention mechanism DQSAT, the PReLU (parametric rectified linear unit) used by the Offset network is a ReLU with parameters; if a i =0, PReLU is degraded to ReLU; if it is a small fixed value (such as =0.01), PReLU is degraded to leaky ReLU (LReLU); Compared with ReLU, LReLU has little effect on the final result; PReLU only adds very few parameters, which means that the computational effort and the risk of overfitting of the network are only increased a little; in particular, when different channels use the same a i When , the parameters are even fewer; the negative slope parameter a i When updating, use the dynamic update mode, such as the formula:

[0017] The feature maps output by the backbone network ResNet50 enter three stages {S3, S4, S5}, and the feature maps of the last three stages {S3, S4, S5} of the backbone are used as the input of the encoder, and the feature maps are input into the efficient hybrid encoder Efficient Hybrid Encoder to convert multi-scale features into image feature sequences through intra-scale feature interaction (AIFI) and cross-scale feature fusion module (CCFM);

[0018] The module structure of the Efficient Hybrid Encoder with DyGSample is described as follows: feature maps of three different sizes (256, 25, 25), (256, 50, 50), and (256, 100, 100) enter the ConvNormlayer module, ConvNormlayer module, and CSPRepLayer module of the first layer of the Efficient Hybrid Encoder respectively; after passing through all modules of the EfficientHybrid Encoder, the feature maps of the three sizes enter the concat module for splicing and then enter the feature map flattening operation, and finally the features are input into the CCFF structure;

[0019] Furthermore, the upsampling operator in the efficient hybrid encoder Efficient Hybrid Encode is replaced by a dynamic grouping upsampler DyGSample; the dynamic grouping upsampler DyGSample uses a point sampling method, which has fewer parameters, floating point operations, GPU memory and latency compared with the previous kernel-based dynamic grouping upsampler; in addition, DyGSample outperforms other upsamplers (the most effective upsampler to date) in five prediction tasks including semantic segmentation, object detection, instance segmentation, panoptic segmentation and unary depth estimation; the implementation of DyGSample is more lightweight and does not require additional CUDA packages or customized hardware support, which makes It is easy to integrate into an efficient hybrid encoder without significantly increasing the complexity of the model; since DyGSample does not rely on high-resolution characteristics, it has good performance in various application scenarios; introducing it into an efficient hybrid encoder can keep the overall model high-performance in a wider range of scenarios; compared with kernel-based dynamic grouping upsamplers such as CARAFE, FADE, and SAPA, DyGSample has fewer parameters, floating-point operations (FLOPs), GPU memory, and latency; this means that the calculation process of using DyGSample in an efficient hybrid encoder c is as follows: O = linear(X); S = G + O; In which, given an upsampling scale factor X and a feature map X of size C×H×W, using an input and output channel number C and 2s 2 The linear layer produces a size of 2s 2 ×H×W, and then reassemble it into 2×sH×sW by pixel reassembly. Then the sample set S is the sum of the offset O and the original sampling grid G.

[0020] Step 4: Transform the original image of the training set through the pre-trained weights in each layer of the neural network to obtain the predicted value Y′, and the training set label is used as the true value Y; input the predicted value Y′ and the true value Y into the loss function L RIoU In , adjust the network weights based on the loss value;

[0021] The present invention adopts a new bounding box regression loss function L based on the minimum point distance for the loss function of BoundingBox in the RT-DETRDecoder network. RIoU Perform bounding box similarity regression adjustment; in order to fully explore the geometric features of horizontal rectangles, a bounding box similarity comparison index L based on the minimum point distance is proposed RIoU ,This metric takes into account all relevant factors considered in existing loss functions,,i.e., overlapping or non-overlapping areas, center point distance, width, and height deviation,,while simplifying the calculation process;

[0022] On this basis, the RIoU-based bounding box regression loss function L is adopted in the present invention. RIoU ; The image features are output from the Decoder of the improved RT-DETR network and then pass through the new loss function L for bounding box regression RIoU Perform bounding box similarity regression adjustment; the bounding box regression loss function L RIoU is a new loss function L for bounding box regression based on the minimum point distance RIoU , and use L in the bounding box regression process RIoU As a new measure to compare the similarity between the predicted bounding box and the ground truth bounding box; the bounding box regression loss function L RIoU The formula is: L RIoU =1-RIoU; Among them, x prd ,y prd , w and h represent the horizontal coordinate, vertical coordinate, width and height of the prediction box center point, respectively, gt ,y gt and represent the horizontal and vertical coordinates of the ground truth center point respectively, and IoU represents the intersection-over-union ratio of the predicted bounding box to the ground truth bounding box; L RIoU Using it in RT-DETR can significantly improve the bounding box regression accuracy, detection performance, and computational efficiency of the model, providing a more accurate and efficient solution for real-time object detection tasks;

[0023] Step 5. After the training is completed, the optimal weight is selected to detect the original signal image dataset, and the evaluation indicators are compared and the loss function is visualized and analyzed. The above experimental steps are repeated using the public aircraft dataset RSOD-DQSATaset to verify the real-time detection performance of the model.

[0024] In order to further verify the advantages and disadvantages of the improved RT-DETR railway signal detection algorithm of the present invention, the convergence process of the loss function of different model training is visualized and analyzed on the railway signal detection dataset and the aircraft dataset RSOD-DQSATaset; in order to ensure that there are no other interference conditions, the different models used in the loss function visualization experiment of the present invention use the same loss function, the number of training iterations is 100 times, and the networks used for comparison are Faster R-CNN, YOLOv5 and the original RT-DETR model;

[0025] The experimental results of the improved RT-DETR network are compared with several other mainstream models, Faster R-CNN, YOLOv5 and the original RT-DETR model, and the loss function is visualized and analyzed. The present invention selects a series of evaluation indicators, including mAP (mean Average Precision), AP (Average Precision), F1 Score, Precision, Recall and time; since Recall and Precision are not the concept of area like AP, the Recall and Precision values ​​of the network are different when the threshold value (Confidence) is different; the evaluation index calculated by the present invention represents the corresponding index value when the threshold value Confidence = 0.5; compared with other algorithms, the present invention can achieve faster processing speed while maintaining high precision and high real-time detection, which is of great significance for the dynamic adjustment of railway signal machines and real-time detection requirements.

[0026] The training method is as follows: the network uses the pre-training model during training, the learning rate is set to 0.01, and the decay weight is 5×10 -4 , the batch size is 8; the number of training iterations is 100 rounds, and the weights are saved every 10 epochs; the experimental configuration is a Windows 10 system with a 64-bit operating system, an Inteli5 CPU, and an NVIDIA Tesla T4 graphics card; the original image data of the signal machine generated by the orthophotography of the DJI M600 drone at an altitude of 400 meters, and the Zenmuse X5S used is a high-performance camera that provides high-quality signal machine image output, which is suitable for professional-level aerial photography tasks. The network uses pre-trained models during training.

[0027] In order to further verify the advantages and disadvantages of the improved RT-DETR railway signal detection algorithm of the present invention, the convergence process of the loss function of different model training is visualized and analyzed on the railway signal detection dataset and the aircraft dataset RSOD-DQSATaset; in order to ensure that there are no other interference conditions, the different models used in the loss function visualization experiment of the present invention use the same loss function, the number of training iterations is 100 times, and the networks used for comparison are Faster R-CNN, YOLOv5 and the original RT-DETR model;

[0028] The experimental results of the improved RT-DETR network are compared with several other mainstream models, such as Faster R-CNN, YOLOv5 and the original RT-DETR model, in terms of evaluation indicators and loss function visualization analysis. The present invention selects a series of evaluation indicators, including mAP (mean Average Precision), AP (Average Precision), F1 Score, Precision, Recall and time. Since Recall and Precision are not the concept of area like AP, the Recall and Precision values ​​of the network are different when the threshold value (Confidence) is different. The evaluation indicator calculated by the present invention represents the corresponding indicator value when the threshold value Confidence = 0.5. Compared with other algorithms, the present invention can achieve a faster processing speed while maintaining high-precision and high-real-time detection, which is of great significance for the dynamic adjustment of railway signal machines and real-time detection requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of a method for dynamic adjustment and real-time detection of railway signal machines based on an improved RT-DETR network according to the present invention;

[0030] Figure 2 An algorithm flow chart of a real-time detection method for dynamic adjustment of railway signal machines based on an improved RT-DETR network provided by the present invention;

[0031] Figure 3 A schematic diagram of the network structure of a railway signal machine dynamic adjustment real-time detection method based on an improved RT-DETR network provided by the present invention;

[0032] Figure 4 The processing flow of feature data by DQSAT, (a) the data processing flow of the dynamic attention mechanism DAM, and (b) the data flow of the Offset network;

[0033] Figure 5The module structure of Efficient Hybrid Encoder with dynamic group upsampler DyGSample;

[0034] Figure 6 It is a structural diagram of the dynamic group upsampler DyGSample, where (a) is the sampling process of the dynamic group upsampler, and (b) is the structure of the dynamic group upsampler;

[0035] Figure 7 This is a railway signal dataset;

[0036] Figure 8 The aircraft dataset is RSOD-DQSATaset;

[0037] Fig. 9 Comparison of the loss function curves of railway signal machines. (a), (b), (c), and (d) are Faster R-CNN, YOLOv5, the original RT-DETR model, and the improved RT-DETR model, respectively.

[0038] Fig.10 Comparison of loss function curves of the aircraft dataset RSOD-DQSATaset. (a), (b), (c), and (d) are Faster R-CNN, YOLOv5, the original RT-DETR model, and the improved RT-DETR model, respectively. DETAILED DESCRIPTION

[0039] The present invention will be further described below in conjunction with the accompanying drawings. Figure 1 Shown is a flow chart of a method for dynamic adjustment and real-time detection of railway signal machines based on an improved RT-DETR network according to the present invention.

[0040] Figure 2 The present invention provides an algorithm flow chart of a real-time detection method for dynamic adjustment of railway signal machines based on an improved RT-DETR network.

[0041] The present application embodiment provides a railway signal machine dynamic adjustment real-time detection method based on an improved RT-DETR network, comprising the following steps:

[0042] Step 1: Collect railway signal machine dataset using drone aerial photography:

[0043] The railway signal machine data set used in the present invention is derived from the railway drone aerial images of multiple urban stations in China; these images not only contain the information of railway tracks, but also capture a large number of railway signal machines; the size of each original aerial image is 4864×3648 pixels, with high resolution and clear details, providing rich visual information for subsequent target detection;

[0044] Step 2: Preprocess the data set:

[0045] Perform contrast enhancement, noise filtering, image sharpening, etc. on the image; perform necessary cropping and splicing on the image, use the Labelimg tool to manually annotate the preprocessed image, generate the corresponding label image, and create a data set dedicated to railway signal detection; divide the prepared data set into training set, test set, and validation set, and use the code to adjust the corresponding label box; divide the data set into training set, test set, and validation set in a ratio of 7:2:1;

[0046] Step 3: Improve the original RT-DETR network:

[0047] The railway signal detection model is based on the RT-DETR network, which includes a ResNet50 backbone network and an efficient hybrid encoder Efficient Hybrid Encode. Some 3*3 convolutions in the backbone network ResNet50 are replaced with a deformable attention mechanism DQSAT of dynamic sampling points; the upsampling operator in the efficient hybrid encoder Efficient Hybrid Encode is replaced with a dynamic grouping upsampler DyGSample; a new bounding box regression loss function L based on the minimum point distance is adopted. RIoU Perform bounding box regression adjustment;

[0048] like Figure 2As shown in the figure, a network structure diagram of a real-time detection method for dynamic adjustment of railway signal machines based on an improved RT-DETR network is shown. The image is input into the backbone of RT-DETR, which integrates the deformable dynamic quantization sampling attention mechanism of the initial DQSAT dynamic sampling point of the deformable attention mechanism, dynamically selects sampling points instead of fixedly processing the entire image, and focuses on those areas that are most important to the current task; then the features of the last three stages of the backbone {S3, S4, S5} are used as the input of the encoder; Efficient Hybrid Encoder Efficient Hybrid The Encoder converts multi-scale features into image feature sequences through intra-scale feature interaction (AIFI) and cross-scale feature fusion module (CCFM); among them, the intra-scale interaction is processed by a single-scale Transformer encoder for the features of each scale, which can more efficiently capture the target information within the same scale; after processing the intra-scale interaction, an efficient fusion mechanism (such as the PANet structure) is used to merge features of different scales, thereby achieving an overall understanding of targets of different sizes; IoU-aware perceptual query selection selects a fixed number of image features as the initial object query of the decoder, and selects features that are more descriptive of the real target by considering the IoU score; then inputs the Decoder with an auxiliary prediction head for the next step. RT-DETR supports flexible adjustment of the inference speed by using different decoder layers without retraining; finally, the features output by the three detection heads are used with the bounding box similarity loss function L based on the minimum point distance RIoU Adjusted, this metric incorporates all relevant factors considered in the existing loss function, namely overlapping or non-overlapping areas, center point distance, width and height deviation, while simplifying the calculation process;

[0049] Specifically, the backbone network ResNet50 is mainly composed of a plurality of bottleneck layers consisting of 1*1 and 3*3 convolutions, and the introduction of the deformable dynamic quantization sampling attention mechanism of the DQSAT dynamic sampling point on this basis can enhance the representation ability and feature extraction ability of the model; the main idea of ​​ResNet50 is to use residual learning to solve the gradient vanishing problem in deep convolutional neural networks; in the 1*1 convolution kernel of the bottleneck layer, since the number of channels has been reduced, the deformable dynamic quantization sampling attention mechanism of the DQSAT dynamic sampling point added after the convolution has a relatively small amount of calculation, which can increase the complexity of the backbone network while ensuring the backbone performance, and improve the calculation accuracy and expressiveness of the backbone network;

[0050] The backbone network ResNet50 includes five layers of benchmark modules, namely Conv1, Conv2_x, Conv3_x, Conv4_x, and Conv5_x. Each layer of the benchmark module includes three convolutional layers. In the embodiment of the present application, the second 3×3 convolution in each benchmark module is replaced with a deformable attention mechanism DQSAT module with dynamic sampling points. The improved ResNet50 network parameters are shown in Table 1 below:

[0051] Table 1 Improved ResNet50 network parameters

[0052]

[0053] like Figure 3 As shown, the processing flow of DQSAT on feature data;

[0054] The dynamic sampling point deformable attention mechanism DQSAT module can dynamically select sampling points instead of processing the entire image in a fixed manner, and focus on those areas that are most important to the current task; in the upper part, a set of reference points are evenly placed on the feature map, and the offsets of these points are learned by the query through the offset network; then, as shown in the lower side, the deformed keys and values ​​are projected from the sampled features according to the deformation points; the relative position deviation is also calculated by the deformation point, which enhances the multi-head attention of the output transformation feature; the size of the input and output feature maps of each layer are annotated, and this Offsetnetwork needs to be controlled in the network code to be added or not; Deformed Multi-Head Attention (DMHA) has a similar computational cost to PVT or Swin Transformer; the only additional overhead comes from the subnetwork used to generate the offset, and the complexity of the entire module can be summarized as: Ω(DMHA)=2HWN s C+2HWC 2 +2N s C 2 +(k 2 +2)N s C (1) The PReLU (parameterized rectified linear unit) used in the Offset network of the dynamic quantization sampling attention mechanism DQSAT is a ReLU with parameters; if = 0, PReLU is downgraded to ReLU; if it is a small fixed value (such as = 0.01), PReLU is downgraded to leaky ReLU (LReLU); compared with ReLU, LReLU has little effect on the final result; PReLU only adds very few parameters, which means that the computational complexity and overfitting risk of the network are only increased a little; in particular, when different channels use the same ai, the parameters are even fewer; when BP updates ai, a dynamic update mode is adopted, such as the formula:

[0055] The feature map output by the backbone network ResNet50 enters three stages {S3, S4, S5}, and the feature maps of the last three stages {S3, S4, S5} of the backbone are used as the input of the encoder, and the feature map is input into the efficient hybrid encoder Efficient Hybrid Encoder to convert the multi-scale features into image feature sequences through intra-scale feature interaction (AIFI) and cross-scale feature fusion module (CCFM);

[0056] like Figure 4 As shown, the structure diagram of the dynamic group upsampler DyGSample;

[0057] The implementation of the dynamic grouping upsampler DyGSample is more lightweight and does not require additional CUDA packages or customized hardware support, which makes it easy to integrate into an efficient hybrid encoder without significantly increasing the complexity of the model; since DyGSample does not rely on high-resolution characteristics, it has good performance in various application scenarios; introducing it into an efficient hybrid encoder can keep the overall model high-performance in a wider range of scenarios; compared with kernel-based dynamic grouping upsamplers such as CARAFE, FADE and SAPA, DyGSample has fewer parameters, floating-point operations (FLOPs), GPU memory and latency; this means that the calculation process of using DyGSample in an efficient hybrid encoder c is as follows: Given an upsampling scale factor X and a feature map X of size C×H×W, a linear layer with input and output channels C and 2s2 is used to produce an offset O of size 2s2×H×W, which is then reorganized into 2×sH×sW by pixel reorganization; then the sample set S is the sum of the offset O and the original sampling grid G, that is: O=linear(X) (4) S=G+O (5)

[0058] like Figure 5 This is the module structure of Efficient Hybrid Encoder with DyGSample.

[0059] Furthermore, the upsampling operator in the efficient hybrid encoder Efficient Hybrid Encode is replaced with a dynamic group upsampler DyGSample; the module structure of the Efficient Hybrid Encoder with DyGSample is described as follows: feature maps of three different sizes (256, 25, 25), (256, 50, 50), and (256, 100, 100) enter the ConvNormlayer module, ConvNormlayer module, and CSPRepLayer module of the first layer of the Efficient Hybrid Encoder respectively; after passing through all modules of the Efficient Hybrid Encoder, the feature maps of the three sizes enter the concat module for splicing and then enter the feature map flattening operation, and finally the features are input into the CCFF structure;

[0060] Figure 6 This is the structure diagram of the dynamic group upsampler DyGSample;

[0061] Step 4: Transform the original image of the training set through the pre-trained weights in each layer of the neural network to obtain the predicted value Y′, and the training set label is used as the true value Y; input the predicted value Y′ and the true value Y into the loss function L RIoU In , adjust the network weights based on the loss value;

[0062] In addition, the embodiment of the present application adopts a new bounding box similarity loss function L based on the minimum point distance as the loss function of the BoundingBox in the RT-DETR network Decoder. RIoU Perform bounding box similarity regression adjustment; the image features are output from the Decoder of the improved RT-DETR network and then pass through the new loss function L for bounding box regression RIoU Perform bounding box similarity regression adjustment;

[0063] The bounding box similarity loss function L RIoU is a new loss function L for bounding box regression based on the minimum point distance RIoU , a new bounding box regression loss function L based on the minimum point distance is adopted RIoU Perform bounding box similarity regression adjustment; in order to fully explore the geometric features of horizontal rectangles, a bounding box similarity comparison index L based on the minimum point distance is proposed RIoU, which takes into account all relevant factors considered in the existing loss function, namely overlapping or non-overlapping areas, center point distance, width and height deviation, and simplifies the calculation process; on this basis, the RIoU-based bounding box regression loss function L RIoU ; The image features are output from the Decoder of the improved RT-DETR network and then pass through the new loss function L for bounding box regression RIoU Perform bounding box similarity regression adjustment; the bounding box regression loss function L RIoU is a new loss function L for bounding box regression based on the minimum point distance RIoU , and use L in the bounding box regression process RIoU As a new measure to compare the similarity between the predicted bounding box and the ground truth bounding box; the bounding box regression loss function L RIoU The formula is: L RIoU =1-RIoU (10) Among them, x prd ,y prd , w and h represent the horizontal coordinate, vertical coordinate, width and height of the prediction box center point, respectively, gt ,y gt and represent the horizontal and vertical coordinates of the ground truth center point respectively, and IoU represents the intersection-over-union ratio of the predicted bounding box to the ground truth bounding box; L RIoU Using it in RT-DETR can significantly improve the bounding box regression accuracy, detection performance, and computational efficiency of the model, providing a more accurate and efficient solution for real-time object detection tasks;

[0064] Step 5: The training method is as follows: the network uses the pre-trained model during training, the learning rate is set to 0.01, and the decay weight is 5×10 -4 , the batch size is 8; the number of training iterations is 100 rounds, and the weights are saved every 10 epochs; the experimental configuration is a Windows 10 system with a 64-bit operating system, an Inteli5 CPU, and an NVIDIA Tesla T4 graphics card;

[0065] Figure 7 This is a railway signal dataset.

[0066] Figure 8 It is the aircraft dataset RSOD-DQSATaset.

[0067] In order to further verify the advantages and disadvantages of the improved RT-DETR railway signal detection algorithm of the present invention, the convergence process of the loss function of different model training is visualized and analyzed on the railway signal detection dataset and the aircraft dataset RSOD-DQSATaset; in order to ensure that there are no other interference conditions, the different models used in the loss function visualization experiment of the present invention use the same loss function, the number of training iterations is 100 times, and the networks used for comparison are Faster R-CNN, YOLOv5 and the original RT-DETR model;

[0068] Fig. 9 Comparison of the loss function curves of railway signal machines. (a), (b), (c), and (d) are the loss function graphs of Faster R-CNN, YOLOv5, the original RT-DETR model, and the improved RT-DETR model, respectively.

[0069] Fig.10 Comparison of loss function curves of aircraft dataset RSOD-DQSATaset, (a), (b), (c), (d) are loss function graphs of Faster R-CNN, YOLOv5, original RT-DETR model and improved RT-DETR model respectively. In the comparative experiment of railway signal dataset, the evaluation index calculated by the present invention represents the corresponding index value when the threshold value Confidence = 0.5. The model of the present invention performs best in recall rate, reaching 90.31%, while the recall rates of other models are between 85.09% and 87.15%; the F1 score of the present invention is 88.93%, which is higher than other models, indicating that it performs well in balancing precision and recall; the precision of the present invention is 89.16%, which is the highest among all compared models, showing high classification accuracy; in terms of processing time, the present invention (291s) is at a medium level compared with other models, and does not show a significant advantage but is not too far behind. The comparison results of the evaluation indicators of the present invention and other models in the railway signal dataset and the aircraft dataset RSOD-DQSATaset are shown in Tables 1 and 2; In the comparative experiment of railway signal data set, the present invention performed well in key evaluation indicators, including recall rate (Recall) reaching 89.45%, F1 score (F1 Score) of 88.73%, and precision (Precision) as high as 90.77%; these indicators all exceeded the models in the other three, showing the superiority of the present invention in classification accuracy and balance; the average precision (AP) of the present invention was 87.98%, and the mean average precision (mAP) was 89.00%, both of which were at a high level, and exceeded the corresponding indicators of two of the models, further proving its good classification performance and generalization ability.

[0070] The experimental results of the improved RT-DETR network are compared with several other mainstream models, Faster R-CNN, YOLOv5 and the original RT-DETR model, and the loss function is visualized and analyzed. The present invention selects a series of evaluation indicators, including mAP (mean Average Precision), AP (Average Precision), F1 Score, Precision, Recall and time; since Recall and Precision are not the concept of area like AP, the Recall and Precision values ​​of the network are different when the threshold value (Confidence) is different; the evaluation index calculated by the present invention represents the corresponding index value when the threshold value Confidence = 0.5; compared with other algorithms, the present invention can achieve faster processing speed while maintaining high precision and high real-time detection, which is of great significance for the dynamic adjustment of railway signal machines and real-time detection requirements.

[0071] A specific example is given below.

[0072] The training method is as follows: the network uses the pre-training model during training, the learning rate is set to 0.01, and the decay weight is 5×10 -4 , the batch size is 8; the number of training iterations is 100 rounds, and the weights are saved every 10 epochs; the experimental configuration is a Windows 10 system with a 64-bit operating system, an Inteli5 CPU, and an NVIDIA Tesla T4 graphics card;

[0073] In order to further verify the advantages and disadvantages of the improved RT-DETR railway signal detection algorithm of the present invention, the convergence process of the loss function of different model training is visualized and analyzed on the railway signal detection dataset and the aircraft dataset RSOD-DQSATaset; in order to ensure that there are no other interference conditions, the different models used in the loss function visualization experiment of the present invention use the same loss function, the number of training iterations is 100 times, and the networks used for comparison are Faster R-CNN, YOLOv5 and the original RT-DETR model;

[0074] Furthermore, the experimental results of the improved RT-DETR network are compared with several other mainstream models, such as Faster R-CNN, YOLOv5 and the original RT-DETR model, for evaluation index comparison and loss function visualization analysis; the present invention selects a series of evaluation indexes, including mAP (mean Average Precision), AP (Average Precision), F1Score, Precision, Recall and time; since Recall and Precision are not the same as AP, which is an area concept, the Recall and Precision values ​​of the network are different when the threshold value (Confidence) is different; the evaluation index calculated by the present invention represents the corresponding index value when the threshold value Confidence = 0.5; compared with other algorithms, the present invention can achieve faster processing speed while maintaining high precision and high real-time detection, which is of great significance for the dynamic adjustment of railway signal machines and real-time detection requirements;

[0075] In the ablation experiment analysis, the accuracy of the original RT-DETR network is 90.21%, and the processing time is 289 seconds; when only the deformable attention machine of the DQSAT dynamic sampling point is used, the accuracy is improved to 90.30%, and the processing time is slightly reduced to 288 seconds, indicating that the deformable attention machine of the DQSAT dynamic sampling point has a slight positive impact on the model performance; after further adding the deformable attention machine and DyGSample of the DQSAT dynamic sampling point, the accuracy is significantly improved to 91.36%, but the processing time is also increased to 300 seconds, indicating that dynamic sampling has a significant effect on improving the performance of the model, but it will also increase the computational burden; finally, when the deformable attention machine of the DQSAT (DQSAT) dynamic sampling point, the dynamic grouping upsampler DyGSample and L RIoU , the accuracy is improved to 91.45% again, while the processing time returns to nearly the original level (291 seconds), indicating that L RIoU While maintaining high efficiency, the accuracy of the model can be further improved. Table 2 shows the ablation experiment of the railway signal data set:

[0076] Table 2 Ablation experiment on railway signal dataset

[0077]

Claims

1. A real-time detection method for dynamic adjustment of railway signal lights based on an improved RT-DETR network, characterized in that: The steps include: Step 1: Collect railway signal machine datasets through drone aerial photography; Step 2: Preprocess the data set: perform contrast enhancement, noise filtering, image sharpening and other operations on the image; Perform necessary cropping and splicing on the images, use the Labelimg tool to manually annotate the preprocessed images, generate the corresponding .txt format label images, and thus produce a dataset dedicated to railway signal recognition; the dataset includes two subfolders, namely images and Labels, where images save the training images and Labels saves the xml files corresponding to the training images; divide the prepared dataset into training set, test set, and validation set, and use the code to adjust the corresponding label boxes; the training set image path is saved in train.txt, the test set image path is saved in test.txt, and the validation set image path is saved in val.txt; Step 3: Improve the original RT-DETR network. The railway signal detection model is based on the RT-DETR network. The RT-DETR network includes a ResNet50 backbone network and an Efficient Hybrid Encode efficient hybrid encoder. Some 3×3 convolutions in the backbone network ResNet50 are replaced with a deformable dynamic quantization sampling attention mechanism DQSAT of dynamic sampling points; the upsampling operator in the Efficient Hybrid Encode efficient hybrid encoder is replaced with a dynamic grouping upsampler DyGSample; a new bounding box regression loss function L based on the minimum point distance is adopted. RloU Perform bounding box regression adjustment; Step 4: Transform the original image of the training set through the pre-trained weights in each layer of the neural network to obtain the predicted value Y′, and the training set label is used as the true value Y; the predicted value Y′ and the true value Y are input into the loss function L RloU In the process, the network weights are adjusted according to the loss value; and the improved RT-DETR network is subjected to ablation experimental analysis; Step 5. After training is completed, select the optimal weight to detect the original signal image test set, compare the evaluation indicators and visualize the loss function, and repeat the above test steps using the public RSOD-DQSATaset aircraft dataset to verify the real-time detection performance of the model.

2. The method for real-time detection of dynamic adjustment of railway signal machines based on improved RT-DETR network according to claim 1 is characterized in that: The railway signal data is derived from the railway drone aerial images of railway stations in multiple cities in China; these images not only contain the information of railway tracks, but also capture a large number of railway signal machines; the size of each original aerial image is 4864×3648 pixels, with high resolution and clear details, providing rich visual information for subsequent target detection; the preprocessing method is: performing operations such as contrast enhancement, noise filtering, and image sharpening on the image; Perform necessary cropping and splicing on the images, use the Labelimg tool to manually annotate the preprocessed images, generate corresponding label images, and produce a dataset dedicated to railway signal recognition; divide the produced dataset into training set, test set, and validation set, and adjust the corresponding label boxes; divide the dataset into training set, test set, and validation set in a ratio of 7:2:

1.

3. The method for dynamic adjustment of railway signaling based on improved RT-DETR network according to claim 1 is characterized in that: The backbone network ResNet50 is composed of a plurality of bottleneck layers composed of 1*1 and 3*3 convolutions alternatingly. On this basis, the introduction of the dynamic quantization sampling attention mechanism DQSAT can enhance the representation ability and feature extraction ability of the model; the main idea of ​​ResNet50 is to use the bottleneck layer residual learning to solve the gradient vanishing problem in the deep convolutional neural network; in the 1*1 convolution kernel of the bottleneck layer, since the number of channels has been reduced, the amount of calculation brought by the DQSAT dynamic quantization sampling attention mechanism added after the convolution is relatively small. The present invention increases the complexity of the backbone network while ensuring the backbone performance, and improves the calculation accuracy and expressiveness of the backbone network; the backbone network ResNet50 includes Conv1, Conv2x, Conv3x, Conv4_x, Conv5_ x five-layer benchmark module, each of which includes three convolutional layers. The present invention replaces the second 3×3 convolution in each of the benchmark modules with a deformable dynamic quantization sampling attention mechanism DQSAT module of dynamic sampling points; the deformable dynamic quantization sampling attention mechanism DQSAT module of dynamic sampling points can dynamically select sampling points instead of fixedly processing the entire image, and focus on those areas that are most important to the current task; in the upper part, a group of reference points are evenly placed on the feature map, and the offsets of these points are learned by the query through the offset network; as shown in the lower side, the deformed keys and values ​​are projected from the sampled features according to the deformation points; the relative position deviation is also calculated by the deformation points, which enhances the multi-head attention of the output conversion features; the size of the input and output feature maps of each layer is marked, and the Offset network needs to be controlled in the network code to be added or not; the deformable dynamic quantization sampling attention mechanism DQSAT module of dynamic sampling points has a computational cost similar to that of PVT or Swin transformer; the only additional overhead comes from the subnetwork used to generate the offset, and the complexity of the entire module can be summarized as: Ω(DQSAT)=2HWN s C+2HWC 2 +2N s C 2 +(k 2 +2)N s C; In the dynamic quantization sampling attention mechanism DQSAT, the PReLU (parametric rectified linear unit) used by the Offset network is a ReLU with parameters; if a i =0, PReLU is degraded to ReLU; If it is a small fixed value (such as = 0.01), PReLU degrades to leaky ReLU (LReLU); Compared with ReLU, LReLU has little effect on the final result; PReLU only adds very few parameters, which means that the computational effort and the risk of overfitting of the network increase only a little; in particular, when different channels use the same a i When , the parameters are even fewer; the negative slope parameter a i When updating, use the dynamic update mode, such as the formula:

4. The method for real-time detection of dynamic adjustment of railway signaling based on improved RT-DETR network according to claim 1 is characterized in that: The feature maps output by the backbone network ResNet50 enter three stages {S3, S4, S5}, and the feature maps of the three stages {S3, S4, S5} are used as the input of the encoder, and the feature maps are input into the efficient hybrid encoder EfficientHybrid Encoder. The multi-scale features are converted into image feature sequences through intra-scale feature interaction (A old) and cross-scale feature fusion module (CCFM); the module structure of the Efficient Hybrid Encoder with DyGSample is described as follows: feature maps of three different sizes (256, 25, 25), (256, 50, 50), and (256, 100, 100) enter the ConvNormlayer module, ConvNormlayer module and CSPRepLayer module of the first layer of the EfficientHybrid Encoder respectively; after passing through all modules of the Efficient Hybrid Encoder, the feature maps of the three sizes enter the concat module for splicing and then enter the feature map flattening operation, and finally the features are input into the CCFF structure; the Efficient Hybrid Encoder The upsampling operator in the Encode efficient hybrid encoder is replaced by a dynamic group upsampler DyGSample; the dynamic group upsampler DyGSample uses a point-based sampling method, which has fewer parameters, floating-point operations, GPU memory, and latency than the previous kernel-based dynamic group upsampler; DyGSample is more lightweight in implementation and does not require additional CUDA packages or custom hardware support, which makes it easy to integrate into an efficient hybrid encoder without significantly increasing the complexity of the model; since DyGSample does not rely on high-resolution features, it has good performance in a variety of application scenarios; introducing it into an efficient hybrid encoder allows the entire model to maintain high performance in a wider range of scenarios; compared with kernel-based dynamic grouping upsamplers such as CARAFE, FADE, and SAPA, DyGSample has fewer parameters, floating-point operations (FLOPs), GPU memory, and latency; this means that using DyGSample in an efficient hybrid encoder c has the following calculation process: 0 = linear (X); S = G + O; In which, given an upsampling scale factor X and a feature map X of size C×H×W, using an input and output channel number C and 2s 2 The linear layer produces a size of 2s 2 ×H×W, and then reorganize it into 2×sH×sW by pixel reorganization; then the sampling set S is the sum of the offset O and the original sampling grid G.

5. The method for real-time detection of dynamic adjustment of railway signaling based on improved RT-DETR network according to claim 1 is characterized in that: A new bounding box regression loss function L based on the minimum point distance is adopted RloU Perform bounding box similarity regression adjustment; in order to fully explore the geometric features of horizontal rectangles, a bounding box similarity comparison index L based on the minimum point distance is proposed RloU , which takes into account all relevant factors considered in the existing loss function, namely overlapping or non-overlapping areas, center point distance, width and height deviation, and simplifies the calculation process; on this basis, the RIoU-based bounding box regression loss function L RIoU ; The image features are output from the Decoder of the improved RT-DETR network and then pass through the new loss function L for bounding box regression RloU Perform bounding box similarity regression adjustment; the bounding box regression loss function L RloU is a new loss function L for bounding box regression based on the minimum point distance RloU , and use L in the bounding box regression process RloU As a new measure to compare the similarity between the predicted bounding box and the ground truth bounding box; the bounding box regression loss function L RloU The formula is: L RIoU =1-RIoU; Among them, x prd ,y prd ,w and h represent the horizontal coordinate, vertical coordinate, width and height of the prediction box center point, respectively, gt ,y gt and represent the horizontal and vertical coordinates of the ground truth center point respectively, and IoU represents the intersection-over-union ratio of the predicted bounding box to the ground truth bounding box; L RloU When used in RT-DETR, it can significantly improve the bounding box regression accuracy, detection performance, and computational efficiency of the model, providing a more accurate and efficient solution for real-time object detection tasks.

6. The method for real-time detection of dynamic adjustment of railway signaling based on improved RT-DETR network according to claim 1 is characterized in that: The training method is as follows: the network uses the pre-training model during training, the learning rate is set to 0.01, and the decay weight is 5×10 -4 , the batch size is 8; the number of training iterations is 100 rounds, and the weights are saved every 10 epochs; the experimental configuration is a Windows 10 system with a 64-bit operating system, an Intel i5 CPU, and an NVIDIA Tesla T4 graphics card; the original image data of the signal machine generated by the orthophotography of the DJI M600 drone at an altitude of 400 meters, and the Zenmuse X5S used is a high-performance camera that provides high-quality signal machine image output, which is suitable for professional-level aerial photography tasks. The network uses pre-trained models during training.

7. The method for real-time detection of dynamic adjustment of railway signaling based on improved RT-DETR network according to claim 1 is characterized in that: The present invention designs and implements an ablation experiment, taking the railway signal recognition dataset as a benchmark. As a systematic and sophisticated research method, the ablation experiment can effectively quantify the contribution of each component to the model performance by gradually introducing model components. In this experiment, a fixed Epoch is set to 100, and the prediction confidence and running time of the original RT-DETR network are first tested. Then, the deformable dynamic quantization sampling attention mechanism of the DQSAT dynamic sampling point and the attention mechanism are added to the ResNet50 backbone network in turn, and the dynamic grouping upsampler DyGSample is added to the Efficient Hybrid Encoder module. The L RloU For the bounding box regression loss function, a new round of experiments is conducted each time a component is introduced, and the data is recorded in detail. By comparing and analyzing the experimental results under different module combinations, the present invention aims to accurately reveal the specific impact of each component on the model's prediction accuracy and operating efficiency, thereby providing solid support for model optimization.

8. The method for real-time detection of dynamic adjustment of railway signal lights based on improved RT-DETR network according to claim 1, characterized in that: In order to further verify the advantages and disadvantages of the improved RT-DETR railway signal detection algorithm of the present invention, the convergence process of the loss function of different model training is visualized and analyzed on the railway signal detection data set and the RSOD-DQSATaset aircraft data set; in order to ensure that there are no other interference conditions, the different models used in the loss function visualization experiment of the present invention use the same loss function, the number of training iterations is 100 times, and the networks used for comparison are Faster R-CNN, YOLOv5 and the original RT-DETR model; the experimental results of the improved RT-DETR network are compared with the evaluation indicators of several other mainstream models Faster R-CNN, YOLOv5 and the original RT-DETR model, and the loss function is visualized and analyzed; the present invention selects a series of evaluation indicators, including mAP (mean Average Precision), AP (Average Precision), and mAP (mean Average Precision). Precision), F1Score, Precision, Recall and time; because Recall and Precision are not the concept of an area like AP, the Recall and Precision values ​​of the network are different when the threshold value (Confidence) is different; the evaluation index calculated by the present invention represents the corresponding index value when the threshold value Confidence = 0.5; compared with other algorithms, the present invention can achieve a faster processing speed while maintaining high-precision and high-real-time detection, which is of great significance for the dynamic adjustment of railway signal machines and real-time detection requirements.

Images