Night pedestrian detection method based on artificial intelligence technology

Abstract

The invention discloses a night pedestrian detection method based on an artificial intelligence technology, and relates to the technical field of target detection, and the method mainly comprises the steps: preparing a night road pedestrian data set in advance, and carrying out the preprocessing; the method comprises the following steps: constructing a night pedestrian detection model NE-RTDETR based on RT-DETR neck enhancement, wherein the model is composed of a backbone network ResNet18, a neck enhancement network and a decoder; the data set is input into a NE-RTDETR model for training, and a trained best weight file is obtained; inputting a to-be-detected night road pedestrian picture into the NE-RTDETR model according to the trained weight to obtain information about whether a pedestrian exists in the night road pedestrian picture or not; and finally outputting a detection result and evaluating model performance. Through the steps, the problems that an existing detection method is insufficient in accuracy, high in missing detection and false detection rate and the like under the conditions of low illumination and fuzzy shielding at night are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular to a night pedestrian detection method based on artificial intelligence technology. BACKGROUND

[0002] In early pedestrian detection methods, two-stage algorithms such as R-CNN and Faster R-CNN are concerned due to their high detection accuracy, but they are difficult to meet the requirements of real-time scenes because of their slow detection speed. With the development of deep learning, single-stage algorithms have gradually become mainstream. The YOLO series is efficient because it does not need to extract candidate frames and directly predicts target classification and position. Although the end-to-end design has improved the speed, it is still limited by the sensitivity to low-light features in night scenes.

[0003] In recent years, the Transformer architecture has made breakthroughs in target detection. DETR (DEtection TRansformer) combines the detection task with the Transformer, replaces the traditional anchor frame design with a query mechanism, and proposes a direct way to solve the set prediction problem. It cleverly bypasses the proxy task and bypasses the artificially designed part, realizes the end-to-end network, and does not require prior knowledge while achieving good results on more difficult datasets. RT-DETR (Real-Time Detection Transformer) is a real-time detection algorithm based on DETR, which overcomes the influence of non-maximum suppression (NMS) on inference speed and accuracy. With the help of an efficient hybrid encoder and an IoU-aware query selection mechanism, it achieves real-time performance while maintaining accuracy, providing a new paradigm for complex scene detection.

[0004] However, the existing RT-DETR model still has limitations in the application of night low-light scenes. On the one hand, the original feature quality of low-light images is poor, and direct input into the backbone network will lead to insufficient feature extraction; on the other hand, for the problems of small target blur and severe occlusion in low-light scenes, the adaptability of the feature fusion and feature extraction module is insufficient. SUMMARY

[0005] To solve the above technical problems, the present application provides a night pedestrian detection method based on artificial intelligence technology, comprising the following steps:

[0006] S1, a night road pedestrian dataset is prepared in advance and a pretreatment operation is performed;

[0007] S2, a night pedestrian detection model NE-RTDETR based on RT-DETR neck enhancement is constructed, the night road pedestrian dataset is input into the NE-RTDETR model for training, and the trained best weight file is obtained; the NE-RTDETR model comprises a backbone network ResNet18, a neck enhancement network and a decoder; the backbone network ResNet18 is used for outputting a feature vector; the neck enhancement network is used for feature extraction and fusion, and outputs the fused features; and the decoder is used for outputting a prediction result;

[0008] S3, according to the trained weight, input the night road pedestrian picture to be detected into the NE-RTDETR model, and obtain information whether the night road pedestrian picture contains a pedestrian;

[0009] S4, output the detection result and evaluate the model performance.

[0010] The further defined technical solution of the application is:

[0011] Further, step S1 specifically comprises the following sub-steps:

[0012] S1.1, a plurality of night road pedestrian images are acquired to form a night road pedestrian dataset, and the night road pedestrian dataset contains night pedestrian targets of different scales and different blur degrees;

[0013] S1.2, the night road pedestrian dataset is divided into a training set, a validation set and a test set according to a proportion;

[0014] S1.3, data enhancement operation is performed on the training set images, and HSV adjustment of hue, saturation and brightness is set.

[0015] As described in the foregoing night pedestrian detection method based on artificial intelligence technology, in step S2, the backbone network ResNet18 performs feature extraction of different scales on the input image through hierarchical convolution and basic block operation, and inputs the feature maps output in the last three stages into the subsequent network; the backbone network ResNet18 adopts a progressive down-sampling strategy, and batch normalization and ReLU activation functions are used after all convolution layers.

[0016] As described in the foregoing night pedestrian detection method based on artificial intelligence technology, in step S2, the neck enhancement network comprises a spatial transformer module, a dynamic up-sampling module and an expanded reparameterization block chain module; the spatial transformer module adopts a two-stage design, the first stage divides the input features into fixed size windows through a local window attention mechanism, then projects each window feature block enhanced with local context into query, key and value, and finally performs regional self-attention calculation in each window feature block; the second stage divides the channels into multiple heads through a global attention mechanism, and simultaneously learns different self-attention maps.

[0017] The night pedestrian detection method based on artificial intelligence technology as described above, the spatial transformer module includes a local window attention module and a global self-attention module, in the first stage, the input feature is first normalized to X∈R H×W×C , and then sent to the local window attention module, wherein H, W and C represent height, width and channel number respectively; the local window attention module adopts a window-based local attention mechanism to divide the input feature into fixed-size windows:

[0018] {X1, X2,..., X n}=Split(X)

[0019] Wherein, represents a group of window feature blocks, N represents the window size and

[0020] Subsequently, by combining 1×1 and 3×3 deep convolution to capture local detail features, then each window feature block X i enhanced with local context is projected as query key and value The process is as follows:

[0021] {Q i , K i , V i}=RS(D(P(X i )))

[0022] Wherein, RS represents a reshaping operator, D represents a depth convolution layer, and P represents a point-wise convolution layer; on this basis, the region self-attention calculation is performed within each window feature block, which is represented as:

[0023]

[0024] Wherein, α represents a learnable parameter, represents the transpose of K i vector; the region attention map A∈R C×C is used to replace the traditional attention map with size A∈R HW×HW , and an information exchange mechanism is introduced to enhance the information exchange between adjacent windows through a moving window;

[0025] The features processed by the local window attention module are reassembled back to the original resolution, and are connected with the input features in residual connection as X′∈R H×W×CX' is taken as the input of the second-stage global self-attention module; in the second stage, local information is first captured from X' using 1x1 and 3x3 depth convolutions; then a different attention mechanism is adopted than in the first stage, instead of dividing windows, the entire feature map is taken as a sequence, the channels C' are divided into multiple heads h, and different self-attention maps are learned simultaneously, and the process of the global self-attention module is represented as follows:

[0026] Attention(Q, K, V) = VReLu(QK T / β)

[0027] wherein β represents a learnable parameter; and the information exchange between the multiple heads is enhanced by fusing the multi-head feature mechanism.

[0028] As described previously, in the dynamic upsampling module, first, the feature X with the size of CxHxW is input, and is sent to the sampling point generator, and a set of sampling sets S with the size of 2gxsHx sW after dynamic adjustment is generated by the sampling point generator, wherein C, H and W respectively represent the number of channels, height and width of the feature map, g represents the number of groups, and s represents the scale factor; then the input feature is resampled by the grid_sample grid sampling function to obtain the feature X' with the size of CxsHx sW after upsampling, and the dynamic upsampling of the feature is realized:

[0029] X' = grid_sample(X, S).

[0030] As described previously, in the dynamic upsampling module of the night pedestrian detection method based on artificial intelligence technology, the point-by-point dynamic range factor is generated by linearly projecting the input feature, the sigmoid function is multiplied by the 0.5 static range factor, so that the dynamic range constraint is equivalent to the original static factor; the generation formula of the offset O is:

[0031] O = 0.5sigmoid(linear1(X))·linear2(X)

[0032] wherein the offset O is obtained by adjusting the shape through the pixelshuffle pixel reshaping operation, and then is added to the original sampling grid G to obtain the final sampling set S.

[0033] The night pedestrian detection method based on artificial intelligence technology, the expansion reparameterization block chain module adopts a double-path design and simultaneously processes feature information of different resolutions; an input tensor is first divided into a main branch and a shortcut branch by a segmentation strategy of a CSP network, the main branch is first subjected to 1*1 convolution for channel dimension reduction, and then enters a core processing unit composed of three series-connected expansion reparameterization blocks, each block gradually extracts and enhances feature representation; the shortcut branch is only projected to the same channel dimension through a 1*1 convolution; the outputs of the main branch and the shortcut branch are combined through an element-wise addition operation, and after processing, a 1*1 convolution is performed to form the final output of the expansion reparameterization block chain module.

[0034] The night pedestrian detection method based on artificial intelligence technology, in the expansion reparameterization block, a 5*5 depth separable convolution is used as the main branch, and two 3*3 convolutions with expansion rates of 1 and 2 are used as auxiliary branches, and the entire expansion reparameterization block chain module forms a feature extraction chain by connecting three expansion reparameterization blocks in series.

[0035] The night pedestrian detection method based on artificial intelligence technology, the expansion reparameterization block chain module uses multiple parallel branches in the training stage, but in the inference stage, the multiple parallel branches are converted and fused into a single efficient convolution operation through the reparameterization technology, and the reparameterization formula is as follows:

[0036]

[0037] Wherein, W fused and b fused represent the weights and biases after reparameterization fusion, W original and b original represent the weights and biases of the main branch convolution kernel, and represent the weights and biases of the i th expansion convolution kernel, D i represents the expansion factor of the i th branch, and pad_to_size represents a function of filling the expansion convolution kernel to match the original convolution kernel size.

[0038] The beneficial effects of the present application are:

[0039] (1) In the present application, the proposed NE-RTDFTR model solves the problems of insufficient feature extraction and insufficient fusion accuracy of the original RT-DETR in low-light night scenes by constructing a neck enhancement network containing a spatial transformer, a dynamic upsampling and an expanded reparameterization block chain. The two-stage attention mechanism (local window and global self-attention) of the spatial transformer can accurately capture the local details and global context of pedestrians, such as fuzzy outlines, limb features, and the spatial relationship between pedestrians and the environment. The dynamic upsampling module optimizes the alignment of multi-scale features by adaptively adjusting the sampling point density, especially improving the feature representation of small target pedestrians (such as distant pedestrians). The expanded reparameterization block chain balances the detection speed and accuracy while expanding the receptive field through multi-branch dilated convolution and reparameterization technology. The synergistic effect of the above three improves the feature processing capability in night low-light, blur, and occlusion scenes, and reduces the miss and false detection rates compared with the original RT-DETR and traditional Y0L0 series models.

[0040] (2) In the present application, ResNet18 is used as the backbone network and cooperates with the neck enhancement network to form an efficient feature extraction processing link. ResNet18 outputs multi-scale feature vectors through hierarchical convolution and progressive downsampling strategy, and all convolution layers are followed by batch normalization and ReLU activation function to ensure stable extraction of night low-quality image features. The neck enhancement network further optimizes the features output by the backbone network, enhances feature correlation through spatial transformer, corrects scale deviation through dynamic upsampling, and strengthens fuzzy feature analysis through expanded reparameterization block chain. This "backbone extraction-neck fine-tuning" architecture can more fully extract effective features of night pedestrians, especially in uneven light and partially occluded target scenes, showing stronger feature robustness.

[0041] (3) In the present application, a dynamic adjustment mechanism is introduced to make the model adapt to the variability of night scenes. The dynamic upsampling module generates a dynamic sampling set through a sampling point generator, and adjusts the sampling density in real time according to the feature distribution, solving the problem of insufficient feature extraction of small targets and fuzzy targets by traditional static upsampling (such as bilinear interpolation). The expanded reparameterization block chain processes different resolution features in parallel in the training stage, and fuses them into a single efficient convolution in the inference stage, balancing the richness and real-time requirements of feature extraction. In addition, HSV adjustment (hue, saturation, brightness) in the data preprocessing stage further enhances the diversity of training data and improves the model's generalization ability to different night lighting conditions. Compared with traditional methods that rely on fixed parameters, this dynamic adaptation mechanism enables NE-RTDETR to maintain stable detection performance in complex night scenes such as rain and fog, backlight, and light interference. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1A schematic diagram of the overall process of the present application;

[0043] Figure 2 A schematic diagram of the structure of the NE-RTDETR model in the embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of the backbone network ResNet18 in the embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of the spatial transformer module in the embodiment of the present application;

[0046] Figure 5 A schematic diagram of the sliding window and fusion multi-head operation in the embodiment of the present application, wherein (a) is a schematic diagram of the sliding window operation, and (b) is a schematic diagram of the fusion multi-head operation;

[0047] Figure 6 A schematic diagram of the sampling process of the DySample module in the embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of the sampling point generator in the embodiment of the present application;

[0049] Figure 8 A schematic diagram of the structure comparison of the DRBC3 module and the RepC3 module in the embodiment of the present application, wherein (a) is a schematic diagram of the structure of the RepC3 module, and (b) is a schematic diagram of the structure of the DRBC3 module;

[0050] Figure 9 A schematic diagram of the structure of the inflated heavy reference block in the embodiment of the present application;

[0051] Figure 10 A comparison diagram of the detection effects of the RT-DETR algorithm and the NE-RTDETR algorithm in the embodiment of the present application, wherein (a) is a schematic diagram of the detection effect of the RT-DETR algorithm, and (b) is a schematic diagram of the detection effect of the NE-RTDETR algorithm of the present application. DETAILED DESCRIPTION

[0052] The night pedestrian detection method based on artificial intelligence technology provided in the embodiment, as shown in the figure, comprises the following steps: Figure 1

[0053] S1, a night road pedestrian data set is prepared in advance and a pretreatment operation is performed, specifically comprising the following steps:

[0054] S1.1, a plurality of night road pedestrian images are obtained to form a night road pedestrian data set, and the night road pedestrian data set contains night pedestrian targets of different scales and different blur degrees;

[0055] ​S1.2, proportionally divide the night road pedestrian dataset into a training set, a validation set and a test set;

[0056] S1.3, perform a data enhancement operation on the training set image, and set the HSV adjustment hue, saturation and brightness.

[0057] In this embodiment, in order to realize the pedestrian detection task under low light at night, the visible light images in the LLVIP public dataset are selected and screened to reserve a small sample of night scene images, a total of 4065 images, and are subdivided into 2939 training set images, 840 validation set images and 420 test set images in the ratio of 7:2:1, so as to enhance the model generalization ability and shorten the training period.

[0058] S2, construct a night pedestrian detection model NE-RTDETR based on RT-DETR neck enhancement, input the night road pedestrian dataset into the NE-RTDETR model for training, and obtain the trained best weight file; as shown in Figure 2 The NE-RTDETR model includes a backbone network ResNet18, a neck enhancement network (NE) and an RTDETRDecoder decoder; step S2 specifically includes the following sub-steps:

[0059] S2.1, cut the images in the night road pedestrian dataset into three-channel images with a size of 640x640 pixels;

[0060] S2.2, input the cut images into the backbone network ResNet18 of the NE-RTDETR model to obtain high-dimensional feature vectors;

[0061] S2.3, input the high-level features extracted by the backbone network ResNet18 into the neck enhancement network, NE, to extract and fuse the features, and obtain the fused features;

[0062] S2.4, input the fused features into the RTDETRDecoder decoder to finally obtain the prediction result.

[0063] In this embodiment, the experimental environment configuration is set as follows: the system environment is Windows 11, the graphics card (GPU) is NVIDIA RTX 4060 (8G), the deep learning framework is Pytorch 2.5, the acceleration environment is CUDA 12.4, and the language is Python 3.10.16; uniform parameters are used during training, and the experimental parameter configuration is shown in Table 1.

[0064] Table 1 Experimental parameter configuration table

[0065]

[0066] AsFigure 3 As shown, the backbone network ResNet18 extracts features of different scales from the input image through hierarchical convolution (ConvNormLayer) and BasicBlock operations, and takes the feature maps S3, S4 and S5 output by the last three stages as core processing data input into the subsequent network; the backbone network ResNet18 adopts a progressive down-sampling strategy, which, while maintaining the expansion of the receptive field, alleviates the gradient vanishing problem through residual connection; batch normalization and ReLU activation functions are used after all convolution layers to ensure training stability.

[0067] The neck enhancement network (NE) includes a spatial transformer (FDT) module, a dynamic up-sampling (DySample) module and a dilated reparameterization block chain (DRBC3) module, and the collaborative design optimizes the effective detection accuracy and reduces the false detection and missed detection rate.

[0068] To address the limitations of AIFI and improve the pedestrian detection capability in low light conditions, the FDT module is introduced, which combines the local window attention mechanism (RSA) and the global self-attention mechanism (GSA); as shown in Figure 4 The FDT module adopts a two-stage design, which includes an RSA module and a GSA module; in the first stage, the input features are first normalized to X∈R H×W×C and then sent to the RSA module, where H, W and C represent height, width and channel number respectively; the RSA module adopts a window-based local attention mechanism to divide the input features into fixed-size windows:

[0069] {X1, X2,..., X n )=Split(X)

[0070] wherein represents a group of window feature blocks, N represents the window size and

[0071] Subsequently, by combining 1x1 and 3x3 depth convolution to capture local detail features, the local spatial context information is enhanced to solve the problem of blurred pedestrian edges and unclear details in low light night environment; then each window feature block X i is projected as query key and value The process is as follows:

[0072] {Q i , K i , V i}=RS(D(P(X i ))

[0073] In this context, RS represents the reshaping operator, D represents a depthwise convolutional layer, and P represents a pointwise convolutional layer.

[0074] Based on this, region self-attention is calculated within each window feature block, as follows:

[0075]

[0076] Where α represents a learnable parameter, K represents i Transpose of a vector; use A∈R to reduce computational complexity. C ×C Region attention maps replace the traditional type A∈R HW×HW Attention map; such as Figure 5 As shown in (a), an information exchange mechanism is introduced to facilitate information exchange between adjacent windows by moving the window. This enhances the detailed features of the region and local areas.

[0077] The features processed by the RSA module are reassembled back to the original resolution and residually concatenated with the input features as X′∈R H×W×C X′ is used as the input to the second-stage GSA module. In the second stage, 1×1 and 3×3 depthwise convolutions are first used to capture local information from X′ to ensure accurate recovery of details. Then, a similar attention mechanism as the first stage is adopted, but instead of segmenting the window, the entire feature map is treated as a sequence, the channel C′ is divided into multiple heads h, and different self-attention maps are learned simultaneously. The process of the global self-attention module is represented as follows:

[0078] Attention(Q,K,V)=VReLU(QK T / β)

[0079] Where β represents a learnable parameter; for example Figure 5 As shown in (b), the information exchange between multiple heads is enhanced by fusing multi-head feature mechanisms, thereby enhancing the local and global features of the input image.

[0080] Traditional upsampling modules (such as bilinear interpolation, transposed convolution, or pixel rearrangement) are clearly insufficient in handling the multiple challenges of complex nighttime detection scenarios. The method in this embodiment constructs a dynamic upsampling (DySample) module, which increases the sampling density of small target regions by introducing a dynamic and learnable spatial resampling mechanism, providing more refined feature representations for these small targets. Traditional methods simply superimpose or stitch features, which can lead to misalignment problems, while DySample can learn to compensate for this offset, achieving more accurate feature alignment and fusion.

[0081] like Figure 6As shown, in the DySample module, the feature X of size C×H×W is first input and then fed into the following... Figure 7 In the sampling point generator shown, a dynamically adjusted sampling set S of size 2g×sH×sW is generated, where C, H, and W represent the number of channels, height, and width of the feature map, respectively, g represents the number of groups, and s represents the scale factor. Subsequently, the input features are resampled using the grid_sample function to obtain an upsampled feature X′ of size C×sH×sW, thus achieving dynamic upsampling of the features.

[0082] X′ = grid_sample(X, S).

[0083] In the sampling point generator of the DySample module, the normalization operation causes the output feature values ​​to be distributed in the interval [-1, 1], resulting in local s 2 The offset ranges at each sampling location exhibit significant overlap; this overlap impairs the prediction accuracy of boundary regions, and its accumulated error propagates in subsequent stages, ultimately leading to output artifacts. To address this challenge and increase the flexibility of the offset, a point-by-point dynamic range factor is generated by linearly projecting the input features. The sigmoid function is multiplied by a 0.5 static range factor, ensuring that the dynamic range constraint remains equivalent to the original static factor. The formula for generating the offset O is:

[0084] O=0.5sigmoid(linear1(X))·linear2(X)

[0085] The offset O is obtained by adjusting the shape through the pixel shuffle operation, and then added to the original sampling grid G ​​to obtain the final sampling set S.

[0086] The dataset collected at night mainly consists of pedestrian target images of different scales, and most of them are blurry. In the feature fusion stage, methods such as... Figure 8 (a) Although the RepC3 module shown has certain processing capabilities, it has limitations in the face of this complex scene. The RepC3 module adopts residual connection and cross-stage feature fusion design, which can enhance the feature extraction capability to a certain extent. However, its fixed receptive field structure is difficult to adapt to the detection requirements of targets of different scales. In order to further solve this problem, this embodiment introduces the DRBC3 module on the basis of RepC3.

[0087] The DRBC3 module inherits the CSP (Cross Stage Partial) network segmentation mechanism from RepC3 and replaces standard convolutions with DRB blocks. It employs an innovative dual-path design to simultaneously process feature information at different resolutions, thereby enhancing its feature extraction capabilities for blurred targets; for example... Figure 8(b) As shown, in the DRBC3 module, the input tensor is first divided into two parallel paths, the main branch and the shortcut branch, by the split strategy of the CSP network. The main branch first passes through a 1x1 convolution for channel dimension reduction, and then enters the core processing unit composed of three series of Dilated Reparam Blocks (DRB). Each block gradually extracts and enhances the feature representation. The shortcut branch is only projected to the same channel dimension through a 1x1 convolution. The outputs of the main branch and the shortcut branch are combined by element-wise addition operation, and then a 1x1 convolution is performed to form the final output of the DRBC3 module after processing.

[0088] The DRB block is the core component of DRBC3, which optimizes the problems of the original RepConv fixed convolution kernel and receptive field. It uses a large kernel convolution and multiple small kernel convolutions with different dilation rates to form a parallel structure. To meet the demand of detection speed, the DRBC3 module effectively simplifies the complex convolution operation during the inference stage, converting it into an efficient 5x5 convolution. This not only retains the advantage of multi-scale feature extraction, but also significantly reduces the computational complexity and the time required for inference.

[0089] As shown in Figure 9 , the DRB block contains a 5x5 depth separable convolution as the main branch, and two 3x3 convolutions with dilation rates of 1 and 2 as auxiliary branches. This design enables the module to efficiently capture features of different scales, making it particularly suitable for processing various target sizes in multi-scale target detection. The entire DRBC3 module forms a powerful feature extraction chain by concatenating three DRB blocks, which not only improves the model's expression ability, but also ensures inference efficiency through the reparameterization technique.

[0090] The combination of the above two parts makes DRBC3 not only maintain the efficient characteristics of the CSP network, but also enhance the feature extraction ability for blurred targets through dilated convolution. During the training stage, multiple parallel branches are used, but during the inference stage, the reparameterization technique is used to convert and fuse multiple parallel branches into a single efficient convolution operation. The reparameterization formula is as follows:

[0091]

[0092] where W fused and b fused represent the weights and biases after reparameterization fusion, W original and b original represent the weights and biases of the main branch convolution kernel, and represent the weights and biases of the i-th dilated convolution kernel, D irepresents the dilation factor of the i-th branch, and pad_to_size represents the function that pads the dilated convolution kernel to match the size of the original convolution kernel.

[0093] The RTDETRDecoder decoder mainly consists of head detection. It uses IoU-aware query selection technology to select a fixed number of image features from the feature sequence output by the encoder as the initial object query for the decoder. At the same time, it introduces DINO's "denoising strategy" to optimize sample matching accuracy and accelerate model convergence.

[0094] S3. Based on the trained weights, input the nighttime road pedestrian images to be detected into the NE-RTDETR model to obtain information on whether pedestrians exist in the nighttime road pedestrian images.

[0095] S4. Output the detection results and evaluate the model performance.

[0096] To objectively evaluate the detection performance of the algorithm model, this embodiment uses Mean Average Precision (mAP), number of model parameters, and Recall (R) to assess the model's performance. mAP employs two different Intersection over Union (IoU) thresholds: one is fixed at 0.5, named mAP0.5, and the other IoU threshold varies between 0.5 and 0.95, referred to as mAP0.5:0.95. Precision (P) is used to assess the degree of false positives, while recall assesses the degree of false negatives. The definitions of precision and recall are shown in the following formulas:

[0097]

[0098] Wherein, TP (True Positive) represents a positive sample predicted as true, FP (False Positive) represents a positive sample predicted as false, and FN (False Negative) represents a negative sample predicted as false.

[0099] The average precision (AP) of a single-class label is defined as follows:

[0100]

[0101] mAP represents the quotient of the sum of the average precision of all labels and the total number of categories M. The definition of mAP is as follows:

[0102]

[0103] To verify the effect of the improved algorithm, the model of the embodiment is compared with YOLOv5m, YOLOv8m, YOLOvl1m, RTDETR-r18 and other algorithms with similar parameter quantities, and the results are shown in Table 2.

[0104] Table 2 Detection effect of each model on the LLVIP visible light dataset

[0105]

[0106] As shown in Table 2, the detection effect of the model of the embodiment is better than that of the mainstream models such as YOLOv5m, YOLOv8m, YOLOvl1m and RTDETR-r18, and the detection effect of the RTDER series model is obviously better than that of the YOLO series. In a good baseline model, the characteristics of the data set in the low-light complex environment make the detection more difficult. Compared with the original RTDETR-r18 model, the mAP0.5, Precision and Recall are improved by 1.39, 2.21 and 3 respectively while the parameter quantity is reduced, the pedestrian detection in low-light conditions is more accurate and the missed detection rate is significantly reduced, and the pedestrian detection effect is improved under lower system overhead.

[0107] To evaluate the detection effect and generalization ability of the improved algorithm of the embodiment on different night pedestrian datasets, further comparative experiments are performed on the NightSurveillance dataset. As shown in the experimental results of Table 3, the improved model is better than RTDETR-r18 in each index, especially the Precision is improved by 3.44%, which enhances the detection accuracy, and the inference speed is also significantly improved to meet the real-time requirement.

[0108] Table 3 Detection effect of the model of the embodiment on the NightSurveillance dataset

[0109]

[0110] To evaluate the influence of each improvement point on pedestrian detection under low-light conditions, a plurality of ablation experiments are performed on the LLVIP visible light dataset, and the results are shown in Table 4.

[0111] Table 4 Ablation experiment data table

[0112]

[0113] From the data in Table 4, the neck optimization algorithm proposed in the embodiment shows significant improvement in each evaluation index compared with the original RTDETR-R18 algorithm. The FDT attention mechanism is used to process the top-level features, which enhances the context modeling capability, and the parameter quantity slightly increases. The mAP0.5 of the model is improved by 0.88, the Precision is improved by 0.89, the Recall is improved by 0.74, and the inference speed is significantly improved. After introducing Dysample in the feature fusion path, it can adaptively process the feature scale and strengthen the sampling density of small targets. The feature extraction core module DRBC3 is optimized in terms of calculation accuracy and parameter quantity. It expands the receptive field by fusing multiple dilated rate convolutions and reparameterization techniques, effectively capturing target details. The introduction of FDT and DySample at the top layer of the neck network can more accurately locate the key area based on rich semantic information, and accurately align and fuse global features and local features. The mAP0.5, Precision and Recall are improved by 1.31, 1.02 and 1.93 percentage points, respectively.

[0114] On this basis, further introduce DRBC3 to expand the receptive field and extract more rich features. The final model reduces the parameter quantity while improving the mAP0.5, Precision and Recall by 1.39, 2.21 and 3 percentage points, respectively, and significantly improves the inference speed. The ablation experiment results show that the RTDETR algorithm with neck optimization can achieve a good balance between detection accuracy, recall rate and parameter quantity, and improve the inference speed.

[0115] As shown in Figure 10 Fig. (a) is a detection effect diagram of the RT-DETR algorithm, and Fig. (b) is a detection effect diagram of the NE-RTDETR algorithm of the embodiment. The improved algorithm proposed in the patent has a significant effect on accuracy and missed detection in low light, blur and multi-scale small target conditions.

[0116] As Figure 10 can be seen, the RTDETR-R18 algorithm has problems of missed detection and inaccurate positioning for pedestrian detection under low light conditions, and the detection effect of small targets is poor. The algorithm of the embodiment can reduce the interference of background information and improve the detection effect of small targets. It can detect more pedestrian targets under low light conditions, which proves that the model of the embodiment has higher detection accuracy and lower risk of missed detection.

[0117] In addition to the above embodiments, the present application can have other implementation manners. Any technical solution formed by equivalent substitution or equivalent transformation falls within the protection scope required by the present application.

Claims

1. A nighttime pedestrian detection method based on artificial intelligence technology, characterized in that: Includes the following steps: S1. Prepare the nighttime pedestrian dataset in advance and perform preprocessing operations; S2. Construct a nighttime pedestrian detection model NE-RTDETR based on RT-DETR neck enhancement. Input the nighttime road pedestrian dataset into the NE-RTDETR model for training to obtain the trained best weight file. The NE-RTDETR model includes a backbone network ResNet18, a neck enhancement network, and a decoder. The backbone network ResNet18 is used to output feature vectors. The neck enhancement network is used for feature extraction and fusion, outputting the fused features. The decoder is used to output the prediction results; S3. Based on the trained weights, input the nighttime road pedestrian images to be detected into the NE-RTDETR model to obtain information on whether there are pedestrians in the nighttime road pedestrian images; S4. Output the detection results and evaluate the model performance.

2. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 1, characterized in that: Step S1 specifically includes the following sub-steps: S1.

1. Acquire several nighttime road pedestrian images to form a nighttime road pedestrian dataset. The nighttime road pedestrian dataset contains nighttime pedestrian targets of different scales and different degrees of blur. S1.2 Divide the nighttime road pedestrian dataset into training set, validation set and test set according to the proportions; S1.3 Perform data augmentation on the training set images, setting HSV to adjust hue, saturation, and brightness.

3. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 1, characterized in that: In step S2, the backbone network ResNet18 extracts features from the input image at different scales through hierarchical convolution and basic block operations, and inputs the feature maps output from the last three stages into the subsequent network. The backbone network ResNet18 employs a progressive downsampling strategy and uses batch normalization and ReLU activation functions after all convolutional layers.

4. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 1, characterized in that: In step S2, the neck enhancement network includes a spatial domain transformer module, a dynamic upsampling module, and an extended reparameterized block chain module. The spatial domain transformer module adopts a two-stage design. In the first stage, the input features are divided into fixed-size windows through a local window attention mechanism. Then, each window feature block with enhanced local context is projected as a query, key, and value. Finally, regional self-attention calculation is performed within each window feature block. In the second stage, the channel is divided into multiple heads through a global attention mechanism, and different self-attention maps are learned simultaneously.

5. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 4, characterized in that: The spatial domain transformer module includes a local window attention module and a global self-attention module. In the first stage, the input features are first normalized to X∈R. H×W×C The input features are then fed into the local window attention module, where H, W, and C represent height, width, and number of channels, respectively. The local window attention module employs a window-based local attention mechanism, dividing the input features into windows of fixed size. {X1,X2,...,X n }=Split(X) in, Let N represent a set of window feature blocks, and N represent the window size. Subsequently, local detail features are captured by combining 1×1 and 3×3 depthwise convolutions, and then each window feature block X that enhances the local context is processed. i Projection for query key and value The process is shown in the following formula: {Q i ,K i ,V i }=RS(D(P(X i ))) Where RS represents the reshaping operator, D represents the depthwise convolutional layer, and P represents the pointwise convolutional layer; based on this, region self-attention is calculated within each window feature block, as shown below: Where α represents a learnable parameter, K represents i Transpose of a vector; using A∈R C×C Region attention maps replace the traditional size A∈R HW×HW The attention graph is used, and an information exchange mechanism is introduced to enhance the information exchange between adjacent windows by moving the window; The features processed by the local window attention module are reassembled back to the original resolution and residually connected with the input features as X′∈R. H×W×C X′ is used as the input to the second-stage global self-attention module. In the second stage, 1×1 and 3×3 depthwise convolutions are first used to capture local information from X′. Then, a different attention mechanism is adopted from the first stage. Instead of segmenting the window, the entire feature map is treated as a sequence, and the channel C′ is divided into multiple heads h. Different self-attention maps are learned simultaneously. The process of the global self-attention module is represented as follows: Attention(Q,K,V)=VReLU(QK T / β) Here, β represents a learnable parameter; and information exchange between multiple heads is enhanced by fusing multi-head feature mechanisms.

6. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 4, characterized in that: In the dynamic upsampling module, a feature X of size C×H×W is first input and fed into a sampling point generator. The sampling point generator generates a dynamically adjusted sampling set S of size 2g×sH×sW, where C, H, and W represent the number of channels, height, and width of the feature map, respectively, g represents the number of groups, and s represents the scale factor. Subsequently, the input feature is resampled using the grid_sample function to obtain an upsampled feature X′ of size C×sH×sW, thus achieving dynamic upsampling of the feature. X′ = grid_sample(X, S).

7. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 6, characterized in that: In the dynamic upsampling module, a point-by-point dynamic range factor is generated by linearly projecting input features. The sigmoid function is multiplied by the 0.5 static range factor to ensure that the dynamic range constraint remains equivalent to the original static factor. The formula for generating the offset O is: O=0.5sigmoid(linear1(X))·linear2(X) The offset O is obtained by adjusting the shape through the pixel shuffle operation, and then added to the original sampling grid G ​​to obtain the final sampling set S.

8. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 4, characterized in that: The dilated reparameterized block chain module adopts a dual-path design to process feature information at different resolutions simultaneously. The input tensor is first divided into two parallel paths, the main branch and the shortcut branch, through the segmentation strategy of the CSP network. The main branch first undergoes channel dimensionality reduction through 1×1 convolution, and then enters the core processing unit consisting of three cascaded dilated reparameterized blocks. Each block gradually extracts and enhances the feature representation. The shortcut branch is projected to the same channel dimension through only a 1×1 convolution; the outputs of the main branch and the shortcut branch are merged by element-wise addition, and after processing, they are converted into the final output of the dilated reparameterized block chain module by a 1×1 convolution.

9. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 8, characterized in that: In the dilated reparameter block, a 5×5 depthwise separable convolution is used as the main branch, and two 3×3 convolutions with dilation rates of 1 and 2 are used as auxiliary branches. The entire dilated reparameter block chain module forms a feature extraction chain by connecting three dilated reparameter blocks in series.

10. The nighttime pedestrian detection method based on artificial intelligence technology according to claim 9, characterized in that: The extended reparameterized block chain module uses multiple parallel branches during the training phase, but during the inference phase, it transforms and merges these multiple parallel branches into a single, efficient convolutional operation through reparameterization technology. The reparameterization formula is as follows: Among them, W fused and b fused W represents the weights and biases after reparameterized fusion. original and b original This represents the weights and biases of the main branch convolution kernel. and D represents the weights and biases of the i-th dilated convolution kernel. i represents the dilation factor of the i-th branch, and pad_to_size represents the function that pads the dilated convolution kernel to match the size of the original convolution kernel.

Images