Methods, devices, systems, and storage media for detecting vehicle high beams in complex environments

Abstract

This invention discloses a method, apparatus, system, and storage medium for detecting vehicle high beams in complex environments. The method includes: acquiring photos of vehicle high beams in complex environments; annotating the acquired photos using the Labelimg annotation tool and dividing the processed data proportionally into training, validation, and test sets; designing an improved NanoDetPlus network model, employing multi-scale convolutional kernels, coordinated attention, and a squeeze excitation module to improve the NanoDetPlus model; inputting data from the training and validation sets into the improved NanoDetPlus model for training and validation to obtain an optimized NanoDetPlus algorithm network model; and inputting data from the test set into the optimized NanoDetPlus model for detection, simulating the detection effect in real-world environments to obtain vehicle high beam detection results in complex environments.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, device, system, and storage medium for detecting vehicle high beams in complex environments. Background Technology

[0002] For vehicle high beam detection under normal weather conditions, there are currently morphology-based methods for detecting the on / off status of high beams on vehicles driving at night, nighttime high beam detection methods that integrate multi-target tracking and evidence chain evidence collection based on machine vision technology, novel vehicle high beam intensity calibration methods based on computer vision, and algorithms such as YOLOv3 and YOLOv8.

[0003] However, a more pressing issue is that in complex weather conditions such as rain and snow, slippery and reflective road surfaces further increase the scattering and reflection range of high beams, significantly impacting driver vision and greatly increasing the risk of traffic accidents. Current research has yet to find an effective solution to this problem.

[0004] To address this issue, a vehicle high beam detection model, NanoDetPlus, was proposed to suit complex weather conditions such as rain and snow. However, due to the lack of sufficient publicly available datasets, a large-scale real-world dataset was established in collaboration with regulatory authorities and enterprises to validate the proposed model. The results show that the model has significant advantages. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention provides a method, device, system, and storage medium for detecting vehicle high beams in complex environments.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for detecting vehicle high beams in complex environments includes:

[0008] S1. Collect photos of vehicle high beams in complex environments, use the Labelimg annotation tool to annotate the photos, and divide the processed data into training set, validation set and test set according to the proportion;

[0009] S2. Design an improved NanoDetPlus network model;

[0010] S3. Input the data from the training set and validation set into the improved NanoDetPlus model for training and validation to obtain the optimized NanoDetPlus algorithm network model;

[0011] S4. Input the data from the test set into the optimized NanoDetPlus model for detection, simulate the detection effect in a real environment, and obtain the detection results of vehicle high beams in complex environments.

[0012] S5 is compared with detection algorithms such as Efficient, Faster RCNN, RTDETR, YOLOv8n, and YOLOv12n to verify detection accuracy and speed.

[0013] Preferably, in step S2, the NanoDetPlus model is improved by using multi-scale convolution kernels, coordinated attention, and squeeze excitation modules.

[0014] Preferably, in step S2, a multi-scale convolutional feature enhancement module (MCE) is designed in the Backbone. The MCE uses four convolutions (1*1, 3*3, 5*5, and 7*7) to perform parallel operations on the input feature map. The outputs of the 3*3, 5*5, and 7*7 branches are added to the output of the 1*1 branch. A coordinated attention module is introduced in Outstage4 of the Backbone, located at the deepest layer of the Backbone. A squeeze-and-excitation convolutional module (SEC) is designed in the Neck, consisting of a squeeze-and-excitation block and a 1×1 convolution.

[0015] The present invention also provides a vehicle high beam detection device for complex environments, comprising:

[0016] The first processing module is used to collect photos of vehicle high beams in complex environments, use the Labelimg annotation tool to annotate the photos, and divide the processed data into training set, validation set and test set according to the proportion;

[0017] The second processing module is used to design the improved NanoDetPlus network model;

[0018] The third processing module is used to input the data from the training set and validation set into the improved NanoDetPlus model for training and validation, so as to obtain the optimized NanoDetPlus algorithm network model.

[0019] The fourth processing module is used to input the data from the test set into the optimized NanoDetPlus model for detection, simulate the detection effect in a real environment, and obtain the detection results of vehicle high beams in complex environments.

[0020] As a preferred approach, the NanoDetPlus model is improved by employing multi-scale convolution kernels, coordinated attention, and squeezed excitation modules.

[0021] As a preferred design, the improved NanoDetPlus network model incorporates a multi-scale convolutional feature enhancement module (MCE) within the Backbone. The MCE uses four convolutions (1x1, 3x3, 5x5, and 7x7) to perform parallel operations on the input feature map. The outputs of the 3x3, 5x5, and 7x7 branches are added to the output of the 1x1 branch. A coordinated attention module is introduced in Outstage4 of the Backbone, located at the deepest layer. A squeeze-and-excitation (SEC) module is designed within the Neck, consisting of a squeeze-and-excitation block and a 1x1 convolution.

[0022] The present invention also provides a vehicle high beam detection system in complex environments, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a vehicle high beam detection method in complex environments when executed by the processor.

[0023] The present invention also provides a storage medium storing a computer program, which executes a method for detecting vehicle high beams under complex environments during runtime.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] To address the lack of available sample datasets, a sample database was established. An improved NanoDetPlus algorithm was used to detect vehicle high beams, achieving an effective balance between detection accuracy and speed. Firstly, a multi-scale convolution feature enhancement module (MCE) and a coordinated attention module (CA) were custom-designed in the backbone network. MCE captures features from different receptive fields through multi-scale convolutional kernels and enhances the high beam features through feature addition and concatenation. CA fully utilizes semantic information for spatial location encoding and channel weight adjustment, enhancing sensitivity to key regions and categories, thus improving the classification confidence and boundary localization accuracy of vehicle high beams. Secondly, a squeeze excitation and conv module (SEC) was designed in the neck network to effectively improve the network's feature representation capabilities. This invention improves the accuracy and recognition efficiency of vehicle high beam detection algorithms while reducing computational load, demonstrating strong practicality. Attached Figure Description

[0026] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the vehicle high beam detection method under complex environments according to the present invention;

[0028] Figure 2(a) is a schematic diagram of the data flow from input to output;

[0029] Figure 2(b) is a schematic diagram of the structure of the Multi-scale Convolution Feature Enhancement Module (MCE);

[0030] Figure 2(c) is a schematic diagram of the CoordinateAttention (CA) module;

[0031] Figure 2(d) is a schematic diagram of the Squeeze Excitation and Conv (SEC) module;

[0032] Figure 3 This is a schematic diagram of a vehicle high beam image database example in the vehicle high beam detection method under complex environments of the present invention;

[0033] Figure 4 This is a diagram showing the detection results of vehicle high beams in the vehicle high beam detection method under complex environments according to the present invention.

[0034] Figure 5(a) is a performance comparison chart of the number of parameters and the mean accuracy Map@0.5 for each model;

[0035] Figure 5(b) is a performance comparison chart of the computational complexity (FLOPs) and average precision (Map@0.5) of each model. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] Example 1:

[0039] like Figure 1 As shown, the present invention provides a method for detecting vehicle high beams in complex environments, comprising:

[0040] S1. Collect photos of vehicle high beams in complex environments, use the Labelimg annotation tool to annotate the photos, and divide the processed data into training set, validation set and test set according to the proportion;

[0041] S1 is implemented according to the following steps:

[0042] S1.1. Industrial video surveillance cameras from Zhejiang Dahua were installed on a gantry crane approximately 6 meters above the ground. Vehicle videos were then recorded at night to collect data on real urban roads. The video surveillance cameras output raw video at a resolution of 4096*2820.

[0043] S1.2. Based on the recorded video, we captured photos of the vehicle's high beams at four locations: 180 meters, 80 meters, 40 meters, and 25 meters away from the capture device.

[0044] S1.3. Use the well-known bounding box annotation software LabelImg to annotate the object and category information of vehicles with high beams in the image. It is important to note that when annotating the data, both vehicle features and high beam features must be considered. Considering only the high beam feature will lead to false positives. For example, a truck's low beam headlights are easily misidentified as a car's high beam headlights, and reflective road surfaces on rainy days are easily misidentified as vehicles with high beams. However, considering vehicle features can reduce the model's false positives. Finally, obtain YOLO format txt label files, Pascal VOC format xml label files, and COCO format json label files.

[0045] S1.4 Divide the vehicle high beam headlight photos and corresponding tag files into training set, validation set and test set in a ratio of 7:1:2.

[0046] S2. An improved NanoDetPlus network model is designed, employing multi-scale convolutional kernels, coordinated attention, and a squeeze excitation and conv module to improve the NanoDetPlus model and enhance detection accuracy. Figure 2 is a schematic diagram of the network structure of the improved NanoDetPlus target detection algorithm in a vehicle high beam detection method under complex environments according to the present invention. The improved NanoDetPlus mainly consists of a backbone network, a neck network, and detection heads. Among them, Figure 2(a) is a schematic diagram of the data flow from input to output; Figure 2(b) is a schematic diagram of the structure of the multiscale convolution feature enhancement module (MCE); Figure 2(c) is a schematic diagram of the structure of the coordinated attention module (CA); Figure 2(d) is a schematic diagram of the structure of the squeeze excitation and conv (SEC) module.

[0047] S2 is implemented according to the following steps:

[0048] S2.1 A multi-scale convolution feature enhancement module (MCE) was designed in Backbone, and its structure is shown in Figure 2(b). The MCE uses four convolutions—1*1, 3*3, 5*5, and 7*7—to perform parallel operations on the input feature map. The outputs of the 3*3, 5*5, and 7*7 branches are added to the output of the 1*1 branch, respectively. This design allows the large-kernel convolution to learn both local details and global contextual information simultaneously. The MCE captures features from different receptive fields through multi-scale convolutional kernels and achieves feature enhancement through feature addition and concatenation fusion.

[0049] S2.2. A Coordinate Attention (CA) module is introduced in Outstage4 of the Backbone, as shown in Figure 2(c). Stage4 is located at the deepest layer of the Backbone and has strong semantic information. Introducing the CA module here can fully utilize semantic information for spatial location encoding and channel weight adjustment, enhancing sensitivity to key regions and categories. This, in turn, improves classification confidence and boundary accuracy.

[0050] To enable attention blocks to accurately capture long-range spatial interactions and their location information, the CA module decomposes global pooling into a one-to-one one-dimensional feature encoding operation. For input X, CA uses two spatially expansive pooling kernels (H, 1) and (1, W) to encode each channel along the horizontal and vertical directions, respectively. Therefore, the output of the c-th channel at height h is... The calculation is shown in Formula 1. Similarly, the output of the c-th channel on width w is... The calculation is shown in Equation 2. The two transformations aggregate features along two spatial directions, generating a pair of direction-aware feature maps. These two transformations enable the attention module to capture long-range dependencies in one direction while preserving precise positional information in the other. This helps the network more accurately locate objects of interest.

[0051]

[0052] Equations 1 and 2 implement the global receptive field and encode precise location information. To utilize the resulting features, coordinate attention generation is proposed. For the aggregated feature maps generated by Equations 1 and 2, they are first concatenated and then fed into a transformation function F1 that shares a 1×1 convolution to obtain f, the calculation of which is shown in Equation 3.

[0053] f=δ(F1([z h ,z w ])) (3)

[0054] Here, [·, ·] denotes the connection operation along the spatial dimension. δ is a nonlinear activation function, f∈R C / r×(H+W) This is an intermediate feature map that encodes spatial information in both the horizontal and vertical directions. Here, r is a reduction ratio used to control the block size, similar to the setting in the SE module. Subsequently, f is partitioned along the spatial dimensions into two independent tensors f0. h ∈R C / r×H and f w ∈R C / r×W Additionally, through two 1×1 convolution transformations F h and F w f respectively h and f w Convert it to a tensor with the same number of channels as the input X, and then pass it through the sigmoid function to obtain g. h and g w g h and g w The calculations are shown in Formula 4 and Formula 5, respectively.

[0055] g h =σ(F h (f h(4)

[0056] g w =σ(F w (f w (5)

[0057] Where σ is the sigmoid function. The output g h and g w These are expanded and used as attention weights. The CA module ultimately outputs Y, y c The calculation is shown in Formula 6.

[0058]

[0059] A Squeeze Excitation and Conv (SEC) module was designed in Neck. The SEC module mainly consists of a Squeeze-and-Excitation block and 1×1 convolutions. The Squeeze-and-Excitation block improves the network's feature representation capability by explicitly modeling the interdependencies between its feature channels. This module enables the network to recalibrate features, selectively highlighting important features while suppressing less useful features. The 1×1 convolutions are responsible for adjusting the number of output channels.

[0060] Input feature map X is processed by convolution F tr After the operation, the output U = [u1, u2, ..., u c ], u c The calculation is shown in Formula 7.

[0061]

[0062] Where V = [v1, v2, ..., v c ] represents the set of filter kernels, v c This represents the parameters of the c-th filter. * indicates a convolution operation. '

[0063] X = [x 1 ,x 2 ,···,x C ]. A two-dimensional space kernel, representing v c One channel and acts on the corresponding channel of X.

[0064] The intermediate feature map U generates channel-wise statistics Z, Z∈R, through global average pooling. C Specifically, Z is generated by shrinking U along the spatial dimension H×W, where the calculation of the c-th element of Z is shown in Equation 8.

[0065]

[0066] To utilize the information accumulated during the squeezing operation, an excitation operation is then performed. This excitation operation aims to comprehensively capture the dependencies between channels. It uses a simple gating mechanism with a sigmoid activation function to obtain S, which is calculated as shown in Equation 9.

[0067] S = F ex (Z,W)=σ(g(Z,W))=σ(W2δ(W1Z)) (9)

[0068] In the formula, δ is the ReLU function and σ is the sigmoid function. To reduce model complexity and improve generalization ability, a bottleneck consisting of two fully connected layers is constructed around the nonlinear layers to parameterize the gating mechanism. One layer is a dimensionality reduction layer with parameter W1, a dimensionality reduction ratio of r, and the activation function ReLU. The other layer is a dimensionality increase layer with parameter W2. Finally, the outputs U and S are rescaled and transformed using the activation function to obtain... The calculation is shown in Formula 10.

[0069]

[0070] in, F scale (u c ,s c ) represents the feature map u c ∈R H×W and scalar s c Channel-level multiplication between them.

[0071] S3. Input the data from the training set and validation set into the improved NanoDetPlus model for training and validation to obtain the optimized NanoDetPlus algorithm network model;

[0072] S3 is implemented according to the following steps:

[0073] S3.1. Input the training set images and corresponding label files divided in S1.4 into the improved NanoDetPlus model obtained in S2 for training. The vehicle high beam image data under rain and snow conditions is as follows: Figure 3As shown. During training, the input image pixel size is 416*416, the batch size is set to 16, the worker is set to 4, and the epoch is 300. The optimizer is AdamW, the optimizer learning rate is 0.001, and the weight decay is 0.05. The warp is linear, the step size is 500, and the initial learning rate ratio is 0.0001.

[0074] S3.2 Validate the improved NanoDetPlus algorithm network model using a validation set and obtain the validation results. Analyze the differences between the training and validation results, such as whether there is overfitting or underfitting. If overfitting occurs, it means the model performs well on the training set but poorly on the validation set, indicating poor generalization ability. This needs to be addressed by increasing the number of training images, reducing model complexity, and fine-tuning hyperparameters. If underfitting occurs, it means the model performs poorly on both the training and validation sets. This needs to be addressed by increasing model complexity, fine-tuning hyperparameters, and adding more target features.

[0075] S3.3. By analyzing the loss curves during training and validation, as well as the accuracy and recall curves, the hyperparameters of the network model are fine-tuned, and the batch-size is adjusted to 8 to obtain the optimized NanoDetPlus model.

[0076] S4. Input the data from the test set into the optimized NanoDetPlus model for detection, simulate the detection effect in a real environment, and obtain the detection result image of vehicle high beams in a complex environment.

[0077] S4 is implemented according to the following steps:

[0078] The test set is input into the optimized NanoDetPlus model obtained in S3.3 to obtain the detection results and a visualization of the detection effect. The detection result diagram of vehicle high beams is shown below. Figure 4 As shown.

[0079] S5 is compared with detection algorithms such as Efficient, Faster RCNN, RTDETR, YOLOv8n, and YOLOv12n to verify detection accuracy and speed.

[0080] S5 is implemented according to the following steps:

[0081] S5.1. Input YOLO format data into YOLOv8n and YOLOv12n for model training and validation. Then, test the model on the test set to obtain the number of parameters, computational complexity, mean accuracy (mAP@0.5), mean accuracy (mAP@0.5:0.95), and detection speed. Input VOC format data into Faster R-CNN for model training and validation. Then, test the model on the test set to obtain the number of parameters, computational complexity, mean accuracy (mAP@0.5), mean accuracy (mAP@0.5:0.95), and detection speed. Input COCO format data into RTDETR, Efficient, and the optimized NanoDetPlus for model training and validation. Then, test the model on the test set to obtain the number of parameters, computational complexity, mean accuracy (mAP@0.5), mean accuracy (mAP@0.5:0.95), and detection speed.

[0082] S5.2. Compare the number of parameters, computational complexity, mean accuracy (mAP@0.5), and detection speed of each model, and select the optimal model based on comprehensive consideration. Figure 5(a) shows a performance comparison of the number of parameters and mean accuracy (mAP@0.5) for each model; Figure 5(b) shows a performance comparison of the computational complexity (FLOPs) and mean accuracy (mAP@0.5) for each model.

[0083] Example 2:

[0084] The present invention also provides a vehicle high beam detection device for complex environments, comprising:

[0085] The first processing module is used to collect photos of vehicle high beams in complex environments, use the Labelimg annotation tool to annotate the photos, and divide the processed data into training set, validation set and test set according to the proportion;

[0086] The second processing module is used to design the improved NanoDetPlus network model;

[0087] The third processing module is used to input the data from the training set and validation set into the improved NanoDetPlus model for training and validation, so as to obtain the optimized NanoDetPlus algorithm network model.

[0088] The fourth processing module is used to input the data from the test set into the optimized NanoDetPlus model for detection, simulate the detection effect in a real environment, and obtain the detection results of vehicle high beams in complex environments.

[0089] As one embodiment of the present invention, the NanoDetPlus model is improved by using multi-scale convolution kernels, coordinated attention, and squeeze excitation modules.

[0090] As one embodiment of the present invention, in designing the improved NanoDetPlus network model, a multi-scale convolutional feature enhancement module (MCE) is designed in the Backbone. The MCE uses four convolutions (1*1, 3*3, 5*5, and 7*7) to perform parallel operations on the input feature map. The outputs of the 3*3, 5*5, and 7*7 branches are added to the output of the 1*1 branch. A coordinated attention module is introduced in Outstage4 of the Backbone, located at the deepest layer of the Backbone. A squeeze-and-excitation convolutional module (SEC) is designed in the Neck, consisting of a squeeze-and-excitation block and a 1×1 convolution.

[0091] Example 3:

[0092] The present invention also provides a vehicle high beam detection system in complex environments, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a vehicle high beam detection method in complex environments when executed by the processor.

[0093] Example 4:

[0094] The present invention also provides a storage medium storing a computer program, which, when executed, performs a method for detecting vehicle high beams under complex environments.

[0095] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for detecting vehicle high beams under complex environments, characterized in that, include: S1. Collect photos of vehicle high beams in complex environments, use the Labelimg annotation tool to annotate the photos, and divide the processed data into training set, validation set and test set according to the proportion; S2. Design an improved NanoDetPlus network model; S3. Input the data from the training set and validation set into the improved NanoDetPlus model for training and validation to obtain the optimized NanoDetPlus algorithm network model; S4. Input the data from the test set into the optimized NanoDetPlus model for detection, simulate the detection effect in a real environment, and obtain the detection results of vehicle high beams in complex environments. S5. Compare with Efficient, Faster RCNN, RTDETR, YOLOv8n, and YOLOv12n detection algorithms to verify detection accuracy and speed; In step S2, the NanoDetPlus model is improved by using multi-scale convolution kernels, coordinated attention, and squeezed excitation modules; In step S2, a multi-scale convolutional feature enhancement module (MCE) is designed in the Backbone; wherein, the MCE uses 1 1,3 3, 5 5 and 7 7. Four convolutions are performed in parallel on the input feature map; 3. 3, 5 5 and 7 The outputs of the three branches are respectively related to 1. The outputs of the first branch are summed; a coordinated attention module is introduced in Outstage4 of the Backbone, which is located at the deepest layer of the Backbone; a squeeze-excitation convolutional (SEC) module is designed in the Neck, consisting of a squeeze-and-excitation block and... Convolutional structure.

2. A vehicle high beam detection device for complex environments, characterized in that, include: The first processing module is used to collect photos of vehicle high beams in complex environments, use the Labelimg annotation tool to annotate the photos, and divide the processed data into training set, validation set and test set according to the proportion; The second processing module is used to design the improved NanoDetPlus network model; The third processing module is used to input the data from the training set and validation set into the improved NanoDetPlus model for training and validation, so as to obtain the optimized NanoDetPlus algorithm network model. The fourth processing module is used to input the data in the test set into the optimized NanoDetPlus model for detection, simulate the detection effect in a real environment, and obtain the detection results of vehicle high beams in complex environments. The NanoDetPlus model is improved by using multi-scale convolutional kernels, coordinated attention, and squeezed excitation modules; In designing the improved NanoDetPlus network model, a multi-scale convolutional feature enhancement module (MCE) is designed within the backbone; the MCE employs a 1 1,3 3, 5 5 and 7 7. Four convolutions are performed in parallel on the input feature map; 3. 3, 5 5 and 7 The outputs of the three branches are respectively related to 1. The outputs of the first branch are summed; a coordinated attention module is introduced in Outstage4 of the Backbone, which is located at the deepest layer of the Backbone; a squeeze-excitation convolutional (SEC) module is designed in the Neck, consisting of a squeeze-and-excitation block and... Convolutional structure.

3. A vehicle high beam detection system for complex environments, characterized in that, include: The system includes a memory and a processor, wherein the memory stores a computer program that is executed by the processor, and the computer program, when executed by the processor, performs the vehicle high beam detection method under complex conditions as described in claim 1.

4. A storage medium, characterized in that, The storage medium stores a computer program, which executes the vehicle high beam detection method under complex conditions as described in claim 1 when it runs.

Images