A low-illumination dynamic target detection method based on vehicle-mounted image recognition

Abstract

The application relates to the field of computer vision and intelligent driving technology, and discloses a low-illumination dynamic target detection method based on vehicle-mounted image recognition. The method continuously collects night driving images by a vehicle-mounted camera, completes reflection component and illumination component decomposition and low-illumination enhancement through Deep Retinex-Net, and then outputs the current detection frame, confidence and ReID appearance feature by a YOLOv8 detection network adopting a BiFPN weighted feature fusion module. The detection frame, ReID appearance feature and confidence of the last five frames are maintained through a fixed-length cache queue, the correlation matching is completed according to the GIoU of the current detection frame and the historical detection frame, the historical ReID appearance feature is propagated according to the motion offset, the propagation weight is determined by combining the GIoU and the historical confidence, and the current feature and the detection result are corrected, so that the image recognition and detection accuracy of small targets, occluded targets and motion blurred targets in the night driving scene is improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent driving technology, and in particular to a low-light dynamic target detection method based on vehicle image recognition. Background Technology

[0002] In target detection tasks for intelligent connected vehicles, the YOLO series of single-stage detection algorithms are widely used due to their advantages such as strong real-time performance and high accuracy. The YOLOv8 model adopts a Backbone-Neck-Head architecture: the backbone network extracts multi-scale image features based on an improved CSPNet structure; the neck network uses a PAN-FPN module for bidirectional feature fusion from top to bottom and bottom to top; and the detection head uses a Decoupled-Head structure to separate classification and regression. In nighttime driving scenarios, due to low light intensity and noise interference, the images captured by the camera are dark and details are difficult to distinguish. Existing solutions typically use traditional Retinex or MSR algorithms to enhance the images before feeding them into the detection model. For small target detection, existing solutions improve feature representation through attention mechanisms or reconstructing the backbone network. For occluded targets, existing solutions mainly rely on single-frame images for detection, and some studies have begun to introduce multi-frame feature fusion to enhance temporal information.

[0003] The aforementioned and existing related technologies have the following drawbacks: Existing target detection models mainly rely on single-frame images for detection. When the target is occluded or blurred by motion, the visible area of ​​the target in a single-frame image is limited, making it difficult for the model to extract effective feature information and easily leading to false positives and false negatives. Some multi-frame fusion schemes are not optimized for the characteristics of nighttime driving scenes, and their performance improvement is limited in static scenes or scenes with small inter-frame differences, and they are prone to introducing redundant calculations. Summary of the Invention

[0004] The technical problem to be solved by this invention is that the existing technology has the disadvantage that the detection model relies on a single frame image, and it is difficult to obtain complete target features when there is occlusion or motion blur, which easily leads to missed detection. To this end, we propose a low-light dynamic target detection method based on vehicle image recognition.

[0005] To achieve the above objectives, this application adopts the following technical solution: a low-light dynamic target detection method based on vehicle image recognition, comprising the following steps: S1. The vehicle-mounted camera continuously captures low-light images of nighttime driving scenes and forms the current image frame by frame; S2. Input the current image into the YOLOv8 detection network. The backbone network extracts multi-scale features, and the detection head outputs the current detection box and confidence score. At the same time, the embedding branch of the detection head extracts the current ReID appearance features. S3. Input the current detection box, current ReID appearance features and confidence level into the multi-frame image fusion module. The multi-frame image fusion module establishes a fixed-length buffer queue and maintains the historical detection boxes, historical ReID appearance features and historical confidence levels of the last five frames in chronological order. S4. Calculate the GIoU between each current detection box and each historical detection box, form candidate target pairs from the detection boxes whose GIoU reaches the matching threshold, and select the historical detection box with the largest GIoU as the matching object. S5. Determine the motion offset based on the matching object and the current detection box, propagate the historical ReID appearance features to the current frame position, and determine the propagation weight based on GIoU and historical confidence, and perform weighted fusion with the current ReID appearance features; S6. Correct the detection results of the current frame with fused features; retain the current features for the current target whose GIoU has not reached the matching threshold, and retain the historical features for the historical target that has not matched the current detection box, so that subsequent frames can be rematched. S7. Output the corrected target detection result for the current frame, write the current detection box, current ReID appearance features and confidence level into the cache queue, and remove the earliest frame data.

[0006] Preferably, S1 further includes an image enhancement process performed after the current image is formed, the image enhancement process including the following steps: S11. Input the current image into the DecomNet decomposition network of Deep Retinex-Net, and output three-channel reflection components and single-channel illumination components through five layers of convolution and ReLU activation stacking. S12. Input the illumination component into the RelightNet relighting network, capture multi-scale illumination information through three levels of downsampling, and then reconstruct features through three levels of upsampling and skip connections. After stitching the feature maps of the three scales together, compress the channels through a one-to-one convolution to output the enhanced illumination component. S13. Perform element-wise multiplication of the reflection component and the enhanced illumination component to obtain the enhanced image input to the YOLOv8 detection network.

[0007] Preferably, the DecomNet decomposition network is trained using paired data of low-light images and normal-light reference images. Each loss component uses the L1 norm as a distance metric. The reconstruction loss reconstructs the input image by multiplying the reflection component and the illumination component. The mutual reconstruction loss keeps the reflection component unchanged across illumination. The reflectivity consistency loss aligns the reflection components of the two images. The exponential decay of the reflectivity gradient is used as the illumination smoothing weight to make the illumination component piecewise smooth. The weight coefficient of the mutual reconstruction loss is 0.001, the weight coefficient of the illumination smoothing loss is 0.1, and the weight coefficient of the reflectivity consistency loss is 0.01. During training, only the DecomNet parameters are updated and the RelightNet parameters are frozen.

[0008] Preferably, the RelightNet network outputs a single-channel illumination increment map required to transition from low-light to normal-light conditions. This single-channel illumination increment map is copied three times along the channel. The total loss for the Relight stage is formed by multiplying the reflection component of the low-light image with the copied illumination increment map, relative to the L1 norm of the normal-light reference image, and the Relight smoothing loss. The weight coefficient of the Relight smoothing loss is 3. During training, only the RelightNet parameters are updated, and the DecomNet parameters are frozen.

[0009] Preferably, in S2, the enhanced image obtained in S13 is input into the YOLOv8n detection network for target detection. The neck network of the YOLOv8n detection network is replaced by the four Concat layers in the PAN-FPN module with the BiFPN weighted feature fusion module. The BiFPN weighted feature fusion module receives 80x80 P3 features from the fourth layer of the backbone network, 40x40 P4 features from the sixth layer of the backbone network, and 20x20 P5 features from the ninth layer of the backbone network. It outputs 64 channels of P3 features, 128 channels of P4 features, and 256 channels of P5 features. The P3, P4, and P5 features are then input into the original Decoupled-Head detection head without adding a P2 detection layer.

[0010] Preferably, the BiFPN weighted feature fusion module is constructed according to the following steps: Remove intermediate nodes in the PAN-FPN module that have only one input edge and contribute little to feature fusion; Add a direct connection path between input and output nodes at the same level, so that the original features bypass intermediate transformations and are transmitted to the fused output. The channel concatenation is replaced by learnable weighted fusion, which sums the input feature maps according to normalized weights, and then restores them to the original number of channels through ReLU activation and one-to-one convolution.

[0011] Preferably, the learnable weights are scalar weights updated through backpropagation, with an initial value of 1. The fusion weights are normalized to be between zero and one. The P5 feature layer receives the original P5 features and the P4 upsampled features and uses a two-dimensional weight vector. The P4 feature layer receives the original P4 features, the P5 downsampled features, and the P3 upsampled features and uses a three-dimensional weight vector. The input feature maps are weighted and fused according to the normalized weights.

[0012] Preferably, Deep Retinex-Net, the YOLOv8 detection network with BiFPN weighted feature fusion module, and the multi-frame image fusion module are trained and inference is performed according to the following steps: Deep Retinex-Net is pre-trained independently on paired data of low-light images and normal-light reference images, and the weights of Deep Retinex-Net are fixed after pre-training is completed. The BiFPN weighted feature fusion module and the multi-frame image fusion module are jointly trained with the YOLOv8 detection network, while Deep Retinex-Net is not involved in the joint training. During inference, the low-light image is input into a Deep Retinex-Net with fixed weights to obtain an enhanced image. The enhanced image is then input into a YOLOv8 detection network with a BiFPN weighted feature fusion module to obtain preliminary detection results. The preliminary detection results are then input into a multi-frame image fusion module to perform association matching and feature fusion with targets in historical frames, and finally output the final detection results.

[0013] Preferably, in S3, a fixed-length buffer queue maintains the detection boxes, ReID appearance feature vectors, and confidence scores of the most recent five frames. Each frame stores the detection box coordinates, the ReID appearance feature vectors synchronously extracted by the embedding branch of the YOLOv8 detection head, and the confidence score. No additional network layer is set. The frame rate of the vehicle camera is 10 Hz, and the most recent five frames form a 0.5-second time window.

[0014] Preferably, S4 includes the following steps: S41. Calculate GIoU based on the intersection region, union region, and minimum rectangular region containing the two detection boxes of the current frame and the historical frame detection boxes. S42. Traverse the historical frame detection boxes, form candidate target pairs with the current frame detection boxes whose GIoU reaches the matching threshold and the historical frame detection boxes, and use the nearest neighbor matching method to select the historical frame detection box with the largest GIoU as the matching object. S5 includes the following steps: S51. Calculate the motion offset between the historical frame detection box and the current frame detection box based on the association results, and propagate the historical ReID appearance features corresponding to the matched object to the current frame position. S52. Combine GIoU similarity and historical confidence to determine the propagation weight, and then perform weighted fusion of the historical ReID appearance features after propagation with the current ReID appearance features according to the propagation weight; S6 includes the following steps: S61. Retain the current ReID appearance features for the current frame detection box where the GIoU has not reached the matching threshold; S62. For targets in historical frames that do not match the detection box in the current frame, retain their historical ReID appearance features for rematching in subsequent frames.

[0015] The technical effects and advantages of this invention are as follows: In this invention, Deep Retinex-Net is used to decompose and re-illuminate the reflection and illumination components of low-light images, reducing the reliance on manual parameters in traditional image enhancement methods and preserving texture, edge, and dark target information in nighttime driving images. A BiFPN weighted feature fusion module replaces the four Concat layers in the YOLOv8n neck network, adjusting the fusion weights based on the contribution of features at different scales to the detection task, thus strengthening the bidirectional transmission of low-level detail information and high-level semantic information. By using the detection boxes, ReID appearance features, and confidence scores of the most recent five frames for GIoU association, motion position propagation, and weighted fusion, historical effective features can be used to supplement the current frame features when the target is temporarily occluded or undergoes motion blur, reducing false detections and missed detections of small and occluded targets. This method uses only images from the vehicle-mounted camera as necessary input, without adding extra sensor burden, and maintains accurate real-time target image recognition and detection capabilities. Attached Figure Description

[0016] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts: Figure 1 This is a flowchart illustrating the overall detection process of a low-light dynamic target detection method based on vehicle image recognition according to the present invention. Figure 2 This is a low-light image enhancement network diagram for a low-light dynamic target detection method based on vehicle image recognition according to the present invention. Figure 3 This is a BiFPN feature fusion structure diagram of a low-light dynamic target detection method based on vehicle image recognition according to the present invention; Figure 4 This is a five-frame buffer and target association map of a low-light dynamic target detection method based on vehicle image recognition according to the present invention; Figure 5This is a flowchart of the GIoU matching process for a low-light dynamic target detection method based on vehicle image recognition according to the present invention. Figure 6 This is a historical feature propagation and fusion diagram of a low-light dynamic target detection method based on vehicle image recognition according to the present invention; Figure 7 This is a diagram illustrating the model training and inference relationship of a low-light dynamic target detection method based on vehicle image recognition according to the present invention. Figure 8 This is a flowchart illustrating the phased training process of the decomposition network and the relighting network in a low-light dynamic target detection method based on vehicle image recognition according to the present invention. Detailed Implementation

[0017] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0018] Reference Figure 1 As shown, the present invention provides a technical solution: a low-light dynamic target detection method based on vehicle image recognition, comprising the following steps: Step 1: The vehicle-mounted camera continuously acquires low-light images of the nighttime driving scene, forming the current image frame by frame. The vehicle-mounted camera only handles continuous image acquisition; no additional LiDAR or other sensors are added as necessary inputs to this method. The vehicle-mounted camera outputs image frames continuously in chronological order, writing a frame number and acquisition time to each frame; the input frames are resized and pixel normalized to form the current image.

[0019] Nighttime driving scenarios were selected based on environmental conditions labeled "night" in the Waymo dataset, corresponding to the period from 18:00 local time to 06:00 the following day. Scenario included nighttime highways, urban roads, and unlit side roads. Road types included two-lane highways and multi-lane urban roads. Images were acquired using an 8-bit RGB color space with pixel values ​​ranging from [0, 255]. The average global brightness across the three channels was no higher than 80, and the grayscale histogram was concentrated in the low grayscale range with a narrow, single-peak distribution. Before inputting the model, the image size was uniformly adjusted to 640×640 pixels.

[0020] The detection categories include Vehicle, Pedestrian, and Cyclist, denoted as V, P, and C respectively. The detection bounding boxes are in the format (x1, y1, x2, y2), where x1 and y1 are the coordinates of the top-left corner of the detection box, and x2 and y2 are the coordinates of the bottom-right corner. Small targets are defined as those with a detection box area less than 32×32 pixels. Occlusion targets are those whose visible feature area within the detection box is reduced due to occlusion by vehicles or environmental objects. For targets that do not have a valid historical match in the current frame, current features are retained; for targets that do not have a current match in historical frames, their historical features are retained so that the target can continue to be matched when it reappears in subsequent frames.

[0021] During the simulation training phase, images from vehicle-mounted cameras in the Waymo dataset were used, with a camera resolution of 1920×1280 and a frame rate of 10Hz. In the real-vehicle phase, a Realtek vehicle-mounted front-view camera with a resolution of 1920×1080 was used, and its horizontal field of view covered the area of ​​the lane in front of the vehicle. The most recent five frames formed a 0.5-second time window.

[0022] When the cache queue is not yet full (less than five frames), historical matching is completed according to the actual number of frames written. If there is no historical target in the first frame, the current detection result is directly retained and written to the cache. After the cache is full, for each frame of current data written, the oldest frame of data is removed at the same time, so that the cache always contains the detection boxes, ReID appearance features, and confidence scores of the most recent five frames.

[0023] After the current image is formed, image enhancement is performed according to the following steps S11 to S13: S11. Input the current image into the DecomNet decomposition network of Deep Retinex-Net, and output three-channel reflection components and single-channel illumination components through five layers of convolution and ReLU activation stacking.

[0024] S12. Input the illumination component into the RelightNet relighting network. After three levels of downsampling, capture multi-scale illumination information. Then, after three levels of upsampling and skip connections, reconstruct features. After stitching the feature maps of the three scales together, compress the channels through a one-to-one convolution to output the enhanced illumination component.

[0025] S13. Perform element-wise multiplication of the reflection component and the enhanced illumination component to obtain the enhanced image input to the YOLOv8 detection network.

[0026] like Figure 2 and Figure 8As shown, Deep Retinex-Net comprises a DecomNet decomposition network and a RelightNet illumination re-illumination network. DecomNet decomposes the current low-light image into a three-channel reflection component and a single-channel illumination component through five layers of convolutions stacked with ReLU activations. RelightNet takes the illumination component as input, acquires illumination information under different receptive fields through three levels of downsampling, then recovers spatial details through three levels of upsampling and skip connections at corresponding levels. The feature maps at the three scales are concatenated channel by channel and compressed through a 1×1 convolution to output a single-channel illumination increment map. The illumination increment map is then copied three times along the channel and element-wise multiplied with the reflection component to obtain the enhanced image.

[0027] DecomNet uses low-light images S low and normal illumination reference image S high The paired data was used for training, and the L1 norm was used as the distance metric for each loss component. R low R high These are the three-channel reflection components corresponding to the low-light image and the normal-light reference image, respectively. low3 I high3 These are the three-channel tensors obtained by copying the single-channel illumination component three times along the channel. The reconstruction loss L is calculated for the low-light image and the normal-light reference image. recon Calculate according to the following formulas: ; In the formula: L recon(low) L recon(high) are the reconstruction losses for the low-light image and the normal-light reference image, respectively, and ||·||1 represents the L1 norm.

[0028] Mutual reconstruction loss L mutal Cross-reconstruction is performed by exchanging reflection components to keep the reflection components unchanged across illumination conditions. The specific calculation is as follows: ; In the formula, L mutal(low) To use R high with I low3 Cross-reconstruction S low The obtained mutual reconstruction loss, L mutal(high) To use R low with I high3 Cross-reconstruction S high The obtained mutual reconstruction loss; R low R high I low3 I high3 S low and S high The meaning is consistent with the aforementioned definition.

[0029] Reflectivity uniformity loss L equal(R) The reflection components of two paired images are directly aligned and calculated using the following formula: ; In the formula, R represents the reflection component variable, R low and R high The meaning is consistent with the previous definition, and ||·||1 represents the L1 norm.

[0030] The illumination smoothing loss uses the exponential decay of the reflectivity gradient as the smoothing weight of the illumination component, and is calculated using the following formula: ; In the formula, L smooth(I,R) For illumination smoothing loss, I represents the single-channel illumination component participating in the smoothing constraint, and R represents the three-channel reflection component corresponding to I. gray R is a grayscale image of the reflection component. r R g and R b These represent the red, green, and blue channels of the reflection component, respectively; 0.2999, 0.587, and 0.114 are the grayscale conversion coefficients for the red, green, and blue channels, respectively; G x (I), G y (I) represent the horizontal and vertical first-order gradients calculated using the convolution kernel [[0,0],[-1,1]], respectively. AG x and AG y , , represent the local average gradients after 3×3 mean pooling, 10 is the exponential decay adjustment coefficient, exp is the exponential function, and mean indicates taking the mean. The weights at points of drastic reflectivity change approach 0, allowing the illumination component to change with the target edge; the weights at points of smooth reflectivity approach 1, keeping the illumination component piecewise smooth.

[0031] The total loss in the Decom phase is calculated using the following formula (weighting coefficients are determined through grid search): ; Among them, L Decom This represents the total loss during the Decom stage; the subscripts low and high represent the corresponding loss terms calculated from the decomposition results of the low-light image and the normal-light reference image, respectively; L recon L mutal L smooth and L equal(R)These are the reconstruction loss, mutual reconstruction loss, illumination smoothing loss, and reflectivity consistency loss, respectively. 0.001 is the mutual reconstruction loss weight coefficient, a small value is chosen to make the mutual reconstruction constraint a weak regularization term to assist training and avoid interfering with the main reconstruction task; 0.1 is the smoothing loss weight coefficient, balancing reconstruction accuracy and illumination smoothness; 0.01 is the reflectivity consistency loss weight coefficient, during the Decom stage only the DecomNet parameters are updated, while the RelightNet parameters are frozen.

[0032] Figure I shows the single-channel illumination increment required for RelightNet output to transition from low-light to normal-light conditions. delta , will I delta The three-channel tensor I is obtained by copying it three times along the channel. delta3 Total loss during the Relight phase L Relight Calculate using the following formula: ; The weighting coefficient for the Relight smoothing loss is 3,I. delta This is a single-channel illumination increment map output by RelightNet, representing the amount of illumination adjustment required to transition from low to normal illumination. I delta3 To make I delta The three-channel tensor, R, is obtained by copying it three times along the channel. low、 S high and L smooth The meaning is consistent with the previous definition: in the Relight phase, only the RelightNet parameters are updated, while the DecomNet parameters are frozen. After completing the two independent pre-training phases, all network weights of Deep Retinex-Net are fixed and used as the pre-image processing part of the YOLOv8 detection network during inference.

[0033] Step 2: Input the current image into the YOLOv8 detection network. The backbone network extracts multi-scale features, and the detection head outputs the current detection box and confidence score. Simultaneously, the embedding branch of the detection head extracts the current ReID appearance features. When executing steps S11 to S13, the obtained enhanced image is used as the current image and input into the YOLOv8n detection network.

[0034] like Figure 3As shown, the four Concat layers in the original PAN-FPN module of the YOLOv8n neck network are replaced entirely with a BiFPN weighted feature fusion module. The BiFPN receives the 80×80 P3 features output from the fourth layer of the backbone network, the 40×40 P4 features output from the sixth layer of the backbone network, and the 20×20 P5 features output from the ninth layer of the backbone network; correspondingly, it outputs 64 channels of P3 features, 128 channels of P4 features, and 256 channels of P5 features, and directly connects to the original Decoupled-Head detection head without adding a P2 detection layer.

[0035] BiFPN removes intermediate nodes with only one input edge and minimal contribution to fusion, establishes direct connections between input and output nodes at the same level, allowing original features to bypass intermediate transformations and reach the fusion output; and replaces fixed-channel concatenation with learnable weights ω. i Weighted fusion. For two inputs, normalize and fuse the weights. i The fusion features are calculated using the following formulas: ; Where ω1 and ω2 are learnable weight parameters (scalars, initially 1), which are automatically updated during training through backpropagation. The value is 0.0001 to prevent the denominator from being 0. `weight1` and `weight2` are the normalized fusion weights. `X1` is the feature map passed in from the top-down path, and `X2` is the feature map passed in from the bottom-up path. fused The weighted fusion result is restored to the original number of channels via ReLU activation and 1x1 convolution. Layer P5 receives two inputs: top-down (original P5 features) and bottom-up (upsampled P4 features), corresponding to a two-dimensional weight vector ω=[ω1, ω2]. Layer P4 receives three inputs (original P4 features + downsampled P5 features + upsampled P3 features), corresponding to a three-dimensional weight vector ω=[ω1, ω2, ω3], and the fusion method is the same as above. During training, the weights ω are automatically updated, enabling the model to adaptively increase the fusion weights of low-level detail features in low-light scenes to enhance the response to small targets, and to balance the contributions of each layer in well-lit conditions.

[0036] The P5 feature layer receives the original P5 features and the upsampled P4 features, and uses a two-dimensional weight vector. The P4 feature layer receives the original P4 features, the downsampled P5 features, and the upsampled P3 features, and uses a three-dimensional weight vector. Features entering the same fusion node are spatially aligned according to the size of the receiving layer feature map before weighting. Weighted summation is performed pixel-wise and channel-wise to avoid inconsistencies in feature sizes across different layers that could prevent fusion.

[0037] The decoupled-head detection head performs classification and bounding box regression on the output features of P3, P4, and P5, respectively, outputting the detection category, bounding box coordinates, and confidence score. The embedding branch reads the target location features corresponding to the bounding boxes from the detection head and outputs ReID appearance feature vectors synchronously according to the bounding box number. No independent backbone network or additional detection network layer is set; each ReID appearance feature vector is bound to the corresponding bounding box, category, and confidence score using the same target number.

[0038] Step 3: Input the current detection box, current ReID appearance features and confidence score into the multi-frame image fusion module. The multi-frame image fusion module establishes a fixed-length cache queue and maintains the historical detection boxes, historical ReID appearance features and historical confidence scores of the most recent five frames in chronological order.

[0039] like Figure 4 As shown, the multi-frame image fusion module establishes a first-in, first-out (FIFO) fixed-length buffer queue. Each frame record includes the frame number, acquisition time, detection box coordinates, target category, ReID appearance feature vector, and confidence score. The queue is arranged in ascending order of acquisition time. After the current frame is detected, only historical frame records are read without overwriting historical data; after the correction result is output, the current frame record is written to the end of the queue. When the queue length exceeds five frames, the record at the head of the queue is removed.

[0040] Historical matching reads the five most recent frames in chronological order from most recent to oldest. For a target within the same historical frame, each historical detection box can only form a final match with one current detection box. When multiple current detection boxes point to the same historical detection box, the target pair with the highest GIoU is retained, and the remaining current detection boxes continue to select matching objects from the remaining historical detection boxes. Detection boxes of different categories do not form candidate target pairs.

[0041] Step 4: Calculate the GIoU between each current detection box and each historical detection box. Detection boxes with GIoU reaching the matching threshold are grouped into candidate target pairs. The historical detection box with the highest GIoU is then selected as the matching target. Step 4 specifically includes: S41. Calculate GIoU based on the intersection region, union region, and minimum rectangular region containing the two detection boxes of the current frame and the historical frame detection boxes.

[0042] S42. Traverse the historical frame detection boxes, form candidate target pairs with the current frame detection boxes whose GIoU reaches the matching threshold and the historical frame detection boxes, and use the nearest neighbor matching method to select the historical frame detection box with the largest GIoU as the matching object.

[0043] like Figure 5 As shown, let A be the detection box region of the current frame and B be the detection box region of the historical frame. Traditional IoU is calculated using the following formula: ; In the formula, A represents the region of the predicted bounding box, and B represents the region of the ground truth bounding box; A∩B represents the intersection region of the two detection boxes, and A∪B represents the union region of the two detection boxes. When the two detection boxes do not intersect, A∩B is 0, and the IoU is 0, making it impossible to distinguish between different non-intersection distances. Furthermore, this method is not sensitive enough to the positional deviation between the predicted and detection boxes, which may result in equal IoU but different overlap. Therefore, GIoU is introduced to improve the sensitivity of the predicted box offset and better reflect the degree of overlap between the predicted and detection boxes. GIoU is calculated using the following formula: ; Where C represents the smallest rectangular region containing A and B.

[0044] For the current frame detection box set α and the historical frame detection box set β, the GIoU matching threshold is set to 0.5, and the candidate target pair set M is determined by the following formula: ; In the formula, α = {a1, a2, ..., a m}, β={b1,b2,…,b n}; m is the number of detection boxes in the current frame, and n is the number of detection boxes in the traversed historical frames. For each current detection box, calculate its GIoU with historical detection boxes of the same category. Target pairs with a GIoU greater than or equal to 0.5 are added to the candidate set M and sorted in descending order of GIoU. The nearest neighbor matching method is used to select the historical detection box with the largest GIoU as the matching object. Detection boxes with a GIoU exactly equal to 0.5 are added to the candidate set; if the GIoU is less than 0.5, feature fusion is not performed, and the current ReID appearance features are retained.

[0045] Targets in historical frames that do not match the current detection box are marked as temporarily invisible targets. Their historical detection boxes, ReID appearance features, and confidence scores are retained in the corresponding frame record until the frame record is removed from the five-frame buffer queue. When a target of the same category appears in a subsequent frame and the matching threshold is reached, a new match is established, thereby covering the feature interruption caused by short-term target occlusion and motion blur.

[0046] Step 5: Determine the motion offset based on the matched object and the current detection box, propagate the historical ReID appearance features to the current frame position, and determine the propagation weights based on GIoU and historical confidence, then perform weighted fusion with the current ReID appearance features. Step 5 specifically includes: S51. Calculate the motion offset between the historical frame detection box and the current frame detection box based on the association results, and propagate the historical ReID appearance features corresponding to the matched object to the current frame position.

[0047] S52. Combine GIoU similarity and historical confidence to determine the propagation weight, and perform weighted fusion of the historical ReID appearance features after propagation with the current ReID appearance features according to the propagation weight.

[0048] like Figure 6 As shown, for the matched current and historical detection boxes, the center position, width, and height of each box are obtained. The horizontal and vertical motion offsets are determined by subtracting the center position of the historical detection box from the center position of the current detection box. The scale of the target region after propagation is determined by the width and height of the current detection box. The ReID appearance features corresponding to the historical detection boxes are propagated to the target position in the current frame. During propagation, position alignment is performed according to the current target region, ensuring that the propagated features have the same length and arrangement order as the current ReID appearance features.

[0049] The propagation weight λ is jointly determined by the GIoU similarity obtained from the association matching and the confidence of the matched historical targets. The GIoU similarity and historical confidence are both restricted to [0,1], and their product is taken as the historical propagation contribution. λ is obtained by subtracting this historical propagation contribution from 1. Therefore, the larger the GIoU and the higher the historical confidence, the larger 1-λ, and the greater the supplementary contribution of the historical targets to the current frame. The following formula is used when performing weighted correction on the matched detection boxes: ; In the formula, α represents the target detection box in the current frame, β represents the historical target detection box propagated to the current frame position, α′ represents the fused and corrected target detection box, and λ represents the weight parameter. The weight of the current target detection box α is λ, and the weight of the historical target detection box β is 1-λ. The ReID appearance features are weighted sequentially using the same weight relationship, with the weight of the current ReID appearance feature being λ and the weight of the historical propagation features being 1-λ. After fusion, the feature vector is normalized; the weight of the current confidence is λ, the weight of the historical confidence is 1-λ, the detection category uses the matching results of the same category, and the target number uses the number of the matched historical target. The current frame correction result is formed by the fused features, corrected detection boxes, corrected confidence, detection category, and target number.

[0050] Step 6: Correct the detection results of the current frame using fused features; retain the current features for current targets whose GIoU does not reach the matching threshold, and retain the historical features for historical targets that do not match the current detection box, for re-matching in subsequent frames. Step S6 specifically includes: S61. For the current frame detection box where GIoU has not reached the matching threshold, retain the current ReID appearance features.

[0051] S62. For targets in historical frames that do not match the detection box in the current frame, retain their historical ReID appearance features for rematching in subsequent frames.

[0052] When the current detection box does not reach the matching threshold, historical features are not used; instead, the current detection box, current ReID appearance features, and current confidence score are directly retained. When a historical target has no current matching object, a current detection box is not generated; only the historical detection box, historical ReID appearance features, and historical confidence score are retained for rematching in subsequent frames. This process avoids incorrectly fusing newly appearing targets with historical targets and prevents the immediate loss of historical appearance information when a target is temporarily occluded.

[0053] Step 7: Output the corrected target detection results for the current frame. Write the current detection box, current ReID appearance features, and confidence score to the cache queue, and remove the earliest frame data. After all detection boxes in the current frame have been matched, output the target category, corrected detection box, corrected confidence score, and target number. When writing to the cache, the corrected detection box and the fused ReID appearance features are used as the current record for that target, and the corresponding confidence score is saved. The earliest frame is removed after the queue reaches five frames. When the next frame arrives, the updated five-frame record is used to continue the association matching, thus forming a frame-by-frame closed-loop processing.

[0054] like Figure 7 and Figure 8 As shown, Deep Retinex-Net is independently pre-trained on paired data of low-light images and normal-light reference images. RelightNet is frozen during the Decom phase, and DecomNet is frozen during the Relight phase; after both phases of training, the weights of Deep Retinex-Net are fixed. The Deep Retinex-Net with fixed weights does not participate in the end-to-end joint training of the YOLOv8 detection network.

[0055] The BiFPN weighted feature fusion module and the multi-frame image fusion module are jointly trained with the YOLOv8 detection network. The joint training data is kept continuous according to scene and frame order. Detection data from the current frame and its five preceding frames are used to complete GIoU association and historical ReID appearance feature propagation, ensuring that the detection head output corresponds to the multi-frame correction results. During inference, DeepRetinex-Net enhancement, YOLOv8n initial detection, five-frame cache association, historical feature propagation, and current frame result correction are executed sequentially.

[0056] Precision (P), recall (R), average precision (AP), and average precision (mAP) for the three target classes are used to evaluate detection accuracy, while inference speed is evaluated using fractional power per second (FPS). The evaluation metrics are based on the following formulas from the technical materials: ; In the formula, TP represents the number of correctly identified positive samples, FP represents the number of negative samples incorrectly identified as positive samples, FN represents the number of positive samples incorrectly identified as negative samples, P represents precision, R represents recall, and d(R) represents the integration of recall, with the lower bound of 0 and the upper bound of 1 corresponding to recall rates ranging from 0 to 1; AP i (i=1, 2, 3) represent the average precision of a single category, namely cars, pedestrians and cyclists, respectively, and mAP represents the average AP of the three categories.

[0057] During model training, cars in the Waymo dataset were labeled "V" (Vehicle), pedestrians were labeled "P" (Pedestrian), and cyclists were labeled "C" (Cyclist). 10,116 images were selected from the dataset for model training, including 8,133 images in the training set, 991 images in the validation set, and 992 images in the test set. The camera specifications of the Waymo dataset are shown in Table 1.

[0058] Table 1 Waymo Camera Specifications

[0059] (1) Ablation test To further verify the effectiveness of the proposed BiFPN fusion module and multi-frame image fusion module, ablation experiments were conducted on the two improved modules. The experimental results are shown in Table 2, where √ indicates the addition of the module.

[0060] Table 2 Ablation Experiment

[0061] As shown in Table 2, both improved modules proposed in this paper enhance the performance of the object detection algorithm. In the YOLOv8n base model, replacing the PAN-FPN module in the neck network with a BiFPN fusion module reduces the loss of feature information in the intermediate layers, improving the model's mAP by 1.9%. Adding a multi-frame image fusion module to the YOLOv8n base model, utilizing the fusion of the current frame and historical frames to address feature information loss, improves the model's mAP by 2.3%. Compared to adding a single improved module, adding both the BiFPN fusion module and the multi-frame image fusion module simultaneously improves the improved model's mAP by 2.8% compared to the base model, reducing false positives and false negatives during detection. Although the final FPS of the improved model is slightly lower than the base model, it still meets the requirements for real-time object detection.

[0062] (2) Comparative experiment To better verify the performance of the proposed improved YOLOv8 object detection algorithm, this paper tests the proposed algorithm model with other object detection models in a nighttime driving scenario. The test results are shown in Table 3.

[0063] Table 3 Comparison of different target detection models

[0064] First, the model in this paper has 5.2M parameters, which is higher than the YOLOv8-DEL model, indicating that the improved model has stronger learning and feature representation capabilities. Compared with the high-parameter-number YOLOv5s, YOLOv8s, and FE-YOLOv8 models, the improved model is less prone to overfitting during training and inference, and has stronger generalization ability with limited training datasets. Second, the model's mAP is 77.3%, which is higher than YOLOv5s, YOLOv8-DEL, YOLOv8s, and FE-YOLOv8 models, indicating that the improved model has improved detection accuracy. The algorithm performs better in nighttime driving scenarios and can effectively reduce false positives and false negatives. Finally, the model's FPS is 98, which is lower than the YOLOv8-DEL model, but its performance still meets the needs of practical detection, achieving a usable balance between accuracy and real-time performance. Therefore, the improved model has better detection performance compared to other models, validating the effectiveness of the algorithm presented in this paper.

[0065] The vehicle-mounted camera continuously captures low-light images of nighttime driving scenes, which are then processed by Deep... Retinex-Net decomposes the reflection and illumination components and performs re-illumination processing to obtain an enhanced image. The enhanced image is input into a YOLOv8n detection network using a BiFPN weighted feature fusion module, which performs weighted fusion of P3, P4, and P5 multi-scale features. The detection head outputs the current detection box, category, confidence score, and ReID appearance features. The multi-frame image fusion module reads the historical detection boxes, historical ReID appearance features, and historical confidence scores from the cache queue of the last five frames, calculates the GIoU between the current detection box and historical detection boxes of the same category, and identifies the historical target with the largest GIoU of 0.5 as the matching object. Based on the motion offset of the detection box, the historical ReID appearance features are propagated to the current frame position, and the propagation weight is determined based on GIoU and historical confidence score to perform weighted correction on the current detection result. Current targets that do not reach the matching threshold retain their current features, and historical targets that do not match the current target retain their historical features. After outputting the correction result, the five-frame cache queue is updated so that the next frame can continue to use historical information to complete association matching, feature propagation, and detection result correction.

[0066] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A low-light dynamic target detection method based on vehicle-mounted image recognition, characterized in that, Includes the following steps: S1. The vehicle-mounted camera continuously captures low-light images of nighttime driving scenes and forms the current image frame by frame; S2. Input the current image into the YOLOv8 detection network. The backbone network extracts multi-scale features, and the detection head outputs the current detection box and confidence score. At the same time, the embedding branch of the detection head extracts the current ReID appearance features. S3. Input the current detection box, current ReID appearance features and confidence level into the multi-frame image fusion module. The multi-frame image fusion module establishes a fixed-length cache queue and maintains the historical detection boxes, historical ReID appearance features and historical confidence levels of the last five frames in chronological order. S4. Calculate the GIoU between each current detection box and each historical detection box, form candidate target pairs from the detection boxes whose GIoU reaches the matching threshold, and select the historical detection box with the largest GIoU as the matching object. S5. Determine the motion offset based on the matching object and the current detection box, propagate the historical ReID appearance features to the current frame position, and determine the propagation weight based on GIoU and historical confidence, and perform weighted fusion with the current ReID appearance features; S6. Correct the detection results of the current frame with fused features; retain the current features for the current target whose GIoU has not reached the matching threshold, and retain the historical features for the historical target that has not matched the current detection box, so that subsequent frames can be rematched. S7. Output the corrected target detection result for the current frame, write the current detection box, current ReID appearance features and confidence level into the cache queue, and remove the earliest frame data.

2. The low-light dynamic target detection method based on vehicle image recognition according to claim 1, characterized in that: S1 further includes an image enhancement process performed after the current image is formed, the image enhancement process including the following steps: S11. Input the current image into the DecomNet decomposition network of Deep Retinex-Net, and output three-channel reflection components and single-channel illumination components through five layers of convolution and ReLU activation stacking. S12. Input the illumination component into the RelightNet relighting network, capture multi-scale illumination information through three levels of downsampling, and then reconstruct features through three levels of upsampling and skip connections. After stitching the feature maps of the three scales together, compress the channels through a one-to-one convolution to output the enhanced illumination component. S13. Perform element-wise multiplication of the reflection component and the enhanced illumination component to obtain the enhanced image input to the YOLOv8 detection network.

3. The low-light dynamic target detection method based on vehicle image recognition according to claim 2, characterized in that: The DecomNet decomposition network is trained using paired data of low-light images and normal-light reference images. Each loss component uses the L1 norm as a distance metric. The reconstruction loss reconstructs the input image by multiplying the reflection component and the illumination component. The mutual reconstruction loss keeps the reflection component unchanged across illumination levels. The reflectivity consistency loss aligns the reflection components of the two images. The exponential decay of the reflectivity gradient is used as the illumination smoothing weight to make the illumination component piecewise smooth. The weight coefficient of the mutual reconstruction loss is 0.001, the weight coefficient of the illumination smoothing loss is 0.1, and the weight coefficient of the reflectivity consistency loss is 0.

01. During training, only the DecomNet parameters are updated and the RelightNet parameters are frozen.

4. The low-light dynamic target detection method based on vehicle image recognition according to claim 3, characterized in that: The RelightNet relighting network outputs a single-channel illumination increment map required to transition from low-light to normal-light conditions. This single-channel illumination increment map is copied three times along each channel. The total loss for the Relight stage is formed by multiplying the reflection component of the low-light image with the copied illumination increment map, relative to the L1 norm of the normal-light reference image, and the Relight smoothing loss. The weight coefficient of the Relight smoothing loss is 3. During training, only the RelightNet parameters are updated, while the DecomNet parameters are frozen.

5. A low-light dynamic target detection method based on vehicle image recognition according to claim 2, characterized in that: In step S2, the enhanced image obtained in step S13 is input into the YOLOv8n detection network for target detection. The neck network of the YOLOv8n detection network is replaced by the four Concat layers in the PAN-FPN module with a BiFPN weighted feature fusion module. The BiFPN weighted feature fusion module receives 80x80 P3 features from the fourth layer of the backbone network, 40x40 P4 features from the sixth layer of the backbone network, and 20x20 P5 features from the ninth layer of the backbone network. It outputs 64 channels of P3 features, 128 channels of P4 features, and 256 channels of P5 features. The P3, P4, and P5 features are then input into the original Decoupled-Head detection head without adding a P2 detection layer.

6. The low-light dynamic target detection method based on vehicle image recognition according to claim 5, characterized in that: The BiFPN weighted feature fusion module is constructed according to the following steps: Remove intermediate nodes in the PAN-FPN module that have only one input edge and contribute little to feature fusion; Add a direct connection path between input and output nodes at the same level, so that the original features bypass intermediate transformations and are transmitted to the fused output. The channel concatenation is replaced by learnable weighted fusion, which sums the input feature maps according to normalized weights, and then restores them to the original number of channels through ReLU activation and one-to-one convolution.

7. The low-light dynamic target detection method based on vehicle image recognition according to claim 6, characterized in that: The learnable weights are scalar weights updated through backpropagation, with an initial value of 1. The fusion weights are normalized to be between zero and one. The P5 feature layer receives the original P5 features and the P4 upsampled features and uses a two-dimensional weight vector. The P4 feature layer receives the original P4 features, the P5 downsampled features, and the P3 upsampled features and uses a three-dimensional weight vector. The input feature maps are weighted and fused according to the normalized weights.

8. The low-light dynamic target detection method based on vehicle image recognition according to claim 6, characterized in that: The Deep Retinex-Net, the YOLOv8 detection network with BiFPN weighted feature fusion module, and the multi-frame image fusion module are trained and inference is performed according to the following steps: Deep Retinex-Net is pre-trained independently on paired data of low-light images and normal-light reference images, and the weights of Deep Retinex-Net are fixed after pre-training is completed. The BiFPN weighted feature fusion module and the multi-frame image fusion module are jointly trained with the YOLOv8 detection network, while DeepRetinex-Net does not participate in the joint training. During inference, the low-light image is input into a Deep Retinex-Net with fixed weights to obtain an enhanced image. The enhanced image is then input into a YOLOv8 detection network with a BiFPN weighted feature fusion module to obtain preliminary detection results. The preliminary detection results are then input into a multi-frame image fusion module to perform association matching and feature fusion with targets in historical frames, and finally output the final detection results.

9. A low-light dynamic target detection method based on vehicle image recognition according to claim 1, characterized in that: In S3, a fixed-length buffer queue maintains the detection boxes, ReID appearance feature vectors, and confidence scores of the last five frames. Each frame stores the detection box coordinates, the ReID appearance feature vectors extracted synchronously from the embedding branch of the YOLOv8 detection head, and the confidence score. No additional network layers are set. The frame rate of the vehicle camera is 10 Hz, and the most recent five frames form a 0.5-second time window.

10. A low-light dynamic target detection method based on vehicle image recognition according to claim 1, characterized in that: S4 includes the following steps: S41. Calculate GIoU based on the intersection region, union region, and minimum rectangular region containing the two detection boxes of the current frame and the historical frame detection boxes. S42. Traverse the historical frame detection boxes, form candidate target pairs with the current frame detection boxes whose GIoU reaches the matching threshold and the historical frame detection boxes, and use the nearest neighbor matching method to select the historical frame detection box with the largest GIoU as the matching object. S5 includes the following steps: S51. Calculate the motion offset between the historical frame detection box and the current frame detection box based on the association results, and propagate the historical ReID appearance features corresponding to the matched object to the current frame position. S52. Combine GIoU similarity and historical confidence to determine the propagation weight, and then perform weighted fusion of the historical ReID appearance features after propagation with the current ReID appearance features according to the propagation weight; S6 includes the following steps: S61. Retain the current ReID appearance features for the current frame detection box where the GIoU has not reached the matching threshold; S62. For targets in historical frames that do not match the detection box in the current frame, retain their historical ReID appearance features for rematching in subsequent frames.

Images