Real-time traffic target detection method for low-illuminance scene
By establishing the detection data set of low-light illuminance scenes and an improved RT-DETR model, integrating the image enhancement module and upgrading the backbone network, the problem of low detection accuracy of the traffic object detection model under low light conditions is solved, and higher detection performance is achieved and speed balance.
Patent Information
- Application Number
- CN202411042009.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-31
AI Technical Summary
The existing traffic object detection model has low detection accuracy under low light conditions, mainly due to the uneven data distribution and the lack of special image enhancement modules, which leads to inaccurate detection in night and inclement weather conditions.
Establish a detection data set for low-light illumination scenes, and improve the traffic object detection model based on RT-DETR, integrate the image enhancement module PENet, upgrade the backbone network to VMamba-tiny, replace the FFN unit to Local-FFN, and introduce the asymptotic feature pyramid network module AFPN , improve the detection performance of the model under low light.
Improve the performance and speed of the detection model under low light conditions, reduce the need for Anchor settings and NMS processing, enhance global relationship modeling capabilities and local feature perception, improve multi-level feature fusion effect, improve detection accuracy and speed .
Smart Images

Figure CN119007149A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of deep learning, computer vision and target detection technology, and in particular to a real-time traffic target detection method for low-light illumination scenes. Background Art
[0002] With the continuous development of social economy and the gradual increase of residents' income, more and more residents buy cars as the main mode of transportation for daily travel, which leads to a rapid increase in the number of cars and an increasingly complex urban traffic environment. Thanks to the rapid development of deep learning and computer vision technology in recent years, autonomous driving technology has received more and more attention and is widely regarded as an important method to deal with complex urban traffic problems. A key basic technology for realizing autonomous driving technology is traffic target detection technology.
[0003] At present, the object detection method based on deep learning and computer vision technology has achieved relatively mature development, and widely used object detection models such as YOLO and SSD have been born. After a long period of training using public traffic object detection datasets, the obtained detection model can usually achieve good detection accuracy on the validation set. However, the above method model still has the following problems:
[0004] Problem 1: Most samples in existing public datasets come from normal lighting environments, while there are fewer low-light samples and the data distribution is uneven. This makes it difficult for the model to fully learn the features under low-light conditions during training, which in turn leads to a decrease in detection accuracy under low-light conditions. Therefore, this traffic target detection method is not reliable in some practical application scenarios, such as at night, in bad weather, and in tunnels.
[0005] Problem 2: Achieving high-precision target detection in low-light scenes still faces great challenges. Because images collected in low-light scenes have a series of problems such as blurred imaging, low clarity, and poor visibility, image details and small targets are difficult to be effectively detected. The traditional solution is to first perform traditional simple image enhancement on the image, such as increasing the brightness of the image, and then input it into the target detection model for training or reasoning. Although this method can improve the detection accuracy in low-light environments to a certain extent, it will damage the details of the bright areas in the image, and the model cannot be trained end-to-end, which is not conducive to the model learning more robust features.
[0006] In order to solve the above-mentioned problems faced by current traffic target detection technology, improve the detection accuracy of detection models in low-light scenarios, and promote the accelerated implementation and widespread application of target detection solutions, it is first necessary to collect a sufficient number of low-light traffic driving pictures to establish the target detection data set required for model training; secondly, it is necessary to use more advanced target detection models and select appropriate improvement modules to further optimize the detection model algorithm for low-light scenarios.
[0007] DETR is a new type of object detection algorithm that uses the Transformer architecture with global relationship modeling capabilities to handle object detection tasks. Unlike traditional anchor-based object detection methods, DETR directly predicts bounding boxes and category labels from images without the need for predefined anchor boxes or region proposal networks, and can achieve high efficiency and accuracy when handling object detection tasks.
[0008] In a nighttime vehicle detection method and device (CN113269119A), the inventor discloses a nighttime vehicle detection method, which includes: acquiring an image to be detected; detecting the image to be detected using a trained target detection model to obtain a detection target; wherein the target detection model processes the image to be detected including: extracting features from the image to be detected to obtain image features; enhancing the image features to obtain enhanced features; inputting the enhanced features into an RPN network to generate a candidate frame; processing the candidate frame through a ROIPooling layer to obtain a feature map of a fixed size; and regressing and classifying the feature map to obtain a detection target.
[0009] When processing low-light images, existing technical methods generally adopt traditional image enhancement methods independent of the model. Usually, there is a lack of image enhancement modules specifically for low-light scenes embedded in the model, which is not conducive to end-to-end feature learning of the model.
[0010] Commonly used object detection models, such as YOLO, need to go through complex pre- and post-processing steps. Before model training, the bounding boxes in the dataset need to be clustered to determine the bounding box distribution, and a set of preset anchor boxes are set accordingly as the basis for prediction. If the anchor box is set improperly, it may cause detection errors. In addition, the anchor box may not be flexible enough when dealing with objects of different sizes and shapes. After the model outputs the prediction results, non-maximum suppression (NMS) processing is usually required. NMS helps to obtain more accurate results by eliminating highly overlapping prediction boxes. However, NMS has a high computational cost and may reduce the detection speed. At the same time, the effect of NMS is also affected by the threshold setting. Improper threshold setting may cause missed detection or false detection problems.
[0011] The backbone network commonly used in current target detection models is generally a CNN or ViT structure. The CNN structure has the advantage of low computational complexity, but its disadvantage is that it does not have a global receptive field and cannot perform global relationship modeling on the detection image, resulting in low detection accuracy. Although the ViT structure has the ability to model global contextual relationships, its quadratic complexity leads to a huge amount of calculation, which cannot meet the requirements of lightweight deployment and real-time detection in actual application scenarios.
[0012] In addition, the commonly used structures in the neck of target detection models are generally FPN, PANet models, etc., and their multi-level feature fusion effects need to be further improved. Summary of the invention
[0013] In order to overcome the aforementioned technical problems, the present invention provides a real-time traffic target detection method for low-light illumination scenes, which achieves a better balance between detection performance and detection speed and has stronger comprehensive performance.
[0014] The present invention is achieved by at least one of the following technical solutions.
[0015] A real-time traffic target detection method for low-light illumination scenes includes the following steps:
[0016] S1. Establish a low-light scene detection dataset;
[0017] S2. Build a traffic target detection model for low-light scenes based on the improved RT-DETR;
[0018] S3, using the detection data set to train the traffic target detection model;
[0019] S4. Use the trained traffic target detection model to detect the target to be detected.
[0020] Furthermore, by acquiring a public traffic target detection data set, collecting image data of low-light scenes therein, and formatting the collected image data, a low-light scene detection data set is formed.
[0021] Furthermore, the improved traffic target detection model for low-light scenes based on RT-DETR integrates an image enhancement module on the basis of the RT-DETR model; the feature extraction backbone network is upgraded to VMamba-tiny, and the original FFN unit in VMamba-tiny is replaced by the Local-FFN unit. At the same time, the cross-scale feature fusion module (CCFM) in the encoder is updated to the asymptotic feature pyramid network module (AFPN), and the FFN units in the encoder and decoder are also replaced by Local-FFN units.
[0022] Furthermore, the image enhancement module uses a pyramid enhancement network (PENet), firstly processes the input image using Gaussian convolution and downsampling to obtain a Gaussian pyramid with four layers of components; processes the Gaussian pyramid using upsampling and Gaussian convolution to obtain a Laplacian pyramid with four layers of components;
[0023] After obtaining the Laplacian pyramid, PENet is used to enhance the image. Specifically, the detail processing module (DPM) is used to enhance the components of each layer of the Laplacian pyramid, and the low-frequency enhancement filter (LEF) is used to extract low-frequency information. The outputs of the detail processing module (DPM) and the low-frequency enhancement filter (LEF) are concatenated and fused, and finally the processed Laplacian components are restored to a high-resolution image.
[0024] Further, the detail processing module (DPM) includes a context branch and an edge branch;
[0025] The edge branch focuses on the edge information of the image. First, the Sobel operator is used to calculate the gradient approximation of the image brightness function in the vertical and horizontal directions to identify the edge information. Then, the convolution operation is used to extract the feature information of the image. Finally, the original image and the processed image are fused by residual connection to achieve information enhancement.
[0026] The context branch is responsible for capturing the long-range dependencies of the image. The context branch includes two residual blocks, which expand and restore the number of channels so that the low-frequency information can be smoothly fused with the high-frequency information after passing through the residual block, thus capturing the global information in the scene.
[0027] Furthermore, a low-frequency enhancement filter (LEF) is used to capture the low-frequency information of the image. The number of channels is first expanded through a point convolution layer, and the input components are evenly divided into four parts along the channel dimension. The features of each part are applied with the corresponding adaptive average pooling layer to obtain low-frequency information at different cutoff frequencies to form a low-pass filter. Then, bilinear interpolation is used to upsample it back to the original height and width, and the four features are concatenated along the channel dimension to form a new feature map. ReLU is used to activate it, and finally, a point convolution layer is used to restore the number of channels.
[0028] Furthermore, the backbone network VMamba-tiny includes a block embedding module and four VMamba levels. The block embedding module scales the size of the input image according to a fixed ratio, expands the channel dimension to a fixed dimension, and obtains embedded image features; the VMamba level includes a visual state space block (VSS BLOCK) and a downsampling layer. The VSS BLOCK flattens the two-dimensional image features into one-dimensional sequence data through a four-way scanning strategy, and then uses a selective state space model (S6) for long-distance relationship modeling, and finally uses a two-dimensional aggregation operation to re-aggregate the sequence data; the downsampling layer downsamples the image features to extract multi-scale features at different levels of the backbone network.
[0029] Furthermore, the Local-FFN unit has a 3×3 deep convolutional layer, which can perceive local features of image pixels and their adjacent small areas.
[0030] Furthermore, the encoder module includes an intra-scale feature interaction module (AIFI) and an asymptotic feature pyramid network module (AFPN); the AIFI structure is a Transformer encoder layer, and the Local-FFN unit is used to replace the original FFN unit in the Transformer encoder layer;
[0031] The AFPN module performs convolution operations on the image features S3, S4, and S5 of the last three levels obtained by the feature extraction module to align the number of channels, and performs progressive feature fusion on S3 and S4 obtained from the backbone network and the fifth-level feature F5 processed by the AIFI module. S3 and S4 are first weightedly fused, and then S3, S4, and F5 are weightedly fused, which helps to reduce the semantic gap between features at different levels.
[0032] Furthermore, the traffic target detection model training process is: setting the hyperparameters of the model training process, inputting the training data into the traffic target detection model, performing image enhancement, feature extraction and fusion, outputting the predicted value and calculating the loss, optimizing the model, and saving the model weights after the model is iteratively trained to convergence.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] The present invention uses RT-DETR as a basic detection model, and does not need to perform Anchor setting and NMS processing before and after detection, thereby accelerating the detection speed; the embedded image enhancement module PENet can extract more effective image feature information and improve the detection performance; the replaced backbone network VMamba-tiny has a global receptive field and linear complexity, which enables the model to obtain global relationship modeling capabilities without increasing the amount of calculation; the Local-FFN unit used can enhance the local area perception capability; the AFPN module introduced into the encoder module of the neck improves the multi-level feature fusion effect, and further enhances the model's detection capability for targets of different scales. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 This is a flow chart of a real-time traffic target detection method for low-light illumination scenes according to an embodiment of the present invention;
[0036] Figure 2 A structural diagram of the traffic target detection model for low-light illumination scenes based on RT-DETR improvement provided by the present invention;
[0037] Figure 3 is a structural diagram of a Laplace pyramid component of an embodiment;
[0038] Figure 4 This is a structural diagram of the image enhancement module PENet of the embodiment;
[0039] Figure 5 is a structural diagram of a detail processing module (DPM) of an embodiment;
[0040] Figure 6 is a structural diagram of a low frequency enhancement filter (LEF) of an embodiment;
[0041] Figure 7 A structural diagram of VMamba-tiny, a backbone network for feature extraction in the embodiment;
[0042] Figure 8 is a structural diagram of the VSS BLOCK of the embodiment;
[0043] Fig. 9 is a structural diagram of the SS2D module of the embodiment;
[0044] Fig.10 This is a specific flow chart of the scanning operation in the fourth embodiment;
[0045] Fig.11 is a structural diagram of a Local-FFN unit of an embodiment;
[0046] Fig.12 This is a structural diagram of the improved encoder module of the embodiment. DETAILED DESCRIPTION
[0047] In order to clarify the technical content and advantages of the present invention, the specific implementation methods of the present invention will be explained in detail below with reference to the accompanying drawings.
[0048] like Figure 1 As shown, the real-time traffic target detection method for low-light illumination scenes of this embodiment includes the following steps:
[0049] S1. Obtain a large-scale traffic target detection dataset, collect pictures and corresponding annotations of low-light scenes such as dusk or night in the above dataset, and organize the data: modify the collected pictures to a unified resolution, convert the corresponding bounding box coordinate annotations of the pictures after the modified resolution, and uniformly modify the annotation format to the coco data format to complete the establishment of the low-light scene traffic target detection dataset.
[0050] As an embodiment, a large-scale traffic target detection dataset includes BDD100k and KITTI.
[0051] S2. Design a traffic target detection model for low-light scenes based on the improved RT-DETR. Its structure is shown in the attached figure. Figure 2 As shown in the figure, the improvement process includes: in the RT-DETR framework, a PENet-based image enhancement module is embedded; the feature extraction backbone network is replaced with VMamba-tiny, and the Local-FFN unit is used to replace the original FFN unit inside VMamba-tiny; the CCFM module in the encoder is replaced with the AFPN module; and the original FFN units in the encoder and decoder modules are also upgraded to Local-FFN units.
[0052] PENet is used as the image enhancement module. Before entering PENet, components of different scales are first obtained through the Laplacian pyramid, as shown in formula (1). A convolution kernel of size 5×5 is used to perform Gaussian convolution on the input, and the downsampling method is to discard pixels with even indexes. After each Gaussian convolution and downsampling, the width and height of the output component are halved, the number of channels remains unchanged, and finally a four-layer Gaussian pyramid {G0, G1, G2, G3} is obtained. Next, the generated Gaussian pyramid is upsampled by expanding the image width and height to twice the original, filling the newly added rows and columns with 0, and multiplying the previous Gaussian convolution kernel pixel value by 4, and then performing Gaussian convolution. Since the Gaussian pyramid downsampling operation is irreversible, the lost information constitutes the components of the Laplacian pyramid, and finally a four-layer Laplacian pyramid {L0, L1, L2, L3} is obtained. The specific formula is shown in formula (2). The Laplacian pyramid is shown in the attached figure. Figure 3 shown.
[0053] G i+1= Down(Gaussian(G i )) (1)
[0054] L i =G i -Gaussian(Up(G i+1 )) (2)
[0055] In the formula, G i represents the i-th layer component of the Gaussian pyramid, Gaussian() represents the Gaussian convolution operation, Down() represents the downsampling operation, Up() represents the upsampling operation, and L i Represents the i-th level component of the Laplacian pyramid.
[0056] The PENet structure is as follows Figure 4 As shown in the figure, PENet first obtains the four-layer components of the Laplacian pyramid as input, then uses the detail processing module (DPM) to enhance the detail features of each layer component, uses the low-frequency enhancement filter (LEF) to extract low-frequency information, and concatenates and convolutionally fuses the outputs of the two modules to obtain the output. Finally, the output components of each layer are restored to a high-resolution image through upsampling.
[0057] The detail processing module (DPM) consists of a context branch and an edge branch. Figure 5 As shown in the figure. The edge branch focuses on the edge information of the image. First, the sobel operator is used to calculate the gradient approximation of the image brightness function in the vertical and horizontal directions to identify the edge information. Then, the convolution operation is used to extract the feature information of the image. Finally, the original image and the processed image are fused by residual connection to achieve information enhancement. The context branch is responsible for capturing the long-range dependencies of the image. First, the residual block is used to expand the channel of the input of this branch to obtain the intermediate features after channel expansion. The intermediate features are convolved and softmaxed to obtain the feature mask. After the feature mask is multiplied point by point with the intermediate features, it is calculated by another convolution layer and residually connected with the intermediate features. Finally, another residual block is used to restore the channel to obtain the output. The context branch allows the low-frequency information to be smoothly fused with the high-frequency information after passing through the residual block, which is conducive to capturing the global information in the scene.
[0058] The low frequency enhancement filter (LEF) can capture the low frequency information of the image. Figure 6As shown. First, a point convolution layer is used to expand the number of channels, and the input components are evenly divided into 4 parts along the channel dimension. The features of each part are applied with the corresponding adaptive average pooling layer, which are 1×1, 2×2, 3×3, and 6×6, respectively, to obtain low-frequency information at different cutoff frequencies and form a low-pass filter. Then bilinear interpolation is used to upsample it back to the original height and width, and the four features are spliced along the channel dimension to form a new feature map, which is activated using ReLU. Finally, a point convolution layer is used to restore the number of channels.
[0059] The features processed by DPM and LEF are concatenated in the channel dimension, and then feature fusion is performed using a point convolution layer. After processing the Laplacian components of different resolutions, the inverse operation of formula (2) is performed to restore the high-resolution image.
[0060] The backbone network used by the feature extraction module is VMamba-tiny, and its structure is shown in the attached figure. Figure 7 As shown in the figure. The network structure of VMamba-tiny can be divided into a block embedding module and four VMamba levels. The training image is first input into the block embedding module. After two convolution operations, the size is reduced to 1 / 2 and 1 / 4 of the original size. At the same time, the number of image channels is expanded to a fixed dimension to obtain embedded image features. The VMamba level contains a downsampling module and several VSS BLOCKs. Among them, the function of the downsampling module is to downsample the input tensor with a scaling ratio of 1 / 2, and at the same time expand the number of channels to twice the original. The network structure of VSSBLOCK is shown in the figure. Figure 8 First, the input feature tensor is layer-normalized, and then the normalized tensor is input into the two-dimensional selective scanning module (SS2D). Fig. 9 The structure of the SS2D module is shown. The SS2D module uses the Selective Scan Space State Sequential Model (S6) for long-distance relationship modeling. First, a four-way scan (2D Scan) operation is performed on the input tensor to convert the two-dimensional graphic data into one-dimensional sequence data. The specific process of the four-way scan operation is: the two-dimensional graphic feature tensor is scanned alternately from the upper left corner pixel to the right by row, from the lower right corner pixel to the left by row, from the upper left corner pixel to the bottom by column, and from the lower right corner pixel to the top by column. The pixels of the feature map are flattened into a one-dimensional sequence in the order of the scan path to obtain four four-way scan one-dimensional sequence feature data. The specific process of the four-way scan operation is shown in the attached figure. Fig.10The four-directional scanning strategy ensures that each feature element can integrate information from all directions and construct a global receptive field, while avoiding the quadratic growth of computational complexity and ensuring the efficiency and accuracy of the model when processing high-resolution data.
[0061] Then the one-dimensional sequence data is input into the S6 model for relational modeling. Finally, the output of the S6 model is processed by two-dimensional aggregation (2D Merge) to restore it to two-dimensional graphic data.
[0062] The output of the SS2D module is residually connected to the input of the VSS BLOCK to obtain the intermediate features. After the intermediate features are normalized, they are input to the Local-FFN unit. The structure of the Local-FFN unit is shown in the attached figure. Fig.11 As shown in the figure, the Local-FFN module first transforms the input two-dimensional feature matrix with the size of (H×W)×C into a three-dimensional feature tensor with the size of C×H×W through the reshape operation, and then performs 1×1 convolution, batch normalization, and h-swish activation on the feature tensor. The h-swish activation function is:
[0063]
[0064] In the formula, x represents the feature vector activated using h-swish, and ReLU() represents the ReLU activation function.
[0065] The feature tensor is then subjected to 3×3 depth convolution, batch normalization, h-swish activation, and input to the SE attention module. The SE attention module can adaptively learn the relationship between feature channels. The output of the SE attention module is then subjected to 1×1 convolution and batch normalization. Finally, the three-dimensional feature tensor of size C×H×W is reshaped into a two-dimensional feature matrix of size (H×W)×C, which is residually connected to the input of the Local-FFN module and output.
[0066] Compared with the original FFN unit, the Local-FFN unit improves on the defect that the original FFN unit only performs linear transformation on each feature separately and lacks local information interaction. By introducing a 3×3 deep convolutional layer, it can perceive the local features of image pixels and their adjacent small areas, allowing adjacent pixels to exchange information, thereby obtaining the ability to model local area relationships.
[0067] The encoder module is mainly composed of the intra-scale feature interaction module (AIFI) and the asymptotic feature pyramid network module (AFPN). Fig.12As shown. First, it is necessary to perform a 1×1 convolution operation on the image features S3, S4, and S5 of the last three levels obtained by the feature extraction module to align the number of channels. Because S5 contains the most advanced semantic features, processing S5 alone can not only maintain the performance of the model, but also significantly reduce the amount of calculation. Therefore, only the processed S5 is flattened and then sent to the AIFI structure. The AIFI structure is a Transformer encoder layer, which uses Local-FFN units to replace the original FFN units in the encoder layer. After S5 is processed by AIFI, the feature shape is restored and recorded as F5.
[0068] Use AFPN to perform cross-scale feature fusion of S3, S4, and F5. AFPN uses upsampling and downsampling to align features of different scales, first fusing S3 and S4, and then fusing S3, S4, and F5. Features at different levels have different weights, and the information of key scale features is guaranteed through weighted fusion. The fusion of S3 and S4 can reduce the semantic gap between them. Since the S4 and F5 features are adjacent, the semantic gap between the S3 and F5 features is also reduced. Therefore, AFPN gradually fuses the image features of the three scales of S3, S4, and F5. The feature information of different scales becomes closer in the process of progressive fusion, which improves the ability of cross-scale feature fusion.
[0069] The multi-scale features obtained by the encoder module are flattened and concatenated, and the query with the least uncertainty is selected to provide more features with accurate classification and precise location, and the true value is denoised to obtain the content information of the border and the location information of the anchor box. These two pieces of information and the concatenated multi-scale feature information are sent to the Transformer decoder for processing to obtain the predicted coordinate information and category information. The FFN unit in the Transformer decoder is upgraded to the Local-FFN unit.
[0070] S3. Iteratively train the traffic target detection model using the low-light detection dataset. First, configure the model training hyperparameters, including the size of the input image, the number of recognition categories, the initial learning rate, and the number of training rounds. Then input the training data into the model for image enhancement, feature extraction, feature fusion, output prediction value, calculation of loss, and back-propagation optimization of the model.
[0071] The loss function of iterative optimization is:
[0072]
[0073] In the formula, The feature vector representing the output of the encoder module; represents the predicted value, represents the predicted bounding box, represents the predicted category; y = {b, c} represents the true value, b represents the true bounding box, and c represents the true category; Represents the difference between the positioning and classification prediction distributions; L represents the total loss value, L box Represents the loss value of the bounding box, L cls Represents the loss value of the category.
[0074] As an embodiment, the back propagation optimization algorithm adopted is Adam.
[0075] After the network model is fully trained and reaches convergence, the model weight file is saved to obtain a trained traffic target detection model.
[0076] S4. Use the trained detection model to detect the target to be detected.
[0077] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well.
Claims
1. A real-time traffic target detection method for low-light illumination scenes, characterized in that: The following steps are involved: S1. Establish a low-light scene detection dataset; S2. Build a traffic target detection model for low-light scenes based on the improved RT-DETR; S3, using the detection data set to train the traffic target detection model; S4. Use the trained traffic target detection model to detect the target to be detected.
2. The method for real-time traffic target detection in low-light illumination scenes according to claim 1, characterized in that: By acquiring a public traffic target detection dataset, collecting image data of low-light scenes, and formatting the collected image data, a low-light scene detection dataset is formed.
3. The real-time traffic target detection method for low-light illumination scenes according to claim 1, characterized in that: The improved RT-DETR-based traffic target detection model for low-light illumination scenes integrates an image enhancement module on the basis of the RT-DETR model; the feature extraction backbone network is upgraded to VMamba-tiny, and the original FFN unit in VMamba-tiny is replaced by the Local-FFN unit. At the same time, the cross-scale feature fusion module CCFM in the encoder is updated to the asymptotic feature pyramid network module AFPN, and the FFN units in the encoder and decoder are also replaced by Local-FFN units.
4. The method for real-time traffic target detection in low-light illumination scenes according to claim 3, characterized in that: The image enhancement module uses a pyramid enhancement network PENet, which first processes the input image using Gaussian convolution and downsampling to obtain a Gaussian pyramid with four layers of components; then processes the Gaussian pyramid using upsampling and Gaussian convolution to obtain a Laplacian pyramid with four layers of components; After obtaining the Laplacian pyramid, PENet is used to enhance the image. Specifically, the detail processing module (DPM) is used to enhance the components of each layer of the Laplacian pyramid, and the low-frequency enhancement filter (LEF) is used to extract low-frequency information. The outputs of the detail processing module DPM and the low-frequency enhancement filter LEF are spliced and fused, and finally the processed Laplacian components are restored to a high-resolution image.
5. The method for real-time traffic target detection in low-light illumination scenes according to claim 4, characterized in that: The detail processing module DPM includes context branch and edge branch; The edge branch focuses on the edge information of the image. First, the Sobel operator is used to calculate the gradient approximation of the image brightness function in the vertical and horizontal directions to identify the edge information. Then, the convolution operation is used to extract the feature information of the image. Finally, the original image and the processed image are fused by residual connection to achieve information enhancement. The context branch is responsible for capturing the long-range dependencies of the image. The context branch includes two residual blocks, which expand and restore the number of channels so that the low-frequency information can be smoothly fused with the high-frequency information after passing through the residual block, thus capturing the global information in the scene.
6. The real-time traffic target detection method for low-light illumination scenes according to claim 4, characterized in that: The low-frequency enhancement filter LEF is used to capture the low-frequency information of the image. It first expands the number of channels through a point convolution layer, and divides the input components into four parts along the channel dimension. The features of each part are applied with the corresponding adaptive average pooling layer to obtain low-frequency information at different cutoff frequencies to form a low-pass filter. Then, bilinear interpolation is used to upsample it back to the original height and width, and the four features are concatenated along the channel dimension to form a new feature map. ReLU is used to activate it, and finally, a point convolution layer is used to restore the number of channels.
7. The real-time traffic target detection method for low-light illumination scenes according to claim 3, characterized in that: The backbone network VMamba-tiny includes a block embedding module and four VMamba levels. The block embedding module scales the size of the input image at a fixed ratio and expands the channel dimension to a fixed dimension to obtain embedded image features. The VMamba level includes a visual state space block VSS BLOCK and a downsampling layer. VSS BLOCK flattens the two-dimensional image features into one-dimensional sequence data through a four-way scanning strategy, and then uses the selective state space model S6 for long-distance relationship modeling, and finally uses a two-dimensional aggregation operation to reaggregate the sequence data. The downsampling layer downsamples the image features to extract multi-scale features at different levels of the backbone network.
8. The method for real-time traffic target detection in low-light illumination scenes according to claim 3, characterized in that: The Local-FFN unit has a 3×3 deep convolutional layer, which can perceive local features of image pixels and their adjacent small areas.
9. The method for real-time traffic target detection in low-light illumination scenes according to claim 3, characterized in that: The encoder module includes an intra-scale feature interaction module AIFI and an asymptotic feature pyramid network module AFPN; the AIFI structure is a Transformer encoder layer, and the Local-FFN unit is used to replace the original FFN unit in the Transformer encoder layer; The AFPN module performs convolution operations on the image features S3, S4, and S5 of the last three levels obtained by the feature extraction module to align the number of channels, and performs progressive feature fusion on S3 and S4 obtained from the backbone network and the fifth-level feature F5 processed by the AIFI module. S3 and S4 are first weightedly fused, and then S3, S4, and F5 are weightedly fused, which helps to reduce the semantic gap between features at different levels.
10. The method for real-time traffic target detection in low-light illumination scenes according to claim 3, characterized in that: The traffic target detection model training process is as follows: setting hyperparameters of the model training process, inputting training data into the traffic target detection model, performing image enhancement, feature extraction and fusion, outputting predicted values and calculating losses, optimizing the model, and saving the model weights after the model is iteratively trained to convergence.
Citation Information
Patent Citations
Container weak and small serial number target detection and identification method based on deep learning
CN117253154A
Method for detecting green crisp plums in complex scene based on improved YOLOv8
CN117612158A
Artificial intelligence system and method serving electric power robot
WO2022188379A1
Cited By
Lightweight road damage detection method based on RT-DETR improvement
CN119784759A
Lightweight Road Damage Detection Method Improved Based on RT-DETR
CN119784759B
Vehicle pedestrian target detection method and device, computer equipment and storage medium
CN119992511A
Eye fundus image lesion segmentation method and device, computer equipment and medium
CN120318249A
