A road target detection method, electronic device and storage medium

Abstract

This invention discloses a road target detection method, electronic device, and storage medium, belonging to the technical field of target detection. The road target detection method includes: acquiring and preprocessing a multimodal image dataset; constructing a dual-stream feature extraction backbone network to extract features from visible light and infrared images respectively; designing an adaptive feature alignment and fusion module to dynamically fuse features from different modalities through feature cascading and cross-attention mechanisms, adaptively adjusting the weights of different modalities; designing a three-branch dynamic parameter enhancement module that integrates defogging and low-light enhancement functions, optimizing scene quality through a lightweight parameter estimation network; and employing a two-stage training strategy: the first stage establishes basic feature extraction capabilities through pre-training, and the second stage focuses on optimizing scene degradation problems. This invention can improve feature complementarity and effectively solve the problem of road target detection under adverse weather conditions.

Description

Technical Field

[0001] This invention relates to the technical field of target detection, and more specifically, to a road target detection method, electronic device, and storage medium. Background Technology

[0002] Over the past decade, thanks to the rapid development of deep learning technology, vision-based object detection has achieved significant breakthroughs in performance and efficiency. As a fundamental task in computer vision, object detection plays an indispensable role in practical applications such as autonomous driving and traffic monitoring. In these applications, traditional object detection methods primarily rely on visible light cameras to acquire image information. In well-lit environments with clear visibility, visible light cameras can provide high-quality images, ensuring the accuracy and reliability of object detection. However, in real-world road environments, single visible light sensors face numerous challenges, especially in adverse weather conditions such as rain, fog, and strong sunlight, often leading to missed and false detections, severely limiting the reliability and practicality of the system.

[0003] To address the limitations of single visible light sensors, researchers have begun exploring multimodal sensing schemes, among which infrared imaging technology has attracted significant attention due to its sensitivity to the thermal radiation characteristics of objects. Infrared sensors can provide clear target contour information under adverse weather conditions, forming a good complementary relationship with visible light sensors. Simultaneously, infrared sensors also offer significant advantages in cost-effectiveness, further enhancing the attractiveness of multimodal sensing schemes. In visible-infrared dual-modal detection research, image fusion technology is crucial for the effective integration of information from the two modalities. Currently, the main image fusion techniques can be categorized into three types: pixel-level fusion, decision-level fusion, and feature-level fusion. While pixel-level fusion is simple to implement, it is prone to introducing fusion artifacts in complex environments, affecting the accuracy of detection results. Decision-level fusion, by integrating the outputs of two independent detectors, avoids image alignment issues, but incurs high computational costs, making it difficult to operate efficiently in real-time applications. In contrast, feature-level fusion integrates information within the network, not only reducing computational costs but also fully utilizing the complementary advantages of the two modalities, thus attracting widespread attention and research.

[0004] However, existing feature-level fusion methods still face several key challenges that urgently need to be addressed. First, image quality is particularly critical in complex environments. Acquired visible light images often suffer from severe quality degradation, directly impacting feature extraction performance and resulting in insufficient feature representation. While image enhancement modules can improve image quality, designing lightweight and efficient enhancement strategies within the framework of detection tasks remains a significant challenge. Second, feature fusion is another unavoidable problem. Features from different modalities have different numerical distributions and representations, and simple feature concatenation or weighted fusion can easily lead to information redundancy or loss. Especially during feature transfer, features from one modality may excessively dominate the fusion result, weakening the complementary advantages of the two modalities. These issues not only restrict the performance improvement of multimodal object detection technology but also hinder its widespread adoption in practical applications. Summary of the Invention

[0005] To overcome the technical problems existing in the prior art, the present invention provides a road target detection method, electronic device and storage medium.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] A road target detection method includes the following steps:

[0008] A multimodal image dataset is acquired and preprocessed, the multimodal image dataset comprising several pairs of visible light images and infrared images;

[0009] A dual-stream feature extraction backbone network is constructed to extract features from the preprocessed visible light image and the infrared image, respectively, to obtain visible light features and infrared features.

[0010] An adaptive feature alignment and fusion module is added to the dual-stream feature extraction backbone network to improve the feature fusion of the visible light features and the infrared features and is trained to obtain pre-trained weights; on the basis of the pre-trained weights, a three-branch dynamic parameter enhancement module for optimizing scene quality is introduced to obtain a road target detection model;

[0011] The image to be detected is input into the road target detection model to obtain the detection results of the road targets.

[0012] Furthermore, the acquisition of the multimodal image dataset specifically involves:

[0013] Obtain a publicly available multimodal image dataset, in which people, cars, buses, motorcycles, trucks, and streetlights are used as labeled objects, and divide the multimodal image dataset into a training set and a test set in an 8:2 ratio.

[0014] Furthermore, the preprocessing includes image normalization, resizing, and data augmentation to improve image quality.

[0015] Furthermore, the construction of the dual-stream feature extraction backbone network specifically involves constructing HGNetv2 backbone networks for feature extraction of the visible light image and the infrared image, respectively.

[0016] Each of the HGNetv2 backbone networks includes:

[0017] The HGStem module is used for initial feature extraction;

[0018] Multiple cascaded HGBlock modules, wherein the multiple HGBlock modules are used for multi-scale feature extraction;

[0019] The DWConv module is used for depthwise separable convolution processing.

[0020] Furthermore, the adaptive feature alignment and fusion module achieves dynamic fusion of features from different modalities through feature concatenation and cross-attention mechanisms, and adaptively adjusts the weights of different modalities;

[0021] The three-branch dynamic parameter enhancement module integrates defogging and low-light enhancement functions, and optimizes scene quality through a lightweight parameter estimation network.

[0022] Furthermore, the adaptive feature alignment and fusion module performs dynamic fusion of the visible light features and the infrared features, including the following steps:

[0023] For the input visible light features F vis and infrared signature F inf Calculate the statistical features of the channel mean and channel maximum value separately, and then concatenate the mean and maximum value features along the channel dimension to obtain the initial statistical features, as shown in the following formula:

[0024]

[0025] F stat =Concat(F avg ,F max )

[0026] Among them, CAA(·) represents the channel-dimensional average aggregation operation, which calculates the average value of the feature map along the channel dimension; CMA(·) represents the channel-dimensional maximum aggregation operation, which calculates the maximum value of the feature map along the channel dimension; Concat(·) represents concatenation by channel.

[0027] The statistical features are rearranged into a sequence, and Q, K, and V are calculated using three transformation matrices. The Q value is then used in conjunction with visible light characteristics. vis K with infrared characteristics inf The relevance score is calculated and normalized using Softmax to obtain the attention weight. The attention weight is then multiplied by V to complete feature enhancement. Simultaneously, reverse attention calculation is performed to achieve bidirectional feature interaction. The formula is as follows:

[0028]

[0029] in, This represents the scaling factor used to adjust the attention score;

[0030] Multi-channel features are compressed into a single channel using a 7×7 convolutional kernel, and a weight map is output using the Sigmoid function:

[0031] F att =σ(Conv(Concat(F) att_vis ,F att_inf )))

[0032] Here, Conv(·) is a 7×7 convolutional kernel used to output a single-channel attention map, achieving mapping compression from multi-channel to single-channel; σ(·) is a sigmoid function used to output the final attention weights; F att It outputs a weighted graph;

[0033] The visible light and infrared features are weighted using the aforementioned weighted graph to achieve dynamic balance and fusion of features:

[0034]

[0035] in, The multiplication operation is represented by . Conv(·) is a 1×1 pointwise convolution operation used to compress the number of channels in the input feature map to the number of channels in the original image.

[0036] Residual connections ensure information flow, and the fused feature map is output for subsequent road target detection.

[0037] Furthermore, the steps of the three-branch dynamic parameter enhancement module in optimizing scene quality include:

[0038] The initial features of the input visible light image are extracted using a shared feature extraction network, as shown in the following formula:

[0039]

[0040] Each convolutional layer is followed by a LeakyReLU activation function, which enables the network to simultaneously preserve the spatial structure information of the image and effectively extract multi-level features from low-level texture to high-level semantics.

[0041] In the dehazing branch, features are extracted through a fifth convolutional network, and a global average pooling is used to obtain a global image description. Atmospheric light values ​​are obtained through a two-layer fully connected network, and a transmission map is output. The atmospheric scattering model formula is then used for image dehazing, as follows:

[0042]

[0043]

[0044] Where t(x) represents the transmission map, A represents the atmospheric light value; fc1 and fc2 represent two fully connected layers; J(x) represents the clear image to be recovered; and I(x) represents the input image with fog.

[0045] In the low-light enhancement branch, eight curve parameters are output through the fifth convolutional network layer and sigmoid mapping is performed. An iterative image enhancement strategy is used to perform eight progressive enhancements, as shown in the following formula:

[0046]

[0047] in, i∈{1,2,...,8} represents the segmentation of the feature map according to the channel dimension, I t+1 It is an image enhancement, I t ε is the original image, r is the curve parameter map; the constant ε is used to ensure the stability of numerical calculations, and the clamp operation ensures that the pixel values ​​remain within a reasonable range during the enhancement process.

[0048] Furthermore, the road target detection model employs a two-stage training strategy to complete model training, specifically as follows:

[0049] In the first stage of training, the initial learning rate was set to 0.0001, the weight decay rate to 0.001, and the batch size to 8. The feature extraction network was pre-trained with the addition of an adaptive feature alignment and fusion module but without the addition of a three-branch dynamic parameter enhancement module to obtain the initial parameters and lay the foundation for feature understanding ability.

[0050] In the second phase of training, the pre-trained weights from the first phase are loaded, and a three-branch dynamic parameter enhancement module is integrated to optimize scene quality, while maintaining the input image size at 640×640 pixels.

[0051] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned road target detection method.

[0052] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned road target detection method.

[0053] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0054] This invention provides a road target detection method, electronic device, and storage medium. The road target detection method includes: acquiring and preprocessing a multimodal image dataset; constructing a dual-stream feature extraction backbone network to extract features from visible light and infrared images respectively; designing an adaptive feature alignment and fusion module to dynamically fuse features from different modalities through feature cascading and cross-attention mechanisms, and adaptively adjusting the weights of different modalities; designing a three-branch dynamic parameter enhancement module that integrates dehazing and low-light enhancement functions, and optimizing scene quality through a lightweight parameter estimation network; and employing a two-stage training strategy, in which the first stage establishes basic feature extraction capabilities through pre-training, and the second stage focuses on optimizing scene degradation problems. This invention can improve feature complementarity and effectively solve the problem of road target detection under adverse weather conditions. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic flowchart of a road target detection method provided in one embodiment of this application;

[0057] Figure 2 This is a schematic diagram of the algorithm structure in the background technology of this application;

[0058] Figure 3 This is a schematic diagram of the AFAF module in the embodiments of this application;

[0059] Figure 4 This is a schematic diagram of the TDPE module in the embodiments of this application;

[0060] Figure 5 This is an optimized structural diagram of the scenario in the embodiments of this application. Detailed Implementation

[0061] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0062] Example 1:

[0063] like Figure 1 As shown, the present invention provides a technical solution:

[0064] A road target detection method includes the following steps:

[0065] A multimodal image dataset is acquired and preprocessed, the multimodal image dataset comprising several pairs of visible light images and infrared images;

[0066] A dual-stream feature extraction backbone network is constructed to extract features from the preprocessed visible light image and the infrared image, respectively, to obtain visible light features and infrared features.

[0067] An adaptive feature alignment and fusion module (AFAF module) is added to the dual-stream feature extraction backbone network to improve the feature fusion of the visible light features and the infrared features and is trained to obtain pre-trained weights; on the basis of the pre-trained weights, a three-branch dynamic parameter enhancement module (TDPE module) for optimizing scene quality is introduced to obtain a road target detection model;

[0068] The image to be detected is input into the road target detection model to obtain the detection results of the road targets.

[0069] Example 2:

[0070] Based on Example 1, the acquisition of the multimodal image dataset specifically involves:

[0071] Obtain a publicly available multimodal image dataset, in which people, cars, buses, motorcycles, trucks, and streetlights are used as labeled objects, and divide the multimodal image dataset into a training set and a test set in an 8:2 ratio.

[0072] Furthermore, the multimodal image dataset is the publicly available multimodal multi-scene object detection datasets M3FD and MSRS;

[0073] The M3FD dataset contains 4,177 pairs of infrared and visible light images, covering different weather conditions such as daytime, nighttime, rainy and foggy days, as well as traffic scenarios in different seasons.

[0074] The MSRS dataset contains 1444 pairs of strictly aligned infrared and visible light image pairs; the dataset covers road scenes both during the day and at night.

[0075] To expand the training samples, the M3FD dataset was combined with the diversity of images under different weather conditions and scenarios, and the high-quality image features of the optimized MSRS dataset. The two datasets were combined to form a training set of 3415 pairs of images and a test set of 854 pairs of images.

[0076] Furthermore, the preprocessing includes image normalization, resizing, and data augmentation to improve image quality.

[0077] Furthermore, the construction of the dual-stream feature extraction backbone network specifically involves constructing HGNetv2 backbone networks for feature extraction of the visible light image and the infrared image, respectively.

[0078] like Figure 2 As shown, each of the HGNetv2 backbone networks includes:

[0079] The HGStem module is used for initial feature extraction;

[0080] Multiple cascaded HGBlock modules, wherein the multiple HGBlock modules are used for multi-scale feature extraction;

[0081] The DWConv module is used for depthwise separable convolution processing.

[0082] Furthermore, the HGStem module consists of a 3×3 convolutional layer with a stride of 2, a BatchNorm layer, a ReLU activation function, and a max pooling layer with a stride of 2;

[0083] The HGBlock module includes a main branch and a skip connection branch. The main branch consists of a 3×3 depthwise separable convolution, a BatchNorm layer, a ReLU activation function, and a 1×1 convolution for channel adjustment. The skip connection branch uses a 1×1 convolution to adjust when the number of input and output channels is different, and uses a convolution with a stride of 2 when the feature map size needs to be changed.

[0084] The visible light and infrared images are processed by the HGStem module to output the C1 feature map, and then processed by four HGBlock modules to obtain the C2, C3, C4 and C5 feature maps, with the number of channels in the feature maps being 64, 128, 256 and 512 respectively. After each HGBlock module, an SE attention mechanism is added to perform channel weighting and residual connections are used to maintain gradient propagation. Finally, a feature pyramid structure is used to upsample the high-level features and fuse them with the low-level features to generate a multi-scale feature representation.

[0085] Furthermore, the adaptive feature alignment and fusion module achieves dynamic fusion of features from different modalities through feature concatenation and cross-attention mechanisms, and adaptively adjusts the weights of different modalities;

[0086] The three-branch dynamic parameter enhancement module integrates defogging and low-light enhancement functions, and optimizes scene quality through a lightweight parameter estimation network.

[0087] Furthermore, the adaptive feature alignment and fusion module performs dynamic fusion of the visible light features and the infrared features, including the following steps:

[0088] like Figure 3 As shown, for the input visible light features F vis and infrared signature F inf Calculate the statistical features of the channel mean and channel maximum value separately, and then concatenate the mean and maximum value features along the channel dimension to obtain the initial statistical features, as shown in the following formula:

[0089]

[0090] F stat =Concat(F avg ,F max )

[0091] Wherein, CAA(·) represents the channel-dimensional average aggregation operation, which calculates the average value of the feature map along the channel dimension; CMA(·) represents the channel-dimensional maximum aggregation operation, which calculates the maximum value of the feature map along the channel dimension; Concat(·) represents concatenation by channel, which allows the network to adaptively learn how to weigh and fuse these features in subsequent layers;

[0092] The statistical features are rearranged into a sequence, and Q, K, and V are calculated using three transformation matrices. The Q value is then used in conjunction with visible light characteristics. vis K with infrared characteristics inf The relevance score is calculated and normalized using Softmax to obtain the attention weight. The attention weight is then multiplied by V to complete feature enhancement. Simultaneously, reverse attention calculation is performed to achieve bidirectional feature interaction. The formula is as follows:

[0093]

[0094] in, This represents the scaling factor used to adjust the attention score;

[0095] Multi-channel features are compressed into a single channel using a 7×7 convolutional kernel, and a weight map is output using the Sigmoid function:

[0096] F att=σ(Conv(Concat(F) att_vis ,F att_inf )))

[0097] Here, Conv(·) is a 7×7 convolutional kernel used to output a single-channel attention map, achieving mapping compression from multi-channel to single-channel; σ(·) is a sigmoid function used to output the final attention weights; F att It outputs a weighted graph;

[0098] The visible light and infrared features are weighted using the aforementioned weighted graph to achieve dynamic balance and fusion of features:

[0099]

[0100] in, The multiplication operation is represented by . Conv(·) is a 1×1 pointwise convolution operation used to compress the number of channels in the input feature map to the number of channels in the original image.

[0101] Residual connections ensure information flow, and the fused feature map is output for subsequent road target detection.

[0102] Furthermore, the steps of the three-branch dynamic parameter enhancement module in optimizing scene quality include:

[0103] like Figure 4 As shown, the initial features of the input visible light image are extracted using a shared feature extraction network, as shown in the following formula:

[0104]

[0105] Each convolutional layer is followed by a LeakyReLU activation function, which enables the network to simultaneously preserve the spatial structure information of the image and effectively extract multi-level features from low-level texture to high-level semantics.

[0106] In the dehazing branch, features are extracted through a fifth convolutional network, and a global average pooling is used to obtain a global image description. Atmospheric light values ​​are obtained through a two-layer fully connected network, and a transmission map is output. The atmospheric scattering model formula is then used for image dehazing, as follows:

[0107]

[0108]

[0109]

[0110] Where t(x) represents the transmission map, A represents the atmospheric light value; fc1 and fc2 represent two fully connected layers; J(x) represents the clear image to be recovered; and I(x) represents the input image with fog.

[0111] In the low-light enhancement branch, eight curve parameters are output through the fifth convolutional network layer and sigmoid mapping is performed. An iterative image enhancement strategy is used to perform eight progressive enhancements, as shown in the following formula:

[0112]

[0113]

[0114]

[0115] in, i∈{1,2,…,8} represents the segmentation of the feature map according to the channel dimension, I t+1 It is an image enhancement, I t This is the original image, and r is the curve parameter map; the constant ε is used to ensure the stability of numerical calculations, and the clamp operation ensures that the pixel values ​​remain within a reasonable range during the enhancement process;

[0116] Finally, scene optimization is performed; such as... Figure 5 As shown, the above dehazing process is performed first to avoid affecting the dehazing effect by first enhancing the exposure in low light; then, low light enhancement is performed to obtain optimized image features, which are then input into the subsequent target detection network.

[0117] Among them, the feature extraction network simultaneously serves the estimation of transmission maps and atmospheric light values ​​in the early stage, realizing the sharing of network parameters and effectively reducing computational overhead.

[0118] Furthermore, the road target detection model employs a two-stage training strategy to complete model training, specifically as follows:

[0119] In the first stage of training, the initial learning rate of 0.0001, the weight decay rate of 0.001, and the batch size of 8 were set. The feature extraction network was pre-trained with the AFAF module but not the TDPE module to obtain reasonable parameter initialization and establish basic feature understanding ability.

[0120] Then, in the second stage of training, the focus of optimization is shifted to scene optimization. The pre-trained weights from the first stage are loaded and the TDPE module is introduced to optimize scene quality, enabling the network to better handle scene degradation issues, while keeping the input image size at 640×640 pixels.

[0121] Furthermore, in this application, all experiments were conducted under the Ubuntu 20.04 operating system, the GPU was an NVIDIA GeForce GTX 3090 with 24GB of video memory, the CPU was an Intel(R) Xeon(R) Gold 5418Y, the deep learning architecture was PyTorch 2.1.0, and the programming language was Python 3.10;

[0122] During training, the initial learning rate was set to 0.0001, the AdamW optimizer with weight decay of 0.0001 was used, the batch size was set to 8, and the number of training epochs was set to 300.

[0123] After training, the optimal weights are used to test the test set to evaluate the model's performance.

[0124] After obtaining the experimental results, comparisons and analyses were conducted. The experimental part used mAP0.5, mAP0.5:0.95, and model size as evaluation indicators. To verify the superiority of the improved strategy proposed in this embodiment over the current mainstream dual-stream target detection algorithms, comparative experiments were carried out. The experimental results are shown in Table 1 below.

[0125] Table 1. Comparative Experiment Results:

[0126]

[0127] Example 3:

[0128] This invention provides a technical solution:

[0129] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned road target detection method.

[0130] In this embodiment, the computer system suitable for implementing the electronic device of the present invention includes a Central Processing Unit (CPU), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or a program loaded from storage into random access memory (RAM), such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0131] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN (Local Area Network) cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0132] Example 4:

[0133] This invention provides a technical solution:

[0134] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the aforementioned road target detection method. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0138] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0139] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more aspects of the application, various features of this application may sometimes be grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, the point of application is that the corresponding technical problem can be solved with fewer features than all of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0140] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0141] It should be noted that the above embodiments are illustrative of this application and not limiting of it, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A road target detection method, characterized in that, Includes the following steps: A multimodal image dataset is acquired and preprocessed, the multimodal image dataset comprising several pairs of visible light images and infrared images; A dual-stream feature extraction backbone network is constructed to extract features from the preprocessed visible light image and the infrared image, respectively, to obtain visible light features and infrared features. An adaptive feature alignment and fusion module is added to the dual-stream feature extraction backbone network to improve the feature fusion of the visible light features and the infrared features and is trained to obtain pre-trained weights; on the basis of the pre-trained weights, a three-branch dynamic parameter enhancement module for optimizing scene quality is introduced to obtain a road target detection model; The image to be detected is input into the road target detection model to obtain the detection results of the road targets; The construction of the dual-stream feature extraction backbone network specifically involves constructing HGNetv2 backbone networks for feature extraction of the visible light image and the infrared image, respectively. Each of the HGNetv2 backbone networks includes: The HGStem module is used for initial feature extraction; Multiple cascaded HGBlock modules, wherein the multiple HGBlock modules are used for multi-scale feature extraction; The DWConv module is used for depth-separable convolution processing. The adaptive feature alignment and fusion module achieves dynamic fusion of features from different modalities through feature concatenation and cross-attention mechanisms, and adaptively adjusts the weights of different modalities. The three-branch dynamic parameter enhancement module integrates defogging and low-light enhancement functions, and optimizes scene quality through a lightweight parameter estimation network; The road target detection model employs a two-stage training strategy to complete model training, specifically: In the first stage of training, the initial learning rate was set to 0.0001, the weight decay rate to 0.001, and the batch size to 8. The feature extraction network was pre-trained with the addition of an adaptive feature alignment and fusion module but without the addition of a three-branch dynamic parameter enhancement module to obtain the initial parameters and lay the foundation for feature understanding ability. In the second phase of training, the pre-trained weights from the first phase are loaded, and a three-branch dynamic parameter enhancement module is integrated to optimize scene quality, while maintaining the input image size at 640×640 pixels.

2. The road target detection method according to claim 1, characterized in that, The specific steps for obtaining the multimodal image dataset are as follows: Obtain a publicly available multimodal image dataset, in which people, cars, buses, motorcycles, trucks, and streetlights are used as labeled objects, and divide the multimodal image dataset into a training set and a test set in an 8:2 ratio.

3. The road target detection method according to claim 1, characterized in that, The preprocessing includes image normalization, resizing, and data augmentation to improve image quality.

4. The road target detection method according to claim 1, characterized in that, The adaptive feature alignment and fusion module performs dynamic fusion of the visible light features and the infrared features, including the following steps: Visible light features of the input and infrared features Calculate the statistical features of the channel mean and channel maximum value separately, and then concatenate the mean and maximum value features along the channel dimension to obtain the initial statistical features, as shown in the following formula: ， ， ， in, This indicates a channel-dimensional average aggregation operation, which calculates the average value of the feature map along the channel dimension. This indicates the maximum value aggregation operation along the channel dimension, which calculates the maximum value of the feature map along the channel dimension. This indicates splicing by channel; The statistical features are rearranged into a sequence, and then calculated using three transformation matrices. , and Utilizing the characteristics of visible light With infrared characteristics Calculate the relevance score and obtain the attention weight after Softmax normalization. Then, combine the attention weight with... Multiplication enhances features, while reverse attention is performed to achieve bidirectional feature interaction, as shown in the following formula: , ， in, This represents the scaling factor used to adjust the attention score; Multi-channel features are compressed into a single channel using a 7×7 convolutional kernel, and a weight map is output using the Sigmoid function: ， in, It is a 7×7 convolutional kernel used to output a single-channel attention map to achieve mapping compression from multi-channel to single-channel; The Sigmoid function is used to output the final attention weights; It outputs a weighted graph; The visible light and infrared features are weighted using the aforementioned weighted graph to achieve dynamic balance and fusion of features: ， in, Indicates the multiplication operation; It is a 1×1 pointwise convolution operation used to compress the number of channels in the input feature map to the number of channels in the original image; Residual connections ensure information flow, and the fused feature map is output for subsequent road target detection.

5. The road target detection method according to claim 1, characterized in that, The steps of the three-branch dynamic parameter enhancement module to optimize scene quality include: The initial features of the input visible light image are extracted using a shared feature extraction network, as shown in the following formula: ， Each convolutional layer is followed by a LeakyReLU activation function, which enables the network to simultaneously preserve the spatial structure information of the image and effectively extract multi-level features from low-level texture to high-level semantics. In the dehazing branch, features are extracted through a fifth convolutional network, and a global average pooling is used to obtain a global image description. Atmospheric light values ​​are obtained through a two-layer fully connected network, and a transmission map is output. The atmospheric scattering model formula is then used for image dehazing, as follows: ， in, This represents a transmission diagram. Indicates atmospheric light value; , This represents two fully connected layers; Indicates the clear image that needs to be restored; This represents an input image with fog. In the low-light enhancement branch, eight curve parameters are output through the fifth convolutional network layer and sigmoid mapping is performed. An iterative image enhancement strategy is used to perform eight progressive enhancements, as shown in the following formula: ， ， in, This indicates that the feature map is segmented according to the channel dimension. It is image enhancement. It is the original image. It is a curve parameter graph; constant To ensure the stability of numerical calculations, the clamp operation ensures that pixel values ​​remain within a reasonable range during the enhancement process.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the road target detection method as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the road target detection method as described in any one of claims 1 to 5.

Images