Real-time Image Enhancement Method and System Based on Multimodality

Abstract

This invention relates to the field of data augmentation technology in intelligent transportation, and particularly to a real-time image enhancement method and system based on multimodal imaging. The method involves acquiring multimodal image data from traffic monitoring, including raw infrared image data and raw RGB image data; performing feature extraction and preprocessing on the multimodal image data; fusing the extracted data features using a cross-modal attention mechanism; and obtaining the weights of a three-dimensional lookup table using a multilayer perception mechanism. The method then uses the raw RGB image and the three-dimensional lookup table to perform trilinear interpolation on the feature fusion result to obtain a color-enhanced target image. Finally, the method aligns the structural consistency features of the raw RGB and raw infrared images, fuses the aligned features through a single convolutional layer, and refines them with skip connections to obtain the final enhanced image. This invention can enhance real-time multimodal traffic images, improving the performance and effectiveness of traffic monitoring systems.

Description

Technical Field

[0001] This invention relates to the field of data augmentation technology in intelligent transportation, and in particular to a real-time image augmentation method and system based on multimodality. Background Technology

[0002] Real-time traffic monitoring image enhancement involves acquiring original traffic images and then processing them using algorithms to make the processed images clearer and more colorful. Traditional image enhancement techniques mainly include histogram equalization, affine transformation, curve fitting functions, and 3-dimensional lookup tables (3D LUTs). However, in the field of traffic monitoring, real-time image enhancement and transmission are challenging tasks. Traditional image enhancement algorithms often can only process images from a single source, failing to meet real-time requirements. Furthermore, while algorithms incorporating neural networks have achieved some success in image enhancement, their high computational complexity cannot meet the needs of real-time traffic monitoring. Therefore, current image enhancement algorithms cannot satisfy the requirements of real-time traffic monitoring. Summary of the Invention

[0003] To address this issue, the present invention provides a multimodal real-time image enhancement method and system to solve the problem of low image quality in real-time traffic monitoring. Based on multimodal real-time traffic image enhancement, the performance and effectiveness of traffic monitoring systems are improved.

[0004] According to the design scheme provided by the present invention, on the one hand, a real-time image enhancement method based on multimodality is provided, comprising:

[0005] Acquire multimodal image data from traffic monitoring, wherein the multimodal image data includes raw infrared image data and raw RGB image data;

[0006] Feature extraction and preprocessing of multimodal image data are performed by resolution feature alignment and channel compression. The extracted data features are fused using a cross-modal attention mechanism, and the weights of the three-dimensional lookup table are obtained using a multilayer perception mechanism.

[0007] The original RGB image is used to perform trilinear interpolation on the feature fusion result through a three-dimensional lookup table to obtain the color-enhanced target image.

[0008] The original RGB image and the original infrared image are aligned with structural consistency features. The aligned features are then fused through a convolutional layer. The fused features are then thinned by skip connections with the target image to obtain the final enhanced image with fused infrared features.

[0009] As a real-time image enhancement method based on multimodality of the present invention, further, feature extraction and preprocessing of multimodal image data features are performed through resolution feature alignment and channel compression, including:

[0010] First, the image data is scaled to a fixed resolution using interpolation, and then the spatial information of the image data is transformed by a preset multiple through a conversion operation to align the resolution of the image data in both infrared and RGB modes.

[0011] Then, for the image features in the two modalities after feature alignment, channel compression is performed using shared parameter convolution to obtain the preprocessed image data features in each modality.

[0012] As a real-time image enhancement method based on multimodality of the present invention, further, a cross-modal attention mechanism is used to perform feature fusion on the extracted data features, including:

[0013] First, for the RGB image data features, the channels are expanded by 3m times and the corresponding Q, K, and V feature vectors are obtained by block splitting. Self-attention feature extraction is performed on the Q, K, and V feature vectors to obtain the target features of the RGB image. For the infrared image data features, the channels are expanded by 2m times and the corresponding K and V feature vectors are obtained by block splitting.

[0014] Then, the target features of the RGB image are used as the query and fused with the K and V feature vectors of the infrared image to form cross-modal channel attention features. The weights of the three-dimensional lookup table are obtained using a multi-layer perception mechanism.

[0015] As a real-time image enhancement method based on multimodality of the present invention, further, the weights of the three-dimensional lookup table are obtained using a multilayer perceptron mechanism, including:

[0016] The feature fusion results are processed using average pooling and convolution operations, and weight prediction is achieved through a fully connected layer.

[0017] As a real-time image enhancement method based on multimodality of the present invention, further, trilinear interpolation calculation is performed on the feature fusion result using the original RGB image and a three-dimensional lookup table, including:

[0018] For a point with target coordinates in the original RGB image, determine its position in the three-dimensional lookup table space and map the RGB values ​​to the three-dimensional lookup table space. Find the control pixel point that is closest to the target position in the three-dimensional lookup table space. For each control pixel point, calculate its comprehensive weight using Euclidean distance and the weight of the three-dimensional lookup table, and use the weighted average color value of the control pixels as the final interpolation result.

[0019] As a real-time image enhancement method based on multimodality of the present invention, further, structural consistency feature alignment is performed on the original RGB image and the original infrared image, including:

[0020] The features of both the original infrared image and the RGB image are structurally aligned at the pixel level using dilated convolution with shared parameters.

[0021] As a real-time image enhancement method based on multimodality in this invention, the process of using the fused features and the image after three-dimensional lookup table to perform skip connection thinning is represented as follows:

[0022] Among them, G v G ir These are image features, I, after structural consistency alignment of the original infrared image and the RGB image, respectively. lut For the color-enhanced target image, Enhanced image for final fusion of infrared features.

[0023] Furthermore, this invention also provides a real-time image enhancement system based on multimodality, comprising: a data acquisition module, a data processing module, a color enhancement module, and an enhancement output module, wherein...

[0024] The data acquisition module is used to acquire multimodal image data from traffic monitoring, including raw infrared image data and raw RGB image data.

[0025] The data processing module is used to extract and preprocess features from multimodal image data through resolution feature alignment and channel compression, perform feature fusion on the extracted data features using a cross-modal attention mechanism, and obtain the weights of the three-dimensional lookup table using a multilayer perception mechanism.

[0026] The color enhancement module is used to perform trilinear interpolation calculation on the feature fusion result using the original RGB image and a three-dimensional lookup table to obtain the color-enhanced target image.

[0027] The enhanced output module is used to perform structural consistency feature alignment on the original RGB image and the original infrared image. The aligned features are fused through a convolutional layer, and the fused features are then refined by skip connections with the target image to obtain the final enhanced image with fused infrared features.

[0028] In another aspect, the present invention also provides a real-time traffic monitoring system based on multiple modules, comprising: a traffic image acquisition terminal with multimodal image acquisition function for real-time collection of traffic multimodal image data; a host computer connected to the traffic image acquisition terminal for traffic image enhancement processing; a central monitoring server cluster connected to the host computer for centralized storage of traffic data; and a monitoring client connected to the central monitoring server cluster for real-time monitoring and analysis based on traffic data. The host computer stores a computer program, which, when executed, can implement the methods described above.

[0029] The beneficial effects of this invention are:

[0030] This invention combines a highly efficient 3D lookup table with a lightweight cross-modal neural network, meeting both the real-time requirements of traffic monitoring and enhancing the quality of the augmented images. Image fusion employs a thinning process, particularly effective in nighttime scenes. This network-based thinning leverages the temperature sensitivity of infrared images to achieve more realistic and clearer details in the augmented image. Furthermore, experimental data demonstrates that this solution significantly reduces the number of model parameters while maintaining image enhancement effectiveness. For 480P images, only 4.6K model parameters are required to complete multimodal traffic image enhancement, meeting the application deployment needs in the field of intelligent transportation. Attached image description:

[0031] Figure 1 This is a schematic diagram illustrating the principle of real-time image enhancement in the embodiment;

[0032] Figure 2 This is a schematic diagram of the deployment topology in the embodiment;

[0033] Figure 3 This is a schematic diagram of the overall neural network architecture in the embodiment;

[0034] Figure 4 This is a schematic diagram of the cross-modal weight prediction network in the embodiment;

[0035] Figure 5 This is a schematic diagram of multimodal image fusion and refinement in the embodiment. Detailed implementation method:

[0036] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described in detail below with reference to the accompanying drawings and technical solutions.

[0037] To address the existing real-time image quality issues in traffic monitoring described in the background section, the present invention provides embodiments, see [link to relevant documentation]. Figure 1 and 2As shown, a multi-module-based real-time traffic monitoring system is provided, comprising: a traffic image acquisition terminal with multimodal image acquisition capabilities for real-time collection of traffic multimodal image data; a host computer connected to the traffic image acquisition terminal for traffic image enhancement processing; a central monitoring server cluster connected to the host computer for centralized storage of traffic data; and a monitoring client connected to the central monitoring server cluster for real-time monitoring and analysis based on traffic data. The host computer stores a computer program, which, when executed, enables real-time image enhancement based on multimodal data. Specifically, the real-time image enhancement method based on multimodal data can be designed to include the following:

[0038] Acquire multimodal image data from traffic monitoring, wherein the multimodal image data includes raw infrared image data and raw RGB image data;

[0039] Feature extraction and preprocessing of multimodal image data are performed by resolution feature alignment and channel compression. The extracted data features are fused using a cross-modal attention mechanism, and the weights of the three-dimensional lookup table are obtained using a multilayer perception mechanism.

[0040] The original RGB image is used to perform trilinear interpolation on the feature fusion result through a three-dimensional lookup table to obtain the color-enhanced target image.

[0041] The original RGB image and the original infrared image are aligned with structural consistency features. The aligned features are then fused through a convolutional layer. The fused features are then thinned by skip connections with the target image to obtain the final enhanced image with fused infrared features.

[0042] LUT stands for Look-Up Table. In the film industry, due to different playback devices, mapping between different color spaces is necessary. 3D LUTs, a technology widely used in film and display industries, are used for precise color correction. A 3D LUT is composed of three 1D LUTs (RGB layers). The input RGB color values ​​are mapped according to the three lookup tables of the 3D LUT to obtain the converted color. Real-time acquisition of multimodal traffic monitoring image data using sensors and other monitoring equipment with both infrared and RGB image acquisition capabilities, and the refined fusion of learnable 3D LUTs with multimodal networks, can improve the enhancement of image color and detail. See also... Figure 3As shown, traffic monitoring images, including infrared and RGB images, are acquired in real time through sensors and other devices. Multimodal fusion features are used to predict the data with three weights. The corresponding 3D LUTs for the multimodal data are calculated using three basic blocks and three weights. Trilinear interpolation is used to perform real-time color enhancement on the RGB images. Local cross-modal feature fusion is performed on the original images using the infrared image modality. The color-enhanced images are then fine-tuned locally using the fusion features, and infrared information is added to the enhanced images to obtain the final enhanced images.

[0043] Specifically, feature extraction and preprocessing of multimodal image data features through resolution feature alignment and channel compression can be designed to include:

[0044] First, the image data is scaled to a fixed resolution using interpolation, and then the spatial information of the image data is transformed by a preset multiple through a conversion operation to align the resolution of the image data in both infrared and RGB modes.

[0045] Then, for the image features in the two modalities after feature alignment, channel compression is performed using shared parameter convolution to obtain the preprocessed image data features in each modality.

[0046] The two modal images can be represented as infrared images I. ir and RGB image I v For these two modalities, feature alignment is performed at varying resolutions. For the input image I, feature alignment can be described as follows:

[0047] F=Unshuffle(4,Interpolate(I)) (1)

[0048] Interpolate represents scaling the image to a fixed resolution using interpolation. Unshuffle is then used to transform the spatial information by a factor of 4. To ensure the stability and efficiency of multimodal features, the aligned features Fir and Fv are compressed by a factor of 2 using convolution with shared parameters. The specific operations are as follows:

[0049] F v ′,F′ ir =Conv * (F v ,F ir (2)

[0050] Among them, feature fusion of extracted data features using a cross-modal attention mechanism can include:

[0051] First, for the RGB image data features, the channels are expanded by 3m times and the corresponding Q, K, and V feature vectors are obtained by block splitting. Self-attention feature extraction is performed on the Q, K, and V feature vectors to obtain the target features of the RGB image. For the infrared image data features, the channels are expanded by 2m times and the corresponding K and V feature vectors are obtained by block splitting.

[0052] Then, the target features of the RGB image are used as the query and fused with the K and V feature vectors of the infrared image to form cross-modal channel attention features. The weights of the three-dimensional lookup table are obtained using a multi-layer perception mechanism.

[0053] For RGB image features F v It employs a 3m-fold channel expansion and splits the chunks into Q, K, and V. The specific description is as follows:

[0054] {Q v ,K v V v}=Chunk(Conv(3m,F v ′)) (3)

[0055] Where m represents the number of heads in the multi-head attention mechanism. Self-attention feature Zv is extracted from the three features {Qv, Kv, Vv} of the RGB image, as described below:

[0056] Z v =softmax(Q) v (K v ) T / β1)V v (4)

[0057] Where β1 is a learnable parameter. For infrared image features F′ ir We use a 2m-fold channel expansion and split the chunk into K and V.

[0058] {K ir V ir}=Chunk(Conv(2m,F′ ir (5)

[0059] Finally, a cross-modal attention mechanism is used to fuse the RGB features Zv and {Kir, Vir}. The specific calculation method is as follows:

[0060] Z = Softmax(Zv(Kir)) T / β2)Vir (6)

[0061] Here, β2 is a learnable parameter, and Z is the multimodal fusion feature. Finally, average pooling (Avgpool) and a convolution with 3 channels are used for feature extraction to process the multimodal fusion feature Z. Then, a fully connected layer is used for prediction of the three weights.

[0062] W=Linear(Conv(Avgpool(Z))) (7)

[0063] The number of prediction weights W is 3.

[0064] See Figure 4 As shown, for RGB and infrared images, the resolution of both modal images is first adjusted, and the feature size is uniformly adjusted to R. 3×480×720 The images are used. PixelUnshuffle is employed to transform the spatial information of the two modalities by a factor of 2, and the transformed information feature size is X (X∈R). 12×224×360 Then, a linear transformation with shared parameters is used to process the two modes respectively, and the feature size after processing is Y (Y∈R). 6×224×360 Then, multi-head self-attention feature extraction was performed on the RGB images separately. Since spatial information transformation was used, efficient channel self-attention was sufficient to effectively extract global features. The extracted features were then used as the Query to perform cross-modal channel attention feature fusion with the infrared features. The fused feature is Z (Z∈R). 6×224×360 The final MLP mainly consists of an average pooling layer, a linear transformation, and a fully connected layer. After processing by the multilayer perceptron MLP, three 3D LUT weights are obtained.

[0065] Furthermore, by using the original RGB image and performing trilinear interpolation on the feature fusion result through a three-dimensional lookup table, the design can include the following:

[0066] For a point with target coordinates in the original RGB image, determine its position in the three-dimensional lookup table space and map the RGB values ​​to the three-dimensional lookup table space. Find the control pixel point that is closest to the target position in the three-dimensional lookup table space. For each control pixel point, calculate its comprehensive weight using Euclidean distance and the weight of the three-dimensional lookup table, and use the weighted average color value of the control pixels as the final interpolation result.

[0067] Employs a learnable three-modal 3DLUT foundation table M (M∈R) 3×33×33×33The target 3DLUT is dynamically acquired using W. For a point P(x,y,z) in the input RGB image with coordinates (x,y,z), eight nearby points are first found in the multimodal 3D LUT space, namely {P1,P2,P3,...,P8}. The weights of each point are calculated as Ur, Ug, and Ub based on the distance. The formula for calculating the pixel C(r`,g`,b` at that point is as follows:

[0068]

[0069] Here, n is the number of elements in the base table (3), and the number of weights is the same. By performing a lookup table on the group pixels of image Iv, the color-enhanced image Ilut can be obtained.

[0070] The algorithm for performing trilinear interpolation on three basic 3D LUTs using real-time generated weights can be described as follows:

[0071] 1. First, determine the position of the RGB image pixel values ​​in the 3D LUT based on the input RGB image pixel values. Map the RGB values ​​to the coordinate space of the three 3D LUTs.

[0072] 2. In each of the three 3D LUTs, find the 24 control points closest to the target location. Each control point includes the 8 surrounding vertices.

[0073] 3. For each pixel, the overall weight of that pixel can be calculated using Euclidean distance and the weights derived in step two.

[0074] 4. The weighted average of the 24 color values ​​is used as the final interpolation result.

[0075] By using the above calculation method and performing trilinear interpolation on the basic 3D LUT with real-time generated weights, the color values ​​of the enhanced image can be obtained more accurately. This interpolation method can improve the color saturation and other performance characteristics of the image, thereby further enhancing the quality and visualization effect of traffic monitoring images.

[0076] The processed image only enhances color saturation; multimodal image fusion is still needed to refine the RGB image in order to incorporate details from the infrared image. The process involves refining and fusing the color-enhanced image with the infrared image. Specifically, structural consistency learning is performed on the original RGB and infrared images using dilated convolution with shared parameters. The operation is as follows:

[0077] G v G ir =dilate_conv * (I v ,I ir(9)

[0078] The fused features are then thinned by skip connections with the enhanced image Ilut processed by S2, resulting in the final enhanced image. The specific steps are as follows:

[0079]

[0080] See Figure 5 As shown, a lightweight design is employed, using dilated convolution with shared parameters for structural consistency alignment, ensuring pixel-level feature consistency between infrared and RGB images. For both infrared and RGB images with a resolution of 480×720, the feature size after feature alignment is R. 6×480×720 .

[0081] To ensure the lightweight nature of the neural network, multimodal fusion is achieved using only one convolutional layer after concatenating the channel dimensions of the two modal features. After ReLU activation of the fused features, a skip connection is made with the color enhancement image from step three to obtain the final enhanced image with infrared features.

[0082] Multimodal real-time image enhancement can be computed on a host computer. Testing showed that for infrared and RGB images with a resolution of 480×720, the number of trainable network parameters is only 4.6K. Therefore, the hardware requirements for the host computer are not high, allowing for large-scale deployment. After image enhancement by multiple host computers, all enhanced image data is stored in a central monitoring server cluster via a routing gateway. This enables real-time enhancement of traffic monitoring images and centralized storage of the enhanced image data. Monitoring clients can access the enhanced traffic images in real time through video monitoring programs. By connecting to the central monitoring server cluster, monitoring clients can obtain real-time enhanced image data and perform real-time monitoring and analysis. In this way, the traffic monitoring system can provide clearer and more accurate images, helping monitoring personnel better observe and judge traffic conditions, improving traffic safety and management efficiency.

[0083] Furthermore, based on the above method, this embodiment of the invention also provides a real-time image enhancement system based on multimodality, comprising: a data acquisition module, a data processing module, a color enhancement module, and an enhancement output module, wherein,

[0084] The data acquisition module is used to acquire multimodal image data from traffic monitoring, including raw infrared image data and raw RGB image data.

[0085] The data processing module is used to extract and preprocess features from multimodal image data through resolution feature alignment and channel compression, perform feature fusion on the extracted data features using a cross-modal attention mechanism, and obtain the weights of the three-dimensional lookup table using a multilayer perception mechanism.

[0086] The color enhancement module is used to perform trilinear interpolation calculation on the feature fusion result using the original RGB image and a three-dimensional lookup table to obtain the color-enhanced target image.

[0087] The enhanced output module is used to perform structural consistency feature alignment on the original RGB image and the original infrared image. The aligned features are fused through a convolutional layer, and the fused features are then refined by skip connections with the target image to obtain the final enhanced image with fused infrared features.

[0088] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0090] The units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations are not considered to be beyond the scope of this invention.

[0091] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This invention is not limited to any particular combination of hardware and software.

[0092] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A real-time image enhancement method based on multimodality, characterized in that, Include: This method acquires multimodal image data from traffic monitoring, including raw infrared image data and raw RGB image data. Feature extraction and preprocessing are performed on the multimodal image data through resolution feature alignment and channel compression. A cross-modal attention mechanism is used to fuse the extracted data features, and a multilayer perception mechanism is used to process the feature fusion results using average pooling and convolution operations. Weight prediction is achieved through a fully connected layer to obtain the weights of a three-dimensional lookup table. Specifically, the feature fusion using the cross-modal attention mechanism includes: first, for RGB image data features, expansion is performed by a 3m-fold increase in channels, and the data is split into blocks to obtain corresponding Q, K, and V feature vectors. Self-attention feature extraction is performed on the Q, K, and V feature vectors to obtain the RGB image target features; for infrared image data features, expansion is performed by a 2m-fold increase in channels, and the corresponding K and V feature vectors are obtained through block splitting; then, the RGB image target features are used as the query, and cross-modal channel attention feature fusion is performed with the K and V feature vectors of the infrared image, and the weights of the three-dimensional lookup table are obtained using a multilayer perception mechanism. For the target coordinates point in the original RGB image, determine its position in the three-dimensional lookup table space and map the RGB values ​​to the three-dimensional lookup table space; find the control pixel point closest to the target position in the three-dimensional lookup table space; for each control pixel point, calculate its comprehensive weight using Euclidean distance and the three-dimensional lookup table weights, and use the weighted average color value of the control pixels as the final trilinear interpolation result to obtain the color-enhanced target image. The original RGB image and the original infrared image are aligned with structural consistency features. The aligned features are then fused through a convolutional layer. The fused features are then thinned by skip connections with the target image to obtain the final enhanced image with fused infrared features.

2. The real-time image enhancement method based on multimodality according to claim 1, characterized in that, Feature extraction and preprocessing of multimodal image data features are performed through resolution feature alignment and channel compression, including: First, the image data is scaled to a fixed resolution using interpolation, and then the spatial information of the image data is transformed by a preset multiple through a conversion operation to align the resolution of the image data in both infrared and RGB modes. Then, for the image features in the two modalities after feature alignment, channel compression is performed using shared parameter convolution to obtain the preprocessed image data features in each modality.

3. The real-time image enhancement method based on multimodality according to claim 1, characterized in that, Perform structural consistency feature alignment on the original RGB image and the original infrared image, including: The features of both the original infrared image and the RGB image are structurally aligned at the pixel level using dilated convolution with shared parameters.

4. The real-time image enhancement method based on multimodality according to claim 1, characterized in that, The process of using the fused features and the target image to perform skip connections for thinning is represented as follows: ,in, These are the image features of the original infrared image and the RGB image after structural consistency alignment, respectively. For the color-enhanced target image, Enhanced image for final fusion of infrared features.

5. A real-time image enhancement system based on multimodality, characterized in that, It includes: a data acquisition module, a data processing module, a color enhancement module, and an enhanced output module. The data acquisition module is used to acquire multimodal image data from traffic monitoring, including raw infrared image data and raw RGB image data. The data processing module is used to extract and preprocess features from multimodal image data through resolution feature alignment and channel compression. It then uses a cross-modal attention mechanism to fuse the extracted data features and employs a multilayer perception mechanism with average pooling and convolution operations to process the feature fusion results. Finally, it uses a fully connected layer to predict weights and obtain the weights for a 3D lookup table. Specifically, the feature fusion using the cross-modal attention mechanism involves: first, expanding the RGB image data features by 3m channels and splitting them into blocks to obtain the corresponding Q, K, and V feature vectors; then, performing self-attention feature extraction on the Q, K, and V feature vectors to obtain the RGB image target features; second, expanding the infrared image data features by 2m channels and splitting them into blocks to obtain the corresponding K and V feature vectors; and finally, using the RGB image target features as the query, fusing them with the K and V feature vectors of the infrared image through cross-modal channel attention feature fusion, and using a multilayer perception mechanism to obtain the weights for the 3D lookup table. The color enhancement module is used to determine the position of the target coordinate point in the original RGB image in the three-dimensional lookup table space and map the RGB values ​​to the three-dimensional lookup table space. It finds the control pixel point closest to the target position in the three-dimensional lookup table space. For each control pixel point, it calculates its comprehensive weight using Euclidean distance and the weight of the three-dimensional lookup table. The color value of the weighted average of the control pixels is used as the final trilinear interpolation result to obtain the color-enhanced target image. The enhanced output module is used to perform structural consistency feature alignment on the original RGB image and the original infrared image. The aligned features are fused through a convolutional layer, and the fused features are then refined by skip connections with the target image to obtain the final enhanced image with fused infrared features.

6. A real-time traffic monitoring system based on multiple modules, characterized in that, The system comprises: a traffic image acquisition terminal with multimodal image acquisition capabilities for real-time collection of traffic multimodal image data; a host computer connected to the traffic image acquisition terminal for traffic image enhancement processing; a central monitoring server cluster connected to the host computer for centralized storage of traffic data; and a monitoring client connected to the central monitoring server cluster for real-time monitoring and analysis based on traffic data. The host computer stores a computer program, which, when executed, enables the implementation of the method described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, enables the implementation of the method as described in any one of claims 1 to 4.

Images