A visual information simulation guided and data driven vehicle-mounted infrared image enhancement method

Abstract

This invention discloses a visual information-guided and data-driven method for enhancing in-vehicle infrared images, belonging to the field of environmental perception technology for intelligent connected vehicles. Addressing the problems of existing in-vehicle infrared image enhancement methods, such as poor global processing, poor real-time performance, lack of environmental information fusion, and insufficient enhancement of key areas, this invention constructs a BVG-Net model based on driver attention data collected by an eye tracker to accurately locate the region of interest (ROI) in the in-vehicle infrared image. It extracts and fuses multimodal features based on weather, time and location, image quality, statistics, and data-driven factors, and adaptively predicts five enhancement parameters, including contrast and brightness, using the TIEFormer model to perform anthropomorphic enhancement of the ROI. The enhanced image is then input into an in-vehicle target detection model to verify the enhancement effect and system real-time performance. This invention can accurately enhance key areas, adapt to adverse weather conditions, meet the real-time deployment requirements of in-vehicle systems, and effectively improve the accuracy of in-vehicle infrared target detection.

Description

Technical Field

[0001] This invention belongs to the field of intelligent connected vehicle environmental perception, computer vision and infrared image processing technology, specifically involving a vehicle infrared image enhancement method that is guided by visual information and driven by data. Background Technology

[0002] In the development of intelligent connected vehicles, the accuracy of environmental perception is crucial for stable vehicle operation. Infrared imaging can capture target thermal radiation and is not limited by low light conditions or glare from strong light, giving it a significant advantage in adverse lighting conditions. However, heavy rain, snow, fog, and other weather conditions cause atmospheric particles to scatter and absorb infrared waves, leading to decreased image contrast, reduced signal-to-noise ratio, blurred target edges, and loss of features, severely impacting perception accuracy. Furthermore, infrared images lack color information and have sparse texture, making it insufficient for distinguishing pedestrians, animals, and hot road obstacles, easily leading to false positives and false negatives, and failing to meet the refined classification requirements of autonomous driving.

[0003] Existing infrared image processing methods have significant drawbacks. First, solutions based on the human visual system and eye-tracking attention rely heavily on offline expert data, lacking real-time driving context and contextual understanding. They cannot accurately identify the driver's actual gaze and are ill-suited for complex traffic scenarios. Second, traditional image enhancement methods depend on fixed rules and hard-coded mathematical formulas, failing to adequately consider the impact of adverse weather conditions, resulting in artifacts, loss of detail, and poor scene adaptability. Third, deep learning methods based on Transformers and GANs primarily involve global pixel mapping or fusion without external environmental parameter intervention. They cannot simulate the nonlinear, comprehensive parameter optimization performed by professionals for specific targets, leading to insufficient enhancement of key regions.

[0004] In general, current vehicle-mounted infrared image enhancement methods generally suffer from three major problems: First, they are processed uniformly globally, making it impossible to accurately locate and enhance key areas; second, they extract features in a single way, failing to integrate multi-dimensional environmental information such as weather, and ignoring the impact of the environment on the image; and third, the algorithms have poor real-time performance, with redundant enhancement of non-critical areas, affecting the environmental perception effect and making it difficult to adapt to the real-time perception requirements of vehicle-mounted embedded deployment and autonomous driving. Summary of the Invention

[0005] Infrared sensors are widely used in environmental perception under adverse weather conditions due to their good adaptability. However, they are still affected by severe weather conditions such as heavy rain, heavy snow, and dense fog. Existing processing methods are mostly based on data-driven approaches or deep learning to enhance infrared images. However, these methods generally process the entire image, resulting in poor real-time performance, and they also perform irrelevant enhancements on non-obstacle areas, thus affecting environmental perception to some extent.

[0006] To address the aforementioned problems, this invention proposes a vehicle-mounted infrared image enhancement method based on visual information guidance and data-driven approaches. It simulates the adjustment of image parameters by mimicking an experienced human image processing expert. First, this invention collects human eye attention hotspot data using an eye tracker and constructs a BVG-Net (Bio-inspired Visual Guidance Network) region-of-interest (ROI) localization model. This model can mimic the distribution of human eye attention during driving, accurately locating the RIO in the image. Then, it performs time-series hotspot analysis and statistical segmentation on the RIO, dividing the RIO images and standardizing image sizes. Next, a multimodal feature extraction module extracts features from multiple dimensions, including weather and time / location information features accessed through an information interface, image quality features and statistical features extracted from the image itself, and data-driven features extracted through a lightweight MobileNetV4 network. These multimodal features are then fused together. The fusion feature vector is obtained by combining the features. After obtaining the fusion features, in the image anthropomorphic enhancement stage, a TIEFormer (Thermal Image Enhancement Transformer) infrared image enhancement model is constructed. Then, the fusion features are normalized and input into the TIEFormer infrared image enhancement model. TIEFormer can automatically predict and adjust five key parameters of image processing based on the comprehensive features of the input image, including contrast parameter, brightness parameter, gamma correction parameter, sharpening intensity, and spatial noise reduction parameter, to imitate the optimization and enhancement of infrared images by experienced human workers, and achieve adaptive processing of images under different shooting conditions. Finally, a high-quality enhanced image is output through the image processing module.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A visual information-guided and data-driven method for enhancing in-vehicle infrared images, comprising the following steps: Step S1: Based on the driver's eye attention hotspot data collected by the eye tracker, construct and train a biomimetic visual guidance region of interest localization model, and output the region of interest attention coordinates of the vehicle infrared image through the region of interest localization model. Step S2: Based on the region of interest localization model trained in step S1, extract and process the region of interest in the input time series image. Based on the attention coordinates, perform hotspot statistical analysis and priority ranking of the region of interest in the time series vehicle infrared image, crop the region of interest image and unify the image size. Step S3: Perform multimodal feature extraction on the region of interest image after unifying the size, and use the region of interest image to fuse environmental information features accessed by the vehicle external interface, quality and statistical features extracted from the image itself, and data-driven features extracted by a lightweight network to generate a multimodal fusion feature vector; Step S4: Construct an image enhancement parameter prediction model, normalize the multimodal fusion feature vector, input the normalized feature into the trained image enhancement parameter prediction model, and output five key image enhancement parameters: contrast, brightness, gamma correction, sharpening intensity, and spatial noise reduction. Step S5: Adaptive processing is performed on the region of interest image using the five key image enhancement parameters. Contrast adjustment, brightness adjustment, gamma correction, sharpening and spatial noise reduction are sequentially performed on the region of interest image using the five key image enhancement parameters to generate an enhanced image. Step S6: Based on the processing results, output the anthropomorphic enhanced vehicle infrared region of interest image; Step S7: Input the anthropomorphically enhanced vehicle-mounted infrared region of interest image into the vehicle-mounted target detection model to perform the vehicle-mounted target detection task and verify the image enhancement effect and the system's real-time processing performance.

[0008] Furthermore, the attention hotspot data acquisition in step S1 includes multi-driver, multi-scene acquisition, invalid data removal, and time-series attention point map generation; the region of interest localization model is based on MobileNetV4 pre-trained on the ImageNet dataset, with its original fully connected classification layer removed and replaced with a regression output head specifically for locating regions of interest in vehicle infrared images. The regression output head reduces the feature map to a 960-dimensional feature vector through a global average pooling layer, and then outputs normalized coordinates (x, y) of the point of interest through a fully connected layer.

[0009] Furthermore, in step S2, the hotspot statistical analysis employs a density clustering algorithm to cluster attention points, and combines regional features to prioritize regions of interest; wherein the clustering parameter is set as: neighborhood radius. The minimum number of points (MinPts) is 5% of the image width, and the minimum number of points (MinPts) is 10% of the total number of points in the window. The priority ranking score takes into account the region area ratio, the spatial density of attention points, the attention duration, and the positional weight of the region relative to the image center.

[0010] Furthermore, the multimodal fusion feature vector generated in step S3 specifically includes: The features are: 10-dimensional weather information features accessed via the interface; 3-dimensional time and location information features accessed via the interface; 6-dimensional image quality features extracted from the image; 10-dimensional grayscale statistical features extracted from the image; 36-dimensional data-driven semantic features extracted from the image and reduced by principal component analysis (PCA); and the features of each modality are fused using a simple splicing method.

[0011] Furthermore, in step S4, the image enhancement parameter prediction model adopts a multi-head self-attention encoder structure, including: an input layer receiving 65-dimensional normalized multimodal fusion features; mapping the 65-dimensional features to a 128-dimensional embedding space through linear transformation; an encoder layer containing an eight-head self-attention mechanism, a feedforward neural network (FFN) with a dimension of 2048, residual connections, and layer normalization, stacking four encoder layers; and a fully connected output layer sequentially mapping the 128-dimensional features to 64-dimensional and 32-dimensional features, finally outputting 5-dimensional image processing parameters.

[0012] Furthermore, in step S4, the five key image enhancement parameters all correspond to preset numerical adjustment ranges, specifically: (1) Contrast parameter, range [0.5, 2.0]: controls the intensity of image contrast adjustment, scaling formula: contrast param =0.5 + 1.5 × sigmoid(output1); (2) Brightness parameter, range [0.5, 2.0]: controls the intensity of image brightness adjustment, scaling formula: brightness param =0.5 + 1.5 × sigmoid(output2); (3) Gamma correction parameters, range [0.1, 2.5]: used for non-linear brightness adjustment, improving the contrast distribution and detail of infrared images by adjusting the gamma curve. Scaling formula gamma param =0.1+2.4×sigmoid(output3); (4) Sharpening intensity, range [0, 2.0]: Controls the intensity of image sharpening. Cropping formula: sharpen strength =clip(relu(output4),0,2.0); (5) Spatial noise reduction adjustment coefficient, range [0, 1.0]: controls the smoothing intensity of the image spatial filter, used to preserve target edge information as much as possible while suppressing thermal imaging background noise. Scaling formula denoise param =sigmoid(output5).

[0013] Furthermore, in step S5, the contrast adjustment adopts adaptive histogram equalization, which divides the image into multiple small blocks, performs histogram equalization on each small block, and limits the contrast enhancement amplitude; the brightness adjustment is achieved by converting the RGB image to an HSV image and then adjusting the V channel; the gamma correction uses power operation to perform non-linear brightness adjustment; the sharpening process is achieved by calculating high-frequency details and superimposing them on the original image; and the spatial noise reduction adopts edge-preserving noise reduction filtering.

[0014] Furthermore, in step S7, the system's real-time processing performance meets the real-time frame rate requirements for vehicle deployment, and the total processing time for a single frame does not exceed 26.3ms.

[0015] Furthermore, in step S2, when cropping the region of interest image, the bounding box coordinates determined according to priority sorting are cropped from the original image. In order to prevent the attention point from being cropped, the bounding box of the cropped part is enlarged by 1.2 times in both length and width, and it is ensured that the coordinates of the enlarged bounding box do not exceed the image boundary. If the calculated bounding box exceeds the image range, boundary cropping is performed.

[0016] Furthermore, the 6-dimensional image quality features in step S5 specifically include: The 1D noise level feature uses a local variance estimation method. First, the image is filtered by median filtering to remove noise. Then, the standard deviation of the difference between the original image and the filtered image is calculated as the noise level estimate and normalized to the range of 0, 10, 1. The image sharpness score is calculated using the Laplacian operator, a Laplacian filter is applied to the image and the variance of the response is calculated, and the sharpness score is normalized to the range of 0, 10, 1. Two-dimensional fuzzy type features are used to determine the fuzzy type through frequency domain analysis. The first dimension is the motion fuzzy direction feature, which is determined by calculating the main direction of energy distribution in the spectrum to determine the motion fuzzy direction angle, normalized to 0, 1 corresponds to 0° to 180°. The second dimension is the weather fuzzy or defocused fuzzy feature, which is determined by analyzing the attenuation characteristics of the spectrum to determine the fuzzy type and output a probability value, where 0 indicates defocused fuzziness and 1 indicates weather fuzziness. One-dimensional exposure feature: calculates the average brightness and histogram distribution of the image, normalizes to the range of -1 to 1, negative values ​​indicate underexposure, positive values ​​indicate overexposure, and values ​​close to 0 indicate normal exposure; One-dimensional pixel contrast characteristics are used to calculate the RMS contrast value and normalize it to the range of 0 and 1.

[0017] The present invention provides a visual information-guided and data-driven method for anthropomorphic enhancement of vehicle-mounted infrared images, which achieves several technical advantages: 1. This invention uses biomimetic vision-guided BVG-Net to locate the region of interest in vehicle-mounted infrared images, breaking through the limitations of traditional global enhancement of infrared images. It achieves human-like enhancement that matches the human eye's attention perception habits, effectively improving the visual presentation of key areas in vehicle-mounted infrared images.

[0018] 2. This invention integrates environmental information features, image features, and data-driven features to generate a multimodal fusion vector, fully exploring the correlation value of multi-dimensional information in vehicle scenarios, solving the problems of poor scene adaptability, loss of details, or noise residue caused by single feature enhancement, and enhancing the generalization ability of infrared image enhancement schemes.

[0019] 3. This invention uses the TIEFormer model to adaptively predict five key image processing parameters, including contrast and brightness, replacing the traditional manual parameter tuning mode. This enables intelligent and automated generation of infrared image enhancement parameters, significantly improving the efficiency of the vehicle-mounted infrared image enhancement process and the accuracy of parameter optimization.

[0020] 4. The overall solution of this invention adopts a lightweight network architecture and modular progressive design. Combined with the real-time verification of vehicle target detection tasks, it ensures that the processing time of the entire process meets the real-time inference requirements of the vehicle end, adapts to the computing power conditions of vehicle embedded devices, and has good engineering feasibility.

[0021] 5. This invention directly inputs the anthropomorphically enhanced infrared image into the vehicle target detection model. Through dual verification of quantitative detection indicators and time consumption statistics, it achieves precise linkage between the infrared image enhancement effect and the actual vehicle application scenario, directly improving the mAP index of vehicle target detection and providing reliable visual data support for intelligent driving environment perception. Attached Figure Description

[0022] Figure 1 This is a flowchart of a vehicle-mounted infrared image enhancement method based on visual information guidance and data-driven principles, according to the present invention. Figure 2 This is a technical roadmap for a vehicle-mounted infrared image enhancement method based on simulated visual information guidance and data-driven principles, according to the present invention. Figure 3 A flowchart illustrating the process of collecting time-series eye-tracking attention hotspot images for this invention; Figure 4 This is a schematic diagram of the installation location of the vehicle-mounted eye tracker used to collect driver attention data according to the present invention; Figure 5 This is a schematic diagram illustrating the time-series eye tracker-based attention hotspot image cropping and unified image size method of the present invention. Figure 6 This is a diagram illustrating the overall framework of the TIEFormer infrared image parameter enhancement model of this invention. Detailed Implementation

[0023] The following detailed implementation of a visual information-guided and data-driven method for anthropomorphic enhancement of vehicle infrared images according to the present invention will be described in detail with reference to specific embodiments. The embodiments of the present invention are only used to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0024] Example 1 like Figure 1 The diagram shown is a flowchart of a vehicle-mounted infrared image enhancement method based on simulated visual information guidance and data-driven principles, according to the present invention. Figure 2 The diagram shows a technical roadmap for a vehicle-mounted infrared image enhancement method based on simulated visual information guidance and data-driven approaches, primarily used to improve image quality and the accuracy of subsequent tasks. The method includes the following steps: Step S1: Construct the BVG-Net region of interest localization model based on eye tracker.

[0025] This step constructs a bio-inspired visual guidance-based region of interest localization model, BVG-Net, to identify and locate regions of interest from input images. The training data for this model comes from human eye attention hotspot data collected by an eye tracker, with all perceptual information sampled at a frequency of 10Hz.

[0026] Step S1 is based on the driver's eye attention hotspot data collected by the eye tracker in the driving scene. It constructs and trains a biomimetic visual guidance region of interest localization model and outputs the region of interest attention coordinates of the vehicle infrared image through the model.

[0027] The attention hotspot data collection in step S1 includes multi-driver, multi-scenario data collection, invalid data removal, and time-series attention point map generation.

[0028] In step S1, all models are trained using supervised learning, and loss functions and early stopping mechanisms to prevent overfitting are configured. The specific process is as follows: Step S1.1, eye tracker data collection and annotation.

[0029] like Figure 3 The diagram shown is a flowchart of the process for collecting time-series eye-tracking attention hotspot images according to this invention. Figure 3 The eye-tracking data collection process shown is divided into four stages: Step S1.1.1, Preparation stage.

[0030] like Figure 4The diagram shows the installation location of the in-vehicle eye tracker used to collect driver attention data according to the present invention. It includes three parts: hardware preparation, software preparation, and experimental preparation. Regarding hardware, the installation location of the eye tracker device is shown below. Figure 4 As shown, the eye tracker is installed below the vehicle's screen, its position adjusted to avoid direct sunlight, and the seating posture adjusted so that the eyes are approximately 60cm away from the screen. On the software side, the relevant drivers and professional analysis software for the eye tracker are installed, and system parameters are configured. For experiments, a 9-point calibration is performed within the software to ensure eye tracking accuracy is within 0.5°.

[0031] Step S1.1.2, Diversified data collection.

[0032] The eye tracker is activated to begin recording, and the driving task commences, simultaneously recording the driver's eye movement data. Twenty different drivers are rotated, and the driving and recording operations are repeatedly calibrated to ensure data diversity and representativeness. Data samples of driving scenarios under different conditions, lighting, and weather are collected, along with simultaneous infrared image acquisition using thread synchronization to ensure temporal and spatial alignment during data collection.

[0033] The experiment recruited 20 volunteers with driving experience (13 men and 7 women), ranging from 2 to 15 years. Each volunteer drove for approximately 30 minutes in both a simulated driving platform and real-world road scenarios. The eye tracker sampling frequency was set to 10Hz, and after 9-point calibration, the tracking accuracy remained stable within the range of 0.3° to 0.5°, meeting the accuracy requirement of within 0.5° in step 1.1.1. A total of 5,200 infrared images were acquired, covering four weather scenarios: sunny, rainy, foggy, and snowy. 3,200 frames were used for BVG-Net testing.

[0034] BVG-Net's region-of-interest (ROI) localization results on a 3,200-frame test set: the average center point prediction error (Euclidean distance) is 14.7 pixels, with an input resolution of 256×256. The average IoU of the ROI reaches 0.73, which is 43% higher than the IoU of approximately 0.51 achieved by traditional saliency detection methods. (DBSCAN clustering results are also mentioned.) With a resolution of 12.8 pixels and MinPts=3, the effective clustering region recognition rate is 91.4%, and the noise point filtering rate is 96.2%, indicating that BVG-Net can accurately locate the key areas that the driver is actually concerned about.

[0035] Step S1.1.3, Data processing.

[0036] Remove invalid points: Remove points whose coordinates are at the origin, points that are outside the screen range, and points with too short a duration (less than 100ms). Also remove data points with an eye-tracking confidence score below 0.7 to ensure data quality.

[0037] Fixation point extraction: Calculate the dispersion of the fixation point using the formula: Dispersion = [max(x) - min(x)] + [max(y) - min(y)], where x and y are the horizontal and vertical coordinates of the fixation point, respectively. A fixation point is considered valid when the Dispersion is less than a threshold and the duration is greater than 100ms.

[0038] Step S1.1.4: Generate a time series attention map.

[0039] Image loading: Driving scenes collected by infrared sensors are loaded as source material in time frames; gaze coordinates are mapped onto infrared image video frames in chronological order.

[0040] The fixation point is blurred to generate a continuous attention point map.

[0041] Output: Obtain the attention point map of the time series, such as... Figure 5 The diagram illustrates the cropping and standardization of attention hotspot images based on time-series eye trackers according to the present invention. Red circles in the diagram represent human eye attention points.

[0042] Step S1.2, Pre-trained model selection.

[0043] MobileNetV4, pre-trained on the ImageNet dataset, was chosen as the backbone network for feature extraction.

[0044] Step S1.3, BVG-Net model architecture design.

[0045] The BVG-Net model is constructed by using MobileNetV4 pre-trained on the ImageNet dataset as the base network, removing its original fully connected classification layers and replacing them with a regression output head specifically for locating regions of interest in vehicle-mounted infrared images. The BVG-Net model architecture includes the following modules: Figure 2 As shown in the dashed box above: (1) Eye tracker data acquisition module: collects hotspot data of human eye attention to provide training samples for model training.

[0046] (2) Unified format and size module: Adjust the input image to a fixed resolution of 256×256 pixels and perform normalization processing to normalize the pixel values ​​to the range of [0,1] to ensure the consistency of the input.

[0047] (3) Coordinate transformation: Convert different coordinate systems into a standard image coordinate system for easier subsequent processing.

[0048] (4) Pre-trained MobileNetV4 backbone network: Extracts deep feature representations of images. Using MobileNetV4-Conv-Medium configuration, the output feature map size is 8×8×960.

[0049] (5) Regression output head: The feature map is reduced to a 960-dimensional feature vector through a global average pooling layer, and then the coordinates (x, y) of the point of interest are output through a fully connected layer.

[0050] (6) Predicted output (x,y): Output the coordinates of the point of interest (x,y), where x and y are the normalized center point coordinates, ranging from [0,1].

[0051] Step 1.4: Model training.

[0052] The BVG-Net model is trained using supervised learning, with smoothed L1 loss as the loss function. Squared loss (L2) is used for smaller errors, and absolute loss (L1) is used for larger errors.

[0053] Step S2: Region of Interest Extraction and Processing.

[0054] This step extracts and processes the region of interest from the input time series image based on the BVG-Net model trained in step S1.

[0055] Step S2 uses attention coordinates as a basis to perform hotspot statistical analysis and priority ranking of the region of interest in the time series vehicle infrared images, and then crops the region of interest images and standardizes the image size.

[0056] In step S2, the hotspot statistical analysis employs a density clustering algorithm to cluster attention points and combines this with regional features to prioritize regions of interest. The specific process is as follows: Step S2.1: Input the time series image and obtain the coordinates of the area where the vehicle is located.

[0057] Input time series images: Input video frames in chronological order, feed the input images into the BVG-Net model, obtain the attention hotspot coordinates (x,y) in each frame image, and convert them into actual coordinates in the pixel coordinate system.

[0058] Step S2.2, statistical analysis of vehicle potential area coordinates.

[0059] A statistical segmentation module is introduced to perform time-series hotspot analysis on the detected regions of interest. For example... Figure 4 As shown, the red box marks the potential area of ​​the vehicle, and the multiple red circles inside the box represent the accumulated attention points in the time series.

[0060] Hotspot clustering analysis: The DBSCAN algorithm is used to cluster all attention points within the window, with the parameter set as: neighborhood radius. The minimum number of points (MinPts) is 5% of the image width, and the minimum number of points (MinPts) is 10% of the total number of points within the window. The clustering results divide the attention points into multiple clusters and noise points.

[0061] Region area ratio analysis: Calculate the ratio of the area of ​​each cluster region to the total area of ​​the image. The area of ​​a cluster region is calculated using the minimum bounding rectangle of all points within that cluster.

[0062] Regional location distribution analysis: Statistically analyze the spatial distribution characteristics of the region of interest in the image, including the mean and variance of the coordinates of the region's center point, the spatial density distribution of the region, and the degree of offset of the region relative to the image center.

[0063] Region Count Statistics: Count the number of clusters of interest detected in the image.

[0064] Region overlap calculation: Calculate the degree of overlap between multiple regions of interest using the IoU (Intersection over Union) metric.

[0065] Selecting the cropping region: Based on the time-series hotspot analysis results, multiple regions of interest are prioritized. The priority score comprehensively considers the region's area proportion, the spatial density of attention points, the duration of attention, and the region's positional weight relative to the image center. Then, the region with the highest priority score is selected.

[0066] Step S2.3: Crop the image of the area where the vehicle is located.

[0067] Based on the highest priority region determined in step S2.2, the region of interest image is cropped from the original image.

[0068] Image cropping: Based on the calculated bounding box coordinates, the region of interest is cropped from the original image. To prevent cropping the attention point, the cropped portion is enlarged by a factor of 1.2 in both length and width. It is necessary to ensure that the coordinates do not exceed the image boundaries. If the calculated bounding box exceeds the image range, boundary cropping is performed.

[0069] Step S2.4: Standardize image size.

[0070] The cropped region of interest (ROI) images are uniformly resized to a standard size of 256×256 pixels. First, the longest side of the cropped image is determined; if no longest side exists, the image's length is selected. Maintaining the aspect ratio, the longest side is enlarged by 256 pixels. Zero-padding is then applied to the enlarged image, filling only the shorter sides to achieve the 256×256 size. This uniformly sized image will then serve as input to the feature extraction module.

[0071] Step S3: Multimodal feature extraction.

[0072] This step performs multimodal feature extraction on the uniformly sized region of interest image, constructing a comprehensive feature vector. The feature extraction module is as follows: Figure 2 As shown in the dashed box below, it includes five major categories of features, totaling 65 dimensions.

[0073] Step S3 uses the image of the region of interest to fuse environmental information features accessed by the vehicle's external interface, quality and statistical features extracted from the image itself, and data-driven features extracted by a lightweight network to generate a multimodal fusion feature vector.

[0074] The environmental information features mentioned in step S3 include weather information features and time and location information features, and the quality and statistical features extracted from the image itself include image quality features and grayscale statistical features.

[0075] In step S3, the data-driven features are extracted and dimensionality-reduced by a lightweight convolutional network, and then concatenated with the remaining features to form a multimodal fusion feature vector. The specific process is as follows: Step S3.1, Weather information feature extraction, accessed through the information interface, totaling 10 dimensions.

[0076] Weather information features are accessed through an external information interface, rather than extracted from images. This interface connects to an onboard weather monitoring system to obtain real-time weather conditions.

[0077] Weather type recognition, 5-dimensional: Recognizes 5 weather types, each weather type corresponds to a value, ranging from [0,1], including: rainy day, snowy day, foggy day, cloudy day, and sunny day.

[0078] Weather intensity level, 5 dimensions: For each weather type, intensity level features are provided, including: temperature, precipitation, snowfall, visibility, and light intensity. These feature values ​​are all normalized to the range [0,1].

[0079] The update frequency is 10Hz, synchronized with the image sampling frequency. The weather information interface uses the on-site weather monitoring system data source associated with the vehicle-mounted weather system. The interface adopts a standardized data format.

[0080] Feature normalization: Normalize all weather features to the range [0,1] to form a 10-dimensional weather information feature vector.

[0081] Step S3.2, extraction of time and location information features.

[0082] Time and location information is obtained from the vehicle's GPS system and system clock via an external information interface.

[0083] (1) Northern and Southern Hemisphere characteristics, 1D: The shooting location is located in the Northern or Southern Hemisphere, used to assist in determining the season. Encoding method: 0 for the Northern Hemisphere, 1 for the Southern Hemisphere. This feature is used to determine the local season in combination with month information; December is winter in the Northern Hemisphere and summer in the Southern Hemisphere. Data source: Latitude information of GPS coordinates.

[0084] (2) Monthly features, 1D: The month at the time of shooting, from January to December, normalized to the range [0,1]. Calculation method: (month-1) / 11. Used to determine the season and the angle of sunlight. Data source: System clock.

[0085] (3) Time characteristics, 1-dimensional: The hour at the time of shooting is 0-23, normalized to the range [0,1]. Calculation method: hour / 23. Used to determine lighting conditions such as day / night, dawn / dusk. Data source: system clock.

[0086] Step S3.3, Image quality information feature extraction.

[0087] Extract various quality features from the image itself.

[0088] (1) Noise level features, 1-dimensional: The noise intensity of the image is calculated using the local variance estimation method. First, the image is filtered by median filtering to remove noise. Then, the standard deviation of the difference between the original image and the filtered image is calculated as an estimate of the noise level. The noise level is normalized to the range [0,1].

[0089] (2) Blur level feature, 1D: The image sharpness score is calculated using the Laplacian operator. A Laplacian filter is applied to the image, and the variance of the response is calculated. The larger the variance, the sharper the image; the smaller the variance, the higher the degree of blur. The sharpness score is normalized to the range [0,1].

[0090] (3) Blur type features, 2-dimensional: Blur type is determined by frequency domain analysis. The first dimension is the motion blur direction feature, which is determined by calculating the main direction of energy distribution in the spectrum and normalizing it to [0,1], corresponding to 0° to 180°. The second dimension is the weather blur / defocus blur feature, which is determined by analyzing the attenuation characteristics of the spectrum and outputting a probability value, where 0 indicates defocus blur and 1 indicates weather blur.

[0091] (4) Exposure characteristics, 1D: Calculate the average brightness and histogram distribution of the image to assess the exposure status, i.e., overexposure / underexposure / normal exposure. Calculation method: (mean...) brightness -128) / 128, normalized to the range [-1,1], where mean brightnessis the average gray value of all pixels in the image, ranging from [0, 255]; 128 is the median gray value, used to center the average brightness value, mapping the calculation result to the interval [-1, 1]. brightness When the mean is less than 128, the result is negative, indicating that the image is underexposed overall; when the mean... brightness When the mean is >128, the result is positive, indicating that the image is overexposed overall; when the mean is >128, the result is positive. brightness When the value approaches 128, the result approaches 0, indicating that the exposure is normal.

[0092] (5) Pixel contrast characteristics, 1D: Calculate the RMS (Root Mean Square) contrast value. The formula is: sqrt(mean((I-mean(I))) 2 This reflects the overall contrast level of the image. It is normalized to the range [0,1].

[0093] Step 3.4: Statistical Feature Extraction High-level statistical features of the image are extracted to describe its overall distribution characteristics. These features are extracted from the image's gray-level histogram and gray-level co-occurrence matrix. The 10 statistical features are: mean, variance, standard deviation, skewness, kurtosis, entropy, energy, smoothness, gray value, and coefficient of variation. All statistical features are normalized to the range [0,1].

[0094] Step S3.5: Data-driven feature extraction.

[0095] A lightweight MobileNetV4 network is used to extract deep features from the region of interest image to obtain data-driven features.

[0096] (1) Input the image into the lightweight MobileNetV4 network: Use the MobileNetV4-Conv-Medium model pre-trained on ImageNet, retaining all layers except the last classification layer. The input image size is 256×256×3.

[0097] (2) Extracting 512-dimensional data-driven feature vectors: Feature maps are extracted before the global average pooling layer of MobileNetV4, and 512-dimensional feature vectors are obtained through global average pooling. These 512-dimensional features contain data-driven information of the image, such as target category, scene type, and texture mode.

[0098] (3) Dimensionality reduction: Principal component analysis (PCA) was used to reduce the 512-dimensional features to 36-dimensional features, retaining principal components with a cumulative contribution rate of over 95%.

[0099] Dimensionality reduction formula: F reduced =PCA(F512 ,n components =36, variance ratio =0.95).

[0100] Among them, F reduced The reduced-dimensional eigenvectors have a dimension of 36; PCA is the principal component analysis function; F 512 The 512-dimensional raw data-driven feature vector output by the MobileNetV4 global average pooling layer; n components This indicates the number of principal components retained, meaning the target dimension after dimensionality reduction is 36; variance ratio =0.95 indicates that the retained principal components must cumulatively explain at least 95% of the variance of the original features, to ensure that most of the effective information is retained while significantly compressing the feature dimensions.

[0101] PCA dimensionality reduction process: First, the 512-dimensional features are zero-mean normalized. Then, the covariance matrix is ​​calculated, the eigenvalues ​​and eigenvectors are solved, and the eigenvectors corresponding to the top 36 largest eigenvalues ​​are selected. The original features are then projected onto these 36 principal components.

[0102] Step S3.6: Generation of multimodal total fusion features.

[0103] All features extracted in steps S3.1 to S3.5 are concatenated and fused to form a 65-dimensional multimodal fusion feature vector: Weather information features (10 dimensions, accessed via interface); time and location information features (3 dimensions, accessed via interface); image information features (6 dimensions, extracted from image); statistical features (10 dimensions, extracted from image); data-driven features (36 dimensions after dimensionality reduction, extracted from image); total: 10 + 3 + 6 + 10 + 36 = 65 dimensions. The feature fusion method uses simple concatenation, combining the features of each modality in sequence into a 65-dimensional feature vector: F multi =[F weather (10),F time _ location (3),F image _ quality (6),F statistics (10),F semantic (36)]. Among them, F weather (10) represents weather information features, 10-dimensional; F time _ location (3) Time and location information features, 3-dimensional; F image _ quality (6) represents image quality features, 6-dimensional; F statistics(10) represents gray-level statistical features, 10 dimensions; F semantic (36) represents data-driven semantic features, which are 36-dimensional.

[0104] The fused feature vectors will be used as input to the TIEFormer parameter prediction model.

[0105] Step S4: Construction and training of the TIEFormer parameter prediction model.

[0106] This step constructs the image enhancement parameter prediction model TIEFormer (ThermalImageEnhancementTransformer), which automatically predicts the optimal image processing parameters based on 65-dimensional multimodal fusion features.

[0107] Step S4 normalizes the multimodal fusion feature vector, inputs the normalized features into a trained infrared image enhancement parameter prediction model, and outputs five key image enhancement parameters: contrast, brightness, gamma correction, sharpening intensity, and spatial noise reduction.

[0108] In step S4, the infrared image enhancement parameter prediction model adopts a multi-head self-attention encoder structure and outputs five enhancement parameters through a fully connected network.

[0109] In step S4, the five key image enhancement parameters are all adaptive prediction values, and they correspond to preset numerical adjustment ranges.

[0110] In step S4, the model is trained using supervised learning, and a loss function and an early stopping mechanism to prevent overfitting are configured. The specific process is as follows: Step S4.1, normalization processing.

[0111] Normalization was performed on the 65-dimensional multimodal fusion features: normalized feature =(feature-mean) / std; Where mean and std are the mean and standard deviation of the features in the training set, respectively. Feature refers to the 65-dimensional multimodal fusion feature vector F generated in step S3.6. multi This refers to a complete 65-dimensional feature vector composed of weather information features, time and location information features, image quality features, grayscale statistical features, and data-driven semantic features. Normalization aims to eliminate dimensional differences between different features, accelerate model convergence, and improve training stability.

[0112] The normalization parameters mean and std are calculated from the training set during the training phase and stored thereafter. The same normalization parameters are used to standardize the input features during the inference phase.

[0113] Step S4.2, TIEFormer model construction.

[0114] The TIEFormer architecture fully leverages self-attention mechanisms to capture global dependencies between features. The model architecture is as follows: Figure 6 The diagram shown is an overall framework diagram of the TIEFormer infrared image parameter enhancement model of this invention. It includes the following modules: (1) Input layer, 65-dimensional: receives 65-dimensional normalized multimodal fusion features.

[0115] (2) Feature embedding layer, linear transformation to 128 dimensions: The 65-dimensional features are mapped to the higher-dimensional embedding space of 128 dimensions through linear transformation.

[0116] Embedding formula: Embedding=Linear(F multi ,output dim =128)+PositionalEncoding; Here, `Embedding` is a 128-dimensional embedding vector obtained by superimposing linear mapping and positional encoding, serving as the input to the TIEFormer encoder; `Linear` is a fully connected linear transformation layer that maps the input features to a specified dimension; F... multi The 65-dimensional normalized multimodal fusion feature vector generated in step S3.6 includes five categories of features: weather, time and location, image quality, statistics, and data-driven semantics; output dim =128 indicates that the output dimension of the linear transformation is 128; PositionalEncoding is the position encoding vector with a dimension of 128, which is used to introduce position information into the unordered feature vector, enabling the model to perceive the relative relationship between the sub-dimensions of each feature.

[0117] (3) Encoder Layer: The encoder is the core module of TIEFormer. Each layer contains the following sub-modules: Eight-head self-attention mechanism: Eight attention heads compute in parallel, each focusing on a different feature subspace.

[0118] Feedforward Neural Network, FFN dimension 2048: FFN contains two linear layers and a ReLU activation function: FFN(x)=Linear2(ReLU(Linear1(x))), where Linear1 maps 128 dimensions to 2048 dimensions, and Linear2 maps 2048 dimensions back to 128 dimensions.

[0119] Residual connections and layer normalization: Output = LayerNorm(x + Sublayer(x)). Four encoder layers are stacked, each with a hidden dimension of 512, and the feedforward network has a dimension of 2048.

[0120] Where Output is the output vector of the current sublayer (self-attention layer or feedforward network layer) after residual connection and layer normalization; LayerNorm is the layer normalization operation, which normalizes the input vector to a mean of 0 and a variance of 1 in the feature dimension, to stabilize the training process and accelerate convergence; x is the input vector of the current sublayer; Sublayer(x) is the output of the current sublayer after transforming the input x; x+Sublayer(x) is the residual connection, which adds the sublayer input and sublayer output, effectively alleviating the gradient vanishing problem in deep networks and preserving the original feature information.

[0121] (4) Fully connected output layer, from 128-dimensional to 64-dimensional to 32-dimensional to 5-dimensional: Through a multi-layer fully connected network, the encoder output is mapped to 5 image processing parameters. Specific structure: First fully connected layer: 128-dimensional to 64-dimensional, activation function ReLU, Dropout(0.3); Second fully connected layer: 64-dimensional to 32-dimensional, activation function ReLU, Dropout(0.3); Fully connected output layer: 32-dimensional to 5-dimensional image processing parameters, with different activation functions used for different parameters.

[0122] Step S4.3, Output parameter design.

[0123] The TIEFormer model outputs five image processing parameters, such as... Figure 2 As shown: (1) Contrast parameter, range [0.5, 2.0]: Controls the intensity of image contrast adjustment. Activation function: Sigmoid. Scaling formula: contrast param =0.5+1.5×sigmoid(output1).

[0124] (2) Brightness parameter, range [0.5, 2.0]: Controls the intensity of image brightness adjustment. Activation function: Sigmoid. Scaling formula: brightness param =0.5+1.5×sigmoid(output2).

[0125] (3) Gamma correction parameters, range [0.1, 2.5]: used for non-linear brightness adjustment, improving the contrast distribution and detail of infrared images by adjusting the gamma curve. Activation function: Sigmoid; scaling formula: gamma. param=0.1+2.4×sigmoid(output3).

[0126] (4) Sharpening intensity, range [0, 2.0]: Controls the intensity of image sharpening. Activation function: ReLU. Crop formula: sharpen strength =clip(relu(output4),0,2.0).

[0127] (5) Spatial denoising adjustment coefficient, range [0, 1.0]: controls the smoothing intensity of the image spatial filter, used to preserve target edge information as much as possible while suppressing thermal imaging background noise. Activation function: Sigmoid; scaling formula: denoise. param =sigmoid(output5).

[0128] Step S4.4, Model training.

[0129] The TIEFormer parameter prediction model is trained using supervised learning. It takes 65-dimensional normalized multimodal fusion features as input and outputs 5-dimensional image processing parameters.

[0130] Early stopping mechanism: Training is stopped when the validation set loss does not decrease for 20 consecutive epochs to prevent overfitting. Training data preparation: The training data consists of image-parameter pairs. The images are the regions of interest images processed in steps 1-3 and their 65-dimensional multimodal features. The parameters are the optimal image processing parameters annotated by experts. Expert annotation process: Image processing experts manually adjust five processing parameters according to the type and degree of image degradation to achieve the best visual effect, and record the optimal parameters as the ground truth.

[0131] The TIEFormer training set was manually labeled by image processing experts, containing 3,000 image-parameter pairs: 800 for sunny days, 750 for rainy days, 750 for foggy days, and 700 for snowy days. These were divided into a training set of 2,400 pairs, a validation set of 300 pairs, and a test set of 300 pairs, in an 8:1:1 ratio. The Adam optimizer was used with an initial learning rate of 1×10⁻⁶. -4 The batch size was 32, and the early stopping threshold was 20 epochs. The model converged on the 48th epoch, and the combined loss on the validation set decreased to 0.0312.

[0132] Prediction accuracy of five image processing parameters on the test set. Mean absolute error (MAE) and relative error: Contrast parameter MAE=0.078, parameter range [0.5, 2.0], relative error 5.2%; Brightness parameter MAE=0.071, parameter range [0.5, 2.0], relative error 4.7%; Gamma correction parameter MAE=0.118, parameter range [0.1, 2.5], relative error 4.9%; Sharpening intensity MAE=0.094, parameter range [0, 2.0], relative error 4.7%; Spatial noise reduction coefficient MAE=0.046, parameter range [0, 1.0], relative error 4.6%. The average relative error of the five parameters is 4.82%, all below 5.5%, indicating that TIEFormer can predict various image enhancement parameters with high accuracy.

[0133] Step S5, image processing.

[0134] This step adaptively processes the region of interest image based on the five parameters predicted by TIEFormer, generating an enhanced image. This step is part of the image processing module, such as... Figure 2 As shown.

[0135] In step S5, the five key image enhancement parameters are used to sequentially perform contrast adjustment, brightness adjustment, gamma correction, sharpening, and spatial noise reduction on the region of interest image.

[0136] In step S5, the contrast adjustment employs adaptive histogram equalization, and the spatial noise reduction uses edge-preserving denoising filtering. The specific process is as follows: Step S5.1, Contrast Adjustment.

[0137] Contrast adjustment is achieved by combining adaptive histogram equalization with predicted contrast parameters. The image is divided into multiple 8×8 blocks, and histogram equalization is performed on each block while limiting the magnitude of contrast enhancement to avoid over-enhancement.

[0138] Step S5.2, brightness adjustment.

[0139] Convert the RGB image to an HSV image and extract the V channel. new =clip(V×brightness param The adjusted V channel (0, 255) is merged with the H and S channels and converted back to an RGB image.

[0140] Among them, V new`brightness` represents the pixel value of the V channel after brightness adjustment; `clip` is a truncation function that limits the calculation result to the legal pixel value range [0, 255] to prevent overflow; `V` is the pixel value of the original HSV image's brightness channel (Value channel); param The brightness adjustment parameter is predicted by the TIEFormer model, with a value range of [0.5, 2.0]: when brightness param When the brightness is less than 1.0, the overall image brightness decreases; when the brightness is less than 1.0, the overall image brightness decreases. param When the brightness is >1.0, the overall image brightness is improved; when the brightness is >1.0, the overall image brightness is improved. param When the value is 1.0, the brightness remains unchanged.

[0141] Step S5.3, Gamma correction adjustment.

[0142] First, the image pixel values ​​are normalized to the [0,1] interval. Then, the predicted gamma coefficients are used for exponentiation. The result is mapped back to [0,255] and truncated. Formula: I new =clip(255×(I / 255) γ (0, 255). When γ < 1, it stretches low grayscale areas, i.e., dark areas, making the overall image brighter and helping to observe the low-temperature background. When γ > 1, it stretches high grayscale areas, i.e., bright areas, increasing the contrast of details inside high-temperature targets.

[0143] Step S5.4, sharpening.

[0144] Apply Gaussian blur to the image, kernel size =5, sigma=1.0, resulting in a blurred image I. blur Calculate high-frequency details: detail=II blur Sharpen the image: I sharp =I+sharpen strength ×detail.

[0145] Among them, kernel size =5 is the size of the Gaussian blur convolution kernel, i.e., a 5×5 pixel Gaussian filter window; sigma=1.0 is the standard deviation of the Gaussian distribution, which controls the degree of blur; the larger the value, the stronger the blur effect; I blur This is the smoothed image obtained after applying Gaussian blur to the original image, used to extract low-frequency components; detail represents the high-frequency detail components of the image, i.e., the original image I and the blurred image I. blur The difference includes high-frequency information such as edges and textures; I sharp The output image is the sharpened version; I represents the input region of interest image; sharpen strengthThe sharpening intensity parameter is predicted by the TIEFormer model, and its value ranges from [0, 2.0]. strength =0 indicates no sharpening, while a larger value results in a more significant sharpening effect.

[0146] Step S5.5, spatial noise reduction processing.

[0147] Using edge-preserving denoising methods such as bilateral filtering or nonlocal mean filtering, the filtering intensity is determined by the denoise. param Parameter control.

[0148] Among them, denoise param The spatial denoising adjustment coefficients predicted by the TIEFormer model range from [0, 1.0] and are obtained by mapping using the sigmoid function; this parameter controls the smoothing intensity of the edge-preserving denoising filter: denoise param The closer the value is to 1.0, the stronger the filtering smoothing effect and the better the noise suppression effect; denoise param The closer it is to 0, the weaker the filtering strength, and the more fully the image details and edge information are preserved; in actual filtering, this parameter will be mapped to the intensity hyperparameter of the corresponding filter.

[0149] Step S6, output processing.

[0150] Step S6, based on the processing results, outputs a human-enhanced vehicle-mounted infrared region of interest image. The specific content is as follows: The output image is a region of interest image after contrast, brightness, gamma correction, sharpening, and spatial noise reduction adjustments. The final output image has been refined to enhance the region of interest, achieving a better visual effect.

[0151] Step S7: Target detection task and system real-time performance verification.

[0152] In step S7, the anthropomorphically enhanced vehicle-mounted infrared region of interest image is input into the vehicle-mounted target detection model to perform the vehicle-mounted target detection task, verifying the image enhancement effect and the system's real-time processing performance.

[0153] In step S7, the system's real-time processing performance meets the real-time frame rate requirements for vehicle deployment. The specific process is as follows: To verify the contribution of image augmentation to practical perception tasks, the YOLOv8-n detection model was used to perform object detection on the baseline and augmented images of each method, with an evaluation metric of mAP@0.5. The test set included three types of targets: pedestrians, vehicles, and obstacles, with a total of 1,860 labeled instances, covering four weather scenarios.

[0154] Target detection mAP@0.5 results: Baseline is 0.631, after enhancement by VCAFusion it is 0.698, after enhancement by AEFusion it is 0.693, and after enhancement by this invention it is 0.752. This invention improves upon the baseline by 19.2% and compared to the best comparison method, VCAFusion, by 7.7%, fully validating the application value of this invention in real-world vehicle environment perception tasks.

[0155] On a test platform with an Intel Core i7-12700H processor, 16GB of RAM, and an NVIDIA RTX 3060 laptop GPU with 6GB of VRAM, the average processing time for each stage of 100 consecutive frames was statistically analyzed: BVG-Net region of interest localization inference 7.8ms; multimodal feature extraction (statistical features, image quality features, and MobileNetV4 inference) 11.4ms; TIEFormer parameter prediction inference 2.9ms; image processing module (contrast, brightness, gamma, sharpening, and noise reduction) 4.2ms; total single-frame processing time 26.3ms, corresponding to approximately 38fps, which meets the 10Hz real-time requirement of this invention and is suitable for actual vehicle deployment.

[0156] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0157] The above embodiments are merely illustrative examples and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A vehicle-mounted infrared image enhancement method guided by visual information and driven by data, characterized in that, The method includes the following steps: Step S1: Based on the driver's eye attention hotspot data collected by the eye tracker, construct and train a biomimetic visual guidance region of interest localization model, and output the region of interest attention coordinates of the vehicle infrared image through the region of interest localization model. Step S2: Based on the region of interest localization model trained in step S1, extract and process the region of interest in the input time series image. Based on the attention coordinates, perform hotspot statistical analysis and priority ranking of the region of interest in the time series vehicle infrared image, crop the region of interest image and unify the image size. Step S3: Perform multimodal feature extraction on the region of interest image after unifying the size, and use the region of interest image to fuse environmental information features accessed by the vehicle external interface, quality and statistical features extracted from the image itself, and data-driven features extracted by a lightweight network to generate a multimodal fusion feature vector; Step S4: Construct an image enhancement parameter prediction model, normalize the multimodal fusion feature vector, input the normalized feature into the trained image enhancement parameter prediction model, and output five key image enhancement parameters: contrast, brightness, gamma correction, sharpening intensity, and spatial noise reduction. Step S5: Adaptive processing is performed on the region of interest image using the five key image enhancement parameters. Contrast adjustment, brightness adjustment, gamma correction, sharpening and spatial noise reduction are sequentially performed on the region of interest image using the five key image enhancement parameters to generate an enhanced image. Step S6: Based on the processing results, output the anthropomorphic enhanced vehicle infrared region of interest image; Step S7: Input the anthropomorphically enhanced vehicle-mounted infrared region of interest image into the vehicle-mounted target detection model to perform the vehicle-mounted target detection task and verify the image enhancement effect and the system's real-time processing performance.

2. The vehicle-mounted infrared image enhancement method guided by visual information and driven by data according to claim 1, characterized in that, The attention hotspot data collection in step S1 includes multi-driver, multi-scene collection, invalid data removal, and time-series attention point map generation. The region of interest localization model is based on MobileNetV4 pre-trained on the ImageNet dataset. Its original fully connected classification layer is removed and replaced with a regression output head specifically for locating regions of interest in vehicle infrared images. The regression output head reduces the feature map to a 960-dimensional feature vector through a global average pooling layer, and then outputs the normalized coordinates (x, y) of the point of interest through a fully connected layer.

3. The vehicle-mounted infrared image enhancement method based on visual information guidance and data-driven approach according to claim 1, characterized in that, In step S2, the hotspot statistical analysis uses a density clustering algorithm to cluster attention points and combines regional features to prioritize regions of interest. The clustering parameter is set to: neighborhood radius. The minimum number of points (MinPts) is 5% of the image width, and the minimum number of points (MinPts) is 10% of the total number of points in the window. The priority ranking score takes into account the region area ratio, the spatial density of attention points, the attention duration, and the positional weight of the region relative to the image center.

4. The vehicle-mounted infrared image enhancement method guided by visual information and driven by data according to claim 1, characterized in that, The multimodal fusion feature vector generated in step S3 specifically includes: The features are: 10-dimensional weather information features accessed via the interface; 3-dimensional time and location information features accessed via the interface; 6-dimensional image quality features extracted from the image; 10-dimensional grayscale statistical features extracted from the image; 36-dimensional data-driven semantic features extracted from the image and reduced by principal component analysis (PCA); and the features of each modality are fused using a simple splicing method.

5. The vehicle-mounted infrared image enhancement method based on visual information guidance and data-driven approach according to claim 4, characterized in that, In step S4, the image enhancement parameter prediction model adopts a multi-head self-attention encoder structure, including: an input layer receiving 65-dimensional normalized multimodal fusion features; mapping the 65-dimensional features to a 128-dimensional embedding space through linear transformation; an encoder layer containing an eight-head self-attention mechanism, a feedforward neural network FFN with a dimension of 2048, residual connections, and layer normalization, stacking four encoder layers; and a fully connected output layer sequentially mapping the 128-dimensional features to 64-dimensional and 32-dimensional features, finally outputting 5-dimensional image processing parameters.

6. The vehicle-mounted infrared image enhancement method guided by visual information and driven by data according to claim 1, characterized in that, In step S4, the five key image enhancement parameters all correspond to preset numerical adjustment ranges, specifically: (1) Contrast parameter, range [0.5, 2.0]: controls the intensity of image contrast adjustment, scaling formula: contrast param =0.5+1.5×sigmoid(output1); (2) Brightness parameter, range [0.5, 2.0]: controls the intensity of image brightness adjustment, scaling formula: brightness param =0.5 + 1.5 × sigmoid(output2); (3) Gamma correction parameters, range [0.1, 2.5]: used for non-linear brightness adjustment, improving the contrast distribution and detail of infrared images by adjusting the gamma curve. Scaling formula gamma param =0.1+2.4×sigmoid(output3); (4) Sharpening intensity, range [0, 2.0]: Controls the intensity of image sharpening. Cropping formula: sharpen strength =clip(relu(output4),0,2.0); (5) Spatial noise reduction adjustment coefficient, range [0, 1.0]: controls the smoothing intensity of the image spatial filter, used to preserve target edge information as much as possible while suppressing thermal imaging background noise. Scaling formula denoise param =sigmoid(output5).

7. The vehicle-mounted infrared image enhancement method guided by visual information and driven by data according to claim 1, characterized in that, In step S5, the contrast adjustment uses adaptive histogram equalization, which divides the image into multiple small blocks, performs histogram equalization on each block, and limits the contrast enhancement. The brightness adjustment is achieved by converting the RGB image to an HSV image and then adjusting the V channel. The gamma correction uses power operation for non-linear brightness adjustment. The sharpening process is achieved by calculating high-frequency details and superimposing them on the original image. The spatial noise reduction uses edge-preserving noise filtering.

8. The vehicle-mounted infrared image enhancement method based on visual information guidance and data-driven approach according to claim 1, characterized in that, In step S7, the system's real-time processing performance meets the real-time frame rate requirements for vehicle deployment, and the total processing time for a single frame does not exceed 26.3ms.

9. The vehicle-mounted infrared image enhancement method based on visual information guidance and data-driven approach according to claim 1, characterized in that, In step S2, when cropping the region of interest image, the bounding box coordinates determined by priority sorting are cropped from the original image. To prevent cropping the attention point, the bounding box of the cropped part is enlarged by 1.2 times in both length and width, and it is ensured that the coordinates of the enlarged bounding box do not exceed the image boundary. If the calculated bounding box exceeds the image range, boundary cropping is performed.

10. A vehicle-mounted infrared image enhancement method guided by visual information and driven by data according to claim 4, characterized in that, The 6-dimensional image quality features in step S5 specifically include: The 1D noise level feature uses a local variance estimation method. First, the image is filtered by median filtering to remove noise. Then, the standard deviation of the difference between the original image and the filtered image is calculated as the noise level estimate and normalized to the range of 0, 10, 1. The image sharpness score is calculated using the Laplacian operator, a Laplacian filter is applied to the image and the variance of the response is calculated, and the sharpness score is normalized to the range of 0, 10, 1. Two-dimensional fuzzy type features are used to determine the fuzzy type through frequency domain analysis. The first dimension is the motion fuzzy direction feature, which is determined by calculating the main direction of energy distribution in the spectrum to determine the motion fuzzy direction angle, normalized to 0, 1 corresponds to 0° to 180°. The second dimension is the weather fuzzy or defocused fuzzy feature, which is determined by analyzing the attenuation characteristics of the spectrum to determine the fuzzy type and output a probability value, where 0 indicates defocused fuzziness and 1 indicates weather fuzziness. One-dimensional exposure feature: calculates the average brightness and histogram distribution of the image, normalizes to the range of -1 to 1, negative values ​​indicate underexposure, positive values ​​indicate overexposure, and values ​​close to 0 indicate normal exposure; One-dimensional pixel contrast characteristics are used to calculate the RMS contrast value and normalize it to the range of 0 and 1.

Images