A vehicle blind spot dynamic enhancement and warning system for low-light environments

Abstract

This invention discloses a vehicle blind spot dynamic enhancement and warning system for low-light environments, relating to the field of automotive active safety technology. It includes a short-wave infrared imaging module for capturing passive thermal feature images of the scene to the side and rear of the vehicle; a multi-source signal acquisition module for acquiring vehicle bus signals characterizing the vehicle's motion state and the driver's operational intentions in real time; a data processing and risk assessment module, connected to the short-wave infrared imaging module and the multi-source signal acquisition module, for fusing the thermal feature images and the vehicle bus signals to identify blind spot targets and calculate their dynamic risk levels; and a graded warning and intervention module, connected to the data processing and risk assessment module. This invention fundamentally overcomes the imaging challenges in extremely low light conditions by employing short-wave infrared imaging technology that is sensitive to nighttime light and has strong anti-interference capabilities, achieving stable detection of vehicles and pedestrians at long distances in completely dark environments.

Description

Technical Field

[0001] This invention relates to the field of automotive active safety technology, and in particular to a vehicle blind spot dynamic enhancement and warning system for low-light environments. Background Technology

[0002] When driving at night, at dusk, or in tunnels, drivers' visual perception is significantly reduced, making blind spots to the sides and rear of the vehicle particularly prominent and greatly increasing safety risks when changing lanes. To address this challenge, existing vehicle blind spot monitoring systems mainly rely on traditional visible light cameras, millimeter-wave radar, or near-infrared technology. However, these technologies all have significant limitations; traditional visible light cameras and streaming rearview mirrors fail or generate a lot of noise in extremely low light conditions due to insufficient light intake; millimeter-wave radar, while capable of detecting objects, cannot provide intuitive image information, resulting in drivers lacking a direct perception of blind spot hazards; and near-infrared solutions based on active illumination have inherent drawbacks such as short effective range and severe attenuation in adverse weather conditions like rain and fog.

[0003] On the one hand, in terms of early warning logic, the existing system's prediction mechanism often relies on the single dimension of turn signal. This judgment is lagging and passive because when the turn signal is on, the lane-changing action has often already begun or is about to begin, leaving the system with a very short window to identify risks and issue warnings, making it impossible to achieve effective early warning. More importantly, in a completely dark environment, drivers may not actively turn on the turn signal due to safety concerns because they cannot see the blind spot. This makes the prediction mechanism that relies on the turn signal completely ineffective.

[0004] On the other hand, existing system warning strategies are often relatively simple, usually only issuing a general alarm after an object is detected. They cannot effectively distinguish risk levels, nor can they be deeply coupled with the driver's real-time operating intentions. This may lead to frequent false alarms, which may interfere with the driver and reduce their trust in the system.

[0005] Therefore, the industry urgently needs a comprehensive solution that can penetrate darkness and severe weather conditions without passive input, provide clear blind spot images, and intelligently understand driver intentions to provide advanced and accurate risk warnings. Summary of the Invention

[0006] The main objective of this invention is to provide a vehicle blind spot dynamic enhancement and warning system for low-light environments, in order to solve the problems mentioned in the background.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is: a dynamic enhancement and warning system for vehicle blind spots in low-light environments, comprising:

[0008] A short-wave infrared imaging module is used to capture passive thermal feature images of the scene behind and to the side of the vehicle.

[0009] The multi-source signal acquisition module is used to acquire vehicle bus signals that represent the vehicle's motion state and the driver's operating intentions in real time.

[0010] The data processing and risk assessment module is connected to the shortwave infrared imaging module and the multi-source signal acquisition module. It is used to fuse the thermal feature image and the vehicle bus signal to identify blind spot targets and calculate their dynamic risk level.

[0011] The graded warning and intervention module is connected to the data processing and risk assessment module and is used to trigger different levels of warnings or vehicle control commands based on the dynamic risk level.

[0012] Preferably, the shortwave infrared imaging module includes:

[0013] The main SWIR imaging unit, using an InGaAs sensor, is mounted below the vehicle's exterior rearview mirror base at a pitch angle of 30°±5°.

[0014] The auxiliary sensing unit includes low-light cameras and safety laser lights mounted on both sides of the rear bumper.

[0015] Preferably, the signals acquired by the multi-source signal acquisition module include:

[0016] Torque value and its rate of change from the steering wheel torque sensor;

[0017] The status of the turn signal switch from the body control module;

[0018] Wheel angle signal from ESC or EPS system;

[0019] Opening signals from the accelerator and brake pedals;

[0020] The vehicle speed signal from the wheel speed sensor.

[0021] Preferably, the data processing and risk assessment module includes:

[0022] An image preprocessing unit is used to perform spatial and temporal filtering on the thermal feature image to improve the signal-to-noise ratio;

[0023] The target extraction and tracking unit is used to cluster moving targets from the filtered image and establish a tracking trajectory for each target;

[0024] The multi-feature fusion computing unit is used to calculate the relative distance, relative velocity and direction of motion of the target based on the target's tracking trajectory, and to fuse the target's thermodynamic and morphological features to generate an initial risk score;

[0025] The intent recognition and risk coupling unit is used to calculate the driver's lane-changing intent confidence level based on the vehicle bus signal, and couple the confidence level with the initial risk score to output the final dynamic risk level.

[0026] Preferably, the execution logic of the image preprocessing unit includes:

[0027] The spatial domain filtering processing subunit uses an adaptive bilateral filtering algorithm that incorporates a thermal gradient weighting factor to process a single frame of SWIR image. The specific steps include:

[0028] Calculate the thermal gradient value H1 between each pixel in the image and its neighboring pixels;

[0029] Set a predetermined thermal gradient threshold H0;

[0030] When the thermal gradient value H1 is less than the predetermined thermal gradient threshold H0, it is determined to be a low thermal gradient region, and the intensity variance is increased. To enhance smoothing and noise reduction, and increase range variance. ;

[0031] When the thermal gradient value H1 is greater than or equal to the predetermined thermal gradient threshold H0, it is determined to be a high thermal gradient region, and the intensity variance is reduced. To preserve edge features and reduce range variance ;

[0032] Perform bilateral filtering calculations using the adjusted parameters to output a spatially enhanced image;

[0033] The temporal filtering subunit performs pixel-level cumulative difference operations on a sequence of N consecutive spatially enhanced image frames to generate a mask of salient moving targets. The specific steps include:

[0034] Perform a difference operation between each frame of the sequence and its previous frame to obtain a binarized difference image;

[0035] Perform a logical OR operation on the binary difference maps of N consecutive frames to generate a cumulative motion region map;

[0036] A morphological opening operation is performed on the accumulated motion region map to eliminate isolated pixels caused by noise and smooth the contour of the moving target.

[0037] The processed cumulative motion region map is used as the final moving target mask output for subsequent target extraction and tracking.

[0038] Preferably, the execution logic of the target extraction and tracking unit includes:

[0039] Foreground target extraction subunit: Performs a logical AND operation between the moving target mask generated by temporal filtering and the enhanced image after spatial filtering to extract the foreground image containing pixels of moving heat source;

[0040] Dynamic parameter clustering subunit: The improved DBSCAN clustering algorithm is used to cluster the pixels in the foreground image. The improvements include:

[0041] Based on the pixel coordinates of the target in the image, combined with camera calibration parameters, the physical size of the target in the real world is estimated;

[0042] Establish a mapping function to map the physical dimensions to the clustering radius parameter Eps of the DBSCAN algorithm.

[0043] in:

[0044] When the physical size C ≥ the first threshold When, the mapping function outputs Threshold;

[0045] When the second threshold ≤Physical size C < First threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold;

[0046] When physical size C < second threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold;

[0047] The minimum number of points (MinPts) required for core point determination is dynamically adjusted based on the vehicle speed. The higher the vehicle speed, the lower the MinPts value, in order to adapt to the sparser target pixels in high-speed scenes.

[0048] Target trajectory initialization and tracking subunit: Initialize a tracker for each successfully clustered independent target:

[0049] A uniform or uniformly accelerated motion model is established for each target using a Kalman filter, and its state vector contains at least the target's position and velocity in the image coordinate system.

[0050] Calculate the feature similarity between the new target and the existing tracked targets. The features include the target's heat intensity histogram and the aspect ratio of its bounding rectangle. Then, use the Hungarian algorithm to perform optimal matching and association.

[0051] Each target is assigned a unique tracking ID and its tracking status is managed. Newly detected but unmatched targets are initialized as transient targets and are only promoted to stable targets and output to the risk assessment module after being tracked for M consecutive frames. For lost tracked targets, a delayed destruction mechanism is activated, and tracking is terminated after the target fails to match a detected target for N consecutive frames.

[0052] Preferably, the formula for calculating the initial risk score by the multi-feature fusion calculation unit is:

[0053] ;

[0054] In the formula, For the initial risk score; , and These are configurable weighting coefficients; The target relative velocity; The relative distance to the target; These are the type weight coefficients derived from target thermal image classification.

[0055] Preferably, the execution logic of the intent recognition and risk coupling unit includes:

[0056] Intent confidence calculation subunit: A finite state machine model is established, whose input is the vehicle bus signal, and whose output is an intent confidence level between 0 and 1. Its state transition logic includes:

[0057] Initial state: When the absolute value of the steering wheel torque |T| < threshold Furthermore, when there is no turn signal, the confidence level is 0;

[0058] Intended state formation: If the detected steering wheel torque T lasts for Δt longer than the threshold value. But below the higher threshold If the torque change rate dT / dt > 0, the confidence level increases linearly from 0; if a turn signal is detected at the same time, the confidence level jumps to 0.8.

[0059] Lane change execution state: If steering wheel torque T > threshold If the wheel angle is detected to be continuously increasing, the confidence level is immediately set to 1 regardless of the turn signal status.

[0060] Intended cancellation state: In any state, if the brake pedal signal or torque value is detected to drop back to within Δt... The state machine is then reset to its initial state, and the confidence level is set to zero.

[0061] Dynamic risk calculation subunit: Calculates the effective risk score using the following formula:

[0062] ;

[0063] In the formula, For effective risk scoring; The initial risk score; α is the amplification factor; For the confidence level of the intention;

[0064] Dynamic threshold adjustment subunit: The distance threshold L1 for Level 1 warning is dynamically adjusted based on the vehicle speed V. ;

[0065] In the formula, As the reference distance, The reference speed is β, which is an adjustment coefficient.

[0066] Preferably, the graded warning and intervention module includes:

[0067] The Level 1 Alert Unit is used to display a yellow outline of the target via AR-HUD when the effective risk score exceeds the first threshold.

[0068] The secondary warning unit is used to trigger a red flashing in the rearview mirror area and a pulse vibration in the driver's seat when the effective risk score exceeds a higher second threshold and the intent confidence is greater than a set threshold.

[0069] The three-level intervention unit is used to apply a reverse torque through the electric power steering system when the effective risk score exceeds the highest third threshold and a change in wheel angle is detected. The torque value is between 0.2 Nm and 0.5 Nm.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] This invention fundamentally overcomes the imaging challenges under extremely low illumination by employing short-wave infrared imaging technology that is sensitive to nighttime light and has strong anti-interference capabilities. It achieves stable detection of vehicles and pedestrians at long distances in completely dark environments, with a range and image quality far exceeding traditional optical solutions, providing a high-quality image foundation for subsequent processing. At the information processing level, through a dual innovative filtering algorithm combining spatial and temporal domains, it successfully suppresses sensor noise while perfectly preserving key target edge features, significantly improving the image signal-to-noise ratio in low-signal-to-noise environments. Furthermore, by introducing size- and speed-adaptive dynamic target clustering and a multi-target tracking management mechanism incorporating thermal features, it effectively solves the challenges of detecting and stably tracking targets of varying sizes and distances, significantly reducing target fragmentation and trajectory breakage, thus laying a solid foundation for accurate risk assessment.

[0072] By constructing a sophisticated finite state machine model to interpret the micro-operation signals of the steering wheel, the system achieves early and accurate prediction of the driver's lane-changing intentions. It dynamically couples the confidence level of this intention with the objective risk of the target for calculation. Compared with mechanical reactive warnings, this system uses a graded warning strategy that combines AR-HUD visual enhancement, tactile vibration, and steering intervention to deliver risk information to the driver in the most intuitive and least disruptive way. This not only provides sufficient warning time before danger occurs but also greatly improves the human-machine interaction experience, avoiding driver fright and annoyance, thereby comprehensively improving the safety and driving confidence of lane-changing operations in dark environments.

[0073] In summary, this invention forms a complete security closed loop through technological innovation across the entire chain of perception, processing, decision-making, and alerting. Attached Figure Description

[0074] Figure 1 This is a flowchart of a vehicle blind spot dynamic enhancement and warning system for low-light environments according to the present invention.

[0075] Figure 2 This is a flowchart of the data processing and risk assessment module of a vehicle blind spot dynamic enhancement and warning system for low-light environments according to the present invention;

[0076] Figure 3 This is a flowchart of the image preprocessing unit of a vehicle blind spot dynamic enhancement and warning system for low-light environments according to the present invention;

[0077] Figure 4 This is a flowchart of the target extraction and tracking unit of a vehicle blind spot dynamic enhancement and warning system for low-light environments according to the present invention;

[0078] Figure 5 This is a flowchart of the intent recognition and risk coupling unit of a vehicle blind spot dynamic enhancement and warning system for low-light environments according to the present invention. Detailed Implementation

[0079] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] Please see Figures 1 to 5 This invention provides a technical solution: a dynamic enhancement and warning system for vehicle blind spots in low-light environments, comprising:

[0081] A short-wave infrared imaging module is used to capture passive thermal feature images of the side and rear scenes of a vehicle in the 1.0-1.7μm band.

[0082] The multi-source signal acquisition module is used to acquire vehicle bus signals that represent the vehicle's motion state and the driver's operating intentions in real time.

[0083] The data processing and risk assessment module is connected to the shortwave infrared imaging module and the multi-source signal acquisition module. It is used to fuse the thermal feature image and the vehicle bus signal to identify blind spot targets and calculate their dynamic risk level.

[0084] The graded warning and intervention module is connected to the data processing and risk assessment module and is used to trigger different levels of warnings or vehicle control commands based on the dynamic risk level.

[0085] In a further embodiment, the shortwave infrared imaging module includes:

[0086] The main SWIR imaging unit uses an InGaAs sensor with a resolution of no less than 1280×960 and a frame rate of no less than 30fps. It is mounted below the vehicle's exterior rearview mirror base at a pitch angle of 30°±5°.

[0087] The auxiliary sensing unit includes low-light cameras and / or 1550nm wavelength safety laser fillers mounted on both sides of the rear bumper.

[0088] The system works by fully utilizing the ability of short-wave infrared technology to perceive nighttime light. The main SWIR imaging unit enables passive imaging in dark environments. Infrared imaging is superior at night; a direct backlight from a source could impair following vehicles and create safety hazards. Furthermore, the performance degradation of the backlight-free solution in rain and fog is significantly lower than that of the source-lit solution, greatly increasing reliability. It also facilitates ultra-long-range detection, allowing ample time for high-speed driving. Its specific pitch angle installation effectively avoids ground reflection interference, ensuring clear images of side and rear thermal features. The auxiliary sensing unit provides supplementary information in extreme environments, collectively providing stable and reliable visual input to the system. This design enables the system to effectively detect vehicles and pedestrians at long distances in complete darkness, fundamentally solving the problem of blind spot vision enhancement in low-light environments and providing a solid foundation for subsequent risk assessment and warnings.

[0089] It breaks through the performance limitations of traditional optical systems in extremely low light conditions, realizing true all-weather blind spot monitoring; secondly, the passive imaging characteristics greatly reduce system energy consumption and dependence on the environment; finally, the dual-unit collaborative design ensures both imaging quality under normal conditions and system reliability in extreme environments, providing drivers with continuous and stable blind spot vision enhancement services.

[0090] In this embodiment, the signals acquired by the multi-source signal acquisition module include:

[0091] Torque value and its rate of change from the steering wheel torque sensor;

[0092] The status of the turn signal switch from the body control module;

[0093] Wheel angle signal from ESC or EPS system;

[0094] Opening signals from the accelerator and brake pedals;

[0095] The vehicle speed signal from the wheel speed sensor.

[0096] The data processing and risk assessment module includes:

[0097] An image preprocessing unit is used to perform spatial and temporal filtering on the thermal feature image to improve the signal-to-noise ratio;

[0098] The target extraction and tracking unit is used to cluster moving targets from the filtered image and establish a tracking trajectory for each target;

[0099] The multi-feature fusion computing unit is used to calculate the relative distance, relative velocity and direction of motion of the target based on the target's tracking trajectory, and to fuse the target's thermodynamic and morphological features to generate an initial risk score;

[0100] The intent recognition and risk coupling unit is used to calculate the driver's lane-changing intent confidence level based on the vehicle bus signal, and couple the confidence level with the initial risk score to output the final dynamic risk level.

[0101] This invention creatively applies shortwave infrared technology to the specific scenario of vehicle side and rear blind spots, and designs a set of data processing and decision-making warning logic tightly coupled with it. It specifically addresses the unique challenges faced by shortwave infrared technology in the vehicle environment. Addressing the technical difficulties of traditional shortwave infrared technology used in military applications, such as the special imaging noise patterns, the large difference between target thermodynamic characteristics and visible light, and strong reflection interference from wet and slippery road surfaces, and the poor performance of directly applying existing computer vision algorithms, this invention provides an innovative solution:

[0102] like Figure 3 As shown in this embodiment, the execution logic of the image preprocessing unit further includes:

[0103] The spatial domain filtering processing subunit uses an adaptive bilateral filtering algorithm that incorporates a thermal gradient weighting factor to process a single frame of SWIR image. The specific steps include:

[0104] Calculate the thermal gradient value H1 between each pixel in the image and its neighboring pixels;

[0105] The range variance of the bilateral filter is dynamically adjusted based on the thermal gradient value. and intensity variance parameter,

[0106] Set a predetermined thermal gradient threshold H0;

[0107] When the thermal gradient value H1 is less than the predetermined thermal gradient threshold H0, it is determined to be a low thermal gradient region, and the intensity variance is increased. To enhance smoothing and noise reduction, and increase range variance. This allows for smoothing by utilizing pixel information within a larger neighborhood, enhancing noise reduction capabilities for large, uniform areas such as roads and vehicle bodies.

[0108] When the thermal gradient value H1 is greater than or equal to the predetermined thermal gradient threshold H0, it is determined to be a high thermal gradient region, and the intensity variance is reduced. To preserve edge features and reduce range variance This limits the spatial influence of the filter to a closer neighborhood, avoiding blurring of details when crossing edges, thus better preserving edge sharpness;

[0109] Perform bilateral filtering calculations using the adjusted parameters to output a spatially enhanced image;

[0110] The temporal filtering subunit performs pixel-level cumulative difference operations on a sequence of N consecutive spatially enhanced image frames to generate a mask of salient moving targets. The specific steps include:

[0111] Perform a difference operation between each frame of the sequence and its previous frame to obtain a binarized difference image;

[0112] Perform a logical OR operation on the binary difference maps of N consecutive frames to generate a cumulative motion region map;

[0113] A morphological opening operation is performed on the accumulated motion region map to eliminate isolated pixels caused by noise and smooth the contour of the moving target.

[0114] The processed cumulative motion region map is used as the final moving target mask output for subsequent target extraction and tracking.

[0115] The core of the image preprocessing algorithm used in this system lies in the dual innovation of spatial and temporal domains, which is specifically optimized for the characteristics of SWIR images with low signal-to-noise ratio and weak target features under extremely low illumination.

[0116] 1. Spatial domain filtering (adaptive bilateral filtering):

[0117] Traditional bilateral filtering can smooth noise while preserving edges, but its parameters are fixed, resulting in poor performance in complex thermal imaging scenarios. This invention improves upon it:

[0118] Thermal gradient calculation: First, the thermal gradient magnitude of each pixel in the image is calculated using gradient operators such as Sobel. This gradient reflects the rate of temperature change in the scene;

[0119] Dynamic parameter adjustment: Filter intensity variance parameter It is no longer a constant, but rather the thermal gradient at that point. Functions:

[0120] ;

[0121] ;

[0122] In the formula, and These are the base values ​​for the intensity variance and the range variance, respectively; and These are adjustment coefficients, and all are negative. For pixels The thermal gradient value at that location;

[0123] In regions with small thermal gradients, such as uniform areas like road surfaces and vehicle bodies, the value of G(x, y) is small. As the value increases, the filter allows for greater grayscale differences, resulting in more thorough smoothing and noise reduction. As the value increases, the filter's effective range in the spatial domain widens, enabling it to utilize pixel information from a larger neighborhood for joint smoothing; through and The two work together to achieve the strongest smoothing effect on sensor inherent noise and large-area uniform regions.

[0124] In regions with large thermal gradients, such as at the edge of a target, the value of G(x, y) is large. As the value decreases, the filter becomes more sensitive, performing a weighted average only on pixels with very similar gray levels; As the value decreases, the effective range of the filter in the spatial domain shrinks, strictly limiting it to the immediate region adjacent to the edge; through and The two methods work together to smooth out minor noise near edges while maximizing edge sharpness and clarity, avoiding edge blurring. This approach implements a site-specific denoising strategy, which is key to improving the quality of low signal-to-noise ratio images.

[0125] 2. Temporal filtering (cumulative difference and morphological post-processing):

[0126] Traditional frame difference methods are sensitive to noise and lose stationary targets. The temporal processing of this invention aims to extract continuously moving targets.

[0127] Cumulative Differentiation: Five consecutive 150ms image sequences are taken. Instead of simple averaging, each frame is differentiated from the previous frame and binarized. Then, these four binary difference results are subjected to a logical OR operation. Through the above operation, any pixel that has moved at any time in the sequence will be recorded in the final cumulative image, ensuring the integrity of the moving target and avoiding the problem of loss in a single difference due to the target being temporarily stationary. At the same time, transient random noise is difficult to be triggered in multiple differences, so it can be effectively suppressed.

[0128] Since the cumulative difference map also contains scatter noise and holes inside the target, morphological processing is performed:

[0129] First, perform a closing operation using a smaller kernel (such as 3x3) to fill the tiny voids inside the target caused by uneven heat distribution, making the target area more continuous.

[0130] Then, an opening operation is performed using a slightly larger kernel (such as 5x5) to eliminate isolated noise points on the periphery and smooth the target boundary, producing a clean and complete moving target mask.

[0131] This process is used to improve the accuracy of extracting real moving targets under harsh imaging conditions.

[0132] like Figure 4 As shown in the figure, in this embodiment, the execution logic of the target extraction and tracking unit further includes:

[0133] Foreground target extraction subunit: Performs a logical AND operation between the moving target mask generated by temporal filtering and the enhanced image after spatial filtering to extract the foreground image containing pixels of moving heat source;

[0134] Dynamic parameter clustering subunit: The improved DBSCAN clustering algorithm is used to cluster the pixels in the foreground image. The improvements include:

[0135] Estimated size calculation: Based on the pixel coordinates of the target in the image and combined with camera calibration parameters, estimate the physical size of the target in the real world;

[0136] Dynamic parameter mapping: Establish a mapping function to map the physical dimensions to the clustering radius parameter Eps of the DBSCAN algorithm, where:

[0137] When the physical size C ≥ the first threshold When, the mapping function outputs Threshold;

[0138] When the second threshold ≤Physical size C < First threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold;

[0139] When physical size C < second threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold;

[0140] This allows for the use of larger Eps values ​​for large vehicle targets and smaller Eps values ​​for small pedestrian or motorcycle targets.

[0141] Density Adaptive: At the same time, the minimum number of points (MinPts) required for core point determination is dynamically adjusted according to the vehicle speed. The higher the vehicle speed, the lower the MinPts value is, in order to adapt to the sparser characteristics of target pixels in high-speed scenes.

[0142] Target trajectory initialization and tracking subunit: Initialize a tracker for each successfully clustered independent target:

[0143] a) Motion model establishment: Use Kalman filters to establish a uniform velocity (CV) or uniform acceleration (CA) motion model for each target, whose state vector contains at least the target's position and velocity in the image coordinate system;

[0144] b) Feature association: Calculate the feature similarity between the new target and the existing tracked targets. The features include the target's heat intensity histogram and the aspect ratio of the bounding rectangle. Then, use the Hungarian algorithm to perform optimal matching and association.

[0145] c) Lifecycle Management: Assign a unique tracking ID to each target and manage its tracking status; for newly detected but unmatched targets, initialize them as transient targets, and only after they have been tracked for M consecutive frames (M≥3) will they be promoted to stable targets and output to the risk assessment module; for lost tracking targets, start a delayed destruction mechanism, and only after they have failed to match a detected target for N consecutive frames (N≥5) will tracking be terminated.

[0146] One of the core innovations of this system is that it solves the problem of fragmentation and trajectory breakage of moving targets in low-light environments. Because traditional fixed-parameter clustering algorithms cannot effectively handle both large, near targets with many pixels and small, distant targets with few pixels at the same time, and targets in high-speed scenes are easily misclassified as noise, the improvements in this embodiment are as follows:

[0147] 1. Dynamic parameter DBSCAN clustering:

[0148] In terms of size perception, this solution no longer uses fixed Eps parameters. First, using the camera's calibrated intrinsic and extrinsic parameters, the pixel size of the target bounding box in the image is converted into the estimated physical size in the real world. For example, if the target width is estimated to be 1.8 meters, it can be preliminarily determined to be a car rather than a motorcycle.

[0149] Regarding parameter adaptation, this system pre-defines a mapping relationship: ;

[0150] In the formula, Eps is the clustering radius parameter of the DBSCAN algorithm; Eps is the input variable of the function, i.e., the estimated physical size of the target; for example, if the physical size is >2m, Eps=8; if the physical size is between 1m and 2m, Eps=5; if the physical size is <1m, Eps=3; it is used to ensure that targets of different sizes can be clustered into a whole at their most suitable scale, avoiding the error of large targets being split or multiple small targets being merged.

[0151] In speed adaptation, the MinPts parameter is dynamically adjusted according to vehicle speed:

[0152] ;

[0153] In the formula, MinPts is the dynamically adjusted minimum number of points threshold; This is a function to find the maximum value. The minimum number of points is the baseline value; floor() is the floor function; V is the independent variable, representing the vehicle speed (km / h); at high speeds, distant targets move faster and have fewer pixels on the image. Reducing the MinPts requirement can avoid misclassifying them as noise and improve the ability to detect distant vehicles on highways.

[0154] 2. Tracking based on multi-feature fusion and lifecycle management:

[0155] This embodiment not only performs Kalman prediction and association in terms of spatial location, but also introduces thermal intensity histogram as an appearance feature. In low-light SWIR images, the thermal radiation characteristics of different objects are relatively stable. This feature can effectively prevent ID switching when targets cross or occlude.

[0156] Transient targets and delayed destruction mechanisms were introduced into state machine management;

[0157] The transient target mechanism works as follows: for a single detection box that suddenly appears, including noise or a target that has just entered the field of view, it is not immediately recognized as a real target. It is required that the target must be matched successfully for more than 3 consecutive frames before it is confirmed as a real target. This is used to suppress the interference of transient noise.

[0158] The delayed destruction mechanism works as follows: when a target disappears briefly, such as when it is obscured by other vehicles, its tracking chain will not be deleted immediately. The Kalman filter will continue to predict its position and attempt to re-associate it in subsequent frames. Only after it cannot be recaptured for 5 consecutive frames will it be determined that the target has left the field of view. This mechanism is used to solve the problem of trajectory breakage caused by occlusion or image quality fluctuations and to ensure the continuity of tracking.

[0159] In this embodiment, the multi-feature fusion calculation unit calculates the initial risk score using the following formula:

[0160] ;

[0161] In the formula, For the initial risk score; , and These are configurable weighting coefficients; The target relative velocity; The relative distance to the target; These are the type weight coefficients derived from target thermal image classification.

[0162] In this embodiment, the execution logic of the intent recognition and risk coupling unit further includes:

[0163] Intent confidence calculation subunit: Establish a state machine model, whose input is the signal collected by the multi-source signal acquisition module, and outputs an intent confidence between 0 and 1;

[0164] Intent confidence calculation subunit: A finite state machine (FSM) model is established, whose input is the vehicle bus signal, and whose output is an intent confidence level between 0 and 1. Its state transition logic includes:

[0165] Initial state: When the absolute value of the steering wheel torque |T| < threshold Furthermore, when there is no turn signal, the confidence level is 0;

[0166] Intended state formation: If the detected steering wheel torque T lasts for Δt longer than the threshold value. But below the higher threshold If the torque change rate dT / dt > 0, the confidence level increases linearly from 0; if a turn signal is detected at the same time, the confidence level jumps to 0.8.

[0167] Lane change execution state: If steering wheel torque T > threshold If the wheel angle is detected to be continuously increasing, the confidence level is immediately set to 1 regardless of the turn signal status.

[0168] Intended cancellation state: In any state, if the brake pedal signal or torque value is detected to drop back to within Δt... The state machine is then reset to its initial state, and the confidence level is set to zero.

[0169] Dynamic risk calculation subunit: Calculates the effective risk score using the following formula:

[0170] ;

[0171] In the formula, For effective risk scoring; The initial risk score; α is the amplification factor; For the confidence level of the intention;

[0172] Dynamic threshold adjustment subunit: The distance threshold L1 for Level 1 warning is dynamically adjusted based on the vehicle speed V. ;

[0173] In the formula, The reference distance is (e.g., 50m). The reference speed is β (e.g., 80 km / h), and β is the adjustment coefficient.

[0174] The core innovation of this system lies in the deep coupling of the driver's implicit intentions with the objective risks of the target. Traditional solutions only detect a single event of torque exceeding a threshold. This invention designs a state machine based on multi-signal fusion, and its innovation lies in:

[0175] It includes multi-factor triggering. Since the state transition condition is a combination of multiple signals such as torque value, duration, rate of change, and turn signal status, for example, "torque increases slowly and turn signals are not activated" is regarded as a "weak intention" that needs to be continuously observed, and its confidence increases linearly; while "suddenly activating the turn signal" is regarded as a "strong intention", and the confidence jumps instantly.

[0176] Based on the priority of execution state, a lane-change execution state is set. Once a substantial steering action, large torque, or wheel angle is detected, other signals are ignored and the intention is directly identified as the highest level; this specifically addresses the recognition problem in scenarios where the driver is "lazy to turn on the lights" or in emergency obstacle avoidance scenarios.

[0177] A safety cancellation mechanism was designed, introducing an intention cancellation state. Once the driver applies the brakes, it indicates that he has perceived the risk and abandoned the lane change. The system should immediately exit the warning state to avoid interfering with the driver, reflecting the concept of machines serving humans in human-machine co-driving.

[0178] In this embodiment, the graded warning and intervention module further includes:

[0179] The Level 1 Alert Unit is used to display a yellow outline of the target via AR-HUD when the effective risk score exceeds the first threshold.

[0180] The secondary warning unit is used to trigger a red flashing in the rearview mirror area and a pulse vibration in the driver's seat when the effective risk score exceeds a higher second threshold and the intent confidence is greater than a set threshold.

[0181] The three-level intervention unit is used to apply a reverse torque through the electric power steering system when the effective risk score exceeds the highest third threshold and a change in wheel angle is detected. The torque value is between 0.2 Nm and 0.5 Nm.

[0182] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0183] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the invention.

Claims

1. A dynamic enhancement and warning system for vehicle blind spots in low-light environments, characterized in that, include: A short-wave infrared imaging module is used to capture passive thermal feature images of the scene behind and to the side of the vehicle. The multi-source signal acquisition module is used to acquire vehicle bus signals that represent the vehicle's motion state and the driver's operating intentions in real time. The data processing and risk assessment module is connected to the shortwave infrared imaging module and the multi-source signal acquisition module. It is used to fuse the thermal feature image and the vehicle bus signal to identify blind spot targets and calculate their dynamic risk level. The graded warning and intervention module is connected to the data processing and risk assessment module and is used to trigger different levels of warnings or vehicle control commands based on the dynamic risk level. The data processing and risk assessment module includes: an image preprocessing unit, used to perform spatial and temporal filtering on the thermal feature image to improve the signal-to-noise ratio; The execution logic of the image preprocessing unit includes: The spatial domain filtering processing subunit uses an adaptive bilateral filtering algorithm that incorporates a thermal gradient weighting factor to process a single frame of SWIR image. The specific steps include: Calculate the thermal gradient value H1 between each pixel in the image and its neighboring pixels; Set a predetermined thermal gradient threshold H0; When the thermal gradient value H1 is less than the predetermined thermal gradient threshold H0, it is determined to be a low thermal gradient region, and the intensity variance is increased. To enhance smoothing and noise reduction, and increase range variance. ; When the thermal gradient value H1 is greater than or equal to the predetermined thermal gradient threshold H0, it is determined to be a high thermal gradient region, and the intensity variance is reduced. To preserve edge features and reduce range variance ; Perform bilateral filtering calculations using the adjusted parameters to output a spatially enhanced image; The temporal filtering subunit performs pixel-level cumulative difference operations on a sequence of N consecutive spatially enhanced image frames to generate a mask of salient moving targets. The specific steps include: Perform a difference operation between each frame of the sequence and its previous frame to obtain a binarized difference image; Perform a logical OR operation on the binary difference maps of N consecutive frames to generate a cumulative motion region map; A morphological opening operation is performed on the accumulated motion region map to eliminate isolated pixels caused by noise and smooth the contour of the moving target. The processed cumulative motion region map is used as the final moving target mask output for subsequent target extraction and tracking.

2. The vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The short-wave infrared imaging module includes: The main SWIR imaging unit, using an InGaAs sensor, is installed below the base of the vehicle's exterior rearview mirror. The auxiliary sensing unit includes low-light cameras and safety laser lights mounted on both sides of the rear bumper.

3. The vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The signals acquired by the multi-source signal acquisition module include: Torque value and its rate of change from the steering wheel torque sensor; The status of the turn signal switch from the body control module; Wheel angle signal from ESC or EPS system; Opening signals from the accelerator and brake pedals; The vehicle speed signal from the wheel speed sensor.

4. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The data processing and risk assessment module also includes: The target extraction and tracking unit is used to cluster moving targets from the filtered image and establish a tracking trajectory for each target; The multi-feature fusion computing unit is used to calculate the relative distance, relative velocity and direction of motion of the target based on the target's tracking trajectory, and to fuse the target's thermodynamic and morphological features to generate an initial risk score; The intent recognition and risk coupling unit is used to calculate the driver's lane-changing intent confidence level based on the vehicle bus signal, and couple the confidence level with the initial risk score to output the final dynamic risk level.

5. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 4, characterized in that, The execution logic of the target extraction and tracking unit includes: Foreground target extraction subunit: Performs a logical AND operation between the moving target mask generated by temporal filtering and the enhanced image after spatial filtering to extract the foreground image containing pixels of moving heat source; Dynamic parameter clustering subunit: The improved DBSCAN clustering algorithm is used to cluster the pixels in the foreground image. The improvements include: Based on the pixel coordinates of the target in the image, combined with camera calibration parameters, the physical size of the target in the real world is estimated; Establish a mapping function to map the physical dimensions to the clustering radius parameter Eps of the DBSCAN algorithm. in: When the physical size C ≥ the first threshold When, the mapping function outputs Threshold; When the second threshold ≤Physical size C < First threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold; When physical size C < second threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold; The minimum number of points (MinPts) required for core point determination is dynamically adjusted based on the vehicle speed. The higher the vehicle speed, the lower the MinPts value, in order to adapt to the sparser target pixels in high-speed scenes. Target trajectory initialization and tracking subunit: Initialize a tracker for each successfully clustered independent target: A uniform or uniformly accelerated motion model is established for each target using a Kalman filter, and its state vector contains at least the target's position and velocity in the image coordinate system. Calculate the feature similarity between the new target and the existing tracked targets. The features include the target's heat intensity histogram and the aspect ratio of its bounding rectangle. Then, use the Hungarian algorithm to perform optimal matching and association. Each target is assigned a unique tracking ID and its tracking status is managed. Newly detected but unmatched targets are initialized as transient targets and are only promoted to stable targets and output to the risk assessment module after being tracked for M consecutive frames. For lost tracked targets, a delayed destruction mechanism is activated, and tracking is terminated after the target fails to match a detected target for N consecutive frames.

6. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 4, characterized in that, The formula for calculating the initial risk score by the multi-feature fusion calculation unit is as follows: In the formula, For the initial risk score; These are configurable weighting coefficients; The target relative velocity; The relative distance to the target; These are the type weight coefficients derived from target thermal image classification.

7. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 4, characterized in that, The execution logic of the intent recognition and risk coupling unit includes: Intent confidence calculation subunit: A finite state machine model is established, whose input is the vehicle bus signal, and whose output is an intent confidence level between 0 and 1. Its state transition logic includes: Initial state: When the absolute value of the steering wheel torque |T| < threshold T1 and there is no turn signal, the confidence level is 0; Intended formation state: If the detected steering wheel torque T lasts for Δt for a duration exceeding the threshold T1 but below the higher threshold T2, and the torque change rate dT / dt > 0, then the confidence level increases linearly from 0; if a turn signal is detected at the same time, then the confidence level jumps to 0.

8. Lane change execution state: If the steering wheel torque T > threshold T2, or if the wheel angle is detected to be continuously increasing, the confidence level is immediately set to 1 regardless of the turn signal status; Intended cancellation state: In any state, if the brake pedal signal or torque value is detected to fall below T1 within Δt, the state machine is reset to the initial state and the confidence level is set to zero. Dynamic risk calculation subunit: Calculates the effective risk score using the following formula: In the formula, For effective risk scoring; The initial risk score; α is the amplification factor; For the confidence level of the intention; Dynamic threshold adjustment subunit: The distance threshold L1 for Level 1 warning is dynamically adjusted based on the vehicle speed V. In the formula, As the reference distance, The reference speed is β, which is an adjustment coefficient.

8. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The tiered warning and intervention module includes: The Level 1 Alert Unit is used to display a yellow outline of the target via AR-HUD when the effective risk score exceeds the first threshold. The secondary warning unit is used to trigger a red flashing in the rearview mirror area and a pulse vibration in the driver's seat when the effective risk score exceeds a higher second threshold and the intent confidence is greater than a set threshold. The three-level intervention unit is used to apply a reverse torque through the electric power steering system when the effective risk score exceeds the highest third threshold and a change in wheel angle is detected. The torque value is between 0.2 Nm and 0.5 Nm.

Images