Vehicle blind area dynamic enhancement and warning system oriented to low-light environment

Abstract

The invention discloses a vehicle blind area dynamic enhancement and warning system for a low-light environment, and relates to the technical field of automobile active safety, and the system comprises a short wave infrared imaging module which is used for capturing a passive thermal feature image of a scene behind the side of a vehicle; the multi-source signal acquisition module is used for acquiring a vehicle bus signal representing a vehicle motion state and an operation intention of a driver in real time; the data processing and risk assessment module is connected with the short wave infrared imaging module and the multi-source signal acquisition module, and is used for performing fusion processing on the thermal feature image and the vehicle bus signal, identifying a blind area target and calculating a dynamic risk level of the blind area target; the grading warning and intervention module is connected with the data processing and risk assessment module; the short-wave infrared imaging technology which is sensitive to night light and high in anti-interference capacity is adopted, the imaging problem under extremely low illumination is fundamentally solved, and stable detection of vehicles and pedestrians in a long distance in a completely light-source-free environment is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automobile active safety technology, in particular to a vehicle blind area dynamic enhancement and warning system for low-light environment. BACKGROUND

[0002] When driving in low-light environment such as night, dusk or tunnel, the visual perception ability of the driver is greatly reduced, and the blind area problem of the side rear of the vehicle becomes particularly prominent, greatly increasing the safety risk when changing lanes. In order to cope with this challenge, the existing vehicle blind area monitoring system mainly relies on traditional visible light camera, millimeter wave radar or near-infrared technology. However, these technical solutions all have significant limitations; traditional visible light camera and streaming rearview mirror fail or produce a lot of noise due to insufficient light in very low illumination; millimeter wave radar can detect objects but cannot provide intuitive image information, resulting in a lack of intuitive perception of the driver to the danger in the blind area; and the near-infrared scheme based on active illumination has inherent shortcomings of short action distance and serious attenuation in bad weather such as rain and fog.

[0003] On the one hand, in the early warning logic, the existing system's prediction mechanism often relies on the single dimension of the turn signal. This judgment is lagging and passive because when the turn signal is on, the merging action has often already started or is about to start, leaving a very short time window for the system to identify risks and issue warnings, which cannot achieve effective early warning. More importantly, in a completely dark environment, the driver may not turn on the turn signal at all due to safety concerns because he cannot see the blind area at all, which makes the prediction mechanism relying on the turn signal completely ineffective.

[0004] On the other hand, the warning strategy of the existing system is often single, usually only issuing a general warning after detecting an object, which cannot effectively distinguish risk levels and cannot deeply couple with the real-time operation intention of the driver, which may lead to frequent false alarms, thereby interfering with the driver and reducing his trust in the system.

[0005] Therefore, the industry urgently needs a comprehensive solution that can penetrate darkness and bad weather under passive conditions, provide clear blind area images, intelligently understand the driver's intention, and provide advanced and accurate risk warnings. SUMMARY

[0006] The main purpose of the present application is to provide a vehicle blind area dynamic enhancement and warning system for low-light environment to solve the problems mentioned in the background.

[0007] To achieve the above purpose, the technical scheme adopted by the present application is as follows: a vehicle blind area dynamic enhancement and warning system for low-light environment, comprising:

[0008] a short-wave infrared imaging module for capturing a passive thermal feature image of a scene behind and beside the vehicle;

[0009] a multi-source signal acquisition module for acquiring vehicle bus signals representing vehicle motion state and driver operation intention in real time;

[0010] a data processing and risk assessment module connected to the short-wave infrared imaging module and the multi-source signal acquisition module, for fusing the thermal feature image and the vehicle bus signals, identifying blind area targets and calculating their dynamic risk levels;

[0011] a hierarchical warning and intervention module connected to the data processing and risk assessment module, for triggering different levels of warning or vehicle control instructions according to the dynamic risk levels.

[0012] Preferably, the short-wave infrared imaging module comprises:

[0013] a main SWIR imaging unit employing InGaAs sensors, installed below the vehicle's exterior rearview mirror base at a pitch angle of 30°±5°;

[0014] an auxiliary perception unit comprising low-light cameras and safety laser illuminators installed on both sides of the rear bumper.

[0015] Preferably, the multi-source signal acquisition module acquires signals including:

[0016] torque values and their rates of change from the steering wheel torque sensor;

[0017] steering light switch states from the body control module;

[0018] wheel rotation angle signals from the ESC or EPS system;

[0019] opening degree signals from the accelerator and brake pedals;

[0020] vehicle speed signals from the wheel speed sensors.

[0021] Preferably, the data processing and risk assessment module comprises:

[0022] an image preprocessing unit for spatial and temporal filtering of the thermal feature image to improve signal-to-noise ratio;

[0023] a target extraction and tracking unit for clustering moving targets from the filtered image and establishing tracking trajectories for each target;

[0024] a multi-feature fusion calculation unit for calculating the relative distance, relative speed, and motion direction of the target based on its tracking trajectory, and fusing the target's thermodynamic and morphological features to generate an initial risk score;

[0025] an intent recognition and risk coupling unit for calculating a lane change intent confidence of the driver based on the vehicle bus signals and coupling this confidence with the initial risk score, outputting a final dynamic risk level.

[0026] Preferably, the execution logic of the image pre-processing unit comprises:

[0027] a spatial filtering processing sub-unit that adopts an adaptive bilateral filtering algorithm with a heat gradient weight factor introduced to process a single frame of SWIR image, the specific steps of which include:

[0028] calculating a heat gradient value H1 of each pixel point in the image and the pixels in its neighborhood;

[0029] setting a predetermined heat gradient threshold H0;

[0030] when the heat gradient value H1 is less than the predetermined heat gradient threshold H0, determining a low heat gradient region, increasing the intensity variance to enhance the smoothing and denoising effect and increase the range variance ;

[0031] when the heat gradient value H1 is greater than or equal to the predetermined heat gradient threshold H0, determining a high heat gradient region, decreasing the intensity variance to retain the edge features and decrease the range variance ;

[0032] performing bilateral filtering calculation using the adjusted parameters, outputting a spatially enhanced image;

[0033] a temporal filtering processing sub-unit that performs pixel-level cumulative difference operation on a sequence of N consecutive spatially enhanced images to generate a significant motion target mask, the specific steps of which include:

[0034] performing difference operation on each frame of image in the sequence and its previous frame of image to obtain a binary difference map;

[0035] performing logical OR operation on the binary difference maps of the N consecutive frames to generate a cumulative motion region map;

[0036] performing morphological opening operation on the cumulative motion region map to eliminate isolated pixel points caused by noise and smooth the outline of the motion target;

[0037] outputting the processed cumulative motion region map as the final motion target mask for subsequent target extraction and tracking.

[0038] Preferably, the execution logic of the target extraction and tracking unit comprises:

[0039] A foreground object extraction subunit: performing a logical AND operation between the motion object mask generated by the time domain filtering and the enhanced image filtered by the spatial domain filtering to extract a foreground image containing the motion heat source pixels;

[0040] A dynamic parameter clustering subunit: using an improved DBSCAN clustering algorithm to cluster the pixels in the foreground image, and the improvement includes:

[0041] According to the pixel coordinates of the target in the image and in combination with the camera calibration parameters, the physical size of the target in the real world is estimated;

[0042] A mapping function is established to map the physical size to a clustering radius parameter Eps of the DBSCAN algorithm,

[0043] Wherein:

[0044] When the physical size C is greater than or equal to a first threshold , the mapping function outputs a threshold value;

[0045] When a second threshold ≤ the physical size C < the first threshold , the mapping function outputs a threshold value, and the threshold value < the threshold value;

[0046] When the physical size C < the second threshold , the mapping function outputs a threshold value, and the threshold value < the threshold value;

[0047] According to the vehicle speed, the minimum number of points MinPts required for the core point determination is dynamically adjusted. The higher the vehicle speed, the lower the MinPts value, so as to adapt to the characteristics of the target pixel points being more sparse in the high-speed scene.

[0048] A target trajectory initialization and tracking subunit: initializing a tracker for each independent target whose clustering is successful:

[0049] Using a Kalman filter to establish a uniform speed or uniform acceleration motion model for each target, and the state vector at least includes the position and speed of the target in the image coordinate system;

[0050] Calculating the feature similarity between the new target and the existing tracking target, the features including the heat intensity histogram and the aspect ratio of the circumscribed rectangle of the target, and using the Hungarian algorithm for optimal matching association;

[0051] Assign a unique tracking ID to each target and manage its tracking state; for newly detected but unmatched targets, initialize as transient targets and only promote to stable targets and output to risk assessment module when it is continuously tracked for M frames; for lost tracking targets, start a delayed destruction mechanism and terminate tracking when it fails to match a detected target for N consecutive frames.

[0052] Preferably, the multi-feature fusion calculation unit calculates the initial risk score according to the formula:

[0053] ;

[0054] wherein, is the initial risk score; 、 and are configurable weighting coefficients; is the relative speed of the target; is the relative distance of the target; is the type weight coefficient based on the classification of the target thermal image.

[0055] Preferably, the execution logic of the intent recognition and risk coupling unit includes:

[0056] An intent confidence calculation subunit: a finite state machine model is established, the input of which is the vehicle bus signal, and the output is an intent confidence between 0 and 1, and the state transition logic includes:

[0057] Initial state: when the absolute value of the steering wheel torque |T| < threshold , and there is no turn signal, the confidence is 0;

[0058] Intent formation state: if the steering wheel torque T is continuously monitored to exceed the threshold for Δt time but is lower than a higher threshold , and the torque change rate dT / dt > 0, the confidence starts to increase linearly from 0; if the turn signal is detected at the same time, the confidence jumps to 0.8;

[0059] Parallel execution state: if the steering wheel torque T > threshold , or the wheel angle is continuously increasing, the confidence is immediately set to 1 regardless of the turn signal state;

[0060] Intent cancellation state: in any state, if the brake pedal signal or the torque value falls to within Δt, the state machine is reset to the initial state and the confidence is zero;

[0061] Dynamic risk calculation subunit: calculate the effective risk score according to the formula:

[0062] ;

[0063] wherein, is an effective risk score; is an initial risk score; and a is a magnification factor; is an intention confidence;

[0064] The dynamic threshold adjustment subunit: the distance threshold L1 of the first-level warning is dynamically adjusted according to the vehicle speed V of the host vehicle: ;

[0065] wherein, is a reference distance, is a reference vehicle speed, and b is an adjustment factor.

[0066] Preferably, the hierarchical warning and intervention module comprises:

[0067] a first-level warning unit for displaying a yellow outline of the target through the AR-HUD when the effective risk score exceeds a first threshold value;

[0068] a second-level warning unit for triggering a red flashing in the rearview mirror area and a pulse vibration of the driver's seat when the effective risk score exceeds a higher second threshold value and the intention confidence is greater than a set threshold;

[0069] a third-level intervention unit for applying a reverse torque through the electric power steering system when the effective risk score exceeds a highest third threshold value and a change in the wheel angle is monitored, and the torque value is between 0.2 Nm and 0.5 Nm.

[0070] Compared with the prior art, the present application has the following beneficial effects:

[0071] The present application fundamentally overcomes the imaging problem under extremely low illumination by adopting the short-wave infrared imaging technology which is sensitive to night sky light and has strong anti-interference ability, realizes stable detection of distant vehicles and pedestrians in a completely light source-free environment, and the effective distance and image quality far exceed the traditional optical scheme, and provides a high-quality image basis for subsequent processing; at the information processing level, through the dual innovation filtering algorithm combining space domain and time domain, the key target edge features are perfectly preserved while the sensor noise is successfully suppressed, greatly improving the image signal-to-noise ratio in a low signal-to-noise ratio environment; further, by introducing a dynamic target clustering adaptive to size and speed and a multi-target tracking management mechanism fused with thermal features, the problems of detection and stable tracking of far, near and large targets are effectively solved, the fragmentation and trajectory breaking phenomenon of the target are significantly reduced, thereby laying a solid foundation for accurate risk assessment.

[0072] The fine state machine model is constructed to interpret the micro-operation signal of the steering wheel, early and accurate intention of the driver is realized, and the intention confidence is dynamically coupled with the objective risk of the target, compared with the mechanical reaction type pre-warning, the risk information is transmitted to the driver in the most intuitive and least interference manner through the AR-HUD visual enhancement, haptic vibration and steering intervention combined grading warning strategy, sufficient pre-warning time is provided before the danger occurs, the human-computer interaction experience is greatly improved, the fright and boredom of the driver are avoided, and the safety and driving confidence of the lane-keeping operation in the dark environment are comprehensively improved.

[0073] In summary, the application forms a complete safety closed loop through the technical innovation of the perception-processing-decision-warning whole link. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 A module flowchart of a vehicle blind area dynamic enhancement and warning system in a low-light environment according to the application; Figure 2 A flowchart of a data processing and risk assessment module of a vehicle blind area dynamic enhancement and warning system in a low-light environment according to the application; Figure 3 A flowchart of an image preprocessing unit of a vehicle blind area dynamic enhancement and warning system in a low-light environment according to the application; Figure 4 A flowchart of a target extraction and tracking unit of a vehicle blind area dynamic enhancement and warning system in a low-light environment according to the application; Figure 5 A flowchart of an intention recognition and risk coupling unit of a vehicle blind area dynamic enhancement and warning system in a low-light environment according to the application. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the application will be apparently and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0076] Please refer to Figures 1 to 5 The application provides a technical solution: a vehicle blind area dynamic enhancement and warning system in a low-light environment, comprising:

[0077] A short-wave infrared imaging module is used to capture a passive thermal feature image of a scene behind and on the side of the vehicle in a 1.0-1.7 mu m wave band;

[0078] The multi-source signal acquisition module is used to acquire vehicle bus signals that represent the vehicle's motion state and the driver's operating intentions in real time.

[0079] The data processing and risk assessment module is connected to the shortwave infrared imaging module and the multi-source signal acquisition module. It is used to fuse the thermal feature image and the vehicle bus signal to identify blind spot targets and calculate their dynamic risk level.

[0080] The graded warning and intervention module is connected to the data processing and risk assessment module and is used to trigger different levels of warnings or vehicle control commands based on the dynamic risk level.

[0081] In a further embodiment, the shortwave infrared imaging module includes:

[0082] The main SWIR imaging unit uses an InGaAs sensor with a resolution of no less than 1280×960 and a frame rate of no less than 30fps. It is mounted below the vehicle's exterior rearview mirror base at a pitch angle of 30°±5°.

[0083] The auxiliary sensing unit includes low-light cameras and / or 1550nm wavelength safety laser fillers mounted on both sides of the rear bumper.

[0084] The system works by fully utilizing the ability of short-wave infrared technology to perceive nighttime light. The main SWIR imaging unit enables passive imaging in dark environments. Infrared imaging is superior at night; a direct backlight from a source could impair following vehicles and create safety hazards. Furthermore, the performance degradation of the backlight-free solution in rain and fog is significantly lower than that of the source-lit solution, greatly increasing reliability. It also facilitates ultra-long-range detection, allowing ample time for high-speed driving. Its specific pitch angle installation effectively avoids ground reflection interference, ensuring clear images of side and rear thermal features. The auxiliary sensing unit provides supplementary information in extreme environments, collectively providing stable and reliable visual input to the system. This design enables the system to effectively detect vehicles and pedestrians at long distances in complete darkness, fundamentally solving the problem of blind spot vision enhancement in low-light environments and providing a solid foundation for subsequent risk assessment and warnings.

[0085] It breaks through the performance limitations of traditional optical systems in extremely low light conditions, realizing true all-weather blind spot monitoring; secondly, the passive imaging characteristics greatly reduce system energy consumption and dependence on the environment; finally, the dual-unit collaborative design ensures both imaging quality under normal conditions and system reliability in extreme environments, providing drivers with continuous and stable blind spot vision enhancement services.

[0086] In this embodiment, the signals acquired by the multi-source signal acquisition module include:

[0087] Torque value and its rate of change from the steering wheel torque sensor;

[0088] The status of the turn signal switch from the body control module;

[0089] Wheel angle signal from ESC or EPS system;

[0090] Opening signals from the accelerator and brake pedals;

[0091] The vehicle speed signal from the wheel speed sensor.

[0092] The data processing and risk assessment module includes:

[0093] An image preprocessing unit is used to perform spatial and temporal filtering on the thermal feature image to improve the signal-to-noise ratio;

[0094] The target extraction and tracking unit is used to cluster moving targets from the filtered image and establish a tracking trajectory for each target;

[0095] The multi-feature fusion computing unit is used to calculate the relative distance, relative velocity and direction of motion of the target based on the target's tracking trajectory, and to fuse the target's thermodynamic and morphological features to generate an initial risk score;

[0096] The intent recognition and risk coupling unit is used to calculate the driver's lane-changing intent confidence level based on the vehicle bus signal, and couple the confidence level with the initial risk score to output the final dynamic risk level.

[0097] This invention creatively applies shortwave infrared technology to the specific scenario of vehicle side and rear blind spots, and designs a set of data processing and decision-making warning logic tightly coupled with it. It specifically addresses the unique challenges faced by shortwave infrared technology in the vehicle environment. Addressing the technical difficulties of traditional shortwave infrared technology used in military applications, such as the special imaging noise patterns, the large difference between target thermodynamic characteristics and visible light, and strong reflection interference from wet and slippery road surfaces, and the poor performance of directly applying existing computer vision algorithms, this invention provides an innovative solution:

[0098] like Figure 3 As shown in this embodiment, the execution logic of the image preprocessing unit further includes:

[0099] The spatial domain filtering processing subunit uses an adaptive bilateral filtering algorithm that incorporates a thermal gradient weighting factor to process a single frame of SWIR image. The specific steps include:

[0100] Calculate the thermal gradient value H1 between each pixel in the image and its neighboring pixels;

[0101] The range variance of the bilateral filter is dynamically adjusted based on the thermal gradient value. and intensity variance parameter,

[0102] Set a predetermined thermal gradient threshold H0;

[0103] When the thermal gradient value H1 is less than the predetermined thermal gradient threshold H0, it is determined to be a low thermal gradient region, and the intensity variance is increased. To enhance smoothing and noise reduction, and increase range variance. This allows for smoothing by utilizing pixel information within a larger neighborhood, enhancing noise reduction capabilities for large, uniform areas such as roads and vehicle bodies.

[0104] When the thermal gradient value H1 is greater than or equal to the predetermined thermal gradient threshold H0, it is determined to be a high thermal gradient region, and the intensity variance is reduced. To preserve edge features and reduce range variance This limits the spatial influence of the filter to a closer neighborhood, avoiding blurring of details when crossing edges, thus better preserving edge sharpness;

[0105] Perform bilateral filtering calculations using the adjusted parameters to output a spatially enhanced image;

[0106] The temporal filtering subunit performs pixel-level cumulative difference operations on a sequence of N consecutive spatially enhanced image frames to generate a mask of salient moving targets. The specific steps include:

[0107] Perform a difference operation between each frame of the sequence and its previous frame to obtain a binarized difference image;

[0108] Perform a logical OR operation on the binary difference maps of N consecutive frames to generate a cumulative motion region map;

[0109] A morphological opening operation is performed on the accumulated motion region map to eliminate isolated pixels caused by noise and smooth the contour of the moving target.

[0110] The processed cumulative motion region map is used as the final moving target mask output for subsequent target extraction and tracking.

[0111] The core of the image preprocessing algorithm used in this system lies in the dual innovation of spatial and temporal domains, which is specifically optimized for the characteristics of SWIR images with low signal-to-noise ratio and weak target features under extremely low illumination.

[0112] 1. Spatial domain filtering (adaptive bilateral filtering):

[0113] Traditional bilateral filtering can smooth noise while preserving edges, but its parameters are fixed, resulting in poor performance in complex thermal imaging scenarios. This invention improves upon it:

[0114] Thermal gradient calculation: First, the thermal gradient magnitude of each pixel in the image is calculated using gradient operators such as Sobel. This gradient reflects the rate of temperature change in the scene;

[0115] Dynamic parameter adjustment: Filter intensity variance parameter It is no longer a constant, but rather the thermal gradient at that point. Functions:

[0116] ;

[0117] ;

[0118] In the formula, and These are the base values ​​for the intensity variance and the range variance, respectively; and These are adjustment coefficients, and all are negative. For pixels The thermal gradient value at that location;

[0119] In regions with small thermal gradients, such as uniform areas like road surfaces and vehicle bodies, the value of G(x, y) is small. As the value increases, the filter allows for greater grayscale differences, resulting in more thorough smoothing and noise reduction. As the value increases, the filter's effective range in the spatial domain widens, enabling it to utilize pixel information from a larger neighborhood for joint smoothing; through and The two work together to achieve the strongest smoothing effect on sensor inherent noise and large-area uniform regions.

[0120] In regions with large thermal gradients, such as at the edge of a target, the value of G(x, y) is large. As the value decreases, the filter becomes more sensitive, performing a weighted average only on pixels with very similar gray levels; As the value decreases, the effective range of the filter in the spatial domain shrinks, strictly limiting it to the immediate region adjacent to the edge; through and The two methods work together to smooth out minor noise near edges while maximizing edge sharpness and clarity, avoiding edge blurring. This approach implements a site-specific denoising strategy, which is key to improving the quality of low signal-to-noise ratio images.

[0121] 2. Temporal filtering (cumulative difference and morphological post-processing):

[0122] Traditional frame difference methods are sensitive to noise and lose stationary targets. The temporal processing of this invention aims to extract continuously moving targets.

[0123] Cumulative Differentiation: Five consecutive 150ms image sequences are taken. Instead of simple averaging, each frame is differentiated from the previous frame and binarized. Then, these four binary difference results are subjected to a logical OR operation. Through the above operation, any pixel that has moved at any time in the sequence will be recorded in the final cumulative image, ensuring the integrity of the moving target and avoiding the problem of loss in a single difference due to the target being temporarily stationary. At the same time, transient random noise is difficult to be triggered in multiple differences, so it can be effectively suppressed.

[0124] Since the cumulative difference map also contains scatter noise and holes inside the target, morphological processing is performed:

[0125] First, perform a closing operation using a smaller kernel (such as 3x3) to fill the tiny voids inside the target caused by uneven heat distribution, making the target area more continuous.

[0126] Then, an opening operation is performed using a slightly larger kernel (such as 5x5) to eliminate isolated noise points on the periphery and smooth the target boundary, producing a clean and complete moving target mask.

[0127] This process is used to improve the accuracy of extracting real moving targets under harsh imaging conditions.

[0128] like Figure 4 As shown in the figure, in this embodiment, the execution logic of the target extraction and tracking unit further includes:

[0129] Foreground target extraction subunit: Performs a logical AND operation between the moving target mask generated by temporal filtering and the enhanced image after spatial filtering to extract the foreground image containing pixels of moving heat source;

[0130] Dynamic parameter clustering subunit: The improved DBSCAN clustering algorithm is used to cluster the pixels in the foreground image. The improvements include:

[0131] Estimated size calculation: Based on the pixel coordinates of the target in the image and combined with camera calibration parameters, estimate the physical size of the target in the real world;

[0132] Dynamic parameter mapping: Establish a mapping function to map the physical dimensions to the clustering radius parameter Eps of the DBSCAN algorithm, where:

[0133] When the physical size C ≥ the first threshold When, the mapping function outputs Threshold;

[0134] When the second threshold ≤Physical size C < First threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold;

[0135] When physical size C < second threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold;

[0136] This allows for the use of larger Eps values ​​for large vehicle targets and smaller Eps values ​​for small pedestrian or motorcycle targets.

[0137] Density Adaptive: At the same time, the minimum number of points (MinPts) required for core point determination is dynamically adjusted according to the vehicle speed. The higher the vehicle speed, the lower the MinPts value is, in order to adapt to the sparser characteristics of target pixels in high-speed scenes.

[0138] Target trajectory initialization and tracking subunit: Initialize a tracker for each successfully clustered independent target:

[0139] a) Motion model establishment: Use Kalman filters to establish a uniform velocity (CV) or uniform acceleration (CA) motion model for each target, whose state vector contains at least the target's position and velocity in the image coordinate system;

[0140] b) Feature association: Calculate the feature similarity between the new target and the existing tracked targets. The features include the target's heat intensity histogram and the aspect ratio of the bounding rectangle. Then, use the Hungarian algorithm to perform optimal matching and association.

[0141] c) Lifecycle Management: Assign a unique tracking ID to each target and manage its tracking status; for newly detected but unmatched targets, initialize them as transient targets, and only after they have been tracked for M consecutive frames (M≥3) will they be promoted to stable targets and output to the risk assessment module; for lost tracking targets, start a delayed destruction mechanism, and only after they have failed to match a detected target for N consecutive frames (N≥5) will tracking be terminated.

[0142] One of the core innovations of this system is that it solves the problem of fragmentation and trajectory breakage of moving targets in low-light environments. Because traditional fixed-parameter clustering algorithms cannot effectively handle both large, near targets with many pixels and small, distant targets with few pixels at the same time, and targets in high-speed scenes are easily misclassified as noise, the improvements in this embodiment are as follows:

[0143] 1. Dynamic parameter DBSCAN clustering:

[0144] In terms of size perception, this solution no longer uses fixed Eps parameters. First, using the camera's calibrated intrinsic and extrinsic parameters, the pixel size of the target bounding box in the image is converted into the estimated physical size in the real world. For example, if the target width is estimated to be 1.8 meters, it can be preliminarily determined to be a car rather than a motorcycle.

[0145] Regarding parameter adaptation, this system pre-defines a mapping relationship: ;

[0146] In the formula, Eps is the clustering radius parameter of the DBSCAN algorithm; Eps is the input variable of the function, i.e., the estimated physical size of the target; for example, if the physical size is >2m, Eps=8; if the physical size is between 1m and 2m, Eps=5; if the physical size is <1m, Eps=3; it is used to ensure that targets of different sizes can be clustered into a whole at their most suitable scale, avoiding the error of large targets being split or multiple small targets being merged.

[0147] In speed adaptation, the MinPts parameter is dynamically adjusted according to vehicle speed:

[0148] ;

[0149] In the formula, MinPts is the dynamically adjusted minimum number of points threshold; This is a function to find the maximum value. The minimum number of points is the baseline value; floor() is the floor function; V is the independent variable, representing the vehicle speed (km / h); at high speeds, distant targets move faster and have fewer pixels on the image. Reducing the MinPts requirement can avoid misclassifying them as noise and improve the ability to detect distant vehicles on highways.

[0150] 2. Tracking based on multi-feature fusion and lifecycle management:

[0151] This embodiment not only performs Kalman prediction and association in terms of spatial location, but also introduces thermal intensity histogram as an appearance feature. In low-light SWIR images, the thermal radiation characteristics of different objects are relatively stable. This feature can effectively prevent ID switching when targets cross or occlude.

[0152] Transient targets and delayed destruction mechanisms were introduced into state machine management;

[0153] The transient target mechanism works as follows: for a single detection box that suddenly appears, including noise or a target that has just entered the field of view, it is not immediately recognized as a real target. It is required that the target must be matched successfully for more than 3 consecutive frames before it is confirmed as a real target. This is used to suppress the interference of transient noise.

[0154] The delayed destruction mechanism works as follows: when a target disappears briefly, such as when it is obscured by other vehicles, its tracking chain will not be deleted immediately. The Kalman filter will continue to predict its position and attempt to re-associate it in subsequent frames. Only after it cannot be recaptured for 5 consecutive frames will it be determined that the target has left the field of view. This mechanism is used to solve the problem of trajectory breakage caused by occlusion or image quality fluctuations and to ensure the continuity of tracking.

[0155] In this embodiment, the multi-feature fusion calculation unit calculates the initial risk score using the following formula:

[0156] ;

[0157] In the formula, For the initial risk score; , and These are configurable weighting coefficients; The target relative velocity; The relative distance to the target; These are the type weight coefficients derived from target thermal image classification.

[0158] In this embodiment, the execution logic of the intent recognition and risk coupling unit further includes:

[0159] Intent confidence calculation subunit: Establish a state machine model, whose input is the signal collected by the multi-source signal acquisition module, and outputs an intent confidence between 0 and 1;

[0160] Intent confidence calculation subunit: A finite state machine (FSM) model is established, whose input is the vehicle bus signal, and whose output is an intent confidence level between 0 and 1. Its state transition logic includes:

[0161] Initial state: When the absolute value of the steering wheel torque |T| < threshold Furthermore, when there is no turn signal, the confidence level is 0;

[0162] Intended state formation: If the detected steering wheel torque T lasts for Δt longer than the threshold value. But below the higher threshold If the torque change rate dT / dt > 0, the confidence level increases linearly from 0; if a turn signal is detected at the same time, the confidence level jumps to 0.8.

[0163] Lane change execution state: If steering wheel torque T > threshold If the wheel angle is detected to be continuously increasing, the confidence level is immediately set to 1 regardless of the turn signal status.

[0164] Intended cancellation state: In any state, if the brake pedal signal or torque value is detected to drop back to within Δt... The state machine is then reset to its initial state, and the confidence level is set to zero.

[0165] Dynamic risk calculation subunit: Calculates the effective risk score using the following formula:

[0166] ;

[0167] In the formula, For effective risk scoring; The initial risk score; α is the amplification factor; For the confidence level of the intention;

[0168] Dynamic threshold adjustment subunit: The distance threshold L1 for Level 1 warning is dynamically adjusted based on the vehicle speed V. ;

[0169] In the formula, The reference distance is (e.g., 50m). The reference speed is β (e.g., 80 km / h), and β is the adjustment coefficient.

[0170] The core innovation of this system lies in the deep coupling of the driver's implicit intentions with the objective risks of the target. Traditional solutions only detect a single event of torque exceeding a threshold. This invention designs a state machine based on multi-signal fusion, and its innovation lies in:

[0171] It includes multi-factor triggering. Since the state transition condition is a combination of multiple signals such as torque value, duration, rate of change, and turn signal status, for example, "torque increases slowly and turn signals are not activated" is regarded as a "weak intention" that needs to be continuously observed, and its confidence increases linearly; while "suddenly activating the turn signal" is regarded as a "strong intention", and the confidence jumps instantly.

[0172] Based on the priority of execution state, a lane-change execution state is set. Once a substantial steering action, large torque, or wheel angle is detected, other signals are ignored and the intention is directly identified as the highest level; this specifically addresses the recognition problem in scenarios where the driver is "lazy to turn on the lights" or in emergency obstacle avoidance scenarios.

[0173] A safety cancellation mechanism was designed, introducing an intention cancellation state. Once the driver applies the brakes, it indicates that he has perceived the risk and abandoned the lane change. The system should immediately exit the warning state to avoid interfering with the driver, reflecting the concept of machines serving humans in human-machine co-driving.

[0174] In this embodiment, the graded warning and intervention module further includes:

[0175] The Level 1 Alert Unit is used to display a yellow outline of the target via AR-HUD when the effective risk score exceeds the first threshold.

[0176] The secondary warning unit is used to trigger a red flashing in the rearview mirror area and a pulse vibration in the driver's seat when the effective risk score exceeds a higher second threshold and the intent confidence is greater than a set threshold.

[0177] The three-level intervention unit is used to apply a reverse torque through the electric power steering system when the effective risk score exceeds the highest third threshold and a change in wheel angle is detected. The torque value is between 0.2 Nm and 0.5 Nm.

[0178] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0179] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the invention.

Claims

1. A dynamic enhancement and warning system for vehicle blind spots in low-light environments, characterized in that, include: A short-wave infrared imaging module is used to capture passive thermal feature images of the scene behind and to the side of the vehicle. The multi-source signal acquisition module is used to acquire vehicle bus signals that represent the vehicle's motion state and the driver's operating intentions in real time. The data processing and risk assessment module is connected to the shortwave infrared imaging module and the multi-source signal acquisition module. It is used to fuse the thermal feature image and the vehicle bus signal to identify blind spot targets and calculate their dynamic risk level. The graded warning and intervention module is connected to the data processing and risk assessment module and is used to trigger different levels of warnings or vehicle control commands based on the dynamic risk level.

2. The vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The short-wave infrared imaging module includes: The main SWIR imaging unit, using an InGaAs sensor, is installed below the base of the vehicle's exterior rearview mirror. The auxiliary sensing unit includes low-light cameras and safety laser lights mounted on both sides of the rear bumper.

3. The vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The signals acquired by the multi-source signal acquisition module include: Torque value and its rate of change from the steering wheel torque sensor; The status of the turn signal switch from the body control module; Wheel angle signal from ESC or EPS system; Opening signals from the accelerator and brake pedals; The vehicle speed signal from the wheel speed sensor.

4. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The data processing and risk assessment module includes: An image preprocessing unit is used to perform spatial and temporal filtering on the thermal feature image to improve the signal-to-noise ratio; The target extraction and tracking unit is used to cluster moving targets from the filtered image and establish a tracking trajectory for each target; The multi-feature fusion computing unit is used to calculate the relative distance, relative velocity and direction of motion of the target based on the target's tracking trajectory, and to fuse the target's thermodynamic and morphological features to generate an initial risk score; The intent recognition and risk coupling unit is used to calculate the driver's lane-changing intent confidence level based on the vehicle bus signal, and couple the confidence level with the initial risk score to output the final dynamic risk level.

5. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 4, characterized in that, The execution logic of the image preprocessing unit includes: The spatial domain filtering processing subunit uses an adaptive bilateral filtering algorithm that incorporates a thermal gradient weighting factor to process a single frame of SWIR image. The specific steps include: Calculate the thermal gradient value H1 between each pixel in the image and its neighboring pixels; Set a predetermined thermal gradient threshold H0; When the thermal gradient value H1 is less than the predetermined thermal gradient threshold H0, it is determined to be a low thermal gradient region, and the intensity variance is increased. To enhance smoothing and noise reduction, and increase range variance. ; When the thermal gradient value H1 is greater than or equal to the predetermined thermal gradient threshold H0, it is determined to be a high thermal gradient region, and the intensity variance is reduced. To preserve edge features and reduce range variance ; Perform bilateral filtering calculations using the adjusted parameters to output a spatially enhanced image; The temporal filtering subunit performs pixel-level cumulative difference operations on a sequence of N consecutive spatially enhanced image frames to generate a mask of salient moving targets. The specific steps include: Perform a difference operation between each frame of the sequence and its previous frame to obtain a binarized difference image; Perform a logical OR operation on the binary difference maps of N consecutive frames to generate a cumulative motion region map; A morphological opening operation is performed on the accumulated motion region map to eliminate isolated pixels caused by noise and smooth the contour of the moving target. The processed cumulative motion region map is used as the final moving target mask output for subsequent target extraction and tracking.

6. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 4, characterized in that, The execution logic of the target extraction and tracking unit includes: Foreground target extraction subunit: Performs a logical AND operation between the moving target mask generated by temporal filtering and the enhanced image after spatial filtering to extract the foreground image containing pixels of moving heat source; Dynamic parameter clustering subunit: The improved DBSCAN clustering algorithm is used to cluster the pixels in the foreground image. The improvements include: Based on the pixel coordinates of the target in the image, combined with camera calibration parameters, the physical size of the target in the real world is estimated; Establish a mapping function to map the physical dimensions to the clustering radius parameter Eps of the DBSCAN algorithm. in: When the physical size C ≥ the first threshold When, the mapping function outputs Threshold; When the second threshold ≤Physical size C < First threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold; When physical size C < second threshold When, the mapping function outputs Threshold, and the Threshold < the Threshold; The minimum number of points (MinPts) required for core point determination is dynamically adjusted based on the vehicle speed. The higher the vehicle speed, the lower the MinPts value, in order to adapt to the sparser target pixels in high-speed scenes. Target trajectory initialization and tracking subunit: Initialize a tracker for each successfully clustered independent target: A uniform or uniformly accelerated motion model is established for each target using a Kalman filter, and its state vector contains at least the target's position and velocity in the image coordinate system. Calculate the feature similarity between the new target and the existing tracked targets. The features include the target's heat intensity histogram and the aspect ratio of its bounding rectangle. Then, use the Hungarian algorithm to perform optimal matching and association. Each target is assigned a unique tracking ID and its tracking status is managed. Newly detected but unmatched targets are initialized as transient targets and are only promoted to stable targets and output to the risk assessment module after being tracked for M consecutive frames. For lost tracked targets, a delayed destruction mechanism is activated, and tracking is terminated after the target fails to match a detected target for N consecutive frames.

7. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 4, characterized in that, The formula for calculating the initial risk score by the multi-feature fusion calculation unit is as follows: ； In the formula, For the initial risk score; , and These are configurable weighting coefficients; The target relative velocity; The relative distance to the target; These are the type weight coefficients derived from target thermal image classification.

8. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 4, characterized in that, The execution logic of the intent recognition and risk coupling unit includes: Intent confidence calculation subunit: A finite state machine model is established, whose input is the vehicle bus signal, and whose output is an intent confidence level between 0 and 1. Its state transition logic includes: Initial state: When the absolute value of the steering wheel torque |T| < threshold Furthermore, when there is no turn signal, the confidence level is 0; Intended state formation: If the detected steering wheel torque T lasts for Δt longer than the threshold value. But below the higher threshold If the torque change rate dT / dt > 0, the confidence level increases linearly from 0; if a turn signal is detected at the same time, the confidence level jumps to 0.

8. Lane change execution state: If steering wheel torque T > threshold If the wheel angle is detected to be continuously increasing, the confidence level is immediately set to 1 regardless of the turn signal status. Intended cancellation state: In any state, if the brake pedal signal or torque value is detected to drop back to within Δt... The state machine is then reset to its initial state, and the confidence level is set to zero. Dynamic risk calculation subunit: Calculates the effective risk score using the following formula: ； In the formula, For effective risk scoring; The initial risk score; α is the amplification factor; For the confidence level of the intention; Dynamic threshold adjustment subunit: The distance threshold L1 for Level 1 warning is dynamically adjusted based on the vehicle speed V. ; In the formula, As the reference distance, The reference speed is β, which is an adjustment coefficient.

9. A vehicle blind spot dynamic enhancement and warning system for low-light environments according to claim 1, characterized in that, The tiered warning and intervention module includes: The Level 1 Alert Unit is used to display a yellow outline of the target via AR-HUD when the effective risk score exceeds the first threshold. The secondary warning unit is used to trigger a red flashing in the rearview mirror area and a pulse vibration in the driver's seat when the effective risk score exceeds a higher second threshold and the intent confidence is greater than a set threshold. The three-level intervention unit is used to apply a reverse torque through the electric power steering system when the effective risk score exceeds the highest third threshold and a change in wheel angle is detected. The torque value is between 0.2 Nm and 0.5 Nm.

Images