A method, system, vehicle, and computer-readable storage medium for dynamic collision risk assessment of autonomous driving systems.

By combining infrared thermal imaging with motion prediction, a dynamic risk assessment model of obstacle thermal field characteristics and motion state is constructed, which solves the defects of obstacle recognition and collision risk assessment in intelligent driving systems and realizes stable and safe driving in complex environments.

CN123069096APending Publication Date: 2026-11-10DONGGUAN XINTAI INSTRUMENT CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610909803.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-06-23
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing intelligent driving systems lack the ability to continuously track obstructions and perform collision risk assessment in environments such as nighttime and inclement weather. They also have a single response strategy and cannot execute differentiated vehicle responses based on dynamic changes, leading to increased safety risks.

Method used

By acquiring thermal image sequences from an infrared thermal imager, heat-generating obstacles are detected and thermal field features are extracted. Combined with motion prediction parameters, Kalman filtering or LSTM models are used to predict the motion state of obstacles, a risk assessment model is constructed, differentiated vehicle responses are executed, and a continuous thermal mass tracking mode is activated when obstacles are obstructed, achieving seamless transition.

Benefits of technology

It achieves stable obstacle identification and dynamic risk assessment in complex environments, improves system stability and safety, avoids over- or under-response, and enhances driving comfort and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123069096A_ABST
    Figure CN123069096A_ABST
Patent Text Reader

Abstract

This invention discloses a method, system, vehicle, and computer-readable storage medium for dynamic collision risk assessment in autonomous driving, relating to the field of autonomous driving technology. The method includes: acquiring an infrared thermal image sequence; detecting heated obstacles and extracting thermal field features; performing motion prediction based on the thermal centroid sequence; inputting the thermal field features and motion prediction parameters into a risk assessment model to obtain low, medium, and high collision risk levels; executing differentiated vehicle responses according to the risk level; and activating a continuous thermal centroid tracking mode when the heated obstacle is obscured, maintaining a virtual target within a time window and continuing tracking through thermal feature matching after signal recovery. This invention achieves dynamic and accurate collision risk assessment by fusing thermal field features and motion prediction, improves system stability through an anti-occlusion continuous tracking mechanism, and balances safety and comfort through a graded response strategy. It can be used in autonomous driving and advanced driver assistance systems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a method, system, vehicle, and computer-readable storage medium for dynamic classification of collision risk in autonomous driving based on thermal characteristics and motion prediction. Background Technology

[0002] With the development of intelligent driving technology in automobiles, the requirements for obstacle recognition and collision avoidance capabilities in autonomous driving systems and advanced driver assistance systems (ADAS) are constantly increasing. Traditional obstacle detection mainly relies on sensors such as visible light cameras, millimeter-wave radar, and lidar. However, the performance of visible light cameras degrades significantly in adverse environments such as nighttime, rainy or foggy weather, or strong glare; while radar has limited detection capabilities for non-metallic objects or stationary pedestrians. Infrared thermal imagers, due to their passive reception of the target's own thermal radiation, are unaffected by lighting conditions and can effectively detect pedestrians, animals, and other heat-generating objects, becoming an important supplement to the field of intelligent driving perception.

[0003] Existing technologies already include solutions for pedestrian detection using infrared thermal imaging. For example, Chinese invention patent CN111183430B discloses a method for detecting passenger boarding and alighting behavior around a vehicle using thermal distribution; Chinese invention patent application CN111879360A proposes an early warning scheme based on multi-sensor semantic segmentation in dark scenes. However, existing solutions have the following shortcomings: 1. Most solutions only determine the presence of a heat-generating object and then trigger a uniform warning or braking operation, without considering the coupling relationship between the heat-generating object's own motion state changes and the evolution of the thermal field. For example, the collision risk of a pedestrian walking towards the lane is significantly different from that of a pedestrian walking away from the lane, but in existing technologies, they are often assigned the same danger level.

[0004] 2. When a hot object (such as a pedestrian) is briefly obscured by obstacles such as other vehicles or buildings, the infrared thermal imaging signal will be temporarily interrupted. Existing systems typically clear the target's status information and re-identify it when it reappears, resulting in discontinuous tracking and increasing safety risks.

[0005] 3. Existing systems typically only provide warnings or a single level of emergency braking, failing to execute differentiated vehicle responses based on dynamically changing collision risks, which can easily lead to unnecessary emergency braking or insufficient warnings.

[0006] Furthermore, after identifying a heated obstacle, most existing systems only perform braking to avoid it. However, in certain scenarios (such as when an obstacle cuts in laterally at high speed and the braking distance is insufficient), simple braking cannot prevent a collision and may even cause a rear-end collision. A few systems attempt to introduce steering avoidance, but they usually rely on visible light or radar alone for decision-making, failing to fully utilize the thermal field characteristics of infrared thermal imaging to accurately determine the lateral movement trend of the heated target. They also lack a steering closed-loop correction mechanism based on thermal image pixel deviations, resulting in inaccurate steering timing and angle, which may actually lead to new collision risks. Summary of the Invention

[0007] The present invention aims to provide an intelligent driving obstacle handling method that can combine the thermal field characteristics of a heated object with motion state prediction for dynamic risk classification, and has continuous occlusion tracking capability and graded response capability. The system, vehicle and computer-readable storage medium are designed to solve the technical problems of static risk assessment, lack of continuous occlusion tracking capability and single response strategy in existing obstacle recognition and collision risk assessment methods based on infrared thermal imaging in intelligent driving systems.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a dynamic classification method for collision risk of autonomous driving based on thermal characteristics and motion prediction, comprising the following steps: S1: Acquire a sequence of thermal images of the area in front of and around the vehicle, continuously collected by an infrared thermal imager; S2: Detect heat-generating obstacles from the thermal image sequence and extract the thermal field features of each heat-generating obstacle; the thermal field features include, but are not limited to, temperature distribution maps, thermal contour information, and thermal centroid coordinate sequences; S3: Based on the thermal centroid coordinate sequence, predict the motion state and generate motion prediction parameters for each thermal obstacle; the motion prediction parameters include predicted velocity vector, predicted acceleration, and predicted trajectory distribution. S4: Input the thermal field characteristics and the motion prediction parameters into a pre-trained risk assessment model, and the risk assessment model outputs the collision risk level of the thermal obstacle; the risk level includes at least low risk level, medium risk level and high risk level. S5: Implement differentiated vehicle response strategies based on the collision risk level; Specifically, when any heat-generating obstacle is temporarily missing in the thermal image sequence due to external occlusion, the continuous thermal tracking mode is activated: based on the motion prediction parameters of the obstacle at the moment before its disappearance, the virtual target state is continuously maintained within a preset time window; if the infrared signal of the obstacle reappears within the time window and its thermal field characteristics match the prediction parameters, the tracking is automatically resumed; if the obstacle does not recover beyond the time window, the tracking is terminated and the status information of the obstacle is archived.

[0009] The thermal centroid refers to the weighted geometric center of a heated obstacle in an infrared thermal image, calculated using its temperature value as the weight. The thermal centroid coordinate sequence is a set of two-dimensional coordinate points (x, y) formed by the change of this center point over time (multiple consecutive frames). By analyzing the changes in the thermal centroid coordinate sequence over time, the instantaneous velocity, acceleration, and trajectory of the target can be calculated, thereby predicting its future position. When the target is briefly occluded, the virtual target is maintained based on the thermal centroid's motion trend (velocity and direction) before it was lost, so that it can be matched after restoration.

[0010] Thermal profiles refer to the shape information formed by the boundaries of regions in an infrared thermal image where the temperature of a heated obstacle differs significantly from the background environment. The shape features of the thermal profile (such as head-to-torso ratio and limb extension direction) can distinguish pedestrians, cyclists, animals, or non-biological heat sources (such as exhaust pipes and engines). For post-occlusion matching, in continuous thermal mass tracking mode, when a target reappears, the system calculates the Intersection over Union (IoU) between the newly detected object's thermal profile and the virtually predicted thermal profile before occlusion. If the IoU is greater than a threshold (e.g., 0.6), the two are considered to be the same target, thus enabling continued tracking. As a target approaches or moves away from the vehicle, its thermal profile area in the image increases or decreases; the rate of change in thermal profile area can be used to assist in estimating distance changes and collision time.

[0011] A further technical solution of the present invention: the thermal field characteristics also include: the highest temperature, average temperature, temperature distribution variance, and thermal profile area change rate of the heat-generating obstacle.

[0012] A further technical solution of the present invention: the motion state prediction in step S3 is implemented by using the Kalman filter algorithm or a time-series prediction model based on a long short-term memory network (LSTM).

[0013] A further technical solution of the present invention: the risk assessment model is a joint model of convolutional neural network and long short-term memory network, wherein the convolutional neural network is used to extract spatial thermal field features in thermal images, and the long short-term memory network is used to process the time series information of motion prediction parameters.

[0014] A further technical solution of the present invention: the classification of the collision risk level is based on the following criteria: the estimated time of collision (TTC) between the heated obstacle and the vehicle, the angle between the direction of motion of the heated obstacle and the direction of travel of the vehicle, and the temperature pulsation frequency characterizing biological activity in the thermal field characteristics of the heated obstacle.

[0015] A further technical solution of the present invention: the differentiated vehicle response strategy includes: When the risk level is low, only routine driving monitoring is performed, and the vehicle driving parameters are not changed. When the risk level is medium risk, both voice and / or visual warnings will be issued, and the braking system will be pre-pressurized, but active braking will not be triggered. When the risk level is high, the system will actively activate the braking system to perform graded braking and deceleration while issuing a warning, and dynamically adjust the deceleration according to the real-time position changes of the heated obstacle.

[0016] A further technical solution of the present invention: In the continuous thermal mass tracking mode, the time window is 0.5 seconds to 2 seconds, and the length of the time window is dynamically adjusted according to the current vehicle speed: the higher the vehicle speed, the shorter the time window.

[0017] A further technical solution of the present invention: the judgment conditions for matching the thermal field characteristics with the prediction parameters include: the overlap between the thermal profile of the reappearing target and the predicted thermal profile is greater than a preset first threshold, and the deviation between the average temperature of the reappearing target and the predicted average temperature is less than a preset second threshold.

[0018] A further technical solution of the present invention: when the collision risk level is high risk and the steering avoidance conditions are met, the method further includes a steering follow-up obstacle avoidance step: S6: By combining the radar unit and the infrared thermal imager, the precise lateral velocity and relative distance change rate of the heated obstacle are obtained; S7: Input the lateral velocity, relative distance change rate and the current vehicle speed into the steering decision model to generate the target steering angle and steering angular velocity curve; S8: During the steering process, based on the deviation between the real-time pixel position of the heated obstacle in the thermal image and the preset safety window, the target steering angle is corrected in a closed loop until the heated obstacle leaves the safe area where the vehicle is traveling.

[0019] A further technical solution of the present invention: the steering avoidance conditions include: the current speed of the vehicle is higher than a first speed threshold, the absolute value of the lateral velocity of the heated obstacle is greater than a second speed threshold, and the braking avoidance distance calculated based on the current braking deceleration is greater than the actual distance between the vehicle and the obstacle.

[0020] A further technical solution of the present invention: the radar unit includes lidar and / or millimeter-wave radar, the infrared thermal imager and the radar unit achieve spatial coordinate system alignment through joint calibration, and target-level association matching is performed through the Hungarian algorithm.

[0021] A further technical solution of the present invention: The steering decision model is a policy network pre-trained based on deep reinforcement learning. Its input state space includes the target lateral velocity, the rate of change of relative distance, the vehicle speed, and the vehicle yaw rate. The output action space is the front wheel steering angle increment and the steering angle rate.

[0022] A further technical solution of the present invention: The preset safety window is a dynamic area in the thermal image relative to the center line of the vehicle. When the pixel coordinates of the heated obstacle deviate from the window, the steering angle correction is triggered. The correction amount is proportional to the deviation amount, and a maximum correction limit is set.

[0023] The present invention also provides an autonomous driving collision risk dynamic rating system, comprising: Infrared thermal image acquisition module, used to acquire thermal image sequences of the environment surrounding the vehicle; The thermal field feature extraction unit is used to detect heat-generating obstacles and extract thermal field features from the thermal image sequence; The motion state prediction module is used to predict the motion parameters of heat-generating obstacles based on historical thermal centroid coordinate sequences. The risk assessment model unit is used to output the collision risk level based on the thermal field characteristics and motion prediction parameters. A continuous thermal mass tracking module is used to maintain the virtual target when the thermal obstacle is occluded and to perform matching continuation when the signal is restored; A graded response execution unit is used to execute differentiated vehicle response strategies based on the collision risk level.

[0024] The present invention also provides an autonomous driving collision risk dynamic rating system, characterized in that it includes: Infrared thermal image acquisition module, used to acquire thermal image sequences of the environment surrounding the vehicle; The thermal field feature extraction unit is used to detect heat-generating obstacles and extract thermal field features from the thermal image sequence; The motion state prediction module is used to predict the motion parameters of heat-generating obstacles based on historical thermal centroid coordinate sequences. The risk assessment model unit is used to output the collision risk level based on the thermal field characteristics and motion prediction parameters. A continuous thermal mass tracking module is used to maintain the virtual target when the thermal obstacle is occluded and to perform matching continuation when the signal is restored; A graded response execution unit is used to execute differentiated vehicle response strategies according to the collision risk level. The steering follow-up obstacle avoidance unit is used to generate and correct the steering angle in a closed loop based on infrared and radar fusion perception when the steering avoidance conditions are met.

[0025] The present invention also provides a vehicle including the above-described dynamic collision risk rating system.

[0026] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0027] Compared with existing technologies, the beneficial effects of this technical solution are: By jointly analyzing the thermal field characteristics and motion state prediction of heat-generating obstacles, a multi-dimensional collision risk level assessment model was constructed. This model can distinguish the actual degree of danger corresponding to heat-generating objects with different motion trends, thus avoiding over- or under-response caused by unified early warning. Continuous thermal mass tracking can maintain the virtual target state by relying on motion prediction when the thermal target is briefly occluded, and achieve seamless continuation through thermal feature matching after the signal is restored. This eliminates decision jitter caused by temporary loss of the target and improves the stability and security of the system. Implementing differentiated vehicle control based on risk level avoids unnecessary intervention in low-risk scenarios while ensuring rapid and effective response in high-risk scenarios, thus improving the balance between driving comfort and safety. By extracting thermal field features such as temperature distribution, thermal profile changes, and thermal centroid coordinates of the heating object, it is possible to effectively distinguish between living and non-living heat sources and reduce the false alarm rate.

[0028] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of the collision risk dynamic classification method of the present invention; Figure 2 This is a circuit block diagram of the collision risk dynamic classification system of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] See Figure 1 This embodiment provides a method for dynamic classification of collision risk in autonomous driving, and the specific steps are as follows.

[0033] Step S1: Obtain an infrared thermal image sequence.

[0034] At least one infrared thermal imager is installed on the vehicle, typically positioned at the front of the roof or behind the front grille, with a field of view covering a certain area in front of and to the left and right sides of the vehicle. The infrared thermal imager continuously acquires 14-bit or 16-bit raw thermal image data at a frame rate of 25Hz or higher, and outputs a temperature matrix after non-uniformity correction (NUC) and bad pixel compensation. In this embodiment, an infrared thermal imager can be selected to achieve a thermal image resolution of 640×512 pixels and a temperature sensitivity better than 50mK.

[0035] Step S2: Detect heat-generating obstacles and extract thermal field features.

[0036] Target detection is performed on each frame of thermal image. A single-stage detection network based on deep learning (such as YOLOv5-tiny thermal imaging version) or a traditional threshold segmentation + morphological processing method can be used to extract all connected regions that are higher than the ambient temperature threshold (e.g., ambient temperature + 5℃) and have an area larger than a preset minimum area (e.g., 200 pixels) as candidate thermal obstacles.

[0037] For each candidate thermal obstacle, the following thermal field features are extracted: 1. Temperature distribution map: The temperature value of each pixel in the region can be downsampled into a 32×32 temperature field matrix; 2. Thermal profile information: minimum bounding rectangle of the obstacle, profile perimeter, ellipse fitting parameters (major axis, minor axis, orientation); 3. Thermal centroid coordinate sequence: Temperature-weighted centroid coordinates (x_c, y_c) within the region, calculated using the following formula: x_c=Σ(T_i*x_i) / ΣTi y_c=Σ(T_i*y_i) / ΣTi Where T_i is the temperature value of pixel i, and (x_i, y_i) is its image coordinate.

[0038] 4. Maximum temperature T_max, average temperature T_avg, temperature distribution variance σ_T²; 5. Thermal profile area change rate: The ratio of the difference in thermal profile area between adjacent frames to the time interval dA / dt.

[0039] Step S3: Motion state prediction.

[0040] Based on the thermal centroid coordinate sequence of the current frame and historical frames (e.g., the previous 5 frames), a Kalman filter is used to predict the target's position and velocity in the next frame (t+Δt) and at future times of 0.5s, 1s, and 2s. The state vector of the Kalman filter is set as [x, y, v_x, v_y, a_x, a_y]. The observation vector is [x_c, y_c]. By recursively applying the state transition matrix and the observation matrix, the predicted velocity vector (v_x_pred, v_y_pred), the predicted acceleration (a_x_pred, a_y_pred), and the predicted trajectory distribution (the uncertainty region represented by a covariance ellipse) are output.

[0041] Alternatively, an LSTM network can be used for temporal prediction: extract the thermal centroid sequence and thermal contour area sequence of each target from the previous 10 frames, input them into an LSTM layer with 64 hidden units, and output the position sequence of future multiple frames.

[0042] Step S4: Input the risk assessment model to obtain the collision risk level.

[0043] The risk assessment model in this embodiment is a joint CNN-LSTM model. Its specific structure is as follows: The input layer receives two data streams simultaneously: one is the thermal image cropping region of the current frame (32×32×1), and the other is the motion prediction parameter sequence (predicted velocity vector and acceleration vector with a length of 5 frames and a shape of 5×4).

[0044] CNN branch: Three convolutional layers (32 / 64 / 128 channels, 3×3 convolutional kernel, ReLU activation), followed by global average pooling, outputting a 128-dimensional feature vector.

[0045] LSTM branch: One LSTM layer (64 units) processes the time series of motion parameters and outputs a 64-dimensional temporal feature vector.

[0046] Fusion layer: The 128-dimensional spatial features and 64-dimensional temporal features are concatenated and input into two fully connected layers (128→64→3). The last layer uses softmax activation and outputs the probability distribution of three categories, corresponding to low risk, medium risk and high risk, respectively.

[0047] This model is pre-trained on a large amount of autonomous driving scenario data (including thermal image sequences, motion state annotations, and accident data), and the loss function is cross-entropy loss.

[0048] Step S5: Implement differentiated vehicle response strategies.

[0049] Based on the risk level output by the risk assessment model, the vehicle response logic is as follows: Low-risk level (e.g., predicted collision time TTC > 5 seconds, and the angle between the target's direction of motion and the vehicle's heading > 45°): The system only performs routine driving monitoring, does not change vehicle driving parameters, and does not issue warnings, in order to reduce unnecessary intervention.

[0050] Medium risk level (e.g., TTC between 2 and 5 seconds, or the target approaching the lane): The system will flash the instrument panel icon / or the central control screen and / or give a voice prompt "Watch out for pedestrians ahead", while sending a brake pre-pressure command to the electronic stability control system (ESC) to build up a pre-pressure of 0.5 to 1 MPa in the master cylinder, shortening the response time during emergency braking, but without triggering actual braking action.

[0051] High-risk level (e.g., TTC < 2 seconds, and the target's movement direction is pointing towards the vehicle's path): While issuing the highest-level alarm (e.g., continuous beeping + full-screen red flashing), the system actively activates the braking actuator for graded braking. The specific strategy for graded braking is as follows: the initial deceleration is 0.2g; if the TTC is still less than 1.5 seconds, it increases to 0.5g; if the TTC is less than 0.8 seconds, the maximum deceleration (0.8~1.0g) is applied. During braking, the system continuously monitors changes in the target's position; if the target suddenly moves away, braking is disengaged.

[0052] Step S20: Continuous thermal mass tracking processing (parallel or interrupt-triggered).

[0053] At any given time, if a previously tracked thermal obstacle is not detected for N consecutive frames (e.g., 3 frames) (i.e., the infrared signal is lost), it is considered occluded. In this case, the system does not immediately remove the target, but instead activates the continuous thermal tracking mode. Step S201: Record the complete state of the target at the moment before the occlusion occurs, including the latest thermal field features (T_max, T_avg, thermal profile template) and motion prediction parameters (velocity vector, acceleration).

[0054] Step S202: Set the time window T_window = max(0.5 seconds, 2.0–0.1×V_ego) seconds, where V_ego is the vehicle speed (m / s). The higher the vehicle speed, the shorter the window, to avoid false targets being maintained for a long time and causing misjudgment.

[0055] Step S203: Within this time window, continue to use the Kalman filter to recursively predict the target's position and thermal profile changes according to the original motion model, forming a virtual target. The virtual target does not participate in the collision risk level calculation, but its state is retained for subsequent follow-up.

[0056] Step S204: Within the time window, if the infrared thermal imager detects a candidate heat-generating object again, calculate the overlap (IoU) between the thermal profile of the object and the predicted thermal profile of the virtual target, as well as the average temperature deviation ΔT_avg. If the overlap > 0.6 and ΔT_avg < 2.0℃, the match is considered successful.

[0057] Step S205: After a successful match, the ID of the virtual target and the continuous motion prediction state are passed to the newly detected target to achieve seamless tracking continuity, and the risk level calculation is restored to use real observation data.

[0058] Step S206: If a match is not found within the time window, the virtual target is terminated and its final state is recorded in the historical database for subsequent analysis.

[0059] Through the aforementioned continuous thermal tracking mechanism, the system can effectively respond to scenarios where pedestrians briefly appear behind vehicles, trees, or large vehicles, avoiding repeated ID switching and drastic fluctuations in risk levels.

[0060] Based on the aforementioned embodiments, this embodiment further specifies that when the collision risk level is high and braking alone cannot safely avoid a collision, the system will activate the steering-following obstacle avoidance mode.

[0061] Steering and obstacle avoidance condition determination: The main controller calculates the following conditions in real time: 1. The vehicle speed V_ego > 36 km / h (first speed threshold); 2. The absolute value of the lateral velocity of the heated obstacle, |V_lat|, is greater than 1.5 m / s (the second velocity threshold, indicating that the target is crossing or diagonally across the lane). 3. The braking distance D_brake = V_ego² / (2×a_max) is greater than the actual distance R between the vehicle and the obstacle (where a_max is the maximum braking deceleration, taken as 0.8g).

[0062] When all three conditions are met, the system determines that pure braking cannot avoid a collision and switches to steering-assisted obstacle avoidance.

[0063] In this embodiment, the vehicle used can be equipped with a forward-facing millimeter-wave radar and an infrared thermal imager, both of which have been jointly calibrated. For the heat-generating obstacle (high-risk level) detected in step S102, the system maps its pixel coordinates in the thermal image to the radar coordinate system through a calibration matrix and performs Hungarian matching with the point cloud target detected by the radar. After successful matching, the precise radial distance R, radial relative velocity V_rad, and azimuth angle φ of the target are extracted from the radar data, and the lateral velocity is calculated accordingly. V_lat = V_rad × sinφ (approximate formula, which can be smoothed over multiple frames in practice).

[0064] In this embodiment, a pre-trained Deep Q-Network (DQN) is used as the steering decision model. The input state of the model is [V_lat, dR / dt, V_ego,ω] (ω is the yaw rate of the vehicle), and the output is a discretized steering action (including steering direction and angle increment, a total of 9 actions: left turn / right turn each have three amplitudes: small, medium, and large, and no steering).

[0065] In this embodiment, the target steering angle Δθ output by the decision model is sent to the electric power steering (EPS) system of the application vehicle via the CAN bus for execution. During steering, the system continuously tracks the pixel horizontal coordinate u (normalized to the [0,1] interval) of the heated obstacle in the thermal image. The preset safety window is [0.3, 0.7] (e.g., corresponding to a width range of 20% to the left and right of the center of the application vehicle in the image). When u < 0.3 or u > 0.7, it indicates that the obstacle has left the center of the safety window, and the system calculates the deviation e = 0.5 - u (if u < 0.3, e is positive, indicating that a correction to the right is needed; if u > 0.7, e is negative, indicating that a correction to the left is needed). The correction amount Δθ_corr = K_p × e, where K_p = 3°, and the single correction is limited to no more than 5°. The final steering angle sent to the EPS is Δθ_final = Δθ + Δθ_corr. This closed-loop correction ensures that the heated obstacle is always kept within a reasonable area of ​​the vehicle's field of vision throughout the entire avoidance process, avoiding over- or under-steering due to a single open-loop turn.

[0066] In this embodiment, during the steering avoidance process, the system simultaneously monitors the vehicle's lateral acceleration and yaw rate. If it determines that the vehicle is at risk of rollover or fishtailing, it actively reduces the steering angular velocity and applies slight braking to maintain vehicle stability. If the heated obstacle suddenly stops or moves in the opposite direction during the steering process, the system can cancel the steering and revert to braking mode.

[0067] See Figure 2 This embodiment provides a dynamic collision risk rating system for autonomous driving based on thermal characteristics and motion prediction, including: Infrared thermal image acquisition module 310: includes at least one infrared thermal imager and related data interface, and outputs digital thermal image sequence.

[0068] Thermal field feature extraction unit 320: connected to the acquisition module, including target detection subunit, temperature feature calculation subunit, and contour analysis subunit, used to extract thermal field features of each heat-generating obstacle from the thermal image.

[0069] Motion state prediction module 330: contains a Kalman filter or LSTM predictor, and outputs motion prediction parameters based on historical thermal centroid sequences.

[0070] Risk assessment model unit 340: Loads a pre-trained CNN-LSTM joint model, inputs thermal field features and motion prediction parameters, and outputs collision risk level.

[0071] The thermal mass continuous tracking module 350 includes an occlusion detection submodule, a virtual target maintenance submodule, and a matching and continuation submodule, enabling continuous tracking of occluded targets.

[0072] The graded response execution unit 360 includes a human-machine interface (instrument panel, speaker), a braking system interface (CAN bus), and a vehicle dynamics control interface, and outputs corresponding control commands according to the risk level.

[0073] Main controller 370: Responsible for scheduling and data flow management of the above modules, it can be an in-vehicle domain controller or a high-performance computing unit.

[0074] In some embodiments, the system further includes a steering follow-up obstacle avoidance unit, which generates and corrects the steering angle in a closed loop based on infrared and radar fusion perception when steering avoidance conditions are met.

[0075] This embodiment also provides a vehicle that includes the system described in the above embodiments. This system can be integrated into the vehicle's autonomous driving domain controller.

[0076] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, can implement all or part of the steps of the method described in the above embodiments. The storage medium can be a non-volatile medium such as ROM, RAM, solid-state drive, or USB flash drive.

[0077] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for dynamically classifying collision risks in autonomous driving, characterized in that, The method includes the following steps: S1: Acquire a sequence of thermal images of the vehicle's surrounding environment continuously acquired by an infrared thermal imager; S2: Detect heat-generating obstacles from the thermal image sequence and extract the thermal field features of each heat-generating obstacle; the thermal field features include temperature distribution map, thermal profile information, and thermal centroid coordinate sequence; S3: Based on the thermal centroid coordinate sequence, predict the motion state and generate motion prediction parameters for each thermal obstacle; the motion prediction parameters include predicted velocity vector, predicted acceleration, and predicted trajectory distribution. S4: Input the thermal field characteristics and the motion prediction parameters into a pre-trained risk assessment model, and the risk assessment model outputs the collision risk level of the thermal obstacle; the risk level includes at least low risk level, medium risk level and high risk level. S5: Implement differentiated vehicle response strategies based on the collision risk level; Among them, when any heat-generating obstacle is temporarily missing in the thermal image sequence due to external occlusion, the continuous thermal mass tracking mode is activated: based on the motion prediction parameters of the obstacle before its disappearance, the virtual target state is continuously maintained within a preset time window. If the infrared signal of the obstacle reappears within the time window and its thermal field characteristics match the predicted parameters, tracking will automatically resume; if the obstacle does not recover beyond the time window, tracking will terminate and the obstacle's status information will be archived.

2. The method according to claim 1, characterized in that, The thermal field characteristics also include: the highest temperature, average temperature, temperature distribution variance, and thermal profile area change rate of the heat-generating obstacle.

3. The method according to claim 1, characterized in that, In step S3, motion state prediction is achieved using a Kalman filter algorithm or a time-series prediction model based on a long short-term memory network.

4. The method according to claim 1, characterized in that, The risk assessment model is a joint model of convolutional neural network and long short-term memory network, in which the convolutional neural network is used to extract spatial thermal field features in thermal images, and the long short-term memory network is used to process the time series information of motion prediction parameters.

5. The method according to claim 1, characterized in that, The classification of collision risk levels is based on: the expected collision time between the heated obstacle and the vehicle, the angle between the direction of movement of the heated obstacle and the direction of travel of the vehicle, and the temperature pulsation frequency that characterizes biological activity in the thermal field characteristics of the heated obstacle.

6. The method according to claim 1, characterized in that, The differentiated vehicle response strategy includes: When the risk level is low, only routine driving monitoring is performed, and the vehicle driving parameters are not changed. When the risk level is medium risk, both voice and / or visual warnings will be issued, and the braking system will be pre-pressurized, but active braking will not be triggered. When the risk level is high, the system will actively activate the braking system to perform graded braking and deceleration while issuing a warning, and dynamically adjust the deceleration according to the real-time position changes of the heated obstacle.

7. The method according to claim 1, characterized in that, The criteria for determining whether the thermal field characteristics match the predicted parameters include: the overlap between the thermal profile of the reappearing target and the predicted thermal profile is greater than a preset first threshold, and the deviation between the average temperature of the reappearing target and the predicted average temperature is less than a preset second threshold.

8. The method according to claim 1, characterized in that, When the collision risk level is high and the steering avoidance conditions are met, the method further includes: S6: By combining the radar unit and the infrared thermal imager, the precise lateral velocity and relative distance change rate of the heated obstacle are obtained; S7: Input the lateral velocity, relative distance change rate and the current vehicle speed into the steering decision model to generate the target steering angle and steering angular velocity curve; S8: During the steering process, based on the deviation between the real-time pixel position of the heated obstacle in the thermal image and the preset safety window, the target steering angle is corrected in a closed loop until the heated obstacle leaves the safe area where the vehicle is traveling.

9. The method according to claim 8, characterized in that, The steering and obstacle avoidance conditions include: The vehicle's current speed is higher than the first speed threshold, the absolute value of the lateral velocity of the heated obstacle is greater than the second speed threshold, and the braking avoidance distance calculated based on the current braking deceleration is greater than the actual distance between the vehicle and the obstacle.

10. The method according to claim 8, characterized in that, The radar unit includes lidar and / or millimeter-wave radar. The infrared thermal imager and the radar unit are aligned in spatial coordinate system through joint calibration and target-level correlation matching is performed through the Hungarian algorithm.

11. The method according to claim 8, characterized in that, The steering decision model is based on a policy network pre-trained by deep reinforcement learning. Its input state space includes the target lateral velocity, the rate of change of relative distance, the vehicle speed, and the vehicle yaw rate. Its output action space includes the front wheel steering angle increment and the steering angle rate.

12. The method according to claim 8, characterized in that, The preset safety window is a dynamic area in the thermal image relative to the center line of the vehicle. When the pixel coordinates of the heated obstacle deviate from this window, the steering angle correction is triggered. The correction amount is proportional to the deviation amount, and a maximum correction limit is set.

13. A dynamic collision risk rating system for autonomous driving, characterized in that, include: Infrared thermal image acquisition module, used to acquire thermal image sequences of the environment surrounding the vehicle; The thermal field feature extraction unit is used to detect heat-generating obstacles and extract thermal field features from the thermal image sequence; The motion state prediction module is used to predict the motion parameters of heat-generating obstacles based on historical thermal centroid coordinate sequences. The risk assessment model unit is used to output the collision risk level based on the thermal field characteristics and motion prediction parameters. A continuous thermal mass tracking module is used to maintain the virtual target when the thermal obstacle is occluded and to perform matching continuation when the signal is restored; A graded response execution unit is used to execute differentiated vehicle response strategies based on the collision risk level.

14. A dynamic collision risk rating system for autonomous driving, characterized in that, include: Infrared thermal image acquisition module, used to acquire thermal image sequences of the environment surrounding the vehicle; The thermal field feature extraction unit is used to detect heat-generating obstacles and extract thermal field features from the thermal image sequence; The motion state prediction module is used to predict the motion parameters of heat-generating obstacles based on historical thermal centroid coordinate sequences. The risk assessment model unit is used to output the collision risk level based on the thermal field characteristics and motion prediction parameters. A continuous thermal mass tracking module is used to maintain the virtual target when the thermal obstacle is occluded and to perform matching continuation when the signal is restored; A graded response execution unit is used to execute differentiated vehicle response strategies according to the collision risk level. The steering follow-up obstacle avoidance unit is used to generate and correct the steering angle in a closed loop based on infrared and radar fusion perception when the steering avoidance conditions are met.

15. A vehicle, characterized in that, Includes the system described in claim 13 or 14.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

Citation Information

Patent Citations

  • Identification processing device, identification processing method and recording medium

    CN111183430B

  • Automatic driving auxiliary safety early warning system in dark scene and early warning method thereof

    CN111879360A