A Method and System for Active Safety Control of Blind Spots in Semi-Trailers Based on Multimodal Fusion

Abstract

This invention belongs to the field of vehicle braking control systems and relates to a method and system for active safety control of blind spots in semi-trailers based on multimodal fusion. The system includes: a non-implantable electrode EEG headband, a camera, a contact sensor, dry electrodes, a data processing module, an attention calculation module, a driving style detection module, an EEG intent correction module, an expected control quantity correction module, a dynamic hazard zone prediction module, a target hazard zone generation module, a projection light, an environmental perception module, and an intervention module. Through closed-loop control of implicit intent prediction, line-of-sight verification, style adaptation, dynamic projection, and graded intervention, it solves key technical problems such as reaction lag, intent misjudgment, insufficient visualization, and uncoordinated intervention in the active safety system for blind spots in semi-trailers, significantly improving the safety, real-time performance, and environmental adaptability of semi-trailers in complex scenarios such as starting, turning, and reversing.

Description

Technical Field

[0001] This invention belongs to the field of vehicle braking control systems and relates to active safety control, specifically to a method and system for active safety control of blind spots in semi-trailers based on multimodal fusion. Background Technology

[0002] Due to their size, semi-trailers have large blind spots in various parts of their field of vision. When the vehicle performs complex actions such as starting, changing lanes, turning, and reversing, if there are pedestrians, non-motorized vehicles, or other obstacles in the blind spots, it is very easy to cause traffic accidents. Therefore, it is crucial to promptly and correctly alert semi-trailer drivers that there are obstacles in the blind spots and to remind pedestrians and non-motorized vehicle drivers that they are currently in the semi-trailer's blind spots.

[0003] There are already some related studies, for example: Chinese patent CN121011110B proposed a low-speed blind spot monitoring method and system based on multi-view camera; Chinese patent CN119611219A proposed a vehicle blind spot monitoring system and truck; Chinese patent CN117734567A proposed a vehicle blind spot dynamic area light marking device. However, the current related studies still have the following shortcomings: (1) System response lag: Existing studies mainly rely on vehicle status, such as steering wheel angle and pedal force, to judge intent. Intervention can only be carried out when the driver takes explicit operations such as stepping on the accelerator and turning the steering wheel, resulting in system lag. (2) Single driver intent judgment: Ignoring the driver's implicit intent and lacking prediction of the driver's implicit intent, resulting in a single judgment of the driver's intent and lack of accuracy. (3) Insufficient blind spot visualization: For the dynamic projection of the blind spot, the influence of terrain, road surface condition and road surface material on the projection accuracy is not considered. (4) The tiered intervention strategy lacks coordination, and the transitions between different levels of intervention are abrupt (such as jumping directly from no intervention to emergency braking). It does not fully consider the unique pushing effect of trailers and the protection of pedestrians behind them, which can easily lead to trailer folding, tail swinging, or secondary injuries. It is evident that the current active safety control for semi-trailers still has shortcomings in terms of real-time performance and accuracy. Summary of the Invention

[0004] In view of the shortcomings and deficiencies of the existing technology, the purpose of this invention is to provide a method and system for active safety control of blind spots in semi-trailers based on multimodal fusion. This method solves the key technical problems of reaction lag, intention misjudgment, insufficient visualization, and uncoordinated intervention in the active safety system of semi-trailers through closed-loop control of implicit intention prediction, line-of-sight verification, style adaptation, dynamic projection and hierarchical intervention. It significantly improves the safety, real-time performance and environmental adaptability of semi-trailers in complex scenarios such as starting, turning and reversing.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for active safety control of blind spots in semi-trailers based on multimodal fusion, comprising the following steps: Step S1. Use an electrode EEG headband to acquire signals, extract slow cortical potential features and event-related desynchronization features, and use the SVM+Platt scaling method to estimate the original probability of various latent intentions; Step S2. Calculate the average attention level of the key sector set corresponding to various implicit intentions based on the gaze direction and the ground landing point, and extract eye movement parameters at the same time; Step S3. Collect physiological signals; Step S4. Obtain long-term driving style. Based on the data obtained in steps S2 and S3, complete short-term driving style detection, fuse long-term and short-term driving styles, and correct them based on the semi-trailer load. Step S5. Correct the original implicit intention probability and average attention, fuse the corrected original implicit intention probability, line of sight and style, calculate the fused intention probability after line of sight and style correction, and output the final implicit intention; at the same time, correct the original expected control quantity based on the corrected driving style, and recalculate the expected vehicle speed and steering wheel angle in combination with the final implicit intention. Step S6. Based on the expected vehicle speed, steering wheel angle and semi-trailer kinematic model, predict the future driving trajectory of the vehicle, generate a dynamic danger zone including the inner wheel differential zone according to the predicted trajectory, and project it onto the road surface using a projection light. Step S7. Determine the driver's state based on the attention level of the key sector corresponding to the final implicit intention. Calculate the comprehensive hazard index and style offset by combining obstacle information, driver state, and driving style. Trigger different levels and execute different control commands according to the comprehensive hazard index and style offset.

[0006] As a preferred embodiment of the present invention, the slow cortical potential characteristics in step S1 The expression is: ; in, Number of sampling points within the window; Sampling interval; : Window start time; : Filtered signal from electrodes, non-implantable electrode EEG headband The electrode is located on the center line of the central area; : Resting state Average voltage of electrodes over 5 seconds; Event-related desynchronization features The method of obtaining it is: ; in, : Average power over time Resting-state reference power Non-implantable electrode EEG headband The electrode is located in the central area on the left side; The electrode is located in the central area on the right side; The electrode is located on the midline of the forehead region; The electrode is located at the midline of the top region.

[0007] As a preferred embodiment of the present invention, the implicit intent category set in step S1 The original probabilities of various implicit intentions are estimated using the SVM+Platt scaling method, followed by smoothing, and the articulation angular velocity of the semi-trailer is introduced. As a threshold: if Then the probability of the "start / accelerate" intention will be forced. Multiply by a coefficient of 0.2; if the semi-trailer's reverse gear is activated, then the probability of forcibly performing the "reverse" intention is increased. Set it to 1.0, and set the rest to zero.

[0008] As a preferred embodiment of the present invention, in step S2, when calculating the average attention, the unit vector of the gaze direction in the vehicle coordinate system is first calculated, and then the gaze starting point, i.e., the coordinates of the eye in the vehicle coordinate system, is obtained; then the coordinates of the gaze ground landing point in the vehicle coordinate system are calculated based on the gaze parameter equation; finally, the ground around the vehicle is divided into multiple sector areas, the cumulative duration of the ground landing point in each sector area within a specified time period is counted, and the attention of each sector area and the average attention of the key sector area set corresponding to various implicit intentions are calculated.

[0009] As a preferred embodiment of the present invention, the eye movement parameters in step S2 include eye movement frequency, blink frequency, and saccade velocity; the physiological signals in step S3 include finger temperature and skin conductance response; in step S4, when performing short-term driving style detection based on the data obtained in steps S2 and S3, the current deviation is calculated according to the baseline median and baseline standard deviation of each indicator, and then the deviation is mapped to... The range is considered, and the direction is taken into account, so that each indicator is transformed into an "aggressive tendency factor". Then, the five aggressive tendency factors are merged to calculate the short-term style factor. After low-pass filtering, it is merged with the long-term style factor to obtain the comprehensive driving style factor. Finally, the comprehensive driving style factor is corrected according to the trailer's current total mass, unloaded mass and maximum permissible mass to obtain the corrected comprehensive driving style factor.

[0010] As a preferred embodiment of the present invention, in step S4, before driving style fusion, a style stability judgment is first performed. If the absolute deviation between the short-term style factor and the long-term style factor is always less than the threshold within a specified period of time, the weight of the long-term style is increased during driving style fusion. If the standard deviation of the short-term style factor exceeds the threshold within a specified period of time, the comprehensive driving style factor is forcibly set to neutral, and normal fusion is resumed after the standard deviation of the short-term style factor falls back below the threshold.

[0011] As a preferred embodiment of the present invention, in step S5, the average attention to the key sector set corresponding to the implicit intention is modified based on the semi-trailer articulation angle to obtain a weighted attention; the left-turn and right-turn smooth probabilities in the original implicit intention probability estimated in step 1 are modified based on the lateralization coefficient and normalized, wherein the lateralization coefficient is calculated based on the event-related desynchronization feature value; then, the load-corrected comprehensive driving style factor is fused. The probability of implicit intention after normalization and correction Weighted attention Calculate the probability of fusion intention after gaze and style correction. The expression is: ; in, κ Visual enhancement factor; ξ Style influence intensity coefficient; : Indicator function, takes the value 1 when the intent is aggressively related, otherwise takes the value 0; The indicator function takes the value 1 if the intent is "aggressive and relevant intent", and 0 otherwise. Number of implicit intent categories; The expected control quantities include expected acceleration and steering rate. The original expected acceleration and steering rate are estimated based on EEG characteristics. The original expected acceleration and steering rate are then corrected based on the modified comprehensive driving style factor.

[0012] As a preferred embodiment of the present invention, in step S5, an intent-attention inconsistency penalty is performed. If the weighted attention corresponding to the current fusion decision implicit intent is less than a set threshold and the original implicit intent probability is greater than a set threshold, a timer is started. When the time exceeds the set time limit, a penalty operation is performed on the fusion intent probability, and the final implicit intent is re-output.

[0013] As a preferred embodiment of the present invention, in step S6, the vehicle's trajectory sequence is obtained based on the expected vehicle speed and steering wheel angle. For each moment, all contour points of the tractor and trailer are projected onto the ground to obtain the occupied area polygon at that moment. The dynamic danger zone is the union of all contour points within a future set time, including the inner wheel differential strip. For turning scenarios, the width of the inner wheel differential strip is calculated. If the lateral slope is greater than... Then, the inner wheel differential band width is corrected, and the offset trajectory of the inner rear wheel of the trailer is calculated based on the corrected inner wheel differential band width, and the dynamic danger zone is updated; at the same time, a style-driven danger zone expansion is introduced. If the corrected comprehensive driving style factor is greater than 0, the dynamic danger zone is expanded outward.

[0014] As a preferred embodiment of the present invention, during the projection in step S6, a projected image is generated by reverse mapping, a ground elevation network around the vehicle is constructed in real time, the ground conditions are identified by the echo intensity of a multispectral camera or lidar, the effective pixel colors are adjusted based on the ground conditions and the projection distance, and the actual output brightness and flicker frequency of the projection lamp are dynamically adjusted according to the current comprehensive hazard index.

[0015] As a preferred embodiment of the present invention, in step S6, a fixed blind spot is preset, which is a fixed area that the driver cannot directly observe in a normal driving posture. The final output target danger area is the union of the dynamic danger area and each fixed blind spot. The projection lamp projects the fixed blind spot and the dynamic danger area onto the actual terrain in turn. When the intention-attention inconsistency penalty is executed in step S5, the brightness and flashing frequency of the projection are increased to the second-level warning level, and the projection of the dynamic danger area is exclusively occupied. When the hinge angle change rate exceeds the threshold, the projection of the right inner wheel difference is exclusively occupied.

[0016] As a preferred embodiment of the present invention, the comprehensive risk index in step S7 The expression is: ; in, Distance factor weights; Collision time factor weight; Intent confidence weight; Attention factor weights; Style risk correction weight; : Fusion confidence, fusion confidence at time t , The probability of the intended fusion after adjustments for perspective and style; Distance factor, based on the minimum distance from the obstacle to the boundary of the projection area. Calculated; Collision time factor; Attention factor ; The current decision-making intention, i.e., the ultimate implicit intention; :intention The corresponding number of key sectors; Sector area The level of attention; Style hazard correction factor, based on the corrected comprehensive driving style factor. Calculated; Style offset ; when At this time, it is level one, with a yellow projection, voice prompts, and no interference with vehicle control; when or At this time, it is at level two, the projection area flashes red, a voice prompt appears, the seat vibrates, and the maximum permissible driving torque is actively limited. ;in, Maximum driving torque, maximum permissible speed ; when or At that time, it was a level three emergency braking system, requesting deceleration. ;in , The absolute value of the relative velocity corresponding to the minimum TTC. To minimize collision time; power output is simultaneously cut off; and the trailer's EBS is triggered 0.1 seconds before the towing vehicle. If a pedestrian is detected in the danger zone behind, restrictions are imposed. .

[0017] This invention also provides a semi-trailer blind spot active safety control system based on multimodal fusion, used to implement the above-mentioned semi-trailer blind spot active safety control method based on multimodal fusion. The system includes: a non-implantable electrode EEG headband, a camera, a contact sensor, dry electrodes, a data processing module, an attention calculation module, a driving style detection module, an EEG intention correction module, an expected control quantity correction module, a dynamic hazard zone prediction module, a target hazard zone generation module, a projection light, an environmental perception module, and an intervention module. The non-implantable electrode EEG headband is used to collect the driver's brain signals; The camera is used to collect data about the driver's eyes; The contact sensor is used to collect the temperature of the driver's fingers; The dry electrode is used to collect the driver's skin conductance response; The data processing module is used to decode implicit intentions based on the collected driver's EEG signals, obtain the driver's gaze direction and ground landing point based on the data collected by the camera, and extract eye movement parameters; and to filter the data collected by the contact sensor and dry electrode. The attention calculation module is used to calculate the average attention of the key sector set corresponding to various implicit intentions based on the driver's line of sight and the ground landing point. The driving style detection module is used to detect short-term driving style based on real-time acquired data, determine long-term driving style based on historical data, merge short-term driving style and long-term driving style to obtain comprehensive driving style, and then make correction based on semi-trailer load to obtain corrected comprehensive driving style. The EEG intention correction module is used to fuse the comprehensive driving style factor after load correction, the normalized and corrected implicit intention probability, and the corrected average attention, calculate the fused intention probability after gaze and style correction, and output the final implicit intention. The expected control quantity correction module is used to correct the original expected acceleration and steering rate according to the corrected comprehensive driving style factor, and recalculate the expected vehicle speed and steering wheel angle based on the corrected acceleration and steering rate combined with the final implicit intention. The dynamic hazard zone prediction module is used to predict the future trajectory of the vehicle based on the expected vehicle speed and steering wheel angle, and calculate the dynamic hazard zone including the inner wheel differential zone. The target hazard area generation module generates a target hazard area to be projected and displayed based on the inherent fixed blind spots and dynamic hazard areas. The projection light is used to project the target danger area onto the road surface in real time; The environmental perception module is used to detect surrounding environmental information and determine whether an obstacle has intruded into the target danger zone; The intervention module calculates a comprehensive hazard index and style deviation based on obstacle information, driver status, and driving style. It then triggers different levels of control commands based on the comprehensive hazard index and style deviation.

[0018] The advantages and beneficial effects of this invention are as follows: (1) This invention uses a non-implantable brain-computer interface to predict the implicit intentions (starting, turning, reversing, etc.) of a semi-trailer driver before they have performed any actions, and projects or actively intervenes in the danger zone 1 to 3 seconds in advance. Compared with existing systems that rely only on explicit signals such as steering wheel angle and pedal force, this invention can perceive the driver's intentions before the driver performs any actions, significantly shortening the system response time and solving the problem of delayed response in existing systems. It can provide valuable avoidance windows for pedestrians and non-motorized vehicles in the semi-trailer's blind spot, effectively avoiding accidents caused by "not having enough time to react".

[0019] (2) This invention integrates the implicit intentions of EEG decoding with real-time gaze points in a spatiotemporal manner to construct an intention-gaze joint probability model, and uses gaze attention to correct the confidence of the intention. Simultaneously, it integrates the driver's long-term driving style (based on historical data clustering) and short-term physiological states (eye movement frequency, blink frequency, saccade velocity, finger temperature, and skin conductance), and introduces the actual load of the semi-trailer to correct the style factors. By verifying EEG intentions through gaze, the dangerous state of "wanting to move but not looking" can be identified, reducing the false judgment rate; by integrating long-term and short-term styles, the intention recognition has personalized adaptability; load correction ensures that aggressive styles are automatically suppressed under heavy loads, avoiding safety hazards caused by the mismatch between driver habits and the actual risks of the vehicle. The above multi-dimensional corrections significantly improve the accuracy of intention judgment, reduce false alarms and missed alarms, and effectively solve the problem of insufficient accuracy in existing systems due to the single driver intention judgment.

[0020] (3) The present invention uses the kinematic model of a semi-trailer to calculate the dynamic danger zone including the inner wheel difference zone (inner wheel difference danger zone) in real time. The danger zone is projected onto the road surface by a single programmable projection light. The dynamic inner wheel difference projection strategy realizes the visualization of blind spots, which can make the driver and surrounding pedestrians intuitively aware of the danger zone that is about to be occupied, and solves the problem that traditional rearview mirrors and central control screens require attention to be diverted.

[0021] (4) When performing blind spot visualization, this invention integrates the laser radar elevation grid and road surface material recognition, automatically adapts to complex terrains such as slopes, snow accumulation, water accumulation, and potholes, and ensures that the projection shape remains accurate on complex road surfaces such as construction sites and ports through terrain adaptive correction, avoiding misjudgment caused by projection deformation; solves the problem that projection is easily affected by terrain; in addition, the projection time slot is dynamically adjusted according to the hinge angle change, and the time slot exclusive strategy of hinge angle linkage can ensure that the warning of the key blind spot (right inner wheel difference) is continuously visible when turning (the warning is not interrupted), making up for the deficiency of insufficient coverage of a single projection lamp.

[0022] (5) This invention integrates obstacle distance, collision time, intention confidence, visual attention and driving style factors to calculate a comprehensive hazard index and implements a three-level intervention system: Level 1 (audio-visual warning), Level 2 (active speed limit), and Level 3 (active emergency braking). In Level 3 braking, the trailer electronic braking system is introduced to coordinate the control triggered in advance by the tractor, and the braking deceleration is limited according to the presence of pedestrians behind. The gradual intervention avoids discomfort and rear-end collision risks caused by sudden braking, while ensuring that the vehicle can be forced to stop in an emergency. The trailer braking coordination eliminates the pushing effect between the tractor and the trailer, preventing the trailer from folding or fishtailing during emergency braking. When there are pedestrians behind, the deceleration is limited to avoid secondary injuries caused by excessive sinking of the trailer tail, which reflects the protection of vulnerable road users.

[0023] (6) This invention solves key technical problems such as delayed response, misjudgment of intent, insufficient visualization, and uncoordinated intervention in the active safety system of semi-trailer blind spots by using a closed-loop control of implicit intent prediction, line of sight verification, style adaptation, dynamic projection, and hierarchical intervention. It significantly improves the safety, real-time performance, and environmental adaptability of semi-trailers in complex scenarios such as starting, turning, and reversing. Attached Figure Description

[0024] Figure 1 A flowchart of the active safety control method for blind spots of semi-trailers based on multimodal fusion provided by the present invention; Figure 2 The block diagram of the active safety control system for blind spots of semi-trailers based on multimodal fusion provided by the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions and advantages of the present invention, the present application will be described in detail below with reference to the accompanying drawings, but this is not intended to limit the scope of protection of the present invention.

[0026] Example 1: like Figure 1 As shown in the figure, this embodiment provides a method for active safety control of blind spots in semi-trailers based on multimodal fusion. The method includes the following steps: Step S1. EEG signal processing and implicit intention decoding: Step S1.1. Signal Acquisition and Preprocessing: In this embodiment, a non-implantable electrode EEG headband (EEG headband) is used for signal acquisition, with a sampling rate of [missing information]. The electrode positions are as follows: Electrode: Corresponds to the motion preparation potential, located in the midline of the central area; Electrodes: Right motor cortex (controls right-side limb movement), located in the left central area; Electrodes: Corresponding to the left motor cortex (controlling left limb movement), located in the right central area; Electrodes: Corresponding to the auxiliary motor area, located on the midline of the frontal region; Electrode: Corresponding to the posterior parietal cortex, located in the midline of the parietal region.

[0027] Bandpass filtering: ;in, Raw EEG signals (unit: ); Time (unit: ); : Low-frequency cutoff frequency; : High-frequency cutoff frequency; : Filtered signal.

[0028] Sliding window segments: window length sliding step size .

[0029] Step S1.2. Feature Extraction: Specifically, this embodiment extracts slow cortical potential (SCP) features (feature 1) and event-related desynchronization (ERD) features (feature 2). The expression for the slow cortical potential (SCP) features is as follows: in, : The number of sampling points within the window; : Sampling interval; Window start time (unit: ); : Filtered signal from the electrodes (unit: ); : Resting state Average voltage of electrodes over 5 seconds (unit: ); Slow cortical potential characteristics (unit: The larger the negative value, the stronger the exercise preparation.

[0030] Resting-state reference power:

[0031] in, Electrode markings; :frequency band( 8~12Hz, (18~26H). : Number of resting-state samples; : Resting state Frequency band within a window Fourier coefficients (unit: ); Resting average power (unit: ).

[0032] The method for obtaining Event-Related Desynchronization (ERD) features is as follows:

[0033] in, : Time-averaged power, compared with resting-state reference power The calculation method is exactly the same, except that the data source is replaced from "resting time period" to "sliding window centered (or ending) at the current time t"; Event-related desynchronization features (in %), negative values ​​indicate cortical activation.

[0034] In addition, to enhance the ability to distinguish between left and right turn intentions, an auxiliary motion zone lateralization coefficient is defined:

[0035] in, Lateralization coefficient (dimensionless, positive values ​​indicate right turn tendency, negative values ​​indicate left turn tendency). : electrode μ Synchronize with rhythmic events; : electrode μ Synchronize with rhythmic events; This coefficient is used to correct the left and right turn probabilities output by the SVM classifier (see step S5.1 for the specific correction formula) to enhance or suppress the initial probability of turning intention, thereby improving the ability to distinguish between left and right turn intentions.

[0036] The final feature vector is:

[0037] in, : Final feature vector; Current moment The characteristic value of slow cortical potential; Current moment electrode μ Rhythmic event-related desynchronization values; Current moment electrode μ Rhythmic event-related desynchronization values; Current moment electrode Rhythmic event-related desynchronization values; Current moment electrode Rhythmic event-related desynchronization values.

[0038] Step S1.3. Implicit Intent Classification and Smoothing: Set of implicit intention categories ,common kind.

[0039] SVM+Platt scaling:

[0040] in, : Decision value of the i-th class SVM (dimensionless); A, B: Platt scaling coefficients (dimensionless, obtained from training); : No. The original probability (dimensionless) of class intent satisfies .

[0041] Exponential moving average smoothing:

[0042] in, : Smoothing factor (dimensionless). : System sampling period; Smoothed intention probability (dimensionless).

[0043] Furthermore, in this embodiment, to prevent false triggering, a threshold filtering of the semi-trailer's motion state is performed, and the semi-trailer's articulation angular velocity is introduced. As a threshold: if (If the trailer is clearly turning), then the probability of forcibly performing a "start / accelerate" instruction will be increased. Multiply by a coefficient of 0.2 to suppress the intention to start. If the semi-trailer's reverse gear is activated, the probability of the "reverse" intention will be forcibly suppressed. Set it to 1.0, and set the rest to zero.

[0044] Preliminary identification of implicit intentions Smoothing confidence level .

[0045] Step S2. Eye tracking, ground landing point calculation, and eye movement parameter extraction: Step S2.1. Direction of line of sight and point of impact on the ground: In this embodiment, the coordinate systems are defined as follows: Vehicle coordinate system: the origin is the center of the rear axle of the tractor, the X-axis is forward, the Y-axis is left, and the Z-axis is upward. Camera coordinate system: the origin is the optical center of the camera, and the transformation relationship between each axis and the vehicle coordinate system is obtained through calibration. Head coordinate system: the origin is the center of the head, and the transformation relationship between each axis and the camera coordinate system is obtained through head pose estimation.

[0046] Step S2.1.1. Calculation of line of sight direction: First, determine the direction of the eye's optical axis (the unit vector in the head coordinate system). Rotate to the camera coordinate system:

[0047] in, : Indicates the direction of the eye's optical axis in the camera's coordinate system. : Rotation matrix from head coordinate system to camera coordinate system (3×3, dimensionless, obtained from calibration).

[0048] Then, the line of sight in the camera coordinate system is transformed to the vehicle coordinate system:

[0049] in, Rotation matrix from camera coordinate system to vehicle coordinate system ( (Dimensionless, obtained by calibration); The unit vector (dimensionless) of the line of sight in the vehicle coordinate system.

[0050] Step S2.1.2. Obtain the coordinates of the starting point of the line of sight (eye) in the vehicle coordinate system: The position of the origin of the head coordinate system in the camera coordinate system is: The offset of the eyeball center relative to the origin of the head coordinate system is denoted as... (Usually a known constant, such as) (meters). Then the coordinates of the eyeball center in the camera coordinate system are:

[0051] Then switch to the vehicle coordinate system:

[0052] in, The coordinates of the camera's mounting location in the vehicle coordinate system (obtained through calibration) are usually known constants. For simplicity, the coordinates of the eye in the vehicle coordinate system are denoted as... ,Right now: .

[0053] Step S2.1.3. Ground landing point calculation: The ground equation is (Vehicle coordinate system). Line-of-sight parameter equations: ; in, Let be the distance parameter (unit: m), and let Solving for:

[0054] in, : Line of sight vector Quantity (dimensionless); The distance parameter (in meters) from the intersection point between the eye and the ground is used to obtain the coordinates of the landing point on the ground. ; : The coordinates of the point on the ground where the line of sight lands in the vehicle coordinate system (unit: m). : Line of sight vector Quantity (dimensionless); : Line of sight vector Quantity (dimensionless).

[0055] Step S2.2. Attention Calculation: Divide the ground around the vehicle into Each sector has an angle. Statistics of the past Cumulative duration of inland ground landing points in each zone Calculate attention:

[0056] in, Sector area Attention level (dimensionless, 0~1).

[0057] Implicit intention The corresponding key sector set Average attention for:

[0058] in, :gather The number of elements in the medium (dimensionless).

[0059] Step S2.3. Extraction of eye movement frequency, blink frequency, and saccade velocity: This embodiment extracts the following parameters from an eye tracker using a source camera (30Hz) or a higher frame rate: Eye movement frequency Within a unit of time (e.g., in the past) Number of gaze movements. Each gaze movement is defined as the change in gaze direction angle between two consecutive frames. .but:

[0060] in, :past Number of eye saccades per second (dimensionless); : Count the length of the window; : , eye sac angle threshold.

[0061] blink frequency The number of times the eyes close per unit of time. Blinking is determined by detecting the eye aspect ratio (EAR) or pupil disappearance.

[0062]

[0063] in, :past The number of blinks within a single eye (dimensionless); : Statistical window (to avoid short-term fluctuations).

[0064] Sagittal speed : The maximum angular velocity of the line of sight during each eye saccade. (Taking past values) The average of all saccade velocities within a second: if in the past There is at least one saccade within a second, that is... ,but:

[0065] in, : No. Angular change between adjacent frames during a saccade (unit: °); Inter-frame time interval. If in the past... No eye saccharidation was detected within seconds, i.e. Then the saccade velocity remains at the value of the previous moment:

[0066] in, This is the system sampling period. If no eye saccades are detected after the system starts, the initial value is set to 0.

[0067] Step S3. Physiological signal acquisition: This embodiment measures finger temperature by installing contact sensors on both sides of the steering wheel. (Unit: °C), preferably via infrared or thermistor; skin conductance response (Unit: μS), preferably measured using two dry electrodes to measure the skin conductance of the hand. Sampling frequency. Hz.

[0068] Real-time moving average filtering:

[0069] in, 20, the number of samples within the sliding window (dimensionless). : Sampling interval; : Filtered finger temperature.

[0070] Similarly, the skin conductance (skin electrical response) is also considered. Filtering (Unit: μS).

[0071] Step S4. Driving Style Detection and Integration: Step S4.1. Obtaining Long-Term Driving Style: Driving behavior indicators from a past period, such as 6 months, are extracted from the vehicle's historical driving database. These indicators include average acceleration, frequency of rapid acceleration, frequency of sharp turns, following distance maintenance, and braking deceleration. Cluster analysis is used to categorize drivers into three types: aggressive drivers (…). ), stable type ( ), conservative type ( Output long-term style factor ,in To be radical, For stability, To be conservative.

[0072] Step S4.2. Short-term driving style detection: Based on eye movement frequency, blink frequency, saccade velocity, finger temperature, and skin conductance, the current driver's emotional arousal and fatigue level are calculated, thereby obtaining short-term style factors.

[0073] In this embodiment, the relationship between each indicator and emotion / state is shown in Table 1: Table 1 shows the relationship between each indicator and mood / state: Eye movement frequency rise Excitement, tension, high arousal Tendency towards radical Sagittal speed rise Excitement, anxiety Tendency towards radical blink frequency reduce High concentration, tension Tendency towards radical blink frequency rise Fatigue, drowsiness Tendency towards conservatism finger temperature reduce Pressure, vasoconstriction Tendency towards radical Skin conductance rise Emotional arousal, sweating Tendency towards radical Specifically, a baseline value is defined for each indicator (the median value calculated from the driver's past hour of normal driving). For each indicator... The current deviation is expressed as:

[0074] in, The current index value (e.g.) ); The baseline median of this indicator; Baseline standard deviation (same unit as indicator); : No. A physiological indicator at the current moment The deviation is dimensionless.

[0075] Map the deviation to Consider the interval and direction.

[0076] Specifically, for the indicator that "the upward trend is becoming more aggressive" ( Contribution value A positive value indicates a radical tendency.

[0077] For the indicator of "reducing the tendency to be aggressive" ( (Decreases when highly concentrated): Contribution value That is, a negative deviation (reduction) produces a positive contribution.

[0078] For the indicator that "the upward trend tends to be conservative" ( (Increases during fatigue): Contribution value ;in, For the first Deviation of a physiological indicator.

[0079] For greater clarity, this embodiment transforms each indicator into a "radical tendency factor". Positive values ​​indicate a more aggressive approach, while negative values ​​indicate a more conservative approach.

[0080] Specifically, the radical factor of eye movement frequency:

[0081] in, Eye movement frequency (Hz); : Median baseline eye movement frequency (Hz); : Standard deviation of baseline eye movement frequency (Hz); Positive value indicates radicalism.

[0082] Aggressive factor in saccade velocity:

[0083] in, : Current average saccade velocity (° / s); : Median baseline saccade velocity (° / s); Baseline standard deviation (° / s).

[0084] Blink frequency radical factor:

[0085] in, Current blink frequency (Hz); Median baseline blink frequency (Hz); Baseline standard deviation (Hz); molecule : Positive (aggressive) when the frequency is below the baseline.

[0086] Finger temperature radical factor:

[0087] in, Current finger temperature (°C); : Median baseline temperature (°C); Baseline standard deviation (°C).

[0088] Skin conductance radical factors:

[0089] in, Current skin conductance (μS); : Median baseline conductance (μS); Baseline standard deviation (μS).

[0090] By combining the above five radical tendency factors and using a weighted average method, the short-term style factor is calculated as follows:

[0091] in, Short-term style factors; Eye movement frequency weighting; : Eye saccade speed weight; Blink frequency weighting; Finger temperature weighting; Skin conductance weight; the sum of all weights should be 1.

[0092] Furthermore, to prevent noise, short-term style factors... Perform low-pass filtering:

[0093] in, : Filter coefficients; : Filtered short-term style factor.

[0094] Step S4.3. Driving Style Integration: Long-term style factors (Values ​​-1, 0, 1) and the filtered short-term style factor (Continuous value) fusion:

[0095] in, :Comprehensive driving style factor (dimensionless, -1~1), positive values ​​indicate aggressiveness, negative values ​​indicate conservatism; Long-term style weight (dimensionless): historical long-term influence accounts for 30%, and short-term real-time status accounts for 70%. Long-term style; : Short-term style after filtering.

[0096] Semi-trailer load correction:

[0097] in, : The revised overall driving style factor; : Current gross vehicle weight (unit: kg); Unloaded mass (unit: kg); Maximum permissible mass (unit: kg); : Load suppression coefficient (dimensionless).

[0098] In this embodiment, long-term style weights are defined. The initial value is This weight is used in the fusion formula of step S4.3. Simultaneously, style stability is assessed; if, over a given period (e.g., 30 seconds), short-term style factors... With long-term style The absolute deviation is always less than the threshold. (Right now If the current driving behavior is highly consistent with historical habits, then long-term style enhancement can be performed, that is: to... The factor is temporarily increased to 0.6 to allow long-term style to dominate. This increase is maintained until the absolute deviation exceeds the threshold again, at which point the factor reverts to its initial value. Additionally, if the short-term style factor has been high over a past period (e.g., 30 seconds), the factor is temporarily increased. If the standard deviation exceeds the upper limit, it indicates drastic fluctuations in physiological signals (possibly due to driver agitation or sensor noise). To avoid misjudgment, style is disabled, i.e., the overall driving style factor is forcibly set to neutral. Normal fusion will resume once the standard deviation falls below the threshold.

[0099] Final style confidence .

[0100] Step S5. Implicit Intention—Gaze Blending and Style Correction: Step S5.1. Correction of the original implicit intent probability: Weighted attention calculation (considering hinge angle):

[0101] in, : Semi-trailer articulation angle, which is the angle between the tractor and the trailer (unit: °); ,when hour ; Represents weighted attention; Correction of lateralization coefficients (from step S1.2):

[0102] in, Step S1.3 outputs the smoothed probabilities of left turn and right turn (dimensionless). Lateralization coefficient (dimensionless, positive values ​​indicate right turn tendency, negative values ​​indicate left turn tendency). : The left and right turn probabilities after lateralization correction.

[0103] The probabilities of other intentions (going straight, starting / accelerating, reversing) remain unchanged:

[0104] Normalization:

[0105] Visual and style corrections:

[0106] in, Normalized and corrected probability of EEG intent (dimensionless); κ : Visual enhancement coefficient (dimensionless); ξ Style influence intensity coefficient (dimensionless). : Load-corrected integrated driving style factor; : Indicator function, takes the value 1 when the intent is aggressively related, otherwise takes the value 0; Indicator function, when the intention The value is 1 if it belongs to "aggressive related intentions" (such as starting, accelerating, or making a sharp turn), otherwise it is 0 (when going straight). ); : Probability of fusion intention after gaze and style correction (dimensionless).

[0107] Furthermore, in this embodiment, an intention-attention inconsistency penalty is applied if... and Start the timer ,when At that time, a penalty operation is performed on the probability of fusion intent, and step S6 is immediately triggered to increase the brightness and flashing frequency of the projection to the second-level warning level (red flashing). At the same time, the dynamic danger area projection is exclusively occupied, which is different from the regular alternating projection, aiming to provide enhanced warning to the driver before actual operation.

[0108]

[0109] in, This represents the weighted attention level corresponding to the implicit intentions of current integrated decision-making. The probability corresponding to the implicit intention of the current fusion decision. The penalty factor, which can be set to 0.3, indicates that when the driver's intention is significantly inconsistent with their visual focus area and the duration exceeds a threshold, the probability of the current fusion intention will be reduced. This reflects the system's distrust of the "want to move but not look" state.

[0110] In this embodiment, the penalty continues until... Or the intention is to disappear.

[0111] Since the probability is reduced, the original maximum probability may change. Therefore, the final implicit intention is recalculated by incorporating lateralization correction, gaze correction, and style correction. , used for subsequent calculations.

[0112] Step S5.2. Adjustment of expected control quantity: Under an aggressive style, expected acceleration and steering rate Multiply by gain:

[0113] in, : Corrected expected acceleration; The original expected acceleration (unit: m / s²) is estimated based on EEG characteristics (through a pre-trained regression model). :Acceleration style gain coefficient (dimensionless); : Corrected integrated driving style factor ( ), positive value indicates radicalism; Results limitations: ,in .

[0114] Corrected steering rate:

[0115] in, The original expected turning rate (in units: ) was estimated based on EEG characteristics (especially lateralization coefficient and activation level of supplementary motor area). 。 / s); Steering rate style gain coefficient (dimensionless). Results limitations: ,in 。 / s.

[0116] Then, based on the corrected acceleration and steering rate, the changes in expected vehicle speed and steering wheel angle over time are recalculated: for starting / acceleration intentions ( The expected speed is:

[0117] in, Current vehicle speed (unit: ); Prediction time (unit: ); :3s.

[0118] For the intended turn (left or right), the steering wheel angle is:

[0119] in, Current steering wheel angle (unit: rad or ); +1 (turn left) or -1 (turn right); Maximum steering wheel angle.

[0120] For a straight-ahead intention, the steering wheel angle decays exponentially:

[0121] For the intention to reverse, the vehicle speed is negative, and the steering wheel angle is the same as the steering formula.

[0122] Step S6. Dynamic Hazard Area Calculation and Projection: In this embodiment, the kinematic model expression for the semi-trailer is:

[0123] in, : Coordinates of the center point of the rear axle of the tractor (unit: m); :Tractor heading angle (unit: rad); Trailer pitch angle (unit: rad); articulation angle (Unit: rad)

[0124] Vehicle parameters: :Trailer wheelbase; : The distance from the rear axle of the tractor to the hinge point; The distance from the articulation point to the rear axle of the trailer; : Width of the tractor; Trailer width.

[0125] Given the corrected expected control quantity: vehicle speed (Positive or negative, unit: m / s), steering wheel angle (Unit: rad), then the turning radius of the tractor is:

[0126] in, Turning radius of the rear axle center point of the tractor (unit: m), when hour (Linear) kinematic differential equations:

[0127] in, : Hinge angle change rate (unit: rad / s); : Center point of the rear axle of the tractor coordinates relative to time The derivative; : Center point of the rear axle of the tractor coordinates relative to time The derivative; : Tractor heading angle Regarding time The derivative; The angle between the tractor and the trailer relative to time The derivative; Predicted time; .

[0128] Numerical integration using the fourth-order Runge-Kutta method, step size Integrating from the current state to The trajectory sequence is obtained as follows:

[0129] in, : No. The center point of the rear axle of the tractor at that moment coordinate; : No. The center point of the rear axle of the tractor at that moment coordinate; : No. The heading angle of the tractor at any given moment; : No. The heading angle of the trailer at a given moment.

[0130] Lateral acceleration constraints: Lateral acceleration of the rear axle of the trailer:

[0131] in, Turning radius of the rear axle center of the trailer (unit: m); like ( , If so, then it is mandatory:

[0132] Vehicle contour envelope and inner wheel differential band: Every moment The tractor unit has a rectangular outline (length) ,Width The trailer's outline is rectangular (length...) ,Width Projecting all contour points onto the ground yields the polygon of the occupied area at that moment. Dynamic hazard zones are defined as future... Union of all contour points:

[0133] Specifically, for turning scenarios, the focus is on calculating the inner wheel difference hazard zone: the area between the front and rear tracks of the tractor and the right rear track of the trailer. Let the radius of the tractor's outermost track be the turning radius of the tractor's outer front wheel. The radius of the trajectory of the innermost point of the trailer, that is, the turning radius of the innermost rear wheel of the trailer, is... The width of the inner wheel differential safety strip (inner wheel differential strip) is:

[0134] in, Turning radius of the outer front wheel of the tractor (unit: m); Turning radius of the inner rear wheel of the trailer (unit: m); : Width of the critical zone of the inner wheel difference (unit: m). This width varies along the trajectory and is used to generate a strip polygon during dynamic projection.

[0135] Roll effect correction: If the transverse slope ( ),but:

[0136] Use replace To calculate the offset trajectory of the inner rear wheel of the trailer, and thus update the danger zone. The inner wheel differential zone boundary. This correction ensures that the projected hazardous area on a sloping road surface is consistent with the actual physical hazardous area.

[0137] In this embodiment, a style-driven danger zone expansion is introduced: if the driving style factor is considered... (Aggressive approach) Expand the danger zone outwards to provide a greater safety margin:

[0138] in, : Morphological dilation operation; Expanding the regional boundaries outwards Rice, of which ; Style extension coefficient; : Corrected integrated driving style factor (dimensionless, ), expands only when the value is positive, if ,but (Not extended).

[0139] Furthermore, in this embodiment, the inherent blind spots of the semi-trailer are determined in advance, and the fixed blind spots are defined as including the blind spot polygon in front of the truck cab, the blind spot polygon directly behind the trailer, etc., and the blind spot polygons in front of the truck cab and directly behind the trailer are respectively... , The final output target hazard area is the union of the dynamic hazard area and the fixed blind area, for example: The final output This is the target hazardous area used by the subsequent projection module. The programmable high-brightness projection light installed below the right rearview mirror projects the fixed blind spot and dynamic hazardous area onto the actual terrain in turn, ensuring accurate shape.

[0140] Furthermore, in this embodiment, the projection lamp is installed below the right rearview mirror and has a built-in electric pan-tilt head or digital micromirror deflection mechanism, which can change the direction of the projected light; therefore, through time-division multiplexing control, each sub-area in the target danger zone can be projected onto the corresponding actual ground in sequence.

[0141] It should be noted that in this embodiment, the target danger area is a broad blind spot, which includes both fixed areas that the driver cannot directly observe in a normal driving posture (such as the blind spot in front of the vehicle, the blind spot of the right A-pillar, and the blind spot directly behind the trailer) and dynamic danger areas (dynamic envelope) generated by vehicle movement, such as the inner wheel differential band when turning and the sway area of ​​the trailer's rear when reversing. Since these areas may not be connected, a single projection light cannot cover multiple unconnected areas simultaneously, so subsequent projections need to be performed in turn.

[0142] Furthermore, in this embodiment, the mathematical model of the projection lamp is: Resolution: , (Unit: pixels), Horizontal field of view: Vertical field of view: Maximum brightness: .

[0143] The following parameters are obtained through factory calibration:

[0144] in, : Intrinsic parameter matrix; , Focal length (unit: pixels); : Coordinates of the station.

[0145] Fixed transformation from the projection lamp to the vehicle coordinate system: rotation matrix is (3×3, dimensionless), translation vector (Unit: m)

[0146] For any pixel in the input image of the projector lamp (Integer coordinates), and their corresponding normalized projection direction (in the projector coordinate system):

[0147] in, Normalized image coordinates (dimensionless).

[0148] Obtain the actual direction unit vector (In the projector lamp coordinate system), the parametric equation of the light rays emitted from the projector lamp is:

[0149] in, Position of the projection light in the vehicle coordinate system (unit: m); Distance parameter along the ray (unit: m).

[0150] This embodiment uses a lidar (single-line or multi-line) mounted on the roof of the cab to construct a real-time ground elevation network around the vehicle. Grid range: , Grid size Each grid cell Storage Elevation (unit: Update frequency Simultaneously, material type is identified by the echo intensity of multispectral cameras or lidar. : Mud and cement :snow, : stagnant water Road surface damage. Define the material attenuation coefficient. : , , , .

[0151] In this embodiment, the projected image is generated by reverse mapping: Generate input image for projector This makes the projected ground light spot equal to the dynamic danger zone or each fixed blind zone.

[0152] Determine valid pixels: For each pixel Calculate the direction of light rays (Considering distortion), solve for the intersection points of the light rays and the terrain elevation grid. A grid traversal algorithm is used, starting from... Start by gradually increasing the height until the light level is below the terrain elevation. Obtain the intersection parameters and ground coordinates. If the intersection point exists and the projected distance is... If the pixel is valid, it is marked as a valid pixel; otherwise, it is marked as invalid.

[0153] For each valid pixel, determine whether it belongs to a dynamic danger zone or a fixed blind zone, for example: check Is it located in Internal. Using the ray casting method (point-polygon containment test):

[0154] in, : Boolean value, indicating that the ground point belongs to the danger zone.

[0155] Set pixel color:

[0156] in, According to the risk index Sure: Yellow (255,255,0) Red (255,0,0); Distance attenuation factor (dimensionless); Attenuation coefficient; Projection distance (unit: m); Material attenuation coefficient (dimensionless).

[0157] Projection timing multiplexing and hinge angle linkage: Since a single projector cannot simultaneously cover multiple hazardous areas in different directions, a time-division multiplexing strategy is employed. Each time slot projects the hazardous area in the corresponding direction, and the persistence of vision creates a simultaneous warning effect across multiple areas. Specifically, within one cycle (e.g., 0.6 seconds), the projector points in multiple directions, such as forward, right, and backward, projecting for 0.2 seconds in each direction. This projection method is low-cost.

[0158] Furthermore, when the rate of change of the articulation angle exceeds a threshold or when reverse gear is activated (during turning or reversing), the system forcibly monopolizes the most dangerous area of ​​the projection, pausing the alternation of other directions. Specifically, if ( The vehicle exclusively occupies the right inner wheel difference projection. If reverse gear is activated, it exclusively occupies the rear projection; otherwise, the projections are alternately allocated. This applies to each time slot. (Unit: s).

[0159] Projection adaptive scaling and flicker:

[0160] in, : Actual output brightness of the projector lamp (unit: lumens, lm). This value is dynamically adjusted based on the current overall risk index; the higher the risk level, the brighter the projection. The maximum rated brightness of the projector lamp (unit: lm) is a fixed constant (e.g., 5000lm). The formula multiplies it by a coefficient. This makes the brightness at Risk=0. (Minimum warning brightness), in At that time (Full brightness); The flicker frequency of the projected pattern (unit: Hz). When Risk=0, the flicker frequency is 1Hz (1 flash per second); when... At this time, the flashing frequency is 5Hz (5 flashes per second). The higher the level of danger, the faster the flashing, in order to enhance the warning effect.

[0161] Furthermore, in this embodiment, when At this time, a dynamically moving arrow is superimposed on the projected pattern. The arrow moves in the direction of the obstacle relative to the vehicle, so as to visually indicate the source of danger to the driver.

[0162] Furthermore, in this embodiment, a degradation strategy is executed based on the projection validity: Define the effective projection index:

[0163] in, : Valid pixels (with ground intersections and The quantity (dimensionless); Total number of pixels (dimensionless).

[0164] like If the ground projection is turned off, the blind spot image will be automatically switched to the central control screen and a voice prompt will be issued.

[0165] Step S7. Risk assessment and tiered intervention: This step integrates obstacle information, driver status, and driving style to calculate a comprehensive hazard index and implement graded safety countermeasures.

[0166] Step S7.1. Calculation of Comprehensive Risk Index:

[0167] in, Overall risk index (dimensionless, 0~1); Distance factor weights; Collision time factor weight; Intent confidence weight; Attention factor weights; Style risk correction weight; : Fusion confidence ; Distance factor :

[0168] in, : Obstacle intrusion depth (negative value, unit: m), taking the most severe value among all obstacles; Minimum distance from the obstacle to the boundary of the target danger zone (positive value, unit: m); Warning distance threshold; : Safe distance threshold.

[0169] Collision time factor :

[0170] in, Minimum collision time (unit: seconds); : Reference time constant, which can be taken as 1.5s.

[0171] Attention factor :

[0172] in, The current decision-making intention, i.e., the ultimate implicit intention; :intention The corresponding number of key sectors; Sector area The level of attention.

[0173] Style Risk Correction Factor :

[0174] in, ： Corrected overall driving style factor (-1~1); Neutral style threshold; An aggressive style increases the risk factor.

[0175] Step S7.2. Tiered Intervention Strategy: The trigger threshold is dynamically adjusted based on the Risk index and style factor.

[0176] Define style offset:

[0177] in, : Threshold reduction amount (dimensionless), only effective for aggressive styles.

[0178] The triggering conditions and actions at each level are shown in Table 2: Table 2. Triggering conditions and actions at each level

[0179] Semi-trailer braking coordination: EBS trailer advance towing vehicle Triggered if a pedestrian is detected in the danger zone behind, restrictions will be lifted. After the danger is over, the braking pressure increases at a slope. release, The value was determined through actual vehicle testing, and the unit is MPa / s.

[0180] In this embodiment, the system performs a one-time initialization when the vehicle starts and continuously calibrates some parameters during operation.

[0181] EEG baseline acquisition: The driver is relaxed and motionless while the vehicle is stationary, maintaining a static eye position. Record each electrode In frequency band Power spectrum and Resting potential of the electrode .

[0182] Gaze calibration: The driver sequentially looks at nine calibration points (3×3 grid) displayed on the central control screen, pausing at each point for 2 seconds. The individual eye model parameters—corneal curvature radius and the angle between the optical axis and the visual axis—are solved using the least squares method. Simultaneously, the mapping relationship between head posture and gaze direction is calculated.

[0183] Projector calibration: The vehicle is parked on a flat, textured surface, and the projector projects a checkerboard pattern. The ground pattern is photographed using an external high-precision camera, and the intrinsic parameter matrix of the projector is calculated using the PnP algorithm. Distortion coefficient And the transformation from projection light to vehicle coordinates Reprojection error requirements Pixel.

[0184] Initial terrain mapping: Vehicles at low speed ( The system travels around the field once, using LiDAR and cameras to collect ground point clouds and construct an initial DEM. It is then continuously updated during operation (using a sliding window to retain the most recent 1000 frames).

[0185] Baseline physiological signal acquisition: After 1 hour (or a cumulative 1 hour) of normal driving by the driver, the median of the following indicators was calculated. and standard deviation Eye movement frequency saccade rate blinking frequency finger temperature Skin conductance These baseline values ​​are stored in the driver's personalized profile. During operation, the baseline is updated every 10 minutes using a sliding window (data from the most recent 10 minutes) to accommodate long-term changes.

[0186] Long-term driving style acquisition: Extract driving behavior indicators from the vehicle's historical driving database over the past 6 months, including average acceleration and frequency of rapid acceleration. Frequency of sharp turns Following distance (median time distance), median braking deceleration, etc. Drivers were categorized into aggressive (three-class) types using K-means clustering. ), stable type ( ), conservative type This value is loaded each time the vehicle starts and is not updated frequently.

[0187] Automatic calibration of trailer geometry parameters: When first connecting or changing trailers, perform a low-speed serpentine driving maneuver (vehicle speed <10 km / h, steering wheel alternately turned left and right to its limit) and automatically identify the distance from the rear axle of the tractor to the articulation point using kinematic inversion. .

[0188] Calibration principle: Acquire hinge angles at multiple moments. Speed Steering wheel angle Fit the following kinematic equations using the least squares method :

[0189] in, From actual measurement Numerical difference is used to obtain, Given the known wheelbase of the tractor unit. During the calibration process, (Distance from the articulation point to the trailer's rear axle) This value uses the trailer's factory parameters or is manually entered by the user; online calculation is not required. After calibration, the obtained distance will be... Automatically stored for kinematic calculations in subsequent step S6.

[0190] Example 2: like Figure 2As shown, this embodiment provides a semi-trailer blind spot active safety control system based on multimodal fusion, used to implement the above-mentioned semi-trailer blind spot active safety control method based on multimodal fusion. The system includes: a non-implantable electrode EEG headband, a camera, a contact sensor, dry electrodes, a data processing module, an attention calculation module, a driving style detection module, an EEG intention correction module, an expected control quantity correction module, a dynamic hazard zone prediction module, a target hazard zone generation module, a projection light, an environmental perception module, and an intervention module. The non-implantable electrode EEG headband is used to collect the driver's brain signals; The camera is used to collect data about the driver's eyes; The contact sensor is used to collect the temperature of the driver's fingers; The dry electrode is used to collect the driver's skin conductance response; The data processing module is used to decode implicit intentions based on the collected driver's EEG signals, obtain the driver's gaze direction and ground landing point based on the data collected by the camera, and extract eye movement parameters; and to filter the data collected by the contact sensor and dry electrode. The attention calculation module is used to calculate the average attention of the key sector set corresponding to various implicit intentions based on the driver's line of sight and the ground landing point. The driving style detection module is used to detect short-term driving style based on real-time acquired data, determine long-term driving style based on historical data, merge short-term driving style and long-term driving style to obtain comprehensive driving style, and then make correction based on semi-trailer load to obtain corrected comprehensive driving style. The EEG intention correction module is used to fuse the comprehensive driving style factor after load correction, the normalized and corrected implicit intention probability, and the corrected average attention, calculate the fused intention probability after gaze and style correction, and output the final implicit intention. The expected control quantity correction module is used to correct the original expected acceleration and steering rate according to the corrected comprehensive driving style factor, and recalculate the expected vehicle speed and steering wheel angle based on the corrected acceleration and steering rate combined with the final implicit intention. The dynamic hazard zone prediction module is used to predict the future trajectory of the vehicle based on the expected vehicle speed and steering wheel angle, and calculate the dynamic hazard zone including the inner wheel differential zone. The target hazard area generation module generates a target hazard area to be projected and displayed based on the inherent fixed blind spots and dynamic hazard areas. The projection light is used to project the target danger area onto the road surface in real time; The environmental perception module is used to detect surrounding environmental information and determine whether an obstacle has intruded into the target danger zone; The intervention module calculates a comprehensive hazard index and style deviation based on obstacle information, driver status, and driving style. It then triggers different levels of control commands based on the comprehensive hazard index and style deviation.

[0191] The present invention also provides an electronic device, comprising: one or more processors and a memory; wherein the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for active safety control of blind spots of semi-trailers based on multimodal fusion.

[0192] The present invention also provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method for active safety control of blind spots in semi-trailers based on multimodal fusion.

[0193] Those skilled in the art will understand that all or part of the functions of the various methods / modules in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved.

[0194] In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the programs can also be stored in storage media such as servers, other computers, disks, optical discs, flash drives, or portable hard drives. They can be downloaded or copied to the memory of the local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be implemented.

[0195] The above describes specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for active safety control of blind spots in semi-trailers based on multimodal fusion, characterized in that, The method includes the following steps: Step S1. Use an electrode EEG headband to acquire signals, extract slow cortical potential features and event-related desynchronization features, and use the SVM+Platt scaling method to estimate the original probability of various latent intentions; Step S2. Calculate the average attention level of the key sector set corresponding to various implicit intentions based on the gaze direction and the ground landing point, and extract eye movement parameters at the same time; Step S3. Collect physiological signals; Step S4. Obtain long-term driving style. Based on the data obtained in steps S2 and S3, complete short-term driving style detection, fuse long-term and short-term driving styles, and correct them based on the semi-trailer load. Step S5. Correct the original implicit intention probability and average attention, fuse the corrected original implicit intention probability, line of sight and style, calculate the fused intention probability after line of sight and style correction, and output the final implicit intention; at the same time, correct the original expected control quantity based on the corrected driving style, and recalculate the expected vehicle speed and steering wheel angle in combination with the final implicit intention. Step S6. Based on the expected vehicle speed, steering wheel angle and semi-trailer kinematic model, predict the future driving trajectory of the vehicle, generate a dynamic danger zone including the inner wheel differential zone according to the predicted trajectory, and project it onto the road surface using a projection light. Step S7. Determine the driver's state based on the attention level of the key sector corresponding to the final implicit intention. Calculate the comprehensive hazard index and style offset by combining obstacle information, driver state, and driving style. Trigger different levels and execute different control commands according to the comprehensive hazard index and style offset.

2. The method for active safety control of blind spots in semi-trailers based on multimodal fusion according to claim 1, characterized in that, Slow cortical potential characteristics in step S1 The expression is: ； in, Number of sampling points within the window; Sampling interval; : Window start time; : Filtered signal from electrodes, non-implantable electrode EEG headband The electrode is located on the center line of the central area; : Resting state Average voltage of electrodes over 5 seconds; Event-related desynchronization features The method of obtaining it is: ； in, : Average power over time Resting-state reference power Non-implantable electrode EEG headband The electrode is located in the central area on the left side; The electrode is located in the central area on the right side; The electrode is located on the midline of the forehead region; The electrode is located at the midline of the top region; Set of implicit intent categories in step S1 The original probabilities of various implicit intentions are estimated using the SVM+Platt scaling method, followed by smoothing, and the articulation angular velocity of the semi-trailer is introduced. As a threshold: if Then the probability of forcibly performing the "start / accelerate" intention will be... Multiply by a coefficient of 0.2; if the semi-trailer's reverse gear is activated, then the probability of forcibly performing the "reverse" intention is increased. Set it to 1.0, and set the rest to zero.

3. The method for active safety control of blind spots in semi-trailers based on multimodal fusion according to claim 1, characterized in that, In step S2, when calculating the average attention, first calculate the unit vector of the gaze direction in the vehicle coordinate system, then obtain the gaze origin, i.e., the coordinates of the eye in the vehicle coordinate system; then calculate the coordinates of the gaze ground landing point in the vehicle coordinate system based on the gaze parameter equation; finally, divide the ground around the vehicle into multiple sector areas, count the cumulative duration of the ground landing point in each sector area within a specified time period, and calculate the attention of each sector area and the average attention of the key sector area set corresponding to various implicit intentions.

4. The active safety control method for blind spots of semi-trailers based on multimodal fusion according to claim 1, characterized in that, The eye movement parameters mentioned in step S2 include eye movement frequency, blink frequency, and saccade velocity; the physiological signals mentioned in step S3 include finger temperature and skin conductance; in step S4, when performing short-term driving style detection based on the data obtained in steps S2 and S3, the current deviation is calculated according to the baseline median and baseline standard deviation of each indicator, and then the deviation is mapped to... The range is considered, and the direction is taken into account, so that each indicator is transformed into an "aggressive tendency factor". Then, the five aggressive tendency factors are merged to calculate the short-term style factor. After low-pass filtering, it is merged with the long-term style factor to obtain the comprehensive driving style factor. Finally, the comprehensive driving style factor is corrected according to the trailer's current total mass, unloaded mass and maximum permissible mass to obtain the corrected comprehensive driving style factor.

5. The method for active safety control of blind spots in semi-trailers based on multimodal fusion according to claim 1, characterized in that, In step S4, before driving style fusion, style stability is first judged. If the absolute deviation between short-term style factor and long-term style factor is always less than the threshold within a specified period of time, the weight of long-term style is increased during driving style fusion. If the standard deviation of the short-term style factor exceeds the threshold within a specified period of time, the overall driving style factor will be forcibly set to neutral. Normal fusion will be restored once the standard deviation of the short-term style factor falls back below the threshold.

6. The method for active safety control of blind spots in semi-trailers based on multimodal fusion according to claim 1, characterized in that, In step S5, the average attention of the key sector set corresponding to the implicit intent is modified based on the semi-trailer articulation angle to obtain the weighted attention; the left turn and right turn smooth probabilities in the original implicit intent probability estimated in step 1 are modified based on the lateralization coefficient and normalized. The lateralization coefficient is calculated based on the event-related desynchronization feature value. Then, the integrated driving style factor after load correction is incorporated. Normalized and corrected probability of implicit intention Weighted attention Calculate the probability of fusion intention after gaze and style correction. The expression is: ； in, κ Visual enhancement factor; ξ Style influence intensity coefficient; : Indicator function, takes the value 1 when the intent is aggressively related, otherwise takes the value 0; The indicator function takes the value 1 if the intent is "aggressive and relevant intent", and 0 otherwise. Number of implicit intent categories; The expected control quantities include expected acceleration and steering rate. The original expected acceleration and steering rate are estimated based on EEG characteristics. The original expected acceleration and steering rate are then corrected based on the modified comprehensive driving style factor. In step S5, an intent-attention inconsistency penalty is performed. If the weighted attention corresponding to the current fusion decision implicit intent is less than a set threshold and the original implicit intent probability is greater than a set threshold, a timer is started. When the time exceeds the set time limit, a penalty operation is performed on the fusion intent probability, and the final implicit intent is re-output.

7. The active safety control method for blind spots of semi-trailers based on multimodal fusion according to claim 1, characterized in that, In step S6, the vehicle's trajectory sequence is obtained based on the expected vehicle speed and steering wheel angle. For each moment, all contour points of the tractor and trailer are projected onto the ground to obtain the occupied area polygon at that moment. The dynamic danger zone is the union of all contour points within a set future time, including the inner wheel differential strip. For turning scenarios, the width of the inner wheel differential strip is calculated. If the lateral slope is greater than... Then, the inner wheel differential band width is corrected, and the offset trajectory of the inner rear wheel of the trailer is calculated based on the corrected inner wheel differential band width, and the dynamic danger zone is updated; at the same time, a style-driven danger zone expansion is introduced. If the corrected comprehensive driving style factor is greater than 0, the dynamic danger zone is expanded outward.

8. The active safety control method for blind spots of semi-trailers based on multimodal fusion according to claim 1, characterized in that, During step S6, a projected image is generated through reverse mapping, a ground elevation network around the vehicle is constructed in real time, ground conditions are identified through the echo intensity of a multispectral camera or lidar, the color of the effective pixels is adjusted based on the ground conditions and the projection distance, and the actual output brightness and flicker frequency of the projection lamp are dynamically adjusted according to the current comprehensive hazard index. In step S6, a fixed blind spot is preset, which is a fixed area that the driver cannot directly observe in a normal driving posture. The final output target danger area is the union of the dynamic danger area and each fixed blind spot. The projection lamp projects the fixed blind spot and the dynamic danger area onto the actual terrain in turn. When step S5 executes the intent-attention inconsistency penalty, the brightness and flicker frequency of the projection are increased to the second-level warning level, and the projection of the dynamic danger area is exclusively controlled. When the rate of change of the hinge angle exceeds the threshold, the projection of the right inner wheel difference is exclusively occupied.

9. The active safety control method for blind spots of semi-trailers based on multimodal fusion according to claim 1, characterized in that, Comprehensive risk index in step S7 The expression is: ； in, Distance factor weights; Collision time factor weight; Intent confidence weight; Attention factor weights; Style risk correction weight; : Fusion confidence, fusion confidence at time t , The probability of the intended fusion after adjustments for perspective and style; Distance factor, based on the minimum distance from the obstacle to the boundary of the projection area. Calculated; Collision time factor; Attention factor ; The current decision-making intention, i.e., the ultimate implicit intention; :intention The corresponding number of key sectors; Sector area The level of attention; Style hazard correction factor, based on the corrected comprehensive driving style factor. Calculated; Style offset ; when At this time, it is level one, with a yellow projection, voice prompts, and no interference with vehicle control; when or At this time, it is at level two, the projection area flashes red, a voice prompt appears, the seat vibrates, and the maximum permissible driving torque is actively limited. ;in, Maximum driving torque, maximum permissible speed ; when or At that time, it was a level three emergency braking system, requesting deceleration. ;in , The absolute value of the relative velocity corresponding to the minimum TTC. To minimize collision time; power output is simultaneously cut off; and the trailer's EBS is triggered 0.1 seconds before the towing vehicle. If a pedestrian is detected in the danger zone behind, restrictions are imposed. .

10. A semi-trailer blind spot active safety control system based on multimodal fusion, characterized in that, The system is used to implement the active safety control method for blind spots of semi-trailers based on multimodal fusion as described in any one of claims 1 to 9, including: a non-implantable electrode EEG headband, a camera, a contact sensor, a dry electrode, a data processing module, an attention calculation module, a driving style detection module, an EEG intention correction module, an expected control quantity correction module, a dynamic hazard zone prediction module, a target hazard zone generation module, a projection light, an environmental perception module, and an intervention module. The non-implantable electrode EEG headband is used to collect the driver's brain signals; The camera is used to collect data about the driver's eyes; The contact sensor is used to collect the temperature of the driver's fingers; The dry electrode is used to collect the driver's skin conductance response; The data processing module is used to decode implicit intentions based on the collected driver's EEG signals, obtain the driver's gaze direction and ground landing point based on the data collected by the camera, and extract eye movement parameters; and to filter the data collected by the contact sensor and dry electrode. The attention calculation module is used to calculate the average attention of the key sector set corresponding to various implicit intentions based on the driver's line of sight and the ground landing point. The driving style detection module is used to detect short-term driving style based on real-time acquired data, determine long-term driving style based on historical data, merge short-term driving style and long-term driving style to obtain comprehensive driving style, and then make correction based on semi-trailer load to obtain corrected comprehensive driving style. The EEG intention correction module is used to fuse the comprehensive driving style factor after load correction, the normalized and corrected implicit intention probability, and the corrected average attention, calculate the fused intention probability after gaze and style correction, and output the final implicit intention. The expected control quantity correction module is used to correct the original expected acceleration and steering rate according to the corrected comprehensive driving style factor, and recalculate the expected vehicle speed and steering wheel angle based on the corrected acceleration and steering rate combined with the final implicit intention. The dynamic hazard zone prediction module is used to predict the future trajectory of the vehicle based on the expected vehicle speed and steering wheel angle, and calculate the dynamic hazard zone including the inner wheel differential zone. The target hazard area generation module generates a target hazard area to be projected and displayed based on the inherent fixed blind spots and dynamic hazard areas. The projection light is used to project the target danger area onto the road surface in real time; The environmental perception module is used to detect surrounding environmental information and determine whether an obstacle has intruded into the target danger zone; The intervention module calculates a comprehensive hazard index and style deviation based on obstacle information, driver status, and driving style. It then triggers different levels of control commands based on the comprehensive hazard index and style deviation.

Images