Layered dangerous lane change intervention method and system based on multi-modal intention recognition

Abstract

The invention relates to the technical field of intelligent driving, and particularly discloses a layered dangerous lane changing intervention method and system based on multi-mode intention recognition, and the method comprises the steps: obtaining a multi-mode intention signal of a driver in real time; judging the current lane changing intention state of the driver based on the multi-mode intention signal; environment information behind the side of the vehicle is detected in real time, and the collision risk level of current lane changing is determined based on the environment information; inputting the lane changing intention state and the collision risk level into a pre-stored decision matrix for matching, and outputting a corresponding hierarchical intervention strategy; the decision matrix comprises intervention strategies corresponding to combinations of different lane changing intention states and different collision risk levels, and early warning and / or vehicle control actions corresponding to the layered intervention strategies are executed. The method solves the core problem that in the prior art, intentional lane changing and unconscious dangerous deviation of a driver cannot be effectively distinguished, and consequently false alarm or untimely intervention is caused.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, specifically to a hierarchical dangerous lane change intervention method and system based on multimodal intent recognition. Background Technology

[0002] With the continuous development of intelligent driving technology in automobiles, various active safety systems have become important means to improve driving safety. Among them, lane change assist and lane keeping assist technologies are key research areas. Currently, the mainstream related technologies on the market mainly include blind spot monitoring systems and lane keeping assist systems.

[0003] Blind spot monitoring systems use radar or camera sensors installed on the sides and rear of the vehicle to detect other road users within the driver's blind spot and warn the driver visually, audibly, or tactilely when a potential risk exists. However, such systems have a significant limitation: they can only sense environmental risks and cannot determine the driver's intention to change lanes. This means that even when the driver has no intention to change lanes and is merely driving close to the lane lines, the system may still generate unnecessary alarms, i.e., "false alarms." Frequent false alarms can distract the driver, reduce the driving experience, and may even cause the driver to become desensitized to the alarms or actively shut down the system, thus creating potential safety hazards.

[0004] Lane Keeping Assist (LKA) systems use a forward-facing camera to identify lane markings. When it detects an unintentional lane departure, it applies a brief steering torque to correct the vehicle's deviation or pulls it back to the center of the lane. However, LKA intervention typically occurs when the vehicle has already crossed or is about to cross the lane lines, resulting in a relatively delayed response. Furthermore, its control strategy is often "bang-bang" control (i.e., a binary control that is either on or off), producing abrupt and harsh corrective torques that lead to a poor driving experience. More importantly, LKA systems cannot distinguish between intentional lane changes and unintentional lane departures. When a driver uses their turn signal to actively change lanes, if the LKA system is not properly countered, the resulting counter-torque will conflict with the driver's intention, creating a "grabbing the steering wheel" experience.

[0005] In summary, existing technical solutions, whether BSD or LKA, share a common fundamental flaw: they separate the perception of environmental risk from the recognition of the driver's subjective intent. BSD "only knows the risk, not the intent"; LKA "only knows the deviation, not the intent." This separation leads to the contradictory situation where the system "does not intervene when it should, and intervenes erroneously when it shouldn't," failing to ensure safety while fully respecting the driver's autonomy and providing a human-centered driving experience.

[0006] Therefore, there is an urgent need in this field for a comprehensive solution that can accurately and early identify a driver's lane-changing intentions in manual driving mode, and can execute smooth, graded, and humanized interventions based on the degree of matching between the intention and the risk. An ideal system should act like a skilled co-pilot, understanding the driver's plan, remaining silent when the risk is low, providing timely and appropriate reminders or assistance when the driver is distracted or unintentionally taking risks, and effectively dissuading the driver from performing high-risk maneuvers, ultimately achieving a perfect balance between safety and user experience. Summary of the Invention

[0007] To achieve the objectives of this invention, this application provides a hierarchical dangerous lane change intervention method based on multimodal intent recognition, comprising: Step S1: Acquire the driver's multimodal intent signal in real time. The multimodal intent signal includes the driver's gaze focus information, turn signal switch status, and steering wheel angle signal. Step S2: Based on the multimodal intention signal, determine the driver's current lane change intention state; the lane change intention state includes: explicit lane change intention, potential lane change intention, and no lane change or distracted intention; Step S3: Detect the environmental information to the side and rear of the vehicle in real time, and determine the collision risk level of the current lane change based on the environmental information; Step S4: Match the lane change intention state with the collision risk level in a pre-stored decision matrix and output the corresponding hierarchical intervention strategy; the decision matrix includes intervention strategies corresponding to different combinations of lane change intention states and different collision risk levels. Step S5: Execute the warning and / or vehicle control actions corresponding to the tiered intervention strategy.

[0008] In some specific embodiments, step S2 includes: When the turn signal is activated, and there is at least one effective gaze at the target side exterior rearview mirror within the preset first time window before and after activation, it is determined to be a clear lane change intention; When the turn signal is not activated and no effective gaze is detected on the left and right exterior rearview mirrors and interior rearview mirror, it is determined that there is no intention to change lanes or be distracted. When the turn signal is not activated, but one of the following conditions is met, it is determined to be a potential lane change intention: a) Within a preset second time window, there are multiple instances of effective gaze at the target-side exterior rearview mirror and / or interior rearview mirror; b) There is at least one effective gaze at the target side rearview mirror, accompanied by continuous steering wheel angle input; The term "effective gaze" refers to the driver's gaze remaining within a preset attention area of ​​the corresponding rearview mirror for a period of time exceeding a set threshold.

[0009] In some specific embodiments, in step S3, the collision risk level is divided based on the expected collision time between the side and rear target vehicle and the vehicle, and includes at least four levels: low risk, medium risk, high risk and emergency risk.

[0010] In some specific embodiments, step S4 includes at least the following matching relationships: When the intent state is a clear lane change intent and the risk level is low risk, the output strategy is no intervention. When the intention state is a clear lane change intention and the risk level is high risk, a dissuasion or suppression strategy is output, which includes a strong warning and the application of a reverse steering torque. When the intention state is a potential lane change intention and the risk level is medium risk, an early warning strategy is output, which includes a medium-level early warning and a tactile early warning. When the intention state is a potential lane change intention and the risk level is high risk, a strong warning and suppression strategy is output, which includes a strong warning, applying a reverse steering torque and coordinated braking. When the intention state is no lane change or distraction intention and the risk level is high risk, a warning and correction strategy is output, which includes early warning and lane keeping assist. When the risk level is emergency risk, an emergency avoidance strategy is output regardless of the intention state.

[0011] In some specific embodiments, in step S5, the tiered intervention strategy includes, in order of increasing intervention intensity, an early warning layer, a warning layer, an active inhibition layer, and an emergency avoidance layer.

[0012] To achieve the same inventive objective, this application also provides a hierarchical hazardous lane change intervention system based on multimodal intent recognition, comprising: Multimodal intent recognition module: used to acquire the driver's multimodal intent signals in real time, including the driver's gaze focus information, turn signal switch status and steering wheel angle signal; Environmental risk perception module: used to determine the driver's current lane change intention state based on the multimodal intention signal; the lane change intention state includes: explicit lane change intention, potential lane change intention, and no lane change or distracted intention; Risk level determination module: used to detect the environmental information to the side and rear of the vehicle in real time, and determine the collision risk level of the current lane change based on the environmental information; Intervention strategy output module: used to match the lane change intention state with the collision risk level in a pre-stored decision matrix and output the corresponding hierarchical intervention strategy; the decision matrix includes intervention strategies corresponding to different combinations of lane change intention states and different collision risk levels; Action execution module: used to execute warning and / or vehicle control actions corresponding to the hierarchical intervention strategy.

[0013] In some specific embodiments, the environmental risk perception module includes the following steps: When the turn signal is activated, and there is at least one effective gaze at the target side exterior rearview mirror within the preset first time window before and after activation, it is determined to be a clear lane change intention; When the turn signal is not activated and no effective gaze is detected on the left and right exterior rearview mirrors and interior rearview mirror, it is determined that there is no intention to change lanes or be distracted. When the turn signal is not activated, but one of the following conditions is met, it is determined to be a potential lane change intention: a) Within a preset second time window, there are multiple instances of effective gaze at the target-side exterior rearview mirror and / or interior rearview mirror; b) There is at least one effective gaze at the target side rearview mirror, accompanied by continuous steering wheel angle input; The term "effective gaze" refers to the driver's gaze remaining within a preset attention area of ​​the corresponding rearview mirror for a period of time exceeding a set threshold.

[0014] In some specific embodiments, the collision risk level in the risk level determination module is divided based on the expected collision time between the target vehicle on the side and the vehicle, and includes at least four levels: low risk, medium risk, high risk, and emergency risk.

[0015] In some specific embodiments, the decision matrix in the intervention strategy output module includes at least the following matching relationships: When the intent state is a clear lane change intent and the risk level is low risk, the output strategy is no intervention. When the intention state is a clear lane change intention and the risk level is high risk, a dissuasion or suppression strategy is output, which includes a strong warning and the application of a reverse steering torque. When the intention state is a potential lane change intention and the risk level is medium risk, an early warning strategy is output, which includes a medium-level early warning and a tactile early warning. When the intention state is a potential lane change intention and the risk level is high risk, a strong warning and suppression strategy is output, which includes a strong warning, applying a reverse steering torque and coordinated braking. When the intention state is no lane change or distraction intention and the risk level is high risk, a warning and correction strategy is output, which includes early warning and lane keeping assist. When the risk level is emergency risk, an emergency avoidance strategy is output regardless of the intention state.

[0016] In some specific embodiments, the hierarchical intervention strategy in the action execution module includes, in order of increasing intervention intensity, an early warning layer, a warning layer, an active inhibition layer, and an emergency avoidance layer.

[0017] The beneficial effects of the above technical solution are as follows: (1) Accurate intent insight: By using the concept of “effective gaze” and combining turn signal and steering wheel signals, accurate and early judgment of the driver’s subjective intent is achieved, which fundamentally reduces the false alarm rate.

[0018] (2) Layered intervention strategy: It changes the traditional single mode of "alarm when there is risk" and provides a smooth transition from prompting to suppression according to the degree of danger of the scene and the driver's responsibility, which not only ensures safety but also optimizes the driving experience.

[0019] (3) Proactive safety protection: In response to the dangerous scenario of "unconscious lane change", it can provide early and low-intrusion warnings through touch and other means before the vehicle has obvious lateral displacement, so as to prevent problems before they occur. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a hierarchical hazardous lane change intervention method based on multimodal intent recognition, provided as an embodiment of the present invention; Figure 2 This is a schematic diagram of a hierarchical hazardous lane change intervention system based on multimodal intent recognition, provided as an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0023] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the invention, and should not be construed as limiting the invention.

[0024] Example 1 One embodiment of the present invention provides a hierarchical dangerous lane change intervention method based on multimodal intent recognition, comprising: Step S1: Acquire the driver's multimodal intent signal in real time. The multimodal intent signal includes the driver's gaze focus information, turn signal switch status, and steering wheel angle signal. Specifically, an in-vehicle camera is used to capture images of the driver's eyes, and an algorithm is used to define virtual attention areas for the left / right exterior rearview mirrors and the interior rearview mirror. When the driver's gaze remains in any area for more than 200 milliseconds (calibration value), it is recorded as a "valid gaze".

[0025] Obtain the turn signal switch status (left / right / off) and steering wheel angle and angular velocity signals from the CAN bus.

[0026] Based on the above input, the driver's intentions are divided into three levels in real time: The turn signal is activated, and within 3 seconds before and after activation, there is at least one "effective gaze" onto the target side exterior rearview mirror.

[0027] Potential lane change intention: Turn signal not activated, but one of the following conditions exists: (a) Within 2 seconds, make multiple effective eye contact with the target side exterior and interior rearview mirrors.

[0028] (b) There is a valid gaze at the target side rearview mirror, accompanied by continuous steering wheel angle input (angle > 5 degrees).

[0029] No intention to change lanes / be distracted: (No intention to change lanes): The vehicle was driving normally in the center without using the turn signal and its line of sight was not within any of the interior or exterior rearview mirror ROIs.

[0030] Distraction / fatigue intent: failure to use turn signals, unfocused vision (not focusing on any interior or exterior rearview mirror ROI or the road ahead), or excessive eye closure (suspected fatigue), but the vehicle begins to unconsciously drift out of its lane.

[0031] This application uses a driver monitoring system (DMS) that does not simply detect head orientation, but precisely determines whether the driver's gaze lingers for more than a set threshold (e.g., 200ms) in specific areas of the left and right exterior rearview mirrors and interior rearview mirror, thereby eliminating unintentional glances and confirming that the driver is intentionally observing the environment.

[0032] Step S2: Based on the multimodal intention signal, determine the driver's current lane change intention state; the lane change intention state includes: explicit lane change intention, potential lane change intention, and no lane change or distracted intention; In one specific embodiment of the present invention, step S2 includes: When the turn signal is activated and there is at least one effective gaze at the target-side outside rearview mirror within a preset first time window before and after activation, it is determined as a clear lane-changing intention; When the turn signal is not activated and no effective gaze at the left and right outside rearview mirrors and the inside rearview mirror is detected, it is determined as no lane-changing or distraction intention; When the turn signal is not activated but meets one of the following conditions, it is determined as a potential lane-changing intention: a) There are multiple effective gazes at the target-side outside rearview mirror and / or the inside rearview mirror within a preset second time window; b) There is at least one effective gaze at the target-side rearview mirror and at the same time there is a continuous input of the steering wheel angle; Wherein, the effective gaze means that the driver's line of sight continuously stays within the preset attention area of the corresponding rearview mirror for more than the set time threshold.

[0033] Step S3: Detect the vehicle's side and rear environment information in real time, and determine the collision risk level of the current lane change based on the environment information; In a specific embodiment of the present invention, in step S3, the collision risk level is divided based on the predicted time to collision (TTC) between the target vehicle on the side and rear and the vehicle itself, and at least includes four levels: low risk, medium risk, high risk and emergency risk.

[0034] Use side and rear millimeter-wave radar and cameras to detect the target on the side and rear of the vehicle.

[0035] Based on the relative distance and speed between the target and the vehicle itself, calculate the predicted time to collision (TTC), and define four risk levels: Low risk: TTC > 5.0 s or no target.

[0036] Medium risk: 3.0 s < TTC ≤ 5.0 s.

[0037] High risk: TTC ≤ 3.0 s.

[0038] Emergency risk: TTC ≤ 1.5 s.

[0039] Step S4: Input the lane-changing intention state and the collision risk level into a pre-stored decision matrix for matching, and output the corresponding hierarchical intervention strategy; the decision matrix includes the intervention strategies corresponding to the combinations of different lane-changing intention states and different collision risk levels; In a specific embodiment of the present invention, in step S4, the decision matrix at least includes the following matching relationships: When the intention state is a clear lane-changing intention and the risk level is low risk, output no intervention strategy; When the intention state is a clear lane change intention and the risk level is high risk, a dissuasion or suppression strategy is output, which includes a strong warning and the application of a reverse steering torque. When the intention state is a potential lane change intention and the risk level is medium risk, an early warning strategy is output, which includes a medium-level early warning and a tactile early warning. When the intention state is a potential lane change intention and the risk level is high risk, a strong warning and suppression strategy is output, which includes a strong warning, applying a reverse steering torque and coordinated braking. When the intention state is no lane change or distraction intention and the risk level is high risk, a warning and correction strategy is output, which includes early warning and lane keeping assist. When the risk level is emergency risk, an emergency avoidance strategy is output regardless of the intention state.

[0040] Specifically, the "intent-risk" fusion decision matrix is ​​shown in the table below: The method provided in this application no longer handles environmental risks or driver operations in isolation, but instead uses a preset decision matrix to match the "driver's intention state" with the "environmental risk level" in real time, thereby making the most reasonable decision.

[0041] Step S5: Execute the warning and / or vehicle control actions corresponding to the tiered intervention strategy.

[0042] In a specific embodiment of the present invention, in step S5, the layered intervention strategy includes, in order of increasing intervention intensity, an early warning layer, a warning layer, an active inhibition layer, and an emergency avoidance layer.

[0043] Specifically, a layered intervention strategy is implemented by setting up a warning unit and a suppression unit. The warning unit includes a buzzer, an instrument panel display, and a steering wheel vibration motor. The suppression unit is deeply integrated with the steering and braking systems to perform active steering and braking interventions.

[0044] This invention addresses the core problem of existing technologies' inability to effectively distinguish between intentional lane changes and unintentional dangerous lane departures, leading to false alarms or delayed interventions. The system uses a Driver Monitoring System (DMS) to accurately detect the driver's effective gaze at the left and right exterior and interior rearview mirrors, fusing this data with turn signal signals and steering wheel angle to construct a multi-layered driver intent state machine. Simultaneously, it uses external sensors to detect side and rear collision risks. The core of the system lies in an intent-risk fusion decision matrix, which triggers progressive intervention strategies—from silent prompts and tiered warnings to active steering torque suppression and coordinated braking—based on different intent-risk combinations. Especially effective against unintentional dangerous lane changes by drivers who fail to check rearview mirrors and use turn signals, it provides early, accurate, and human-centered warnings and suppression, significantly improving driving safety.

[0045] Example 2 One embodiment of the present invention provides a hierarchical hazardous lane change intervention system based on multimodal intent recognition, referring to... Figure 2 As shown, it includes: Multimodal intent recognition module 10: used to acquire the driver's multimodal intent signals in real time, including the driver's gaze focus information, turn signal switch status and steering wheel angle signal; Environmental risk perception module 20: used to determine the driver's current lane change intention state based on the multimodal intention signal; the lane change intention state includes: clear lane change intention, potential lane change intention, and no lane change or distraction intention; Risk level determination module 30: used to detect the environmental information behind and to the side of the vehicle in real time, and to determine the collision risk level of the current lane change based on the environmental information; Intervention strategy output module 40: is used to match the lane change intention state with the collision risk level in a pre-stored decision matrix and output the corresponding hierarchical intervention strategy; the decision matrix includes intervention strategies corresponding to different combinations of lane change intention states and different collision risk levels; Action execution module 50: used to execute warning and / or vehicle control actions corresponding to the hierarchical intervention strategy.

[0046] In one specific embodiment of the present invention, the environmental risk perception module 20 includes the following steps: When the turn signal is activated, and there is at least one effective gaze at the target side exterior rearview mirror within the preset first time window before and after activation, it is determined to be a clear lane change intention; When the turn signal is not activated and no effective gaze is detected on the left and right exterior rearview mirrors and interior rearview mirror, it is determined that there is no intention to change lanes or be distracted. When the turn signal is not activated, but one of the following conditions is met, it is determined to be a potential lane change intention: a) Within a preset second time window, there are multiple instances of effective gaze at the target-side exterior rearview mirror and / or interior rearview mirror; b) There is at least one effective gaze at the target side rearview mirror, accompanied by continuous steering wheel angle input; The term "effective gaze" refers to the driver's gaze remaining within a preset attention area of ​​the corresponding rearview mirror for a period of time exceeding a set threshold.

[0047] In a specific embodiment of the present invention, the collision risk level in the risk level determination module 30 is divided based on the expected collision time between the target vehicle at the side and the vehicle, and includes at least four levels: low risk, medium risk, high risk and emergency risk.

[0048] In one specific embodiment of the present invention, the decision matrix in the intervention strategy output module 40 includes at least the following matching relationships: When the intent state is a clear lane change intent and the risk level is low risk, the output strategy is no intervention. When the intention state is a clear lane change intention and the risk level is high risk, a dissuasion or suppression strategy is output, which includes a strong warning and the application of a reverse steering torque. When the intention state is a potential lane change intention and the risk level is medium risk, an early warning strategy is output, which includes a medium-level early warning and a tactile early warning. When the intention state is a potential lane change intention and the risk level is high risk, a strong warning and suppression strategy is output, which includes a strong warning, applying a reverse steering torque and coordinated braking. When the intention state is no lane change or distraction intention and the risk level is high risk, a warning and correction strategy is output, which includes early warning and lane keeping assist. When the risk level is emergency risk, an emergency avoidance strategy is output regardless of the intention state.

[0049] In a specific embodiment of the present invention, the action execution module 50 includes, in order of increasing intervention intensity, a layered intervention strategy comprising: an early warning layer, a warning layer, an active inhibition layer, and an emergency avoidance layer.

[0050] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0051] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the functions specified in one or more boxes. Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the invention. Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0052] The methods and apparatus provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0053] In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "a specific embodiment" or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A hierarchical dangerous lane change intervention method based on multi-modal intent recognition, characterized in that, Comprising: Step S1: Real-time acquisition of the driver's multi-modal intention signal, the multi-modal intention signal including the driver's line-of-sight focal point information, the turn signal switch state, and the steering wheel turning angle signal; Step S2: Based on the multi-modal intention signal, determine the current lane change intention state of the driver; The lane change intention state includes: explicit lane change intention, potential lane change intention, and no lane change or distraction intention; Step S3: Real-time detection of the environment information behind the vehicle, and determination of the current lane change collision risk level based on the environment information; Step S4: Input the lane change intention state and the collision risk level into the pre-stored decision matrix for matching, and output the corresponding hierarchical intervention strategy; the decision matrix includes the intervention strategy corresponding to the combination of different lane change intention states and different collision risk levels; Step S5: Perform the warning and / or vehicle control actions corresponding to the hierarchical intervention strategy.

2. The hierarchical hazardous lane change intervention method based on multi-modal intent recognition according to claim 1, characterized in that, Step S2 includes: When the turn signal is activated, and within a preset first time window before and after activation, there is at least one valid gaze on the target side outside mirror, it is determined as explicit lane change intention; When the turn signal is not activated, and no valid gaze on the left and right outside mirrors and the inside mirror is detected, it is determined as no lane change or distraction intention; When the turn signal is not activated, but one of the following conditions is met, it is determined as potential lane change intention: a) Within a preset second time window, multiple valid gazes on the target side outside mirror and / or inside mirror occur; b) There is at least one valid gaze on the target side mirror, accompanied by continuous steering wheel turning angle input; Wherein, the valid gaze refers to the driver's line of sight staying in the preset attention area of the corresponding rearview mirror for more than a set time threshold.

3. The hierarchical hazardous lane change intervention method based on multi-modal intent recognition according to claim 1, characterized in that, In step S3, the collision risk level is divided based on the predicted collision time of the target vehicle behind the vehicle and the vehicle, including at least four levels: low risk, medium risk, high risk, and emergency risk.

4. The hierarchical hazardous lane change intervention method based on multi-modal intent recognition according to claim 1, characterized in that, In step S4, the decision matrix includes at least the following matching relationship: When the intention state is explicit lane change intention and the risk level is low risk, output no intervention strategy; When the intention state is explicit lane change intention and the risk level is high risk, output discouragement or suppression strategy, including strong warning and applying reverse steering torque; When the intention state is potential lane change intention and the risk level is medium risk, output warning strategy, including medium-level warning and haptic warning; When the intention state is potential lane change intention and the risk level is high risk, output strong warning and suppression strategy, including strong warning, applying reverse steering torque, and coordinated braking; When the intention state is no lane change or distraction intention and the risk level is high risk, output warning and correction strategy, including early warning and lane keeping assistance; When the risk level is emergency risk, regardless of the intention state, output emergency avoidance strategy.

5. The hierarchical hazardous lane change intervention method based on multi-modal intent recognition according to claim 1, characterized in that, In step S5, the hierarchical intervention strategy includes, from light to heavy in terms of intervention intensity, in order: warning layer, warning layer, active suppression layer, and emergency avoidance layer.

6. A hierarchical dangerous lane change intervention system based on multi-modal intent recognition, characterized in that, Comprising: A multi-modal intention recognition module is configured to acquire a multi-modal intention signal of a driver in real time, the multi-modal intention signal including a line-of-sight focus information of the driver, a turn signal switch state, and a steering wheel turning angle signal; An environmental risk perception module is configured to determine a current lane-changing intention state of the driver based on the multi-modal intention signal; The lane-changing intention state includes an explicit lane-changing intention, a potential lane-changing intention, and no lane-changing or distraction intention; A risk level determination module is configured to detect environmental information of a side rear of the vehicle in real time, and determine a collision risk level of current lane-changing based on the environmental information; An intervention strategy output module is configured to input the lane-changing intention state and the collision risk level into a pre-stored decision matrix to match, and output a corresponding hierarchical intervention strategy; the decision matrix includes intervention strategies corresponding to combinations of different lane-changing intention states and different collision risk levels; An action execution module is configured to execute a warning and / or vehicle control action corresponding to the hierarchical intervention strategy.

7. The multi-modal intent recognition based hierarchical hazardous lane change intervention system of claim 6, wherein, The environmental risk perception module includes the following steps: When the turn signal is activated, and within a pre-set first time window before and after the activation, there is at least one valid gaze on the target side outside mirror, it is determined as an explicit lane-changing intention; When the turn signal is not activated, and no valid gaze on the left and right outside mirrors and the inside mirror is detected, it is determined as no lane-changing or distraction intention; When the turn signal is not activated, but one of the following conditions is met, it is determined as a potential lane-changing intention: a) Within a pre-set second time window, multiple valid gazes on the target side outside mirror and / or inside mirror occur; b) There is at least one valid gaze on the target side mirror, accompanied by a continuous steering wheel turning angle input at the same time; Wherein, the valid gaze refers to the driver's line of sight staying in the pre-set attention area of the corresponding mirror for more than a set time threshold.

8. The multi-modal intent recognition based hierarchical hazardous lane change intervention system of claim 6, wherein, In the intervention strategy output module, the collision risk level is divided based on the predicted collision time of the side rear target vehicle and the vehicle, and at least includes four levels of low risk, medium risk, high risk and emergency risk.

9. The multi-modal intent recognition based hierarchical hazardous lane change intervention system of claim 6, wherein, In the intervention strategy output module, the decision matrix at least includes the following matching relationship: When the intention state is an explicit lane-changing intention and the risk level is low, no intervention strategy is outputted; When the intention state is an explicit lane-changing intention and the risk level is high, a dissuasion or suppression strategy is outputted, including strong warning and applying a reverse steering torque; When the intention state is a potential lane-changing intention and the risk level is medium, a warning strategy is outputted, including a medium-level warning and a haptic warning; When the intention state is a potential lane-changing intention and the risk level is high, a strong warning and suppression strategy is outputted, including a strong warning, applying a reverse steering torque, and cooperative braking; When the intention state is no lane-changing or distraction intention and the risk level is high, an alert and correction strategy is outputted, including an early warning and a lane keeping assistance; When the risk level is emergency risk, no matter what the intention state is, an emergency avoidance strategy is outputted.

10. The multi-modal intent recognition based hierarchical hazardous lane change intervention system of claim 6, wherein, In the action execution module, the layered intervention strategy comprises, in order of intervention intensity from light to heavy, a pre-warning layer, a warning layer, an active inhibition layer and an emergency avoidance layer.

Images