Lane departure intention recognition method and device, electronic equipment and storage medium

Abstract

The invention discloses a lane departure intention recognition method and device, electronic equipment and a storage medium, and relates to the technical field of driving safety recognized.The lane departure intention recognition method comprises the steps that before a vehicle deviates transversely, a sight area transfer sequence of a target driving object in the vehicle in a first preset time window is obtained, the first preset time window is a time window of a preset duration before the current moment; under the condition that the sight line area transfer sequence does not conform to a preset sight line transfer mode, determining an intention recognition probability that the target driving object has an active lane departure intention based on the sight line area transfer sequence and a trained intention recognition model; and based on the intention recognition probability and a preset probability threshold, determining whether the target driving object has an active lane departure intention at the current moment. According to the invention, the accuracy of lane departure intention recognition can be improved, and the user experience and the acceptability of the user to a driving assistance system can be improved.

Description

Technical Field

[0001] This application relates to the field of driving safety recognition technology, and in particular to a method, device, electronic device and storage medium for lane departure intention recognition. Background Technology

[0002] Currently, with social development and technological progress, advanced driver assistance systems have become standard equipment in intelligent vehicles. Traditional lane departure warning systems can effectively prevent a large number of traffic accidents by monitoring vehicles' unintentional deviation from their lanes.

[0003] However, existing systems generally suffer from poor accuracy in recognizing lane departure intentions, which means that unnecessary alarms are generated when the driver is consciously changing lanes, affecting user experience and user acceptance of the driver assistance system. Summary of the Invention

[0004] This application provides a lane departure intention recognition method, device, electronic device, and storage medium. The embodiments provided by this application solve the problem of poor accuracy in recognizing lane departure intentions in the prior art, which affects user experience and user acceptance of driving assistance systems. The embodiments provided by this application can improve the accuracy of lane departure intention recognition, thereby improving user experience and user acceptance of driving assistance systems.

[0005] In a first aspect, this application provides a lane departure intention recognition method, which includes: Before the vehicle deviates laterally, the sequence of the line of sight movement of the target driving object inside the vehicle under the first preset time window is obtained, wherein the first preset time window is a time window of preset duration before the current moment; When the gaze region transfer sequence does not conform to the preset gaze transfer pattern, the probability of determining the intention of the target driving object to actively deviate from the lane is determined based on the gaze region transfer sequence and the trained intention recognition model. Based on the intent recognition probability and a preset probability threshold, it is determined whether the target driver has an intention to actively deviate from the lane at the current moment.

[0006] In one feasible implementation, based on the intent recognition probability and a preset probability threshold, it is determined whether the target driver intends to actively deviate from the lane at the current moment, including: If the probability of intent recognition is greater than or equal to a preset probability threshold, then it is determined that the target driving object has an intention to actively deviate from the lane at the current moment. If the probability of intent recognition is less than a preset probability threshold, it is determined that the target driving object does not have an intention to actively deviate from the lane at the current moment.

[0007] In one feasible implementation, the method further includes: If the gaze transfer sequence matches the preset gaze transfer pattern, it is determined that the target driver has an intention to actively deviate from the lane at the current moment.

[0008] In one feasible implementation, the preset gaze shift mode includes a scanning observation mode and a continuous gaze mode, and the method further includes: When the gaze region transfer sequence does not conform to the scanning observation mode and does not conform to the continuous gaze mode, it is determined that the gaze region transfer sequence does not conform to the preset gaze transfer mode. When the gaze region transfer sequence matches either the scanning observation mode or the continuous gaze mode, it is determined that the gaze region transfer sequence matches the preset gaze transfer mode. Among them, the scanning observation mode is used to characterize the observation mode in which the gaze moves from the road area ahead to at least one target monitoring area and then returns to the road area ahead; the continuous gaze mode is used to characterize the gaze mode in which the gaze is continuously focused on at least one target monitoring area for a duration exceeding a preset time.

[0009] In one feasible implementation, acquiring the gaze region shift sequence of the target driving object inside the vehicle within a first preset time window includes: Based on the vehicle operation information of the target driver inside the vehicle, and in the case that it is determined that the target driver is not actively controlling the vehicle, real-time line-of-sight information of the target driver is obtained. Among them, the real-time gaze area information is obtained by mapping the facial image of the target driver to multiple preset discretized gaze areas after gaze tracking processing. The preset discretized gaze areas include the road area ahead and the target monitoring area. Based on real-time line-of-sight information, the line-of-sight transfer sequence of the target driving object under the first preset time window is determined.

[0010] In one feasible implementation, the operational information includes turn signal information, brake pedal opening information, accelerator pedal opening information, and the target driver's hand torque information. Based on the target driver's operational information on the vehicle, it is determined that the target driver actively controls the vehicle, including: If the target driver inside the vehicle controls the turn signal to "turn on," then it is determined that the target driver is actively controlling the vehicle; or If the brake pedal opening information of the target driver inside the vehicle is greater than a first preset opening value, then it is determined that the target driver is actively controlling the vehicle; or If the accelerator pedal opening information of the target driver inside the vehicle is greater than the second preset opening value, then it is determined that the target driver is actively controlling the vehicle; or If the hand torque of the target driver inside the vehicle is greater than a preset hand torque threshold, then it is determined that the target driver is actively controlling the vehicle.

[0011] In one feasible implementation, after determining that the target driver intends to actively deviate from the lane at the current moment, the method further includes: Determine whether the vehicle has lateral deviation within the second preset time window, wherein the second preset time window is the effective time window corresponding to the active lane departure intention, and the start time of the second preset time window is the current time; If so, the vehicle's status information and the target driver's hand torque information are obtained, and the intention to actively deviate from the lane is verified based on the status information and hand torque information.

[0012] In one feasible implementation, the state information includes the vehicle's lateral deviation direction, and the hand torque information includes the hand torque direction and magnitude. Based on the state information and hand torque information, the active lane departure intention is verified, including: If the lateral deviation direction of the vehicle is consistent with the direction of the hand torque, and the magnitude of the hand torque is greater than or equal to the preset hand torque deviation threshold, then the active lane departure intention of the target driving object at the current moment is determined to be a true active lane departure intention.

[0013] In one feasible implementation, before determining the probability of an active lane departure intention by the target driver based on the gaze region transfer sequence and a trained intention recognition model, the method further includes: Construct the graph nodes corresponding to each preset discretized gaze region to obtain the node set; Based on the spatial relationships between multiple preset discrete view regions, the edges between each graph node are determined, and the edge set is obtained. The target graph structure is generated based on the set of nodes and the set of edges.

[0014] In one feasible implementation, a graph node corresponding to each preset discretized gaze region is constructed to obtain a node set, including: Determine the region categories of the preset discretized gaze area, which include the road ahead area, the left rearview mirror area, the right rearview mirror area, the left window area, the right window area, the interior rearview mirror area, the dashboard area, the center console area, and the interior gaze discretized area. Based on the region category of each preset discretized gaze region, construct the graph node corresponding to each preset discretized gaze region; The set of graph nodes is defined as the node set.

[0015] In one feasible implementation, a graph node corresponding to each preset discretized gaze region is constructed to obtain a node set, including: Determine the region categories of the preset discretized gaze area, which include the road ahead area, the left rearview mirror area, the right rearview mirror area, the left window area, the right window area, the interior rearview mirror area, the dashboard area, the center console area, and the interior gaze discretized area. Based on the region category of each preset discretized gaze region, construct the graph node corresponding to each preset discretized gaze region; The set of graph nodes is defined as the node set.

[0016] In one feasible implementation, the spatiotemporal convolution module includes at least two sequentially cascaded spatiotemporal convolutional blocks, each including a sequentially cascaded temporal convolutional layer and a spatial graph convolutional layer; the initial node feature tensor is subjected to spatiotemporally enhanced convolution processing by the spatiotemporal convolution module to obtain a spatiotemporally enhanced feature tensor, including: For each spatiotemporal convolutional block, the initial node feature tensor is processed by convolution of the feature sequence in the time dimension through a temporal convolutional layer to obtain a temporally enhanced feature tensor. The temporal augmentation feature tensor is obtained by performing spatial graph convolution processing on the temporal augmentation feature tensor through spatial graph convolution layers.

[0017] In one feasible implementation, the trained intent recognition model is determined in the following manner: Based on at least one historical line-of-sight area information of a historical driving object output by the driver monitoring system, determine at least one historical line-of-sight area transfer sequence of the historical driving object under a historical preset time window; Based on all historical line-of-sight region transfer sequences and the historical intent label corresponding to each historical line-of-sight region transfer sequence, a historical training dataset is constructed. The historical intent label is used to characterize whether the historical driving object had an active lane departure intention. Input the historical training dataset into the initial spatiotemporal graph convolutional model to determine the predicted historical intent recognition probability of historical driving objects having active lane departure intent; Based on historical intent labels and predicted historical intent recognition probabilities, the initial spatiotemporal graph convolutional model is iteratively trained to determine the trained intent recognition model.

[0018] In one feasible implementation, the historical intent label includes positive sample labels, and the historical intent label corresponding to the historical gaze region transfer sequence is determined by the following method: If, within the historical preset time window, the historical driving status information of the historical driving object meets the first preset historical driving condition, then the historical intention label corresponding to the historical line-of-sight area transfer sequence is determined to be a positive sample label. The first preset historical driving conditions include: The driver in the previous video turned on the vehicle's turn signal; Furthermore, the vehicle controlled by the historical driver completed the lane change operation; Furthermore, the direction of the hand torque applied by the driver in the past was consistent with the direction of the vehicle's lane change; Furthermore, the duration of the historical driving subject's dwell time in the line of sight area on the same side as the lane change direction is longer than the preset dwell time.

[0019] In one feasible implementation, the historical intent label includes negative sample labels, and the historical intent label corresponding to the historical gaze region transfer sequence is determined by the following method: If, within the historical preset time window, the historical driving status information of the historical driving object meets the second preset historical driving condition, then the historical intention label corresponding to the historical line-of-sight area transfer sequence is determined to be a negative sample label. The second preset historical driving conditions include: The vehicle's turn signals were not activated by the driver in the past; Furthermore, the direction of the hand torque applied by the driver in the past was inconsistent with the direction of the vehicle's lane change; Furthermore, the dwell time of the historical driving object in the line of sight area on the same side as the lane change direction is less than or equal to the preset dwell time.

[0020] In a second aspect, this application provides a lane departure intention recognition device, which includes: The acquisition module is used to acquire the line-of-sight region transfer sequence of the target driving object inside the vehicle under a first preset time window before the vehicle causes lateral deviation, wherein the first preset time window is a time window of a preset duration before the current moment; The first determining module is used to determine the probability of intention recognition that the target driving object has an active lane departure intention based on the line-of-sight transfer sequence and the trained intention recognition model when the line-of-sight transfer sequence does not conform to the preset line-of-sight transfer pattern. The second determination module is used to determine whether the target driving object has an intention to actively deviate from the lane at the current moment, based on the intent recognition probability and a preset probability threshold.

[0021] In a third aspect of this application, an electronic device is provided, including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the lane departure intention recognition method described above.

[0022] In a fourth aspect, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the intent recognition method described above.

[0023] In a fifth aspect of this application, an embodiment of this application provides a vehicle equipped with a lane departure intention recognition device.

[0024] In a sixth aspect of this application, an embodiment of this application provides a computer program product that stores a computer program or computer-executable instructions. When the computer program product is run by a processor, it performs the steps of the intent recognition method described above.

[0025] Compared with the prior art, the lane departure intention recognition method, device, electronic device and storage medium provided in this application have the following advantages: Before the vehicle deviates laterally, the embodiments of this application acquire the gaze region transfer sequence of the target driving object in the vehicle under a first preset time window. If the gaze region transfer sequence does not conform to the preset gaze transfer pattern, the probability of the target driving object having an active lane departure intention is determined based on the gaze region transfer sequence and the trained intention recognition model. Then, based on the intention recognition probability and a preset probability threshold, it is determined whether the target driving object has an active lane departure intention at the current moment. This application can improve the accuracy of lane departure intention recognition, thereby improving the user experience and the user's acceptance of the driving assistance system. Attached Figure Description

[0026] Figure 1 A flowchart illustrating a lane departure intention recognition method provided in an embodiment of this application is shown. Figure 2 The flowchart illustrates the verification of active lane departure intent in a lane departure intent recognition method provided in an embodiment of this application. Figure 3 This diagram illustrates a structural block diagram of a lane departure intention recognition device provided in an embodiment of this application. Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown.

[0027] Figure 3 and Figure 4 The correspondence between the figure labels and figure titles in the accompanying drawings is as follows: 300 Lane departure intention recognition device; 310 Acquisition module; 320 First determination module; 330 Second determination module; 340 Third determination module; 350 Judgment module; 360 Verification module; 400 Electronic device; 410 Processor; 420 Memory; 430 Bus. Detailed Implementation

[0028] To better understand the technical solutions provided in the embodiments of this specification, the technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0029] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element. The term "two or more" includes two or more cases.

[0030] First, the applicable application scenarios of this application will be introduced. The embodiments provided in this application are applicable to the field of driving safety recognition technology, and in particular relate to a lane departure intention recognition method, device, electronic device and storage medium.

[0031] Currently, with social development and technological progress, advanced driver assistance systems have become standard equipment in intelligent vehicles. Traditional lane departure warning systems can effectively prevent a large number of traffic accidents by monitoring vehicles' unintentional deviation from their lanes.

[0032] However, existing systems generally suffer from poor accuracy in recognizing lane departure intentions, which means that unnecessary alarms are generated when the driver is consciously changing lanes, affecting user experience and user acceptance of the driver assistance system.

[0033] Traditional lane departure intention estimation methods use a trained machine learning model to predict lane departure intention when the driver is in a normal state, and directly determine that there is no lane departure intention when an abnormal state is detected. This achieves intelligent recognition of driver intention and accurate matching of vehicle control strategy. However, traditional machine learning models cannot handle memory defects in long-distance temporal dependency problems. They treat continuous gaze data as a one-dimensional sequence and ignore the inherent spatial topological relationship between different gaze areas.

[0034] Furthermore, traditional lane departure warning systems have a common problem: they beep erratically whenever the lane line is crossed, failing to distinguish between actions such as using turn signals, checking rearview mirrors, and changing lanes, which indicate an intention to actively deviate from the lane. This inaccurate alarms also negatively impact the user experience.

[0035] Based on this, the embodiments of this application provide a lane departure intention recognition method, device, electronic device and storage medium. The embodiments provided by this application solve the problem of poor accuracy in recognizing lane departure intentions in the prior art, which affects user experience and user acceptance of driving assistance systems. The embodiments provided by this application can improve the accuracy of lane departure intention recognition, thereby improving user experience and user acceptance of driving assistance systems.

[0036] Figure 1 This is a flowchart illustrating a lane departure intention recognition method provided in an embodiment of this application. Figure 1 As shown, the lane departure intention recognition method includes the following steps: S101. Before the vehicle deviates laterally, obtain the line-of-sight region transfer sequence of the target driving object inside the vehicle under the first preset time window, wherein the first preset time window is a time window of preset duration before the current moment.

[0037] In this step, the lane departure intention recognition method provided in the embodiments of this application is generally integrated into the lane departure warning system or the lane keeping assist system to suppress false alarms of the target driver's active lane departure behavior.

[0038] In the embodiments provided in this application, during lane keeping, the target driver's gaze is usually fixed in front of the vehicle. Before the vehicle deviates laterally, that is, when the target driver intends to actively control the vehicle to deviate from the lane, they usually check the left and right side mirrors, left and right side windows and interior rearview mirror to observe the road environment and confirm safety. Only after confirming that the road environment is safe will they perform lane changing or other operations on the vehicle they are driving. At this time, the vehicle will deviate laterally.

[0039] Furthermore, in the embodiments provided in this application, when the vehicle is in a driving state, it is necessary to detect the line of sight area of ​​the target driving object in real time, and before the vehicle deviates laterally, obtain the line of sight area transfer sequence of the target driving object in the vehicle under the first preset time window, and cache the real-time line of sight area transfer sequence of the target driving object in real time according to the sliding first preset time window.

[0040] It is understood that the target driver in the embodiments provided in this application may check the blind spot by turning their head to look over the shoulder blind spot, that is, by turning their head to look left and right through the side windows. In other words, the target driver's line of sight may shift. Therefore, it is possible to determine whether the target driver is consciously deviating from the lane by judging the target driver's line of sight area, the shift of the target driver's line of sight area, and the number of shifts.

[0041] In the embodiments provided in this application, the target driving object can be customized and used according to different application scenarios and usage conditions. Specifically, the target driving object in the embodiments provided in this application can be a driver.

[0042] The size of the first preset time window in the embodiments provided in this application can be customized and used according to different application scenarios and usage conditions. The first preset time window in the embodiments provided in this application can be specifically used... The preset duration can be specified, but is not limited to, 5 seconds.

[0043] In this application, the vehicle, under natural driving conditions, simultaneously records timestamps, vehicle data (including turn signal status and vehicle lateral position), and line-of-sight area data output by the driver monitoring system (DMS), and detects the occurrence time of all lane departure events. (Judged by the vehicle's position relative to the lane lines), and for each deviation event, a fixed time window will be traced back. , extract from arrive All data within the time period.

[0044] S102. When the gaze region transfer sequence does not conform to the preset gaze transfer mode, the probability of determining the intention of the target driving object to actively deviate from the lane is determined based on the gaze region transfer sequence and the trained intention recognition model.

[0045] In this step, the embodiments provided in this application pre-set a preset gaze shift mode to simulate gaze shifting from the main task area to one or more target monitoring areas and finally back to the main task area, or to simulate a gaze that maintains a continuous and stable gaze on the target monitoring area for a duration exceeding a preset time. This is used to initially determine whether the above gaze area shift sequence has the intention of actively deviating from the lane. Only when it is determined that the gaze area shift sequence does not conform to the preset gaze shift mode will the above gaze area shift sequence be input into the trained intention recognition model to determine the probability of the target driving object having the intention of actively deviating from the lane.

[0046] It is understood that in the embodiments provided in this application, a preset gaze shift mode is used to initially determine whether there is an intention to actively deviate from the above gaze area shift sequence, and a trained intention recognition model is used to further accurately determine the probability of the target driving object having an intention to actively deviate from the lane, thereby achieving dual authentication of the intention to actively deviate from the lane, and the two are serially coordinated and progressively advanced.

[0047] Here, in the embodiments provided in this application, since the preset gaze shift pattern heavily relies on the completeness and accuracy of the preset rules, and in actual complex driving scenarios, the visual scanning behavior (i.e., gaze region shift sequence) of the target driving object exhibits significant individual differences and contextual dependence, it is difficult to cover all conscious lane-changing patterns in all real-world scenarios with only a limited set of preset typical gaze region shift sequences, resulting in insufficient system recall. Therefore, it is necessary to design a well-trained intent recognition model to further determine the probability of the target driving object having an intention to actively deviate from the lane, even when the gaze region shift sequence does not conform to the preset gaze shift pattern.

[0048] It should be noted that the main model of the trained intent recognition model in the embodiments provided in this application can be customized and used according to different application scenarios and usage conditions. The trained intent recognition model in the embodiments provided in this application can be specifically set as follows: The Spatial-Temporal Graph Convolutional Network (ST-GCN) model.

[0049] In the embodiments provided in this application, the main task area and the target-side monitoring area can be customized and used according to different application scenarios and usage conditions. The main task area in the embodiments provided in this application can be specifically, but is not limited to, the road area in front; the target-side monitoring area in the embodiments provided in this application can be specifically, but is not limited to, the left rearview mirror area, the left side window area, the right rearview mirror area, and the right side window area, etc.

[0050] S103. Based on the intent recognition probability and the preset probability threshold, determine whether the target driving object has an intention to actively deviate from the lane at the current moment.

[0051] In this step, the embodiments provided in this application, after determining the probability of the target driving object having an intention to actively deviate from the lane, compare the probability of the intention to identify with a preset probability threshold, and determine whether the target driving object has an intention to actively deviate from the lane at the current moment based on the comparison result.

[0052] In the embodiments provided in this application, the preset probability threshold can be customized and used according to different application scenarios and usage conditions. Specifically, the preset probability threshold in the embodiments provided in this application can be set to 0.6.

[0053] Compared with the prior art, the lane departure intention recognition method provided in this application obtains the gaze region transfer sequence of the target driver in the vehicle under a first preset time window before the vehicle lateral deviates. If the gaze region transfer sequence does not conform to the preset gaze transfer pattern, the probability of the target driver having an intention to actively deviate from the lane is determined based on the gaze region transfer sequence and the trained intention recognition model. Then, based on the intention recognition probability and a preset probability threshold, it is determined whether the target driver has an intention to actively deviate from the lane at the current moment. This application can improve the accuracy of lane departure intention recognition, thereby improving the user experience and the user's acceptance of the driving assistance system.

[0054] In one feasible embodiment, determining whether the target driver intends to actively deviate from the lane at the current moment, based on the intent recognition probability and a preset probability threshold, includes: If the intent recognition probability is greater than or equal to a preset probability threshold, it is determined that the target driver has an intention to actively deviate from the lane at the current moment; if the intent recognition probability is less than the preset probability threshold, it is determined that the target driver does not have an intention to actively deviate from the lane at the current moment.

[0055] In the embodiments provided in this application, after determining the probability of the target driving object having an intention to actively deviate from the lane, it is necessary to compare the above-mentioned intention recognition probability with a preset probability threshold. When it is determined that the intention recognition probability is greater than or equal to the preset probability threshold, it is determined that the target driving object has an intention to actively deviate from the lane at the current moment; when it is determined that the intention recognition probability is less than the preset probability threshold, it is determined that the target driving object does not have an intention to actively deviate from the lane at the current moment.

[0056] It should be noted that the probability of recognizing a target driver's intention to actively deviate from the lane can be used... To represent, if If a preset probability threshold is set, the target driver is determined to have an intention to actively deviate from the lane; if If a preset probability threshold is set, it is determined that the target driver has no intention of actively deviating from the lane.

[0057] In this application, a trained intent recognition model is used to learn the gaze region transfer sequence of the target driver and thereby determine the probability of the target driver having an intention to actively deviate from the lane. Compared with simple logical rules and simple braking, this method of learning gaze region transfer sequences to determine the intention to actively deviate from the lane can cover more different drivers' lane departure intentions, can more accurately identify the probability of the target driver having an intention to actively deviate from the lane, and can be applied to various different working conditions.

[0058] In one feasible embodiment, the method further includes: If the gaze transfer sequence matches the preset gaze transfer pattern, it is determined that the target driver has an intention to actively deviate from the lane at the current moment.

[0059] In the embodiments provided in this application, when it is determined that the gaze region transfer sequence of the target driving object under the first preset time window conforms to the preset gaze transfer mode, it is not necessary to input the gaze region transfer sequence into the trained intention recognition model. It can be directly determined that the target driving object has an active lane departure intention at the current moment. At this time, it is not necessary to issue an alarm to the vehicle to stop the target driving object.

[0060] In this application, by directly determining that the target driving object has an active lane departure intention at the current moment by the gaze area transfer sequence conforming to the preset gaze transfer mode, false alarms caused by incorrect recognition of the vehicle's active lane departure intention are reduced, thereby enhancing the user experience.

[0061] In one feasible embodiment, the preset gaze shift mode includes a scanning observation mode and a continuous gaze mode, and the method further includes: When the gaze region transfer sequence does not conform to the scanning observation mode and does not conform to the continuous gaze mode, it is determined that the gaze region transfer sequence does not conform to the preset gaze transfer mode; when the gaze region transfer sequence conforms to either the scanning observation mode or the continuous gaze mode, it is determined that the gaze region transfer sequence conforms to the preset gaze transfer mode; wherein, the scanning observation mode is used to characterize the observation mode in which the gaze moves from the road area ahead to at least one target monitoring area and then returns to the road area ahead; the continuous gaze mode is used to characterize the gaze mode in which the gaze is continuously focused on at least one target monitoring area for a duration exceeding a preset duration.

[0062] In the embodiments provided in this application, the preset gaze shift mode includes a scanning observation mode and a continuous gaze mode. The scanning observation mode is used to characterize a complete and dynamic scanning closed-loop mode of the gaze. The scanning observation mode is used to characterize the gaze to maintain a continuous and stable gaze on the target-side monitoring area for a duration exceeding a preset preset duration (e.g., 0.5 seconds, 1 second, or 2 seconds), indicating that the target driving object is conducting a deep and careful observation of the area to assess safety risks.

[0063] The scanning observation mode can be specific, but is not limited to: [Road ahead, left rearview mirror, road ahead]; [Road ahead, left side window, road ahead]; And [the road ahead, the left rearview mirror, the left side window, the road ahead], etc.

[0064] In the above, the [road ahead, left rearview mirror, road ahead] in the scan observation mode can be interpreted as the target driver glancing at the left rearview mirror from directly in front to confirm safety, and then returning their gaze to the road ahead. This is the simplest pre-lane change check in the scan observation mode.

[0065] For example, in the scanning observation mode, [road ahead, left rearview mirror, left side window, road ahead] can be interpreted as the target driver glancing at the left rearview mirror from directly in front to confirm safety, then looking at the left side window, and then returning their gaze to the road ahead. This is a more cautious and complete inspection process in the scanning observation mode.

[0066] Returning our gaze to the front, this is one of the simplest pre-lane change checks in the scanning observation mode.

[0067] The sustained gaze mode can be specific, but is not limited to: Continuous gaze on the left rearview mirror area; A sustained gaze over the left-side window area; And patterns such as sustained gaze on the right rearview mirror area.

[0068] It should be noted that, in the embodiments provided in this application, when the gaze region transfer sequence conforms to either the scanning observation mode or the continuous gaze mode, it is determined that the gaze region transfer sequence does not conform to the preset gaze transfer mode. At this time, the subsequent determination of the probability of active lane departure intention can be performed. That is, as long as the gaze region transfer sequence meets the scanning observation mode or the continuous gaze mode, it is not necessary to further identify the active lane departure intention through the trained intention recognition model.

[0069] It is understood that the preset gaze shift mode in the embodiments provided in this application can be specifically a series of normative visual attention patterns followed to confirm the safety of the side and rear environment before determining that the target driving object is making a conscious and safe lane change (i.e., the vehicle is laterally deviating). Its essential feature is the target monitoring area.

[0070] In the embodiments provided in this application, the target monitoring area can be customized and used according to different application scenarios and usage conditions. The target monitoring area in the embodiments provided in this application can be specifically set as the target side monitoring area, and more specifically as the monitoring area of ​​the road in front, the left rearview mirror, the right rearview mirror, the left window, the right window, the interior rearview mirror, and other non-road areas not mentioned above.

[0071] In this application, before inputting the gaze region transfer sequence into the trained intent recognition model, the gaze region transfer sequence is first judged to see if it conforms to the preset gaze transfer mode. This avoids re-inputting the gaze region transfer sequence of the target driving object that already has the intention to actively deviate from the lane into the trained intent recognition model for repeated recognition, thereby improving the efficiency and accuracy of lane departure intent recognition. Moreover, the above recognition method has low requirements for computing resources and is easy to debug and implement.

[0072] In one feasible embodiment, obtaining the gaze region transfer sequence of a target driving object inside the vehicle within a first preset time window includes: Based on the vehicle operation information of the target driver inside the vehicle, and assuming that the target driver is not actively controlling the vehicle, the real-time gaze region information of the target driver is obtained. The real-time gaze region information is obtained by mapping the facial image of the target driver to multiple preset discretized gaze regions after gaze tracking processing. The preset discretized gaze regions include the road area ahead and the target monitoring area. Based on the real-time gaze region information, the gaze region transfer sequence of the target driver under the first preset time window is determined.

[0073] In the above-described embodiments, the target driver's operation information on the vehicle before the lateral drift event occurs is monitored. Based on the operation information, it is determined whether the target driver actively controls the vehicle. If it is determined that the target driver actively controls the vehicle, it is not necessary to obtain the target driver's line of sight transfer sequence in the first preset time window. The target driver's target operation on the vehicle can be directly executed, such as turning on the turn signal, changing the driver's hand torque, pressing the brake pedal, etc.

[0074] If it is determined that the target driver is not actively controlling the vehicle, then a high-resolution image of the target driver's face needs to be captured by a Driver Monitoring System (DMS). Based on the facial image and the internal algorithm of the DMS, the target driver's eye movement posture and head posture are determined. Based on the eye movement posture, head posture, and a preset algorithm, the target driver's gaze position coordinates are determined. Then, based on the gaze position coordinates and a preset region discretization algorithm, at least one preset discretized gaze region of the target driver is determined, thereby obtaining real-time gaze region information of the target driver under multiple preset discretized gaze regions. Finally, based on the real-time gaze region information, the gaze region transfer sequence of the target driver under a first preset time window is determined.

[0075] It is understood that the multiple preset discretized gaze areas include: the road ahead, the left rearview mirror, the right rearview mirror, the left window, the right window, the interior rearview mirror, the dashboard, the center console, and other vehicle detection areas. Furthermore, the multiple preset discretized gaze areas in the embodiments provided in this application can be specifically used... This expression represents, where... , It is a collection of multiple pre-defined discretized gaze regions. Here, the embodiments provided in this application are defined as follows: This means dividing the target driver's field of vision into nine key regions of interest, namely the road ahead. Left rearview mirror area Right side rearview mirror area Left side window area Right side window area Rearview mirror area Dashboard area Central control area Other vehicle inspection areas .

[0076] In the embodiments provided in this application, the high-definition facial image of the target driving object can be customized and used according to different application scenarios and usage conditions. Specifically, the high-definition facial image of the target driving object in the embodiments provided in this application can be set as an image of the eye area and a head posture image.

[0077] Here, eye movement posture includes the angle of eye movement, which is calculated by identifying the center of the pupil and the corneal reflective point; head posture includes the head tilt and pitch angles.

[0078] The first preset time window can be customized and used according to different application scenarios and usage conditions. The first preset time window in the embodiments provided in this application can be used... Seconds are used to represent time.

[0079] In this application, by obtaining the gaze region transfer sequence of the target driver under a first preset time window, and matching the gaze region transfer sequence under the first preset time window with a predefined gaze transfer pattern representing a "safety confirmation" behavior pattern, it is determined whether the target driver has a clear intention to actively deviate from the lane. Instead of directly inputting all gaze region transfer sequences into the trained intention recognition model to calculate the probability of intention recognition, the accuracy and efficiency of intention recognition probability are improved, and the robustness of the trained intention recognition model is enhanced.

[0080] In one feasible embodiment, the operational information includes turn signal information, brake pedal opening information, accelerator pedal opening information, and the target driver's hand torque information. Based on the target driver's operational information on the vehicle, it is determined that the target driver actively controls the vehicle, including: If the target driver inside the vehicle controls the turn signal to turn on, it is determined that the target driver is actively controlling the vehicle; or if the target driver inside the vehicle controls the brake pedal opening to a value greater than a first preset value, it is determined that the target driver is actively controlling the vehicle; or if the target driver inside the vehicle controls the accelerator pedal opening to a value greater than a second preset value, it is determined that the target driver is actively controlling the vehicle; or if the target driver inside the vehicle controls the hand torque to a value greater than a preset hand torque threshold, it is determined that the target driver is actively controlling the vehicle.

[0081] In the embodiments provided in this application, when it is determined that the target driver inside the vehicle controls the turn signal to turn off, or the target driver controls the brake pedal opening to be less than or equal to a first preset opening value, or the target driver controls the accelerator pedal opening to be less than or equal to a second preset opening value, or the target driver controls the hand torque to be less than or equal to a preset hand torque threshold, then it is determined that the target driver is not actively controlling the vehicle.

[0082] In the embodiments provided in this application, the first preset opening value and the second preset opening value can be customized and used according to different application scenarios and usage conditions.

[0083] In this application, by pre-determining the target driver's operation information on the vehicle, high-precision active lane departure intention recognition is achieved, reducing the occurrence of false alarms.

[0084] In one feasible implementation, after determining that the target driver intends to actively deviate from the lane at the current moment, the method further includes: determining whether the vehicle has laterally deviated within a second preset time window, wherein the second preset time window is the effective time window corresponding to the active lane deviation intention, and the start time of the second preset time window is the current moment; if so, then acquiring the vehicle's state information and the target driver's hand torque information, and verifying the active lane deviation intention based on the state information and hand torque information.

[0085] The status information includes the vehicle's lateral deviation direction, and the hand torque information includes the hand torque direction and the hand torque magnitude. Based on the status information and the hand torque information, the active lane departure intention is verified, including: if the vehicle's lateral deviation direction is consistent with the hand torque direction, and the hand torque magnitude is greater than or equal to a preset hand torque deviation threshold, then the active lane departure intention of the target driving object at the current moment is determined to be a true active lane departure intention.

[0086] In the embodiments provided in this application, after determining that the target driver intends to actively deviate from the lane at the current moment, it is necessary to determine whether the vehicle has laterally deviated within the effective window specified by the second preset time window. If the vehicle laterally deviates within the effective window specified by the second preset time window, the target driver's hand torque information and the vehicle's state information are cross-validated. At this time, the movement of the vehicle relative to the lane line is continuously monitored. After completing the determination of the target driver's intention to actively deviate from the lane, within the second preset time window of the intention to actively deviate from the lane, the vehicle's movement is continuously monitored. Only when a lateral offset event is detected on the corresponding side of the vehicle will the final cross-validation process be initiated, which involves cross-validating the hand torque information of the target driver with the vehicle's status information.

[0087] It should be noted that the embodiments provided in this application require the target driver's control input signal during the cross-validation process of the target driver's hand torque information and the vehicle's state information. Specifically, it detects whether there is dominant hand torque information consistent with the deviation direction. When it is determined that the hand torque direction in the hand torque information is the same as the direction of the vehicle's lateral deviation, and the torque magnitude is greater than a preset hand torque threshold, it is determined that the vehicle's lateral deviation event is caused by the target driver's active control. Together with his visual intention (i.e., the intention recognition probability output by the trained intention recognition model), it constitutes a complete cross-validation process, thereby ultimately confirming it as a genuine active lane departure intention.

[0088] Understandably, if the direction of the hand torque in the hand torque information is not the same as the direction of the vehicle's lateral deviation or does not reach the preset hand torque threshold, it indicates that the vehicle's lateral movement was not intentional on the part of the target driver, and thus the vehicle's lateral deviation event is ultimately determined to be a "passive (unintentional) deviation".

[0089] The second preset time window provided in the embodiments of this application can be specifically used. To express.

[0090] The preset hand torque threshold provided in the embodiments of this application can be customized and used according to different application scenarios and usage conditions.

[0091] Figure 2 This diagram illustrates a flowchart of a lane departure intention recognition method provided in this application, which verifies the intention to actively deviate from lanes. Figure 2 As shown, verifying the intention to actively deviate from lanes mainly includes the following sub-steps: S0. The vehicle is in lane keeping mode.

[0092] S1. When the target driver is detected to have an intention to actively deviate from the lane at the current moment, wait for the vehicle to lateral deviate event to occur. This state will last for a fixed second preset time window. .

[0093] S2. Triggered by an event—in The window enters this state after detecting a lateral drift event of the vehicle. In this state, the final decision is made and the final judgment result is output to confirm whether the target driver does indeed have a genuine intention to actively deviate from the lane.

[0094] S3. When the lateral deviation direction of the vehicle is consistent with the direction of the hand torque, and the magnitude of the hand torque is greater than or equal to the preset hand torque deviation threshold, it is determined that the vehicle meets the hand torque condition and that the target driver does indeed have a genuine intention to actively deviate from the lane; otherwise, it is determined that the vehicle does not meet the hand torque condition and that the target driver does not have a genuine intention to actively deviate from the lane.

[0095] In this application, after determining that the target driver intends to actively deviate from the lane at the current moment, the embodiments provided in this application will also perform differential verification between the intention recognition probability of active lane deviation at the current moment output by the trained intention recognition model and the hand torque information occurring within the second preset time window, thereby achieving comprehensive and robust recognition of the target driver's lane deviation intention and greatly improving the user experience.

[0096] In one feasible embodiment, before determining the probability of an active lane departure intention by the target driver based on the gaze region transfer sequence and the trained intention recognition model, the method further includes: Construct graph nodes corresponding to each preset discretized gaze region to obtain a node set; determine the edges between each graph node based on the spatial relationship between multiple preset discretized gaze regions to obtain an edge set; generate the target graph structure based on the node set and the edge set.

[0097] In the embodiments provided in this application, as described above, a unique graph node needs to be constructed for each preset discretized gaze region. And based on each graph node The corresponding node set is constructed, and the embodiments provided in this application define the edges between each graph node based on spatial adjacency and driving task logic. Based on the edges, the edge set is obtained, and then the target graph structure is generated based on the node set and the edge set.

[0098] It should be noted that, in the embodiments provided in this application, the edges between the graph nodes are used to represent that two preset discretized gaze regions have a direct and meaningful logical relationship in the driving context. For example, there are edges between the vehicle's "windshield" and "left rearview mirror", and between the "left rearview mirror" and "left blind spot".

[0099] It is understandable that, in a specific embodiment of this method, a fully connected strategy is used to determine the edges between graph nodes. That is, there exists a connection between any two nodes, i.e.: .

[0100] The node set in the embodiments provided in this application can be specifically obtained through... To represent, and each graph node permanently represents a pre-defined discretized gaze region. , arrive It represents 9 graph nodes, and each graph node can be initialized with a feature vector.

[0101] In this application, a graph node corresponding to each preset discretized gaze region is constructed to obtain a node set, including: The region categories of the preset discretized gaze regions are determined, including the road ahead region, the left rearview mirror region, the right rearview mirror region, the left window region, the right window region, the interior rearview mirror region, the dashboard region, the center console region, and the interior gaze discrete region. Based on the region category of each preset discretized gaze region, graph nodes corresponding to each preset discretized gaze region are constructed. The set of graph nodes is determined as a node set.

[0102] In the embodiments provided in this application, the set of graph nodes is determined as a node set that can be represented by a graph structure. To characterize.

[0103] It should be noted that, in the embodiments provided in this application, it is also necessary to first define an adjacency matrix that defines the information transmission paths between nodes in the spatiotemporal convolution module. ,and .

[0104] It is understood that in the embodiments provided in this application, the fully connected layers in the trained intent recognition model are fully connected graphs. Therefore, the adjacency matrix can initially be a matrix with all elements equal to 1, excluding the diagonal (self-loops), indicating that there are connections between all nodes, i.e.: ; Here, the elements representing edge connections in the initial adjacency matrix are set as learnable parameters. In this way, during subsequent end-to-end training, the values ​​of the learnable parameters are automatically adjusted based on the gradient backpropagation of the training data to optimize the spatial dependency strength between different preset discretized gaze regions.

[0105] Here, the embodiments provided in this application are illustrated using a graphical structure. It can be specifically set as a spacetime graph containing spatial and temporal dimensions. .

[0106] In this application, by defining the graph node and the corresponding node set for each preset discretized gaze region, as well as the edges between each graph node, the trained intent recognition model is able to automatically learn the importance of lane departure intent, thereby improving the recognition accuracy and importance of the trained intent recognition model.

[0107] In one feasible embodiment, the intent recognition model includes a cascaded embedding layer, a spatiotemporal convolutional module, an average pooling layer, a fully connected layer, and an output layer. Based on the gaze region transfer sequence and the trained intent recognition model, the probability of determining whether the target driving object has an intention to actively deviate from its lane is determined includes: inputting the gaze region transfer sequence and the target graph structure into the trained intent recognition model, and extracting initial node feature tensors through the embedding layer; performing spatiotemporally enhanced convolution processing on the initial node feature tensors through the spatiotemporal convolutional module to obtain spatiotemporally enhanced feature tensors; performing global average pooling processing on the spatiotemporally enhanced feature tensors through the average pooling layer to obtain a global feature vector; performing deviation intent analysis processing on the global feature vector through the fully connected layer to obtain an intent score; and converting the intent score into a probability distribution through the output layer to obtain the probability of determining whether the target driving object has an intention to actively deviate from its lane.

[0108] In the embodiments provided in this application, the gaze region transfer sequence can be a gaze sequence. To characterize this, this application caches the gaze region transfer sequence of the target driving object in real time through a first preset time window, and inputs the sequence within the window into a trained intent recognition model used to characterize the spatiotemporal graph convolutional network for recognition at a fixed frequency or event triggering method. In the embodiment provided by this application, the gaze region transfer sequence will pass through the embedding layer, spatiotemporal convolution module, average pooling layer, fully connected layer and output layer of the intent recognition model in sequence, and finally obtain the intent recognition probability that the target driving object has an active lane departure intent.

[0109] It should be noted that the intent recognition model in the embodiments provided in this application is an end-to-end architecture. Its design follows the principle of complete mapping from symbol sequence to classification probability. The core is a hierarchical feature extractor composed of L spatiotemporal convolutional blocks (ST-Conv Blocks).

[0110] Understandably, the input to the intent recognition model is the original gaze sequence. and predefined graph structures This application will first By mapping a trainable embedding layer operation to a continuous and dense vector representation, this vector representation can be effectively learned through semantically rich and computationally efficient feature representations.

[0111] For example, the construction of the embedding layer of the intent recognition model in the embodiments provided in this application includes: Initialize a trainable weight matrix with dimensions equal to the number of nodes and a preset embedding dimension. Each row of the trainable weight matrix corresponds to a learnable feature representation of a graph node. Configure a lookup mapping operation, which uses the discrete gaze region identifier of each time step in the input sequence as the row index to extract the corresponding row vector from the trainable weight matrix and determine the row vector as the feature vector of that time step. Then, based on the feature vector output by the embedding layer, generate the initial node features of all graph nodes in the node set at each time step to obtain the initial node feature tensor.

[0112] In the above, based on the feature vector output by the embedding layer, initial node features of all graph nodes in the node set at each time step are generated, resulting in an initial node feature tensor, including: Obtain the feature vector sequence output by the embedding layer, which corresponds one-to-one with each time step in the input sequence. The shape of the feature vector sequence is the number of time steps plus a preset embedding dimension. For each time step, copy the feature vector corresponding to the current time step into multiple identical copies, equal to the number of graph nodes in the node set. Arrange the multiple identical copies in graph node order to generate the initial node feature matrix for all graph nodes at that time step. Stack the initial node feature matrices of each time step in chronological order to form the initial node feature tensor. The weight matrix of the embedding layer in the embodiments provided in this application is as follows:

[0113] here, It is the embedding dimension; for the gaze sequence Each symbol in Its embedding vector Obtained by looking up the table: ; in Represents the weight matrix of the first element. The row vector of a row.

[0114] In this application, the entire line-of-sight sequence After the embedding layer, the initial node features are obtained. ,here It is the number of time steps. It refers to the number of nodes. Furthermore, in the embodiments provided in this application, during initialization, for the same time step... ,all The features of each node are initialized to the same vector. This is because at the time step All we know is which area the target driver is looking at. However, it does not know the relationship between this region and other regions; this relationship will be learned by the subsequent spatiotemporal convolution module.

[0115] After obtaining the initial node characteristics Next, the initial node features pass Feature extraction is performed on spatiotemporal convolutional blocks to obtain spatiotemporal augmented feature tensors. Then, the output spatiotemporal augmented feature tensors are input into an average pooling layer for global average pooling to obtain a global feature vector. Next, the global feature vector is input into a fully connected layer for lane departure intention analysis to obtain an intention score. Finally, the intention score is input into the output layer for probability distribution transformation to determine the probability of intention recognition that the target driving object has an active lane departure intention.

[0116] The pooling layer is a global average pooling layer. Constructing the pooling layer includes: Configure a global average pooling layer. The global average pooling layer is configured to calculate the average value of all time step features in the time step dimension and the average value of all graph node features in the node dimension. The average values ​​in the time step dimension and the node dimension are combined to generate a global feature vector of fixed dimension.

[0117] Here, the output layer for classification includes a fully connected layer and a normalized exponential function. The output layer for classification is constructed as follows: A fully connected layer is configured to perform a linear transformation on the global feature vector, outputting an original score vector corresponding to the number of intent categories. A normalized exponential function is configured to convert the original score vector into a predicted probability for each intent category, wherein each intent category includes at least the presence of an active lane departure intent and the absence of an active lane departure intent. The predicted probability of the presence of an active lane departure intent output by the normalized exponential function is determined as the intent recognition probability of the target driving object having an active lane departure intent.

[0118] In summary, the embodiments provided in this application initialize the node feature tensor via... After several spatiotemporal convolutional modules, the refined spatiotemporal augmented feature tensor Then, global average pooling is used to compress it into a fixed graph-level global feature vector. : ; In the above, the global feature vector in the embodiments provided in this application Integrating the entire gaze sequence Global spatiotemporal information across all regions.

[0119] Next, The data is then fed into a fully connected layer for final classification. ; in .

[0120] Finally, we use the Softmax function in the output layer to calculate the score. Converting to a probability distribution yields the probability of the target driver having an intention to actively deviate from the lane: in, ; ; The Softmax function is a function that maps a real vector to a probability distribution. In classification tasks, it is often used in the output layer of multi-class classification problems to convert the raw scores (logits) output by the neural network into probabilities.

[0121] In one feasible implementation, the spatiotemporal convolution module includes at least two sequentially cascaded spatiotemporal convolution blocks, each spatiotemporal convolution block including a sequentially cascaded temporal convolution layer and a spatial graph convolution layer; the initial node feature tensor is subjected to spatiotemporally enhanced convolution processing by the spatiotemporal convolution module to obtain a spatiotemporally enhanced feature tensor, including: for each spatiotemporal convolution block, the initial node feature tensor is subjected to feature sequence convolution processing in the temporal dimension by the temporal convolution layer to obtain a temporally enhanced feature tensor; the temporally enhanced feature tensor is subjected to graph convolution processing in the spatial dimension by the spatial graph convolution layer to obtain a spatiotemporally enhanced feature tensor.

[0122] In the embodiments provided in this application, after the initial node feature tensor is output from the embedding layer, it is first input into the temporal convolutional layer of the spatiotemporal convolutional block for temporal feature sequence convolution processing to obtain a temporal augmentation feature tensor. Then, it is processed by spatial graph convolution processing through the spatial graph convolutional layer to obtain a spatiotemporal augmentation feature tensor. .

[0123] For example, for each time step, the node feature matrix of the current time step and the initial adjacency matrix are input together into the spatial graph convolution module; The spatial graph convolution module aggregates the features of each graph node’s neighbor nodes based on the neighbor relationships defined by the initial adjacency matrix to update the feature representation of the graph node; the node feature matrices updated at each time step are then stacked in chronological order to generate a spatiotemporally enhanced feature tensor.

[0124] The goal of the temporal convolutional layer in the spatiotemporal convolutional block is to extract sequence patterns along the time dimension for each graph node. , its initial node feature tensor Treating it as an independent time series, the feature sequence of each graph node in the time dimension is subjected to one-dimensional convolution operation through the temporal convolution module. The dilation rate parameter is introduced in the above one-dimensional convolution operation to expand the temporal receptive field of the convolution kernel in the time dimension by sampling at intervals. Only the feature data of the current time and the historical time are used for calculation in the one-dimensional convolution operation to ensure causality.

[0125] It should be noted that the temporal convolutional layer in the embodiments provided in this application uses dilated causal convolution.

[0126] Understandably, for an input sequence and a convolution kernel In time step The output is: ; in, Indicates the expansion rate Hollow convolution. It is the kernel size.

[0127] Here, causality is determined by access only. Time-bound data guarantee; dilated convolution uses dilation rate The receptive field expands exponentially, enabling deep networks to capture long-range dependencies.

[0128] To control the information flow and stabilize deep training, we employ gated TCN units. The gating mechanism adaptively controls the information flow, better capturing the dynamic evolution of the gaze sequence. Its core formula is:

[0129] in It is element-wise multiplication. It is the Sigmoid function.

[0130] Thus, the temporal convolutional layer in the embodiments provided in this application integrates a gating mechanism, specifically: performing one-dimensional convolution operation on the feature sequence of each graph node in the time dimension through the temporal convolution module, and further including: performing convolution transformation on the input feature sequence through a parallel main convolution path to generate main path features; performing convolution transformation on the input feature sequence through a parallel gated convolution path, and generating gated weights after processing by an activation function; and multiplying the main path features and gated weights element-wise to achieve adaptive adjustment of the temporal information flow.

[0131] The output of the temporal convolutional layer in the spatiotemporal convolutional block It captured the temporal evolution pattern of each node.

[0132] In another embodiment, the output of the spatial convolutional layer in the spatiotemporal convolutional block... Its goal is to learn the spatial dependencies between nodes at each time step using the graph structure.

[0133] In this application, a graph convolutional network represented by a spatial convolutional layer is used to learn the spatial dependencies between graph nodes. Specifically: In target graph structure data, the legal connections between nodes are irregular, making it impossible to directly apply traditional convolution. The core idea of ​​graph convolution is message passing, where each node receives information from its neighboring nodes and aggregates this information to update its own features.

[0134] Specifically: Each node collects features from its neighbors; aggregates these features through weighted sums and other methods; and updates its representation by combining its own features. This operation enables each node to perceive the structure of its local neighborhood. Through multi-layer stacking, nodes can capture a wider range of structural information.

[0135] At each time step The temporal augmentation feature tensor represented by the node feature matrix Graph convolutional network layers are used to aggregate neighbor information and update node features. The formula is as follows: ; in, Used to characterize the normalized adjacency matrix It is an adjacency matrix with added self-loops to ensure that nodes do not lose their own characteristics when aggregating neighbor information.

[0136] yes The degree matrix is ​​used to characterize normalization.

[0137] These are the trainable weights of the spatial graph convolutional layer. It is an activation function.

[0138] In the above, this operation enables each node to aggregate information from its neighboring nodes through a weighted summation, update its own representation, and thus understand the strong spatial and functional association between things such as "left rearview mirror" and "left window".

[0139] Here, the embodiments provided in this application use the spatiotemporal augmented feature tensor output by each spatiotemporal convolutional block as the input to the next spatiotemporal convolutional block, and use the spatiotemporal augmented feature tensor output by the last spatiotemporal convolutional block as the input to the pooling layer, so that the intent recognition model can learn hierarchical and complex spatiotemporal features.

[0140] For example, the spatial graph convolution module aggregates the neighbor node features of each graph node based on the neighbor relationships defined in the initial adjacency matrix to update the feature representation of the graph node. Specifically, this involves: adding self-loop connections to the initial adjacency matrix to generate an enhanced adjacency matrix; calculating the degree matrix based on the enhanced adjacency matrix and performing symmetric normalization on the enhanced adjacency matrix using the degree matrix; multiplying the normalized enhanced adjacency matrix with the node feature matrix at the current time step to obtain the aggregated neighbor node features; performing a linear transformation on the aggregated neighbor node features and a trainable weight matrix, and outputting the updated node feature matrix through a non-linear activation function.

[0141] In one feasible implementation, the trained intent recognition model is determined by the following method: Based on at least one historical line-of-sight area information of a historical driving object output by the driver monitoring system, determine at least one historical line-of-sight area transfer sequence of the historical driving object under a historical preset time window; Based on all historical line-of-sight region transfer sequences and the historical intent labels corresponding to each historical line-of-sight region transfer sequence, a historical training dataset is constructed. The historical intent labels are used to characterize whether the historical driving object had an intention to actively deviate from the lane. The historical training dataset is input into an initial spatiotemporal graph convolutional model to determine the predicted historical intent recognition probability of the historical driving object having an intention to actively deviate from the lane. Based on the historical intent labels and the predicted historical intent recognition probability, the initial spatiotemporal graph convolutional model is iteratively trained to determine the trained intent recognition model.

[0142] In the embodiments provided in this application, a historical training dataset is constructed by determining at least one historical line-of-sight region transfer sequence and a historical intent label corresponding to each historical line-of-sight region transfer sequence based on at least one historical line-of-sight region information of a historical driving object, so as to train the initial spatiotemporal graph convolutional model and determine the trained intent recognition model.

[0143] It should be noted that the specific training process is as follows: based on the historical training dataset and historical intent labels, an initial spatiotemporal graph convolutional model is subjected to end-to-end supervised iterative training. When the model performance meets the preset conditions, the training is stopped, and the trained intent recognition model is obtained.

[0144] It is understood that the embodiments provided in this application can train an initial convolutional model based on real labels and predicted labels, determine the loss value, and determine the target convolutional model after training until the loss value reaches a preset loss threshold.

[0145] In this application, the trained target volume model constructs the gaze region transfer sequence into a graph structure, and automatically learns the spatial dependencies between gaze regions and the temporal dynamic evolution of gaze transfer. This overcomes the shortcomings of traditional temporal models that ignore spatial correlations, and achieves high-precision recognition of complex and atypical intention patterns. Furthermore, the trained target volume model in the embodiments provided in this application can capture richer and more discriminative visual behavioral features, such as gaze region transfer sequences, and obtains better recognition performance.

[0146] In one feasible implementation, the historical intent label includes positive sample labels, and the historical intent label corresponding to the historical gaze region transfer sequence is determined by the following method: If, within a preset historical time window, the historical driving status information of the historical driving object meets the first preset historical driving conditions, then the historical intent label corresponding to the historical line-of-sight area transfer sequence is determined to be a positive sample label. The first preset historical driving conditions include: the historical driving object turns on the vehicle's turn signal; the historical driving object controls the vehicle to complete a lane change operation; the direction of the historical driving object's hand torque is consistent with the direction of the vehicle's lane change; and the duration of the historical driving object's dwell time in the line-of-sight area on the same side as the direction of the lane change is greater than the preset dwell time.

[0147] In the embodiments provided in this application, if the first preset historical driving conditions are met, the historical driving state information of the historical driving object is positive sample information (i.e., its corresponding historical line-of-sight area transfer sequence is marked as 1), and the positive sample information specifically needs to meet the following conditions: the historical driving object turns on the vehicle's turn signal; the historical driving object controls the vehicle to complete the lane change operation; the direction of the historical driving object's hand torque is consistent with the direction of the vehicle's lane change; and the duration of the historical driving object's dwell time in the line-of-sight area on the same side of the lane change direction is greater than the preset dwell time.

[0148] It should be noted that, in the embodiments provided in this application, the positive sample information also specifically needs to satisfy the target driving object's hand torque deviation value being greater than a first deviation threshold.

[0149] It is understood that the positive sample information provided in the embodiments of this application also specifically needs to satisfy that the target driver's physical and mental state is not negative, that is, to determine that the target driver is not in a state of distraction and / or fatigue.

[0150] In the embodiments provided in this application, the lane changing operation is determined by the position of the vehicle relative to the lane lines.

[0151] Here, the first deviation threshold provided in the embodiments of this application can be customized and used according to different application scenarios and usage conditions.

[0152] In this application, historical driving status information that meets all the above-mentioned first preset historical driving conditions is determined as positive sample information.

[0153] In one feasible implementation, the historical intent label includes negative sample labels, and the historical intent label corresponding to the historical gaze region transfer sequence is determined by the following method: If, within a preset historical time window, the historical driving status information of the historical driving object meets the second preset historical driving conditions, then the historical intention label corresponding to the historical line-of-sight area transfer sequence is determined to be a negative sample label. The second preset historical driving conditions include: the historical driving object did not turn on the vehicle's turn signal; the direction of the historical driving object's hand torque is inconsistent with the vehicle's lane change direction; and the dwell time of the historical driving object in the line-of-sight area on the same side as the lane change direction is less than or equal to the preset dwell time.

[0154] In the embodiments provided in this application, if the second preset historical driving conditions are met, the historical driving state information of the historical driving object is negative sample information (i.e., its corresponding historical line-of-sight area transfer sequence is marked as 0), and the positive sample information specifically needs to meet the following conditions: the historical driving object does not turn on the vehicle's turn signal; the direction of the historical driving object's hand torque is inconsistent with the vehicle's lane change direction; and the dwell time of the historical driving object in the line-of-sight area on the same side as the lane change direction is less than or equal to the preset dwell time.

[0155] It is understood that the positive sample information provided in the embodiments of this application also specifically needs to satisfy the target driver's physical and mental state as negative, that is, to determine that the target driver is in a state of distraction and / or fatigue.

[0156] It should be noted that, in the embodiments provided in this application, driving data segments that simultaneously satisfy "vehicle turn signal is on" but "target driver's line of sight is not lingering in the corresponding side mirror or window area" are regarded as fuzzy samples and removed from the historical training dataset.

[0157] In the embodiments provided in this application, the lane changing operation is determined by the position of the vehicle relative to the lane lines.

[0158] In this application, historical driving status information that meets all the above-mentioned second preset historical driving conditions is determined as negative sample information.

[0159] Figure 3 This is a structural block diagram of a lane departure intention recognition device provided in an embodiment of this application. Figure 3 As shown, the lane departure intention recognition device 300 includes: The acquisition module 310 is used to acquire the line-of-sight region transfer sequence of the target driving object inside the vehicle under a first preset time window before the vehicle causes lateral deviation, wherein the first preset time window is a time window of a preset duration before the current moment.

[0160] The first determining module 320 is used to determine the probability of intention recognition that the target driving object has an active lane departure intention based on the line-of-sight transfer sequence and the trained intention recognition model when the line-of-sight transfer sequence does not conform to the preset line-of-sight transfer mode.

[0161] The second determining module 330 is used to determine whether the target driving object has an intention to actively deviate from the lane at the current moment based on the intention recognition probability and a preset probability threshold.

[0162] The third determining module 340 is used to determine that the target driving object has an intention to actively deviate from the lane at the current moment when the line-of-sight area transfer sequence conforms to the preset line-of-sight transfer mode.

[0163] The judgment module 350 is used to determine whether the vehicle has lateral deviation within a second preset time window, wherein the second preset time window is the effective time window corresponding to the active lane departure intention, and the start time of the second preset time window is the current time.

[0164] The verification module 360 ​​is used to obtain the vehicle's status information and the target driver's hand torque information if the condition is met, and to verify the active lane departure intention based on the status information and hand torque information.

[0165] In one feasible embodiment, the second determining module 330 is specifically used for: If the probability of intent recognition is greater than or equal to a preset probability threshold, then it is determined that the target driving object has an intention to actively deviate from the lane at the current moment.

[0166] If the probability of intent recognition is less than a preset probability threshold, it is determined that the target driving object does not have an intention to actively deviate from the lane at the current moment.

[0167] In one feasible embodiment, the preset gaze shift mode includes a scanning observation mode and a continuous gaze mode, and the third determining module 340 is specifically used for: When the gaze region transfer sequence does not conform to the scanning observation mode and does not conform to the continuous gaze mode, it is determined that the gaze region transfer sequence does not conform to the preset gaze transfer mode.

[0168] When the gaze region transfer sequence matches either the scanning observation mode or the continuous gaze mode, it is determined that the gaze region transfer sequence matches the preset gaze transfer mode.

[0169] Among them, the scanning observation mode is used to characterize the observation mode in which the gaze moves from the road area ahead to at least one target monitoring area and then returns to the road area ahead; the continuous gaze mode is used to characterize the gaze mode in which the gaze is continuously focused on at least one target monitoring area for a duration exceeding a preset time.

[0170] In one feasible embodiment, the acquisition module 310 is specifically used for: Based on the vehicle operation information of the target driver inside the vehicle, and assuming that the target driver is not actively controlling the vehicle, the real-time gaze area information of the target driver is obtained. The real-time gaze area information is obtained by mapping the target driver's facial image to multiple preset discretized gaze areas after gaze tracking processing. The preset discretized gaze areas include the road area ahead and the target monitoring area.

[0171] Based on real-time line-of-sight information, the line-of-sight transfer sequence of the target driving object under the first preset time window is determined.

[0172] In one feasible embodiment, the operation information includes turn signal information, brake pedal opening information, accelerator pedal opening information, and the hand torque information of the target driver. If the target driver in the vehicle controls the turn signal information to turn on the turn signal, it is determined that the target driver is actively controlling the vehicle.

[0173] If the brake pedal opening information controlled by the target driver inside the vehicle is greater than the first preset opening value, then it is determined that the target driver is actively controlling the vehicle.

[0174] Alternatively, if the accelerator pedal opening information of the target driver inside the vehicle is greater than the second preset opening value, then it is determined that the target driver is actively controlling the vehicle.

[0175] If the hand torque information of the target driver inside the vehicle is greater than the preset hand torque threshold, then it is determined that the target driver is actively controlling the vehicle.

[0176] In one feasible embodiment, the state information includes the vehicle's lateral offset direction, the hand torque information includes the hand torque direction and the hand torque magnitude, and the verification module 360 ​​is specifically used for: If the lateral deviation direction of the vehicle is consistent with the direction of the hand torque, and the magnitude of the hand torque is greater than or equal to the preset hand torque deviation threshold, then the active lane departure intention of the target driving object at the current moment is determined to be a true active lane departure intention.

[0177] In one feasible embodiment, before determining the probability of a target driver's intention to actively deviate from the lane departure target based on the gaze region transfer sequence and the trained intention recognition model, the lane departure intention recognition device will also construct a graph node corresponding to each preset discretized gaze region to obtain a node set.

[0178] Based on the spatial relationships between multiple preset discrete view regions, the edges between each graph node are determined, resulting in an edge set.

[0179] The target graph structure is generated based on the set of nodes and the set of edges.

[0180] In one feasible embodiment, a graph node corresponding to each preset discretized gaze region is constructed to obtain a node set, including: The preset discretized gaze area is categorized into regions, including the road ahead region, left rearview mirror region, right rearview mirror region, left window region, right window region, interior rearview mirror region, dashboard region, center console region, and interior gaze discretization region.

[0181] Based on the region category of each preset discretized gaze region, construct the graph node corresponding to each preset discretized gaze region.

[0182] The set of graph nodes is defined as the node set.

[0183] In one feasible embodiment, the intent recognition model includes a cascaded embedding layer, a spatiotemporal convolutional module, an average pooling layer, a fully connected layer, and an output layer. The first determination module 320 is specifically used for: The gaze region transfer sequence and target map structure are input into the trained intent recognition model to extract the initial node feature tensor through the embedding layer.

[0184] The initial node feature tensor is subjected to spatiotemporal augmentation convolution processing by the spatiotemporal convolution module to obtain the spatiotemporal augmented feature tensor.

[0185] The global feature vector is obtained by performing global average pooling on the spatiotemporal augmentation feature tensor through an average pooling layer.

[0186] An intent score is obtained by performing deviation analysis on the global feature vector through a fully connected layer.

[0187] The intent score is converted into a probability distribution by the output layer to obtain the intent recognition probability that the target driver has an intention to actively deviate from the lane.

[0188] In one feasible embodiment, the spatiotemporal convolution module includes at least two sequentially cascaded spatiotemporal convolutional blocks, each spatiotemporal convolutional block including a sequentially cascaded temporal convolutional layer and a spatial graph convolutional layer; the initial node feature tensor is subjected to spatiotemporally enhanced convolution processing by the spatiotemporal convolution module to obtain a spatiotemporally enhanced feature tensor, including: For each spatiotemporal convolutional block, the initial node feature tensor is processed by convolution of the feature sequence in the time dimension through a temporal convolutional layer to obtain a temporally enhanced feature tensor.

[0189] The temporal augmentation feature tensor is obtained by performing spatial graph convolution processing on the temporal augmentation feature tensor through spatial graph convolution layers.

[0190] In one feasible embodiment, the first determining module 320 is specifically configured to determine the trained intent recognition model by: Based on at least one historical line-of-sight area information of a historical driving object output by the driver monitoring system, determine at least one historical line-of-sight area transfer sequence of the historical driving object under a historical preset time window.

[0191] A historical training dataset is constructed based on all historical line-of-sight transfer sequences and the historical intent label corresponding to each historical line-of-sight transfer sequence. The historical intent label is used to characterize whether the historical driving object had an intention to actively deviate from the lane.

[0192] The historical training dataset is input into the initial spatiotemporal graph convolutional model to determine the predicted historical intent recognition probability of historical driving objects having active lane departure intent.

[0193] Based on historical intent labels and predicted historical intent recognition probabilities, the initial spatiotemporal graph convolutional model is iteratively trained to determine the trained intent recognition model.

[0194] In one feasible embodiment, the historical intent label includes positive sample labels, and the historical intent label corresponding to the historical gaze region transfer sequence is determined by the following method: If, within the historical preset time window, the historical driving status information of the historical driving object meets the first preset historical driving condition, then the historical intention label corresponding to the historical line-of-sight area transfer sequence is determined to be a positive sample label. The first preset historical driving conditions include: The driver in the previous video turned on the vehicle's turn signal; Furthermore, the vehicle controlled by the historical driver completed the lane change operation; Furthermore, the direction of the hand torque applied by the driver in the past was consistent with the direction of the vehicle's lane change; Furthermore, the duration of the historical driving subject's dwell time in the line of sight area on the same side as the lane change direction is longer than the preset dwell time.

[0195] In one feasible embodiment, the historical intent label includes negative sample labels, and the historical intent label corresponding to the historical gaze region transfer sequence is determined by the following method: If, within a preset historical time window, the historical driving status information of the historical driving object meets the second preset historical driving conditions, then the historical intent label corresponding to the historical line-of-sight area transfer sequence is determined to be a negative sample label.

[0196] The second preset historical driving conditions include:

[0197] The historical driver did not activate the vehicle's turn signal.

[0198] Furthermore, the direction of the hand torque applied by the driver in the past was inconsistent with the direction of the vehicle's lane change.

[0199] Furthermore, the dwell time of the historical driving object in the line of sight area on the same side as the lane change direction is less than or equal to the preset dwell time.

[0200] Compared with the prior art, the lane departure intention recognition device 300 provided in this application obtains the gaze region transfer sequence of the target driver in the vehicle under a first preset time window before the vehicle lateral deviates. If the gaze region transfer sequence does not conform to the preset gaze transfer mode, the application determines the probability of the target driver having an active lane departure intention based on the gaze region transfer sequence and the trained intention recognition model. Then, based on the intention recognition probability and a preset probability threshold, the application determines whether the target driver has an active lane departure intention at the current moment. This application can improve the accuracy of lane departure intention recognition, thereby improving the user experience and the user's acceptance of the driving assistance system.

[0201] Please see Figure 4 , Figure 4 This application provides a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0202] Memory 420 stores machine-readable instructions executable by processor 410. When electronic device 400 is running, processor 410 and memory 420 communicate via bus 430. When the machine-readable instructions are executed by processor 410, they can perform the operations described above. Figures 1 to 2 The steps of the lane departure intention recognition method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0203] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figures 1 to 2 The steps of the lane departure intention recognition method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0204] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0205] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0206] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0207] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0208] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0209] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0210] This application also provides a computer program product including computer software instructions that, when executed on a processing device, cause the processing device to perform a process of lane departure intention recognition.

[0211] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0212] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0213] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0214] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0215] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0216] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0217] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0218] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this specification.

[0219] Obviously, those skilled in the art can make various modifications and variations to this specification without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, this specification is also intended to include such modifications and variations.

Claims

1. A method for lane departure intention recognition, characterized in that, The lane departure intention recognition method includes: Before the vehicle deviates laterally, the sequence of the line of sight area shift of the target driving object inside the vehicle under the first preset time window is obtained, wherein the first preset time window is a time window of a preset duration before the current moment; If the gaze region transfer sequence does not conform to the preset gaze transfer pattern, the probability of the target driving object having an intention to actively deviate from the lane is determined based on the gaze region transfer sequence and the trained intention recognition model. Based on the intent recognition probability and the preset probability threshold, it is determined whether the target driving object has an intention to actively deviate from the lane at the current moment.

2. The lane departure intention recognition method according to claim 1, characterized in that, The step of determining whether the target driver intends to actively deviate from the lane at the current moment based on the intent recognition probability and a preset probability threshold includes: If the intent recognition probability is greater than or equal to the preset probability threshold, then it is determined that the target driving object has the active lane departure intent at the current moment; If the probability of intent recognition is less than the preset probability threshold, then it is determined that the target driving object does not have the intention to actively deviate from the lane at the current moment.

3. The lane departure intention recognition method according to claim 1, characterized in that, The method further includes: If the gaze shift sequence matches the preset gaze shift pattern, it is determined that the target driver has the intention to actively deviate from the lane at the current moment.

4. The lane departure intention recognition method according to claim 3, characterized in that, The preset gaze shifting mode includes a scanning observation mode and a continuous gaze mode, and the method further includes: When the gaze region transfer sequence does not conform to the scanning observation mode and does not conform to the continuous gaze mode, it is determined that the gaze region transfer sequence does not conform to the preset gaze transfer mode. When the gaze region transfer sequence conforms to either the scanning observation mode or the continuous gaze mode, it is determined that the gaze region transfer sequence conforms to the preset gaze transfer mode. The scanning observation mode is used to characterize the observation mode in which the gaze moves from the road area ahead to at least one target monitoring area and then returns to the road area ahead; the continuous gaze mode is used to characterize the gaze mode in which the gaze is continuously focused on at least one of the target monitoring areas for a duration exceeding a preset duration.

5. The lane departure intention recognition method according to claim 1, characterized in that, The step of obtaining the gaze region shift sequence of the target driving object inside the vehicle within a first preset time window includes: Based on the operation information of the target driver inside the vehicle, if it is determined that the target driver is not actively controlling the vehicle, the real-time line-of-sight area information of the target driver is obtained. The real-time gaze area information is obtained by mapping the facial image of the target driver to multiple preset discretized gaze areas after gaze tracking processing. The preset discretized gaze areas include the road area ahead and the target monitoring area. Based on the real-time line-of-sight information, the line-of-sight transfer sequence of the target driving object under the first preset time window is determined.

6. The lane departure intention recognition method according to claim 5, characterized in that, The operation information includes turn signal information, brake pedal opening information, accelerator pedal opening information, and the hand torque information of the target driver. Determining that the target driver is actively controlling the vehicle based on the operation information of the target driver within the vehicle includes: If the target driver inside the vehicle controls the turn signal information to turn on the turn signal, then it is determined that the target driver is actively controlling the vehicle; or If the target driver inside the vehicle controls the brake pedal opening to be greater than a first preset value, then it is determined that the target driver is actively controlling the vehicle; or If the target driver inside the vehicle controls the accelerator pedal opening to be greater than a second preset value, then it is determined that the target driver is actively controlling the vehicle; or If the target driver inside the vehicle controls the hand torque information and the hand torque is greater than a preset hand torque threshold, then it is determined that the target driver is actively controlling the vehicle.

7. The lane departure intention recognition method according to claim 2 or 3, characterized in that, After determining that the target driving object has the active lane departure intention at the current time, the method further includes: Determine whether the vehicle has lateral deviation within a second preset time window, wherein the second preset time window is the effective time window corresponding to the active lane departure intention, and the start time of the second preset time window is the current time; If so, the vehicle's status information and the target driver's hand torque information are obtained, and the active lane departure intention is verified based on the status information and the hand torque information.

8. The lane departure intention recognition method according to claim 7, characterized in that, The status information includes the vehicle's lateral deviation direction, and the hand torque information includes the hand torque direction and magnitude. The verification of the active lane departure intention based on the status information and the hand torque information includes: If the lateral deviation direction of the vehicle is consistent with the direction of the hand torque, and the magnitude of the hand torque is greater than or equal to a preset hand torque deviation threshold, then the active lane departure intention of the target driving object at the current moment is determined to be a genuine active lane departure intention.

9. The lane departure intention recognition method according to claim 1, characterized in that, Before determining the probability of the target driver's intention to actively deviate from lane based on the gaze region transfer sequence and the trained intention recognition model, the method further includes: Construct a graph node corresponding to each of the preset discretized gaze regions to obtain a node set; Based on the spatial relationship between multiple preset discrete line-of-sight regions, the edges between each graph node are determined to obtain an edge set; Based on the set of nodes and the set of edges, a target graph structure is generated.

10. The lane departure intention recognition method according to claim 9, characterized in that, The process of constructing graph nodes corresponding to each of the preset discretized gaze regions yields a node set, including: The region category of the preset discretized gaze area is determined, wherein the region category includes the road ahead area, the left rearview mirror area, the right rearview mirror area, the left window area, the right window area, the interior rearview mirror area, the dashboard area, the center console area, and the interior gaze discrete area. Based on the region category of each preset discretized gaze region, construct graph nodes corresponding to each preset discretized gaze region; The set of graph nodes is defined as the node set.

11. The lane departure intention recognition method according to claim 9, characterized in that, The intent recognition model includes a cascaded embedding layer, a spatiotemporal convolutional module, an average pooling layer, a fully connected layer, and an output layer. The determination of the intent recognition probability that the target driving object has an active lane departure intent based on the gaze region transfer sequence and the trained intent recognition model includes: The gaze region transfer sequence and the target graph structure are input into the trained intent recognition model to extract the initial node feature tensor through the embedding layer; The initial node feature tensor is subjected to spatiotemporal enhanced convolution processing by the spatiotemporal convolution module to obtain the spatiotemporal enhanced feature tensor; The spatiotemporal enhancement feature tensor is subjected to global average pooling through the average pooling layer to obtain a global feature vector; The global feature vector is subjected to deviation intent analysis through the fully connected layer to obtain an intent score. The output layer converts the intent score into a probability distribution to obtain the intent recognition probability that the target driving object has an intention to actively deviate from the lane.

12. The lane departure intention recognition method according to claim 11, characterized in that, The spatiotemporal convolution module includes at least two cascaded spatiotemporal convolution blocks, each of which includes a cascaded temporal convolution layer and a spatial graph convolution layer; the spatiotemporal augmented convolution processing of the initial node feature tensor through the spatiotemporal convolution module to obtain a spatiotemporally augmented feature tensor includes: For each of the spatiotemporal convolutional blocks, the initial node feature tensor is subjected to temporal-dimensional feature sequence convolution processing through the temporal convolutional layer to obtain a temporally enhanced feature tensor; The temporal enhancement feature tensor is obtained by performing spatial graph convolution processing on the spatial dimension of the spatial graph convolution layer.

13. The lane departure intention recognition method according to claim 1, characterized in that, The trained intent recognition model is determined in the following ways: Based on at least one historical line-of-sight area information of a historical driving object output by the driver monitoring system, at least one historical line-of-sight area transfer sequence of the historical driving object under a historical preset time window is determined. Based on all the historical line-of-sight region transfer sequences and the historical intent label corresponding to each historical line-of-sight region transfer sequence, a historical training dataset is constructed, wherein the historical intent label is used to characterize whether the historical driving object has an active lane departure intention; The historical training dataset is input into the initial spatiotemporal graph convolutional model to determine the predicted historical intent recognition probability of the historical driving object having the active lane departure intent; Based on the historical intent labels and the predicted historical intent recognition probability, the initial spatiotemporal graph convolutional model is iteratively trained to determine the trained intent recognition model.

14. The lane departure intention recognition method according to claim 13, characterized in that, The historical intent labels include positive sample labels, and the historical intent labels corresponding to the historical gaze region transfer sequences are determined in the following way: If, within the historical preset time window, the historical driving status information of the historical driving object meets the first preset historical driving condition, then the historical intent label corresponding to the historical gaze area transfer sequence is determined to be a positive sample label. The first preset historical driving conditions include: The historical driver turns on the vehicle's turn signal; Furthermore, the historical driving object controls the vehicle to complete the lane change operation; Furthermore, the direction of the hand torque of the historical driver is consistent with the lane change direction of the vehicle; Furthermore, the duration of the historical driving object's stay in the line of sight area on the same side as the lane change direction is greater than the preset stay duration.

15. The lane departure intention recognition method according to claim 13, characterized in that, The historical intent labels include negative sample labels, and the historical intent labels corresponding to the historical gaze region transfer sequences are determined in the following way: If, within the historical preset time window, the historical driving status information of the historical driving object meets the second preset historical driving condition, then the historical intention label corresponding to the historical gaze area transfer sequence is determined to be a negative sample label. The second preset historical driving conditions include: The driver in question did not turn on the vehicle's turn signal. Furthermore, the direction of the hand torque of the historical driver is inconsistent with the lane change direction of the vehicle; Furthermore, the duration of the historical driving object's stay in the line of sight area on the same side as the lane change direction is less than or equal to the preset stay duration.

16. A lane departure intention recognition device, characterized in that, The lane departure intention recognition device includes: The acquisition module is used to acquire the line-of-sight region transfer sequence of the target driving object inside the vehicle under a first preset time window before the vehicle deviates laterally, wherein the first preset time window is a time window of a preset duration before the current moment; The first determining module is used to determine the probability of intention recognition that the target driving object has an active lane departure intention based on the gaze region transfer sequence and the trained intention recognition model when the gaze region transfer sequence does not conform to the preset gaze transfer mode. The second determining module is used to determine, based on the intent recognition probability and a preset probability threshold, whether the target driving object has an intention to actively deviate from the lane at the current moment.

17. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the lane departure intention recognition method as described in any one of claims 1-15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the lane departure intention recognition method as described in any one of claims 1-15.

19. A vehicle, characterized in that, The vehicle is equipped with a lane departure intention recognition device as described in claim 16.

20. A computer program product, characterized in that, The method includes a computer program or computer-executable instructions, characterized in that, when the computer program or computer-executable instructions are executed by a processor, they implement the steps of the lane departure intention recognition method as described in any one of claims 1 to 15.

Images