A personalized lane departure warning method and device, and a storage medium

Abstract

The present application relates to the field of advanced auxiliary driving technology, especially to a new personalized lane departure warning method, device and storage medium. The method first acquires real-time data of vehicle operation and determines the point of departure trend occurrence; secondly, vehicle characteristic data and driver characteristic data are acquired, and the Non-stationary Crossformer model is used to predict the lateral deviation trajectory; thirdly, the correction behavior discrimination result and the deviation area are acquired; finally, when the correction behavior discrimination result meets the deviation area judgment condition, it is judged whether the deviation area is greater than the pre-set deviation area threshold, if yes, the warning is executed, otherwise the warning is not executed; the deviation area is surrounded by the horizontal line where the point of departure trend occurrence is located and the lateral deviation trajectory. Compared with the prior art, the present application has the advantages of effectively improving the prediction accuracy of vehicle deviation trajectory and the individualization degree of vehicle deviation threshold setting, thereby reducing the false alarm rate of LDW system.

Description

Technical Field

[0001] This invention relates to the field of advanced driver assistance technology, and in particular to a personalized lane departure warning method, device, and storage medium. Background Technology

[0002] False alarms occur when a warning is triggered when it is not necessary, which can lead to driver frustration and reduce their trust in the LDW (Lane Departure Warning). The key to reducing false alarm rates lies in identifying driver corrective behaviors and setting personalized thresholds. Driver corrective behaviors are defined as the driver returning the vehicle to the lane line within a certain period after a deviation trend occurs. However, it is not enough to only consider the occurrence of corrective behaviors; it is also necessary to determine whether the deviation has a significant impact on the driver during this period. Therefore, when the driver successfully performs a corrective behavior and the deviation does not have a significant impact on the driver, both conditions must be met to trigger a warning. All of this is based on the prediction of future lateral trajectories. Due to the differences in characteristics among drivers, the same deviation will have different effects on different drivers. Therefore, the reasons for the false alarm rate are: (1) the driver corrective behavior identification is not accurate enough, that is, the trajectory prediction accuracy is not high enough; (2) no personalized thresholds are set.

[0003] Existing technologies have explored solutions to address these two reasons for the high false alarm rate, but certain problems remain. First, most current LDW algorithms do not consider the interrelationships between variables or special handling for large trajectory changes when predicting the trajectory, and most determine whether to issue a warning by finding the point of maximum trajectory deviation. This approach cannot consider the complete deviation process and cannot measure the risks caused throughout the process, thus limiting its ability to reduce the false alarm rate. Second, regarding the formulation of personalized thresholds, most solutions do not specify personalized thresholds for individual drivers, but only use a uniform maximum deviation displacement threshold or a category-based threshold. Some drivers do not accept this uniform threshold, which still leads to an increased false alarm rate. Therefore, how to further improve the prediction accuracy of vehicle deviation trajectory and the personalization of vehicle deviation threshold settings has become a problem that needs to be solved in this field. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art, such as insufficient accuracy in predicting vehicle deviation trajectory and low degree of personalization in setting vehicle deviation threshold, and to provide a personalized lane departure warning method, device and storage medium.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] According to a first aspect of the present invention, a personalized lane departure warning method is provided, comprising the following steps:

[0007] S1, acquire real-time vehicle operation data and determine the point where the deviation trend occurs;

[0008] S2, based on the deviation trend occurrence point, obtain vehicle feature data and driver feature data, and use the Non-stationary Crossformer model to predict the lateral deviation trajectory;

[0009] S3, based on the lateral deviation trajectory, obtain the correction behavior discrimination result and the deviation area;

[0010] S4. When the correction behavior judgment result meets the deviation area judgment condition, determine whether the deviation area is greater than the deviation area threshold. If yes, then execute the warning; if no, then do not execute the warning.

[0011] The deviation area is defined by the horizontal line where the deviation trend occurs and the lateral deviation trajectory, and the deviation area threshold is preset.

[0012] As a preferred technical solution, the key to constructing the Non-stationary Crossformer model lies in introducing the Non-stationary structure into the Crossformer model to achieve the fusion of the two-stage attention layer, the hierarchical encoding-decoding structure, and non-stationary information.

[0013] As a preferred technical solution, the process of presetting the deviation area threshold includes obtaining relevant historical deviation area data and changes in driver handling stability before and after deviation for different drivers, and calculating and setting the optimal deviation area threshold using kernel density estimation and classification information entropy.

[0014] As a preferred technical solution, the changes in the driver's control stability before and after the deviation are obtained using the K-means clustering method.

[0015] As a preferred technical solution, the vehicle characteristic data includes relative yaw angle, speed, lateral position and yaw rate, and the driver characteristic data includes steering wheel angle.

[0016] As a preferred technical solution, the process of determining the deviation trend occurrence point includes:

[0017] Based on the real-time vehicle operation data, obtain the current TLC value;

[0018] The current TLC value is compared with the time threshold. If the TLC value is less than the time threshold, it is determined that there is a deviation trend and the point where the deviation trend occurs is determined. Otherwise, it is determined that there is no deviation trend and no warning is issued. The time threshold is preset.

[0019] As a preferred technical solution, the real-time vehicle operation data used in the process of obtaining the current TLC value includes the radius of curvature of the vehicle path, the center angle of the trajectory, the yaw rate, the distance between the front wheel on the side deviating from the lane line and the lane line, and the relative yaw angle.

[0020] As a preferred technical solution, the meaning of meeting the deviation area judgment condition includes that, after a pre-set time interval, the judgment result of the corrective behavior is that the vehicle has returned to the lane line.

[0021] According to a second aspect of the present invention, a personalized lane departure warning device is provided, comprising a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement the method described herein.

[0022] According to a third aspect of the present invention, a storage medium is provided having a program stored thereon, which, when executed, implements the method described thereon.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. This invention uses a non-stationary Crossformer combined model to predict lateral deviation trajectories and uses the Crossformer model for correction behavior discrimination. It can simultaneously capture cross-time and cross-dimensional dependencies, effectively utilize the dependencies between input variable data, and capture information at different scales using a hierarchical structure. This allows key variables that change in deviation trajectory prediction, such as speed and relative yaw angle, which are variables at different time scales, to be captured, predicted, and reflect dependencies at different levels. Since deviation trajectories often change significantly during correction behavior, the introduction of the non-stationary structure can fully retain the non-stationary information in the deviation trajectory sequence, more accurately predict trajectories when changes are large, effectively improve the accuracy of vehicle deviation trajectory prediction, and thus reduce the false alarm rate of the LDW system.

[0025] 2. This invention introduces the concept of deviation area and its judgment process, which can take into account both lateral deviation displacement and vehicle movement position. It combines the determination of the deviation trend occurrence point, the prediction of lateral deviation trajectory and the judgment results of corrective behavior to realize the multiple judgment mechanism of the LDW system. It takes into account both the immediacy of vehicle deviation warning and the complete deviation process, and measures the risks caused in the entire deviation process, thereby further improving the accuracy of vehicle deviation trajectory prediction.

[0026] 3. This invention acquires historical deviation area data and changes in driver handling stability before and after deviation for different drivers. It uses kernel density estimation and classification information entropy to calculate and set personalized deviation area thresholds, distinguishing the critical risks that different drivers can tolerate, rather than based on a uniform threshold. This allows the system to determine whether a driver can successfully return to the original lane if the deviation is less than the corresponding threshold, thus deciding whether to issue a warning. This approach can more accurately and meticulously capture the characteristic differences between drivers, effectively improving the personalization of vehicle deviation threshold settings, increasing the likelihood that different drivers will accept the warning results, thereby reducing the false alarm rate of the LDW system and increasing its utilization rate. Attached Figure Description

[0027] Figure 1 This is a schematic flowchart of the method of the present invention;

[0028] Figure 2 This is a schematic diagram illustrating the implementation process of the method in this embodiment of the invention;

[0029] Figure 3 This is a schematic diagram illustrating the determination of vehicle deviation trend in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the Non-stationary Crossformer model structure in an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram illustrating the offset area in an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of the segments before and after deviation and their categories in an embodiment of the present invention. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0034] Example

[0035] like Figure 1 As shown, this embodiment provides a personalized lane departure warning method, which includes the following steps:

[0036] Step S1: Obtain real-time vehicle operation data and determine the point where the deviation trend occurs;

[0037] Step S2: Based on the deviation trend occurrence point, obtain vehicle feature data and driver feature data, and use the Non-stationary Crossformer model to predict the lateral deviation trajectory;

[0038] Step S3: Based on the lateral deviation trajectory, obtain the correction behavior discrimination result and the deviation area;

[0039] Step S4: When the result of the corrective behavior judgment meets the deviation area judgment condition, determine whether the deviation area is greater than the deviation area threshold. If yes, then execute the warning; if no, then do not execute the warning.

[0040] The deviation area is composed of the horizontal line where the deviation trend occurs and the horizontal deviation trajectory, and the deviation area threshold is preset.

[0041] like Figure 2 As shown, the specific implementation process of the above steps is as follows:

[0042] In step S1, after acquiring real-time vehicle operation data, the vehicle deviation trend is judged based on the real-time vehicle operation data. At the same time as obtaining the judgment result, the location of the deviation trend occurrence point can also be determined, instead of starting the algorithm only when the vehicle actually deviates from the lane line.

[0043] Determining whether a vehicle exhibits a deviation trend is a prerequisite for personalized early warning. Real-time vehicle operation data includes the radius of curvature of the vehicle path, center angle, yaw rate, distance between the front wheel and the lane line on the side deviating from the lane line, relative yaw angle, and steering wheel angle, etc. The aforementioned determination of vehicle deviation trend and location of the deviation trend occurrence point can be achieved using the TCL method: after calculating the current TLC value using a portion of the acquired real-time vehicle operation data, the current TLC value is compared with a time threshold. If the TLC value is less than the preset time threshold, a deviation trend is determined, and the deviation trend occurrence point is identified; otherwise, no deviation trend is determined, and no warning is issued.

[0044] This embodiment utilizes the Time-Limited Chance (TLC) method, which estimates the time it takes for the vehicle to approach the lane line and compares this estimate with the time required for the driver to successfully correct the lane. If the former is less than the latter, the vehicle responds immediately, allowing the driver to receive an early warning and initiate steering maneuvers, thus corresponding to the determination of a deviation trend. Simultaneously, the change in the relative yaw angle β is considered. The time required for the driver to successfully correct the lane is a pre-set time threshold.

[0045] like Figure 3As shown, the road is mainly straight. The scenario is set up as a vehicle deviating from its trajectory on a straight road using a curved path. Both T1 and T2 in the diagram can be used to simulate the vehicle's deviation, but T2 is more realistic than T1. Therefore, T2 is selected as the deviation trajectory. The derivation formula is as follows:

[0046] Based on geometric relationships:

[0047]

[0048]

[0049] In equation (2), α is the central angle of the trajectory; R is the radius of curvature of the vehicle path, which can be obtained by the following formula:

[0050]

[0051] In equation (3), ω is the yaw rate, which can be considered to remain constant over a short period of time, and α can be obtained from the calculation of the cosine function:

[0052]

[0053]

[0054] Given AC=R, substituting equation (5) into equation (4), we get:

[0055]

[0056] and The value of is negative, and its magnitude represents BE, therefore:

[0057]

[0058] In ΔACE, AE is the foot of the perpendicular from A to the lane line, according to geometric relationships:

[0059]

[0060] then:

[0061]

[0062] OD is the distance between the front wheel (leaning towards the lane line) and the lane line, and β is the relative yaw angle. The obtained TLC is compared with a time threshold, which can be flexibly adjusted. In this embodiment, 1 second is selected; if it is less than 1 second, there is a deviation trend.

[0063] In step S2, after determining the point where the deviation trend occurs, vehicle characteristic data and driver characteristic data are acquired, and a Non-stationary Crossformer model is used to predict the lateral deviation trajectory in the future time period. The vehicle characteristic data refers to vehicle state variables, including relative yaw angle, speed, lateral position, and relative yaw rate, while the driver characteristic data refers to driver intention variables, including steering wheel angle, etc.

[0064] This embodiment selects variables such as relative yaw angle, speed, lateral position, steering wheel angle, relative yaw rate, driver intention, and vehicle state to predict deviation from the trajectory. The Non-stationary Crossformer model is a multivariate time series prediction model based on an improvement on the Transformer. The dimensional segmentation embedding structure in the Crossformer model changes the conventional embedding mode, enhancing the dependencies between dimensions, and thus proposes a two-stage attention layer and a hierarchical encoder-decoder structure to capture information from different time scales (such as speed and lateral position) in the trajectory variables. Based on the Crossformer model, a Non-stationary structure is introduced, and a de-stationary process is added to the attention layer to better preserve the trend components of the sequence. Therefore, the Non-stationary Crossformer can combine the advantages of both Non-stationary and Crossformer models, better improving the prediction accuracy when deviations from the trajectory vary significantly. The algorithm framework is as follows: Figure 4 As shown, where x is the input sequence, The input sequence is standardized. , For the mean and variance, the non-stationary factors τ and Δ are obtained through multilayer perceptron mapping; in the encoder, , , Represents the standardized query, key, and value in the Transformer. and These represent the outputs of the attention layers across time and across stages, respectively, where n represents the size of the dimension. This is the predicted output when standardized, and This is the predicted output after destandardization.

[0065] When implementing step S3, it is necessary to obtain the correction behavior judgment result and the deviation area based on the lateral deviation trajectory predicted in step S2.

[0066] After the lateral trajectory prediction in step S2, the following is obtained: Figure 5 The deviation trajectory is shown. Figure 5In the deviation process shown, point 1 is the point where the deviation trend occurs, {Δ } represents the distance from the lateral trajectory to the lane line within the time interval T, negative when inside the lane line and positive when outside the lane line; point 2 represents the maximum deviation point max{Δ} in the deviation event. Point 3 represents the trajectory point after time interval T (point 3 may coincide with point 2). Based on point 3, determine whether the driver's corrective action was successful.

[0067]

[0068] This represents the distance of point 3 from the lane line after the time interval T. Equation (10) is used to determine whether the vehicle is within the lane line after the time interval T. If the vehicle has returned to the lane line, the corrective action is considered successful, meeting the deviation area judgment condition, and step S4 can continue; if the vehicle is still outside the lane line, the corrective action is considered unsuccessful, not meeting the deviation area judgment condition, and a warning is immediately issued. For example, the time interval T can be set to 1 second, and it can be determined whether the vehicle will return to the lane line after 1 second. If it does, it is considered that the driver has performed a corrective action.

[0069] This embodiment introduces a new concept of deviation area S. For example... Figure 5 As shown, the deviation area S is bounded by the predicted lateral deviation trajectory within the time interval T, starting from the point where the deviation trend occurred, and the horizontal line where the deviation trend occurred. Therefore, the deviation area S is calculated as follows:

[0070]

[0071] in, Let Y represent the distance from the trajectory point to the lane line at time t. The value is negative if the distance is inside the lane line and positive if it's outside. Y represents the lateral position of point 1 on the horizontal line. D is the lateral distance of the trajectory point relative to this horizontal line within the time interval T. The deviation area S is the integral of the relative lateral distance over time within the time interval T. The area of ​​the integral inside the horizontal line is negative, and the area outside is positive. It represents the overall degree of deviation in the deviation process. For example, we can take the time interval T as 1 second, and use the horizontal line where the deviation trend occurred and the lateral trajectory within the next 1 second to form the deviation area. This deviation area can be used to measure the risk caused by this deviation.

[0072] When implementing step S4, since the corrective action was successful in step S3, meaning the deviation area judgment condition is met, the current step can continue. The deviation area S obtained in step S3 is used to measure the risk caused by the deviation event. Specifically, it is determined whether the deviation area S is greater than the deviation area threshold. If yes, a warning is issued; otherwise, no warning is issued. In other words, if the deviation area S is less than the driver's corresponding deviation area threshold Q, and the driver successfully performed the corrective action, no warning is triggered; otherwise, if either of the two conditions is not met, a warning is triggered.

[0073] Because different drivers have different risk tolerance levels, it is necessary to consider the characteristics of each driver; that is, each driver has a specific deviation area threshold Q. Drivers with strong maneuvering skills can maintain good driving performance even when deviating from a large deviation area S, while some drivers may make mistakes and lead to greater risks under the same conditions. Therefore, for different drivers, it is necessary to statistically analyze the impact of their historical deviation areas, i.e., to obtain each driver's historical deviation area and the driver's reaction caused by the current deviation, and to determine the deviation area threshold using kernel density estimation combined with information entropy. Taking fixed-length driving segments before and after each deviation, the standard deviations of lateral position, yaw angle, and speed are extracted for each segment, and K-means clustering is performed to divide them into four categories. The larger these three indicators are, the greater the fluctuation and instability in the driver's operation. Lateral position and yaw angle reflect the driver's lateral operation, while speed reflects the driver's longitudinal operation. Figure 6 As shown, A1 represents the category before deviation, and A2 represents the category after deviation. In the clustering, category 4 represents the driver with the best operational stability, and category 1 represents the driver with the worst operational stability. Based on the change in driver operational stability before and after deviation, the deviation area of ​​the driver's history is labeled:

[0074]

[0075] Among them, the label "+" indicates that the deviation area has a significant impact on the driver or that the driver was already in an unstable state before the deviation, while the label "" indicates that the deviation area has a significant impact on the driver or that the driver was already in an unstable state before the deviation. "This indicates that the driver can still tolerate the deviation area." This embodiment defines an optimal threshold for a driver if, under ideal conditions, the size of a deviation area has a significant impact on the driver. Therefore, kernel density estimation and classification information entropy are combined to calculate the optimal personalized threshold for each driver. Specifically, after labeling the deviation area based on the change in stability before and after the deviation, kernel density estimation is used to determine the cut-off point (candidate threshold) for the overall sample. The optimal threshold is then determined using classification information entropy, which is the optimal personalized threshold for the corresponding driver. Finally, the predicted deviation area for the current deviation event is compared with this optimal personalized threshold to determine whether a warning is needed.

[0076] Furthermore, this embodiment also provides a personalized lane departure warning device, including a memory, a processor, and a program stored in the memory, which, when executed by the processor, implements the aforementioned method. The device processor includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from the memory unit into a random access memory (RAM). The RAM can also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. An input / output (I / O) interface is also connected to the bus. Multiple components in the device are connected to the I / O interface, including: input units, such as a keyboard, mouse, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical discs, etc.; and communication units, such as network interface cards, modems, wireless transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks. The processing unit executes the various methods and processes described above, such as steps S1 to S4 in the aforementioned embodiments. For example, in some embodiments, steps S1 to S4 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps S1 to S4 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S4 by any other suitable means (e.g., by means of firmware). The functions described above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0077] Furthermore, this embodiment also provides a storage medium on which a program is stored, which, when executed, implements the aforementioned method. The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine as a standalone software package, or entirely on a remote machine or server. In the context of this invention, a computer-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0078] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A personalized lane departure warning method, characterized in that, Includes the following steps: S1, acquire real-time vehicle operation data and determine the point where the deviation trend occurs; S2, based on the deviation trend occurrence point, obtain vehicle feature data and driver feature data, and use the Non-stationary Crossformer model to predict the lateral deviation trajectory; S3, based on the lateral deviation trajectory, obtain the correction behavior discrimination result and the deviation area; S4. When the correction behavior judgment result meets the deviation area judgment condition, determine whether the deviation area is greater than the deviation area threshold. If yes, then execute the warning; if no, then do not execute the warning. The deviation area is enclosed by the horizontal line where the deviation trend occurs and the lateral deviation trajectory, and the deviation area threshold is preset. The key to constructing the Non-stationary Crossformer model lies in introducing the Non-stationary structure into the Crossformer model to achieve the fusion of the two-stage attention layer, the hierarchical encoding-decoding structure, and non-stationary information. The process of presetting the deviation area threshold includes obtaining relevant historical deviation area data and changes in driver handling stability before and after deviation for different drivers, and calculating and setting the optimal deviation area threshold using kernel density estimation and classification information entropy. The changes in driver stability before and after deviation were obtained using the K-means clustering method. The process of determining the point where the deviation trend occurs includes: Based on the real-time vehicle operation data, obtain the current TLC value; The current TLC value is compared with the time threshold. If the TLC value is less than the time threshold, it is determined that there is a deviation trend and the point where the deviation trend occurs is determined. Otherwise, it is determined that there is no deviation trend and no warning is issued. The time threshold is preset.

2. The personalized lane departure warning method according to claim 1, characterized in that, The vehicle characteristic data includes relative yaw angle, speed, lateral position, and yaw rate, and the driver characteristic data includes steering wheel angle.

3. The personalized lane departure warning method according to claim 1, characterized in that, The real-time vehicle data used in obtaining the current TLC value includes the radius of curvature of the vehicle path, the center angle of the trajectory, the yaw rate, the distance between the front wheel on the side deviating from the lane line and the lane line, and the relative yaw angle.

4. The personalized lane departure warning method according to claim 1, characterized in that, The meaning of meeting the deviation area judgment condition includes that, after a pre-set time interval, the judgment result of the corrective behavior is that the vehicle has returned to the lane line.

5. A personalized lane departure warning device, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-4.

6. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-4.

Images