Trajectory deviation early warning method, terminal and storage medium

Abstract

The embodiment of the application discloses a trajectory deviation early warning method, a terminal and a storage medium. The trajectory deviation early warning method is applied to the terminal. The terminal is installed in a target vehicle. When the target vehicle travels, the terminal acquires real-time turning angle information and real-time image information in front of a road of the target vehicle. The trajectory deviation early warning method performs driving trajectory prediction processing based on a wavelet analysis method and the real-time turning angle information, and obtains prediction trajectory information. The trajectory deviation early warning method performs lane line detection processing based on a target detection model and the real-time image information in front of the road, and obtains lane line information. If it is determined that there is a deviation behavior according to the prediction trajectory information and the lane line information, early warning processing is performed.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a trajectory deviation warning method, a terminal, and a storage medium. Background Technology

[0002] Unintentional lane departures are a major cause of traffic accidents while driving on highways. These unintentional departures may be due to driver inattention, fatigue, or other reasons. Providing warnings for such unintentional departures to prevent potential traffic accidents is one of the functions of driver assistance systems.

[0003] Current lane departure warning technology cannot predict the vehicle's position over a future period, resulting in poor foresight and low accuracy in lane line detection, thus affecting the effectiveness of lane departure warning. Summary of the Invention

[0004] This application provides a trajectory deviation early warning method, a terminal, and a storage medium, which can effectively improve the deviation early warning effect.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a trajectory deviation warning method, wherein the trajectory deviation warning method is applied to a terminal, the terminal being installed in a target vehicle, and the method includes:

[0007] When the target vehicle is driving, acquire the real-time turning angle information of the target vehicle and the real-time image information of the road ahead;

[0008] Based on wavelet analysis and the real-time turning angle information, driving trajectory prediction processing is performed to obtain predicted trajectory information;

[0009] Lane line detection processing is performed based on the target detection model and the real-time image information of the road ahead to obtain lane line information;

[0010] If a deviation is determined based on the predicted trajectory information and the lane line information, an early warning is issued.

[0011] Secondly, embodiments of this application provide a terminal installed in a target vehicle. The terminal includes an acquisition unit, a prediction unit, a detection unit, and a warning unit.

[0012] The acquisition unit is used to acquire the real-time turning angle information and real-time image information of the road ahead of the target vehicle when the target vehicle is driving.

[0013] The prediction unit is used to perform driving trajectory prediction processing based on wavelet analysis method and the real-time turning angle information to obtain predicted trajectory information.

[0014] The detection unit is used to perform lane line detection processing based on the target detection model and the real-time image information of the road ahead to obtain lane line information.

[0015] The warning unit is used to issue a warning if it is determined that there is a deviation behavior based on the predicted trajectory information and the lane line information.

[0016] Thirdly, this application provides a terminal, which further includes a processor and a memory storing processor-executable instructions. When the instructions are executed by the processor, the trajectory deviation warning method described above is implemented.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program applied in a terminal, wherein when the program is executed by a processor, it implements the trajectory deviation warning method described above.

[0018] This application provides a trajectory deviation warning method, a terminal, and a storage medium. The terminal is installed in a target vehicle. When the target vehicle is driving, it acquires real-time turning angle information of the target vehicle and real-time image information of the road ahead. Based on wavelet analysis and real-time turning angle information, it performs trajectory prediction processing to obtain predicted trajectory information. Based on a target detection model and real-time image information of the road ahead, it performs lane line detection processing to obtain lane line information. If deviation behavior is determined based on the predicted trajectory information and lane line information, a warning is issued. Therefore, in this application, the terminal is installed in the target vehicle. During the vehicle's movement, the terminal can perform trajectory prediction processing on real-time turning angle information using wavelet analysis to obtain predicted trajectory information for a future period, thus enabling the estimation of the vehicle's position over a future period, providing excellent foresight. Simultaneously, using a target detection model to perform lane line detection processing on real-time image information of the road ahead improves the accuracy of lane line detection. Thus, based on the predicted trajectory information and lane line information, it can predict whether the target vehicle will deviate, and issue a warning when deviation behavior is detected, effectively improving the deviation warning effect. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the implementation process of the trajectory deviation early warning method proposed in the embodiments of this application. Figure 1 ;

[0020] Figure 2 This is a schematic diagram of the terminal structure proposed in the embodiments of this application. Figure 1 ;

[0021] Figure 3 This is a schematic diagram of the predicted trajectory information proposed in the embodiments of this application. Figure 1 ;

[0022] Figure 4 This is a schematic diagram of the predicted trajectory information proposed in the embodiments of this application. Figure 2 ;

[0023] Figure 5 This is a schematic diagram of lane line information proposed in an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of the implementation process of the trajectory deviation early warning method proposed in the embodiments of this application. Figure 2 ;

[0025] Figure 7 This is a schematic diagram illustrating the implementation of wavelet decomposition processing proposed in the embodiments of this application. Figure 1 ;

[0026] Figure 8 This is a schematic diagram illustrating the implementation of wavelet decomposition processing proposed in the embodiments of this application. Figure 2 ;

[0027] Figure 9 This is a schematic diagram illustrating the implementation of wavelet decomposition processing proposed in the embodiments of this application. Figure 3 ;

[0028] Figure 10 This is a schematic diagram of the target signal proposed in the embodiments of this application;

[0029] Figure 11 This is a schematic diagram of the implementation process of the trajectory deviation early warning method proposed in the embodiments of this application. Figure 3 ;

[0030] Figure 12 This is a schematic diagram of the predicted trajectory information proposed in the embodiments of this application. Figure 3 ;

[0031] Figure 13 This is a schematic diagram of the implementation process of the trajectory deviation early warning method proposed in the embodiments of this application. Figure 4 ;

[0032] Figure 14 This is a schematic diagram of the terminal structure proposed in the embodiments of this application. Figure 2 ;

[0033] Figure 15 This is a schematic diagram of the terminal structure proposed in the embodiments of this application. Figure 3 . Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the relevant application and not for limiting the application. Furthermore, it should be noted that, for ease of description, only the parts related to the relevant application are shown in the accompanying drawings.

[0035] Unintentional lane departures are a major cause of traffic accidents while driving on highways. These unintentional departures may be due to driver inattention, fatigue, or other reasons. Providing warnings for such unintentional departures to prevent potential traffic accidents is one of the functions of driver assistance systems.

[0036] Common lane departure warning methods typically require additional hardware installed on the vehicle or road to assist lane detection, such as radar or infrared devices. These are not only expensive and resource-intensive to implement, but also difficult to implement and have limited applicability. Furthermore, because they cannot accurately distinguish between intentional lane changes and unintentional lane departures, warnings may be issued even during intentional lane changes, causing driver interference. Additionally, current lane departure warning technology cannot predict the vehicle's position over a future period, resulting in poor foresight.

[0037] Meanwhile, existing methods for lane line detection are only applicable to straight lane lines. However, in real life, lane lines on roads are diverse, including straight lines and curves, solid lines and dashed lines, and white and yellow lines. Therefore, traditional morphological and image processing algorithms have difficulty accurately detecting all of these different types of lane lines. Weather, lighting, occlusion and other factors also have a significant impact on the detection results.

[0038] Furthermore, for existing object detection networks, the training process typically involves Hungarian matching to calculate the error between the parameters of each paired detected target and the real target. The principle behind this can be expressed by the following formula:

[0039]

[0040] Where d() is the infinitesimal symbol; ∑ is the accumulation symbol.

[0041] In lane detection, each lane line can be curve-fitted using several parameters, including: cubic curve equation parameters (k1, k2, k3, b), camera intrinsic and extrinsic parameters (f, R), and vertical start and end values ​​(α, β); then d(l) in the above equation... i ,gt z(i)The error calculation components can be composed of these parameters between the detected target and the real target, and can be expressed by the following formula:

[0042]

[0043] The above represents the general idea behind constructing error models in existing object detection methods. However, considering the specific methods and data characteristics of lane line annotation, it has a significant drawback: lane line annotation methods use software like LabelMe to mark points on the image. After annotation, the resulting data is a series of discrete points on each lane line, with each point represented by (x, y, n), where x represents the x-coordinate, y represents the y-coordinate, and n indicates which lane line the point belongs to. Therefore, the parameters required in the above formula, including the parameters of the cubic curve equation, need to be calculated by fitting discrete point coordinate data. Real-world lane lines do not necessarily perfectly conform to cubic curve characteristics, and there will be some error during the fitting process. Image distortion and projection errors during the shooting process also affect the accuracy of the fitting process. Fitting parameters from scattered points, participating in error calculation, and then converting the parameters back to scattered points—the cumulative errors from these two conversions will affect the accuracy of the error model. In other words, the existing error model is not precise in calculating errors during training, thus affecting the accuracy of object detection and making it unable to detect lane lines effectively.

[0044] It is evident that existing deviation warning methods generally suffer from low accuracy in detecting vehicle deviations and poor warning effectiveness.

[0045] To address the problems existing in current trajectory deviation warning methods, this application provides a trajectory deviation warning method, including a terminal and a storage medium. The terminal is installed in the target vehicle. When the target vehicle is driving, the terminal acquires the target vehicle's real-time turning angle information and real-time image information of the road ahead. Based on wavelet analysis and the real-time turning angle information, the terminal performs trajectory prediction processing to obtain predicted trajectory information. Based on a target detection model and the real-time image information of the road ahead, the terminal performs lane line detection processing to obtain lane line information. If a deviation is determined based on the predicted trajectory information and lane line information, a warning is issued. This method has good foresight and can also improve the lane line detection accuracy, ultimately effectively improving the deviation warning effect.

[0046] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0047] This application provides a trajectory deviation warning method, which is applied to a terminal installed in a target vehicle. Figure 1This is a schematic diagram of the implementation process of the trajectory deviation early warning method proposed in the embodiments of this application. Figure 1 ,like Figure 1 As shown, the trajectory deviation warning method may include the following steps:

[0048] Step 101: When the target vehicle is driving, acquire the real-time turning angle information of the target vehicle and the real-time image information of the road ahead.

[0049] In the embodiments of this application, when the target vehicle is driving, the terminal can obtain the target vehicle's real-time turning angle information and real-time image information of the road ahead.

[0050] It should be noted that, in the embodiments of this application, real-time steering angle information can determine the steering angle of the target vehicle, thereby determining the forward trajectory of the target vehicle; for example, real-time steering angle information can be obtained based on the real-time steering angle information of the steering wheel of the target vehicle.

[0051] It should be noted that, in the embodiments of this application, during the driving of the target vehicle, the instantaneous steering angle of the steering wheel can determine the deflection angle of the front wheels of the target vehicle, thus the short-term forward trajectory can be determined by the steering wheel angle; for example, when the steering wheel angle is 0, the deflection angle of the front wheels is 0, and the driving trajectory of the target vehicle is two straight lines forward; when the steering wheel angle is not 0, the target vehicle turns, and the trajectory of the front wheels is a pair of concentric arcs, the radius and length of which are determined by the wheel deflection angle and body parameters of the target vehicle; therefore, by obtaining the steering wheel angle of the target vehicle, relevant information about the driving trajectory of the target vehicle can be obtained.

[0052] For example, in the embodiments of this application, Figure 2 This is a schematic diagram of the terminal structure proposed in the embodiments of this application. Figure 1 ,like Figure 2 As shown, the terminal 10 is installed in the target vehicle 20, and the target vehicle 20 may also be equipped with a camera 30; the camera can collect real-time image information of the road ahead when the target vehicle is driving; the terminal can acquire the real-time image information of the road ahead collected by the camera.

[0053] Furthermore, in the embodiments of this application, the camera in the target vehicle is pre-calibrated; the purpose of calibration is to obtain the camera's interior orientation elements, distortion parameters, and exterior orientation elements, so that with these parameters, the transformation relationship between a three-dimensional point in space and its projection point in a two-dimensional image can be established.

[0054] For example, in the embodiments of this application, firstly, a custom spatial rectangular coordinate system is established as the benchmark for all subsequent spatial transformation relationships and spatial absolute distance measurements; the custom spatial rectangular coordinate system may have its origin at the vertical projection point of the center point of the front axle of the target vehicle on the ground, with the X-axis pointing to the right side of the target vehicle, the Y-axis pointing in the opposite direction of the target vehicle's forward movement, and the Z-axis pointing directly upward; then, in this custom spatial rectangular coordinate system, according to Zhang Zhengyou's calibration method, multiple photos are first taken using a moving checkerboard to calculate the camera's interior orientation elements, then distortion parameter optimization is performed, and finally, a calibration board laid flat on the ground is photographed to calculate the camera's exterior orientation elements in this spatial rectangular coordinate system.

[0055] Wherein, the interior orientation elements are denoted as: focal length f, pixel sizes dx and dy of the camera in the x and y directions, and pixel coordinates (u0, v0) of the principal point; the exterior orientation elements are denoted as: translation matrix T and rotation matrix R; thus, the transformation relationship between a point (x, y, z) in space and its position (u, v) in the image captured by the camera in the target vehicle can be expressed by the following formula:

[0056]

[0057] z in the formula c This represents a constant parameter.

[0058] It should be noted that, in the embodiments of this application, for any target vehicle, after the camera is installed, only one camera calibration needs to be performed, and the parameters obtained after calibration can be used as fixed system parameters in the camera.

[0059] Step 102: Perform driving trajectory prediction processing based on wavelet analysis method and real-time turning angle information to obtain predicted trajectory information.

[0060] In the embodiments of this application, after the terminal obtains the real-time turning angle information of the target vehicle and the real-time image information of the road ahead, it can perform driving trajectory prediction processing based on wavelet analysis method and real-time turning angle information to obtain predicted trajectory information.

[0061] It should be noted that, in the embodiments of this application, the predicted trajectory information refers to the predicted trajectory of the target vehicle over a future period of time.

[0062] Furthermore, in some embodiments of this application, the terminal may first determine the signal sequence based on real-time corner information; then perform wavelet decomposition on the signal sequence to obtain a signal combination; perform noise filtering on the signal combination according to a preset noise filtering interval to obtain the target signal; then process the target signal based on time information to obtain the predicted signal corresponding to the time information; and finally reconstruct the predicted signal to obtain the predicted trajectory information.

[0063] It should be noted that, in the embodiments of this application, the wavelet analysis method can predict the driving trajectory of the target vehicle in the future, which has good foresight. Therefore, it can issue an early warning when a deviation is detected, so as to avoid the occurrence of an accident.

[0064] For example, Figure 3 This is a schematic diagram of the predicted trajectory information proposed in the embodiments of this application. Figure 1 ,like Figure 3 The image shows the predicted trajectory information projected onto a real-time image of the road ahead captured by a camera. After projection, it can be seen that the predicted trajectory is a straight path. Figure 4 This is a schematic diagram of the predicted trajectory information proposed in the embodiments of this application. Figure 2 ,like Figure 4 As shown, the predicted trajectory information is the trajectory for turning.

[0065] Step 103: Perform lane line detection processing based on the target detection model and real-time image information of the road ahead to obtain lane line information.

[0066] In the embodiments of this application, after the terminal obtains the real-time turning angle information of the target vehicle and the real-time image information of the road ahead, it can perform lane line detection processing based on the target detection model and the real-time image information of the road ahead to obtain lane line information.

[0067] It should be noted that, in the embodiments of this application, the target detection model is obtained after training the initial detection model based on the preset error calculation model; the preset error calculation model proposed in this application can improve the training effect of the initial detection model, thereby improving the detection accuracy of the target detection model and achieving more accurate lane line detection.

[0068] It is understood that, in the embodiments of this application, the target detection model can perform lane line detection processing based on real-time image information of the road ahead, and the obtained lane line information is the detected lane line; for example, Figure 5 This is a schematic diagram of lane line information proposed in an embodiment of this application, such as... Figure 5 As shown, in the real-time image information ahead of the road, the target detection model can detect and label the lane lines ahead.

[0069] It should be noted that, in the embodiments of this application, when using the target detection model to detect lane lines in real-time image information ahead of the road, not only can the lane line of the current lane where the target vehicle is located be detected, but if there are lanes in the same direction on both sides of the current lane, the lane lines of the lanes in the same direction can also be detected; that is, the lane line information can include the lane line of the lane where the target vehicle is currently located, and if there are lanes in the same direction on both sides of the current lane, it also includes the lane lines in the same direction as the current lane.

[0070] Furthermore, after obtaining lane line information, the terminal can directly determine the current lane of the target vehicle and whether there are lanes in the same direction based on the lane line information.

[0071] Step 104: If deviation behavior is determined based on the predicted trajectory information and lane line information, an early warning is issued.

[0072] In the embodiments of this application, the terminal performs driving trajectory prediction processing based on wavelet analysis method and real-time turning angle information to obtain predicted trajectory information, and performs lane line detection processing based on target detection model and real-time image information of the road ahead to obtain lane line information. If it is determined that there is deviation behavior based on the predicted trajectory information and lane line information, a warning processing is performed.

[0073] It should be noted that in some embodiments of this application, the terminal may first calculate the distance information between the predicted trajectory information and the lane line information; if the distance information is less than a preset threshold, the predicted driving direction is determined based on the predicted trajectory information, and the current lane of the target vehicle is determined based on the lane line information; if it is determined based on the lane line information that there is no lane in the same direction as the predicted driving direction in the current lane, then it is determined that there is a deviation behavior.

[0074] Further, in the embodiments of this application, if the distance information is less than a preset threshold, the predicted driving direction is determined based on the predicted trajectory information, and the current lane of the target vehicle is determined based on the lane line information; then, if there is a lane in the same direction as the current lane in the predicted driving direction, the first historical steering wheel angle information corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval are obtained; a difference curve is determined based on the first historical steering wheel angle information and the first historical lane line information, and a turning point information is determined based on the difference curve; wherein, the turning point information represents the starting information of the driving angle deviating from the lane line; a first vector, a second vector, and a third vector are determined based on the turning point information and the predicted trajectory information. Three vectors are used; if the product of the first and second vectors is greater than or equal to the product of the second and third vectors, then deviation behavior is determined to exist; if the product of the first and second vectors is less than the product of the second and third vectors, then the second historical steering angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval are obtained; at least one difference integral information between the second historical steering angle information and the second historical lane line information is calculated based on a preset time interval; the change trend of the difference integral corresponding to the second historical time interval is determined based on at least one difference integral information; if the change trend of the difference integral is an increasing trend, then deviation behavior is determined to exist.

[0075] It should be noted that, in the embodiments of this application, deviation behavior refers to the target vehicle's unintentional deviation, rather than active deviation; that is, the target vehicle can actively deviate, such as actively changing lanes. In the case of such active deviation, no warning processing is performed. Warning processing is only performed when the target vehicle deviates unintentionally.

[0076] It is understood that in the embodiments of this application, if it is determined that there is no deviation behavior based on the predicted trajectory information and lane line information, no warning processing will be performed.

[0077] Furthermore, in the embodiments of this application, warning processing can be carried out by means of sound warning, warning information display, etc.

[0078] Furthermore, in the embodiments of this application, before the terminal performs lane line detection processing based on the target detection model and real-time image information of the road ahead, i.e., before step 103, the following steps may also be included:

[0079] Step 105: Obtain the training dataset; the training dataset includes lane line annotation point information.

[0080] In the embodiments of this application, the terminal performs lane line detection processing based on the target detection model and real-time image information of the road ahead. Before obtaining lane line information, a training dataset can be obtained first; wherein, the training dataset includes lane line annotation point information.

[0081] It should be noted that, in the embodiments of this application, the lane line marking point information is the relevant data of the marking points obtained by marking actual lane lines.

[0082] Step 106: Use the initial detection model to perform object detection processing on the training dataset to obtain the lane line marking point detection results corresponding to the lane line marking point information.

[0083] In the embodiments of this application, after obtaining the training dataset, the terminal can use the initial detection model to perform target detection processing on the training dataset to obtain the lane line marking point detection results corresponding to the lane line marking point information.

[0084] It is understood that, in the embodiments of this application, since the training dataset contains lane line marking point information, the initial detection model can perform target detection processing on this lane line marking point information to obtain lane line marking point detection results based on this lane line marking point information; there is an error between the lane line marking point detection results and the lane line marking point information, and these errors are calculated based on the preset error calculation model proposed in this application to update and optimize the initial detection model.

[0085] It should be noted that, in the embodiments of this application, the initial detection model can be a transformer-based deep learning network; its structure may include convolutional layers, encoders, decoders, parameter feedforward layers, etc.

[0086] Step 107: Calculate the error information between the lane line marking point detection results and the lane line marking point information, and update the initial detection model based on the error information to obtain the target detection model.

[0087] In the embodiments of this application, after the terminal performs target detection processing on the training dataset using the initial detection model and obtains the lane line marking point detection result corresponding to the lane line marking point information, it can calculate the error information between the lane line marking point detection result and the lane line marking point information, and update the initial detection model according to the error information to obtain the target detection model.

[0088] It should be noted that, in the embodiments of this application, a preset error calculation model can be used to calculate the error between the lane line marking point information (marking point coordinates) and the lane line marking point detection result (detected point coordinates). The preset error calculation model can be expressed as the following formula:

[0089]

[0090] Where cost represents error information; p l,x p represents the x-axis coordinate in the lane line marker detection results. gt,x This represents the x-axis coordinate in the lane line marking point information; p l,y p represents the y-axis coordinate in the lane line marker detection results. gt,y This represents the y-axis coordinate in the lane line marking point information.

[0091] Therefore, the preset error calculation model proposed in this application uses the error sequence of points on the lane line as the component of error calculation, which is a discretized error acquisition method. This can avoid the error accumulation caused by the fitting and transformation process of existing error calculation methods, and also avoid the influence of errors caused by image distortion or projection. This can improve the training effect and improve the detection accuracy of the target detection model.

[0092] Figure 6 This is a schematic diagram of the implementation process of the trajectory deviation early warning method proposed in the embodiments of this application. Figure 2 ,like Figure 6 As shown, the method for the terminal to perform driving trajectory prediction processing based on wavelet analysis and real-time turning angle information, and to obtain the predicted trajectory information, namely the method proposed in step 102, may include the following steps:

[0093] Step 102a: Determine the signal sequence based on real-time rotation angle information.

[0094] In some embodiments of this application, the terminal performs driving trajectory prediction processing based on wavelet analysis and real-time turning angle information to obtain predicted trajectory information; in some embodiments of this application, the terminal may first determine the signal sequence based on real-time turning angle information.

[0095] For example, in an embodiment of this application, the application can collect real-time corner information over a period of time and organize this real-time corner information into a one-dimensional signal sequence that varies with time. This signal sequence can be represented as: A = f(t); where t = t0, t1, t2, ... t k .

[0096] In other words, the terminal can acquire and store real-time turning information over a period of time. After converting the real-time turning information over this period of time into a signal sequence, the vehicle trajectory prediction is transformed into a problem of predicting the trend of a signal.

[0097] Step 102b: Perform wavelet decomposition on the signal sequence based on wavelet analysis to obtain the signal combination.

[0098] In the embodiments of this application, after the terminal determines the signal sequence based on real-time corner information, it can perform wavelet decomposition processing on the signal sequence based on wavelet analysis to obtain the signal combination.

[0099] It should be noted that in the embodiments of this application, due to sensor acquisition errors, transmission errors, and occasional steering fluctuations when the driver operates the steering wheel, the real-time turning angle information acquired by the terminal contains some noise. Therefore, it is necessary to remove this noise from the signal sequence. Experiments have shown that the true information in the signal sequence is concentrated in the lower frequency components, while the disturbances caused by factors such as acquisition, transmission, and operation errors, i.e., noise, are characterized by high frequency and sporadic occurrence. Therefore, the approach adopted in this application is to perform wavelet decomposition processing on the signal sequence, which can decompose the signal sequence into an approximate part and a detail part, thereby obtaining a signal combination. Among them, the approximate part represents the true signal, and the detail part represents the noise.

[0100] For example, in an embodiment of this application, a trigonometric function-like wavelet is constructed as the wavelet basis for wavelet decomposition processing, thereby performing wavelet decomposition processing on the signal sequence through a two-level decomposition method; the wavelet basis can be expressed as the following formula:

[0101]

[0102] Where C is a constant parameter, e is a natural constant, t represents time, and cos(t) represents the cosine function.

[0103] Figure 7 This is a schematic diagram illustrating the implementation of wavelet decomposition processing proposed in the embodiments of this application. Figure 1 ,like Figure 7 The image shows the signal sequence before wavelet decomposition; furthermore, Figure 8 This is a schematic diagram illustrating the implementation of wavelet decomposition processing proposed in the embodiments of this application. Figure 2 ,like Figure 8 The diagram shows the wavelet basis pairs described above. Figure 7 The signal sequence in the data is subjected to two-level wavelet decomposition to obtain the approximation part (Approximation A1) in the signal combination; Figure 9 This is a schematic diagram illustrating the implementation of wavelet decomposition processing proposed in the embodiments of this application. Figure 3 ,like Figure 9 The diagram shows the wavelet basis pairs described above. Figure 3 The signal sequence is subjected to two-level wavelet decomposition to obtain the detail part (Detail D1) of the signal combination.

[0104] Furthermore, after wavelet decomposition, the signal sequence is transformed into a linear combination of multi-resolution signals, equivalent to wavelet function fitting at different scales and times, thus obtaining the signal combination; the signal combination can be expressed by the following formula:

[0105] f(t)=∑ k ∑ j α j,k ψ j,k (t) (6)

[0106] Where, α j,k For the harmonic parameter, ψ j,k (t) represents a family of wavelet functions, and j and k are wavelet coefficients.

[0107] Step 102c: Perform noise filtering on the signal combination according to the preset noise filtering interval to obtain the target signal.

[0108] In the embodiments of this application, after the terminal performs wavelet decomposition on the signal sequence to obtain the signal combination, it can perform noise filtering on the signal combination according to a preset noise filtering interval to obtain the target signal.

[0109] It should be noted that in the embodiments of this application, since in the wavelet domain, the real information is concentrated on the low-frequency components, and their wavelet coefficients are generally large, while noise is generally concentrated on the higher-frequency components, and their wavelet coefficients are generally small, this application collects some typical noise components, statistically analyzes their wavelet coefficients, records the mean of their wavelet coefficients as σ, and obtains the preset noise filtering interval based on the mean of the wavelet coefficients.

[0110] For example, in an embodiment of this application, the preset noise filtering interval is [-3×σ, 3×σ]; thus, signals in the signal combination whose wavelet coefficients fall within this preset noise filtering interval are filtered out to complete the noise filtering process, and the obtained target signal can be expressed as the following formula:

[0111] f′(t)=∑ k ∑ j α j,k ψ j,k (t) (7)

[0112] Where α>3σ or α<3σ.

[0113] For example, in the embodiments of this application, Figure 10 This is a schematic diagram of the target signal proposed in the embodiments of this application, such as... Figure 10 The image shows the target signal obtained after noise filtering.

[0114] Step 102d: Determine the predicted trajectory information based on the target signal.

[0115] In the embodiments of this application, after the terminal performs noise filtering processing on the signal combination according to the preset noise filtering interval and obtains the target signal, it can determine the predicted trajectory information based on the target signal.

[0116] In some embodiments of this application, the terminal can process the target signal based on time information to obtain the predicted signal corresponding to the time information; then, the predicted signal is reconstructed to obtain the predicted trajectory information.

[0117] Figure 11 This is a schematic diagram of the implementation process of the trajectory deviation early warning method proposed in the embodiments of this application. Figure 3 ,like Figure 11 As shown, the method for the terminal to determine the predicted trajectory information based on the target signal, i.e., the method proposed in step 102d, may include the following steps:

[0118] Step 102d1: Process the target signal based on time information to obtain the prediction signal corresponding to the time information.

[0119] In some embodiments of this application, the terminal determines the predicted trajectory information based on the target signal; in some embodiments of this application, the terminal may first process the target signal based on time information to obtain the predicted signal corresponding to the time information.

[0120] It should be noted that, in the embodiments of this application, time information refers to a future period of time, and the prediction signal to be obtained is the signal corresponding to that future period of time.

[0121] For example, in an embodiment of this application, the target signal is processed based on time information t+Δt to obtain the predicted signal corresponding to t+Δt, which can be expressed by the following formula:

[0122] f′(t+△t)=∑ k ∑ j α j,k ψ j,k (t+△t) (8)

[0123] In other words, by substituting the time information t+Δt into the target signal, the predicted signal corresponding to the time information can be obtained.

[0124] Step 102d2: Reconstruct the predicted signal to obtain the predicted trajectory information.

[0125] In the embodiments of this application, after the terminal processes the target signal based on time information to obtain the predicted signal corresponding to the time information, it can reconstruct the predicted signal to obtain the predicted trajectory information.

[0126] It should be noted that, in the embodiments of this application, the reconstruction process refers to the inverse transform of decomposing the predicted signal.

[0127] Furthermore, Figure 12 This is a schematic diagram of the predicted trajectory information proposed in the embodiments of this application. Figure 3 ,like Figure 12 The diagram shows the predicted trajectory information obtained after processing by the above series of wavelet analysis methods.

[0128] Furthermore, in the embodiments of this application, if the terminal determines that there is a deviation behavior based on the predicted trajectory information and lane line information, the following steps may be included before the warning processing, i.e. before step 104:

[0129] Step 108: Calculate the distance information between the predicted trajectory information and the lane line information.

[0130] In the embodiments of this application, if the terminal determines that there is a deviation behavior based on the predicted trajectory information and lane line information, the distance information between the predicted trajectory information and lane line information is calculated before the warning processing is performed.

[0131] It should be noted that, in the embodiments of this application, the distance information can be calculated by calculating the distance on the image and then projecting and converting it.

[0132] Step 109: If the distance information is less than the preset threshold, determine the predicted driving direction based on the predicted trajectory information and determine the current lane of the target vehicle based on the lane line information.

[0133] In the embodiments of this application, after the terminal calculates the distance information between the predicted trajectory information and the lane line information, if the distance information is less than a preset threshold, the terminal determines the predicted driving direction based on the predicted trajectory information and determines the current lane of the target vehicle based on the lane line information.

[0134] It should be noted that in the embodiments of this application, if only the distance information between the predicted trajectory information and the lane line information is used as the basis for judging deviation, then when the vehicle actively changes lanes, it will also be judged as a deviation, and warnings will be issued continuously, which is obviously inappropriate. Therefore, this application adopts a series of behavioral analysis methods to distinguish between the situation of the target vehicle actively changing lanes and the situation of deviation. When deviation occurs, a deviation warning is issued, and when actively changing lanes, a lane change prompt can be issued.

[0135] Therefore, when the distance information is less than a preset threshold, this application first determines the predicted driving direction based on the predicted trajectory information and determines the current lane of the target vehicle based on the lane line information; thereby determining whether there is a lane in the same direction on the side where the target vehicle is about to cross the line when it is driving in the predicted direction.

[0136] For example, in an embodiment of this application, if the distance information obtained at a certain moment during the driving of the target vehicle is 19cm and the preset threshold is 20cm, then it is necessary to determine the predicted driving direction based on the predicted trajectory information and determine the current lane of the target vehicle based on the lane line information.

[0137] It is understood that, in the embodiments of this application, the terminal can directly determine the current lane of the target vehicle based on the lane line information.

[0138] Step 110: If the lane information indicates that there is no lane in the same direction as the current lane in the predicted driving direction, then a deviation behavior is confirmed.

[0139] In the embodiments of this application, after the terminal calculates the distance information between the predicted trajectory information and the lane line information, if it determines based on the lane line information that there is no lane in the same direction as the current lane in the predicted driving direction, then it is determined that there is a deviation behavior.

[0140] It is understood that, in the embodiments of this application, if it is determined from the lane line information that there is no lane in the same direction as the current lane in the predicted driving direction, then the target vehicle is definitely not actively changing lanes, and it is determined that there is a deviation behavior.

[0141] Furthermore, in the embodiments of this application, if the distance information is less than a preset threshold, the predicted driving direction is determined based on the predicted trajectory information, and the current lane of the target vehicle is determined based on the lane line information, i.e., after step 109, the following steps may be included:

[0142] Step 110: If there is a lane in the same direction as the current lane in the predicted driving direction, then obtain the first historical steering angle information of the steering wheel corresponding to the first historical time interval, and the first historical lane line information corresponding to the first historical time interval.

[0143] In the embodiments of this application, if the distance information is less than a preset threshold, the terminal determines the predicted driving direction based on the predicted trajectory information, and determines the current lane of the target vehicle based on the lane line information. If there is a lane in the same direction as the current lane in the predicted driving direction, the terminal obtains the first historical turning angle information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval.

[0144] It is understood that in the embodiments of this application, if there is a lane in the same direction as the current lane in the predicted driving direction, there is a possibility of lane change, and it is necessary to combine other information to further determine whether there is a deviation.

[0145] It should be noted that, in the embodiments of this application, the first historical time interval refers to a period of time before the current moment for the target vehicle, the first historical turning angle information is the steering wheel turning angle information corresponding to the first historical time interval, and the first historical lane line information is the lane line information corresponding to the first historical time interval.

[0146] The first historical lane line information is obtained based on real-time image information of the road ahead corresponding to the first historical time interval.

[0147] Step 111: Determine the difference curve based on the first historical turning angle information and the first historical lane line information, and determine the turning point information based on the difference curve; wherein, the turning point information represents the starting information of the driving angle deviating from the lane line.

[0148] In the embodiments of this application, if there is a lane in the same direction as the current lane in the predicted driving direction, the terminal obtains the first historical turning angle information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval. Then, it can determine the difference curve based on the first historical turning angle information and the first historical lane line information, and determine the turning point information based on the difference curve. The turning point information represents the starting information of the driving angle deviating from the lane line.

[0149] It should be noted that, in the embodiments of this application, the turning point information represents the starting information of the vehicle's angle deviating from the lane line.

[0150] It is understandable that when the target vehicle is driving normally, the steering wheel angle information and the lane line angle should be "consistent". However, from the moment the target vehicle begins to deviate, the steering wheel angle information and the lane line angle become "inconsistent". This application determines the difference curve based on the first historical steering angle information and the first historical lane line information, and then performs extreme value analysis on the difference curve to obtain the turning point from "consistent" to "inconsistent" in the difference curve, i.e., the turning point information.

[0151] Step 112: Determine whether there is any deviation behavior based on the inflection point information.

[0152] In the embodiments of this application, after the terminal determines the difference curve based on the first historical turning angle information and the first historical lane line information, and determines the turning point information based on the difference curve, it can determine whether there is a deviation behavior based on the turning point information.

[0153] In some embodiments of this application, the terminal can determine the first vector, the second vector, and the third vector based on the turning point information and the predicted trajectory information; then, if the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector, it is determined that there is a deviation behavior.

[0154] It should be noted that, in the embodiments of this application, the turning point information includes the time information and the location information of the turning point; it can be understood that the time information of the turning point is the moment when the turning point occurs, and the location information of the turning point is the location where the turning point occurs.

[0155] Furthermore, in the embodiments of this application, the method by which the terminal determines whether there is deviation behavior based on the inflection point information, i.e., the method proposed in step 112, may include the following steps:

[0156] Step 112a: Determine the first vector, the second vector, and the third vector based on the turning point information and the predicted trajectory information.

[0157] In some embodiments of this application, the terminal determines whether there is deviation behavior based on the turning point information; in some embodiments of this application, the terminal may first determine the first vector, the second vector, and the third vector based on the turning point information and the predicted trajectory information.

[0158] It should be noted that, in the embodiments of this application, the first vector is a vector that starts with the position information of the turning point and ends with the position information corresponding to the next moment of the time information of the turning point in the predicted trajectory information.

[0159] It should be noted that, in the embodiments of this application, a line parallel to the lane line is drawn through the position information of the turning point, and a vector of a unit length on this line is taken to form a second vector.

[0160] It should be noted that, in the embodiments of this application, the third vector is a vector that starts with the position information of the turning point and ends with the position information corresponding to the previous moment of the time information of the turning point in the predicted trajectory information.

[0161] Step 112b: If the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector, then it is determined that there is deviation behavior.

[0162] In the embodiments of this application, after the terminal determines the first vector, the second vector, and the third vector based on the turning point information and the predicted trajectory information, if the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector, then it is determined that there is a deviation behavior.

[0163] For example, in an embodiment of this application, the first vector is represented as The second vector is represented as The third vector is represented as Calculate the vector product:

[0164]

[0165]

[0166] If L1 > L2 or L1 = L2, then a deviation behavior is identified and an early warning is required.

[0167] Furthermore, in embodiments of this application, after the terminal determines the first vector, the second vector, and the third vector based on the turning point information and the predicted trajectory information, i.e., after step 112a, the following steps may also be included:

[0168] Step 112c: If the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, then obtain the second historical steering angle information of the steering wheel corresponding to the second historical time interval, and the second historical lane line information corresponding to the second historical time interval.

[0169] In the embodiments of this application, after the terminal determines the first vector, the second vector, and the third vector based on the turning point information and the predicted trajectory information, if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, then the second historical turning angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval are obtained.

[0170] It should be noted that, in the embodiments of this application, if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, it can be determined that the target vehicle is likely to have turned in the direction of deviating from the lane by the driver manipulating the steering wheel; further, the second historical turning angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval are obtained to determine whether it is an active lane change behavior. If it is not an active lane change behavior, a warning is issued.

[0171] Furthermore, in the embodiments of this application, the second historical time interval refers to the time period from the turning point to the current moment; the second historical turning angle information is the steering wheel turning angle information corresponding to the second historical time interval, and the second historical lane line information is the lane line information corresponding to the second historical time interval.

[0172] It is understood that, in the embodiments of this application, the second historical lane line information is obtained based on the real-time image information of the road ahead corresponding to the second historical time interval.

[0173] Step 112d: Calculate at least one difference integral between the second historical turning angle information and the second historical lane line information based on a preset time interval.

[0174] In the embodiments of this application, if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, then after the terminal obtains the second historical turning angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval, it can calculate at least one difference integral information between the second historical turning angle information and the second historical lane line information based on a preset time interval.

[0175] It should be noted that, in the embodiments of this application, the corresponding driving trajectory information can be determined based on the second historical turning angle information, and then at least one difference integral information between the driving trajectory information corresponding to the second historical turning angle information and the second historical lane line information can be calculated based on a preset time interval.

[0176] In other words, by calculating the difference integral between the driving trajectory information corresponding to the second historical turning angle information and the second historical lane line information at preset time intervals, at least one difference integral information can be obtained.

[0177] For example, in an embodiment of this application, the method for calculating the difference integral information can be expressed as the following formula:

[0178]

[0179] Among them, f car (t) represents the second historical turning point information, f lane (t) represents the second historical lane line information.

[0180] Step 112e: Determine the trend of the difference integral change corresponding to the second historical time interval based on at least one difference integral information.

[0181] In the embodiments of this application, after the terminal calculates at least one difference integral information between the second historical turning angle information and the second historical lane line information based on a preset time interval, it can determine the difference integral change trend corresponding to the second historical time interval based on at least one difference integral information.

[0182] It should be noted that, in the embodiments of this application, the trend of difference integral change refers to the trend of change of at least one difference integral information obtained from the time information of the turning point in the second historical time interval to the current time.

[0183] Step 112f: If the trend of the difference integral is increasing, then it is determined that there is no deviation behavior.

[0184] In the embodiments of this application, after the terminal determines the trend of the difference integral change corresponding to the second historical time interval based on at least one difference integral information, if the trend of the difference integral change is an increasing trend, it is determined that there is no deviation behavior.

[0185] It should be noted that in the embodiments of the present application, if the change trend of the difference integral is an increasing trend, it indicates that the target vehicle is making an active lane change behavior by the driver actively operating the steering wheel, rather than a deviation behavior. At this time, lane change prompt information can be generated.

[0186] It should be noted that in the embodiments of the present application, if the change trend of the difference integral is not an increasing trend, such as decreasing or remaining unchanged, it is determined that there is a deviation behavior and early warning processing needs to be performed.

[0187] In summary, Figure 13 is a schematic implementation process of the trajectory deviation early warning method proposed by the embodiments of the present application Figure 4 , as Figure 13 shown, it is judged whether the distance information between the predicted trajectory information and the lane line information is less than a preset threshold (step 201). If so, it is further judged whether there is a same-direction lane on the side where it is about to cross the line (step 202). If there is a same-direction lane, the turning point information is first determined (step 203), and then the vector products L1 and L2 are calculated based on the turning point information (step 204), and then it is judged whether L1 is less than L2 (step 205); if there is no same-direction lane, deviation early warning is performed (step 206); then, if it is determined based on step 205 that L1 < L2, at least one difference integral information is calculated (step 207), and then it is judged whether the change trend of at least one difference integral information is an increasing trend (step 208). If the change trend is an increasing trend, it is determined that there is no deviation behavior, and lane change prompt information can be generated (step 209). If the change trend is not an increasing trend, it is determined that there is a deviation behavior and deviation early warning is performed (step 206); if it is judged based on step 201 that it is not less than the preset threshold, it is determined that there is no deviation (step 210), and deviation early warning is not required.

[0188] Therefore, the trajectory deviation warning method in this application is based on wavelet analysis for vehicle trajectory prediction. The error model of the traditional deep learning network for object detection has been improved, resulting in better training performance and greater suitability for lane line detection. Furthermore, it can distinguish between active lane changes and unintentional deviations based on predicted trajectory and lane line information. This trajectory deviation warning method can be applied to automated intelligent vehicle-road cooperative projects. Autonomous trucks equipped with this technology have demonstrated excellent detection capabilities for lane lines of various shapes and road scenarios during field testing, with predicted trajectories matching actual trajectories, exhibiting high overall accuracy. It can promptly detect danger and issue warnings when unintentional deviations or abnormal driving conditions lead to lane departures, contributing to lane keeping and driving safety. Simultaneously, when actively maneuvering the steering wheel for lane changes, it correctly identifies the lane change intention, avoiding lane departure warnings and instead displaying a lane change prompt, making the autonomous driving function more intelligent. This solves the previously existing problem of false lane change warnings, improves the efficiency of the remote driving control console in automated intelligent vehicle-road cooperative projects, and achieves intelligent trajectory deviation warning.

[0189] This application provides a trajectory deviation warning method, a terminal, and a storage medium. The terminal is installed in a target vehicle. When the target vehicle is driving, it acquires real-time turning angle information of the target vehicle and real-time image information of the road ahead. Based on wavelet analysis and real-time turning angle information, it performs trajectory prediction processing to obtain predicted trajectory information. Based on a target detection model and real-time image information of the road ahead, it performs lane line detection processing to obtain lane line information. If deviation behavior is determined based on the predicted trajectory information and lane line information, a warning is issued. Therefore, in this application, the terminal is installed in the target vehicle. During the vehicle's movement, the terminal can perform trajectory prediction processing on real-time turning angle information using wavelet analysis to obtain predicted trajectory information for a future period, thus enabling the estimation of the vehicle's position over a future period, providing excellent foresight. Simultaneously, using a target detection model to perform lane line detection processing on real-time image information of the road ahead improves the accuracy of lane line detection. Thus, based on the predicted trajectory information and lane line information, it can predict whether the target vehicle will deviate, and issue a warning when deviation behavior is detected, effectively improving the deviation warning effect.

[0190] In another embodiment of this application, Figure 14 This is a schematic diagram of the terminal structure proposed in the embodiments of this application. Figure 2 ,like Figure 14 As shown, the terminal 10 proposed in this application embodiment may include an acquisition unit 11, a prediction unit 12, a detection unit 13, an early warning unit 14, and a calculation unit 15.

[0191] The acquisition unit 11 is used to acquire the real-time turning angle information and real-time image information of the road ahead of the target vehicle when the target vehicle is driving.

[0192] The prediction unit 12 is used to perform driving trajectory prediction processing based on wavelet analysis and the real-time turning angle information to obtain predicted trajectory information.

[0193] The detection unit 13 is used to perform lane line detection processing based on the target detection model and the real-time image information of the road ahead to obtain lane line information.

[0194] The early warning unit 14 is used to issue an early warning if it is determined that there is a deviation behavior based on the predicted trajectory information and the lane line information.

[0195] Furthermore, the acquisition unit 11 is also used to acquire a training dataset before performing lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information; wherein, the training dataset includes lane line annotation point information.

[0196] Furthermore, the detection unit 13 is also used to perform target detection processing on the training dataset using the initial detection model to obtain the lane line marking point detection result corresponding to the lane line marking point information; and to calculate the error information between the lane line marking point detection result and the lane line marking point information, and update the initial detection model according to the error information to obtain the target detection model.

[0197] Furthermore, the acquisition unit 11 is also used to determine a signal sequence based on the real-time turning information; to perform wavelet decomposition processing on the signal sequence to obtain a signal combination; to perform noise filtering processing on the signal combination according to a preset noise filtering interval to obtain a target signal; and to determine the predicted trajectory information based on the target signal.

[0198] Furthermore, the acquisition unit 11 is also used to process the target signal based on time information to obtain a prediction signal corresponding to the time information; and to reconstruct the prediction signal to obtain the prediction trajectory information.

[0199] The calculation unit 15 is configured to calculate the distance information between the predicted trajectory information and the lane line information before the warning unit 14 performs warning processing if it determines that there is a deviation behavior based on the predicted trajectory information and the lane line information; if the distance information is less than a preset threshold, the predicted driving direction is determined based on the predicted trajectory information, and the current lane of the target vehicle is determined based on the lane line information; if it is determined based on the lane line information that there is no lane in the same direction as the predicted driving direction in the current lane, the deviation behavior is determined to exist.

[0200] Furthermore, the acquisition unit 11 is also used to, if the distance information is less than a preset threshold, then after the calculation unit 15 determines the predicted driving direction based on the predicted trajectory information and determines the current lane of the target vehicle based on the lane line information, if there is a lane in the same direction as the current lane in the predicted driving direction, then acquire the first historical turning angle information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval.

[0201] Furthermore, the calculation unit 15 is also configured to determine a difference curve based on the first historical turning angle information and the first historical lane line information, and to determine turning point information based on the difference curve; wherein the turning point information represents the starting information of the driving angle deviating from the lane line; and to determine whether the deviation behavior exists based on the turning point information.

[0202] Furthermore, the calculation unit 15 is also used to determine a first vector, a second vector, and a third vector based on the turning point information and the predicted trajectory information; and if the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector, then it is determined that the deviation behavior exists.

[0203] Furthermore, the calculation unit 15 is also configured to: after determining the first vector, the second vector, and the third vector based on the turning point and the predicted trajectory information; if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector; then obtain the second historical turning angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval; calculate at least one difference integral information between the second historical turning angle information and the second historical lane line information based on a preset time interval; and determine the difference integral change trend corresponding to the second historical time interval based on the at least one difference integral information; if the difference integral change trend is an increasing trend, then determine that there is no deviation behavior.

[0204] Figure 15 This is a schematic diagram of the terminal structure proposed in the embodiments of this application. Figure 3 ,like Figure 15 As shown, the terminal 10 proposed in this application embodiment may further include a processor 16, a memory 17 storing instructions executable by the processor 16, and further, the terminal 10 may also include a communication interface 18 and a bus 19 for connecting the processor 16, the memory 17 and the communication interface 18.

[0205] In the embodiments of this application, the processor 16 can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, and microprocessor. It is understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other types, and this application embodiment does not specifically limit this. The processor 16 may also include a memory 17, which can be connected to the processor 16. The memory 17 is used to store executable program code, which includes computer operation instructions. The memory 17 may include high-speed RAM memory and may also include non-volatile memory, such as at least two disk drives.

[0206] In embodiments of this application, bus 19 is used to connect communication interface 18, processor 16, and memory 17, as well as the mutual communication between these devices.

[0207] In embodiments of this application, memory 17 is used to store instructions and data.

[0208] Furthermore, in the embodiments of this application, the processor 16 is configured to, when the target vehicle is driving, acquire the real-time turning angle information of the target vehicle and the real-time image information of the road ahead; perform driving trajectory prediction processing based on wavelet analysis method and the real-time turning angle information to obtain predicted trajectory information; perform lane line detection processing based on target detection model and the real-time image information of the road ahead to obtain lane line information; and if it is determined that there is deviation behavior based on the predicted trajectory information and the lane line information, perform early warning processing.

[0209] In practical applications, the aforementioned memory 17 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 16.

[0210] Furthermore, in this embodiment, the functional modules can be integrated into one analysis unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0211] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] This application provides a trajectory deviation warning method, a terminal, and a storage medium. The terminal is installed in a target vehicle. When the target vehicle is driving, it acquires real-time turning angle information of the target vehicle and real-time image information of the road ahead. Based on wavelet analysis and real-time turning angle information, it performs trajectory prediction processing to obtain predicted trajectory information. Based on a target detection model and real-time image information of the road ahead, it performs lane line detection processing to obtain lane line information. If deviation behavior is determined based on the predicted trajectory information and lane line information, a warning is issued. Therefore, in this application, the terminal is installed in the target vehicle. During the vehicle's movement, the terminal can perform trajectory prediction processing on real-time turning angle information using wavelet analysis to obtain predicted trajectory information for a future period, thus enabling the estimation of the vehicle's position over a future period, providing excellent foresight. Simultaneously, using a target detection model to perform lane line detection processing on real-time image information of the road ahead improves the accuracy of lane line detection. Thus, based on the predicted trajectory information and lane line information, it can predict whether the target vehicle will deviate, and issue a warning when deviation behavior is detected, effectively improving the deviation warning effect.

[0213] Specifically, the program instructions corresponding to a trajectory deviation warning method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives; when the program instructions corresponding to a trajectory deviation warning method in the storage media are read or executed by an electronic device, the following steps are included:

[0214] When the target vehicle is driving, acquire the real-time turning angle information of the target vehicle and the real-time image information of the road ahead;

[0215] Based on wavelet analysis and the real-time turning angle information, driving trajectory prediction processing is performed to obtain predicted trajectory information;

[0216] Lane line detection processing is performed based on the target detection model and the real-time image information of the road ahead to obtain lane line information;

[0217] If a deviation is determined based on the predicted trajectory information and the lane line information, an early warning is issued.

[0218] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0219] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the schematic and / or block diagrams can be implemented by computer program instructions, and combinations of blocks in the schematic and / or block diagrams can be implemented. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the schematic and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0220] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0221] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0222] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A trajectory deviation early warning method, characterized in that, The trajectory deviation warning method is applied to a terminal, which is installed in the target vehicle, and the method includes: When the target vehicle is driving, real-time turning angle information of the target vehicle and real-time image information of the road ahead are acquired; driving trajectory prediction processing is performed based on wavelet analysis method and the real-time turning angle information to obtain predicted trajectory information; lane line detection processing is performed based on target detection model and the real-time image information of the road ahead to obtain lane line information. Calculate the distance information between the predicted trajectory information and the lane line information; if the distance information is less than a preset threshold, determine the predicted driving direction based on the predicted trajectory information and determine the current lane of the target vehicle based on the lane line information; if it is determined based on the lane line information that there is no lane in the same direction as the predicted driving direction in the current lane, then determine that the deviation behavior exists. If there is a lane in the same direction as the current lane in the predicted driving direction, then the first historical steering angle information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval are obtained; a difference curve is determined based on the first historical steering angle information and the first historical lane line information, and a turning point information is determined based on the difference curve; wherein, the turning point information represents the starting information of the driving angle deviating from the lane line. Based on the turning point information and the predicted trajectory information, a first vector, a second vector, and a third vector are determined; if the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector, then the deviation behavior is determined to exist. If the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, then the second historical steering angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval are obtained; at least one difference integral information between the second historical steering angle information and the second historical lane line information is calculated based on a preset time interval; the difference integral change trend corresponding to the second historical time interval is determined according to the at least one difference integral information; if the difference integral change trend is an increasing trend, then it is determined that there is no deviation behavior. If a deviation is determined based on the predicted trajectory information and the lane line information, an early warning is issued.

2. The method according to claim 1, characterized in that, Before obtaining lane line information by performing lane line detection processing based on the target detection model and the real-time image information ahead of the road, the method includes: Obtain a training dataset; wherein the training dataset includes lane line annotation point information; The initial detection model is used to perform target detection processing on the training dataset to obtain the lane line marking point detection results corresponding to the lane line marking point information. Calculate the error information between the lane line marking point detection results and the lane line marking point information, and update the initial detection model based on the error information to obtain the target detection model.

3. The method according to claim 1, characterized in that, The process of predicting the driving trajectory based on wavelet analysis and the real-time turning angle information to obtain predicted trajectory information includes: The signal sequence is determined based on the real-time rotation information; The signal sequence is decomposed using the wavelet analysis method described above to obtain a signal combination. The signal combination is subjected to noise filtering processing according to a preset noise filtering interval to obtain the target signal; The predicted trajectory information is determined based on the target signal.

4. The method according to claim 3, characterized in that, Determining the predicted trajectory information based on the target signal includes: The target signal is processed based on time information to obtain the predicted signal corresponding to the time information; The predicted signal is reconstructed to obtain the predicted trajectory information.

5. A terminal, characterized in that, The terminal is installed in the target vehicle and includes an acquisition unit, a prediction unit, a detection unit, an early warning unit, and a calculation unit. The acquisition unit is used to acquire the real-time turning angle information and real-time image information of the road ahead of the target vehicle when the target vehicle is driving. The prediction unit is used to perform driving trajectory prediction processing based on wavelet analysis method and the real-time turning angle information to obtain predicted trajectory information. The detection unit is used to perform lane line detection processing based on the target detection model and the real-time image information of the road ahead to obtain lane line information. The warning unit is used to issue a warning if it is determined that there is a deviation behavior based on the predicted trajectory information and the lane line information. The calculation unit is configured to calculate the distance information between the predicted trajectory information and the lane line information before the warning unit performs warning processing if it determines that there is a deviation behavior based on the predicted trajectory information and the lane line information; if the distance information is less than a preset threshold, the predicted driving direction is determined based on the predicted trajectory information, and the current lane of the target vehicle is determined based on the lane line information; if it is determined based on the lane line information that there is no lane in the same direction as the predicted driving direction in the current lane, the deviation behavior is determined to exist. The acquisition unit is further configured to, if the distance information is less than a preset threshold, then after the calculation unit determines the predicted driving direction based on the predicted trajectory information and determines the current lane of the target vehicle based on the lane line information, if there is a lane in the same direction as the current lane in the predicted driving direction, then acquire the first historical turning angle information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval. The calculation unit is further configured to determine a difference curve based on the first historical turning angle information and the first historical lane line information, and to determine turning point information based on the difference curve; wherein the turning point information represents the starting information of the driving angle deviating from the lane line; and to determine a first vector, a second vector, and a third vector based on the turning point information and the predicted trajectory information; and to determine that the deviation behavior exists if the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector. The calculation unit is further configured to: determine the first vector, the second vector, and the third vector based on the turning point and the predicted trajectory information; if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, then obtain the second historical turning angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval; calculate at least one difference integral information between the second historical turning angle information and the second historical lane line information based on a preset time interval; determine the difference integral change trend corresponding to the second historical time interval based on the at least one difference integral information; if the difference integral change trend is an increasing trend, then determine that there is no deviation behavior.

6. A terminal, characterized in that, The terminal includes a processor and a memory storing processor-executable instructions, which, when executed by the processor, implement the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a program that is applied to a terminal, and when the program is executed by a processor, it implements the method as described in any one of claims 1-4.

Images