Lane intelligent keeping method, device and equipment, and storage medium

Abstract

The application discloses a lane intelligent keeping method and device, equipment and storage medium, relates to the technical field of automobile control, and comprises the following steps: identifying a vehicle road image to obtain a lateral offset distance and a heading angle of a target vehicle relative to a lane center; predicting the lateral offset distance, the heading angle and real-time motion parameters to obtain a future motion trajectory, and judging whether the target vehicle has a lane deviation trend according to the future motion trajectory to obtain a judgment result; when the judgment result is that the target vehicle has the lane deviation trend, constructing a target cost function according to the future motion trajectory and the real-time motion parameters, solving the target cost function according to a preset constraint condition, obtaining a target auxiliary torque and generating a correction control instruction, and controlling the target vehicle to maintain the lane unchanged, so that the technical effect that the application time of the correction torque is matched with the actual deviation trend of the vehicle under complex driving conditions is realized.

Description

Technical Field

[0001] This application relates to the field of automotive control technology, and in particular to a lane-keeping intelligent method, device, equipment, and storage medium. Background Technology

[0002] With the continuous expansion of highway networks and the increase in vehicle mileage, the demand for lane keeping in high-speed driving scenarios is becoming increasingly prominent. The system is required to be able to stably identify lane boundaries under various weather, lighting and road conditions, and adjust the lane keeping strategy in real time according to the dynamic changes of the vehicle to ensure driving safety.

[0003] Existing lane keeping systems generally use a monocular camera as the sole source of perception. The reliability of their lane line recognition results decreases significantly in rainy, snowy, or foggy weather, sudden changes in lighting at tunnel entrances and exits, or when lane lines are worn or dirty, leading to frequent system failures or misjudgments. At the same time, the intervention decisions of traditional systems rely solely on the current lateral distance as a single indicator, which can easily result in intervention lag or over-intervention in complex road sections such as curves. Furthermore, the system does not consider the driver's operating status when performing corrections, and continues to apply correction torque even when the driver intends to change lanes, causing conflicts between human and machine operation. Summary of the Invention

[0004] The main objective of this application is to provide a lane keeping intelligent method, device, equipment, and storage medium, which aims to solve the technical problem of how to ensure the accuracy of lane keeping in complex environments.

[0005] To achieve the above objectives, this application proposes a lane-keeping intelligent method, the method comprising: Acquire road images and real-time motion parameters of the target vehicle; The road image is identified to obtain the lateral offset distance and heading angle of the target vehicle relative to the center of the lane; The future trajectory is predicted based on the lateral offset distance, the heading angle, and the real-time motion parameters. Based on the future trajectory, it is determined whether the target vehicle has a tendency to deviate from the lane, and a judgment result is obtained. When the judgment result indicates that the target vehicle has a tendency to deviate from the lane, a target cost function is constructed based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters. The target cost function is then solved according to preset constraints to obtain the target auxiliary torque. Based on the target auxiliary torque, generate a correction control command; According to the correction control command, the target vehicle is controlled to maintain its lane position.

[0006] In one embodiment, the step of identifying the road image to obtain the lateral offset distance and heading angle of the target vehicle relative to the lane center includes: The road image is preprocessed to obtain a preprocessed image; The preprocessed image is input into the lane line recognition model, and the curve equation of the lane line output by the lane line recognition model is obtained. Based on the curve equation, the lane center is obtained; Based on the center of the lane line, the lateral offset distance and heading angle of the target vehicle relative to the center of the lane are obtained.

[0007] In one embodiment, the process of training a lane line recognition model includes: Obtain road image samples and annotate the lane lines in the road image samples; The labeled road image samples are used as input parameters and the labeled lane lines are used as output parameters. The convolutional network model is trained based on the input parameters and the output parameters; The trained convolutional network model is used as a lane line recognition model.

[0008] In one embodiment, determining whether the target vehicle has a tendency to deviate from the lane based on the future motion trajectory, and obtaining the determination result, includes: Obtain lane curvature; Based on the future trajectory, the lane crossing time of the target vehicle reaching the lane line boundary is obtained; The current vehicle speed is obtained based on the real-time motion parameters. An interference threshold is obtained based on the current vehicle speed and the lane curvature, wherein the interference threshold includes an intervention threshold and an emergency threshold, and the intervention threshold is greater than the emergency threshold; Based on the target vehicle's turn signal, active driving steering wheel torque, and driver image, determine whether the driver intends to change lanes; When the lane crossing time is less than or equal to the interference threshold and it is determined that the driver does not intend to change lanes, the tendency of the target vehicle to deviate from the lane is taken as the judgment result.

[0009] In one embodiment, determining whether the driver intends to change lanes based on the turn signal signal of the target vehicle, the active driving steering wheel torque, and the driver image includes: The system acquires the turn signal signals of the target vehicle and the driver image captured by the built-in camera. Face detection is performed on the driver image to obtain the gaze direction, eyelid opening, and head posture; The fatigue index is determined based on the direction of gaze, the eyelid opening, and the head posture. When the turn signal is off, the active driving steering wheel torque is less than a preset torque value, and the fatigue index is less than a preset threshold, it is determined that the driver does not intend to change lanes. When the duration of the active driving steering wheel torque being greater than or equal to a preset torque value is greater than or equal to a preset duration, or when the turn signal is on, it is determined that the driver intends to change lanes.

[0010] In one embodiment, after determining whether the driver intends to change lanes based on the turn signal signal of the target vehicle, the active driving steering wheel torque, and the driver image, the method further includes: Acquire the steering wheel pressure signal of the target vehicle; A warning threshold is obtained based on the current vehicle speed and the lane curvature, wherein the warning threshold is greater than the intervention threshold; When the lane crossing time is less than or equal to the warning threshold, the lane crossing time is greater than or equal to the interference threshold, the turn signal is off and the steering wheel pressure signal is active, control the steering wheel and instrument panel of the target vehicle to issue a warning. When the lane crossing time is less than or equal to the emergency threshold and it is determined that the driver does not intend to change lanes, the deflection side is determined based on the future trajectory, and a preset braking force is applied to the front wheel of the deflection side to straighten the target vehicle.

[0011] In one embodiment, when the determination result indicates that the target vehicle has a tendency to deviate from the lane, a target cost function is constructed based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters, and the target cost function is solved according to preset constraints to obtain the target auxiliary torque, including: When the judgment result indicates that the vehicle has a tendency to deviate from the lane, the steering wheel assist torque is obtained based on the real-time motion parameters; Based on the future trajectory, the lateral offset distance, the heading angle, and the steering wheel auxiliary torque, a discretized vehicle dynamics equation is established. Based on the discretized vehicle dynamics equations, a target cost function is constructed, wherein the target cost function includes a tracking error penalty term, a control quantity penalty term, and a control change rate penalty term; Within a preset control time domain, the target cost function is optimized and solved according to preset constraints to obtain an auxiliary torque sequence, wherein the preset constraints include control magnitude constraints, control rate of change constraints, and position constraints. The target auxiliary torque is obtained based on the auxiliary torque sequence.

[0012] Furthermore, to achieve the above objectives, this application also proposes a lane keeping intelligent device, which includes: The input module is used to acquire road images and real-time motion parameters of the target vehicle; The recognition module is used to recognize the road image and obtain the lateral offset distance and heading angle of the target vehicle relative to the center of the lane; The judgment module is used to predict the future trajectory based on the lateral offset distance, the heading angle and the real-time motion parameters, and to judge whether the target vehicle has a tendency to deviate from the lane based on the future trajectory, and to obtain the judgment result. The solution module is used to construct a target cost function based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters when the judgment result indicates that the target vehicle has a tendency to deviate from the lane, and to solve the target cost function according to preset constraints to obtain the target auxiliary torque. The generation module is used to generate correction control commands based on the target auxiliary torque; The output module is used to control the target vehicle to maintain its lane position according to the correction control command.

[0013] In addition, to achieve the above objectives, this application also proposes a lane keeping intelligent device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the lane keeping intelligent method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the lane-keeping intelligent method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the lane-keeping intelligent method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: The system acquires road images and real-time motion parameters of the target vehicle; identifies the road images to obtain the lateral offset distance and heading angle of the target vehicle relative to the lane center; predicts the future trajectory based on the lateral offset distance, heading angle, and real-time motion parameters, and determines whether the target vehicle has a tendency to deviate from the lane based on the future trajectory, obtaining a judgment result; when the judgment result indicates that the target vehicle has a tendency to deviate from the lane, constructs a target cost function based on the future trajectory, lateral offset distance, heading angle, and real-time motion parameters, and solves the target cost function according to preset constraints to obtain a target auxiliary torque; generates a correction control command based on the target auxiliary torque; and controls the target vehicle to maintain its lane position based on the correction control command. By using a torque calculation method based on future trajectory prediction and constrained optimization to generate correction commands, the correction control commands can reflect the dynamic change trend of the vehicle in the prediction time domain rather than relying solely on the current instantaneous state. This solves the technical problem of existing systems making decisions based solely on the current lateral distance, which leads to delayed or excessive intervention. It achieves the technical effect of matching the application time of the correction torque with the actual deviation trend of the vehicle under complex driving conditions. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the lane-keeping intelligent method of this application. Figure 2 A schematic diagram illustrating the implementation process of the model prediction controller provided in Embodiment 1 of the lane-keeping intelligent method of this application; Figure 3 This is a schematic diagram of the lane keeping system architecture provided in Embodiment 1 of the lane keeping intelligent method of this application; Figure 4 This is a flowchart illustrating Embodiment 2 of the lane-keeping intelligent method of this application; Figure 5 This is a schematic diagram of the lane keeping system execution flow provided in Embodiment 2 of the lane keeping intelligent method of this application; Figure 6This is a schematic diagram of the module structure of the lane keeping intelligent device according to an embodiment of this application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the lane-keeping intelligent method in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is as follows: acquiring road images and real-time motion parameters of the target vehicle; identifying the road images to obtain the lateral offset distance and heading angle of the target vehicle relative to the center of the lane; predicting the future trajectory based on the lateral offset distance, the heading angle, and the real-time motion parameters, and determining whether the target vehicle has a tendency to deviate from the lane based on the future trajectory, obtaining a judgment result; when the judgment result indicates that the target vehicle has a tendency to deviate from the lane, constructing a target cost function based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters, and solving the target cost function according to preset constraints to obtain a target auxiliary torque; generating a correction control command based on the target auxiliary torque; and controlling the target vehicle to maintain its lane position according to the correction control command.

[0024] In this embodiment, for ease of description, the vehicle-mounted system will be used as the execution subject in the following description.

[0025] Because existing lane keeping systems generally use a monocular camera as the sole source of perception, the reliability of their lane line recognition results decreases significantly in scenarios such as rain, snow, fog, sudden changes in lighting at tunnel entrances and exits, or lane line wear and tear, leading to frequent system failures or misjudgments. At the same time, the intervention decisions of traditional systems rely solely on the current lateral distance as a single indicator, which can easily result in intervention lag or over-intervention in complex road sections such as curves. Furthermore, the system does not consider the driver's operating status when performing corrections, and continues to apply correction torque even when the driver intends to change lanes, causing conflicts between human and machine operation.

[0026] This application provides a solution that acquires road images and real-time motion parameters of a target vehicle, identifies the lateral offset distance and heading angle from the road images, predicts the future trajectory based on the lateral offset distance, heading angle, and real-time motion parameters to determine the deviation trend, constructs a target cost function based on the future trajectory and real-time motion parameters when a deviation trend exists, and solves for the target auxiliary torque based on preset constraints, and finally generates a correction control command based on the target auxiliary torque to control the target vehicle to maintain lane stability. Because the correction command is generated using a torque calculation method based on future trajectory prediction and constrained optimization, the correction control command reflects the dynamic change trend of the vehicle in the prediction time domain rather than relying solely on the current instantaneous state. This solves the technical problem of existing systems making decisions based solely on the current lateral distance, leading to delayed or excessive intervention, and achieves the technical effect of matching the application time of the correction torque with the actual deviation trend of the vehicle under complex driving conditions.

[0027] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or in-vehicle system capable of performing the above functions. The following description uses an in-vehicle system as an example to illustrate this embodiment and the subsequent embodiments.

[0028] Based on this, embodiments of this application provide a lane-keeping intelligent method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the lane keeping intelligent method of this application.

[0029] In this embodiment, the lane keeping intelligent method includes steps S10 to S60: Step S10: Obtain road images and real-time motion parameters of the target vehicle; It should be noted that road images refer to image data of the road in front of the vehicle collected by a forward-facing camera installed inside the windshield of the target vehicle, while real-time motion parameters refer to vehicle speed, yaw rate, lateral acceleration, steering wheel angle, and steering wheel torque signals obtained through the vehicle's Controller Area Network (CAN) bus.

[0030] Specifically, the forward-facing camera continuously collects road images at a preset sampling frequency. The vehicle speed sensor, yaw rate sensor, and steering angle sensor in the vehicle status sensor group send real-time motion parameters to the CAN bus at a sampling frequency of not less than 100 Hz. The domain controller obtains the above real-time motion parameters through the CAN communication link and synchronizes the road images and real-time motion parameters on the timestamp.

[0031] It is understandable that the reliability of a single camera decreases in adverse weather or drastic lighting conditions, and the vehicle's motion state has a direct impact on the deviation trend. Therefore, performing step S10 to acquire multi-source data can avoid insufficient perception caused by relying solely on image information, thereby improving the data completeness of the system in complex environments.

[0032] Step S20: Identify the road image to obtain the lateral offset distance and heading angle of the target vehicle relative to the center of the lane; It should be noted that the lateral offset distance refers to the lateral distance between the target vehicle's current position and the lane centerline, and the heading angle refers to the angle between the target vehicle's current direction of travel and the direction of the lane tangent.

[0033] For example, after receiving a road image from the forward-facing camera module, the image recognition unit in the domain controller first performs preprocessing operations on the road image: converting the original color image to a grayscale image to reduce computational load, using a median filtering algorithm to eliminate salt-and-pepper noise and Gaussian noise introduced during image acquisition to smooth image edges, and then using inverse perspective transformation to map the road image in the image coordinate system to a top-view image in the vehicle coordinate system, thereby eliminating the distortion effect of perspective on the geometry of lane lines. After preprocessing, the image recognition unit divides the top-view image into multiple grid regions, performs gradient calculation on the pixels in each grid region to extract edge features, and uses the coordinates of the edge pixels as input to the lane line recognition model.

[0034] Furthermore, after receiving the edge feature data of the pre-processed image, the pre-trained lane recognition model performs downsampling and convolution operations on the input data through the encoder structure to extract high-level semantic features of the lane lines. Then, the decoder structure upsamples these high-level semantic features step-by-step to restore the original image size, ultimately outputting the curve equations of the left and right lane lines in the vehicle coordinate system. These curve equations are in quadratic polynomial form. After obtaining the curve equations of the left and right lane lines, the image recognition unit calculates the lateral coordinates of the left and right lane lines at the vehicle's current position, taking their average as the lateral coordinate of the lane centerline. The difference between the lateral coordinates of the lane centerline and the lateral coordinates of the vehicle's own center is taken as the lateral offset distance. Simultaneously, the first derivative of the lane centerline equation at the current vehicle position is calculated to obtain the lane tangent direction angle, and the difference between this direction angle and the vehicle's current driving direction angle is taken as the heading angle. Thus, the image recognition unit completes the entire recognition process from road image input to lateral offset distance and heading angle output, providing the vehicle's pose information relative to the lane center for subsequent steps.

[0035] In one feasible implementation, step S20 may include: preprocessing the road image to obtain a preprocessed image; inputting the preprocessed image into a lane line recognition model and obtaining the curve equation of the lane line output by the lane line recognition model; obtaining the lane line center based on the curve equation; and obtaining the lateral offset distance and heading angle of the target vehicle relative to the lane center based on the lane line center.

[0036] It should be noted that preprocessing refers to sequentially performing grayscale conversion, filtering and denoising, and inverse perspective transformation on the road image. The lane line recognition model is a model based on a convolutional neural network used to extract lane line features from the road image. The curve equation is a mathematical expression describing the geometric shape of the lane line in the vehicle coordinate system. The lane line center refers to the geometric center position of the left and right lane lines.

[0037] Specifically, the road image is converted from a color image to a grayscale image through grayscale conversion, and noise points in the image are eliminated through filtering and denoising. The image coordinates are converted to vehicle coordinates in a top view through inverse perspective transformation. After the preprocessed image is input into the lane line recognition model, the model extracts and classifies the features of the lane line pixels in the image, and outputs the curve equations of the left and right lane lines in the vehicle coordinate system. Based on the curve equations of the left and right lane lines, the lateral midpoint of the two curves at the current position of the vehicle is calculated as the lane line center. The lateral coordinates of the lane line center are compared with the lateral coordinates of the vehicle's own center to obtain the lateral offset distance. The heading angle is obtained by comparing the current driving direction of the vehicle with the tangent direction of the lane line centerline.

[0038] Furthermore, the process of training the lane line recognition model includes: acquiring road image samples and labeling the lane lines in the road image samples; using the labeled road image samples as input parameters and the labeled lane lines as output parameters; training the convolutional network model based on the input parameters and the output parameters; and using the trained convolutional network model as the lane line recognition model.

[0039] It should be noted that road image samples refer to a collection of road images collected under different weather conditions, lighting conditions, and road conditions. Annotation refers to manually marking the pixel positions of the left and right lane lines in the road image samples. The convolutional neural network model includes an input layer, a convolutional layer, a pooling layer, and an output layer.

[0040] Specifically, road images including those from sunny days, rainy days, snowy days, foggy days, backlighting, nighttime, tunnel entrances and exits, and lane line wear and dirt scenes are collected as a sample set. The positions of the left and right lane lines in each frame of the sample set are labeled at the pixel level. The labeled image samples are input into an initialized convolutional neural network. The network receives image data through the input layer and extracts features such as the edge, direction, and position of the lane lines layer by layer through multiple convolutional and pooling layers. The predicted position of the lane lines is output through the output layer. The predicted position is compared with the labeled position and the network parameters are adjusted by backpropagation. The above process is repeated until the difference between the predicted position and the labeled position output by the network converges to a preset range. The trained network is used as the lane line recognition model.

[0041] In this embodiment, a convolutional neural network is used to identify lane lines in road images, which solves the problem of low recognition rate of traditional image processing algorithms in scenarios with worn, dirty, and changing lighting conditions.

[0042] The above are merely feasible implementations of step S20 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S20.

[0043] Step S30: Based on the lateral offset distance, the heading angle, and the real-time motion parameters, a future trajectory is predicted, and based on the future trajectory, it is determined whether the target vehicle has a tendency to deviate from the lane, and a judgment result is obtained. It should be noted that the future trajectory refers to the predicted path of the target vehicle along the direction of travel within a preset time period.

[0044] Specifically, the lateral offset distance and heading angle obtained in step S20, as well as the vehicle speed and yaw rate from the real-time motion parameters obtained in step S10, are used as inputs to the vehicle dynamics model. The dynamics model performs recursive calculations based on the current vehicle state and outputs the predicted lateral position and predicted heading angle of the vehicle at multiple future moments. These predicted positions are connected in chronological order to form the future motion trajectory. The predicted lateral position in the future motion trajectory is compared with the lateral position of the lane boundary line. If the predicted lateral position exceeds the lane boundary line, it is determined that there is a deviation trend. If the predicted lateral position does not exceed the lane boundary line, it is determined that there is no deviation trend.

[0045] It is understandable that since a vehicle deviating from its lane at high speed is a dynamic process, being in the lane at the current moment does not mean that it will not deviate in the future. Therefore, performing step S30 to predict the future trajectory can avoid the intervention lag caused by relying solely on the current lateral position, thereby improving the timeliness of deviation judgment.

[0046] Step S40: When the judgment result indicates that the target vehicle has a tendency to deviate from the lane, a target cost function is constructed based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters. The target cost function is then solved according to preset constraints to obtain the target auxiliary torque. It should be noted that the target cost function is an evaluation function used to quantify the control effect, and the target auxiliary torque refers to the torque value applied to the steering wheel through the electric power steering system to assist the vehicle in returning to the lane.

[0047] In one feasible implementation, step S40 may include: when the determination result indicates that the vehicle has a tendency to deviate from the lane, obtaining the steering wheel assist torque based on the real-time motion parameters; establishing a discretized vehicle dynamics equation based on the future motion trajectory, the lateral offset distance, the heading angle, and the steering wheel assist torque; constructing a target cost function based on the discretized vehicle dynamics equation, wherein the target cost function includes a tracking error penalty term, a control quantity penalty term, and a control change rate penalty term; optimizing and solving the target cost function within a preset control time domain according to preset constraints to obtain an assist torque sequence, wherein the preset constraints include control quantity amplitude constraints, control change rate constraints, and position constraints; and obtaining the target assist torque based on the assist torque sequence.

[0048] It should be noted that the steering wheel assist torque refers to the assist torque sequence corresponding to the current moment, obtained by establishing and solving the cost function. The discretized vehicle dynamics equation is to convert the continuous-time vehicle lateral motion and yaw motion equations into discrete-time equations according to the sampling period. The tracking error penalty term is a term that quantifies the degree of vehicle deviation from the lane centerline. The control quantity penalty term is a term that quantifies the magnitude of the applied control torque. The control change rate penalty term is a term that quantifies the change amplitude of control torque between adjacent moments. The control quantity amplitude constraint is a boundary condition that limits the maximum and minimum values ​​of control torque. The control change rate constraint is a boundary condition that limits the change in control torque between adjacent moments. The position constraint is a boundary condition that limits the lateral position of the vehicle from exceeding the lane boundary. The assist torque sequence refers to the set of control torque values ​​at each moment within the preset control time domain.

[0049] Specifically, such as Figure 2As shown, based on the lateral offset distance and heading angle at different times, the rate of change of the lateral offset distance and the rate of change of the heading angle at different times are obtained. The lateral offset distance and heading angle at different times, the rate of change of the lateral offset distance at different times and the rate of change of the heading angle at different times are spliced ​​together. The spliced ​​vehicle state data and steering wheel auxiliary torque are used as inputs to the vehicle dynamics model. The vehicle dynamics model is discretized in the prediction time domain according to the sampling period to obtain the recursive relationship equation between the lateral position and the steering wheel auxiliary torque at each time.

[0050] Among them, vehicle status data is y includes the lateral offset distance at different times. Including heading angles at different times, Including the rate of change of lateral offset distance at different times, This includes the rate of change of heading angle at different times, where x(k+1) represents the lateral offset distance, heading angle, rate of change of lateral offset distance, and rate of change of heading angle at time k+1, x(k) represents the lateral offset distance, heading angle, rate of change of lateral offset distance, and rate of change of heading angle at time k, A and B are state space matrices determined based on vehicle speed, and u(k) represents the steering wheel auxiliary torque at time k, which can be the target auxiliary torque calculated at time k extracted from the steering wheel auxiliary torque.

[0051] Based on this recursive equation, the objective cost function J is constructed as follows:

[0052] Where N represents the number of time steps in the prediction time domain, which is 10 in this embodiment, and M represents the number of time steps in the control time domain, which is 5 in this embodiment. It is represented by a pre-calibrated weighting coefficient, the specific value of which is usually adjusted according to actual vehicle testing and driving comfort requirements. y(k) represents the reference lateral offset distance at time k extracted from the future trajectory (a reference state sequence for the next N steps), and y(k) represents the actual lateral offset distance at time k obtained by lane recognition. This represents the reference heading angle at time k extracted from the future trajectory. Let u(k) represent the actual heading angle obtained by lane recognition at time k, u(k) represent the steering wheel assist torque at time k, and u(k+1) represent the steering wheel assist torque at time k+1. The corresponding tracking error penalty term is set as the sum of squares of the deviations between the predicted lateral position and the lane centerline; The corresponding control quantity penalty term is set as the sum of the squares of the steering wheel auxiliary torque at each moment; The corresponding control rate of change penalty term is set as the sum of the squares of the differences in steering wheel auxiliary torque between adjacent time points, and the minimum value of the objective cost function is taken as the optimization objective.

[0053] Within a preset control time domain, a control magnitude constraint is set for the steering wheel auxiliary torque to not exceed five Newton-meters, a control rate constraint is set for the steering wheel auxiliary torque to not exceed thirty Newton-meters per second, and a position constraint is set for the vehicle's lateral position to be within the lane boundary. Under the above constraints, a steering wheel auxiliary torque sequence that minimizes the objective cost function is obtained as the auxiliary torque sequence, and the first steering wheel auxiliary torque value is extracted from the auxiliary torque sequence as the target auxiliary torque.

[0054] In this embodiment, by employing model predictive control to perform constrained optimization of the objective cost function, which includes tracking error penalty, control quantity penalty, and control change rate penalty, the problem of torque abrupt change caused by the existing system directly calculating the correction torque based solely on the current lateral distance is solved.

[0055] The above are merely feasible implementations of step S40 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S30.

[0056] Step S50: Generate a correction control command based on the target auxiliary torque.

[0057] It should be noted that the correction control command refers to the control command that includes the target auxiliary torque value and its execution timestamp.

[0058] Specifically, the decision control unit in the domain controller encapsulates the target auxiliary torque value obtained in step S40 into a data frame format that conforms to the communication protocol of the electric power steering system, writes the target auxiliary torque value and the current timestamp into the data frame, and sends the data frame to the electric power steering system via the CAN bus.

[0059] Understandably, since the target auxiliary torque is a calculated result in numerical form, the electric power steering system cannot directly recognize this value. Therefore, step S50 converts the target auxiliary torque into a control command that conforms to the communication protocol, which can avoid execution failure caused by the actuator's inability to parse the control command, thereby improving the system's executability.

[0060] Step S60: According to the correction control command, control the target vehicle to maintain its lane position.

[0061] It should be noted that controlling the target vehicle to maintain its lane position means using the actuator to output physical torque to return the vehicle to the center line of the lane and keep it driving near that center line.

[0062] Specifically, the motor controller of the electric power steering system receives the correction control command generated in step S50, parses the target auxiliary torque value from the data frame, and drives the motor rotor to generate the corresponding electromagnetic torque according to the torque value. The electromagnetic torque is amplified by the reduction mechanism and applied to the steering column, forming an auxiliary torque on the steering wheel. The auxiliary torque is superimposed with the steering torque applied by the driver and drives the front wheels to rotate, causing the vehicle's driving direction to deflect and return to the center line of the lane.

[0063] It is understandable that, since the motor of the electric power steering system requires a specific torque drive signal to generate physical torque, step S60 converts the lane correction control command into the motor's drive current. This can prevent the control command from remaining at the data level and causing the vehicle to be unable to perform lane correction, thereby improving the effectiveness of lane keeping.

[0064] Combination such as Figure 3 The system architecture diagram shown illustrates that the overall hardware signal flow begins at the multi-source information sensing unit. The forward-facing camera module transmits the acquired road images to the information fusion and decision-making unit in the domain controller via interface 1. Simultaneously, vehicle status information is synchronously input to this unit via the CAN bus through interface 2. The driver's facial image acquired by the in-vehicle camera module is also fed into the information fusion and decision-making unit via interface 1. After the above multi-source data is timestamped and synchronized within the domain controller, the acquisition of the target vehicle's road image and real-time motion parameters is complete, providing a data foundation for subsequent image recognition.

[0065] The information fusion and decision unit in the domain controller processes the road image transmitted from the forward-facing camera module. Using an internal image recognition algorithm, it extracts the curve equations of the left and right lane lines from the road image, and then calculates the lateral offset distance and heading angle of the target vehicle relative to the lane center. Based on this lateral offset distance and heading angle, the information fusion and decision unit combines real-time motion parameters obtained from the CAN bus to perform recursive prediction, generating a sequence of predicted lateral positions for multiple future moments as the future trajectory. The predicted lateral positions in this future trajectory are compared with the lane boundary positions to determine if the target vehicle is trending towards lane departure. Simultaneously, fatigue indicators obtained from driver images captured by the in-vehicle camera module after face detection, along with turn signal and steering wheel torque signals from vehicle status information, are used to determine if the driver intends to change lanes. This intention recognition result and the deviation trend judgment result are comprehensively evaluated within the domain controller.

[0066] When the comprehensive judgment indicates that the target vehicle is trending out of its lane and the driver has no intention of changing lanes, the information fusion and decision-making unit constructs a target cost function based on the predicted lateral position sequence, heading angle sequence, and real-time motion parameters in the future trajectory. This function includes a tracking error penalty term, a control quantity penalty term, and a control change rate penalty term. Within a preset control time domain, the unit optimizes and solves this target cost function according to the control quantity amplitude constraint and the control change rate constraint to minimize the target cost function, thereby obtaining the steering wheel assist torque as the target assist torque. This target assist torque is encapsulated into a data frame conforming to the communication protocol and sent via the CAN bus through interface 2 to the electric power steering (EPS) controller in the vehicle's human-machine collaborative execution unit. The EPS controller drives the motor to generate the corresponding assist torque and applies it to the steering wheel. Simultaneously, depending on the degree of deviation, it determines whether to send a braking command to the hydraulic unit of the electronic stability control (ESC) system to apply braking force to the front wheels on the deviating side, ultimately controlling the target vehicle to maintain its lane position. Throughout the process, the dashboard display and warning module in the human-computer interaction component synchronously output visual and auditory warning information according to the instructions of the information fusion and decision-making unit, forming a complete closed loop of perception, decision-making, and execution.

[0067] This embodiment provides a lane-keeping intelligent method, which acquires a road image and real-time motion parameters of a target vehicle; identifies the road image to obtain the lateral offset distance and heading angle of the target vehicle relative to the lane center; predicts the future trajectory based on the lateral offset distance, heading angle, and real-time motion parameters, and determines whether the target vehicle has a tendency to deviate from the lane based on the future trajectory, obtaining a judgment result; when the judgment result indicates that the target vehicle has a tendency to deviate from the lane, constructs a target cost function based on the future trajectory, lateral offset distance, heading angle, and real-time motion parameters, and solves the target cost function according to preset constraints to obtain a target auxiliary torque; generates a correction control command based on the target auxiliary torque; and controls the target vehicle to maintain lane stability based on the correction control command. By using a torque calculation method based on future trajectory prediction and constrained optimization to generate correction commands, the correction control commands can reflect the dynamic change trend of the vehicle in the prediction time domain rather than relying solely on the current instantaneous state. This solves the technical problem of existing systems making decisions based solely on the current lateral distance, which leads to delayed or excessive intervention. It achieves the technical effect of matching the application time of the correction torque with the actual deviation trend of the vehicle under complex driving conditions.

[0068] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 The lane keeping intelligent method further includes steps S31 to S36 in step S30: Step S31, obtain the lane curvature; It should be noted that lane curvature refers to the degree of curvature of the lane centerline in the horizontal plane.

[0069] Specifically, the quadratic coefficients of the curves are extracted from the curve equations of the left and right lane lines output by the lane line recognition model in step S20, and the curvature value of the lane centerline at the current vehicle position is calculated based on the quadratic coefficients.

[0070] It is understandable that, since the lane lines are curved when a vehicle is driving on a curve, different curvatures of curves have different effects on the rate of change of the vehicle's lateral position. Therefore, obtaining the lane curvature in step S31 can avoid misjudgment caused by using the straight road threshold in a curve scenario, thereby improving the accuracy of judgment in a curve scenario.

[0071] Step S32: Based on the future motion trajectory, obtain the lane crossing time when the target vehicle reaches the lane line boundary; It should be noted that Lane Crossing Time (TLC) refers to the estimated time required for a vehicle to reach the lane boundary line in its current motion state.

[0072] Specifically, the predicted lateral position at each moment in the future motion trajectory generated in step S30 is compared with the lateral position of the lane boundary line to determine the moment when the predicted lateral position first reaches the lane boundary line position. The time difference between this moment and the current moment is calculated, and this time difference is used as the lane crossing time.

[0073] Understandably, since lane crossing time directly reflects the urgency of the deviation, step S32, which quantifies the trajectory prediction result as a time indicator, avoids the problem that directly using lateral distance cannot distinguish between rapid and slow deviations, thereby improving the accuracy of deviation risk quantification.

[0074] Step S33: Obtain the current vehicle speed based on the real-time motion parameters.

[0075] It should be noted that the current speed refers to the longitudinal speed of the target vehicle at the current moment.

[0076] Specifically, the longitudinal speed value of the vehicle collected by the vehicle speed sensor is extracted from the real-time motion parameters obtained in step S10, and this value is used as the current vehicle speed.

[0077] It is understandable that, since the time required for a vehicle to complete the same lateral displacement varies at different speeds, and the available reaction time is shorter at high speeds, obtaining the current vehicle speed in step S33 can avoid the delay in intervention in high-speed scenarios caused by ignoring the influence of vehicle speed, thereby improving safety in high-speed scenarios.

[0078] Step S34: Based on the current vehicle speed and the lane curvature, an interference threshold is obtained, wherein the interference threshold includes an intervention threshold and an emergency threshold, and the intervention threshold is greater than the emergency threshold.

[0079] It should be noted that the intervention threshold includes the intervention threshold and the emergency threshold. The intervention threshold is the critical value that triggers an early warning or gentle intervention, while the emergency threshold is the critical value that triggers a strong intervention.

[0080] Specifically, the current vehicle speed obtained in step S33 is used as input, and the basic threshold is calculated according to the rule that the higher the vehicle speed, the smaller the threshold. The lane curvature obtained in step S31 is used as input, and the basic threshold is corrected according to the rule that the greater the curvature, the larger the threshold. The larger value among the corrected basic thresholds is used as the intervention threshold, and the smaller value is used as the emergency threshold.

[0081] It is understandable that, since the vehicle's reaction time is shortened when driving at high speeds, earlier intervention is needed, and when driving on curves, there is lateral deviation inherent in normal cornering, so the triggering conditions need to be appropriately relaxed. Therefore, performing step S34 to adjust the threshold based on vehicle speed and curvature can avoid the fixed threshold intervening too late at high speeds and triggering intervention erroneously on curves, thereby improving the scenario adaptability of the threshold.

[0082] Step S35: Determine whether the driver intends to change lanes based on the turn signal signal of the target vehicle, the active driving steering wheel torque, and the driver image.

[0083] It should be noted that the turn signal refers to the turn signal switch status signal of the target vehicle, the active driving steering wheel torque refers to the operating torque actively applied to the steering wheel by the driver, and the driver image refers to the facial image of the driver captured by the built-in camera facing the driver.

[0084] In one feasible implementation, step S35 may include: acquiring the turn signal of the target vehicle and the driver image captured by the built-in camera; performing face detection on the driver image to obtain the gaze direction, eyelid opening, and head posture; determining a fatigue index based on the gaze direction, eyelid opening, and head posture; determining that the driver does not intend to change lanes when the turn signal is off, the active driving steering wheel torque is less than a preset torque value, and the fatigue index is less than a preset threshold; and determining that the driver intends to change lanes when the active driving steering wheel torque is greater than or equal to the preset torque value for a duration greater than or equal to a preset duration or when the turn signal is on.

[0085] It should be noted that gaze direction refers to the direction of the driver's pupils, eyelid opening refers to the degree of opening and closing between the driver's upper and lower eyelids, head posture refers to the angle of deflection of the driver's head in three-dimensional space, and fatigue index is an indicator used to quantify the degree of driver fatigue, calculated based on eyelid opening and gaze direction.

[0086] Specifically, the built-in camera captures facial images of the driver at a preset sampling frequency. Face detection is performed on the captured images to determine the facial region. Key eye points and head feature points are located within the facial region. The direction of gaze is determined based on the relative offset between the pupil position and the center of the eye region. The eyelid opening is determined based on the distance between key points on the upper and lower eyelid edges. The head posture is determined based on the spatial changes of head feature points. The proportion of eye closure time to the total duration of the eyelid opening sequence within a preset time period is calculated as a fatigue indicator. When the turn signal is off, the active driving steering wheel torque is less than a preset torque value, and the fatigue indicator is less than a preset threshold, it is determined that the driver's intention to maintain lane keeping is clear, and there is no intention to change lanes. When the active driving steering wheel torque is greater than or equal to a preset torque value for a preset duration, it is determined that the driver is actively turning the steering wheel to prepare for a lane change. Alternatively, when the turn signal is on, it is determined that the driver has issued a lane change signal. If any of the above conditions are met, it is determined that the driver has an intention to change lanes.

[0087] In this embodiment, lane change intention recognition is achieved by integrating turn signal signals, steering wheel torque, and driver visual attention status, thus solving the problem of low accuracy in judging driver intentions by relying on only a single signal.

[0088] In another feasible implementation, step S35 may include: acquiring the steering wheel pressure signal of the target vehicle; obtaining a warning threshold based on the current vehicle speed and the lane curvature, wherein the warning threshold is greater than the intervention threshold; controlling the steering wheel and instrument panel of the target vehicle to issue a warning when the lane crossing time is less than or equal to the warning threshold, the lane crossing time is greater than or equal to the interference threshold, the turn signal is off and the steering wheel pressure signal is active; when the lane crossing time is less than or equal to the emergency threshold and it is determined that the driver does not intend to change lanes, determining the deviating side based on the future trajectory, and applying a preset braking force to the front wheel of the deviating side to straighten the target vehicle.

[0089] It should be noted that the steering wheel pressure signal refers to the signal output by the capacitive grip force detection sensor integrated into the steering wheel rim, which indicates whether the driver's hands are in contact with the steering wheel. The warning threshold is the critical value that triggers the warning and its value is greater than the intervention threshold. The deviation side refers to the predicted trajectory that will deviate from the center line of the lane to the side it is facing.

[0090] Specifically, the capacitive grip force sensor continuously detects the capacitance change at the steering wheel rim. When the capacitance change exceeds a preset threshold when the driver grips the steering wheel, the steering wheel pressure signal is activated. The current vehicle speed and lane curvature obtained in step S34 are used to calculate the warning threshold according to the rule that the warning threshold is greater than the intervention threshold, i.e., the warning threshold is greater than both the intervention threshold and the emergency threshold. When all four conditions are met simultaneously—lane crossing time less than or equal to the warning threshold, lane crossing time greater than or equal to the intervention threshold, turn signal off, and steering wheel pressure signal activated—the domain controller... The controller sends a flashing icon command to the instrument panel display, a buzzer command to the audible and visual alarm, and a micro-vibration command to the electric power steering system. This causes the steering wheel to produce a three-part micro-vibration, the instrument panel icon to flash yellow, and the buzzer to emit a beeping alarm every three seconds. When the lane crossing time is less than or equal to the emergency threshold and the driver does not intend to change lanes, the controller determines the deflection side based on the predicted lateral position deviation direction in the future trajectory and sends a braking command to the electronic stability control system. The electronic stability control system applies a progressive small-amplitude braking force to the front wheels on the deflection side, generating a yaw torque to straighten the vehicle.

[0091] In this embodiment, a three-tiered threshold system consisting of a warning threshold, an intervention threshold, and an emergency threshold is established. Warnings or braking are executed based on the threshold range into which the lane crossing time falls. This solves the problem that a single intervention strategy cannot distinguish between the degree of urgency, leading to over-intervention or under-intervention.

[0092] The above are merely feasible implementations of step S35 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S35.

[0093] Step S36: When the lane crossing time is less than or equal to the interference threshold and it is determined that the driver does not intend to change lanes, the tendency of the target vehicle to deviate from the lane is taken as the judgment result.

[0094] It should be noted that the judgment result refers to a Boolean quantity used to indicate whether the target vehicle has a tendency to deviate from the lane.

[0095] Specifically, the lane crossing time obtained in step S32 is compared with the interference threshold obtained in step S34. At the same time, the judgment result of whether the driver intends to change lanes obtained in step S35 is used as a condition. When the lane crossing time is less than or equal to the intervention threshold in the interference threshold and the driver does not intend to change lanes, the judgment result is set to have a tendency to deviate from the lane; otherwise, the judgment result is set to not have a tendency to deviate from the lane.

[0096] Understandably, since a driver's lane departure is an active behavior that does not require system intervention if the driver intends to change lanes, step S36 only determines that there is a deviation trend when both the time condition and the intention condition are met. This can avoid the system intervening incorrectly in the driver's active lane change operation, thereby improving the coordination of human-machine collaboration.

[0097] For example, combined with Figure 5 The diagram illustrates the execution flow of the lane keeping system. The overall decision-making process begins in the standby state after a successful system self-check. In the standby state, the information fusion and decision-making unit continuously acquires lane images from the forward-facing camera module and identifies lane lines, extracting lane curvature from them. Simultaneously, it compares the predicted lateral position at each moment in the generated future trajectory with the lane boundary line position to calculate the lane crossing time for the target vehicle to reach the lane boundary; and extracts the current vehicle speed from the vehicle status information. Based on the current vehicle speed and lane curvature, an intervention threshold including an intervention threshold and an emergency threshold is calculated, where the intervention threshold is greater than the emergency threshold. On this basis, after the state transition condition C1 in the flowchart (lane crossing time less than the warning threshold, no turn signal, and hands on the steering wheel) is triggered, the system transitions from the standby state to the first-level warning state, corresponding to the introduction of the warning threshold and the judgment of the steering wheel pressure signal—that is, when the lane crossing time is less than or equal to the warning threshold, the lane crossing time is greater than or equal to the intervention threshold, the turn signal is off, and the steering wheel pressure signal is active, the system controls the steering wheel to generate a micro-vibration and causes the instrument panel icon to flash to issue a warning.

[0098] When the system is in a Level 1 warning state, if the lane crossing time continues to decrease within 1.5 seconds of the warning being issued and the driver does not take any corrective action, transition conditions C2 and C3 in the flowchart are triggered sequentially, and the system transitions from the Level 1 warning state to a gentle intervention state. During this process, the information fusion and decision-making unit continuously monitors the turn signal, the active driving steering wheel torque, and the driver image captured by the built-in camera. Face detection is performed on the driver image to obtain the gaze direction, eyelid opening, and head posture, thereby determining fatigue indicators. When the turn signal is off, the active driving steering wheel torque is less than a preset torque value, and the fatigue indicator is less than a preset threshold, it is determined that the driver does not intend to change lanes. When the lane crossing time is less than or equal to the intervention threshold and it is determined that the driver does not intend to change lanes, the system determines that the target vehicle has a tendency to deviate from the lane, and then enters a gentle intervention state. If the driver remains unresponsive under gentle intervention and the lane crossing time further decreases to less than or equal to the emergency threshold, the system switches to strong intervention. At this time, the deviation side is determined based on the future trajectory, and a preset braking force is applied to the front wheel on the deviation side to generate a yaw moment to straighten the vehicle. At the same time, the EPS controller continues to output auxiliary torque to achieve combined steering and braking correction.

[0099] During the entire state transition process, condition C4 in the flowchart (turn signal on or driver torque greater than threshold) is used to detect driver intent. Once the turn signal is detected as on or the active driving steering wheel torque is greater than or equal to the preset torque value for a duration greater than or equal to the preset duration, the system determines that the driver intends to change lanes and immediately transitions from the current intervention state to the driver takeover state, stopping all corrective torque outputs and completely transferring control to the driver. If the risk is eliminated and lasts for 2 seconds, the system returns from each intervention state to the standby state to continue monitoring. When a sensor fault or system anomaly is detected, condition C5 is triggered, the system transitions to the system fault state and performs degraded operation, providing only basic warning functions. After the fault is cleared, it is reinitialized and restored to the standby state, thus forming a complete finite state machine decision-making closed loop. The detailed description of the system states is shown in Table 1 below.

[0100] Table 1

[0101] This embodiment provides a lane-keeping intelligent method, which acquires lane curvature; obtains the lane crossing time of the target vehicle to the lane boundary based on the future motion trajectory; obtains the current vehicle speed based on the real-time motion parameters; obtains an interference threshold based on the current vehicle speed and the lane curvature, wherein the interference threshold includes an intervention threshold and an emergency threshold, and the intervention threshold is greater than the emergency threshold; determines whether the driver intends to change lanes based on the target vehicle's turn signal, the active driving steering wheel torque, and the driver's image; when the lane crossing time is less than or equal to the interference threshold and it is determined that the driver does not intend to change lanes, the method judges that the target vehicle has a tendency to deviate from the lane. Because it uses a dual judgment method based on comparing the lane crossing time with the dynamic interference threshold and combining it with the driver's intention, it avoids misjudgments caused by relying solely on the lateral distance as a single indicator. This solves the technical problem that existing systems cannot distinguish between active lane changes and unintentional deviations, thus causing unnecessary intervention, thereby reducing interference with the driver's normal operation while ensuring safety.

[0102] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the lane keeping method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0103] This application also provides a lane keeping assist device; please refer to... Figure 6 The lane keeping assist device includes: Input module 10 is used to acquire road images and real-time motion parameters of the target vehicle; The recognition module 20 is used to recognize the road image and obtain the lateral offset distance and heading angle of the target vehicle relative to the center of the lane; The judgment module 30 is used to predict the future trajectory based on the lateral offset distance, the heading angle and the real-time motion parameters, and to judge whether the target vehicle has a tendency to deviate from the lane based on the future trajectory, and to obtain the judgment result. The solution module 40 is used to construct a target cost function based on the future trajectory, the lateral offset distance, the heading angle and the real-time motion parameters when the judgment result is that the target vehicle has a tendency to deviate from the lane, and to solve the target cost function according to preset constraints to obtain the target auxiliary torque. The generation module 50 is used to generate a correction control command based on the target auxiliary torque; The output module 60 is used to control the target vehicle to maintain its lane position according to the correction control command.

[0104] The lane keeping assist device provided in this application, employing the lane keeping assist method in the above embodiments, can solve the technical problem of ensuring lane keeping accuracy in complex environments. Compared with the prior art, the beneficial effects of the lane keeping assist device provided in this application are the same as those of the lane keeping assist method provided in the above embodiments, and other technical features in the lane keeping assist device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0105] The recognition module 20 is also used to preprocess the road image to obtain a preprocessed image; The preprocessed image is input into the lane line recognition model, and the curve equation of the lane line output by the lane line recognition model is obtained. Based on the curve equation, the lane center is obtained; Based on the center of the lane line, the lateral offset distance and heading angle of the target vehicle relative to the center of the lane are obtained.

[0106] The recognition module 20 is also used to acquire road image samples and mark the lane lines in the road image samples; The labeled road image samples are used as input parameters and the labeled lane lines are used as output parameters. The convolutional network model is trained based on the input parameters and the output parameters; The trained convolutional network model is used as a lane line recognition model.

[0107] The judgment module 30 is also used to obtain the lane curvature; Based on the future trajectory, the lane crossing time of the target vehicle reaching the lane line boundary is obtained; The current vehicle speed is obtained based on the real-time motion parameters. An interference threshold is obtained based on the current vehicle speed and the lane curvature, wherein the interference threshold includes an intervention threshold and an emergency threshold, and the intervention threshold is greater than the emergency threshold; Based on the target vehicle's turn signal, active driving steering wheel torque, and driver image, determine whether the driver intends to change lanes; When the lane crossing time is less than or equal to the interference threshold and it is determined that the driver does not intend to change lanes, the tendency of the target vehicle to deviate from the lane is taken as the judgment result.

[0108] The judgment module 30 is also used to acquire the turn signal of the target vehicle and the driver image captured by the built-in camera; Face detection is performed on the driver image to obtain the gaze direction, eyelid opening, and head posture; The fatigue index is determined based on the direction of gaze, the eyelid opening, and the head posture. When the turn signal is off, the active driving steering wheel torque is less than a preset torque value, and the fatigue index is less than a preset threshold, it is determined that the driver does not intend to change lanes. When the duration of the active driving steering wheel torque being greater than or equal to a preset torque value is greater than or equal to a preset duration, or when the turn signal is on, it is determined that the driver intends to change lanes.

[0109] The judgment module 30 is also used to acquire the steering wheel pressure signal of the target vehicle; A warning threshold is obtained based on the current vehicle speed and the lane curvature, wherein the warning threshold is greater than the intervention threshold; When the lane crossing time is less than or equal to the warning threshold, the lane crossing time is greater than or equal to the interference threshold, the turn signal is off and the steering wheel pressure signal is active, control the steering wheel and instrument panel of the target vehicle to issue a warning. When the lane crossing time is less than or equal to the emergency threshold and it is determined that the driver does not intend to change lanes, the deflection side is determined based on the future trajectory, and a preset braking force is applied to the front wheel of the deflection side to straighten the target vehicle.

[0110] The solving module 40 is also used to obtain the steering wheel auxiliary torque based on the real-time motion parameters when the judgment result indicates that the vehicle has a tendency to deviate from the lane. Based on the future trajectory, the lateral offset distance, the heading angle, and the steering wheel auxiliary torque, a discretized vehicle dynamics equation is established. Based on the discretized vehicle dynamics equations, a target cost function is constructed, wherein the target cost function includes a tracking error penalty term, a control quantity penalty term, and a control change rate penalty term; Within a preset control time domain, the target cost function is optimized and solved according to preset constraints to obtain an auxiliary torque sequence, wherein the preset constraints include control magnitude constraints, control rate of change constraints, and position constraints. The target auxiliary torque is obtained based on the auxiliary torque sequence.

[0111] This application provides a lane keeping intelligent device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the lane keeping intelligent method in the first embodiment described above.

[0112] The following is for reference.Figure 7 The diagram illustrates a structural schematic suitable for implementing the lane keeping intelligent device in the embodiments of this application. The lane keeping intelligent device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), tablets, PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The lane keeping device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0113] like Figure 7 As shown, the lane keeping assist device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the lane keeping assist device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the lane-keeping intelligent device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show lane-keeping intelligent devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0114] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0115] The lane keeping intelligent device provided in this application, employing the lane keeping intelligent method in the above embodiments, can solve the technical problem of ensuring lane keeping accuracy in complex environments. Compared with the prior art, the beneficial effects of the lane keeping intelligent device provided in this application are the same as those of the lane keeping intelligent method provided in the above embodiments, and other technical features of this lane keeping intelligent device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0116] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0117] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0118] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the lane-keeping intelligent method in the above embodiments.

[0119] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0120] The aforementioned computer-readable storage medium may be included in the lane keeping device; or it may exist independently and not be installed in the lane keeping device.

[0121] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the lane-keeping intelligent device, the lane-keeping intelligent device causes the following actions: acquiring a road image and real-time motion parameters of the target vehicle; identifying the road image to obtain the lateral offset distance and heading angle of the target vehicle relative to the lane center; predicting the future trajectory based on the lateral offset distance, the heading angle, and the real-time motion parameters; determining whether the target vehicle has a tendency to deviate from the lane based on the future trajectory; when the determination result indicates that the target vehicle has a tendency to deviate from the lane, constructing a target cost function based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters; solving the target cost function according to preset constraints to obtain a target auxiliary torque; generating a correction control command based on the target auxiliary torque; and controlling the target vehicle to maintain its lane position according to the correction control command.

[0122] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0124] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0125] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described intelligent lane keeping method, thereby solving the technical problem of ensuring lane keeping accuracy in complex environments. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the intelligent lane keeping method provided in the above embodiments, and will not be repeated here.

[0126] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the lane-keeping intelligent method described above.

[0127] The computer program product provided in this application can solve the technical problem of ensuring lane keeping accuracy in complex environments. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the intelligent lane keeping method provided in the above embodiments, and will not be repeated here.

[0128] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A lane-keeping intelligent method, characterized in that, The method includes: Acquire road images and real-time motion parameters of the target vehicle; The road image is identified to obtain the lateral offset distance and heading angle of the target vehicle relative to the center of the lane; The future trajectory is predicted based on the lateral offset distance, the heading angle, and the real-time motion parameters. Based on the future trajectory, it is determined whether the target vehicle has a tendency to deviate from the lane, and a judgment result is obtained. When the judgment result indicates that the target vehicle has a tendency to deviate from the lane, a target cost function is constructed based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters. The target cost function is then solved according to preset constraints to obtain the target auxiliary torque. Based on the target auxiliary torque, generate a correction control command; According to the correction control command, the target vehicle is controlled to maintain its lane position.

2. The method as described in claim 1, characterized in that, The process of identifying the road image to obtain the lateral offset distance and heading angle of the target vehicle relative to the lane center includes: The road image is preprocessed to obtain a preprocessed image; The preprocessed image is input into the lane line recognition model, and the curve equation of the lane line output by the lane line recognition model is obtained. Based on the curve equation, the lane center is obtained; Based on the center of the lane line, the lateral offset distance and heading angle of the target vehicle relative to the center of the lane are obtained.

3. The method as described in claim 2, characterized in that, The process of training a lane line recognition model includes: Obtain road image samples and annotate the lane lines in the road image samples; The labeled road image samples are used as input parameters and the labeled lane lines are used as output parameters. The convolutional network model is trained based on the input parameters and the output parameters; The trained convolutional network model is used as a lane line recognition model.

4. The method as described in claim 1, characterized in that, Based on the future trajectory, determine whether the target vehicle has a tendency to deviate from the lane, and obtain the determination result, including: Obtain lane curvature; Based on the future trajectory, the lane crossing time of the target vehicle reaching the lane line boundary is obtained; The current vehicle speed is obtained based on the real-time motion parameters. An interference threshold is obtained based on the current vehicle speed and the lane curvature, wherein the interference threshold includes an intervention threshold and an emergency threshold, and the intervention threshold is greater than the emergency threshold; Based on the target vehicle's turn signal, active driving steering wheel torque, and driver image, determine whether the driver intends to change lanes; When the lane crossing time is less than or equal to the interference threshold and it is determined that the driver does not intend to change lanes, the tendency of the target vehicle to deviate from the lane is taken as the judgment result.

5. The method as described in claim 4, characterized in that, The step of determining whether the driver intends to change lanes based on the turn signal signal of the target vehicle, the active driving steering wheel torque, and the driver image includes: The system acquires the turn signal signals of the target vehicle and the driver image captured by the built-in camera. Face detection is performed on the driver image to obtain the gaze direction, eyelid opening, and head posture; The fatigue index is determined based on the direction of gaze, the eyelid opening, and the head posture. When the turn signal is off, the active driving steering wheel torque is less than a preset torque value, and the fatigue index is less than a preset threshold, it is determined that the driver does not intend to change lanes. When the duration of the active driving steering wheel torque being greater than or equal to a preset torque value is greater than or equal to a preset duration, or when the turn signal is on, it is determined that the driver intends to change lanes.

6. The method as described in claim 4, characterized in that, After determining whether the driver intends to change lanes based on the turn signal signal of the target vehicle, the active driving steering wheel torque, and the driver image, the process further includes: Acquire the steering wheel pressure signal of the target vehicle; A warning threshold is obtained based on the current vehicle speed and the lane curvature, wherein the warning threshold is greater than the intervention threshold; When the lane crossing time is less than or equal to the warning threshold, the lane crossing time is greater than or equal to the interference threshold, the turn signal is off and the steering wheel pressure signal is active, control the steering wheel and instrument panel of the target vehicle to issue a warning. When the lane crossing time is less than or equal to the emergency threshold and it is determined that the driver does not intend to change lanes, the deflection side is determined based on the future trajectory, and a preset braking force is applied to the front wheel of the deflection side to straighten the target vehicle.

7. The method as described in claim 1, characterized in that, When the judgment result indicates that the target vehicle has a tendency to deviate from the lane, a target cost function is constructed based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters. The target cost function is then solved according to preset constraints to obtain the target auxiliary torque, including: When the judgment result indicates that the vehicle has a tendency to deviate from the lane, the steering wheel assist torque is obtained based on the real-time motion parameters; Based on the future trajectory, the lateral offset distance, the heading angle, and the steering wheel auxiliary torque, a discretized vehicle dynamics equation is established. Based on the discretized vehicle dynamics equations, a target cost function is constructed, wherein the target cost function includes a tracking error penalty term, a control quantity penalty term, and a control change rate penalty term; Within a preset control time domain, the target cost function is optimized and solved according to preset constraints to obtain an auxiliary torque sequence, wherein the preset constraints include control magnitude constraints, control rate of change constraints, and position constraints. The target auxiliary torque is obtained based on the auxiliary torque sequence.

8. A lane-keeping intelligent device, characterized in that, The device includes: The input module is used to acquire road images and real-time motion parameters of the target vehicle; The recognition module is used to recognize the road image and obtain the lateral offset distance and heading angle of the target vehicle relative to the center of the lane; The judgment module is used to predict the future trajectory based on the lateral offset distance, the heading angle and the real-time motion parameters, and to judge whether the target vehicle has a tendency to deviate from the lane based on the future trajectory, and to obtain the judgment result. The solution module is used to construct a target cost function based on the future trajectory, the lateral offset distance, the heading angle, and the real-time motion parameters when the judgment result indicates that the target vehicle has a tendency to deviate from the lane, and to solve the target cost function according to preset constraints to obtain the target auxiliary torque. The generation module is used to generate correction control commands based on the target auxiliary torque; The output module is used to control the target vehicle to maintain its lane position according to the correction control command.

9. A lane keeping intelligent device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the lane-keeping intelligent method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the lane intelligent keeping method as described in any one of claims 1 to 7.

Images