A method and system for real-time simulation and visualization of lane keeping assistance

Abstract

The present invention belongs to the field of intelligent driving assistance technology and automobile simulation technology, and discloses a method for real-time simulation and visualization of lane keeping assistance. The specific use steps of the method are as follows: S1: software GUI interface display and Carsim connection: S2: image preprocessing and road line recognition: S3: road parameter calculation and display: S4: vehicle initial driving control: S5: real-time image recognition and lane keeping activation: S6: lane keeping strategy implementation: S7: steering wheel control and strategy adjustment. The present invention further expands its application capabilities by integrating the road driving simulation and image output functions of Carsim software, and innovatively develops additional tools. The tool not only integrates advanced lane image recognition technology, but also embeds a lane keeping assistance strategy execution module. The integrated design enables the vehicle to display its lane keeping ability in real time under simulated LKA conditions.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving assistance technology and automobile simulation technology, and specifically relates to a method and system for real-time simulation and visualization of lane keeping assistance. Background Art

[0002] At present, the mainstream test method for lane keeping assist (LKA) system is real vehicle testing, that is, engineers use real vehicles on specific test sections to verify the control effect of LKA strategy. Although this method can intuitively reflect the performance of the strategy in actual application, its high cost, long test cycle and complex debugging process have become defects that cannot be ignored. As a popular automotive simulation software, Carsim plays an important role in the research and development of modern automotive control systems. It can accurately simulate the driving state of vehicles under various complex road conditions. However, as a software focusing on vehicle dynamics analysis, Carsim does not directly provide the function of accessing LKA control strategy, so it cannot be directly used for simulation testing of LKA system. Although Carsim performs well in simulating the vehicle's response to driver operation, road surface and aerodynamic input, lane keeping assist is not within its established functional scope. Summary of the invention

[0003] The object of the present invention is to provide a method and system for real-time simulation and visualization of lane keeping assistance to solve the problems raised in the above background technology.

[0004] In order to achieve the above object, the present invention provides the following technical solution: a method for real-time simulation and visualization of lane keeping assistance, the specific steps of the method are as follows:

[0005] S1: Software GUI interface display and Carsim connection: Develop a user-friendly GUI to display vehicle status and road information, and integrate CarsimDLL to achieve real-time data communication;

[0006] S2: Image preprocessing and road line recognition: convert the perspective to get the top view, grayscale the edge detection, and fit the road line with a quadratic function to get y=f(x);

[0007] S3: Road parameter calculation and display: Fit the road line equation to calculate the curvature, take the average and convert the unit; calculate the centerline deviation of the vehicle body and the distance between the two sides, and display the GUI in real time;

[0008] S4: Initial driving control: The software controls the vehicle to follow a preset curve and approach the roadside at a specific angle to warm up for the lane keeping test;

[0009] S5: Real-time image recognition and lane keeping activation: image recognition and distance measurement, lane keeping is activated when the distance is close to the edge line, and intelligent steering wheel angle calculation ensures safety;

[0010] S6: Lane keeping strategy implementation: The software controls the turning angle according to the time curve to keep the lane, or adjusts the direction according to the offset linear interpolation to stabilize the lane center;

[0011] S7: Steering wheel control and strategy adjustment: Steering wheel takeover and strategy dynamic adjustment optimization to ensure accurate and reliable lane keeping.

[0012] Preferably, the software GUI interface display in S1 and the connection with Carsim refer to building an efficient lane keeping assist system simulation platform, designing an intuitive and user-friendly graphical interface (GUI), which can clearly display the current status information of the vehicle, including speed and direction, and present road conditions in real time, such as lane lines and curvature key data. The GUI also integrates the feedback mechanism of the lane keeping assist system, so that users can intuitively see the operating status and effect of the system. In order to achieve seamless docking with the Carsim simulation software, the DLL file provided by it is fully utilized to build a stable data communication interface. The interface ensures that the software can receive the image and vehicle status data transmitted by Carsim in real time and accurately.

[0013] Preferably, the specific steps of image preprocessing and road line recognition in S2 are as follows:

[0014] Step 1: Perspective Transformation

[0015] Receive and transform images: Receive images from the driver's perspective from Carsim, and then use image processing techniques, perspective transformation or affine transformation to transform the image from the driver's perspective to a top-down view;

[0016] Step 2: Grayscale and edge detection

[0017] Grayscale processing: convert the converted color image into a grayscale image. The grayscale image only contains brightness information and removes color information, thereby simplifying the complexity of subsequent processing;

[0018] Edge detection: Apply edge detection algorithms such as Canny edge detection on grayscale images. This algorithm can identify edge features in images, especially the edges of road lines. By adjusting the parameters of the algorithm, the effect of edge detection is optimized to ensure that road lines are accurately identified.

[0019] Step 3: Road line fitting

[0020] Extract road line feature points: extract the feature points of the road lines on the left and right sides of the image after edge detection; Fit the road lines: use a quadratic function to fit the extracted road line feature points to obtain the relationship y=f(x).

[0021] Preferably, the specific steps of calculating and displaying the road parameters in S3 are as follows:

[0022] Step 1: Curvature calculation steps and formula

[0023] A1: Calculate the curvature of the left and right road lines

[0024] The road line equations on the left and right sides obtained by fitting are usually expressed as quadratic functions, such as y = ax 2 + bx + c， Curvature κ The calculation formula for a general quadratic curve is more complicated, but for a parabola, at a specific point ( x 0, y The curvature on 0) is approximately: ,

[0025] The formula is the parabola at point ( x 0, y 0), where is the coefficient of the quadratic term of the parabola, x 0 is the horizontal coordinate of the point of interest;

[0026] A2: Pixel to meter conversion

[0027] In order to convert the calculated curvature from the image coordinate system to the actual road coordinate system (in meters), it is necessary to know the ratio of the image resolution and the lane width. Assume that the image resolution is W×H pixels, and the lane width is W lane meters, then each pixel represents the actual distance d pixel-to-meter for: ;

[0028] Step 2: Deviation and distance calculation steps and formulas

[0029] B1: Calculate the deviation between the center line of the vehicle body and the center line of the road

[0030] In order to calculate the deviation between the centerline of the vehicle and the centerline of the road, it is first necessary to determine the positions of the centerline of the vehicle and the centerline of the road. The centerline of the vehicle is determined by the center point of the vehicle, and the centerline of the road is approximated by the average of the left and right road line equations.

[0031] Assume that the horizontal coordinate of the centerline of the vehicle body is x vehicle , the horizontal coordinate of the road centerline is x road-center , then the deviation Δ x for:

[0032] Δ x = x vehicle − x road-center ,

[0033] B2: Calculate the distance from both sides of the vehicle to the road line

[0034] The distance from the two sides of the vehicle body to the road line is calculated by the point-to-line distance formula. For the left and right road lines, the point-to-line distance formula is used to find the shortest distance from the two sides of the vehicle body to the road line. However, in most cases, the distance from the center line of the vehicle body to the left and right road lines is more concerned, which is obtained by calculating the perpendicular distance between the center line of the vehicle body and the equations of the left and right road lines; assuming that the coordinates of the center line of the vehicle body are ( x vehicle , y vehicle ), the equation of the left road is y = f left ( x ), the right road route equation y = f right ( x ), then the distance from the center line of the vehicle to the left lane d left and the distance to the right lane d right They are:

[0035] d left =∣ y vehicle − f left ( x vehicle )∣

[0036] d right =∣ y vehicle − f right ( x vehicle )∣,

[0037] Similarly, the distance is in the image coordinate system and needs to be converted to the distance in the actual road coordinate system, which requires the use of a pixel to meter conversion ratio. These parameters are displayed in real time on the GUI interface for the driver's reference or for use in the control algorithm of the autonomous driving system.

[0038] Preferably, the initial driving control of the vehicle in S4 refers to the stage of vehicle test preparation, in which the software controls the vehicle to strictly follow the preset initial curve trajectory, and the vehicle needs to gradually adjust the body posture according to the instructions, and smoothly approach the road edge at a specific angle, in order to simulate the complex scenes in the real road environment, lay a solid foundation for the subsequent lane keeping assist system test, and ensure that the system can accurately identify the lane under various road conditions and realize stable and safe autonomous driving.

[0039] Preferably, the real-time image recognition and lane keeping activation in S5 means that during the automatic driving process, the system accurately recognizes the real-time image captured by the camera and quickly calculates the distance between the vehicle body and the road sideline. Once the distance is lower than a preset safety threshold, the lane keeping assist system is immediately activated, and based on advanced algorithm strategies, the required steering wheel angle is intelligently calculated, and the vehicle's driving trajectory is automatically adjusted to ensure that the vehicle remains stably in the lane, thereby improving driving safety and comfort.

[0040] Preferably, the specific steps of implementing the lane keeping strategy in S6 are as follows:

[0041] Step 1: Monitor and calculate offset:

[0042] First, the system captures the road image through the camera, identifies the coordinates of the center position of the vehicle body and the center line of the lane, and then calculates the distance between the center of the vehicle body and the center line of the lane, that is, the offset;

[0043] Step 2: Select and execute strategy:

[0044] Based on the calculated offset, the system selects the corresponding lane keeping strategy. If the time curve strategy is used, the software will gradually adjust the steering wheel angle according to the preset time curve to keep the vehicle stable in the lane. If the offset is large, the system is more inclined to use the offset strategy, that is, the steering wheel angle is calculated according to the linear interpolation method to ensure that the vehicle can smoothly return to the center of the lane.

[0045] Step 3: Adjust the steering wheel angle in real time:

[0046] Finally, the system adjusts the steering wheel in real time based on the steering wheel angle calculated by the selected strategy, so that the vehicle can stay accurately within the lane. Under the offset strategy, the closer to the road edge, the greater the steering wheel angle, to ensure that the vehicle can smoothly and quickly return to the center of the lane, improving driving safety and stability.

[0047] Preferably, the steering wheel control and strategy adjustment in S7 means that during the automatic driving process, the software will accurately take over the control of the steering wheel according to the calculated steering wheel angle, and ensure that the vehicle remains stably in the lane by intelligently adjusting the vehicle posture. At the same time, the system has a high degree of flexibility and adaptability, and can dynamically adjust the lane keeping strategy according to real-time road condition feedback and the current state of the vehicle, thereby further improving the accuracy and reliability of the system.

[0048] Preferably, based on a method for real-time simulation and visualization of lane keeping assist, the system consists of a front-end module and a back-end data module. The front-end module provides an interface to receive strategies, modes and Carsimsim files input by users, and transmits this information to the back-end module for processing. The back-end data module receives the display view and vehicle speed data transmitted by Carsim through software, determines the vehicle body position through image recognition of road information, executes the LKA strategy according to the offset between the center line of the vehicle body and the center line of the road surface, and feeds back the road and vehicle body posture information to the front-end view in real time to assist engineers in evaluating the effect of the LKA strategy.

[0049] The beneficial effects of the present invention are as follows:

[0050] 1. The present invention further expands the application capabilities of Carsim software by integrating its road driving simulation and image output functions, and innovatively develops additional tools. This tool not only integrates advanced lane image recognition technology, but also embeds a lane keeping assist (LKA) strategy execution module. The integrated design enables the vehicle to demonstrate its lane keeping ability in real time under simulated LKA conditions, and outputs relevant data and results through an intuitive graphical interface. This feature greatly facilitates the user's work in the product development stage, enabling the user to quickly analyze the execution effect of the LKA strategy and make adjustments as needed. Compared with traditional test methods, this solution significantly reduces the number of tests and effectively shortens the entire development cycle, thereby saving users valuable development costs. This innovation not only improves the development efficiency of automotive active safety systems, but also lays a solid foundation for the continuous advancement of intelligent driving technology.

[0051] 2. The present invention innovatively integrates image recognition and road deviation calculation technology, accurately identifies lane lines through advanced image processing methods, and calculates the deviation of the vehicle relative to the center of the road in real time, providing instant feedback for the lane keeping assist strategy to ensure that the system can efficiently perform the lane keeping function. The system has built-in multiple LKA strategies, such as time curve control and offset interpolation control, which can be flexibly adjusted according to the deviation. At the same time, the relative position and posture of the vehicle and the lane are displayed in real time through an intuitive graphical interface, which greatly facilitates engineers to evaluate the effectiveness of the strategy. At the same time, it is also equipped with a simple and easy-to-use user graphical interface, which supports users to easily input and adjust the LKA strategy, and monitor the execution effect of the strategy in real time during the simulation process, so as to achieve continuous optimization of the strategy and effectively improve the development efficiency and quality of intelligent driving technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0053] Figure 2 A flow chart for using the system of the present invention;

[0054] Figure 3 Generating graphs for the vehicle model and the road model of the present invention;

[0055] Figure 4 It is a schematic diagram of the control strategy of the auxiliary system of the present invention;

[0056] Figure 5 This is a schematic diagram of the lane recognition structure of the present invention;

[0057] Figure 6 It is a schematic diagram of the simulation results of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] like Figures 1 to 6 As shown, an embodiment of the present invention provides a method for real-time simulation and visualization of lane keeping assistance, and the specific steps of using the method are as follows:

[0060] S1: Software GUI interface display and Carsim connection: Develop a user-friendly GUI to display vehicle status and road information, and integrate CarsimDLL to achieve real-time data communication;

[0061] S2: Image preprocessing and road line recognition: convert the perspective to get the top view, grayscale the edge detection, and fit the road line with a quadratic function to get y=f(x);

[0062] S3: Road parameter calculation and display: Fit the road line equation to calculate the curvature, take the average and convert the unit; calculate the centerline deviation of the vehicle body and the distance between the two sides, and display the GUI in real time;

[0063] S4: Initial driving control: The software controls the vehicle to follow a preset curve and approach the roadside at a specific angle to warm up for the lane keeping test;

[0064] S5: Real-time image recognition and lane keeping activation: image recognition and distance measurement, lane keeping is activated when the distance is close to the edge line, and intelligent steering wheel angle calculation ensures safety;

[0065] S6: Lane keeping strategy implementation: The software controls the turning angle according to the time curve to keep the lane, or adjusts the direction according to the offset linear interpolation to stabilize the lane center;

[0066] S7: Steering wheel control and strategy adjustment: Steering wheel takeover and strategy dynamic adjustment optimization ensure accurate and reliable lane keeping.

[0067] Among them, the software GUI interface display in S1 and the connection with Carsim refer to the construction of an efficient lane keeping assist system simulation platform. An intuitive and user-friendly graphical interface (GUI) is designed, which can clearly display the current status information of the vehicle, including speed and direction, and present road conditions in real time, such as lane lines and curvature key data. The GUI also integrates the feedback mechanism of the lane keeping assist system, so that users can intuitively see the operating status and effect of the system. In order to achieve seamless docking with the Carsim simulation software, the DLL file provided by it is fully utilized to build a stable data communication interface. The interface ensures that the software can receive the images and vehicle status data transmitted by Carsim in real time and accurately, providing a solid foundation for the simulation and testing of the system.

[0068] The specific steps of image preprocessing and road line recognition in S2 are as follows:

[0069] The specific steps of image preprocessing and road line recognition in S2 are as follows:

[0070] Step 1: Perspective Transformation

[0071] Receive and convert images: Receive images from the driver's perspective from Carsim, and then use image processing techniques such as perspective transformation or affine transformation to convert the image from the driver's perspective to a bird's-eye view. The purpose of this step is to eliminate image distortion caused by different perspectives, making the road information more intuitive and accurate in the bird's-eye view.

[0072] Step 2: Grayscale and edge detection

[0073] Grayscale processing: convert the converted color image into a grayscale image. The grayscale image only contains brightness information and removes color information, thereby simplifying the complexity of subsequent processing;

[0074] Edge detection: Apply edge detection algorithms, such as Canny edge detection, to grayscale images. This algorithm can identify edge features in images, especially the edges of road lines. By adjusting the parameters of the algorithm, the effect of edge detection can be optimized to ensure that road lines are accurately identified.

[0075] Step 3: Road line fitting

[0076] Extracting road line feature points: Extracting the feature points of the road lines on the left and right sides of the image after edge detection; these feature points are usually located on the edges of the road lines and have a certain regularity and continuity.

[0077] Fitting road lines: Use a quadratic function (such as a parabola or hyperbola) to fit the extracted road line feature points to obtain the relationship y=f(x). The purpose of this step is to extract the road line from the image and express it in the form of a mathematical function, providing a basis for subsequent road parameter calculation and lane keeping assist system control.

[0078] The specific steps of calculating and displaying the road parameters in S3 are as follows:

[0079] Step 1: Curvature calculation steps and formula

[0080] A1: Calculate the curvature of the left and right road lines

[0081] The fitted road line equations on the left and right sides can usually be expressed as a quadratic function, such as y = ax 2 + bx + c (For simplicity, we assume that the road line is a parabola. In fact, it may be a more complex curve, but the principle is similar.) Curvature κ The calculation formula for a general quadratic curve is more complicated, but for a parabola, at a specific point ( x 0, y The curvature on 0) can be approximated as:

[0082] ,

[0083] Note that this formula is for the parabola at point ( x 0, y 0), where is the coefficient of the quadratic term of the parabola, x 0 is the horizontal coordinate of the point of interest (which may be a reference point on the vehicle body in the image). For the left and right road routes, their curvatures are calculated separately and the average is taken as the road curvature.

[0084] A2: Pixel to meter conversion

[0085] In order to convert the calculated curvature from the image coordinate system to the actual road coordinate system (in meters), it is necessary to know the ratio of the image resolution and the lane width. Assume that the image resolution is W×H pixels, and the lane width is W lane meters, then each pixel represents the actual distance d pixel-to-meter for: ,

[0086] The "pixel width of the lane in the image" is measured by an image processing algorithm. Once you have this ratio, you can convert the curvature from pixels to meters. However, please note that curvature itself is a dimensionless quantity. What really needs to be converted is the radius of curvature (if necessary), which can be calculated by taking the inverse of the curvature and multiplying it by the appropriate scale factor to convert it to meters.

[0087] Step 2: Deviation and distance calculation steps and formulas

[0088] B1: Calculate the deviation between the center line of the vehicle body and the center line of the road

[0089] In order to calculate the deviation between the centerline of the vehicle and the centerline of the road, we first need to determine the positions of the centerline of the vehicle and the centerline of the road. The centerline of the vehicle can be determined by the center point of the vehicle (which may be a feature point of the vehicle in the image, such as the center of the rear axle). The centerline of the road can be approximated by the average of the equations of the left and right road lines (if the road lines are parallel or approximately parallel);

[0090] Assume that the horizontal coordinate of the centerline of the vehicle body is x vehicle, the horizontal coordinate of the road centerline is x road-center (this may require some additional calculations or assumptions, such as assuming that the road is horizontal and symmetrical), then the deviation Δ x for:

[0091] Δ x = x vehicle − x road-center ,

[0092] Note that the horizontal coordinate here is in the image coordinate system. If you need to convert it into the deviation in the actual road coordinate system, you need to use the conversion ratio from pixels to meters;

[0093] B2: Calculate the distance from both sides of the vehicle to the road line

[0094] The distance from the sides of the vehicle to the road line can be calculated using the point-to-line distance formula. For the left and right road lines (assuming they can be approximated as straight line segments), the point-to-line distance formula is used to find the shortest distance from the sides of the vehicle (such as the left and right edges of the vehicle) to the road line. However, in most cases, you may be more concerned with the distance from the centerline of the vehicle to the left and right road lines, which can be obtained by calculating the perpendicular distance between the centerline of the vehicle and the equations of the left and right road lines;

[0095] Assume that the coordinates of the centerline of the vehicle body are ( x vehicle , y vehicle ), the equation of the left road is y = f left ( x ), the right road route equation is y = f right ( x ), then the distance from the center line of the vehicle to the left lane d left and the distance to the right lane d right They are:

[0096] d left =∣ y vehicle − f left ( x vehicle )∣

[0097] d right =∣ y vehicle − f right ( x vehicle )∣,

[0098] Similarly, these distances are in the image coordinate system. If you need to convert them into the actual road coordinate system, you need to use the pixel to meter conversion ratio. These parameters (deviation and distance) are displayed in real time on the GUI interface for the driver's reference or for use in the control algorithm of the autonomous driving system.

[0099] Road parameter calculation and display technology achieves an in-depth understanding of the road environment by accurately calculating the road curvature, the deviation between the centerline of the vehicle body and the centerline of the road, and the distance from the two sides of the vehicle body to the road line. This technology uses image recognition and algorithm processing to convert the parameters in the image coordinate system into values ​​in the actual road coordinate system, improving the accuracy and practicality of the parameters. At the same time, the real-time displayed parameter information provides an intuitive reference for the driver and also provides key input for the control algorithm of the autonomous driving system, which helps to improve the safety and stability of autonomous driving and lays a solid foundation for the development of intelligent transportation systems.

[0100] Among them, the initial driving control of the vehicle in S4 refers to the fact that during the vehicle test preparation stage, the software will control the vehicle to strictly follow the preset initial curve trajectory. The vehicle needs to gradually adjust its body posture according to the instructions and approach the road edge steadily at a specific angle. This is to simulate complex scenes in real road environments, lay a solid foundation for subsequent lane keeping assist system tests, and ensure that the system can accurately identify lanes under various road conditions and achieve stable and safe autonomous driving.

[0101] During the initial driving control phase of the vehicle, the software guides the vehicle to drive along the preset initial curve trajectory, steadily approach the road edge, simulate the complex scenes of real roads, and lay a solid foundation for the subsequent lane keeping assist system test. This process ensures that the system can accurately identify lanes under various road conditions, providing a strong guarantee for the realization of stable and safe autonomous driving, and improving the overall test efficiency and system reliability.

[0102] Among them, the real-time image recognition and lane keeping activation in S5 means that during the automatic driving process, the system accurately recognizes the real-time image captured by the camera and quickly calculates the distance between the vehicle body and the road sideline. Once the distance is lower than the preset safety threshold, the lane keeping assist system is immediately activated. Based on advanced algorithm strategies, it intelligently calculates the required steering wheel angle, automatically adjusts the vehicle's driving trajectory, ensures that the vehicle remains stable in the lane, and improves driving safety and comfort.

[0103] The real-time image recognition and lane keeping activation mechanism accurately identifies the real-time images captured by the camera and quickly responds to changes in the distance between the vehicle body and the road sideline. Once it falls below the safety threshold, the lane keeping assist system is activated, the steering wheel angle is intelligently calculated, and the driving trajectory is automatically adjusted to ensure that the vehicle drives in a stable lane, significantly improving driving safety and comfort, and providing strong support for autonomous driving.

[0104] The specific steps for implementing the lane keeping strategy in S6 are as follows:

[0105] Step 1: Monitor and calculate offset:

[0106] First, the system captures the road image through the camera, identifies the center position of the vehicle body (usually in the middle of the image) and the coordinates of the lane centerline (determined by the road line fitting formula f(x) at y=0). Then, the distance between the center of the vehicle body and the lane centerline, i.e., the offset, is calculated.

[0107] Step 2: Select and execute strategy:

[0108] Based on the calculated offset, the system selects the corresponding lane keeping strategy. If the time curve strategy is used, the software will gradually adjust the steering wheel angle according to the preset time curve to keep the vehicle stable in the lane. If the offset is large, the system may prefer to use the offset strategy, that is, calculate the steering wheel angle according to the linear interpolation method to ensure that the vehicle can smoothly return to the center of the lane.

[0109] Step 3: Adjust the steering wheel angle in real time:

[0110] Finally, the system adjusts the steering wheel in real time based on the steering wheel angle calculated by the selected strategy, so that the vehicle can accurately stay in the lane. Under the offset strategy, the closer to the road edge, the greater the steering wheel angle, to ensure that the vehicle can smoothly and quickly return to the center of the lane, improving driving safety and stability.

[0111] The lane keeping strategy monitors and calculates the offset between the vehicle body and the center line of the lane in real time, accurately selects and executes the corresponding lane keeping strategy, such as the time curve or offset strategy, and adjusts the steering wheel angle in real time to ensure that the vehicle stays in the lane smoothly and quickly, improving driving safety and stability. The implementation of this strategy effectively improves the performance and reliability of the autonomous driving system, providing drivers with a more comfortable and intelligent driving experience.

[0112] Among them, the steering wheel control and strategy adjustment in the S7 refers to the fact that during the automatic driving process, the software will accurately take over the control of the steering wheel according to the calculated steering wheel angle, and ensure that the vehicle remains stable in the lane by intelligently adjusting the vehicle posture. At the same time, the system has a high degree of flexibility and adaptability, and can dynamically adjust the lane keeping strategy based on real-time road condition feedback and the current state of the vehicle, further improving the accuracy and reliability of the system. In addition, the system will continuously optimize and upgrade the strategy to better adapt to various complex road conditions and the personalized driving habits of different drivers, thereby providing a more comfortable and intelligent driving experience while ensuring safety.

[0113] Steering wheel control and strategy adjustment take over the steering wheel accurately during autonomous driving, intelligently adjust the vehicle posture, and ensure that the lane remains stable. The system is highly flexible and adaptable, and dynamically adjusts the lane keeping strategy in real time to improve accuracy and reliability. The continuously optimized and upgraded strategy better adapts to complex roads and personalized driving habits, ensures safety, provides a comfortable and intelligent driving experience, and improves the overall performance and user satisfaction of autonomous driving.

[0114] Among them, based on a method for real-time simulation and visualization of lane keeping assist, the system consists of a front-end module and a back-end data module. The front-end module provides an interface to receive user-input strategies, modes and Carsimsim files, and transmits this information to the back-end module for processing. The back-end data module receives the display view and vehicle speed data transmitted by Carsim through software, determines the vehicle body position through image recognition of road information, executes the LKA strategy according to the offset between the center line of the vehicle body and the center line of the road surface, and feeds back the road and vehicle body posture information to the front-end view in real time to assist engineers in evaluating the effectiveness of the LKA strategy.

[0115] Specific implementation example: Set up the vehicle model and road model in the Carsim software and store the generated simfile.sim in the software source file path such as Figure 3 As shown;

[0116] Open the software and set the initial driving curve and lane keeping assist system control strategy, such as Figure 4 As shown:

[0117] Click Execute. The software connects to the carsim software, receives the camera image and performs lane recognition, while calculating the vehicle's motion status and its relative position to the road. This information is superimposed on the original camera image and displayed on the software interface. Users can watch the real-time image to judge the vehicle's driving posture, the intervention time of the lane keeping assist system, and the strategy execution results, such as Figure 5 shown.

[0118] After execution, the simulation results can be played back in the form of driving path lines and characteristic variable curves, such as Figure 6 shown.

[0119] If necessary, users can adjust the control strategy data of the lane keeping assist system at any time and re-execute it to view the assistance effects of different strategies.

[0120] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0121] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for real-time simulation and visualization of lane keeping assistance, characterized in that: The specific steps of using this method based on software are as follows: S1: Software GUI interface display and Carsim connection: Develop a user-friendly GUI to display vehicle status and road information, and integrate CarsimDLL to achieve real-time data communication; S2: Image preprocessing and road line recognition: convert the perspective to get the top view, grayscale the edge detection, and fit the road line with a quadratic function to get y=f(x); S3: Road parameter calculation and display: Fit the road line equation to calculate the curvature, take the average and convert the unit; calculate the centerline deviation of the vehicle body and the distance between the two sides, and display the GUI in real time; S4: Initial driving control: The software controls the vehicle to follow a preset curve and approach the roadside at a specific angle to warm up for the lane keeping test; S5: Real-time image recognition and lane keeping activation: image recognition and distance measurement, lane keeping is activated when the distance is close to the edge line, and intelligent steering wheel angle calculation ensures safety; S6: Lane keeping strategy implementation: The software controls the turning angle according to the time curve to keep the lane, or adjusts the direction according to the offset linear interpolation to stabilize the lane center; S7: Steering wheel control and strategy adjustment: Steering wheel takeover and strategy dynamic adjustment optimization to ensure accurate and reliable lane keeping; The specific steps of calculating and displaying the road parameters in S3 are as follows: Step 1: Curvature calculation steps and formula A1: Calculate the curvature of the left and right road lines The fitted road line equations on the left and right sides are expressed as quadratic functions. y = ax 2 + bx + c， Curvature κ The calculation formula for a general quadratic curve is more complicated, but for a parabola, at a specific point ( x 0, y The curvature on 0) is approximately: , The formula is the parabola at point ( x 0, y 0), where is the coefficient of the quadratic term of the parabola, x 0 is the horizontal coordinate of the point of interest; A2: Pixel to meter conversion In order to convert the calculated curvature from the image coordinate system to the actual road coordinate system, it is necessary to know the ratio of the image resolution and the lane width. Assume that the image resolution is W×H pixels, and the lane width is W lane meters, then each pixel represents the actual distance d pixel-to-meter for: ， Step 2: Deviation and distance calculation steps and formulas B1: Calculate the deviation between the center line of the vehicle body and the center line of the road In order to calculate the deviation between the centerline of the vehicle and the centerline of the road, it is first necessary to determine the positions of the centerline of the vehicle and the centerline of the road. The centerline of the vehicle is determined by the center point of the vehicle, and the centerline of the road is approximated by the average of the left and right road line equations. Assume that the horizontal coordinate of the centerline of the vehicle body is x vehicle , the horizontal coordinate of the road centerline is x road-center , then the deviation Δ x for: D x=x vehicle - x road-center， B2: Calculate the distance from both sides of the vehicle to the road line The distance from the two sides of the vehicle body to the road line is calculated by the point-to-line distance formula. For the left and right road lines, the point-to-line distance formula is used to find the shortest distance from the two sides of the vehicle body to the road line. However, in most cases, the distance from the center line of the vehicle body to the left and right road lines is more concerned, which is obtained by calculating the perpendicular distance between the center line of the vehicle body and the equations of the left and right road lines; assuming that the coordinates of the center line of the vehicle body are ( x vehicle , y vehicle ), the equation of the left road is y = f left ( x ), the right road route equation y = f right ( x ), then the distance from the center line of the vehicle to the left lane d left and the distance to the right lane d right They are: ， Similarly, the distance is in the image coordinate system, and needs to be converted to the distance in the actual road coordinate system, which requires the use of a pixel to meter conversion ratio. These parameters are displayed in real time on the GUI interface for the driver's reference or used in the control algorithm of the autonomous driving system; The specific steps for implementing the lane keeping strategy in S6 are as follows: Step 1: Monitor and calculate offset: First, the system captures the road image through the camera, identifies the coordinates of the center position of the vehicle body and the center line of the lane, and then calculates the distance between the center of the vehicle body and the center line of the lane, that is, the offset; Step 2: Select and execute strategy: Based on the calculated offset, the system selects the corresponding lane keeping strategy. If the time curve strategy is used, the software will gradually adjust the steering wheel angle according to the preset time curve to keep the vehicle stable in the lane. If the offset is large, the system is more inclined to use the offset strategy, that is, the steering wheel angle is calculated according to the linear interpolation method to ensure that the vehicle can smoothly return to the center of the lane. Step 3: Adjust the steering wheel angle in real time: Finally, the system adjusts the steering wheel in real time based on the steering wheel angle calculated by the selected strategy, so that the vehicle can stay accurately within the lane. Under the offset strategy, the closer to the center line, the greater the steering wheel angle, to ensure that the vehicle can smoothly and quickly return to the center of the lane, improving driving safety and stability.

2. The method for real-time simulation and visualization of lane keeping assistance according to claim 1, characterized in that: The software GUI interface display in S1 and the connection with Carsim refer to the construction of an efficient lane keeping assist system simulation platform, and the design of an intuitive and user-friendly graphical interface; The interface can clearly display the vehicle's current status information, including speed and direction, and present key data such as road conditions, lane lines, and curvature in real time. The GUI also integrates the feedback mechanism of the lane keeping assist system, allowing users to intuitively see the system's operating status and effects; In order to achieve seamless connection with the Carsim simulation software, the DLL files provided by it were fully utilized to build a stable data communication interface. The interface ensures that the software can receive the images and vehicle status data transmitted by Carsim in real time and accurately.

3. The method for real-time simulation and visualization of lane keeping assistance according to claim 1, characterized in that: The specific steps of image preprocessing and road line recognition in S2 are as follows: Step 1: Perspective Transformation Receive and transform images: Receive images from the driver's perspective from Carsim, and then use image processing techniques, perspective transformation or affine transformation to transform the image from the driver's perspective to a top-down view; Step 2: Grayscale and edge detection Grayscale processing: convert the converted color image into a grayscale image. The grayscale image only contains brightness information and removes color information, thereby simplifying the complexity of subsequent processing; Edge detection: Apply edge detection algorithm, Canny edge detection, to grayscale images. This algorithm can identify edge features in images, especially the edges of road lines. By adjusting the parameters of the algorithm, the effect of edge detection is optimized to ensure that road lines are accurately identified. Step 3: Road line fitting Extract road line feature points: extract the feature points of the road lines on the left and right sides of the image after edge detection; Fit the road lines: use a quadratic function to fit the extracted road line feature points to obtain the relationship y=f(x).

4. The method for real-time simulation and visualization of lane keeping assistance according to claim 1, characterized in that: The initial driving control of the vehicle in S4 refers to the stage of vehicle test preparation, in which the software will control the vehicle to strictly follow the preset initial curve trajectory. The vehicle needs to gradually adjust its body posture according to the instructions and steadily approach the road edge at a specific angle. This is to simulate complex scenarios in real road environments, lay a solid foundation for subsequent lane keeping assist system testing, and ensure that the system can accurately identify lanes under various road conditions to achieve stable and safe autonomous driving.

5. The method for real-time simulation and visualization of lane keeping assistance according to claim 1, characterized in that: The real-time image recognition and lane keeping activation in S5 means that during the autonomous driving process, the system accurately recognizes the real-time image captured by the camera and quickly calculates the distance between the vehicle body and the road sideline. Once the distance is lower than the preset safety threshold, the lane keeping assist system is immediately activated. Based on advanced algorithm strategies, it intelligently calculates the required steering wheel angle and automatically adjusts the vehicle's driving trajectory to ensure that the vehicle remains stable in the lane, thereby improving driving safety and comfort.

6. The method for real-time simulation and visualization of lane keeping assistance according to claim 1, characterized in that: The steering wheel control and strategy adjustment in the S7 refers to the fact that during the autonomous driving process, the software will accurately take over the control of the steering wheel based on the calculated steering wheel angle, and intelligently adjust the vehicle posture to ensure that the vehicle remains stable in the lane. At the same time, the system is highly flexible and adaptable, and can dynamically adjust the lane keeping strategy based on real-time road condition feedback and the current status of the vehicle, further improving the accuracy and reliability of the system.

7. A lane keeping assist real-time simulation and visualization system, implementing the method according to claim 1, characterized in that: The system consists of a front-end module and a back-end data module. The front-end module provides an interface to receive user-input strategies, modes and Carsimsim files, and transmits this information to the back-end module for processing. The back-end data module receives the display view and vehicle speed data transmitted by Carsim through software, determines the vehicle body position through image recognition of road information, executes the LKA strategy based on the offset between the center line of the vehicle body and the center line of the road surface, and feeds back the road and vehicle body posture information to the front-end view in real time to assist engineers in evaluating the effectiveness of the LKA strategy.

Images