A lane line ground truth acquisition method and system

Abstract

The application belongs to the field of intelligent driving of automobiles, and provides a lane line true value acquisition method and system. The method comprises the following steps: acquiring current map information and a center point on a map road closest to a position of a vehicle; searching for a road point in front of the position of the vehicle based on the center point on the map road, and calculating a lateral compensation distance based on a vehicle heading vector, so as to acquire lane line discrete point true value information; fitting a three-dimensional curve to the lane line discrete point true value information through a multi-dimensional curve fitting method, so as to acquire true value parameters of a current lane line through the three-dimensional curve; and sending the true value parameters of the current lane line. The method meets the application requirement of automatic driving simulation technology, meets the data input requirement of a downstream module in a software structure, and provides high-precision numerical values on this basis.

Description

Technical Field

[0001] This invention pertains to the field of intelligent driving for automobiles, and specifically relates to a method and system for obtaining lane line truth values. Background Technology

[0002] To reduce road traffic accidents, improve travel efficiency, and enhance the travel experience, intelligent driving technology has developed rapidly in recent years. Intelligent driving technology encompasses multiple aspects, including perception, planning, decision-making, and control. Virtual simulation technology is an indispensable tool in automotive R&D, manufacturing, and verification testing, effectively shortening the R&D cycle and reducing costs. With the development of intelligent and connected vehicles, virtual simulation technology has found greater application in the field of autonomous driving. Currently, the industry has reached a consensus on a "three-pillar" approach for testing intelligent driving: simulation testing, closed-course testing, and open-road testing. Virtual simulation testing is a crucial step in realizing the application of advanced intelligent assisted driving systems. Vehicles with autonomous driving capabilities must undergo extensive virtual simulation testing and real-world road testing before commercialization. Simulation testing includes the following types: Model-in-the-Loop (MIL), Software-in-the-Loop (SIL), Hardware-in-the-Loop (HIL), and Vehicle-in-the-Loop (VIL).

[0003] Existing lane line extraction methods or technologies have the following drawbacks: 1) Related algorithms and products heavily rely on sensor data, with relatively few simulation studies and software products available. This makes it difficult to generate stable ground truth data in a simulation environment, increasing development difficulty. For example, a Chinese patent application titled "A Method and Apparatus for Obtaining Truth Values ​​of Lane Line Coordinates Based on Unmanned Vehicles" describes a lane line acquisition method based on images acquired by a camera. This method depends on pixel recognition after camera exposure. When the environment is unstable, such as excessively high or low light intensity, it cannot effectively provide pixel information. Furthermore, such calculations involve recognition and computational errors, resulting in poor convergence of the lane line ground truth results. 2) Lane line data results are obtained using perception algorithms, breakpoint prediction, and fitting methods. These methods have systematic errors, and their reliability needs to be considered. A Chinese patent application titled "Method, Apparatus, Device, and Storage Medium for Determining Lane Centerline Accuracy" describes a method to improve lane line centerline accuracy by obtaining edge distances through breakpoints on the lane line centerline ground truth. This method obtains raw data based on the actual driving trajectory of the vehicle, which means that the data itself has the error of being acquired after the event. Furthermore, when obtaining edge distances at breakpoints, this processing method leads to unstable output when dealing with curves and traffic intersections; 3) Poor real-time performance and slow iteration convergence speed. A Chinese patent application titled "Lane Line Recognition Anomaly Time Determination Method, Lane Line Recognition Device and System" describes updating lane lines based on posterior lane line results and using a confidence-building method to obtain the true value of the lane lines. This method can provide valid information about the elapsed time after the system has been running for a period of time. Although it provides detailed information for subsequent inspections, it cannot provide valid information directly when the raw data is acquired, and cannot quickly detect problems, resulting in poor data real-time performance and validity; 4) The method developed based on the Carla client has slow data iteration and computation speed, affecting the overall system efficiency. In addition, the architecture is unstable, easily modified illegally from the Python API, and security is limited. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a method and system for obtaining lane line truth values. This method meets the application requirements of autonomous driving simulation technology, satisfies the data input requirements of downstream modules in the software structure, and provides high-precision numerical values.

[0005] To achieve the above objectives, the present invention employs the following technical solution:

[0006] In a first aspect, the present invention provides a method for obtaining the true value of lane lines, comprising:

[0007] Obtain the current map information and the center point of the nearest road on the map to the vehicle's location;

[0008] Based on the center point of the road on the map, the system searches for road points forward from the vehicle's location. It calculates the lateral compensation distance based on the vehicle's facing vector, thereby obtaining the true value information of the discrete points of the lane line. The system then uses a multi-dimensional curve fitting method to fit a three-dimensional curve to the discrete points of the lane line, and obtains the true value parameters of the current lane line through the three-dimensional curve.

[0009] Send the truth value parameter of the current lane line.

[0010] As a further improvement of the present invention, the step of obtaining the current map information and the center point of the map road closest to the vehicle's position includes:

[0011] In a world snapshot, obtain the current map information and the center point of the nearest map road to the vehicle's location, and determine whether the center point of the nearest map road is valid;

[0012] If successful, proceed to the loop logic step of searching for the center point of the next map road. The loop logic step is: set an adaptive forward search distance, and search for the center point of the map road from near to far.

[0013] If invalid, check the validity of the center point on the map road in the previous world snapshot. If valid, copy the data according to the data at the end of the previous world snapshot, complete the saving and output. If invalid, initialize the output data structure, complete the saving and output, and set the data of this world snapshot as invalid.

[0014] As a further improvement of the present invention, the adaptive forward search distance formula is as follows:

[0015]

[0016] in: For each adaptive distance, This is the initial forward search distance. This represents the number of loop iterations.

[0017] After searching for the center point of the road on the map from near to far, the process also includes: determining the validity of the center point of the next map road; if valid, the data will be saved and recorded; if invalid, the data will be discarded.

[0018] As a further improvement of the present invention, the step of searching for road points forward from the vehicle's location based on the center point of the road on the map, and calculating the lateral compensation distance based on the vehicle's facing vector, includes:

[0019] The system acquires center point data on the map roads, then calculates the coordinate data of the center points on the map roads. From the center point coordinate data, it calculates the coordinate transformation relationship and the vehicle heading vector. Based on the vehicle heading vector, it obtains the lateral lane line compensation value, and based on the lateral lane line compensation value, it obtains the main road edge line points. Then, it acquires the center points and related data on the map roads from the left, left-left, right, and right-right sides of the vehicle.

[0020] As a further improvement of the present invention, the step of acquiring center point data on the map road, and then solving for the center point coordinate data on the map road, and solving for the coordinate transformation relationship and the vehicle heading vector from the center point coordinate data, includes:

[0021] Using the obtained Actor ID as an index, the coordinate vector is obtained on the map;

[0022] Based on the map's inherent coordinate system, and the obtained coordinate vectors and vehicle position, the data is transformed into the vehicle's coordinate system to obtain the coordinate data of the center point on the map's road.

[0023] The process of solving the coordinate transformation relationship and the vehicle heading vector from the center point coordinate data includes:

[0024] Obtain the Actor ID of the vehicle to get the current orientation vector of the vehicle, and return the orientation vector of the target based on the currently specified target.

[0025] As a further improvement of the present invention, the step of obtaining the lateral lane line compensation value based on the vehicle's facing vector, obtaining the main road edge line points based on the lateral lane line compensation value, and then obtaining the center points and related data of the left side, left-left side, right side, and right-right side of the map road from the vehicle's left side, left-left side, right side, and right-right side includes:

[0026] After obtaining the vehicle's orientation vector, calculate the unit vector of the vehicle's orientation vector, and then calculate the projection of the lane width onto the direction of the lane width. The projection calculation formula is as follows:

[0027]

[0028] in, The vector after projection calculation, which includes components. These correspond to the compensation values ​​on each axis. Indicates the calculation direction of the three components; This refers to the vector pointing towards the front of the vehicle; For the vehicle's heading vector The formula for determining the modulus of the axial component is as follows: L refers to lane width;

[0029] The compensation value for the lateral lane lines is obtained by projection calculation;

[0030] By extending the compensation value to the left and right once on the center point data of the map road, the corresponding edge point of the current main road is obtained. In the iterative process of gradually obtaining the center point of the next map road, all discrete points of the edge line of the main road can be obtained in sequence.

[0031] Based on all discrete points along the edge of the main road, search to the left, left-left, right, and right-right respectively;

[0032] After obtaining the center point coordinates of the main road on the map, the compensation value is shifted in the negative direction by a preset multiple to obtain the coordinates of the center point that may exist on the left side of the map. Based on these coordinates, it is determined whether there is a corresponding center point on the map. If there is, all discrete points of the edge line of the left side of the road are obtained, and then the left and right sides of the road are calculated. If there is no such point, the calculation of the left and right sides of the road is skipped.

[0033] The right-hand road and the right-right road are calculated using the same method as the left-hand road.

[0034] As a further improvement of the present invention, the step of fitting a three-dimensional curve to discrete points using a multi-dimensional curve fitting method based on the true value information of discrete points of lane lines includes:

[0035] Fit the true value information of discrete points of the lane line; when the lane line is segmented, calculate the curve parameters for the straight and curved sections; take multiple discrete points at the end of the first curve segment and multiple discrete points at the beginning of the second curve segment to form a new data set, and calculate the average curvature of this data set. Use the average curvature to characterize the slope of the fitted curve, realize the curvature smoothing operation at the junction of the lane line segments, obtain the expression of the lane line, and construct a cubic curve model.

[0036] Points with variance distributions exceeding 3σ of the normal distribution are removed from the data. A least-squares fitting function is constructed based on the input data of the cubic curve model. The given fitting function is solved using the matrix coefficient equation to achieve the fitting of a three-dimensional curve to discrete points.

[0037] As a further improvement of the present invention, after obtaining the true value parameters of the current lane line through the three-dimensional curve, the method further includes:

[0038] The true value parameters of the lane lines and the center points of the acquired map roads are structured and organized, including:

[0039] The center point structure of a single map road includes: waypoint coordinates x, y, z, the road where the waypoint is located, the road type where the waypoint is located, the road width where the waypoint is located, the point, the lane line type corresponding to the waypoint, the lane line color corresponding to the waypoint, and the lane line width corresponding to the waypoint.

[0040] Lane line parameter structure, including: lane line output validity and lane line length.

[0041] As a further improvement of the present invention, sending the truth parameters of the current lane line refers to sending prepared data, which is converted into a string type and sent sequentially in the form of a data stream.

[0042] After the truth parameters of the current lane line are sent, the current world snapshot is completed; then, world snapshots are continuously taken, returning to the step of obtaining the current map information and the center point of the map road closest to the vehicle's position;

[0043] By comparing the true values ​​of the current lane lines with the sensor recognition results, we can determine the optimization direction of the sensor perception algorithm; or the true values ​​of the current lane lines can be used as input to the planning and control module to participate in downstream path planning or be integrated into the navigation system.

[0044] Secondly, the present invention provides a lane line truth value acquisition system, comprising:

[0045] A lane line truth value acquisition system, characterized in that it includes:

[0046] The acquisition module is used to obtain the current map information and the center point of the nearest road on the map to the vehicle's location;

[0047] The solution module is used to search for road points forward from the vehicle's location based on the center point of the road on the map, calculate the lateral compensation distance based on the vehicle's facing vector, and thus obtain the true value information of the discrete points of the lane line. Based on the true value information of the discrete points of the lane line, a three-dimensional curve is fitted to the discrete points using a multi-dimensional curve fitting method, and the true value parameters of the current lane line are obtained through the three-dimensional curve.

[0048] The sending module is used to send the true value parameters of the current lane line.

[0049] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the lane line truth value acquisition method.

[0050] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the lane line truth value acquisition method.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] This invention utilizes the model provided by the Open-drive module in Carla as the raw data input to obtain the current map information and the center point of the nearest map road to the vehicle's position. This avoids system measurement errors caused by data acquisition and perception. It progressively reads and saves information such as the map, lane lines, and waypoints on the map roads. The design logic ensures data security at each stage. An improved least squares method is used as the lane line curve fitting method to obtain the lane line parameters and related ground truth values ​​under the current frame environment. This method meets the application requirements of autonomous driving simulation technology and the data input requirements of downstream modules in the software structure, and provides high-precision numerical values. This invention can easily obtain raw data and can be quickly applied to different types of maps. The data results obtained by this method are more reliable. This method does not involve perception algorithms, confidence assessment, posterior iterative updates, or other methods that introduce computational errors, providing more robust data input for downstream applications and a strong reference for multiple experiments. The lane line results obtained by this method have extremely high accuracy, achieving the same level of accuracy as map modeling. Compared with the results obtained by sensor recognition under the same conditions, the accuracy of this method is improved by at least one order of magnitude. Attached Figure Description

[0053] Figure 1 This is a flowchart of a method for obtaining the true value of lane lines according to the present invention;

[0054] Figure 2 The basic structure of Carla is given in the embodiments of the present invention;

[0055] Figure 3 This is an architecture diagram of the lane line truth value acquisition module provided in an embodiment of the present invention;

[0056] Figure 4 This is a detailed flowchart of the Lane Detect software provided in an embodiment of the present invention;

[0057] Figure 5 The following are the detailed steps for obtaining the center point data on a map according to an embodiment of the present invention;

[0058] Figure 6 The Lane Detect software described in this embodiment of the invention sends data to the Carla Server.

[0059] Figure 7 This is a schematic diagram of the lane line truth value acquisition system provided by the present invention;

[0060] Figure 8 This is a schematic diagram of an electronic device provided by the present invention. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0063] Terminology Explanation:

[0064] Autonomous driving simulation technology: To ensure the safe operation of intelligent connected vehicles on the road, with the development of computer technology, people use mathematical modeling, scene reconstruction and other methods to restore autonomous driving application scenarios and establish system models that are as close as possible to the real world. This enables the design, development and verification of autonomous driving software for scenarios that are difficult to occur or reproduce in reality. This field of technology is called autonomous driving simulation technology.

[0065] SIL stands for Software in the Loop, a technology used in autonomous driving simulation. It refers to a process where a computer runs a vehicle model within a virtual system environment, processes signals from various sensors and controllers within the virtual model, and receives control signals from the virtual ECU and bench sensors. This connects the virtual ECU under test with the system environment model, achieving a closed-loop simulation and control system. SIL testing is an essential step in the V-shaped development model of automotive software testing.

[0066] Autonomous driving simulation simulators are software platforms providing virtual simulation testing for autonomous vehicles. They offer rich data environment models and support multiple sensors, such as LiDAR and cameras, as well as hardware and software integration with vehicle controllers. They also feature powerful APIs and diverse development interfaces, and their software architecture is highly extensible, providing an excellent platform for the research and development of autonomous driving simulation technology. Examples of excellent autonomous driving simulation simulators on the market include Carla, PreScan, and Carsim.

[0067] Lane line ground truth: Lane line ground truth refers to the real data of road lane information obtained through certain methods. As an ideal state quantity, it can be used for comparison of the results of autonomous driving perception algorithms, participate in fusion, or be directly used as input for planning and control.

[0068] The purpose of this invention is to provide a method for obtaining lane line ground truth data based on Carla. For example... Figure 1 As shown, the first objective of this invention is to provide a method for obtaining the true value of lane lines, characterized by comprising:

[0069] S100: Obtain the current map information and the center point of the nearest road on the map to the vehicle's location;

[0070] S200 searches for road points forward from the vehicle's location based on the center point of the road on the map, calculates the lateral compensation distance based on the vehicle's facing vector, and obtains the true value information of the discrete points of the lane line. Based on the true value information of the discrete points of the lane line, a three-dimensional curve is fitted to the discrete points using a multi-dimensional curve fitting method, and the true value parameters of the current lane line are obtained through the three-dimensional curve.

[0071] S300, send the true value parameters of the current lane line.

[0072] This invention utilizes the model provided by the Open-drive module in Carla as the raw data input, avoiding system measurement errors caused by data acquisition and perception. It progressively reads and saves information such as the map, lane lines, and waypoints on the map. The design logic ensures data security at each stage. An improved least squares method is used as the lane line curve fitting method to obtain the lane line parameters and related ground truth values ​​under the current frame environment. This method meets the application requirements of autonomous driving simulation technology, satisfies the data input requirements of downstream modules in the software structure, and provides high-precision numerical values.

[0073] The methods and results involved in this invention correspond to the SIL part. This method is based on Carla, obtains map information from Open-drive, and is implemented from the underlying UE architecture to the server side. The method searches for road points forward from the vehicle's current location and calculates the lateral compensation distance based on the vehicle's facing vector, thereby obtaining the ground truth information of discrete lane line points. The parameters of the current lane line are then obtained through a multi-dimensional curve fitting method and output.

[0074] This invention uses simulation technology to obtain the true values ​​of lane lines, and its main improvement over existing methods is as follows:

[0075] 1) This method does not rely on sensors to perceive the environment to generate data, but acquires data based on a simulated environment model, thus avoiding measurement errors from a system perspective, and the results are true value data;

[0076] 2) It reduces computational and maintenance costs while expanding environmental adaptability. The method is implemented at the underlying layer from UE to Carla-Server. By utilizing the properties of Lane and Actor in Carla, it is easy to obtain raw data and can be quickly applied to different types of maps.

[0077] 3) The data results obtained by this method are more reliable. This method does not involve perceptual algorithms, confidence assessment, posterior iterative updates and other methods that may introduce computational errors. This provides more robust data input for downstream applications and also provides a strong reference for multiple experiments.

[0078] 4) The lane line results obtained by this method are extremely accurate, achieving the same level of accuracy as map modeling. Compared with the results obtained by sensor recognition under the same conditions, the accuracy of this method is improved by at least one order of magnitude.

[0079] 5) The software is developed based on Carla's server-side architecture, resulting in faster overall response speed, more stable functional architecture, and avoidance of repeated compilation when making changes, thus improving efficiency.

[0080] As a specific example of the present invention, obtaining the current map information and the center point of the map road closest to the vehicle's position includes:

[0081] S1: In a world snapshot, obtain the current map information and the center point of the nearest map road to the vehicle's location, and determine whether the center point of the nearest map road is valid;

[0082] S2: If valid, proceed to the loop logic step of searching for the center point of the next map road. The loop logic step is: set an adaptive forward search distance, and search for the center point of the map road from near to far.

[0083] S3: If invalid, check the validity of the center point on the map road in the previous world snapshot. If valid, copy the data according to the data at the end of the previous world snapshot, and complete the saving and output. If invalid, initialize the output data structure, complete the saving and output, and set the data of this world snapshot to invalid.

[0084] The present invention also provides a specific embodiment, wherein the adaptive forward search distance formula is:

[0085]

[0086] in: For each adaptive distance, This is the initial forward search distance. This represents the number of loop iterations.

[0087] After searching for the center point of the road on the map from near to far, the process also includes: determining the validity of the center point of the next map road; if valid, the data will be saved and recorded; if invalid, the data will be discarded.

[0088] As a specific example of the present invention, the step of searching for road points forward from the vehicle's location based on the center point of the road on the map, and calculating the lateral compensation distance based on the vehicle's facing vector, includes:

[0089] The system acquires center point data on the map roads, then calculates the coordinate data of the center points on the map roads. From the center point coordinate data, it calculates the coordinate transformation relationship and the vehicle heading vector. Based on the vehicle heading vector, it obtains the lateral lane line compensation value, and based on the lateral lane line compensation value, it obtains the main road edge line points. Then, it acquires the center points and related data on the map roads from the left, left-left, right, and right-right sides of the vehicle.

[0090] In step S200, obtaining the center point data on the map road and then solving for the center point coordinate data on the map road includes:

[0091] Using the obtained Actor ID as an index, the coordinate vector is obtained on the map;

[0092] Based on the map's inherent coordinate system, the obtained coordinate vectors, and the vehicle's position, the data is transformed into the vehicle's coordinate system to obtain the coordinate data of the center point on the map's road.

[0093] In step S200, the process of solving the coordinate transformation relationship and the vehicle heading vector from the center point coordinate data includes:

[0094] Obtain the Actor ID of the vehicle to get the current orientation vector of the vehicle, and return the orientation vector of the target based on the currently specified target.

[0095] In step S200, the lateral lane line compensation value is obtained based on the vehicle's facing vector. Based on this value, the edge points of the main road are obtained, and then the center points and related data of the left, left-left, right, and right-right sides of the map road are acquired, including:

[0096] By extending the compensation value to the left and right once on the center point data of the map road, the corresponding edge point of the current main road is obtained. In the iterative process of gradually obtaining the center point of the next map road, all discrete points of the edge line of the main road can be obtained in sequence.

[0097] Based on all discrete points along the edge of the main road, search to the left, left-left, right, and right-right respectively;

[0098] After obtaining the center point coordinates of the main road on the map, the compensation value is shifted in the negative direction by a preset multiple to obtain the coordinates of the center point that may exist on the left side of the map. Based on these coordinates, it is determined whether there is a corresponding center point on the map. If there is, all discrete points of the edge line of the left side of the road are obtained, and then the left and right sides of the road are calculated. If there is no such point, the calculation of the left and right sides of the road is skipped.

[0099] The right-hand road and the right-right road are calculated using the same method as the left-hand road.

[0100] In this step S200, the step of fitting a three-dimensional curve to the discrete points of the lane line using a multi-dimensional curve fitting method includes:

[0101] Fit the true value information of discrete points of the lane line; when the lane line is segmented, calculate the curve parameters for the straight and curved sections; take multiple discrete points at the end of the first curve segment and multiple discrete points at the beginning of the second curve segment to form a new data set, and calculate the average curvature of this data set. Use the average curvature to characterize the slope of the fitted curve, realize the curvature smoothing operation at the junction of the lane line segments, obtain the expression of the lane line, and construct a cubic curve model.

[0102] Points with variance distributions exceeding 3σ of the normal distribution are removed from the data. A least-squares fitting function is constructed based on the input data of the cubic curve model. The given fitting function is solved using the matrix coefficient equation to achieve the fitting of a three-dimensional curve to discrete points.

[0103] In this step S200, in order to perform data transmission and hardware matching, after obtaining the true value parameters of the current lane line through the three-dimensional curve, the following steps are also included:

[0104] The true value parameters of the lane lines and the center points of the acquired map roads are structured and organized, including:

[0105] The center point structure of a single map road includes: waypoint coordinates x, y, z, the road where the waypoint is located, the road type where the waypoint is located, the road width where the waypoint is located, the point, the lane line type corresponding to the waypoint, the lane line color corresponding to the waypoint, and the lane line width corresponding to the waypoint.

[0106] Lane line parameter structure, including: lane line output validity and lane line length.

[0107] In this step S300, sending the truth parameter of the current lane line means sending the prepared data, which is converted into a string type and sent in the form of a data stream.

[0108] This invention acquires data in real time. Therefore, once the true value parameters of the current lane line are sent, the current world snapshot is completed. Then, world snapshots are continuously taken, and the process of obtaining the current map information and the center point of the map road closest to the vehicle's position is repeated.

[0109] The final data from this invention can have multiple uses. For example, by comparing the true parameters of the current lane line with the sensor recognition results, the optimization direction of the sensor perception algorithm can be obtained; or the true parameters of the current lane line can be used as input to the planning and control module to participate in downstream path planning or access the navigation system.

[0110] The steps of each embodiment of the present invention will be described in detail below.

[0111] Example 1

[0112] In one embodiment of the present invention, the background of the basic hardware and system application software described below is as follows:

[0113] Computer hardware: Host: HP-TE01, equipped with CPU name: i5-12400F, quantity: 20, memory: 32GB; Graphics card: NVIDIA RTX-3080, quantity: 1, video memory: 12GB, Linux operating system Ubuntu 20.04 version;

[0114] Carla, as a development dependency of the lane line truth acquisition software mentioned in this invention, has a version no lower than 0.9.13, which was released in November 2021;

[0115] Carla's dependent component is Unreal Engine, version 4.26 or higher, which was released in December 2020;

[0116] like Figure 2This is a schematic diagram of the Carla client and server architecture. Carla consists of a client and a server. The server is implemented using C++ for technical functions such as information communication, while the client uses Python scripts to call the server's high-level function interfaces to present information to the user. The underlying architecture is mainly implemented by the user interface (UE). The method and software mentioned in this invention are based on the architecture between the underlying UE and the server. Typically, Carla calls data and API interfaces at the UE level, performs data serialization operations on the processing results in its own modules, and then sends them out from the server. The software architecture mentioned in this invention follows Carla's deployment model. This has the advantages of improving the stability of the embedded functions in the system, ensuring data security, not exposing core technical code to users, and improving running speed.

[0117] The architecture of the lane line truth value acquisition module is as follows: Figure 3 As shown: The User to Web part refers to the upstream receiving the instruction to obtain lane lines from the web page. This instruction is controlled by the node task manager. After confirming thread safety and normal communication establishment, it enters the Lane Detect part to obtain lane lines.

[0118] Lane Detect is the core method and software implemented in this application, which includes the following processes: World Tick (world snapshot), Lane Function (lane line acquisition), and Send Data (data reconstruction and transmission). A world snapshot refers to the operation of the sensor acquiring one accessible data point from the world, analogous to a "frame" in a conventional sensor. World snapshots and Send Data are part of the CALA's interconnected architecture, representing the computation of a single frame and the data reconstruction and transmission process, respectively.

[0119] After the Lane Detect results are sent, the Task Manager sends feedback to the web, and then to the user. In addition, the Task Manager also decides whether to save the data results to the Data section or to perform visualization rendering, i.e., the Visualization section.

[0120] like Figure 4 The following is a detailed flowchart of the Lane Detect software, which includes the following steps:

[0121] S1: In a world snapshot, the software extracts the current map information and the Waypoint, the center point of the map road closest to the vehicle's position, from the simulated world model. The Waypoint contains the point's ID, the lane it is in, and route information. This data comes from Carla's Open Drive module and is highly matched with the Carla world model.

[0122] Considering the possibility of abnormalities in the current vehicle's driving status, such as driving in the wrong direction or driving in wilderness or jungle, it is necessary to make a logical judgment on whether the center point of the road on the map has been obtained. If the validity is true, proceed to S2; if the validity is false, proceed to S3.

[0123] S2: After confirming that the center point of the map road closest to the vehicle is valid, enter the loop logic to search for the center point of the next map road. The loop logic is: set an adaptive forward search distance and search for the center point of the map road from the nearest to the farthest.

[0124] The adaptive forward search distance formula is as follows:

[0125]

[0126] in: For each adaptive distance, This is the initial forward search distance. This represents the number of iterations.

[0127] Determine the validity of the center point on the next map road. If it is true, proceed to S2.1; if it is false, proceed to S2.2. When the loop search reaches the set upper limit, exit the loop and proceed to S4.

[0128] S2.1: The center point of the next map road obtained in the loop is valid, and this data will be saved and recorded;

[0129] S2.2: The center point of the next map road obtained in the loop is invalid and this data will be discarded;

[0130] S3: If the center point of the obtained map road is invalid, then the environment for obtaining lane lines on the map is not available. Therefore, the validity of the step of obtaining the center point of the map road in the previous world snapshot is checked. If it is true and valid, proceed to S3.1; if it is false and invalid, proceed to S3.2.

[0131] S3.1: If the center point of the map road is valid in the previous world snapshot, then copy the data according to the data at the end of the previous world snapshot, complete the saving and output, and set the data of this world snapshot as valid;

[0132] S3.2: If the center point of the map road in the previous world snapshot is invalid, the output data structure will be initialized, the data will be saved and output, and the data of this world snapshot will be set to invalid.

[0133] S4: Figure 4 The dashed box portion falls under step S4, which is the core method of this invention for obtaining true lane line data. The method implementation flow is as follows:

[0134] S4.1: Obtain center point data on the map roads. The main steps include: S4.1.1 Solving for the coordinates of the center points on the map roads; S4.1.2 Solving for coordinate transformation relationships; S4.1.3 Solving for the vehicle's facing vector; S4.1.4 Obtaining the lateral lane line compensation value based on the vehicle's facing vector; S4.1.5 Obtaining the main road edge line points based on the compensation value; S4.1.6 Obtaining the center points and related data on the map roads from the left, left-left, right, and right-right sides of the vehicle. Figure 5 As shown;

[0135] S4.1.1 Solving for the center point coordinates on the map road uses the obtained Actor ID as an index. At this time, the Actor ID is the ID identifier of the center point on the current map road in the map, which is unique. The coordinate vector is obtained on the map by calling the interface of the UEOpen Drive module.

[0136] S4.1.2 Based on the Map's inherent coordinate system, and the coordinate vector and vehicle position obtained in S4.1.1, transform the data to the vehicle coordinate system;

[0137] S4.1.3 The process of solving the vehicle's orientation vector is similar to that in S4.1.1. The Actor ID of the vehicle is obtained, and the Forward method for solving the orientation in the UE's underlying layer is called to obtain the current vehicle's orientation vector. The Forward method returns the orientation vector of the target based on the currently specified target. This method is extremely efficient compared to ordinary matrix operations or trigonometric function operations.

[0138] S4.1.4 After obtaining the vehicle's orientation vector, calculate its unit vector and then calculate the lane width by projecting it onto the direction of the lane width. The projection calculation formula is as follows:

[0139]

[0140] in The vector after projection calculation, which includes components. These correspond to the compensation values ​​on each axis. This indicates the calculation direction of the three components, where... The axial direction remains unchanged. Rotate along the axis. The axis result has no effect on this vector, so it is treated as 0; This refers to the vector pointing towards the front of the vehicle; For the vehicle's heading vector The formula for determining the modulus of the axial component is as follows: L refers to the lane width.

[0141] The sum of the absolute values ​​of the two components of this compensation value is the lane width. Each component corresponds to the curvature and bending direction of the road. Therefore, when using this formula for calculation, it can effectively handle lane line position compensation calculations in situations such as turning and changing lanes.

[0142] S4.1.5 Based on S4.1.1 and S4.1.4, by extending the compensation value to the left and right once on the center point data of the map road, the corresponding edge point of the current main road can be obtained. In the iterative process of gradually obtaining the center point of the next map road, all discrete points of the edge line of the main road can be obtained in sequence.

[0143] S4.1.6 Based on the results of step S4.1.5, search to the left, left-left, right, and right-right respectively. Taking the left and left-left as examples, after obtaining the center point coordinates of the main road on the map, shift the compensation value by 3 times in the negative direction to obtain the coordinates of the center point that may exist on the left map road. Based on these coordinates, determine whether there is a corresponding center point on the map road. If there is, the operation steps are the same as other processes in S4, obtain all discrete points of the left road edge line, and then calculate the left-left road in the same way (the left-left road is compensated by 5 times the compensation value in the negative direction based on the center point coordinates of the map road); if there is no such point, return and skip the calculation of the left-left road; the right road is calculated in the same way as the right-right road (the right road is calculated by compensating by 3 times the compensation value in the positive direction based on the center point coordinates of the map road, and the right-right road is compensated by 5 times the compensation value in the positive direction).

[0144] Section S4.2, Curve Fitting, uses an improved least squares method to fit discrete points to a 3D curve. The basic process of this method is as follows:

[0145] 1. Consider the expression of the lane lines and construct a cubic curve model; 2. Construct a least squares problem using the input data; 3. Solve the given fitting function using the matrix coefficient equation.

[0146] This application's method further improves upon the aforementioned least squares method, effectively enhancing the validity of the results and increasing the aesthetic appeal of the visualizations. The improvements include:

[0147] 1): Curvature smoothing operation at the junction of lane segments. When lanes are segmented, such as straight sections with curves, curve parameters are calculated for the straight and curved sections separately. Four discrete points at the end of the first curve segment and four discrete points at the beginning of the second curve segment are taken to form a new data set. The average curvature of this data set is then calculated, and the average curvature is used to characterize the slope of the fitted curve.

[0148] 2): Data validity review. To obtain a more accurate model, before solving the least squares problem, points whose variance distribution exceeds 3σ of the normal distribution are removed to avoid these points affecting the original data set in the solution.

[0149] The S4.3 Reconstructing Data section restructures the lane line truth parameter data from S4.2 and the road, lane, and other attribute data obtained from S1, S2, and S3, and adapts them to the software architecture. It consists of two parts:

[0150] 1. Design data structures to adapt to the Lane Detect software architecture.

[0151] The designed data structure includes the following:

[0152] The structure of the center point on a separate map road includes: the coordinates of the waypoint (x, y, z), the road ID of the waypoint, the road type of the waypoint (road_type), the width of the road (road_width), the point, the lane type of the waypoint (lane_type), the lane color of the waypoint (lane_color), and the lane width of the waypoint (lane_width).

[0153] Table 1. Schematic diagram of center point structure on the map road

[0154]

[0155] The lane line parameter structure includes: the output validity of the lane line (lane_validity), the length and size of the lane line (section_size), and the 0th-order term (x0), 1st-order term (x1), 2nd-order term (x2), and 3rd-order term (x3) of the lane line parameters.

[0156] Table 2. Schematic diagram of lane line parameter structure

[0157]

[0158] 2. Interface design between Lane Detect software and Carla Server

[0159] The Lane Detect software sends prepared data to the Carla Server. All data is converted to string type via an interface and sent sequentially as a data stream, such as... Figure 6 As shown.

[0160] S5: Data transmission complete, current world snapshot complete. The software will continuously take world snapshots and repeat the above process after it runs.

[0161] Based on the test results in this example, the time consumed for data acquisition and transmission is less than 50 microseconds, which is much less than Carla's official rated frequency, such as 40Hz, or 25 milliseconds.

[0162] After the true lane line data is sent out, it can be compared with the sensor recognition results to quickly obtain the optimization direction of the sensor perception algorithm. It can also be used as the absolute input of the planning and control module to directly participate in the downstream path planning algorithm or be connected to the navigation system.

[0163] Therefore, through the description of specific embodiments of the present invention, it can be concluded that the present invention has the following advantages:

[0164] 1) A method and software for acquiring lane line ground truth data based on Carla are proposed. The software is deployed between the Carla underlying UE architecture and the server. This method aims to improve data authenticity and data security by not relying on sensor perception. It performs calculations and outputs results based on Open Drive, which has higher data accuracy than using sensors and achieves real-time updates, resulting in higher efficiency.

[0165] 2) The methods and software mentioned in this invention have high adaptability and strong portability on different systems, avoiding repeated compilation and repeated settings when using multiple terminals, thus improving the efficiency of multi-machine use;

[0166] 3) The lane line truth calculation method mentioned in this invention does not involve perception fusion or recognition algorithms, thus avoiding the introduction of additional system noise. The data has high authenticity and stability, and at the same time provides a control group for related perception algorithms and provides absolute truth values ​​for downstream modules.

[0167] 4) Compared with the implementation on the Carla Client, the method mentioned in this application does not change the software architecture and data transmission, and does not expose core data to users, making it more secure in terms of data storage and use;

[0168] 5) The various structural designs and process implementations mentioned in this method do not increase the complexity of the system while ensuring the completion of functions.

[0169] like Figure 7 As shown, the present invention provides a lane line truth value acquisition system, which mainly includes:

[0170] The acquisition module is used to obtain the current map information and the center point of the nearest road on the map to the vehicle's location;

[0171] The solution module is used to search for road points forward from the vehicle's location based on the center point of the road on the map, calculate the lateral compensation distance based on the vehicle's facing vector, and thus obtain the true value information of the discrete points of the lane line. Based on the true value information of the discrete points of the lane line, a three-dimensional curve is fitted to the discrete points using a multi-dimensional curve fitting method, and the true value parameters of the current lane line are obtained through the three-dimensional curve.

[0172] The sending module is used to send the true value parameters of the current lane line.

[0173] This system is based on the lane line truth acquisition method that takes into account the delay characteristics of multiple interactive functions. This system and method can be applied to, for example... Figure 2 The location of the lane line truth value acquisition module architecture. Figure 3 In Lane Detect.

[0174] like Figure 8 As shown, a third aspect of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the lane line truth value acquisition method taking into account the delay characteristics of multiple interactive functions.

[0175] A fourth aspect of the present invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method for obtaining the truth value of lane lines taking into account the delay characteristics of multiple interactive functions.

[0176] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for obtaining the true value of lane lines, characterized in that, include: Obtain the current map information and the center point of the nearest road on the map to the vehicle's location; Based on the center point of the road on the map, the system searches for road points forward from the vehicle's location. It calculates the lateral compensation distance based on the vehicle's facing vector, thereby obtaining the true value information of the discrete points of the lane line. The system then uses a multi-dimensional curve fitting method to fit a three-dimensional curve to the discrete points of the lane line, and obtains the true value parameters of the current lane line through the three-dimensional curve. Send the truth value parameter for the current lane line; The method of searching for road points forward from the vehicle's location based on the center point of the road on the map, and calculating the lateral compensation distance based on the vehicle's facing vector, includes: The system acquires center point data on the map roads, then solves for the center point coordinates on the map roads. From the center point coordinates, it solves for coordinate transformation relationships and vehicle heading vectors. Based on the vehicle heading vectors, it obtains lateral lane line compensation values, and based on the lateral lane line compensation values, it acquires the main road edge line points. Then, it acquires the center points and related data on the map roads from the left, left-left, right, and right-right sides of the vehicle. The process involves obtaining lateral lane line compensation values ​​based on the vehicle's orientation vector, acquiring main road edge points based on these compensation values, and then obtaining the center points and related data for the left, left-left, right, and right-right sides of the map roads, including: After obtaining the vehicle's orientation vector, calculate the unit vector of the vehicle's orientation vector, and then calculate the projection of the lane width onto the direction of the lane width. The projection calculation formula is as follows: in, The vector after projection calculation, which includes components. These correspond to the compensation values ​​on each axis. Indicates the calculation direction of the three components; This refers to the vector pointing towards the front of the vehicle; For the vehicle's heading vector The formula for determining the modulus of the axial component is as follows: L refers to lane width; The compensation value for the lateral lane lines is obtained by projection calculation; Extend the compensation value to the left and right once on the center point data of the map road to obtain the corresponding edge line point of the current main road. In the iterative process of gradually obtaining the center point of the next map road, obtain all discrete points of the edge line of the main road in sequence. Based on all discrete points along the edge of the main road, search to the left, left-left, right, and right-right respectively; After obtaining the center point coordinates of the main road on the map, the compensation value is shifted in the negative direction by a preset multiple to obtain the coordinates of the center point that may exist on the left side of the map. Based on these coordinates, it is determined whether there is a corresponding center point on the map. If there is, all discrete points of the edge line of the left side of the road are obtained, and then the left and right sides of the road are calculated. If there is no such point, the calculation of the left and right sides of the road is skipped. The right-hand road and the right-right road are calculated using the same method as the left-hand road.

2. The method for obtaining the true value of lane lines according to claim 1, characterized in that, The process of obtaining the current map information and the center point of the map road closest to the vehicle's position includes: In a world snapshot, obtain the current map information and the center point of the nearest map road to the vehicle's location, and determine whether the center point of the nearest map road is valid; If successful, proceed to the loop logic step of searching for the center point of the next map road. The loop logic step is: set an adaptive forward search distance, and search for the center point of the map road from near to far. If invalid, check the validity of the center point on the map road in the previous world snapshot. If valid, copy the data according to the data at the end of the previous world snapshot, complete the saving and output. If invalid, initialize the output data structure, complete the saving and output, and set the data of this world snapshot as invalid.

3. The method for obtaining the true value of lane lines according to claim 2, characterized in that, The adaptive forward search distance formula is as follows: in: For each adaptive distance, This is the initial forward search distance. This represents the number of loop iterations. After searching for the center point of the road on the map from near to far, the process also includes: determining the validity of the center point of the next map road; if valid, the data will be saved and recorded; if invalid, the data will be discarded.

4. The method for obtaining the true value of lane lines according to claim 1, characterized in that, The process of acquiring center point data on the map road, then solving for the center point coordinate data on the map road, and using the center point coordinate data to solve for coordinate transformation relationships and vehicle heading vectors includes: Using the obtained Actor ID as an index, the coordinate vector is obtained on the map; Based on the map's inherent coordinate system, and the obtained coordinate vectors and vehicle position, the data is transformed into the vehicle's coordinate system to obtain the coordinate data of the center point on the map's road. The process of solving the coordinate transformation relationship and the vehicle heading vector from the center point coordinate data includes: Obtain the Actor ID of the vehicle to get the current orientation vector of the vehicle, and return the orientation vector of the target based on the currently specified target.

5. The method for obtaining the true value of lane lines according to claim 1, characterized in that, The method of fitting a three-dimensional curve to discrete points based on the true value information of lane line discrete points using a multi-dimensional curve fitting method includes: Fit the true value information of discrete points of the lane line; when the lane line is segmented, calculate the curve parameters for the straight and curved sections; take multiple discrete points at the end of the first curve segment and multiple discrete points at the beginning of the second curve segment to form a new data set, and calculate the average curvature of this data set. Use the average curvature to characterize the slope of the fitted curve, realize the curvature smoothing operation at the junction of the lane line segments, obtain the expression of the lane line, and construct a cubic curve model. Points with variance distributions exceeding 3σ of the normal distribution are removed from the data. A least-squares fitting function is constructed based on the input data of the cubic curve model. The given fitting function is solved using the matrix coefficient equation to achieve the fitting of a three-dimensional curve to discrete points.

6. The method for obtaining the true value of lane lines according to claim 1, characterized in that, After obtaining the true parameters of the current lane line through the three-dimensional curve, the process also includes: The true value parameters of the lane lines and the center points of the acquired map roads are structured and organized, including: The center point structure of a single map road includes: waypoint coordinates x, y, z, the road where the waypoint is located, the road type where the waypoint is located, the road width where the waypoint is located, the point, the lane line type corresponding to the waypoint, the lane line color corresponding to the waypoint, and the lane line width corresponding to the waypoint. Lane line parameter structure, including: lane line output validity and lane line length.

7. The method for obtaining the true value of lane lines according to claim 1, characterized in that, Sending the truth value parameters of the current lane line refers to sending prepared data, which is converted into a string type and sent sequentially in the form of a data stream. After the truth parameters of the current lane line are sent, the current world snapshot is completed; then, world snapshots are continuously taken, returning to the step of obtaining the current map information and the center point of the map road closest to the vehicle's position; By comparing the true values ​​of the current lane lines with the sensor recognition results, we can determine the optimization direction of the sensor perception algorithm; or the true values ​​of the current lane lines can be used as input to the planning and control module to participate in downstream path planning or be integrated into the navigation system.

8. A lane line truth value acquisition system, implementing the lane line truth value acquisition method according to any one of claims 1 to 7, characterized in that, include: The acquisition module is used to obtain the current map information and the center point of the nearest road on the map to the vehicle's location; The solution module is used to search for road points forward from the vehicle's location based on the center point of the road on the map, calculate the lateral compensation distance based on the vehicle's facing vector, and thus obtain the true value information of the discrete points of the lane line. Based on the true value information of the discrete points of the lane line, a three-dimensional curve is fitted to the discrete points using a multi-dimensional curve fitting method, and the true value parameters of the current lane line are obtained through the three-dimensional curve. The sending module is used to send the true value parameters of the current lane line.

Images