Method, apparatus, vehicle and computer program product for optimizing lane line

Abstract

The invention provides a method and device for optimizing a lane line, a vehicle and a computer program product, and the method comprises the steps: aiming at a spatial point, which is commonly observed by a current frame lane line and at least one historical frame lane line in a common view area under a global coordinate system, of the current frame lane line and the at least one historical frame lane line; according to the observed multiple frames of lane lines of the spatial point and vehicle pose information corresponding to the multiple frames of lane lines, first coordinates of the spatial point in a global coordinate system are correspondingly converted into second coordinates in a vehicle body coordinate system; and taking the sum of the first deviation degrees of the second coordinates and the lane lines of the corresponding frames under the vehicle body coordinate system as a first residual error of the spatial point. And finally, based on the sum of the first residual errors of the spatial points observed by the current frame lane line and the at least one historical frame lane line in the common view area, optimizing the lane line point of the current frame lane line in the common view area and the vehicle pose information corresponding to the current frame lane line.

Description

Technical Field

[0001] This specification relates to the field of autonomous driving technology, and in particular to methods, devices, vehicles, and computer program products for optimizing lane lines. Background Art

[0002] Lane mapping involves collecting road information through sensors (such as cameras, lidar, and GPS) and combining it with computer vision and Simultaneous Localization and Mapping (SLAM) technology to generate a high-precision digital map that includes lane geometry, topological relationships, and semantic attributes. This map provides key support for autonomous driving, including lane-level positioning and path planning.

[0003] However, due to the probability of deviation in lane line recognition by the perception model, directly tracking lane lines based on the raw lane line information identified by the perception model will result in inaccurate lane line recognition, leading to tracking failure and affecting the accurate construction of the lane map. Furthermore, in scenarios with poor satellite signal quality, lane tracking also relies on estimated vehicle pose information, and positioning based directly on this estimated vehicle pose information will also have certain errors, further affecting the construction of the lane map. Summary of the Invention

[0004] To overcome the problems existing in the related art, this specification provides a method, device, vehicle and computer program product for optimizing lane lines.

[0005] According to a first aspect of an embodiment of this specification, a method for optimizing lane lines is provided, the method comprising:

[0006] Get the lane line point set corresponding to the vehicle's current frame lane line.

[0007] A common view area between the lane line of the current frame and the lane line of at least one historical frame in a global coordinate system is determined, where the common view area includes at least one lane line point in the lane line point set.

[0008] For each spatial point P in the common view area that is observed by the current frame lane line and the at least one historical frame lane line: determine a number of frame lane lines l that observe the spatial point P i and their corresponding vehicle posture information, according to the lane lines l i The first coordinate a of the spatial point P in the global coordinate system is converted into the second coordinates b in the vehicle body coordinate system. i , and the second coordinates b i Respectively with each frame lane line li The sum of the first deviations in the vehicle body coordinate system is taken as the first residual of the spatial point P; i is a positive integer.

[0009] Based on the sum of the first residuals of each spatial point P in the common view area observed by the current frame lane line and the at least one historical frame lane line, the vehicle posture information corresponding to the at least one lane line point and the current frame lane line is optimized.

[0010] According to a second aspect of an embodiment of this specification, a device for optimizing lane lines is provided, the device comprising:

[0011] The lane line point set acquisition module is used to obtain the lane line point set corresponding to the current frame lane line of the vehicle.

[0012] A common view area determination module is used to determine a common view area between the lane line of the current frame and at least one lane line of a historical frame in a global coordinate system, wherein the common view area includes at least one lane line point in the lane line point set.

[0013] The first residual calculation module is used to determine, for each spatial point P in the common view area that is observed by the current frame lane line and the at least one historical frame lane line: a number of frame lane lines l that observe the spatial point P i and their corresponding vehicle posture information, according to the lane lines l i The first coordinate a of the spatial point P in the global coordinate system is converted into the second coordinates b in the vehicle body coordinate system. i , and the second coordinates b i Respectively with each frame lane line l i The sum of the first deviations in the vehicle body coordinate system is taken as the first residual of the spatial point P; i is a positive integer.

[0014] An optimization module is used to optimize the vehicle posture information corresponding to the at least one lane line point and the current frame lane line based on the sum of the first residuals of each spatial point P observed jointly by the current frame lane line and the at least one historical frame lane line in the common view area.

[0015] According to a third aspect of an embodiment of this specification, a vehicle is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor implements the steps of the method described in the first aspect by running the executable instructions.

[0016] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, comprising: a computer program / instructions, which implement the steps of the method described in the first aspect when executed by a processor.

[0017] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0018] This solution targets each spatial point observed in the common view area between the current lane line and at least one historical lane line in the global coordinate system. Based on the multiple lane lines observed at that spatial point and their corresponding vehicle pose information, the first coordinate of that spatial point in the global coordinate system is converted to a corresponding second coordinate in the vehicle coordinate system. The sum of the first deviations between each second coordinate and the corresponding lane line in the vehicle coordinate system is used as the first residual of the spatial point. Finally, based on the sum of the first residuals of each spatial point observed in the common view area by the current lane line and at least one historical lane line, the lane line point of the current lane line in the common view area and the vehicle pose information corresponding to the current lane line are optimized.

[0019] It can be understood that this solution optimizes the lane line points of the current frame lane line in the common view area and the vehicle pose information corresponding to the current frame lane line by continuously reducing the deviation between the second coordinate of each spatial point and the corresponding frame lane line in the vehicle coordinate system. This approach can improve the accuracy of the position of the lane line points of the current frame lane line in the common view area and the vehicle pose information, thereby improving the accuracy of the constructed lane map.

[0020] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0022] Figure 1 This is a flowchart of a method for optimizing lane lines according to an exemplary embodiment of this specification.

[0023] Figure 2 This is a schematic diagram illustrating the association between the lane lines of the current frame and the lane lines of the historical frames according to an exemplary embodiment of this specification.

[0024] Figure 3 Schematic diagram of a co-viewing area between lane lines in a current frame and lane lines in a historical frame according to an exemplary embodiment of the present specification.

[0025] Figure 4 This is a schematic diagram of the correspondence between a spatial point and a lane line observing the spatial point according to an exemplary embodiment of this specification.

[0026] Figure 5Schematic diagram of the point-to-line distance between a spatial point P and a line connecting q1 and q2 according to an exemplary embodiment of the present specification.

[0027] Figure 6 It is a structural diagram of an electronic device according to an exemplary embodiment of this specification.

[0028] Figure 7 This is a block diagram of a device for optimizing lane lines according to an exemplary embodiment of the present specification. DETAILED DESCRIPTION

[0029] Lane mapping involves collecting road information through sensors (such as cameras, lidar, and GPS) and combining it with computer vision and Simultaneous Localization and Mapping (SLAM) technology to generate a high-precision digital map that includes lane geometry, topological relationships, and semantic attributes. This map provides key support for autonomous driving, including lane-level positioning and path planning.

[0030] However, due to the probability of deviation in lane line recognition by the perception model, directly tracking lane lines based on the raw lane line information identified by the perception model will result in inaccurate lane line recognition, leading to tracking failure and affecting the accurate construction of the lane map. Furthermore, in scenarios with poor satellite signal quality, lane tracking also relies on estimated vehicle pose information, and positioning based directly on this estimated vehicle pose information will also have certain errors, further affecting the construction of the lane map.

[0031] In response to the above technical problems, this specification provides a method for optimizing lane lines, which improves the accuracy of constructing a lane line map by optimizing the lane line point set of the current frame lane line and the vehicle posture information corresponding to the current frame lane line.

[0032] Figure 1 This is a flow chart of a method for optimizing lane lines according to an exemplary embodiment of this specification. Figure 1 As shown, it includes steps 101-104:

[0033] Step 101: Obtain a lane line point set corresponding to the current frame lane line of the vehicle.

[0034] The lane line in the current frame can be any lane line information identified by the vehicle at the current moment from real-time image data and / or point cloud data collected by sensors (such as cameras and lidar). A lane line point set can be the position of a lane line described by a series of two-dimensional or three-dimensional coordinate points (i.e., "lane line points"), each of which can correspond to a specific physical location in the global coordinate system.

[0035] In one embodiment, when obtaining the lane line point set corresponding to the current frame lane line of the vehicle, the lane line point set identified from the road data collected from the vehicle can be obtained, or the lane line point set can be determined from the equation for fitting the lane line corresponding to the current frame lane line of the vehicle.

[0036] Specifically, by obtaining the original lane line point set identified by the perception model, this method can be used to optimize the original lane line point set, and construct the lane line equation based on the optimized lane line point set.

[0037] Of course, for lane equations already constructed in related technologies that don't utilize this method, this method can still be used to further optimize the relevant parameters in the lane equations. Specifically, this method can be used to re-determine the lane point set based on the relevant parameters of the lane equations, and then optimize the lane point set using this method to update the relevant parameters of the lane equations. Therefore, regardless of whether the original lane point set identified by the perception model is optimized when constructing the lane equations in related technologies, this method can be used to further optimize the lane equations constructed in related technologies.

[0038] Step 102: Determine a common view area between the lane line of the current frame and at least one lane line of a historical frame in a global coordinate system, wherein the common view area includes at least one lane line point in the lane line point set.

[0039] Vehicle pose information is a comprehensive parameter describing the vehicle's translation and rotation in three-dimensional space. In areas with poor satellite signal quality, the continuously calculated changes in the vehicle's pose can be used to recursively infer vehicle positioning. However, this recursive positioning based on the vehicle's pose changes can lead to a certain degree of cumulative error over long periods of operation. This cumulative error can be reduced to a certain extent by optimizing the calculated vehicle pose information before use.

[0040] Specifically, the vehicle's angular velocity and speed data collected by the vehicle's inertial measurement unit (IMU) and wheel odometry can be obtained, and the vehicle's current position information can be calculated based on the vehicle's position information calculated at the previous moment.

[0041] For example, the timestamp of the IMU and the timestamp of the wheel speed meter can be matched to ensure that they are output at the same time. Then, the IMU provides angular velocity observations, and the wheel speed meter provides speed observations. Between the two frames of the wheel speed meter, the angular velocity is integrated. At the same time, it is assumed that the vehicle is in a uniform motion state during this time difference, and the position and angle are obtained by integration. Specifically, referring to formula 1-2, the rotation P of the vehicle relative to the global coordinate system in the current frame can be calculated. k and pan R k :

[0042]

[0043] R k =R k-1 ExP((w k -bias)Δt) Formula 2

[0044] Among them, P k-1 is the rotation of the vehicle at the previous moment, V k-1 is the speed at the previous moment, Δt is the time difference between the previous moment and the current moment, R k-1 is the displacement of the vehicle relative to the global coordinate system at the previous moment, a k is the acceleration of the vehicle at the current moment, bias is the zero bias of the IMU, g is the acceleration of gravity, and w k is the angular velocity of the vehicle at the current moment, and ExP(*) is the exponential mapping that converts Lie algebra into Lie group (rotation matrix).

[0045] For the calculated rotation P k and pan R k After complementary filtering, the vehicle position information of the current moment relative to the global coordinate system is output. The vehicle position information includes the rotation P k and pan R k Among them, in the complementary filtering process, it is assumed that if acceleration is used, the acceleration information is used to observe the rotation information. The main observation model assumes that the two frames of IMU are in uniform motion (that is, the lateral acceleration and forward acceleration of the IMU are 0), and the extended Kalman filter is used for complementary filtering.

[0046] The lane line point set of the current frame lane line is converted from the vehicle body coordinate system to the global coordinate system using the vehicle posture information corresponding to the lane line of the current frame, and the lane line of the historical frame associated with the lane line of the current frame is determined in the global coordinate system. For example, Figure 2 As shown, the lane lines in the current frame are 4, 5, and 6, while the lane lines in the historical frame are 1, 2, and 3. It can be determined that lane line 4 is associated with lane line 1, lane line 5 is associated with lane line 2, and lane line 6 is associated with lane line 3.

[0047] In order to determine the lane line of the historical frame associated with the lane line of the current frame in the global coordinate system, for each lane line point P0 on the lane line of the current frame, the lane line point P closest to P0, the nearby P1 and P2 are determined on each lane line of the historical frames. For example, the lane line point P can be the previous index P1 and the next index P2. Construct P , The distance cost of the line connecting the two lane line points P1 and P2 can eventually obtain the cost matrix of the current frame lane line relative to each historical frame lane line. This cost matrix can be determined by the distance cost corresponding to each lane line point P0 on the current frame lane line. For the cost matrices corresponding to each historical frame lane line, Hungarian matching is used to determine the historical frame lane line associated with the current frame lane line.

[0048] The common view area can be understood as the region where the current frame lane line and the historical frame lane line observe the same spatial point in the global coordinate system. To determine the common view area between the current frame lane line and the historical frame lane line in the global coordinate system, the distance between the lane line point of the current frame lane line and the corresponding lane line point of the associated historical frame lane line can be determined in the global coordinate system. The region containing lane line points whose distance is less than a distance threshold is considered the common view area.

[0049] For example, Figure 3 As shown in , for the lane line of the current frame and its associated lane line of the historical frame, the distance between the corresponding lane line points of the two can be determined, and the area where the lane line points with a distance less than the distance threshold are located is taken as the common view area, that is, Figure 3 In the common view area 30, for each lane line point on the lane line of the current frame, the distance between it and the corresponding lane line point on the lane line of the historical frame is less than the distance threshold.

[0050] Step 103: For each spatial point P in the common view area that is observed by the current frame lane line and the at least one historical frame lane line: determine the number of frame lane lines l that observe the spatial point P. i and their corresponding vehicle posture information, according to the lane lines l i The first coordinate a of the spatial point P in the global coordinate system is converted into the second coordinates b in the vehicle body coordinate system. i , and the second coordinates b i Respectively with each frame lane line l i The sum of the first deviations in the vehicle body coordinate system is taken as the first residual of the spatial point P; i is a positive integer.

[0051] Step 104: Based on the sum of the first residuals of each spatial point P in the common view area observed by the current frame lane line and the at least one historical frame lane line, optimize the vehicle posture information corresponding to the at least one lane line point and the current frame lane line.

[0052] This solution, for each spatial point observed in the common view area between the current lane line and at least one historical lane line in the global coordinate system, converts the first coordinate of each spatial point in the global coordinate system to a corresponding second coordinate in the vehicle coordinate system based on the multiple lane lines and their corresponding vehicle pose information. The sum of the first deviations between each second coordinate and the corresponding lane line in the vehicle coordinate system is used as the first residual for the spatial point. Finally, based on the sum of the first residuals for each spatial point observed in the common view area by the current lane line and at least one historical lane line, the lane point of the current lane line in the common view area and the vehicle pose information corresponding to the current lane line are optimized. By continuously reducing the deviation between the second coordinate of each observed spatial point and the corresponding lane line in the vehicle coordinate system, the lane point of the current lane line in the common view area and the vehicle pose information corresponding to the current lane line are optimized. This improves the accuracy of lane map construction by increasing the accuracy of the position of the lane point of the current lane line in the common view area and the vehicle pose information.

[0053] In the common view area, there may be a situation where the same spatial point P is observed by the lane line of the current frame and at least one lane line of the historical frame at the same time. Figure 4 As shown, for a spatial point P in the map real , which may be observed by the current frame image 40 taken by the vehicle at the current moment and the previous frame image 41 taken at the previous moment (of course, it is not limited to being observed by two frames of images, the number can be greater than 2), that is, P real P on road line 401 and road line 411 respectively observe Corresponding. Through the vehicle posture information corresponding to each frame image, P real The corresponding transformation is to P* in the vehicle coordinate system or image coordinate system where the road line 401 and the road line 402 are located. Due to the recognition error of the perception model or the calculation error of the vehicle posture information, P* is not completely consistent with P observe By continuously reducing the deviation between P* and lane line 401 or lane line 402, the vehicle posture information and lane line point P corresponding to each frame image can be optimized. observe In this solution, the vehicle posture information corresponding to the current frame lane line in the current frame image and the P position of the current frame lane line in the common view area are optimized. observe .

[0054] Based on the above principle, for each spatial point P in the common view area that is observed by the current frame lane line and at least one historical frame lane line, the spatial point P is converted from the global coordinate system to the vehicle body coordinate system using the vehicle pose information corresponding to the lane lines in the frames (including the current frame lane line and at least one historical frame lane line) that observe the spatial point P. Ideally, the spatial point P converted to the vehicle body coordinate system using the vehicle pose information corresponding to any frame lane line should be located on that frame lane line, that is, the deviation from the frame lane line is zero. Therefore, the sum of the first deviations of the spatial point P converted to the body coordinate system and the corresponding frame lane line in the vehicle coordinate system can be used as the first residual of the spatial point P. By continuously reducing the first residual, the optimized vehicle pose information corresponding to the current frame lane line and the lane line point corresponding to the spatial point can be determined.

[0055] It should be noted that the above description is the calculation logic of the first residual for any spatial point P. The above logic can be used to calculate the first residual for each spatial point P in the common view area. Finally, based on the sum of the first residuals of each spatial point P, the lane line point of the current frame lane line in the common view area and the vehicle posture information corresponding to the current frame lane line can be optimized.

[0056] In one embodiment, for any spatial point P in the common view area that is observed by n frames of lane lines (including the current frame lane lines and the historical frame lane lines), the n frames of lane lines l that observe the spatial point P are determined. i and their corresponding vehicle posture information, converting the first coordinate a of the spatial point P in the global coordinate system into the second coordinates b in the vehicle coordinate system i , and determine the lane line l of each frame in the global coordinate system i The two lane line points closest to the spatial point P and And each second coordinate b in the vehicle coordinate system i Respectively with each frame lane line l i Two lane line points on and The sum of the point-to-line distances between the connecting lines is taken as the first residual of the spatial point P. For details, see Formula 3 and Formula 4:

[0057]

[0058] b i =R i P+t i Formula 4

[0059] Among them, R i represents the rotation information of the vehicle posture information corresponding to the lane line of frame i, t iIndicates the translation information in the vehicle pose information corresponding to the lane line in frame i. and Represents two lane line points on the lane line of frame i.

[0060] like Figure 5 As shown in the figure, for two lane points q1 and q2 on any lane line in a frame, the first residual r can be expressed as the point-to-line distance between the spatial point P and the line connecting q1 and q2. In this embodiment, the residual is calculated as the point-to-line distance between the spatial point P and the line connecting q1 and q2, rather than the point-to-point distance between the spatial point P and the corresponding lane point on the lane line. This point-to-line distance more naturally expresses the geometric characteristics of the lane line, thereby improving the stability of the optimization.

[0061] In one embodiment, in addition to measuring the deviation between the spatial point P and the corresponding frame lane line in the vehicle coordinate system, the deviation between the spatial point P and the corresponding frame lane line can also be measured in the image coordinate system. Specifically, according to the number of frame lane lines l i The corresponding vehicle posture information is converted from the first coordinate a of the spatial point P in the global coordinate system to the third coordinate c in the image coordinate system. i , and each third coordinate c i Respectively with each frame lane line l i The sum of the second deviations in the image coordinate system is used as the second residual of the spatial point P. Based on the sum of the second residuals of each spatial point P observed in the common view area by the current frame lane line and at least one historical frame lane line, the lane line point of the current frame lane line in the common view area and the vehicle posture information corresponding to the current frame lane line are optimized.

[0062] In one embodiment, when each third coordinate c i Respectively with each frame lane line l i When the sum of the second deviations in the image coordinate system is used as the second residual of the spatial point P, the lane line l of each frame in the global coordinate system can be determined separately. i The two lane line points closest to the spatial point P on the image coordinate system are i Respectively with each frame lane line l i The sum of the point-line distances between the two lane line points on is taken as the second residual of the spatial point P.

[0063] In this embodiment, the weight of the residual on the XY axis can be increased by converting the spatial point P into the image coordinate system to measure the deviation between the spatial point P and the corresponding frame lane line.

[0064] In one embodiment, in order to address the situation where negative optimization occurs in the vehicle posture information corresponding to the lane line of the current frame based on the first residual and the second residual due to perception errors, and to further improve the effect of optimizing the vehicle posture information corresponding to the lane line of the current frame based on the first residual and the second residual, a residual can be made between the prior posture information and the posture information variable to ensure that the deviation between the posture before and after optimization is not too large, that is, to ensure that it is not greatly affected by erroneous perception.

[0065] Specifically, the priori pose information corresponding to the lane line of the current frame can be obtained (specifically, the P k and R k ), the distance between the priori pose information and the pose information variable is used as the third residual, and the vehicle pose information corresponding to the lane line of the current frame is optimized based on the third residual. The vehicle pose information may include the vehicle rotation R and the vehicle displacement t, and the third residual may include the residual between the priori rotation and the rotation variable and the residual between the priori displacement and the displacement variable. Specifically, the priori rotation R init and the rotation variable R optimi The distance between them is the residual res q and the prior displacement t init and displacement variable t optimi The distance between them is the residual res t , see formula 4-5 for details:

[0066]

[0067] res t =t optimi -t init Formula 5

[0068] In one embodiment, in general driving scenarios, lane lines are relatively smooth, whether they are straight roads or curves. In order to cope with the jagged and confusing lane line points that appear in perception, an assumption is made about the line shape of the lane lines, that is, it is assumed that the lane lines are evenly distributed in the horizontal and vertical directions, and there will not be too large a gap in the horizontal and vertical directions.

[0069] Therefore, residuals can be set according to the smoothness of the lane lines. Specifically, for each lane line point in the middle position in the lane line point set to be optimized, the third deviation between the lane line point and the adjacent lane line points on both sides is used as the fourth residual, and the lane line points in the lane line point set to be optimized are optimized based on the fourth residual corresponding to each lane line point.

[0070] For example, for any middle lane line point P in the lane line point set of the current frame lane line, i , according to the lane line point Pi The index of the lane line point P at its previous position is determined i-1 and the lane line point P at the next position i+1 , based on the lane line point P i Respectively with P i-1 and P i+1 The third deviation is used as the fourth residual, where the lane line point P i Respectively with P i-1 and P i+1 The third deviation can be expressed as (P i+1 -P i )-(P i -P i-1 ) indicates that the fourth residual r i The formula can be found in Formula 6:

[0071] r i =(P i+1 -P i )-(P i -P i-1 ) Formula 6

[0072] By optimizing the lane line point set to be optimized in the lane line of the current frame by using the fourth residual, the smoothness of the lane line point distribution in the lane line of the current frame can be improved, thereby improving the accuracy of constructing the lane line map.

[0073] For the residual optimization method of any of the above settings, the lane line point set of the current frame lane line and the vehicle posture information corresponding to the current frame lane line can be used as optimization variables and optimized using a nonlinear optimization library.

[0074] After optimization is complete, lane line points in the optimized lane line point set for the current frame can be fitted to obtain a lane line equation, and a lane line map can be constructed based on the lane line equation. The fitting method can be a least squares method using a cubic function model, and the smoothing method can be a smoothing method using an average method by rotating the polynomial coefficients of at least one lane line frame near the current frame.

[0075] Corresponding to the aforementioned method embodiments, this specification also provides embodiments of an apparatus and a terminal to which it is applied.

[0076] Figure 6 This is a structural diagram of an electronic device according to an exemplary embodiment of the present specification. Figure 6As shown, at the hardware level, the electronic device 600 includes a processor 602, an internal bus 604, a network interface 606, a memory 608, and a non-volatile memory 610, and may also include hardware required for other services. One or more embodiments of this specification can be implemented based on software, such as the processor 602 reading the corresponding computer program from the non-volatile memory 610 into the memory 608 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic module, but can also be hardware or logic devices.

[0077] Figure 7 This is a block diagram of a device for optimizing lane lines according to an exemplary embodiment of this specification. Figure 7 As shown, the device can be used in Figure 6 The electronic device 600 shown in the figure is used to implement the technical solution of this specification. The device includes:

[0078] The lane line point set acquisition module 702 is used to acquire a lane line point set corresponding to the lane line of the current frame of the vehicle;

[0079] The common view area determination module 704 is configured to determine a common view area between the lane line of the current frame and at least one lane line of the historical frame in a global coordinate system, wherein the common view area includes at least one lane line point in the lane line point set.

[0080] The first residual calculation module 706 is used to determine, for each spatial point P in the common view area that is observed by the current frame lane line and the at least one historical frame lane line: i and their corresponding vehicle posture information, according to the lane lines l i The first coordinate a of the spatial point P in the global coordinate system is converted into the second coordinates b in the vehicle body coordinate system. i , and the second coordinates b i Respectively with each frame lane line l i The sum of the first deviations in the vehicle body coordinate system is taken as the first residual of the spatial point P; i is a positive integer.

[0081] The optimization module 708 is used to optimize the vehicle posture information corresponding to the at least one lane line point and the current frame lane line based on the sum of the first residuals of each spatial point P observed jointly by the current frame lane line and the at least one historical frame lane line in the common view area.

[0082] Optionally, the first residual calculation module 706 is specifically configured to determine the lane line l of each frame in the global coordinate system. i The two lane line points closest to the spatial point P are located on the vehicle body coordinate system, and the second coordinates b are i Respectively with each frame lane line l i The sum of the point-to-line distances between the two lane line points on is taken as the first residual of the spatial point P.

[0083] Optionally, the device further includes a second residual calculation module for calculating the lane line l of the plurality of frames. i The corresponding vehicle posture information is converted into the first coordinate a of the space point P in the global coordinate system and the third coordinate c in the image coordinate system. i , and the third coordinates c i Respectively with each frame lane line l i The sum of the second deviations in the image coordinate system is used as the second residual of the spatial point P.

[0084] Optionally, the optimization module 708 is also used to optimize the vehicle posture information corresponding to the at least one lane line point and the current frame lane line based on the sum of the second residuals of each spatial point P in the common view area observed jointly by the current frame lane line and the at least one historical frame lane line.

[0085] Optionally, the device further includes a third residual calculation module, which is used to obtain the priori posture information corresponding to the lane line of the current frame, and use the distance between the priori posture information and the posture information variable as the third residual.

[0086] The optimization module 708 is further configured to optimize the vehicle posture information corresponding to the lane line of the current frame based on the third residual.

[0087] Optionally, the device further includes a fourth residual determination module for determining, for each lane point in the middle position in the lane point set to be optimized, a fourth residual based on a third deviation between the lane point and the adjacent lane points on both sides.

[0088] The optimization module 708 is further configured to optimize the lane line points in the set of lane line points to be optimized based on the fourth residual corresponding to each lane line point.

[0089] Optionally, the common view area determination module 704 is specifically used to determine the distance between the lane line point of the current frame lane line and the corresponding lane line point of its associated historical frame lane line in the global coordinate system, and take the area where the lane line point whose distance is less than the distance threshold is located as the common view area.

[0090] Optionally, the lane line point set acquisition module 702 is specifically used to determine the lane line point set from the equation for fitting the lane line corresponding to the current frame lane line of the vehicle; or, to obtain the lane line point set identified from the road data collected from the vehicle.

[0091] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0092] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0093] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the aforementioned methods for optimizing lane lines provided in this application.

[0094] Specifically, computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks.

[0095] This specification also provides a computer program product, including a computer program / instruction, which implements the steps of any of the aforementioned methods for optimizing lane lines when executed by a processor.

[0096] This specification also provides a vehicle, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the steps of any of the aforementioned methods for optimizing lane lines by running the executable instructions.

Claims

1. A method for optimizing lane lines, characterized in that: The method comprises: Get the lane line point set corresponding to the vehicle's current frame lane line; Determine a common view area between the lane line of the current frame and at least one lane line of a historical frame in a global coordinate system, wherein the common view area includes at least one lane line point in the lane line point set; For each spatial point P in the common view area that is observed by the current frame lane line and the at least one historical frame lane line: determine a number of frame lane lines l that observe the spatial point P i and their corresponding vehicle posture information, according to the lane lines l i The first coordinate a of the spatial point P in the global coordinate system is converted into the second coordinates b in the vehicle body coordinate system. i , and the second coordinates b i Respectively with each frame lane line l i The sum of the first deviations in the vehicle coordinate system is taken as the first residual of the spatial point P; i is a positive integer; Based on the sum of the first residuals of each spatial point P in the common view area observed by the current frame lane line and the at least one historical frame lane line, the vehicle posture information corresponding to the at least one lane line point and the current frame lane line is optimized.

2. The method according to claim 1, characterized in that The second coordinates b i Respectively with each frame lane line l i The sum of the first deviations in the vehicle body coordinate system is used as the first residual of the spatial point P, including: Determine the lane line l of each frame in the global coordinate system i The two lane line points closest to the spatial point P are located on the vehicle body coordinate system, and the second coordinates b are i Respectively with each frame lane line l i The sum of the point-to-line distances between the two lane line points on is taken as the first residual of the spatial point P.

3. The method according to claim 1, characterized in that The method further comprises: According to the lane lines l i The corresponding vehicle posture information is converted into the first coordinate a of the space point P in the global coordinate system and the third coordinate c in the image coordinate system. i , and the third coordinates c i Respectively with each frame lane line l i The sum of the second deviations in the image coordinate system is used as the second residual of the spatial point P; Based on the sum of the second residuals of each spatial point P in the common view area observed by the current frame lane line and the at least one historical frame lane line, the vehicle posture information corresponding to the at least one lane line point and the current frame lane line is optimized.

4. The method according to claim 1, wherein The method further comprises: Obtain prior pose information corresponding to the lane line of the current frame, use the distance between the prior pose information and the pose information variable as a third residual, and optimize the vehicle pose information corresponding to the lane line of the current frame based on the third residual.

5. The method according to claim 1, wherein The method further comprises: For each lane line point in the middle position in the lane line point set to be optimized, the third deviation between the lane line point and the adjacent lane line points on both sides is used as the fourth residual, and the lane line points in the lane line point set to be optimized are optimized based on the fourth residual corresponding to each lane line point.

6. The method according to claim 1, characterized in that Determining a common view area between the lane line of the current frame and the lane line of at least one historical frame in a global coordinate system includes: The distance between the lane line point of the current frame lane line and the corresponding lane line point of the associated historical frame lane line is determined in the global coordinate system, and the area where the lane line point with the distance less than the distance threshold is located is used as the common view area.

7. The method according to claim 1, characterized in that The obtaining of a lane line point set corresponding to a lane line of a current frame of the vehicle includes: Determine the lane line point set from the equation for fitting the lane line corresponding to the current frame lane line of the vehicle; or, A lane line point set identified from road data collected by the vehicle is obtained.

8. A device for optimizing lane lines, characterized in that: The device comprises: A lane line point set acquisition module is used to obtain a lane line point set corresponding to the lane line of the vehicle in the current frame; a common view area determination module, configured to determine a common view area between the lane line of the current frame and the lane line of at least one historical frame in a global coordinate system, wherein the common view area includes at least one lane line point in the lane line point set; The first residual calculation module is used to determine, for each spatial point P in the common view area that is observed by the current frame lane line and the at least one historical frame lane line: a number of frame lane lines l that observe the spatial point P i and their corresponding vehicle posture information, according to the lane lines l i The first coordinate a of the spatial point P in the global coordinate system is converted into the second coordinates b in the vehicle body coordinate system. i , and the second coordinates b i Respectively with each frame lane line l i The sum of the first deviations in the vehicle coordinate system is taken as the first residual of the spatial point P; i is a positive integer; An optimization module is used to optimize the vehicle posture information corresponding to the at least one lane line point and the current frame lane line based on the sum of the first residuals of each spatial point P observed jointly by the current frame lane line and the at least one historical frame lane line in the common view area.

9. A vehicle, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer program product, characterized in that include: A computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images