A lane line association registration method, device, equipment and storage medium

Abstract

This application provides a lane line association registration method, apparatus, device, and storage medium, including: acquiring a visual lane line point cloud and a map lane line point cloud; transforming the visual lane line point cloud to a global plane coordinate system using a transformation matrix; searching for visual lane line points in the visual lane line point cloud; searching for line segments in the map lane line point cloud; combining the searched line segments with the corresponding visual lane line points to form search point-line pairs; determining whether the search point-line pairs meet matching conditions; and when the matching conditions are met, performing a registration transformation on the transformation matrix to ensure that the visual lane line points in the visual lane line point cloud have matching line segments in the map lane lines. This application adopts a point-line pair matching registration method, which performs fuzzy matching to a certain extent, resulting in better registration accuracy and faster convergence speed for the two point clouds with different distributions: the visual lane line point cloud and the lane line points in the high-precision map. This method can handle scenarios where map points are relatively sparse.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a lane line association registration method and device, equipment and a storage medium. BACKGROUND

[0002] Lane boundary line (referred to as lane line) is a traffic marking used to separate traffic flow in the same direction, which is generally white dotted, solid line or yellow dotted, solid line. The white dotted line separates the same direction vehicles, and can change lanes and overtake in safe conditions. The white solid line also separates the same direction vehicles, but cannot change lanes. The yellow solid line separates vehicles driving in different directions, and sometimes separates vehicles driving in the same direction, which can be used as a boundary line or a center line, and cannot change lanes. The yellow dotted line can be used as a boundary line or a center line, and can change lanes when used as a boundary line.

[0003] At present, when the visual lane line and the map lane line are associated and registered, the existing method is generally as follows: collecting the road lane line cubic curve, estimating the vehicle position according to the curve coincidence degree and similarity. Then, sample points are configured around the estimated position and the next time position is predicted. However, the existing method will introduce errors when fitting the map points into a cubic curve, which will cause partial distortion. If distortion occurs, the number of samples will change, and the accuracy of estimating the vehicle pose is related to the number of configured sample points. Fewer sample points will increase the error, and more sample points will increase the calculation cost. At the same time, when the ICP algorithm is used for matching, it is necessary to find a one-to-one matching point pair on the lane line and the local map, which has high requirements for point selection and error convergence, and will affect the error of the matching result. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the present application provides a lane line association registration method, device, equipment and storage medium to solve the above-mentioned technical problems.

[0005] The present application provides a lane line association registration method, comprising the following steps:

[0006] detecting lane lines from pre-shooting or real-time shooting road images, denoted as visual lane lines; and sampling the visual lane lines to obtain visual lane line point clouds;

[0007] obtaining the latitude and longitude of the lane lines from pre-extracted or real-time extracted map data to generate map lane lines; and converting the map lane lines into global plane coordinate points to obtain map lane line point clouds;

[0008] transforming the visual lane line point cloud into the global plane coordinate system by using the transformation matrix obtained in advance or in real time, searching for visual lane line points in the visual lane line point cloud, searching for a line segment from the map lane line point cloud, and combining the searched line segment and the corresponding visual lane line point into a search point-line pair;

[0009] determining whether the search point-line pair meets a preset matching condition, and performing registration transformation on the transformation matrix to make each visual lane line point in the visual lane line point cloud have a matching line segment in the map lane line when the search point-line pair meets the preset matching condition.

[0010] In an embodiment of the present application, the process of determining whether the search point-line pair meets the preset matching condition includes:

[0011] obtaining a first visual lane line point in the visual lane line point cloud, and searching for a line segment associated with the first visual lane line point from the map lane line point cloud, denoted as a first map line segment;

[0012] obtaining a projection point of the first visual lane line point to a direct projection point of the first map line segment, and determining whether the projection point is on the first map line segment; and,

[0013] calculating a distance of the first visual lane line point to the first map line segment, and determining whether the distance is less than a maximum matching threshold; and,

[0014] calculating an included angle of the visual lane line and the first map line segment, and determining whether the included angle is less than a maximum angle threshold;

[0015] if the projection point is on the first map line segment, the distance is less than the maximum matching threshold, and the included angle is less than the maximum angle threshold, it is determined that the search point-line pair meets the preset matching condition, and constitutes a matching point-line pair;

[0016] if the projection point is not on the first map line segment, the distance is greater than or equal to the maximum matching threshold, and / or the included angle is greater than or equal to the maximum angle threshold, it is determined that the search point-line pair does not meet the preset matching condition, and does not constitute a matching point-line pair;

[0017] wherein the search point-line pair is formed by the first visual lane line point and the first map line segment; the maximum matching threshold and the maximum angle threshold are obtained according to actual engineering values.

[0018] In an embodiment of the present application, the process of performing registration transformation on the transformation matrix includes:

[0019] The distance from the first visual lane line point to the first map line segment is defined as the matching error;

[0020] An error equation is constructed based on the coordinates of the first visual lane line points and the straight line equation of the first map line segment;

[0021] Calculate the registration transformation matrix that minimizes the sum of squares of all matching errors based on the error equation.

[0022] In one embodiment of this application, the process of obtaining the transformation matrix includes:

[0023] Obtain the target vehicle's latitude and longitude, heading, wheel speed, and yaw rate;

[0024] Based on the latitude and longitude information, heading information, wheel speed information, and yaw rate, the pose of the target vehicle in the global plane coordinate system is determined;

[0025] The transformation matrix is ​​calculated based on the pose of the target vehicle in the global plane coordinate system.

[0026] In one embodiment of this application, the process of calculating the transformation matrix based on the pose of the target vehicle in the global plane coordinate system includes:

[0027] The pose of the target vehicle in the global plane coordinate system is denoted as P(t). x , t y ,θ);

[0028] Let the transformation matrix be denoted as T. w_v Then, based on the pose of the target vehicle in the global plane coordinate system, the transformation matrix is ​​calculated as follows:

[0029]

[0030] In the formula, T w_v This represents the transformation matrix.

[0031] In one embodiment of this application, the process of constructing an error equation based on the coordinates of the first visual lane line point and the straight line equation of the first map line segment includes:

[0032] Obtain the coordinates of the first visual lane line point, denoted as (x i y i );as well as,

[0033] Obtain the equation of the straight line of the first map line segment, denoted as A. i x+B i y+C i =0;

[0034] Based on the coordinates of the first visual lane line points and the straight line equation of the first map line segment, an error equation is constructed:

[0035]

[0036] In the formula, e i Representing the error equation, A i B i C i Let be a constant, and i be a natural number.

[0037] In one embodiment of this application, the process of capturing the road image includes: pre-setting an image capturing device on a target vehicle, and using the image capturing device to capture the road within a preset distance range from the target vehicle while the target vehicle is in motion or stationary, thereby obtaining the road image; wherein the target vehicle includes at least one of the following: a new energy vehicle or a fuel vehicle.

[0038] This application also provides a lane line association registration device, the device comprising:

[0039] A visual lane line module is used to detect lane lines from pre-captured or real-time road images, denoted as visual lane lines; and to sample the visual lane lines to obtain a visual lane line point cloud.

[0040] The map lane line module is used to obtain the latitude and longitude of lane lines from pre-extracted or real-time map data to generate map lane lines; and to convert the map lane lines into global planar coordinate points to obtain map lane line point clouds.

[0041] The point-line pair search module is used to transform the visual lane line point cloud to the global plane coordinate system using a transformation matrix obtained in advance or in real time, and to search for visual lane line points in the visual lane line point cloud, search for line segments from the map lane line point cloud, and combine the searched line segments with the corresponding visual lane line points to form search point-line pairs.

[0042] The matching module is used to determine whether the search point-line pair meets the preset matching conditions;

[0043] The registration transformation module is used to perform registration transformation on the transformation matrix when the search point-line pair meets the preset matching conditions, so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane line.

[0044] This application also provides a lane line association registration device, the device comprising:

[0045] One or more processors;

[0046] A storage device for storing one or more programs that, when executed by one or more processors, cause the device to implement the lane line association registration method as described above.

[0047] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer's processor, causes the computer to perform the lane line association registration method as described above.

[0048] As described above, this application provides a lane line association registration method, apparatus, device, and storage medium, which have the following beneficial effects:

[0049] This application detects lane lines from pre- or real-time captured road images, denoted as visual lane lines; samples the visual lane lines to obtain a visual lane line point cloud; obtains the latitude and longitude of the lane lines from pre- or real-time extracted map data to generate map lane lines; converts the map lane lines into global planar coordinate points to obtain a map lane line point cloud; then uses a pre- or real-time acquired transformation matrix to transform the visual lane line point cloud into the global planar coordinate system, searches for visual lane line points in the visual lane line point cloud to find line segments from the map lane line point cloud, and combines the searched line segments with the corresponding visual lane line points to form search point-line pairs; finally, it determines whether the search point-line pairs meet preset matching conditions, and when the search point-line pairs meet the preset matching conditions, it performs a registration transformation on the transformation matrix so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane lines. Therefore, this application employs a point-line pair matching registration method, which to some extent performs fuzzy matching. This results in better registration accuracy and faster convergence speed between the visual lane line point cloud and the lane line points on the high-precision map, two point clouds with different distributions. It can also handle scenarios where map points are relatively sparse. The search method used in this application fully utilizes the geometric and data characteristics of lane lines, reducing the computational load and accelerating processing speed. This application can be used for visual semantic localization and other applications requiring registration of visual lane lines with map lane lines.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0052] Figure 1 This is a schematic diagram illustrating an exemplary system architecture that applies the technical solutions in one or more embodiments of this application;

[0053] Figure 2 This is a schematic flowchart of a lane line association registration method provided in one embodiment of this application;

[0054] Figure 3 A schematic diagram of a global planar coordinate system and a vehicle coordinate system provided in one embodiment of this application;

[0055] Figure 4 A schematic flowchart illustrating a lane line association registration method provided in another embodiment of this application;

[0056] Figure 5 A schematic diagram of the hardware structure of a lane line association registration system provided in an embodiment of this application;

[0057] Figure 6 This is a schematic diagram of the hardware structure of a lane line association registration device suitable for implementing one or more embodiments of this application. Detailed Implementation

[0058] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0059] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0060] In this application, "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0061] The term "multiple" in this application refers to two or more.

[0062] In the description of this application, the terms "first," "second," etc., are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0063] Additionally, in the embodiments of this application, the term "exemplary" is used to indicate that it is an example, illustration, or description. Any embodiment or implementation described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or implementations. Rather, the use of the term "exemplary" is intended to present the concept in a specific manner.

[0064] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0065] Figure 1 A schematic diagram of an exemplary system architecture that can apply the technical solutions of one or more embodiments of this application is shown. Figure 1 As shown, the system architecture 100 may include terminal device 110, network 120, and server 130. Terminal device 110 may include various electronic devices such as smartphones, tablets, laptops, and desktop computers. Server 130 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal device 110 and server 130, such as a wired communication link or a wireless communication link.

[0066] Depending on the implementation requirements, the system architecture in this application embodiment can have any number of terminal devices, networks, and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solutions provided in this application embodiment can be applied to terminal device 110, or to server 130, or can be implemented jointly by terminal device 110 and server 130. This application does not impose any special limitations on this.

[0067] In one embodiment of this application, the terminal device 110 or server 130 can detect lane lines from road images captured in advance or in real time, denoted as visual lane lines; sample the visual lane lines to obtain a visual lane line point cloud; obtain the latitude and longitude of the lane lines from map data extracted in advance or in real time to generate map lane lines; convert the map lane lines into global plane coordinate points to obtain a map lane line point cloud; then use a transformation matrix obtained in advance or in real time to transform the visual lane line point cloud into the global plane coordinate system, and search for visual lane line points in the visual lane line point cloud to find line segments from the map lane line point cloud, and combine the searched line segments with the corresponding visual lane line points to form search point-line pairs; finally, determine whether the search point-line pairs meet preset matching conditions, and when the search point-line pairs meet the preset matching conditions, perform registration transformation on the transformation matrix so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane lines. The lane line association registration method is executed using terminal device 110 or server 130. By employing a point-line pair matching registration approach, fuzzy matching is achieved to a certain extent, resulting in better registration accuracy and faster convergence speed between the visual lane line point cloud and the lane line points on the high-precision map, which have different distributions. It can also handle scenarios where map points are relatively sparse. The search method fully utilizes the geometric and data characteristics of lane lines, reducing the computational load of the algorithm and accelerating processing speed. Therefore, it can be used for visual semantic localization and other applications requiring registration of visual lane lines with map lane lines.

[0068] The above section introduced an exemplary system architecture that applies the technical solution of this application. Next, we will continue to introduce the lane line association registration method of this application.

[0069] Figure 2 A schematic flowchart of a lane line association registration method according to an embodiment of this application is shown. Specifically, in an exemplary embodiment, as follows... Figure 2 As shown, this embodiment provides a lane line association registration method, which includes the following steps:

[0070] S210, detect lane lines from road images captured in advance or in real time, and record them as visual lane lines; and sample the visual lane lines to obtain a visual lane line point cloud.

[0071] S220, obtain the latitude and longitude of the lane lines from the map data extracted in advance or in real time, and generate map lane lines; and convert the map lane lines into global planar coordinate points to obtain map lane line point cloud;

[0072] S230, using a transformation matrix acquired in advance or in real time, the visual lane line point cloud is transformed to the global plane coordinate system, and the visual lane line points in the visual lane line point cloud are searched to find line segments from the map lane line point cloud, and the searched line segments are combined with the corresponding visual lane line points to form search point-line pairs; as an example, a schematic diagram of the vehicle coordinate system and the global plane coordinate system is shown below. Figure 3 As shown. In Figure 3 In the middle, o w Using the global coordinate system, the coordinates of a point in the global coordinate system are (x... w y w );o v Let the vehicle coordinate system be used, and the coordinates of a point in the vehicle coordinate system be (x, y, y). v y v ).

[0073] S240, determine whether the search point-line pair meets the preset matching conditions, and when the search point-line pair meets the preset matching conditions, perform registration transformation on the transformation matrix so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane line.

[0074] Therefore, this embodiment, by employing a point-line pair matching registration method, performs fuzzy matching to a certain extent, resulting in better registration accuracy and faster convergence speed between the visual lane line point cloud and the lane line points on the high-precision map, which have different distributions. It can also handle scenarios where map points are relatively sparse. The search method used in this embodiment fully utilizes the geometric and data characteristics of lane lines, reducing the computational load of the algorithm and accelerating processing speed. This embodiment can be used for visual semantic localization and other applications requiring registration of visual lane lines with map lane lines.

[0075] According to the above description, in an exemplary embodiment, step S240, determining whether the search point-line pair satisfies the preset matching conditions, includes:

[0076] Obtain the first visual lane line point in the visual lane line point cloud, and search for the line segment associated with the first visual lane line point from the map lane line point cloud, and denot it as the first map line segment;

[0077] Obtain the projection point directly from the first visual lane line point to the first map line segment, and determine whether the projection point lies on the first map line segment; and,

[0078] Calculate the distance from the first visual lane line point to the first map line segment, and determine whether the distance is less than the maximum matching threshold; and,

[0079] Calculate the angle between the visual lane line and the first map line segment, and determine whether the angle is less than the maximum angle threshold;

[0080] If the projection point is on the first map line segment, the distance is less than the maximum matching threshold, and the included angle is less than the maximum angle threshold, then the search point-line pair is determined to meet the preset matching conditions and constitute a matching point-line pair.

[0081] If the projection point is not on the first map line segment, the distance is greater than or equal to the maximum matching threshold, and / or the included angle is greater than or equal to the maximum angle threshold, then the search point-line pair is determined to not meet the preset matching conditions and does not constitute a matching point-line pair.

[0082] The search point pair is formed by combining the first visual lane line point and the first map line segment; the maximum matching threshold and the maximum angle threshold are obtained based on actual engineering values. As an example, the maximum matching threshold and the maximum angle threshold in this embodiment can be set according to actual conditions, and this embodiment does not impose specific numerical limitations. For example, in this embodiment, the maximum matching threshold can be set to 1 meter; the maximum angle threshold can be set to 70°.

[0083] According to the above description, in an exemplary embodiment, the process of performing registration transformation on the transformation matrix includes: defining the distance from the first visual lane line point to the first map line segment as the matching error; constructing an error equation based on the coordinates of the first visual lane line point and the linear equation of the first map line segment; and calculating the registration transformation matrix that minimizes the sum of squares of all matching errors based on the error equation. As an example, specifically, the process of constructing the error equation based on the coordinates of the first visual lane line point and the linear equation of the first map line segment includes: obtaining the coordinates of the first visual lane line point, denoted as (x... i y i ); and, obtain the straight line equation of the first map line segment, denoted as A. i x+B i y+C i =0; Based on the coordinates of the first visual lane line points and the straight line equation of the first map line segment, an error equation is constructed:

[0084]

[0085] In the formula, e i Representing the error equation, A i B i C i Let be a constant, and i be a natural number.

[0086] In an exemplary embodiment, the process of obtaining the transformation matrix includes: obtaining the latitude and longitude information, heading information, wheel speed information, and yaw rate of the target vehicle; determining the pose of the target vehicle in the global plane coordinate system based on the latitude and longitude information, heading information, wheel speed information, and yaw rate; and calculating the transformation matrix based on the pose of the target vehicle in the global plane coordinate system. Specifically, as an example, the process of calculating the transformation matrix based on the pose of the target vehicle in the global plane coordinate system includes: denoting the pose of the target vehicle in the global plane coordinate system as P(t... x , t y The transformation matrix is ​​denoted as T. w_v Then, based on the pose of the target vehicle in the global plane coordinate system, the transformation matrix is ​​calculated as follows:

[0087]

[0088] In the formula, T w_v This represents the transformation matrix.

[0089] In an exemplary embodiment, the process of capturing the road image includes: mounting an image capturing device on a target vehicle, and capturing a road within a preset distance range from the target vehicle while the target vehicle is in motion or stationary, thereby obtaining the road image; wherein the target vehicle includes at least one of the following: a new energy vehicle or a fuel vehicle.

[0090] According to the above records, such as Figure 4 As shown, in another exemplary embodiment of this application, this embodiment provides a lane line association registration method, including the following steps:

[0091] Step 1: Input Information Acquisition and Preprocessing. First, the smart camera acquires lane lines and their corresponding type and other attribute information detected from the road image ahead. The map module extracts high-precision map data within a certain range around the vehicle based on the vehicle's positioning information obtained from the positioning device. The high-precision map data contains the latitude, longitude, shape points, and type attribute information of the lane lines. The positioning device (such as GPS) acquires the vehicle's latitude, longitude, and heading information. Wheel speed information is obtained from the vehicle's wheel speed sensor, and yaw rate is obtained through inertial navigation equipment. After acquiring the input data, the visual lane lines, high-precision map data, and vehicle motion data are processed separately. The visual lane lines are sampled to obtain a lane line point cloud. The high-precision map lane line shape points are transformed to a planar coordinate system. The current pose of the vehicle is estimated using GPS pose and vehicle motion data.

[0092] Step 2: Point-Line Pair Search. Step 2 is used to search for matching point-line pairs in the visual lane lines and the high-precision map lane lines. This embodiment does not use the commonly used point-to-point feature matching method, but instead uses point-line matching. That is, for each point in the visual lane line point cloud, a matching line segment is searched in the line segment formed by connecting the lane line points in the high-precision map. There are two main reasons for using point-line matching: First, the distribution of the visual lane line point cloud is different from the distribution of the lane line points in the high-precision map, making it difficult to determine one-to-one corresponding point pairs. Even if the optimal matching point pair is found by iterating the nearest point, the final registration error will be large. Second, the high-precision map has relatively sparse points in some areas, while the effective range of the visual lane lines is usually within 50 meters. In this scenario, point-to-point matching cannot find enough corresponding point pairs. The search is a relatively time-consuming process. In this embodiment, the correlation between lane line points, the one-to-one correspondence of lane line matching, and binary search are used to speed up the search.

[0093] Step 3: Matching Condition Judgment. Step 3 is used to determine whether the pair of visual lane line points and high-precision map point segments found in Step 2 are matching point-line pairs. The judgment conditions mainly include the following three aspects: (1) whether the projection point of the visual lane line point to the straight line containing the map point segment is on the line segment; (2) whether the distance from the point to the line segment is less than the maximum matching threshold; (3) whether the angle between the visual lane line and the map point segment is less than the maximum angle threshold. If this pair of point-line pairs meets the matching conditions, the equations of the straight lines containing the visual lane line points and line segments are stored as a matching pair. At the same time, a weight is assigned to this pair of point-line pairs, which is the proportion of the matching pair constraint in the registration process. According to the characteristics of visual perception, the farther away from the vehicle, the greater the error of the detected lane line. Therefore, this embodiment will allocate weights according to this characteristic.

[0094] Step 4: Solving the Registration Transformation. Step 4 calculates the registration transformation based on all matching point-line pairs found in Steps 2 and 3. First, an error equation is constructed. Since this embodiment uses point-line matching, the error is defined as the distance from the point to the line. Then, an optimization problem is constructed to solve for the registration transformation that minimizes the sum of squared errors of all matching pairs. The solution to this problem is the registration transformation from the visual lane line point cloud to the lane line points on the high-precision map.

[0095] According to the above description, the specific process of obtaining and preprocessing input information in step 1 includes: the lane line curve output by the intelligent camera is represented as a cubic curve in the vehicle coordinate system: y = C0 + C1x + C2x 2 +C3x 3Each visual lane line is uniformly sampled. In this embodiment, each lane line has 200 sampling points (num = 200), forming a visual lane line point cloud (src_cloud). The positioning device acquires the vehicle's pose information, latitude, longitude, and heading. The map module outputs the lane line latitude and longitude points [longitude, latitude] within a certain range around the vehicle based on the pose information, which are then converted into global planar coordinate points (x, y). ref y ref This forms the map lane line point set ref_cloud. Based on the wheel speed information obtained from the vehicle's wheel speed sensor and the yaw rate obtained from the inertial navigation system, the current pose P(t) of the vehicle in the global plane coordinate system is estimated according to the uniform linear motion model. x , t y ,θ).

[0096] Step 2, the specific process of point-line pair search, includes: As we know from Step 1, the visual lane line point cloud (src_cloud) is based on the vehicle coordinate system, while the map lane line point cloud (ref_cloud) is based on the global coordinate system. Therefore, before searching, the visual lane line points to be matched need to be transformed to the global coordinate system. The transformation formula is p w =T w_v ·p v Among them, p w p is a point in the global coordinate system. v T is a point in the vehicle coordinate system. w_v T is the transformation matrix. w_v The position and pose P(t) can be estimated from this vehicle. x , t y We obtain the following from θ):

[0097]

[0098] The visual lane line point p_w is searched within the map lane line point cloud. Because the point cloud data for each map lane line is continuous and ordered, a binary search can be used to speed up the search. Based on the geometric characteristics of lane lines, each visual lane line can only match one map lane line. Therefore, for a visual point p... w If a matching map line segment is found, and the lane line number of that map line segment is ref_line_index, then for p w Other points on the visual lane line only need to be searched on the map lane line with the number ref_line_index, thereby speeding up the search.

[0099] Step 3, the specific process of determining the matching conditions, includes: determining whether the pair of visual lane line points and high-precision map point segments found in Step 2 are matching point-line pairs. The first condition is the visual point p. wAt point p on the map j1 p j2 The projection point p0 of the line lies on line segment j1j2. Let vector... For point p j1 The vector pointing to point p0, and the vector b pointing to point p0. j2 The vector pointing to point p0, if the product of the vectors... Then the projection point p0 lies on line segment j1j2. The second condition is that the visual point p... w The distance to line segment j1j2 is the distance to line segment p. w The length of p0 is less than the maximum matching threshold, which is set to 1 meter in this embodiment. The third criterion is whether the angle between the visual lane line and map line segments j1 and j2 is less than the maximum angle threshold, which is set to 70° in this embodiment. The current visual point p is then taken. w The previous point p w0 Line segment p w p0, calculate line segment p w0 p w The angle between the point and line segment j1j2 is sufficient. If the point-line pair satisfies the above conditions, it is considered a match.

[0100] Step 4, the specific process of performing the registration transformation solution, includes: First, constructing the error equation. In this embodiment, point-line matching is used, therefore the error e is defined. i For point p w The distance to line segment j1j2. Let point p. w The coordinates are (x i y i The equation of the line containing j1j2 is A. i x+B i y+C i =0, then the error formula is Solving the registration problem between visual lane lines and map lane lines, i.e., finding the registration transformation T′ that minimizes the sum of squared errors for all matching pairs. w_v Given p w =T′ w_v ·p v Solving the optimization problem Where N is the number of matching pairs. In this embodiment, the registration transformation T′ w_v The pose of the vehicle in the global coordinate system is also obtained through registration.

[0101] In summary, this application provides a lane line association registration method, which involves detecting lane lines from pre- or real-time captured road images, denoted as visual lane lines; sampling the visual lane lines to obtain a visual lane line point cloud; obtaining the latitude and longitude of the lane lines from pre- or real-time extracted map data to generate map lane lines; converting the map lane lines into global planar coordinate points to obtain a map lane line point cloud; then using a transformation matrix obtained pre- or real-time, transforming the visual lane line point cloud into the global planar coordinate system, searching for visual lane line points in the visual lane line point cloud to find line segments from the map lane line point cloud, and combining the searched line segments with the corresponding visual lane line points to form search point-line pairs; finally, determining whether the search point-line pairs meet preset matching conditions, and when the search point-line pairs meet the preset matching conditions, performing a registration transformation on the transformation matrix to ensure that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane lines. Therefore, this method employs a point-line pair matching registration approach, which incorporates fuzzy matching to a certain extent. This results in better registration accuracy and faster convergence speed between the visual lane line point cloud and the lane line points on the high-precision map, two point clouds with different distributions. It also handles scenarios where map points are sparse. The search method used in this approach fully utilizes the geometric and data characteristics of lane lines, reducing the computational load and accelerating processing. This method can be used for visual semantic localization and other applications requiring registration of visual lane lines with map lane lines.

[0102] like Figure 5 As shown, this application also provides a lane line association registration device, the device comprising:

[0103] The visual lane line module 510 is used to detect lane lines from road images captured in advance or in real time, denoted as visual lane lines; and to sample the visual lane lines to obtain a visual lane line point cloud.

[0104] The map lane line module 520 is used to obtain the latitude and longitude of lane lines from map data extracted in advance or in real time, generate map lane lines, and convert the map lane lines into global planar coordinate points to obtain map lane line point clouds.

[0105] The point-line pair search module 530 is used to transform the visual lane line point cloud to the global plane coordinate system using a transformation matrix obtained in advance or in real time, and to search for visual lane line points in the visual lane line point cloud, search for line segments from the map lane line point cloud, and combine the searched line segments with the corresponding visual lane line points to form search point-line pairs.

[0106] Matching module 540 is used to determine whether the search point-line pair meets the preset matching conditions;

[0107] The registration transformation module 550 is used to perform registration transformation on the transformation matrix when the search point line pair meets the preset matching conditions, so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane line.

[0108] Therefore, this embodiment, by employing a point-line pair matching registration method, performs fuzzy matching to a certain extent, resulting in better registration accuracy and faster convergence speed between the visual lane line point cloud and the lane line points on the high-precision map, which have different distributions. It can also handle scenarios where map points are relatively sparse. The search method used in this embodiment fully utilizes the geometric and data characteristics of lane lines, reducing the computational load of the algorithm and accelerating processing speed. This embodiment can be used for visual semantic localization and other applications requiring registration of visual lane lines with map lane lines.

[0109] According to the above description, in an exemplary embodiment, the process by which the matching module 540 determines whether the search point-line pair meets the preset matching conditions includes:

[0110] Obtain the first visual lane line point in the visual lane line point cloud, and search for the line segment associated with the first visual lane line point from the map lane line point cloud, and denot it as the first map line segment;

[0111] Obtain the projection point directly from the first visual lane line point to the first map line segment, and determine whether the projection point lies on the first map line segment; and,

[0112] Calculate the distance from the first visual lane line point to the first map line segment, and determine whether the distance is less than the maximum matching threshold; and,

[0113] Calculate the angle between the visual lane line and the first map line segment, and determine whether the angle is less than the maximum angle threshold;

[0114] If the projection point is on the first map line segment, the distance is less than the maximum matching threshold, and the included angle is less than the maximum angle threshold, then the search point-line pair is determined to meet the preset matching conditions and constitute a matching point-line pair.

[0115] If the projection point is not on the first map line segment, the distance is greater than or equal to the maximum matching threshold, and / or the included angle is greater than or equal to the maximum angle threshold, then the search point-line pair is determined to not meet the preset matching conditions and does not constitute a matching point-line pair.

[0116] The search point pair is formed by combining the first visual lane line point and the first map line segment; the maximum matching threshold and the maximum angle threshold are obtained based on actual engineering values. As an example, the maximum matching threshold and the maximum angle threshold in this embodiment can be set according to actual conditions, and this embodiment does not impose specific numerical limitations. For example, in this embodiment, the maximum matching threshold can be set to 1 meter; the maximum angle threshold can be set to 70°.

[0117] According to the above description, in an exemplary embodiment, the process of performing registration transformation on the transformation matrix includes: defining the distance from the first visual lane line point to the first map line segment as the matching error; constructing an error equation based on the coordinates of the first visual lane line point and the linear equation of the first map line segment; and calculating the registration transformation matrix that minimizes the sum of squares of all matching errors based on the error equation. As an example, specifically, the process of constructing the error equation based on the coordinates of the first visual lane line point and the linear equation of the first map line segment includes: obtaining the coordinates of the first visual lane line point, denoted as (x... i y i ); and, obtain the straight line equation of the first map line segment, denoted as A. i x+B i y+C i =0; Based on the coordinates of the first visual lane line points and the straight line equation of the first map line segment, an error equation is constructed:

[0118]

[0119] In the formula, e i Representing the error equation, A i B i C i Let be a constant, and i be a natural number.

[0120] In an exemplary embodiment, the process of obtaining the transformation matrix includes: obtaining the latitude and longitude information, heading information, wheel speed information, and yaw rate of the target vehicle; determining the pose of the target vehicle in the global plane coordinate system based on the latitude and longitude information, heading information, wheel speed information, and yaw rate; and calculating the transformation matrix based on the pose of the target vehicle in the global plane coordinate system. Specifically, as an example, the process of calculating the transformation matrix based on the pose of the target vehicle in the global plane coordinate system includes: denoting the pose of the target vehicle in the global plane coordinate system as P(t... x , t y The transformation matrix is ​​denoted as T. w_v Then, based on the pose of the target vehicle in the global plane coordinate system, the transformation matrix is ​​calculated as follows:

[0121]

[0122] In the formula, T w_v This represents the transformation matrix.

[0123] In an exemplary embodiment, the process of capturing the road image includes: mounting an image capturing device on a target vehicle, and capturing a road within a preset distance range from the target vehicle while the target vehicle is in motion or stationary, thereby obtaining the road image; wherein the target vehicle includes at least one of the following: a new energy vehicle or a fuel vehicle.

[0124] According to the above description, in another exemplary embodiment of this application, this embodiment provides a lane line association registration device for performing the following steps:

[0125] Step 1: Input Information Acquisition and Preprocessing. First, the smart camera acquires lane lines and their corresponding type and other attribute information detected from the road image ahead. The map module extracts high-precision map data within a certain range around the vehicle based on the vehicle's positioning information obtained from the positioning device. The high-precision map data contains the latitude, longitude, shape points, and type attribute information of the lane lines. The positioning device (such as GPS) acquires the vehicle's latitude, longitude, and heading information. Wheel speed information is obtained from the vehicle's wheel speed sensor, and yaw rate is obtained through inertial navigation equipment. After acquiring the input data, the visual lane lines, high-precision map data, and vehicle motion data are processed separately. The visual lane lines are sampled to obtain a lane line point cloud. The high-precision map lane line shape points are transformed to a planar coordinate system. The current pose of the vehicle is estimated using GPS pose and vehicle motion data.

[0126] Step 2: Point-Line Pair Search. Step 2 is used to search for matching point-line pairs in the visual lane lines and the high-precision map lane lines. This embodiment does not use the commonly used point-to-point feature matching method, but instead uses point-line matching. That is, for each point in the visual lane line point cloud, a matching line segment is searched in the line segment formed by connecting the lane line points in the high-precision map. There are two main reasons for using point-line matching: First, the distribution of the visual lane line point cloud is different from the distribution of the lane line points in the high-precision map, making it difficult to determine one-to-one corresponding point pairs. Even if the optimal matching point pair is found by iterating the nearest point, the final registration error will be large. Second, the high-precision map has relatively sparse points in some areas, while the effective range of the visual lane lines is usually within 50 meters. In this scenario, point-to-point matching cannot find enough corresponding point pairs. The search is a relatively time-consuming process. In this embodiment, the correlation between lane line points, the one-to-one correspondence of lane line matching, and binary search are used to speed up the search.

[0127] Step 3: Matching Condition Judgment. Step 3 is used to determine whether the pair of visual lane line points and high-precision map point segments found in Step 2 are matching point-line pairs. The judgment conditions mainly include the following three aspects: (1) whether the projection point of the visual lane line point to the straight line containing the map point segment is on the line segment; (2) whether the distance from the point to the line segment is less than the maximum matching threshold; (3) whether the angle between the visual lane line and the map point segment is less than the maximum angle threshold. If this pair of point-line pairs meets the matching conditions, the equations of the straight lines containing the visual lane line points and line segments are stored as a matching pair. At the same time, a weight is assigned to this pair of point-line pairs, which is the proportion of the matching pair constraint in the registration process. According to the characteristics of visual perception, the farther away from the vehicle, the greater the error of the detected lane line. Therefore, this embodiment will allocate weights according to this characteristic.

[0128] Step 4: Solving the Registration Transformation. Step 4 calculates the registration transformation based on all matching point-line pairs found in Steps 2 and 3. First, an error equation is constructed. Since this embodiment uses point-line matching, the error is defined as the distance from the point to the line. Then, an optimization problem is constructed to solve for the registration transformation that minimizes the sum of squared errors of all matching pairs. The solution to this problem is the registration transformation from the visual lane line point cloud to the lane line points on the high-precision map.

[0129] According to the above description, the specific process of obtaining and preprocessing input information in step 1 includes: the lane line curve output by the intelligent camera is represented as a cubic curve in the vehicle coordinate system: y = C0 + C1x + C2x 2 +C3x 3 Each visual lane line is uniformly sampled. In this embodiment, each lane line has 200 sampling points (num = 200), forming a visual lane line point cloud (src_cloud). The positioning device acquires the vehicle's pose information, latitude, longitude, and heading. The map module outputs the lane line latitude and longitude points [longitude, latitude] within a certain range around the vehicle based on the pose information, which are then converted into global planar coordinate points (x, y). ref y ref This forms the map lane line point set ref_cloud. Based on the wheel speed information obtained from the vehicle's wheel speed sensor and the yaw rate obtained from the inertial navigation system, the current pose P(t) of the vehicle in the global plane coordinate system is estimated according to the uniform linear motion model. x , t y ,θ).

[0130] Step 2, the specific process of point-line pair search, includes: As we know from Step 1, the visual lane line point cloud (src_cloud) is based on the vehicle coordinate system, while the map lane line point cloud (ref_cloud) is based on the global coordinate system. Therefore, before searching, the visual lane line points to be matched need to be transformed to the global coordinate system. The transformation formula is p w=T w_v ·p v Among them, p w p is a point in the global coordinate system. v T is a point in the vehicle coordinate system. w_v T is the transformation matrix. w_v The position and pose P(t) can be estimated from this vehicle. x , t y We obtain the following from θ):

[0131]

[0132] The visual lane line point p_w is searched within the map lane line point cloud. Because the point cloud data for each map lane line is continuous and ordered, a binary search can be used to speed up the search. Based on the geometric characteristics of lane lines, each visual lane line can only match one map lane line. Therefore, for a visual point p... w If a matching map line segment is found, and the lane line number of that map line segment is ref_line_index, then for p w Other points on the visual lane line only need to be searched on the map lane line with the number ref_line_index, thereby speeding up the search.

[0133] Step 3, the specific process of determining the matching conditions, includes: determining whether the pair of visual lane line points and high-precision map point segments found in Step 2 are matching point-line pairs. The first condition is the visual point p. w At point p on the map j1 p j2 The projection point p0 of the line lies on line segment j1j2. Let vector... For point p j1 The vector pointing to point p0, the vector For point p j2 The vector pointing to point p0, if the product of the vectors... Then the projection point p0 lies on line segment j1j2. The second condition is that the visual point p... w The distance to line segment j1j2 is the distance to line segment p. w The length of p0 is less than the maximum matching threshold, which is set to 1 meter in this embodiment. The third criterion is whether the angle between the visual lane line and map line segments j1 and j2 is less than the maximum angle threshold, which is set to 70° in this embodiment. The current visual point p is then taken. w The previous point p w0 Line segment p w p0, calculate line segment p w0 p w The angle between the point and line segment j1j2 is sufficient. If the point-line pair satisfies the above conditions, it is considered a match.

[0134] Step 4, the specific process of performing the registration transformation solution, includes: First, constructing the error equation. In this embodiment, point-line matching is used, therefore the error e is defined. i For point p w The distance to line segment j1j2. Let point p. w The coordinates are (x i y i The equation of the line containing j1j2 is A. i x+B i y+C i =0, then the error formula is Solving the registration problem between visual lane lines and map lane lines, i.e., finding the registration transformation T′ that minimizes the sum of squared errors for all matching pairs. w_v Given p w =T′ w_v ·p v Solving the optimization problem Where N is the number of matching pairs. In this embodiment, the registration transformation T′ w_v The pose of the vehicle in the global coordinate system is also obtained through registration.

[0135] In summary, this application provides a lane line association registration device, which detects lane lines from pre- or real-time captured road images and denotes them as visual lane lines; samples the visual lane lines to obtain a visual lane line point cloud; obtains the latitude and longitude of the lane lines from pre- or real-time extracted map data to generate map lane lines; converts the map lane lines into global planar coordinate points to obtain a map lane line point cloud; then uses a transformation matrix obtained in advance or in real-time to transform the visual lane line point cloud into the global planar coordinate system, searches for visual lane line points in the visual lane line point cloud, searches for line segments from the map lane line point cloud, and combines the searched line segments with the corresponding visual lane line points to form search point-line pairs; finally, it determines whether the search point-line pairs meet preset matching conditions, and when the search point-line pairs meet the preset matching conditions, performs registration transformation on the transformation matrix so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane lines. Therefore, this device employs a point-line pair matching registration method, which incorporates fuzzy matching to a certain extent. This results in better registration accuracy and faster convergence speed between the visual lane line point cloud and the lane line points on the high-precision map, two point clouds with different distributions. It can also handle scenarios where map points are sparse. The search method used in this device fully utilizes the geometric and data characteristics of lane lines, reducing the computational load and accelerating processing speed. This device can be used for visual semantic localization and other applications requiring registration of visual lane lines with map lane lines.

[0136] It should be noted that the lane line association registration device provided in the above embodiments and the lane line association registration method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the lane line association registration device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0137] Embodiments of this application also provide a lane line association registration device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the lane line association registration device to implement the lane line association registration method provided in the above embodiments.

[0138] Figure 6 A schematic diagram of a computer device suitable for implementing the lane line association registration device of the embodiments of this application is shown. It should be noted that... Figure 6 The computer system 1000 for lane line association registration shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0139] like Figure 6 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0140] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0141] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the various functions defined in the apparatus of this application.

[0142] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0144] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0145] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a computer's processor, causes the computer to perform the lane line association registration method as described above. This computer-readable storage medium may be included in the lane line association registration device described in the above embodiments, or it may exist independently and not incorporated into the lane line association registration device.

[0146] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the lane line association registration method provided in the various embodiments described above.

[0147] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A lane line association registration method, characterized in that, The method includes the following steps: Lane lines are detected from road images captured in advance or in real time and denoted as visual lane lines; and the visual lane lines are sampled to obtain a visual lane line point cloud. The latitude and longitude of lane lines are obtained from map data extracted in advance or in real time to generate map lane lines; and the map lane lines are converted into global planar coordinate points to obtain map lane line point clouds. Using a transformation matrix acquired in advance or in real time, the visual lane line point cloud is transformed into a global plane coordinate system, and the visual lane line points in the visual lane line point cloud are searched to find line segments from the map lane line point cloud, and the searched line segments are combined with the corresponding visual lane line points to form search point-line pairs. Determine whether the search point-line pair meets the preset matching conditions, and when the search point-line pair meets the preset matching conditions, perform a registration transformation on the transformation matrix so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane line. The process of determining whether the search point-line pair meets the preset matching conditions includes: Obtain the first visual lane line point in the visual lane line point cloud, and search for the line segment associated with the first visual lane line point from the map lane line point cloud, and denot it as the first map line segment; Obtain the projection point from the first visual lane line point to the straight line containing the first map line segment, and determine whether the projection point lies on the first map line segment; and, Calculate the distance from the first visual lane line point to the first map line segment, and determine whether the distance is less than the maximum matching threshold; and, Calculate the angle between the visual lane line and the first map line segment, and determine whether the angle is less than the maximum angle threshold; If the projection point is on the first map line segment, the distance is less than the maximum matching threshold, and the included angle is less than the maximum angle threshold, then the search point-line pair is determined to meet the preset matching conditions and constitute a matching point-line pair. If the projection point is not on the first map line segment, the distance is greater than or equal to the maximum matching threshold, and / or the included angle is greater than or equal to the maximum angle threshold, then the search point-line pair is determined to not meet the preset matching conditions and does not constitute a matching point-line pair. The search point pair is formed by combining the first visual lane line point and the first map line segment; the maximum matching threshold and the maximum angle threshold are obtained based on actual engineering values.

2. The lane line association registration method according to claim 1, characterized in that, The process of performing registration transformation on the transformation matrix includes: The distance from the first visual lane line point to the first map line segment is defined as the matching error; An error equation is constructed based on the coordinates of the first visual lane line points and the straight line equation of the first map line segment; Calculate the registration transformation matrix that minimizes the sum of squares of all matching errors based on the error equation.

3. The lane line association registration method according to claim 1 or 2, characterized in that, The process of obtaining the transformation matrix includes: Obtain the target vehicle's latitude and longitude, heading, wheel speed, and yaw rate; Based on the latitude and longitude information, heading information, wheel speed information, and yaw rate, the pose of the target vehicle in the global plane coordinate system is determined; The transformation matrix is ​​calculated based on the pose of the target vehicle in the global plane coordinate system.

4. The lane line association registration method according to claim 3, characterized in that, The process of calculating the transformation matrix based on the pose of the target vehicle in the global plane coordinate system includes: The pose of the target vehicle in the global plane coordinate system is denoted as... ; Let the transformation matrix be denoted as Then, based on the pose of the target vehicle in the global plane coordinate system, the transformation matrix is ​​calculated as follows: ； In the formula, This represents the transformation matrix.

5. The lane line association registration method according to claim 2, characterized in that, The process of constructing the error equation based on the coordinates of the first visual lane line points and the straight line equation of the first map line segment includes: Obtain the coordinates of the first visual lane line point, denoted as ;as well as, Obtain the straight line equation of the first map line segment, denoted as ; Based on the coordinates of the first visual lane line points and the straight line equation of the first map line segment, an error equation is constructed: ； In the formula, Representing the error equation, , , It is a constant. It is a natural number.

6. The lane line association registration method according to claim 1, characterized in that, The process of capturing the road image includes: pre-setting an image capturing device on the target vehicle, and using the image capturing device to capture the road within a preset distance range from the target vehicle while the target vehicle is in motion or stationary, thereby obtaining the road image; wherein the target vehicle includes at least one of the following: a new energy vehicle or a fuel vehicle.

7. A lane line association registration device, characterized in that, The device includes: A visual lane line module is used to detect lane lines from pre-captured or real-time road images, denoted as visual lane lines; and to sample the visual lane lines to obtain a visual lane line point cloud. The map lane line module is used to obtain the latitude and longitude of lane lines from pre-extracted or real-time map data and generate map lane lines. Furthermore, the map lane lines are converted into global planar coordinate points to obtain a map lane line point cloud; The point-line pair search module is used to transform the visual lane line point cloud to the global plane coordinate system using a transformation matrix obtained in advance or in real time, and to search for visual lane line points in the visual lane line point cloud, search for line segments from the map lane line point cloud, and combine the searched line segments with the corresponding visual lane line points to form search point-line pairs. The matching module is used to determine whether the search point-line pair meets preset matching conditions, including: acquiring a first visual lane line point in the visual lane line point cloud, and searching for a line segment associated with the first visual lane line point from the map lane line point cloud, denoted as a first map line segment; acquiring the projection point of the first visual lane line point to the straight line containing the first map line segment, and determining whether the projection point is on the first map line segment; calculating the distance from the first visual lane line point to the first map line segment, and determining whether the distance is less than a maximum matching threshold; and calculating the angle between the visual lane line and the first map line segment, and determining whether the angle is... If the projection point is on the first map line segment, the distance is less than the maximum matching threshold, and the included angle is less than the maximum angle threshold, then the search point-line pair is determined to meet the preset matching conditions and constitutes a matching point-line pair; if the projection point is not on the first map line segment, the distance is greater than or equal to the maximum matching threshold, and / or the included angle is greater than or equal to the maximum angle threshold, then the search point-line pair is determined to not meet the preset matching conditions and does not constitute a matching point-line pair; wherein, the search point-line pair is formed by combining the first visual lane line point and the first map line segment; the maximum matching threshold and the maximum angle threshold are obtained based on actual engineering values; The registration transformation module is used to perform registration transformation on the transformation matrix when the search point-line pair meets the preset matching conditions, so that each visual lane line point in the visual lane line point cloud has a matching line segment in the map lane line.

8. A lane line association registration device, characterized in that, The device includes: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the device to implement the lane line association registration method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the lane line association registration method as described in any one of claims 1 to 6.

Images