A Map Construction Method and Device Based on Lane Line Semantics

Abstract

An embodiment of the present application provides a map construction method and device based on lane line semantics, which relates to the technical field of map construction. The method includes obtaining original data, where the original data includes camera images and IMU data; respectively obtaining ORB feature points and lane line information by using the original data, and converting the obtained lane line information into lane line constraint information; performing a dynamic point elimination operation on the ORB feature points to obtain an ORB feature point set; using the ORB feature point set for camera pose estimation and pose optimization to obtain a camera pose estimation result; constructing a map based on the camera pose estimation result and the ORB feature point set. This method adds a dynamic point deletion function and adds constraint information to the solution problem of SLAM, improving the robustness and accuracy, and solving the problem of low robustness and accuracy caused by the fact that existing methods do not consider dynamic scenarios.

Description

Technical Field

[0001] The present application relates to the technical field of map construction, and more particularly, to a map construction method and device based on lane line semantics. Background Art

[0002] Currently, the mainstream method for constructing high-precision maps is the Simultaneous Localization and Mapping (SLAM) technology, which is mainly divided into two directions: laser SLAM and visual SLAM. Due to the low cost and large amount of information of visual SLAM, it is commonly used in products such as autonomous vehicles and drones.

[0003] However, the visual SLAM technology has insufficient adaptability to high-dynamic scenes. Currently, most visual SLAM systems are designed based on the assumption of static scenes, and there are certain difficulties in detecting, tracking, and mapping in fast-moving scenes or high-dynamic scenes. Such dynamic objects will not only affect the estimation of the odometer during map construction, but also affect the local geometric features of the constructed map and the subsequent inter-frame matching and positioning process, thus affecting the robustness and accuracy of the map construction process. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a map construction method and device based on lane line semantics, which add a dynamic point deletion function and add constraint information to the solution problem of SLAM, improve the robustness and accuracy, and solve the problem of low robustness and accuracy caused by the existing methods not considering dynamic scenes.

[0005] The embodiments of the present application provide a map construction method based on lane line semantics, and the method includes:

[0006] Obtain original data, where the original data includes camera images and IMU data;

[0007] Use the original data to obtain ORB feature points and lane line information respectively, and convert the obtained lane line information into lane line constraint information;

[0008] Based on the lane line constraint information, perform a dynamic point elimination operation on the ORB feature points to obtain an ORB feature point set;

[0009] Use the ORB feature point set for camera pose estimation and pose optimization to obtain a camera pose estimation result;

[0010] Construct a map based on the camera pose estimation result and the ORB feature point set.

[0011] In the above implementation process, through dynamic point culling operations, map construction in dynamic scenarios is achieved. Meanwhile, lane line constraint information is added, which can further improve the robustness and accuracy of ORB-SLAM2, making it applicable to real-time mapping and positioning tasks in more scenarios, enhancing the robustness and accuracy, and solving the problem of low robustness and accuracy caused by the failure of existing methods to consider dynamic scenarios.

[0012] Furthermore, the method of separately obtaining ORB feature points and lane line information from the original data and converting the obtained lane line information into lane line constraint information includes:

[0013] Using the lane line detection network CLRNet to perform lane line detection on the preprocessed original data to obtain lane line information and road marking images;

[0014] Extracting ORB feature points from the preprocessed original data;

[0015] Converting the obtained lane line information into lane line constraint information.

[0016] In the above implementation process, converting the lane line information based on ORB feature points into lane line constraint information can improve the accuracy of subsequent camera pose estimation and lane line detection.

[0017] Furthermore, the lane line constraint information includes lane line direction constraints. The process of converting the lane line information based on ORB feature points into lane line constraint information includes:

[0018] The lane line direction constraint can be represented as the rotation matrix of the camera between two adjacent frames. Then, the relationship between the rotation matrix and the lane line direction vector can be expressed as:

[0019] d i = R i-1 d i-1 ;

[0020] where d i and d i-1 respectively represent the lane line direction vectors between two adjacent frames; R i-1 represents the rotation matrix of the camera between two adjacent frames.

[0021] In the above implementation process, through the direction information of the lane line, the movement direction of the camera between two adjacent frames can be constrained, thereby improving the accuracy of pose estimation.

[0022] Further, the lane line constraint information includes lane line distance constraint, and the conversion of the obtained lane line information into lane line constraint information includes:

[0023] The lane line distance between two adjacent frames can be expressed as:

[0024]

[0025] Wherein, T i+1 and T i represent the poses of the cameras of two adjacent frames of images; d represents the direction vector of the lane line in the camera coordinate system;

[0026] Then the lane line distance constraint can be expressed as an algebraic equation:

[0027]

[0028] Wherein, const represents the position offset of the lane line in the camera coordinate system.

[0029] In the above implementation process, through the distance information of the lane line, the moving distance of the camera between two adjacent frames can be constrained, thereby improving the accuracy of pose estimation.

[0030] Further, the lane line constraint information includes lane line curvature constraint, and the conversion of the obtained lane line information into lane line constraint information includes:

[0031] The lane line curvature constraint can be expressed as the motion trajectory of the camera between two adjacent frames, and the motion trajectory is consistent with the curvature information of the lane line:

[0032]

[0033] Wherein, x and y respectively represent the horizontal and vertical coordinates of the motion trajectory of the camera between two adjacent frames in the world coordinate system; f ( x ) represents the curve function of the lane line in the world coordinate system.

[0034] In the above implementation process, through the curvature information of the lane line, the motion trajectory of the camera between two adjacent frames can be constrained, thereby improving the accuracy of pose estimation.

[0035] Further, the lane line constraint information includes lane line smoothness constraint, and the conversion of the obtained lane line information into lane line constraint information includes:

[0036] The lane line smoothness constraint can be expressed as the continuity of the curvature and the curvature change rate of the lane line, and the lane line smoothness constraint is described using an equation:

[0037]

[0038] where, k i represents the curvature of the lane line on the i th frame image; v i represents the timestamp of the i th frame image;

[0039] The lane line smoothness constraint equation is transformed into matrix form:

[0040] Ax = 0;

[0041] where, A represents the n × n matrix, x represents the n -dimensional vector, and the least squares solution obtained satisfies the smoothness constraint, n represents the number of ORB feature points corresponding to the lane line.

[0042] In the above implementation process, through the smoothness information of the lane line, the continuity of the lane line between two adjacent frames can be constrained, thereby improving the accuracy of lane line detection.

[0043] Furthermore, the dynamic point elimination operation is performed on the ORB feature points to obtain an ORB feature point set, including:

[0044] Determine whether the ORB feature point falls within the road marking range corresponding to the road marking image;

[0045] If it is within the road marking range, the ORB feature point is deleted.

[0046] In the above implementation process, through the dynamic point elimination operation, ORB-SLAM2 is applicable to dynamic scenarios, and map construction in dynamic scenarios is realized.

[0047] This application embodiment also provides a map construction device based on lane line semantics, and the device includes:

[0048] A data acquisition module, configured to acquire original data, where the original data includes camera images and IMU data;

[0049] A lane line detection module, configured to respectively obtain ORB feature points and lane line information using the original data, and convert the obtained lane line information into lane line constraint information;

[0050] A dynamic point culling module, which is used to perform a dynamic point culling operation on the ORB feature points based on the lane line constraint information to obtain an ORB feature point set;

[0051] A pose estimation module, which is used to perform camera pose estimation and pose optimization using the ORB feature point set to obtain a camera pose estimation result;

[0052] A map construction module, which is used to construct a map based on the camera pose estimation result and the ORB feature point set.

[0053] In the above implementation process, through the dynamic point culling operation, map construction in a dynamic scenario is realized, and at the same time, lane line constraint information is added, which can further improve the robustness and accuracy of ORB-SLAM2, is applicable to real-time mapping and positioning tasks in more scenarios, improves the robustness and accuracy, and solves the problem of low robustness and accuracy caused by the fact that existing methods do not consider dynamic scenarios.

[0054] An embodiment of the present application also provides an electronic device, which includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the map construction method based on lane line semantics described in any one of the above.

[0055] An embodiment of the present application also provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and run by a processor, the map construction method based on lane line semantics described in any one of the above is executed. Description of the Drawings

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative efforts.

[0057] Figure 1 It is a flowchart of a map construction method based on lane line semantics provided by an embodiment of the present application;

[0058] Figure 2 It is a specific implementation flowchart after adding a dynamic point deletion function provided by an embodiment of the present application;

[0059] Figure 3 It is a lane line detection flowchart provided by an embodiment of the present application;

[0060] Figure 4Flowchart of the specific implementation of dynamic point deletion provided by the embodiments of the present application;

[0061] Figure 5 Block diagram of a map construction device based on lane line semantics provided by the embodiments of the present application;

[0062] Figure 6 Block diagram of another map construction device based on lane line semantics provided by the embodiments of the present application.

[0063] Icon:

[0064] 100 - Data acquisition module; 200 - Lane line detection module; 201 - Road marking image acquisition module; 202 - Feature point extraction module; 203 - Lane line constraint module; 300 - Dynamic point elimination module; 301 - Judgment module; 302 - Deletion module; 400 - Pose estimation module; 500 - Map construction module. Detailed implementation manners

[0065] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0066] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0067] Embodiment 1

[0068] Please refer to Figure 1 , Figure 1 which is a flowchart of a map construction method based on lane line semantics provided by the embodiments of the present application.

[0069] The mathematical optimization problem of ORB - SLAM2 can be expressed as:

[0070] Assume that the input image sequence is , and the camera pose sequence is , then the optimization problem can be expressed as minimizing the objective function:

[0071]

[0072] where, represents the operation of the pose, represents the motion between adjacent frames, e i ( T ) represents the reprojection error of the i th frame image, is the weight matrix, is the weight parameter for the balance error term and the regularization term.

[0073] The optimization problem can be solved by a non - linear optimization method (e.g., the Gauss - Newton method or Levenberg - Marquardt algorithm) to obtain the optimal solution of the camera pose, thereby achieving SLAM the goal.

[0074] This application adds the function of dynamic point elimination on the basis of orb - slam2. The main method is to use the output of the lane line detection network CLRNet to establish a mask for the input image. After extracting the orb feature points of the input image, it is judged whether the matching points are within the range of the road mask. If the orb feature points fall on the road, they are determined as the feature points of dynamic targets. After eliminating the dynamic points, the mapping process of orb - slam2 continues, thereby realizing the construction of a road semantic map in a dynamic scene, as Figure 2 shown in the specific implementation flowchart after adding the dynamic point deletion function.

[0075] The method specifically includes the following steps:

[0076] Step S100: Obtain the original data, where the original data includes camera images and IMU data;

[0077] Step S200: Use the original data to obtain ORB feature points and lane line information respectively, and convert the obtained lane line information into lane line constraint information;

[0078] It should be noted that the lane line information here is extracted by the lane line detection network CLRNet, and the ORB feature points can be extracted using the ORB algorithm, which belongs to a parallel network structure.

[0079] Step S300: Perform a dynamic point elimination operation on the ORB feature points based on the lane line constraint information to obtain an ORB feature point set;

[0080] The lane line constraint is used to eliminate dynamic points before constructing the map, which is equivalent to deleting some dirty data that is likely to cause errors.

[0081] Step S400: Use the ORB feature point set for camera pose estimation and pose optimization to obtain the camera pose estimation result;

[0082] Specifically: Screen and match the ORB feature points, calculate the relative motion information between adjacent frames; Use the RANSAC algorithm to estimate the camera pose, and at the same time remove the mis - matched feature points; Use an optimization algorithm (such as BA) to further optimize the pose estimation result.

[0083] Step S500: Construct a map based on the camera pose estimation result and the set of ORB feature points.

[0084] Specifically, construct a map using the above-extracted feature points and pose information, including a local map and a global map; use IMU data to predict and correct the camera movement; repeat the above steps S200 - S500 to achieve real-time mapping and positioning functions.

[0085] Among them, as Figure 3 shown, it is a flowchart of lane line detection. Step S200 specifically includes the following steps:

[0086] Step S201: Use the lane line detection network CLRNet to perform lane line detection on the preprocessed original data to obtain lane line information and a road marking image;

[0087] Step S202: Extract ORB feature points using the preprocessed original data;

[0088] Step S203: Convert the obtained lane line information into lane line constraint information.

[0089] Generate a mask (road marking image) of the road using the output of the lane line detection network CLRNet: Convert the lane line detection result into a binary image, mark the lane line area as white (foreground), and the remaining area as black (background) to obtain a binary mask image.

[0090] Preprocess the input image, such as cropping, scaling, etc., to adapt to the input size of the ORB feature point extraction algorithm. Use the ORB feature point extraction algorithm to extract feature points in the image.

[0091] The lane line constraint information includes lane line direction constraint, lane line distance constraint, lane line curvature constraint, and lane line smoothness constraint.

[0092] Among them, the lane line direction constraint can be represented as the rotation matrix of the camera between two adjacent frames. Represent the camera poses of two adjacent frames as T i-1 and T i , then:

[0093]

[0094] Among them, R i and R i-1 represent the rotation matrix of the camera between two adjacent frames, t i-1 and ti respectively represent the translation vector of the camera between two frames.

[0095] Express the direction vector of the lane line between two adjacent frames as d i-1 and d i , then and The direction relationship between can be expressed as:

[0096] ;

[0097] where d i and d i-1 respectively represent the direction vectors of the lane line between two adjacent frames; R i-1 represents the rotation matrix of the camera between two adjacent frames.

[0098] This is because d i-1 The coordinates in the camera coordinate system are , where R T i-1 represents R i-1 The transpose matrix of. Therefore, what the camera sees in the coordinate system of the i -1 frame d i The direction vector of is R i-1 d i-1 , while what the camera sees in the coordinate system of the i frame d i The direction vector of is d i . From this, the above equation can be obtained. Expanding the above equation, we can get:

[0099] ;

[0100] where r ij represents R i-1 The element in the i th j row and

[0101] Writing the above equation in vector form, we can get:

[0102] .

[0103] Lane line direction constraint: Through the direction information of the lane line, the movement direction of the camera between adjacent frames can be constrained, thereby improving the accuracy of pose estimation.

[0104] The lane line distance constraint can be expressed as the translation matrix of the camera between two adjacent frames.

[0105] Specifically, for two adjacent frames of images, let the poses of the camera be T i+1 and T i , then based on the pixel coordinates of the lane line in the two frames of images ( u i , v i ) and ( u i+1 , v i+1 ), as well as the internal parameter matrix K of the camera, the spatial coordinates P i+1 and P i of the lane line in the camera coordinate system can be calculated, that is:

[0106] ;

[0107] Let the direction vector of the lane line in the camera coordinate system be d , then the lane line distance D between two adjacent frames can be expressed as:

[0108]

[0109] where T i+1 and T i represent the poses of the camera in two adjacent frames of images; d represents the direction vector of the lane line in the camera coordinate system;

[0110] Then the lane line distance constraint can be expressed as an algebraic equation:

[0111] ;

[0112] where const represents the position offset of the lane line in the camera coordinate system.

[0113] Lane line distance constraint: Through the distance information of the lane line, the movement distance of the camera between adjacent frames can be constrained, thereby improving the accuracy of pose estimation.

[0114] The lane line curvature constraint can be expressed as the motion trajectory of the camera between adjacent frames, and this trajectory should conform to the curvature information of the lane line.

[0115] The lane line curvature constraint can be expressed as the motion trajectory of the camera between two adjacent frames, and the motion trajectory is consistent with the curvature information of the lane line:

[0116]

[0117] Among them, x and y respectively represent the horizontal and vertical coordinates of the motion trajectory of the camera between two adjacent frames in the world coordinate system; f ( x ) represents the curve function of the lane line in the world coordinate system.

[0118] Adding this constraint to the SLAM solution process makes the curvature of the motion trajectory the same as the lane line curvature.

[0119] Lane line curvature constraint: Through the curvature information of the lane line, the motion trajectory of the camera between adjacent frames can be constrained, thereby improving the accuracy of pose estimation.

[0120] The lane line smoothing constraint can be expressed as the continuity of the curvature and the curvature change rate of the lane line, which can be expressed as a series of difference equations. Let k i represent the curvature of the lane line on the i -th frame image; v i represent the timestamp of the i -th frame image; The smoothing constraint is described by the following equations:

[0121] ;

[0122] Among them, the first equation is a first-order difference equation for keeping the curvature smooth, and the second equation is a second-order difference equation for keeping the curvature change rate smooth.

[0123] These equations can be transformed into matrix form Ax = 0, where A represents the matrix of , x represents the n -dimensional vector, and the obtained least squares solution satisfies the smoothing constraint, n represents the number of ORB feature points corresponding to the lane line. By solving Ax = 0, a least squares solution x can be obtained, which can satisfy the smoothing constraint.

[0124] Lane line smooth constraint: Through the smooth information of lane lines, the continuity of lane lines between adjacent frames can be constrained, thereby improving the accuracy of lane line detection.

[0125] These constraint information are converted into a series of algebraic equations and applied to the solution problem of slam.

[0126] After adding constraint conditions SLAM The process is a non-linear optimization problem with constraints. The optimization goal is to minimize the sum of squares of all observation residuals while satisfying the constraint conditions. Therefore, this problem can be written in the following form:

[0127] ;

[0128] Among them, x represents the variable vector to be optimized, f i represents the i th observation function, represents the robust loss function, represents the constraint function. The optimization variable x includes the camera pose and the map point cloud. The constraint functions include road surface constraints, lane line direction constraints, lane line distance constraints, lane line curvature constraints and lane line smooth constraints. By using an optimization algorithm (such as a non-linear least squares optimization algorithm) to solve this problem, the optimal camera pose and map point cloud can be obtained while satisfying all the constraint conditions.

[0129] As Figure 4 shown, it is the specific implementation flowchart of dynamic point deletion. Step S300 specifically includes the following steps:

[0130] Step S301: Determine whether the ORB feature point falls within the road marking range corresponding to the road marking image;

[0131] Step S302: If it is within the road marking range, delete the ORB feature point.

[0132] Determine whether the ORB feature point falls within the road mask range. If it is on the road, it is determined as a dynamic point and deleted from the set of ORB feature points.

[0133] The process of ORB-SLAM2 with dynamic point culling adds one step compared to the original ORB-SLAM2. That is, after feature point extraction, the input image is masked using the output of the lane detection network, and then the dynamic points within the mask are culled. After obtaining the output of the lane detection network, by adding lane direction constraints, lane distance constraints, lane curvature constraints, and lane smoothness constraints to the SLAM solution problem, the robustness and accuracy of ORB-SLAM2 can be further improved, making it applicable to real-time mapping and localization tasks in more scenarios.

[0134] This method can be applied to the field of unmanned driving mapping and also to the field of map construction in other dynamic scenarios, without any limitation here.

[0135] Embodiment 2

[0136] An embodiment of this application provides a map construction device based on lane line semantics, which is applied to the map construction method based on lane line semantics described in Embodiment 1. As Figure 5 shown, it is a structural block diagram of a map construction device based on lane line semantics. The device includes but is not limited to:

[0137] A data acquisition module 100, configured to acquire raw data, where the raw data includes camera images and IMU data;

[0138] A lane detection module 200, configured to perform lane detection based on the preprocessed raw data, extract ORB feature points, and convert the obtained lane line information into lane line constraint information;

[0139] A dynamic point culling module 300, configured to perform a dynamic point culling operation on the ORB feature points based on the lane line constraint information to obtain an ORB feature point set;

[0140] A pose estimation module 400, configured to use the ORB feature point set to perform camera pose estimation and pose optimization to obtain a camera pose estimation result;

[0141] A map construction module 500, configured to construct a map based on the camera pose estimation result and the ORB feature point set.

[0142] Among them, as Figure 6 shown, it is a structural block diagram of another map construction device based on lane line semantics. The lane detection module 200 includes:

[0143] A road marking image acquisition module 201, configured to use the lane detection network CLRNet to perform lane detection on the preprocessed raw data to obtain lane line information and road marking images;

[0144] The feature point extraction module 202 is used to extract ORB feature points by using the preprocessed original data;

[0145] The lane line constraint module 203 is used to convert the obtained lane line information into lane line constraint information.

[0146] The specific implementation process of the lane line constraint module 203 has been specifically described in Embodiment 1 and will not be elaborated here.

[0147] The dynamic point elimination module 300 includes:

[0148] The judgment module 301 is used to judge whether the ORB feature points fall within the road marking range corresponding to the road marking image; [[ID=,14]]

[0149] The deletion module 302 is used to delete the ORB feature points if they are within the road marking range.

[0150] Through the dynamic point elimination operation, the map construction in the dynamic scenario is realized, and at the same time, the lane line constraint information is added, which can further improve the robustness and accuracy of ORB-SLAM2, is applicable to the real-time mapping and positioning tasks in more scenarios, improves the robustness and accuracy, and solves the problem of low robustness and accuracy caused by the existing methods not considering the dynamic scenario.

[0151] This application embodiment also provides an electronic device, which includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the map construction method based on lane line semantics described in Embodiment 2.

[0152] This application embodiment also provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and run by a processor, the map construction method based on lane line semantics described in Embodiment 2 is executed.

[0153] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0154] In addition, each functional module in various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0155] If the described functions are implemented in the form of software functional modules and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes.

[0156] The above are only embodiments of the present application and are not intended to limit the protection scope of the present application. For those skilled in the art, various modifications and changes can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application. It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0157] As mentioned above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or replacements, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0158] It should be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

Claims

1. A map construction method based on lane line semantics, characterized in that, The method includes: Obtaining original data, where the original data includes camera images and IMU data; Respectively obtaining ORB feature points and lane line information from the original data, and converting the obtained lane line information into lane line constraint information. The lane line constraint information includes lane line direction constraint, lane line distance constraint, lane line curvature constraint, and lane line smoothness constraint, and converting the lane line constraint information into a series of algebraic equations to be applied to the solution problem of SLAM. Specifically: For the lane line direction constraint, the movement direction of the camera between adjacent frames is constrained by the direction information of the lane line; for the lane line distance constraint, the movement distance of the camera between adjacent frames is constrained by the distance information of the lane line; for the lane line curvature constraint, the movement trajectory of the camera between adjacent frames is constrained by the curvature information of the lane line; for the lane line smoothness constraint, the continuity of the lane line between adjacent frames is constrained by the smoothness information of the lane line; Performing a dynamic point elimination operation on the ORB feature points based on the lane line constraint information to obtain an ORB feature point set; Using the ORB feature point set for camera pose estimation and pose optimization to obtain a camera pose estimation result; Constructing a map based on the camera pose estimation result and the ORB feature point set.

2. The method for constructing a map based on lane line semantics according to claim 1, wherein The respectively obtaining ORB feature points and lane line information from the original data, and converting the obtained lane line information into lane line constraint information includes: Using the lane line detection network CLRNet to perform lane line detection on the preprocessed original data to obtain lane line information and road marking images; Extracting ORB feature points from the preprocessed original data; Converting the obtained lane line information into lane line constraint information.

3. The method for constructing a map based on lane line semantics according to claim 2, wherein The lane line constraint information includes a lane line direction constraint. Converting the obtained lane line information into lane line constraint information includes: The lane line direction constraint is represented as the rotation matrix of the camera between two adjacent frames, and the relationship between the rotation matrix and the lane line direction vector is expressed as: ； Among them, d i and d i-1 respectively represent the lane line direction vectors between two adjacent frames; R i-1 represents the rotation matrix of the camera between two adjacent frames.

4. The method for constructing a map based on lane line semantics according to claim 2, wherein The lane line constraint information includes a lane line distance constraint. Converting the lane line information based on ORB feature points into lane line constraint information includes: The lane line distance between two adjacent frames is expressed as: ； Among them, T i+1 and T i represent the poses of the cameras of two adjacent frames of images; d represents the direction vector of the lane line in the camera coordinate system; Then the lane line distance constraint is expressed as an algebraic equation: ； where const represents the position offset of the lane line in the camera coordinate system.

5. The method for constructing a map based on lane line semantics according to claim 2, wherein The lane line constraint information includes a lane line curvature constraint. Converting the lane line information based on ORB feature points into lane line constraint information includes: The lane line curvature constraint is represented as the movement trajectory of the camera between two adjacent frames, and the movement trajectory is consistent with the curvature information of the lane line: ； Among them, x and y respectively represent the horizontal and vertical coordinates of the camera's motion trajectory between two adjacent frames in the world coordinate system; f ( x ) represents the curve function of the lane line in the world coordinate system.

6. The method for constructing a map based on lane line semantics according to claim 2, wherein The lane line constraint information includes a lane line smoothness constraint. Converting the lane line information based on ORB feature points into lane line constraint information includes: The lane line smoothness constraint is represented as the continuity of the curvature and the curvature change rate of the lane line, and the lane line smoothness constraint is described using an equation: ； Among them, represents the curvature of the lane line on the th frame image; represents the timestamp of the th frame image; Converting the equation representing the lane line smoothness constraint into matrix form: Ax =0； Among them, A represents matrix, represents dimensional vector, and the least squares solution obtained satisfies the smoothing constraint, represents the number of ORB feature points corresponding to the lane line.

7. The map construction method based on lane line semantics according to claim 1, wherein The performing a dynamic point elimination operation on the ORB feature points based on the lane line constraint information to obtain an ORB feature point set includes: Determine whether the ORB feature points fall within the road marking range corresponding to the road marking image; If within the road marking range, delete the ORB feature points.

8. A map construction device based on lane line semantics, characterized in that The device includes: A data acquisition module for acquiring original data, where the original data includes camera images and IMU data; A lane line detection module for respectively obtaining ORB feature points and lane line information using the original data, and converting the obtained lane line information into lane line constraint information, where the lane line constraint information includes lane line direction constraint, lane line distance constraint, lane line curvature constraint, and lane line smoothness constraint, and converting the lane line constraint information into a series of algebraic equations to be applied to the solution problem of SLAM. Specifically: the lane line direction constraint restricts the movement direction of the camera between adjacent frames through the direction information of the lane line; the lane line distance constraint restricts the movement distance of the camera between adjacent frames through the distance information of the lane line; the lane line curvature constraint restricts the movement trajectory of the camera between adjacent frames through the curvature information of the lane line; the lane line smoothness constraint restricts the continuity of the lane line between adjacent frames through the smoothness information of the lane line; A dynamic point elimination module for performing a dynamic point elimination operation on the ORB feature points based on the lane line constraint information to obtain an ORB feature point set; A pose estimation module for using the ORB feature point set to perform camera pose estimation and pose optimization to obtain a camera pose estimation result; A map construction module for constructing a map based on the camera pose estimation result and the ORB feature point set.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the map construction method based on lane line semantics according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that, Computer program instructions are stored in the readable storage medium, and when the computer program instructions are read and run by a processor, the map construction method based on lane line semantics according to any one of claims 1 to 7 is executed.

Images