Lane line determination method and system, vehicle, and storage medium

Abstract

This application relates to a lane line determination method, a lane line determination system, a vehicle, and a storage medium. The lane line determination method includes: capturing a road image of a current location in a vehicle coordinate system; recognizing the road image and generating basic lane lines for the current location; extracting map lane lines for the current location; mapping the map lane lines to the vehicle coordinate system to obtain auxiliary lane lines; registering the basic lane lines and the auxiliary lane lines, wherein the registration is performed based on the confidence level of the basic lane lines; and generating a target lane line based on the registered auxiliary lane lines and the basic lane lines. The lane line determination method can improve the accuracy of lane line recognition by performing operations such as correcting visually captured lane lines using map lane lines based on conditions.

Description

Technical Field

[0001] This application relates to the field of lane recognition, and more specifically, to a lane line determination method, a lane line determination method system, a vehicle, and a storage medium. Background Technology

[0002] In driver assistance systems, accurate and stable identification of the vehicle's current lane and adjacent lanes is fundamental to many control functions. Lane recognition and construction heavily rely on lane line detection capabilities. Currently, most vehicles with driver assistance features use cameras to detect lane lines within a certain range in front of the vehicle and construct a lane based on these detection results. However, because camera detection is affected by environmental factors such as the actual scene, weather conditions, and other surrounding vehicles, the detection results may contain omissions and errors compared to the real physical world. This can lead to the vehicle being unable to construct correct lane information, ultimately causing the driver assistance function to disengage. This could potentially even jeopardize driving safety. Summary of the Invention

[0003] Embodiments of this application provide a lane line determination method, a lane line determination method system, a vehicle, and a storage medium, which can correct visually captured lane lines based on conditions using map lane lines.

[0004] According to one aspect of this application, a lane line determination method is provided, comprising: capturing a road image of a current location in a vehicle coordinate system; identifying the road image and generating a basic lane line for the current location; extracting a map lane line for the current location; mapping the map lane line to the vehicle coordinate system to obtain an auxiliary lane line; registering the basic lane line and the auxiliary lane line, the registration being performed based on the confidence level of the basic lane line; and generating a target lane line based on the registered auxiliary lane line and the basic lane line.

[0005] In some embodiments of this application, optionally, registering the base lane lines and the auxiliary lane lines includes: using lane lines in the base lane lines whose confidence exceeds a first threshold as lane lines to be registered; and registering the base lane lines and the auxiliary lane lines based on at least a portion of the lane lines to be registered.

[0006] Optionally, in some embodiments of this application, registering the base lane line and the auxiliary lane line includes: generating a point cloud group to be registered based on the point cloud in the auxiliary lane line; for each of the lane lines to be registered: calculating the relative distance between the point cloud group to be registered and the current lane line to be registered; and selecting the point cloud group to be registered with the lowest relative distance that is less than a second threshold as the registration point cloud group of the current lane line to be registered.

[0007] In some embodiments of this application, optionally, generating a target lane line based on the registered auxiliary lane line and the basic lane line includes: for the lane line to be registered, if the proportion of the lane line in which the registered point cloud group does not exist exceeds a third threshold, then the basic lane line is output as the target lane line.

[0008] Optionally, in some embodiments of this application, registering the base lane lines and the auxiliary lane lines includes: for the lane lines to be registered that have the registration point cloud group, performing point cloud registration between them and the corresponding registration point cloud group, wherein the point cloud registration includes: discretizing each of the lane lines to be registered corresponding to a predetermined field of view of the road image and using it as the target point cloud for the point cloud registration, wherein the density of the discretization is greater than the point cloud density of the registration point cloud group; using the registration point cloud group corresponding to the predetermined field of view as the starting point cloud for the point cloud registration; and determining the translation / rotation parameters from the starting point cloud to the target point cloud through an iterative method.

[0009] In some embodiments of this application, optionally, generating a target lane line based on the registered auxiliary lane line and the base lane line includes: if the translation / rotation parameter exceeds a predetermined threshold, then the base lane line is output as the target lane line.

[0010] In some embodiments of this application, optionally, generating a target lane line based on the registered auxiliary lane line and the basic lane line includes: during the point cloud registration process, if the remaining error between the point cloud obtained by transforming the starting point cloud according to the translation / rotation parameters and the target point cloud exceeds a fourth threshold, then the basic lane line is output as the target lane line.

[0011] Optionally, in some embodiments of this application, registering the base lane line and the auxiliary lane line includes: registering the auxiliary lane line and the base lane line according to the translation / rotation parameters; and generating a target lane line based on the registered auxiliary lane line and the base lane line includes: for each of the base lane lines: calculating the relative distance between the point cloud group in the registered auxiliary lane line and the current base lane line; using the point cloud group with the lowest relative distance as the calibration point cloud group of the current base lane line; and correcting the current base lane line using the calibration point cloud group.

[0012] In some embodiments of this application, optionally, generating a target lane line based on the registered auxiliary lane line and the base lane line includes: for lane lines in the base lane line whose length is less than a fifth threshold, if the calibration point cloud group exists, then the calibration point cloud group is used to fit a first lane curve, and the first lane curve is used to supplement it.

[0013] In some embodiments of this application, optionally, generating a target lane line based on the registered auxiliary lane line and the base lane line includes: for a first lane line in the base lane line with a confidence level less than a sixth threshold, if the first lane line contains the calibration point cloud group, then a second lane curve is fitted using the calibration point cloud group, and the fitting parameters of the second lane curve are used as parameters of the first lane line; and / or for a second lane line in the base lane line where the calibration point cloud group is not present, if: the second lane line overlaps with any point cloud group in the registered auxiliary lane line, but the relative distance of the overlapping portion exceeds a seventh threshold; and the relative distance between the base lane line and the registered auxiliary lane line is less than an eighth threshold; then: a third lane curve is fitted using the point cloud group in the overlapping portion, and the fitting parameters of the third lane curve are used as parameters of the second lane line.

[0014] According to another aspect of this application, a lane line determination system is provided, comprising: a capture unit configured to capture a road image of a current location in a vehicle coordinate system; a recognition unit configured to recognize the road image and generate basic lane lines for the current location; an extraction unit configured to extract map lane lines for the current location; a mapping unit configured to map the map lane lines to the vehicle coordinate system to obtain auxiliary lane lines; a registration unit configured to register the basic lane lines and the auxiliary lane lines, the registration being performed based on the confidence level of the basic lane lines; and a determination unit configured to generate a target lane line based on the registered auxiliary lane lines and the basic lane lines.

[0015] In some embodiments of this application, optionally, the registration unit is further configured to: use lane lines in the base lane lines whose confidence exceeds a first threshold as lane lines to be registered; and register the base lane lines with the auxiliary lane lines based on at least a portion of the lane lines to be registered.

[0016] In some embodiments of this application, optionally, the registration unit is further configured to: generate a set of point clouds to be registered based on the point clouds in the auxiliary lane lines; for each of the lane lines to be registered: calculate the relative distance between the set of point clouds to be registered and the current lane line to be registered; and take the set of point clouds to be registered with the lowest relative distance and less than a second threshold as the registration point cloud set of the current lane line to be registered.

[0017] In some embodiments of this application, optionally, the determining unit is further configured to: for the lane line to be registered, if the proportion of the lane line in which the registration point cloud group does not exist exceeds a third threshold, instruct the determining unit to output the base lane line as the target lane line.

[0018] In some embodiments of this application, optionally, the registration unit is further configured to: perform point cloud registration between the lane lines to be registered that have the registration point cloud group and the corresponding registration point cloud group, wherein the point cloud registration includes: discretizing each of the lane lines to be registered corresponding to a predetermined field of view of the road image and using it as the target point cloud for the point cloud registration, wherein the density of the discretization is greater than the point cloud density of the registration point cloud group; using the registration point cloud group corresponding to the predetermined field of view as the starting point cloud for the point cloud registration; and determining the translation / rotation parameters from the starting point cloud to the target point cloud through an iterative method.

[0019] In some embodiments of this application, optionally, the determining unit is further configured to output the base lane line as the target lane line if the translation / rotation parameter exceeds a predetermined threshold.

[0020] In some embodiments of this application, optionally, the determining unit is further configured to output the base lane line as the target lane line when the remaining error between the point cloud obtained by the registration unit transforming the starting point cloud according to the translation / rotation parameters and the target point cloud exceeds a fourth threshold.

[0021] In some embodiments of this application, optionally, the registration unit is further configured to: register the auxiliary lane line with the base lane line according to the translation / rotation parameters; and the determination unit is further configured to, for each of the base lane lines: calculate the relative distance between the point cloud group in the registered auxiliary lane line and the current base lane line; take the point cloud group with the lowest relative distance as the calibration point cloud group of the current base lane line; and correct the current base lane line with the calibration point cloud group.

[0022] In some embodiments of this application, optionally, the determining unit is further configured to: for lane lines in the basic lane lines whose length is less than the fifth threshold, if the calibration point cloud group exists, then use the calibration point cloud group to fit a first lane curve, and supplement it with the first lane curve.

[0023] Optionally, in some embodiments of this application, the determining unit is further configured to: for a first lane line in the basic lane lines with a confidence level less than a sixth threshold, if the first lane line contains the calibration point cloud group, then fit a second lane curve using the calibration point cloud group, and use the fitting parameters of the second lane curve as parameters of the first lane line; and / or for a second lane line in the basic lane lines where the calibration point cloud group does not exist, if:

[0024] If the second lane line overlaps with any point cloud group in the registered auxiliary lane line, but the relative distance of the overlapping part exceeds the seventh threshold; and the relative distance between the base lane line and the registered auxiliary lane line is less than the eighth threshold; then: fit the point cloud group in the overlapping part to obtain the third lane curve, and use the fitting parameters of the third lane curve as the parameters of the second lane line.

[0025] According to another aspect of this application, a vehicle is provided, including any of the lane line determination systems described above.

[0026] According to another aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored therein, characterized in that, when the instructions are executed by a processor, the processor performs any of the lane line determination methods described above. Attached Figure Description

[0027] The above and other objects and advantages of this application will become more fully clear from the following detailed description taken in conjunction with the accompanying drawings, wherein the same or similar elements are indicated by the same reference numerals.

[0028] Figure 1 A lane line determination method according to an embodiment of this application is shown.

[0029] Figure 2 A lane line determination system according to an embodiment of this application is shown. Detailed Implementation

[0030] For purposes of brevity and illustrativeness, the principles of this application are described herein primarily with reference to exemplary embodiments thereof. However, those skilled in the art will readily recognize that the same principles are equivalently applicable to all types of lane line determination methods, lane line determination system, vehicles, and storage media, and that these same or similar principles can be implemented therein, without departing from the true spirit and scope of this application.

[0031] The following specific implementation describes various thresholds. Although these thresholds may have different names, they may also have the same size if they have the same dimensions / units.

[0032] According to one aspect of this application, a lane line determination method is provided. In some examples, the lane line determination method uses lane line coordinate point clouds from a high-precision map and registers them with lane lines perceived by a camera, unifying the map lane lines in the positioning result into the camera coordinate system. Furthermore, by comparing the two sets of lane lines, lane lines that are missed / misidentified / have low confidence by the camera are identified, and corrected by fitting a curve using the corresponding lane line coordinate points from the high-precision map.

[0033] like Figure 1 As shown, the lane line determination method 10 includes the following steps: In step S102, the lane line determination method 10 captures a road image of the current location in the vehicle coordinate system; in step S104, it identifies the road image and generates basic lane lines for the current location; in step S106, it extracts map lane lines for the current location; in step S108, it maps the map lane lines to the vehicle coordinate system to obtain auxiliary lane lines; in step S110, it registers the basic lane lines and auxiliary lane lines, and the registration is performed based on the confidence level of the basic lane lines; in step S112, it generates target lane lines based on the registered auxiliary lane lines and basic lane lines.

[0034] Lane line determination method 10 captures a road image of the current location in the vehicle coordinate system in step S102, and identifies the road image and generates basic lane lines for the current location in step S104. In the following example, since a camera can be used to capture images and thereby generate basic lane lines, the basic lane lines are also called camera lane lines. Lane lines acquired and identified using image capture devices such as cameras can serve as a basis for autonomous driving / assisted driving. Since devices such as cameras are relatively fixed to the vehicle body, image capture, processing, and identification are more convenient in the vehicle coordinate system. The vehicle coordinate system in this invention refers to a coordinate system with a certain position on the vehicle body (e.g., the position of the camera) as the origin. In some examples, if the position of the camera is taken as the origin, the vehicle coordinate system is also called the camera coordinate system.

[0035] On the other hand, because vehicles may be in constant motion and the subjects captured by devices such as cameras are limited to a certain range, the identified lane lines are geographically dependent. In other words, the base lane lines obtained for each (current) location may be different.

[0036] Lane line determination method 10 extracts the map lane lines for the current location in step S106. For example, the vehicle's latitude and longitude coordinates can be calculated using the vehicle's GPS sensor, speed sensor, cornering sensor, and other motion sensors, and further calculated using the vehicle's positioning module. Then, based on these coordinates, the latitude and longitude coordinates of the lane lines within a certain range ahead, stored in point cloud (group) form, can be obtained from the high-precision map engine, and used as the (high-precision) map lane lines.

[0037] In step S108, lane line determination method 10 maps the map lane lines to the vehicle coordinate system to obtain auxiliary lane lines. For example, the vehicle's latitude and longitude coordinates and orientation calculated by the vehicle positioning module can be used to convert the high-precision map lane line point cloud to the vehicle coordinate system. Since the high-precision map lane lines are pre-collected and created, they are generally formed with the earth as a reference system. Therefore, in order to make the high-precision map lane lines usable on moving vehicles, they can be converted to the vehicle coordinate system.

[0038] In step S110 of lane line determination method 10, the basic lane lines and auxiliary lane lines are registered, and the registration is performed based on the confidence level of the basic lane lines. Considering that the positioning module has a certain error, and that the origin / orientation of the vehicle coordinate system under the positioning module is not consistent with the origin / orientation of the vehicle coordinate system (e.g., the camera coordinate system), registration is needed to correct the deviation between the two coordinate systems. It should be noted that registration is a prerequisite for a series of subsequent operations such as correction, therefore, the accuracy of registration must be improved. Therefore, the registration in step S110 is performed based on the confidence level of the basic lane lines to ensure the reliability of the registration.

[0039] In step S112, lane line determination method 10 generates the target lane line based on the registered auxiliary lane lines and the base lane lines. For example, lane lines that are missed / misidentified / recognized by the camera with low confidence can be corrected by fitting a curve using the corresponding lane line coordinate points in a high-precision map. In some examples, if the conditions described in detail below are not met, the registered auxiliary lane lines can be disregarded when generating the target lane line, and the target lane line can be generated solely based on the base lane lines. This can be considered a special case of "generating the target lane line based on the registered auxiliary lane lines and the base lane lines".

[0040] In some embodiments of this application, the registration of the base lane lines and auxiliary lane lines in step S110 is specifically achieved as follows: lane lines in the base lane lines with a confidence level exceeding a first threshold are designated as lane lines to be registered; and the base lane lines and auxiliary lane lines are registered based on at least a portion (e.g., all or some of the lane lines to be registered) of the lane lines to be registered. Thus, the following registration process will exclude lane lines with low confidence levels in the base lane lines, preventing these lane lines from affecting the registration effect and consequently impacting subsequent correction operations.

[0041] In some embodiments of this application, the process of registering the base lane lines and auxiliary lane lines in step S110 specifically includes: generating a set of point clouds to be registered based on the point clouds in the auxiliary lane lines, and performing the following operations for each of the lane lines to be registered until the operation is completed for all lane lines to be registered. Wherein, if the operation is performed on the i-th lane line, the i-th lane line is also referred to as the current lane line to be registered. Specific operations include: calculating the relative distance between the set of point clouds to be registered and the current lane line to be registered; and using the set of point clouds to be registered with the lowest relative distance that is less than a second threshold as the registration point cloud set for the current lane line to be registered. If the relative distances of all the set of point clouds to be registered with the lowest relative distances are less than the second threshold, then the base lane lines and auxiliary lane lines will be registered based on all the lane lines to be registered. If the relative distances of some of the set of point clouds to be registered with the lowest relative distances are all less than the second threshold, then the base lane lines and auxiliary lane lines will be registered based on these partial lane lines to be registered.

[0042] In some examples, for instance, the connection relationships between the point cloud groups of each lane line given by the high-definition map can be used to merge the connectable point cloud groups and generate all possible high-definition map point cloud groups to be paired. Then, for each lane line to be paired, the relative distance with each point cloud group to be paired on the map is calculated, and the point cloud group with the smallest relative distance and a relative distance less than a second threshold is selected as the paired point cloud group for that lane line to be paired (also known as the camera lane line).

[0043] In some embodiments of this application, step S112 of lane line determination method 10 specifically includes: for the lane line to be registered, if the proportion of lane lines without a registration point cloud group exceeds a third threshold, it is considered that the current camera measurement result does not match the map, and at this time, the basic lane line can be output as the target lane line. Furthermore, other calculation steps can be stopped, allowing the vehicle to return to a single-camera perception state.

[0044] In some embodiments of this application, if the proportion of lane lines without registration point cloud groups does not exceed a third threshold, step S110 further performs point cloud registration with the corresponding registration point cloud groups for lane lines with existing registration point cloud groups. More specifically, point cloud registration may include the following process: discretizing each lane line to be registered corresponding to a predetermined field of view of the road image and using it as the target point cloud for point cloud registration, wherein the density of the discretization is greater than the point cloud density of the registration point cloud group; using the registration point cloud group corresponding to the predetermined field of view as the starting point cloud for point cloud registration; and determining the translation / rotation parameters from the starting point cloud to the target point cloud through an iterative method.

[0045] Continuing the example above, we can perform registration using camera lane lines that have paired point cloud groups and their corresponding paired point cloud groups. First, the camera lane lines can be discretized, transforming them from continuous lines into several point clouds, with a discretization density greater than the density of the paired point cloud groups. The result of this discretization is called the registration target point cloud. Considering the reduced ability of cameras to recognize lane lines at a distance, we can select only lane lines within a certain range for discretization. For example, we can discretize only lane lines within a certain field of view.

[0046] Secondly, only points within the discrete range of the corresponding camera lane lines are selected from the paired point cloud group to form the registration starting point cloud. In other words, the registration point cloud group corresponding to the predetermined field of view mentioned above can be used as the starting point cloud for point cloud registration.

[0047] Finally, iterative optimization methods can be used to calculate the relative translation / rotation parameters between the starting point cloud and the target point cloud. During this process, it is necessary to select the points in the starting point cloud that are closest to each other within the target point cloud. Since the pairing relationship between the high-precision map lane line point cloud group and the camera lane line point cloud group has already been obtained, the selection range can be narrowed when choosing the nearest point. Furthermore, considering the relative relationships between points in the lane line point cloud, the selection range can be further narrowed based on the selection results of preceding points. This reduces the computational burden of registration calculations on the system.

[0048] In some embodiments of this application, if the translation / rotation parameters exceed a predetermined threshold, step S112 of lane line determination method 10 may specifically use the basic lane line as the target lane line output. In some examples, if the iteratively obtained translation / rotation parameters are large, it is considered that there is a large initial deviation between the camera and the high-precision map, and that one of the sensors has failed. At this time, other calculation steps can be stopped and the system can revert to a single-camera perception state, outputting only the basic lane line as the target lane line.

[0049] In some embodiments of this application, step S112 of lane line determination method 10 specifically includes: during point cloud registration, if the remaining error between the point cloud obtained by transforming the starting point cloud according to translation / rotation parameters and the target point cloud exceeds a fourth threshold, then the basic lane line is output as the target lane line. In some examples, the starting point cloud is transformed using the iteratively obtained translation / rotation parameters, and the remaining error between the transformed point cloud and the target point cloud is calculated. If the error is too large, the registration is considered to have failed. At this time, other calculation steps can be stopped and the system can return to the single-camera perception state, outputting only the basic lane line as the target lane line.

[0050] In some embodiments of this application, the process of registering the base lane lines and auxiliary lane lines in step S110 specifically includes: registering the auxiliary lane lines and base lane lines according to translation / rotation parameters. The process of generating the target lane line based on the registered auxiliary lane lines and base lane lines in step S112 specifically includes the following steps: For each base lane line: calculating the relative distance between the point cloud group in the registered auxiliary lane line and the current base lane line; using the point cloud group with the lowest relative distance as the calibration point cloud group for the current base lane line; and correcting the current base lane line using the calibration point cloud group (e.g., supplementing, correcting, etc.).

[0051] For example, based on the registration results, the coordinates of all map lane line points can be transformed to the camera coordinate system. At this point, the map lane lines and the camera lane lines are most similar. Then, the relative distances between each lane line perceived by the camera and each point cloud group of the map lane lines can be compared, and the point cloud group with the smallest relative distance can be selected as the corrected pair.

[0052] In some embodiments of this application, generating a target lane line based on the registered auxiliary lane line and the base lane line includes: for lane lines in the base lane line whose length is less than a fifth threshold, if a calibration point cloud group exists, then the calibration point cloud group is used to fit a first lane curve, and the first lane curve is used to supplement it.

[0053] In some examples, map lane line information can be used to supplement the camera's perception results. For instance, unpaired map lane line point cloud groups can be selected, merged according to connectivity, and fitted into a cubic curve to supplement the lane line pool for subsequent lane generation. Alternatively, for lane lines perceived by the camera that are relatively short, and for which there are paired map lane line point cloud groups, the paired point cloud groups can be fitted into a cubic curve to extend the camera's lane lines.

[0054] In some embodiments of this application, generating a target lane line based on the registered auxiliary lane line and the base lane line includes: for a first lane line in the base lane line with a confidence level less than a sixth threshold, if the first lane line has a calibration point cloud group, then a second lane curve is fitted with the calibration point cloud group, and the fitting parameters of the second lane curve are used as parameters of the first lane line; and / or for a second lane line in the base lane line without a calibration point cloud group, if the following conditions are met simultaneously: (1) the second lane line overlaps with any point cloud group in the registered auxiliary lane line, but the relative distance of the overlapping part exceeds a seventh threshold; (2) the relative distance between the base lane line and the registered auxiliary lane line is less than an eighth threshold; then a third lane curve can be fitted with the point cloud group in the overlapping part, and the fitting parameters of the third lane curve are used as parameters of the second lane line.

[0055] In some examples, map lane line information can be used to correct camera perception results. For instance, for lane lines with low camera perception confidence, if a paired map lane line point cloud group exists, a cubic curve can be fitted from the map point cloud group, and the fitting parameters can replace the corresponding camera lane line parameters. As another example, if a camera-perceived lane line has no paired map lane line point cloud group, and partially overlaps with a map lane line point cloud group but at a large relative distance, and most other camera lane lines are well-paired with the remaining map lane line point cloud groups, then this camera lane line can be considered a misidentification, and its parameters can be replaced with the fitted curve from the partially overlapping map lane line point cloud group.

[0056] According to another aspect of this application, a lane line determination system is provided. For example... Figure 2 As shown, the lane line determination system 20 includes a capture unit 202, a recognition unit 204, an extraction unit 206, a mapping unit 208, a registration unit 210, and a determination unit 212.

[0057] The lane determination system 20's capture unit 202 is configured to capture a road image of the current location in a vehicle coordinate system, and the recognition unit 204 is configured to recognize the road image and generate basic lane lines for the current location. In the following example, since a camera can be used to capture images and thereby generate basic lane lines, the basic lane lines are also referred to as camera lane lines. Lane lines acquired and recognized using image capture devices such as cameras can serve as a basis for autonomous / assisted driving. Since devices such as cameras are relatively fixed to the vehicle body, image capture, processing, and recognition are more convenient in a vehicle coordinate system. The vehicle coordinate system in this invention refers to a coordinate system with a certain position on the vehicle body (e.g., the position of the camera) as the origin. In some examples, if the position of the camera is taken as the origin, the vehicle coordinate system is also called the camera coordinate system.

[0058] On the other hand, because vehicles may be in constant motion and the subjects captured by devices such as cameras are limited to a certain range, the identified lane lines are geographically dependent. In other words, the base lane lines obtained for each (current) location may be different.

[0059] The extraction unit 206 of the lane line determination system 20 is configured to extract the map lane lines for the current location. For example, the vehicle's latitude and longitude coordinates can be calculated using the vehicle's GPS sensor, speed sensor, cornering sensor, and other motion sensors, and further calculated using the vehicle's positioning module. Then, the extraction unit 206 can obtain the latitude and longitude coordinates of the lane lines within a certain range ahead, stored in point cloud (group) form, from the high-precision map engine based on these coordinates, and use them as the (high-precision) map lane lines.

[0060] The mapping unit 208 of the lane line determination system 20 is configured to map map lane lines to the vehicle coordinate system to obtain auxiliary lane lines. For example, the mapping unit 208 can use the vehicle's latitude and longitude coordinates and orientation calculated by the vehicle positioning module to transform the high-precision map lane line point cloud into the vehicle coordinate system. Since the high-precision map lane lines are pre-collected and created, they are generally formed with the earth as a reference system. Therefore, in order to make the high-precision map lane lines usable on a moving vehicle, they can be transformed into the vehicle coordinate system.

[0061] The registration unit 210 of the lane line determination system 20 is configured to register base lane lines and auxiliary lane lines. Registration is performed based on the confidence level of the base lane lines. Considering that the positioning module has a certain degree of error, and that the origin / orientation of the vehicle's coordinate system under the positioning module is not consistent with the origin / orientation of the vehicle coordinate system (e.g., the camera coordinate system), registration is needed to correct the deviation between the two coordinate systems. It should be noted that registration is a prerequisite for a series of subsequent operations such as correction, therefore, the accuracy of registration must be improved. Therefore, the registration operation of the registration unit 210 is performed based on the confidence level of the base lane lines to ensure the reliability of the registration.

[0062] The determination unit 212 of the lane line determination system 20 is configured to generate a target lane line based on the registered auxiliary lane lines and the base lane lines. For example, lane lines that are missed / misidentified / recognized by the camera with low confidence can be corrected by fitting a curve using the corresponding lane line coordinate points in a high-precision map. In some examples, if the conditions described in detail below are not met, the determination unit 212 may also generate the target lane line without considering the registered auxiliary lane lines, and only based on the base lane lines. This can be considered a special case of "generating the target lane line based on the registered auxiliary lane lines and the base lane lines".

[0063] In some embodiments of this application, the registration unit 210 is further configured to: designate lane lines in the base lane lines with a confidence level exceeding a first threshold as lane lines to be registered; and register the base lane lines and auxiliary lane lines based on at least a portion (e.g., all or some of the lane lines to be registered) of the lane lines to be registered. Thus, the following registration process will exclude lane lines with low confidence levels in the base lane lines, preventing these lane lines from affecting the registration effect and consequently impacting subsequent correction operations.

[0064] In some embodiments of this application, the registration unit 210 is further configured to: generate a set of point clouds to be registered based on the point clouds in the auxiliary lane lines and perform the following operations for each of the lane lines to be registered until the operation is completed for all lane lines to be registered. Wherein, if the operation is performed on the i-th lane line, the i-th lane line is also referred to as the current lane line to be registered. Specific operations include: calculating the relative distance between the set of point clouds to be registered and the current lane line to be registered; and using the set of point clouds to be registered with the lowest relative distance that is less than a second threshold as the registration point cloud set for the current lane line to be registered. If the relative distances of all the set of point clouds to be registered with the lowest relative distances are less than the second threshold, the registration unit 210 will register the base lane line and the auxiliary lane line based on all the lane lines to be registered. If the relative distances of some of the set of point clouds to be registered with the lowest relative distances are all less than the second threshold, the registration unit 210 will register the base lane line and the auxiliary lane line based on these partial lane lines to be registered.

[0065] In some examples, for instance, the connection relationships between the point cloud groups of each lane line given by the high-definition map can be used to merge the connectable point cloud groups and generate all possible high-definition map point cloud groups to be paired. Then, for each lane line to be paired, the relative distance with each point cloud group to be paired on the map is calculated, and the point cloud group with the smallest relative distance and a relative distance less than a second threshold is selected as the paired point cloud group for that lane line to be paired (also known as the camera lane line).

[0066] In some embodiments of this application, the determining unit 212 is further configured to: for a lane line to be registered, if the proportion of lane lines without registered point cloud groups exceeds a third threshold, instruct the determining unit 212 to output the base lane line as the target lane line. Furthermore, other calculation steps can be stopped, allowing the vehicle to return to a single-camera perception state.

[0067] In some embodiments of this application, if the proportion of lane lines without registration point cloud groups does not exceed a third threshold, the registration unit 210 is further configured to perform point cloud registration with the corresponding registration point cloud group for lane lines with existing registration point cloud groups. More specifically, the point cloud registration may include the following process: discretizing each lane line to be registered corresponding to a predetermined field of view of the road image and using it as the target point cloud for point cloud registration, wherein the density of the discretization is greater than the point cloud density of the registration point cloud group; using the registration point cloud group corresponding to the predetermined field of view as the starting point cloud for point cloud registration; and determining the translation / rotation parameters from the starting point cloud to the target point cloud through an iterative method.

[0068] Continuing the example above, we can perform registration using camera lane lines that have paired point cloud groups and their corresponding paired point cloud groups. First, the camera lane lines can be discretized, transforming them from continuous lines into several point clouds, with a discretization density greater than the density of the paired point cloud groups. The result of this discretization is called the registration target point cloud. Considering the reduced ability of cameras to recognize lane lines at a distance, we can select only lane lines within a certain range for discretization. For example, we can discretize only lane lines within a certain field of view.

[0069] Secondly, only points within the discrete range of the corresponding camera lane lines are selected from the paired point cloud group to form the registration starting point cloud. In other words, the registration point cloud group corresponding to the predetermined field of view mentioned above can be used as the starting point cloud for point cloud registration.

[0070] Finally, iterative optimization methods can be used to calculate the relative translation / rotation parameters between the starting point cloud and the target point cloud. During this process, it is necessary to select the points in the starting point cloud that are closest to each other within the target point cloud. Since the pairing relationship between the high-precision map lane line point cloud group and the camera lane line point cloud group has already been obtained, the selection range can be narrowed when choosing the nearest point. Furthermore, considering the relative relationships between points in the lane line point cloud, the selection range can be further narrowed based on the selection results of preceding points. This reduces the computational burden of registration calculations on the system.

[0071] In some embodiments of this application, the determining unit 212 is further configured to output the basic lane line as the target lane line when the translation / rotation parameters exceed a predetermined threshold. In some examples, if the iteratively obtained translation / rotation parameters are large, it is considered that there is a large initial deviation between the camera and the high-precision map, and that one of the sensors has failed. At this time, other calculation steps can be stopped and the system can revert to a single-camera perception state, outputting only the basic lane line as the target lane line.

[0072] In some embodiments of this application, the determining unit 212 is further configured to output the basic lane line as the target lane line if the residual error between the point cloud obtained by the registration unit 210 transforming the starting point cloud according to the translation / rotation parameters and the target point cloud exceeds a fourth threshold. In some examples, the starting point cloud is transformed using the iteratively obtained translation / rotation parameters, and the residual error between the transformed point cloud and the target point cloud is calculated. If the error is too large, the registration is considered to have failed. At this time, other calculation steps can be stopped and the system can return to the single-camera perception state, outputting only the basic lane line as the target lane line.

[0073] In some embodiments of this application, the registration unit 210 is further configured to register the auxiliary lane line with the base lane line according to translation / rotation parameters, and the determination unit 212 is further configured to, for each of the base lane lines: calculate the relative distance between the point cloud group in the registered auxiliary lane line and the current base lane line; take the point cloud group with the lowest relative distance as the calibration point cloud group of the current base lane line; and correct the current base lane line with the calibration point cloud group (e.g., supplement, correct, etc.).

[0074] For example, based on the registration results, the coordinates of all map lane line points can be transformed to the camera coordinate system. At this point, the map lane lines and the camera lane lines are most similar. Then, the relative distances between each lane line perceived by the camera and each point cloud group of the map lane lines can be compared, and the point cloud group with the smallest relative distance can be selected as the corrected pair.

[0075] In some embodiments of this application, the determining unit 212 is further configured to: for lane lines in the basic lane lines whose length is less than the fifth threshold, if a calibration point cloud group exists, fit the first lane curve using the calibration point cloud group and supplement it with the first lane curve.

[0076] In some examples, map lane line information can be used to supplement the camera's perception results. For instance, unpaired map lane line point cloud groups can be selected, merged according to connectivity, and fitted into a cubic curve to supplement the lane line pool for subsequent lane generation. Alternatively, for lane lines perceived by the camera that are relatively short, and for which there are paired map lane line point cloud groups, the paired point cloud groups can be fitted into a cubic curve to extend the camera's lane lines.

[0077] In some embodiments of this application, the determining unit 212 is further configured to: for a first lane line in the basic lane line with a confidence level less than the sixth threshold, if the first lane line has a calibration point cloud group, then the second lane curve is fitted with the calibration point cloud group, and the fitting parameters of the second lane curve are used as the parameters of the first lane line; and / or for a second lane line in the basic lane line without a calibration point cloud group, if the following conditions are met simultaneously: (1) the second lane line overlaps with any point cloud group in the registered auxiliary lane line, but the relative distance of the overlapping part exceeds the seventh threshold; (2) the relative distance between the basic lane line and the registered auxiliary lane line is less than the eighth threshold; then the third lane curve can be fitted with the point cloud group in the overlapping part, and the fitting parameters of the third lane curve are used as the parameters of the second lane line.

[0078] In some examples, map lane line information can be used to correct camera perception results. For instance, for lane lines with low camera perception confidence, if a paired map lane line point cloud group exists, a cubic curve can be fitted from the map point cloud group, and the fitting parameters can replace the corresponding camera lane line parameters. As another example, if a camera-perceived lane line has no paired map lane line point cloud group, and partially overlaps with a map lane line point cloud group but at a large relative distance, and most other camera lane lines are well-paired with the remaining map lane line point cloud groups, then this camera lane line can be considered a misidentification, and its parameters can be replaced with the fitted curve from the partially overlapping map lane line point cloud group.

[0079] According to another aspect of this application, a vehicle is provided, including any of the lane line determination systems described above. A vehicle equipped with a lane line determination system can perform operations such as correcting visually captured lane lines using map lane lines based on conditions, thereby improving the accuracy of lane line recognition.

[0080] According to another aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored that, when executed by a processor, cause the processor to perform any of the lane line determination methods described above. The computer-readable medium referred to in this application includes various types of computer storage media, and can be any available medium accessible by a general-purpose or special-purpose computer. For example, the computer-readable medium may include RAM, ROM, EPROM, E... 2 PROM, registers, hard disks, removable disks, CD-ROMs or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other temporary or non-temporary medium capable of carrying or storing desired program code units in the form of instructions or data structures and accessible by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. As used herein, disks typically magnetically copy data, while discs optically copy data using lasers. Combinations of the above should also be included within the scope of computer-readable media. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0081] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Those skilled in the art can conceive of other feasible variations or substitutions based on the technical scope disclosed in this application, and such variations or substitutions are all covered within the scope of protection of this application. Where there is no conflict, the embodiments and features described in the embodiments of this application can also be combined with each other. The scope of protection of this application is determined by the claims.

Claims

1. A method for determining lane lines, comprising: Capture a road image of the current location in the vehicle coordinate system; Identify the road image and generate basic lane lines for the current location; Extract the map lane lines for the current location; The auxiliary lane lines are obtained by mapping the map lane lines to the vehicle coordinate system. Registering the base lane lines with the auxiliary lane lines, wherein the registration is performed based on the confidence level of the base lane lines, and wherein registering the base lane lines with the auxiliary lane lines includes: Lanes with a confidence level exceeding a first threshold in the basic lane lines are designated as lanes to be registered. Register the base lane lines and the auxiliary lane lines based on at least a portion of the lane lines to be registered; A point cloud group to be registered is generated based on the point cloud in the auxiliary lane lines; For each of the lane lines to be registered: Calculate the relative distance between the point cloud group to be registered and the current lane line to be registered; and The point cloud group to be registered that has the lowest relative distance and is less than the second threshold is taken as the registration point cloud group of the current lane line to be registered. For the lane line to be registered that exists in the registration point cloud group, perform point cloud registration between it and the corresponding registration point cloud group. The point cloud registration includes: Each lane line to be registered corresponding to a predetermined field of view of the road image is discretized and used as the target point cloud for the point cloud registration, wherein the density of the discretization is greater than the point cloud density of the registration point cloud group. The registration point cloud group corresponding to the predetermined field of view is used as the starting point cloud for point cloud registration; and The translation / rotation parameters from the starting point cloud to the target point cloud are determined iteratively; and The target lane line is generated based on the registered auxiliary lane line and the basic lane line.

2. The method according to claim 1, wherein, Generating a target lane line based on the registered auxiliary lane line and the basic lane line includes: for the lane line to be registered, if the proportion of the lane line in which the registered point cloud group does not exist exceeds a third threshold, then the basic lane line is output as the target lane line.

3. The method according to claim 2, wherein, Generating a target lane line based on the registered auxiliary lane line and the base lane line includes: if the translation / rotation parameter exceeds a predetermined threshold, then the base lane line is output as the target lane line.

4. The method according to claim 2, wherein, The target lane line is generated based on the registered auxiliary lane line and the basic lane line: During the point cloud registration process, if the remaining error between the point cloud obtained by transforming the starting point cloud according to the translation / rotation parameters and the target point cloud exceeds the fourth threshold, the basic lane line is output as the target lane line.

5. The method according to claim 2, wherein, Registering the basic lane lines and the auxiliary lane lines includes: Register the auxiliary lane lines with the base lane lines according to the translation / rotation parameters; and Generating a target lane line based on the registered auxiliary lane line and the basic lane line includes: For each of the aforementioned basic lane lines: Calculate the relative distance between the point cloud group in the registered auxiliary lane line and the current base lane line; The point cloud group with the lowest relative distance is used as the calibration point cloud group for the current basic lane line; and The current basic lane lines are corrected using the calibration point cloud group.

6. The method according to claim 5, wherein, Generating a target lane line based on the registered auxiliary lane line and the basic lane line includes: For lane lines in the basic lane lines whose length is less than the fifth threshold, if the calibration point cloud group exists, the calibration point cloud group is used to fit the first lane curve, and the first lane curve is used to supplement it.

7. The method according to claim 5, wherein, Generating a target lane line based on the registered auxiliary lane line and the basic lane line includes: For the first lane line in the basic lane lines with a confidence level less than the sixth threshold, if the first lane line exists within the calibration point cloud group, then the second lane curve is fitted using the calibration point cloud group, and the fitting parameters of the second lane curve are used as the parameters of the first lane line; and / or For a second lane line in the basic lane lines that does not contain the calibration point cloud group, if: The second lane line overlaps with any point cloud group in the registered auxiliary lane line, but the relative distance of the overlapping portion exceeds the seventh threshold; and The relative distance between the basic lane line and the registered auxiliary lane line is less than the eighth threshold. but: The third lane curve is fitted using the point cloud group in the overlapping part, and the fitting parameters of the third lane curve are used as the parameters of the second lane line.

8. A lane line determination system, comprising: A capture unit configured to capture a road image of the current location in the vehicle coordinate system; A recognition unit configured to recognize the road image and generate basic lane lines for the current location; An extraction unit configured to extract map lane lines about the current location; A mapping unit configured to map the map lane lines to the vehicle coordinate system to obtain auxiliary lane lines; A registration unit configured to register the base lane lines and the auxiliary lane lines, wherein the registration is performed based on the confidence level of the base lane lines; as well as A determining unit is configured to generate a target lane line based on the registered auxiliary lane line and the base lane line, wherein the registration unit is further configured to: Lanes with a confidence level exceeding a first threshold in the basic lane lines are designated as lanes to be registered. Register the base lane lines and the auxiliary lane lines based on at least a portion of the lane lines to be registered. A point cloud group to be registered is generated based on the point cloud in the auxiliary lane lines; For each of the lane lines to be registered: Calculate the relative distance between the point cloud group to be registered and the current lane line to be registered; and The point cloud group to be registered that has the lowest relative distance and is less than the second threshold is taken as the registration point cloud group of the current lane line to be registered. For the lane line to be registered that exists in the registration point cloud group, perform point cloud registration between it and the corresponding registration point cloud group. The point cloud registration includes: Each lane line to be registered corresponding to a predetermined field of view of the road image is discretized and used as the target point cloud for the point cloud registration, wherein the density of the discretization is greater than the point cloud density of the registration point cloud group. The registration point cloud group corresponding to the predetermined field of view is used as the starting point cloud for point cloud registration; and The translation / rotation parameters from the starting point cloud to the target point cloud are determined iteratively.

9. The system according to claim 8, wherein, The determining unit is further configured to: for the lane line to be registered, if the proportion of the lane line in which the registration point cloud group does not exist exceeds a third threshold, instruct the determining unit to output the basic lane line as the target lane line.

10. The system according to claim 9, wherein, The determining unit is further configured to output the base lane line as the target lane line when the translation / rotation parameter exceeds a predetermined threshold.

11. The system according to claim 9, wherein, The determining unit is further configured to output the base lane line as the target lane line when the remaining error between the point cloud obtained by the registration unit transforming the starting point cloud according to the translation / rotation parameters and the target point cloud exceeds a fourth threshold.

12. The system according to claim 9, wherein, The registration unit is further configured to: Register the auxiliary lane lines with the base lane lines according to the translation / rotation parameters; and The determining unit is further configured to target each of the basic lane lines: Calculate the relative distance between the point cloud group in the registered auxiliary lane line and the current base lane line; The point cloud group with the lowest relative distance is used as the calibration point cloud group for the current basic lane line; and The current basic lane lines are corrected using the calibration point cloud group.

13. The system according to claim 12, wherein, The determining unit is further configured to: for lane lines in the basic lane lines whose length is less than the fifth threshold, if the calibration point cloud group exists, then use the calibration point cloud group to fit the first lane curve, and supplement it with the first lane curve.

14. The system according to claim 13, wherein, The determining unit is further configured to: For the first lane line in the basic lane lines with a confidence level less than the sixth threshold, if the first lane line exists within the calibration point cloud group, then the second lane curve is fitted using the calibration point cloud group, and the fitting parameters of the second lane curve are used as the parameters of the first lane line; and / or For a second lane line in the basic lane lines that does not contain the calibration point cloud group, if: The second lane line overlaps with any point cloud group in the registered auxiliary lane line, but the relative distance of the overlapping portion exceeds the seventh threshold; and The relative distance between the basic lane line and the registered auxiliary lane line is less than the eighth threshold. but: The third lane curve is fitted using the point cloud group in the overlapping part, and the fitting parameters of the third lane curve are used as the parameters of the second lane line.

15. A vehicle comprising any one of the lane marking systems as described in claims 8-14.

16. A computer-readable storage medium storing instructions, characterized in that, When the instruction is executed by the processor, it causes the processor to perform the method as described in any one of claims 1-7.

Images