Lane line topological relation identification method and device, controller and mobile equipment

Abstract

The invention provides a lane line topological relation identification method and device, a controller, mobile equipment and a computer storage medium, and the method comprises the steps: obtaining a road image comprising images of a plurality of lane lines, carrying out the recognition of the position relation of the lane lines in the road image, determining at least one group of adjacent lane lines, and carrying out the recognition of the position relation of the lane lines in the road image; the method comprises the following steps: projecting lane lines in a road image to a reference plane, determining a starting point and an ending point of the obtained projected lane lines, carrying out nearest neighbor search on two projected lane lines corresponding to adjacent lane lines, and respectively determining a starting point nearest neighbor point and an ending point nearest neighbor point of the two projected lane lines, and determining the topological relation between the two lane lines corresponding to the adjacent lane lines according to the starting points, the ending points and the distance between at least parts of the nearest point of the starting points and the nearest point of the ending points of the two projection lane lines corresponding to the adjacent lane lines. According to the lane line topological relation identification method, identification of the lane line topological relation can be completed without a high-precision map.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of autonomous driving technology, and in particular to a method, apparatus, controller, mobile device, and computer storage medium for identifying lane line topological relationships. Background Art

[0002] On structured roads, lane markings define the vehicle's driving boundaries and serve as crucial reference for vehicle planning and control. When using visual sensors to reconstruct and associate lane markings from images, lane topology is often required as prior input, and different processing logic is employed based on the different lane topologies.

[0003] Currently, the primary method for identifying lane topology relationships is through pre-created high-definition maps (HD-Maps). HD-Maps segment roads and provide lane and lane boundary information for each segment. For each lane, information about the left and right lane lines, as well as their lateral and vertical connections, is provided. This information is manually annotated and stored in the HD-Map during the HD-Map production process.

[0004] However, existing lane topology recognition methods rely too heavily on high-precision maps, making it difficult to directly identify lane topology in scenarios not covered by these maps. Furthermore, the high cost of producing, annotating, and updating high-precision maps, coupled with their limited coverage, mean that existing technologies suffer from significant disadvantages in terms of cost and coverage.

[0005] Application Contents

[0006] In view of this, embodiments of the present application provide a lane line topology relationship identification method, apparatus, controller, mobile device, and computer storage medium to at least partially solve the above-mentioned problems.

[0007] According to a first aspect of an embodiment of the present application, a method for identifying lane line topological relationships is provided, comprising: acquiring a road image including an image of a plurality of lane lines; identifying the positional relationship of lane lines in the road image, and determining at least one group of adjacent lane lines, wherein the adjacent lane lines include two adjacent lane lines in the road image; projecting the lane lines in the road image onto a reference plane, and determining the start and end points of the obtained projected lane lines, wherein the reference plane is parallel to the plane where the vehicle is located, and the road image is collected by a visual sensor installed on the vehicle; and performing positional analysis on the two projected lane lines corresponding to the adjacent lane lines. Perform nearest neighbor searches on each other to respectively determine the starting point nearest neighbor points and the ending point nearest neighbor points of the two projected lane lines, wherein the starting point nearest neighbor point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the starting point of the projected lane line, and the ending point nearest neighbor point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the ending point of the projected lane line; determine the topological relationship between the two lane lines corresponding to the adjacent lane lines based on the distances between at least part of the starting points, ending points, starting point nearest neighbor points, and ending point nearest neighbor points of the two projected lane lines corresponding to the adjacent lane lines.

[0008] According to a second aspect of an embodiment of the present application, a lane line topology relationship recognition device is provided, comprising: an acquisition module for acquiring a road image including an image of a plurality of lane lines; a determination module for identifying the positional relationship of lane lines in the road image and determining at least one group of adjacent lane lines, wherein the adjacent lane lines include two adjacent lane lines in the road image; a projection module for projecting the lane lines in the road image onto a reference plane and determining the start and end points of the obtained projected lane lines, wherein the reference plane is parallel to the plane where the vehicle is located, and the road image is collected by a visual sensor mounted on the vehicle; a search module for searching the adjacent lane line pairs. The two corresponding projected lane lines perform a nearest neighbor search on each other, and determine the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines respectively, wherein the nearest neighbor point of the starting point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the starting point of the projected lane line, and the nearest neighbor point of the ending point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the ending point of the projected lane line; an identification module is used to determine the topological relationship between the two lane lines corresponding to the adjacent lane lines based on the distance between at least part of the starting points, the ending points, the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines corresponding to the adjacent lane lines.

[0009] According to a third aspect of an embodiment of the present application, a controller is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, and the memory is used to store a computer program; the processor is used to execute the method steps described in any one of the first aspects of the embodiment by running the computer program stored on the memory.

[0010] According to a fourth aspect of an embodiment of the present application, a mobile device is provided, comprising: a sensor, a processor, a memory, a communication interface, and a communication bus, wherein the sensor, the processor, the memory, and the communication interface communicate with each other via the communication bus; the sensor is used to collect environmental data of the environment in which the mobile device is located and generate perception data; the memory is used to store at least one executable instruction, wherein the executable instruction enables the processor to perform an operation corresponding to the method described in any one of the first aspects of the embodiment based on the perception data.

[0011] According to a fifth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method as described in any one of the first aspects of the embodiment is implemented.

[0012] The above technical solution processes a road image containing multiple lane lines and determines the topological relationship between two adjacent lane lines using a nearest neighbor search to determine the nearest neighbor points of the start and end points of the two projected lane lines. This allows lane line topological relationship recognition without the need for a high-precision map. Furthermore, since only the distances between the start and end points, and at least a portion of the nearest neighbor points of the start and end points of the two projected lane lines corresponding to the adjacent lane lines need to be obtained, manual labeling is not required, thereby improving the efficiency of lane line topological relationship recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0014] Figure 1 Flowchart of the steps of a method for identifying lane topological relationships according to an embodiment of the present application;

[0015] Figure 2 This is a flowchart of a method for determining adjacent lane lines according to an embodiment of the present application;

[0016] Figure 3 This is a flowchart of the steps of a method for determining the starting point and end point of a projected lane line according to an embodiment of the present application;

[0017] Figure 4 Schematic diagram of the topological relationship between two projected lane lines corresponding to adjacent lane lines according to an embodiment of the present application;

[0018] Figure 5 1 is a structural block diagram of a lane topology relationship recognition device according to an embodiment of the present application;

[0019] Figure 6 is a structural block diagram of a controller according to an embodiment of the present application;

[0020] Figure 7 The figure is a structural block diagram of a mobile device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0022] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0023] Figure 1 This is a flowchart of a method for identifying lane topology relationships according to an embodiment of the present application. Figure 1 As shown, the lane line topology relationship recognition method includes:

[0024] Step 101: Acquire a road image including images of multiple lane lines.

[0025] To identify lane line topology, a road image including multiple lane lines must first be acquired. The road image can be acquired using a visual sensor, which can be a camera, a camera, a light source sensor, and / or any other suitable type of device.

[0026] It should be noted that the road image can be acquired through one or more visual sensors, which is not limited here.

[0027] Step 102: Identify the positional relationship of lane lines in the road image and determine at least one group of adjacent lane lines.

[0028] After acquiring the road image, the positional relationship of the lane lines in the road image is identified. The identification method can be through a pre-trained model or through feature points in the lane lines. After the lane lines are identified, the adjacent lane lines are determined based on the positional relationship of the lane lines in the road image, or the lane line feature points satisfying a preset position judgment relationship. Adjacent lane lines include two adjacent lane lines in the road image. Since there can be multiple lane lines in the road image, at least one group of adjacent lane lines can be determined.

[0029] Step 103: Project the lane lines in the road image onto a reference plane, and determine the start and end points of the projected lane lines.

[0030] Because lane markings are painted on the road, they lie on the same two-dimensional plane as the road itself. Therefore, lane line topology must be based on relationships on this plane. Therefore, the lane lines in the road image can be projected onto a reference plane parallel to the vehicle's plane. The road image is captured by a visual sensor mounted on the vehicle. After projection, the start and end points of the projected lane lines can be determined. This can be done by using lane line feature points or by constructing lane line vectors.

[0031] It should be noted that since this application is based on an autonomous driving scenario, the plane where the vehicle is located can be a plane parallel to the road on which the vehicle is traveling, and then it can be determined that the reference plane is parallel to the two-dimensional plane where the road is located, and then the lane line topological relationship identified based on the starting point and end point of the projected lane line on the reference plane can indicate the topological relationship of the lane line in the road.

[0032] Step 104: Perform a nearest neighbor search on the two projected lane lines corresponding to the adjacent lane lines to determine the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines respectively.

[0033] After determining the start and end points of the projected lane line, a nearest neighbor search is performed on the two projected lane lines corresponding to the adjacent lane lines in combination with at least one group of adjacent lane lines. The nearest neighbor search can be implemented based on searching for the shortest Euclidean distance or Manhattan distance between the two projected lane lines corresponding to the adjacent lane lines to determine the nearest neighbor points of the start and end points of the two projected lane lines.

[0034] The nearest point to the starting point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the starting point of the projected lane line, and the nearest point to the ending point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the ending point of the projected lane line.

[0035] Step 105: Determine a topological relationship between the two projected lane lines corresponding to the adjacent lane lines based on a distance between at least a portion of the starting point, the ending point, the nearest neighbor point of the starting point, and the nearest neighbor point of the ending point of the two projected lane lines corresponding to the adjacent lane lines.

[0036] After determining the starting point nearest neighbor points and the ending point nearest neighbor points of the two projected lane lines, the topological relationship between the two lane lines corresponding to the adjacent lane lines is determined based on the distances between at least part of the starting point, the ending point, the starting point nearest neighbor points, and the ending point nearest neighbor points of the two projected lane lines corresponding to the adjacent lane lines, such as the distance between the starting point and the starting point nearest neighbor points of the two projected lane lines, the distance between the ending point and the ending point nearest neighbor points of the two projected lane lines, and the distance between the starting point and the ending point nearest neighbor points of the two projected lane lines.

[0037] In an embodiment of the present application, by processing a road image including multiple lane lines and determining the topological relationship between two adjacent lane lines using a nearest neighbor search to determine the nearest neighbor points of the starting and ending points of the two projected lane lines, lane line topological relationship recognition can be completed without the need for a high-precision map. Furthermore, since only the distances between the starting and ending points, and at least a portion of the nearest neighbor points of the starting and ending points of the two projected lane lines corresponding to the adjacent lane lines need to be obtained, manual labeling and other methods are not required, thereby improving the efficiency of lane line topological relationship recognition.

[0038] Figure 2 FIG. 1 is a flowchart of a method for determining adjacent lane lines according to an embodiment of the present application. Figure 2 As shown, the adjacent lane line determination method includes:

[0039] Step 201: perform lane line extraction on a road image to obtain a lane line extraction result.

[0040] After acquiring a road image, lane line extraction can be performed on the road image through methods such as a lane line extraction model. Taking the lane line extraction model as an example, after the road image is input into the model, the lane line extraction result output by the model is obtained. The lane line extraction result includes at least one pixel point set. Different pixel point sets correspond to different lane line images. The lane line image is an image of multiple lane lines included in the road image. The pixel point set includes at least part of the pixel points of the corresponding lane line image.

[0041] The lane line extraction model can be a RESA model based on a deep learning algorithm or any other suitable type of lane line extraction model.

[0042] Step 202: Determine the starting point and end point of the lane line in the pixel coordinate system based on the coordinate information of the pixel points in the pixel point set corresponding to the lane line.

[0043] After obtaining the lane line extraction result, the starting point and end point of the lane line in the pixel coordinate system can be determined based on the coordinate information of the pixel points in the pixel point set corresponding to the lane line, such as the coordinate value, or by determining the vector of the lane line based on the coordinate information.

[0044] Step 203: Determine at least one set of adjacent lane lines based on the start and end points of the lane lines in the pixel coordinate system.

[0045] After determining the start and end points of the lane lines in the pixel coordinate system, the position of each lane line in the pixel coordinate system can be determined based on the determined start and end points in the pixel coordinate system. Furthermore, the positional relationship between each lane line can be determined. By comparing the positional relationships between each lane line, at least one group of adjacent lane lines can be identified. Alternatively, the positional relationship between each lane line can be directly determined by calculating the vector of each lane line in the pixel coordinate system based on the start and end points of the lane line in the pixel coordinate system.

[0046] In the embodiment of the present application, lane line extraction is performed only through a road image, and at least one group of adjacent lane lines can be finally determined through corresponding processing. The calculation process is highly automated, thereby making the accuracy and efficiency of adjacent lane line determination higher.

[0047] In one feasible manner, the u-axis and v-axis of the pixel coordinate system are located in the same plane as the road image, the positive direction of the u-axis points to the right of the road image, and the positive direction of the v-axis points to the bottom of the road image.

[0048] Correspondingly, the process of determining the starting point and end point of the lane line in the pixel coordinate system based on the coordinate information of the pixel points in the pixel point set corresponding to the lane line can also include: sorting the pixel points in the pixel point set according to the size of the v coordinate of the pixel points in the pixel point set, and determining the pixel point with the smallest v coordinate in the pixel point set as the end point of the lane line corresponding to the pixel point set in the pixel coordinate system, and determining the pixel point with the largest v coordinate in the pixel point set as the starting point of the lane line corresponding to the pixel point set in the pixel coordinate system.

[0049] Since the positive direction of the u-axis of the pixel coordinate system points to the right of the road image, and the positive direction of the v-axis points to the bottom of the road image, the larger the v coordinate of the pixel point in the lane line pixel point set in the road image, the closer the pixel point is to the lower edge of the road image, and vice versa, the closer the pixel point is to the upper edge of the road image. It can be understood that the pixel point with the largest v coordinate in the lane line pixel point set is the starting point of the lane line in the pixel coordinate system, and the pixel point with the smallest v coordinate in the lane line pixel point set is the end point of the lane line in the pixel coordinate system.

[0050] In an embodiment of the present application, the starting point and end point of the lane line are determined by the v coordinates of the pixel points in the lane line pixel point set, which has high accuracy, thereby improving the accuracy of lane line topological relationship recognition.

[0051] In one feasible embodiment, the process of determining at least one group of adjacent lane lines based on the starting point and end point of the lane line in the pixel coordinate system may also include: determining the lane line vector based on the starting point and end point of the lane line in the pixel coordinate system, and determining at least one group of adjacent lane lines based on the first lane line vector to the Nth lane line vector.

[0052] After determining the start and end points of the lane lines in the pixel coordinate system, lane line vectors corresponding to the N lane lines can be determined based on their start and end points in the pixel coordinate system. The lane line vectors include the first lane line vector through the Nth lane line vector of the N lane lines included in the road image, where N is a positive integer greater than or equal to 2. For example, if the road image includes three lane lines, the first lane line vector corresponding to the first lane line can be determined based on its start and end points in the pixel coordinate system. Similarly, the second lane line vector corresponding to the second lane line and the third lane line vector corresponding to the third lane line can be determined. Based on the lane line vectors, it can be determined whether there are other lane lines between every two lane lines, and then two lane lines without other lane lines between them can be determined as a group of adjacent lane lines.

[0053] In an embodiment of the present application, adjacent lane lines are determined by calculating lane line vectors, which can be achieved using only mathematical operations, with higher accuracy and lower complexity.

[0054] In one feasible manner, the process of determining at least one group of adjacent lane lines based on the first lane line vector to the Nth lane line vector may further include: grouping the N lane lines in the road image into groups of two, and performing the following operations on the third lane line and the fourth lane line in each group:

[0055] Take the center of the third lane line vector as the starting point and the center of the fourth lane line vector as the end point to obtain the first vector, cross-product the third lane line vector and the first vector to obtain the second vector, and determine the positional relationship between the third lane line and the fourth lane line in the pixel coordinate system by comparing the direction between the second vector and the unit vector of the pixel coordinate system. Then, integrate the positional relationship between the third lane line and the fourth lane line in each group in the pixel coordinate system to determine at least one group of adjacent lane lines.

[0056] Another method for determining adjacent lane lines is to first group the N lane lines in the road image into groups of two, ensuring that each lane line is grouped with all other lane lines in the road image except the lane line itself, and the two lane lines in each group are referred to as the third lane line and the fourth lane line, respectively. Then, the first vector is obtained by connecting the three lane lines with the center of the third lane line vector as the starting point and the center of the fourth lane line vector as the end point. The center of the third lane line vector is the average value of the coordinates of the pixels in the set of pixels corresponding to the third lane line, and the center of the fourth lane line vector is the average value of the coordinates of the pixels in the set of pixels corresponding to the fourth lane line. The coordinates of the pixels are determined based on the coordinate information of the pixels. The third lane line vector and the first vector are cross-producted to obtain a second vector. The third lane line vector points from the starting point of the third lane line to the end point of the third lane line. Then, by comparing the direction between the second vector and the unit vector of the pixel coordinate system, the positional relationship between the third lane line and the fourth lane line in the pixel coordinate system is determined. For example, if the second vector and the unit vector of the pixel coordinate system are in the same direction, the fourth lane line is determined to be to the right of the third lane line. If the second vector and the unit vector of the pixel coordinate system are in opposite directions, the fourth lane line is determined to be to the right of the third lane line. This process is repeated until the positional relationships between two lane lines in all groups are obtained. By integrating these positional relationships, adjacent lane lines can be determined. For example, if the third lane line a in one group is to the right of the fourth lane line b, and the third lane line b in another group is to the right of the fourth lane line c, it can be determined that lane line b must exist between lane lines a and c. Therefore, it can be inferred that lane lines a and c are not adjacent, and lane lines a and b, and lane lines b and c are temporarily adjacent. The positional relationships in the remaining groups are then integrated to ultimately determine the adjacent lane lines.

[0057] In an embodiment of the present application, by calculating the positional relationships in groups and ultimately integrating the positional relationships in each group, it is possible to more finely determine the positional relationships of all lane lines in a road image while determining adjacent lane lines, thereby providing richer data for the generation of a virtual map.

[0058] In one feasible embodiment, the reference plane coincides with the xOz plane in the visual sensor coordinate system, the X-axis of the visual sensor coordinate system is perpendicular to the vehicle's traveling direction, and the positive direction of the X-axis points to the right side of the vehicle, the Y-axis of the visual sensor coordinate system is perpendicular to the vehicle's traveling direction, and the positive direction of the Y-axis points directly below the vehicle, and the positive direction of the Z-axis of the visual sensor coordinate system points to the vehicle's traveling direction.

[0059] According to the above scenario, a flowchart of a method for determining the starting point and end point of a projected lane line can be as follows: Figure 3 As shown, the method for determining the starting point and end point of the projected lane line includes:

[0060] Step 301: According to the conversion relationship between the pixel coordinate system and the visual sensor coordinate system, a set of pixel points corresponding to lane lines in a road image is converted into the visual sensor coordinate system.

[0061] To project lane markings from a road image onto a reference plane, the pixel points corresponding to the lane markings in the road image can first be converted to the vision sensor coordinate system based on the conversion relationship between the pixel coordinate system and the vision sensor coordinate system. This conversion relationship can be obtained using traditional camera calibration methods, active vision camera calibration methods, camera self-calibration methods, and zero-distortion camera calibration methods, using the lane marking pixel points as a calibration target.

[0062] Step 302: Project the pixel point set in the visual sensor coordinate system onto the xOz plane of the visual sensor coordinate system to obtain the projected lane line.

[0063] After converting the pixel point set corresponding to the lane line in the road image into the visual sensor coordinate system, the obtained pixel point set is a three-dimensional pixel point set. However, the image that we ultimately want to present to the user is still a two-dimensional image. Therefore, the pixel point set in the visual sensor coordinate system is projected onto the xOz plane of the visual sensor coordinate system. The pixel point set on the xOz plane of the sensor coordinate system is a two-dimensional pixel point set, and the projected lane line is obtained.

[0064] Step 303: Determine the start and end points of the projected lane line based on the z-coordinate values ​​of the pixel point set corresponding to the projected lane line in the xOz plane.

[0065] After obtaining the projected lane lines, since the positive direction of the Z axis in the xOz plane points in the direction of vehicle travel, the larger the pixel's z coordinate, the farther away from the vehicle it is. Conversely, the smaller the pixel's z coordinate, the closer it is to the vehicle. Therefore, the pixel with the largest z coordinate in the set of pixels can be used as the end point of the lane line, and the pixel with the smallest z coordinate can be used as the starting point of the lane line.

[0066] In an embodiment of the present application, the starting point and end point of the lane line are determined by projecting the z coordinate of the pixel point set corresponding to the lane line in the xOz plane, which has high accuracy, thereby improving the accuracy of lane line topological relationship recognition.

[0067] In one feasible embodiment, the process of determining the topological relationship between the two lane lines corresponding to the adjacent lane lines based on the distances between at least part of the starting points, end points, nearest points of the starting points, and nearest points of the end points of the two projected lane lines corresponding to the adjacent lane lines may also include: determining the topological relationship between the two projected lane lines corresponding to the adjacent lane lines based on a first distance between a first starting point of a projected lane line corresponding to the adjacent lane line and the nearest point of the first starting point, a second distance between the nearest point of the first starting point and the second end point of another projected lane line corresponding to the adjacent lane line, a third distance between the nearest point of the first starting point and the first end point of a projected lane line corresponding to the adjacent lane line, a fourth distance between the second starting point of another projected lane line corresponding to the adjacent lane line and the nearest point of the second starting point, a fifth distance between the nearest points of the second starting point and the first end point, a sixth distance between the nearest point of the second starting point and the second end point, a seventh distance between the first end point and the nearest point of the first end point, and an eighth distance between the second end point and the nearest point of the second end point.

[0068] To identify the topological relationship between two projected lane lines corresponding to adjacent lane lines, a conditional judgment can be performed using the distances between at least some of the start points, end points, and nearest neighbors of the start points and the nearest neighbors of the end points of the two projected lane lines corresponding to the adjacent lane lines as parameters. The distances between the start points, end points, and nearest neighbors of the start points and the nearest neighbors of the end points of the two projected lane lines include: a first distance between a first start point of a projected lane line corresponding to the adjacent lane line and its nearest neighbor, a second distance between the nearest neighbor of the first start point and the second end point of another projected lane line corresponding to the adjacent lane line, a third distance between the nearest neighbor of the first start point and the first end point of the projected lane line corresponding to the adjacent lane line, a fourth distance between the second start point of the other projected lane line corresponding to the adjacent lane line and its nearest neighbor, a fifth distance between the nearest neighbor of the second start point and the first end point, a sixth distance between the nearest neighbor of the second start point and the second end point, a seventh distance between the first end point and its nearest neighbor, and an eighth distance between the second end point and its nearest neighbor. Based on the first to eighth distances, the topological relationship between the two projected lane lines corresponding to the adjacent lane lines can be determined. The topological relationship between two projected lane lines corresponding to adjacent lane lines may include bifurcation, disconnection, parallel and independent relationships, which can be specifically as follows: Figure 4 As shown, two projected lane lines with an independent topological relationship can be the left and right lane lines of the same lane, two projected lane lines with a side-by-side topological relationship can be double solid lines, double dashed lines, dashed-solid lines, etc., two projected lane lines with a forked topological relationship can be lane lines where the main road diverts into the ramp, and two projected lane lines with a disconnected topological relationship can be the same lane line that is broken into two parts due to wear or misdetection.

[0069] In an embodiment of the present application, the topological relationship between two projected lane lines corresponding to adjacent lane lines is determined by using the first distance to the eighth distance. Since the acquisition of the first to the eighth distances is relatively simple, the complexity of identifying the lane line topological relationship is reduced.

[0070] In one feasible embodiment, the process of determining the topological relationship between the two projected lane lines corresponding to the adjacent lane lines may also include: when one projected lane line corresponding to the adjacent lane line and another projected lane line corresponding to the adjacent lane line meet a first condition, determining that the two projected lane lines corresponding to the adjacent lane lines are in a forked or side-by-side relationship; when one projected lane line corresponding to the adjacent lane line and another projected lane line corresponding to the adjacent lane line do not meet the first condition, determining that the two projected lane lines corresponding to the adjacent lane lines are in a disconnected or independent relationship; when one projected lane line corresponding to the adjacent lane line and another projected lane line corresponding to the adjacent lane line meet the first condition, determining that the two projected lane lines corresponding to the adjacent lane lines are in a forked or side-by-side relationship.

[0071] The first condition is: the first distance is less than the first distance threshold, the second distance is greater than the second distance threshold, and the first distance is less than the third distance, or the fourth distance is less than the first distance threshold, the fifth distance is greater than the second distance threshold, and the fourth distance is less than the sixth distance. When the two projected lane lines corresponding to adjacent lane lines meet the first condition, the two projected lane lines cannot be disconnected and independent. For example, the first distance threshold can be 1.5 meters, and the second distance threshold can be 2 meters.

[0072] In the embodiment of the present application, by defining the first condition as a comparison rule, it is possible to determine whether the two projected lane lines are in a forked or parallel relationship or a disconnected or independent relationship.

[0073] In one feasible embodiment, the process of determining the topological relationship between the two projected lane lines corresponding to the adjacent lane lines may also include: when one projected lane line corresponding to the adjacent lane line and another projected lane line corresponding to the adjacent lane line meet the first condition and the second condition, determining that the two projected lane lines corresponding to the adjacent lane lines are in a side-by-side relationship; when one projected lane line corresponding to the adjacent lane line and another projected lane line corresponding to the adjacent lane line meet the first condition and do not meet the second condition, determining that the two projected lane lines corresponding to the adjacent lane lines are in a bifurcated relationship.

[0074] The second condition is that the first distance or the fourth distance is less than a third distance threshold, and the seventh distance or the eighth distance is less than the third distance threshold. For example, the third distance threshold may be 0.5 m.

[0075] In the embodiment of the present application, under the judgment result of the first condition, by defining the second condition as a comparison rule, it is possible to determine whether the two projected lane lines are in a forked relationship or a side-by-side relationship.

[0076] In one feasible embodiment, the process of determining the topological relationship between the two projected lane lines corresponding to the adjacent lane lines may also include: when one projected lane line corresponding to the adjacent lane line and another projected lane line corresponding to the adjacent lane line meet the first condition and the third condition, determining that the two projected lane lines corresponding to the adjacent lane lines are in a disconnected relationship; when one projected lane line corresponding to the adjacent lane line and another projected lane line corresponding to the adjacent lane line meet the first condition and do not meet the third condition, determining that the two projected lane lines corresponding to the adjacent lane lines are in an independent relationship.

[0077] The third condition is: the second distance is less than the fourth distance threshold and the first distance is less than the third distance, or the fifth distance is less than the fourth distance threshold and the fourth distance is less than the sixth distance. For example, the fourth distance threshold may be 1 meter.

[0078] In the embodiment of the present application, under the judgment result of the first condition, by defining the third condition as a comparison rule, it is possible to determine whether the relationship between the two projected lane lines is disconnected or independent.

[0079] Figure 5 FIG. 1 is a structural block diagram of a lane topology relationship recognition device according to an embodiment of the present application. Figure 5 As shown, the lane topology relationship recognition device 400 may include: an acquisition module 401 , a determination module 402 , a projection module 403 , a search module 404 and a recognition module 405 .

[0080] The acquisition module 401 is configured to acquire a road image including an image of a plurality of lane lines.

[0081] To identify the lane line topology, a road image including multiple lane lines needs to be first acquired by the acquisition module 401. The road image can be acquired using a visual sensor, which can be a camera, a camera, a light source sensor, and / or any other suitable type of device.

[0082] It should be noted that the road image can be acquired through one or more visual sensors, which is not limited here.

[0083] The determination module 402 is configured to identify the positional relationship of lane lines in the road image and determine at least one set of adjacent lane lines, wherein the adjacent lane lines include two adjacent lane lines in the road image.

[0084] After the acquisition module 401 acquires the road image, the determination module 402 identifies the positional relationship of the lane lines in the road image. The identification method can be through a pre-trained model or through feature points in the lane lines. After the lane lines are identified, the adjacent lane lines are determined based on the positional relationship of the lane lines in the road image, or the lane line feature points satisfying a preset position judgment relationship. The adjacent lane lines include two adjacent lane lines in the road image. Since there can be multiple lane lines in the road image, at least one group of adjacent lane lines can be determined.

[0085] The projection module 403 is used to project the lane lines in the road image onto a reference plane and determine the start and end points of the projected lane lines, wherein the reference plane is parallel to the plane where the vehicle is located, and the road image is collected by a visual sensor installed on the vehicle.

[0086] Because lane markings are painted on the road, they are equivalent to being located on the two-dimensional plane of the road. Therefore, the lane line topology should be based on relationships on this two-dimensional plane. Therefore, projection module 403 can project the lane lines in the road image onto a reference plane, which is parallel to the plane of the vehicle. The road image is captured by a visual sensor mounted on the vehicle. After projection, the start and end points of the projected lane lines can be further determined. This can be done by using lane line feature points or by constructing lane line vectors.

[0087] It should be noted that since this application is based on an autonomous driving scenario, the plane where the vehicle is located can be a plane parallel to the road on which the vehicle is traveling, and then it can be determined that the reference plane is parallel to the two-dimensional plane where the road is located, and then the lane line topological relationship identified based on the starting point and end point of the projected lane line on the reference plane can indicate the topological relationship of the lane line in the road.

[0088] The search module 404 is used to perform a nearest neighbor search on two projected lane lines corresponding to adjacent lane lines, and respectively determine the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines, wherein the nearest neighbor point of the starting point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the starting point of the projected lane line, and the nearest neighbor point of the ending point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the ending point of the projected lane line.

[0089] After the projection module 403 determines the start and end points of the projected lane line, in combination with at least one set of adjacent lane lines determined by the determination module 402, the search module 404 performs a nearest neighbor search on the two projected lane lines corresponding to the adjacent lane lines. The nearest neighbor search can be implemented by searching for the shortest Euclidean distance or Manhattan distance between the two projected lane lines corresponding to the adjacent lane lines, to determine the nearest neighbor points of the start and end points of the two projected lane lines.

[0090] The nearest point to the starting point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the starting point of the projected lane line, and the nearest point to the ending point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the ending point of the projected lane line.

[0091] The identification module 405 is used to determine the topological relationship between the two lane lines corresponding to the adjacent lane lines based on the distance between at least part of the starting point, the end point, the nearest neighbor point of the starting point and the nearest neighbor point of the end point of the two projected lane lines corresponding to the adjacent lane lines.

[0092] After the search module 404 determines the starting point nearest neighbor points and the ending point nearest neighbor points of the two projected lane lines, the topological relationship between the two lane lines corresponding to the adjacent lane lines is determined based on the distances between at least part of the starting point, the ending point, the starting point nearest neighbor points, and the ending point nearest neighbor points of the two projected lane lines corresponding to the adjacent lane lines, such as the distance between the starting point and the starting point nearest neighbor points of the two projected lane lines, the distance between the ending point and the ending point nearest neighbor points of the two projected lane lines, and the distance between the starting point and the ending point nearest neighbor points of the two projected lane lines.

[0093] In an embodiment of the present application, by processing a road image including multiple lane lines and determining the topological relationship between two adjacent lane lines using a nearest neighbor search to determine the nearest neighbor points of the starting and ending points of the two projected lane lines, lane line topological relationship recognition can be completed without the need for a high-precision map. Furthermore, since only the distances between the starting and ending points, and at least a portion of the nearest neighbor points of the starting and ending points of the two projected lane lines corresponding to the adjacent lane lines need to be obtained, manual labeling and other methods are not required, thereby improving the efficiency of lane line topological relationship recognition.

[0094] It should be noted that the information interaction, execution process, etc. between the modules within the above-mentioned lane line topology relationship identification device are based on the same concept as the above-mentioned lane line topology relationship identification method embodiment. The specific contents can be found in the description in the above-mentioned lane line topology relationship identification method embodiment and will not be repeated here.

[0095] In this embodiment, a controller 500 is provided, such as Figure 6As shown, the controller 500 may include: a processor 501, a communications interface 502, a memory 503, and a communication bus 504.

[0096] The processor 501 , the communication interface 502 , and the memory 503 communicate with each other via the communication bus 504 .

[0097] The communication interface 502 is used to communicate with other electronic devices or servers.

[0098] The processor 501 is used to execute the program 505, and specifically can execute the relevant steps in the above-mentioned lane line topology relationship recognition method embodiment.

[0099] Specifically, the program 505 may include program codes, which include computer operation instructions.

[0100] The processor 501 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0101] The memory 503 is used to store the computer program 505. The memory 503 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0102] The program 505 can be specifically used to enable the processor 501 to execute the lane line topology relationship recognition method in the aforementioned embodiment.

[0103] The specific implementation of each step in procedure 505 can be found in the corresponding descriptions of the corresponding steps and units in the aforementioned lane topology relationship identification method embodiment, and will not be repeated here. Those skilled in the art will clearly understand that for ease and brevity of description, the specific operating processes of the above-described devices and modules can refer to the corresponding process descriptions in the aforementioned method embodiment, and will not be repeated here.

[0104] The controller of an embodiment of the present application processes a road image including multiple lane lines and determines the topological relationship between two adjacent lane lines using a nearest neighbor search to determine the nearest neighbor points of the start and end points of the two projected lane lines. This allows lane line topological relationship recognition without the need for a high-precision map. Furthermore, since only the distances between the start and end points, and at least a portion of the nearest neighbor points of the start and end points of the two projected lane lines corresponding to the adjacent lane lines need to be obtained, manual labeling and other methods are not required, thereby improving the efficiency of lane line topological relationship recognition.

[0105] In this embodiment, a mobile device is provided, such as Figure 7 As shown, the mobile device 600 may include: a processor 601, a communications interface 602, a memory 603, a communication bus 604, and a sensor 606. Among them:

[0106] The processor 601 , the communication interface 602 , and the memory 603 communicate with each other via the communication bus 604 .

[0107] The communication interface 602 is used to communicate with other electronic devices or servers.

[0108] The processor 601 is used to execute the program 605, and specifically can execute the relevant steps in the above-mentioned lane line topology relationship recognition method embodiment.

[0109] Specifically, the program 605 may include program codes, which include computer operation instructions.

[0110] The processor 601 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0111] The memory 603 is used to store the computer program 605. The memory 603 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0112] The program 605 can be specifically used to enable the processor 601 to execute the lane line topology relationship recognition method in the aforementioned embodiment.

[0113] The sensor 606 is used to collect environmental data of the environment in which the mobile device 600 is located and generate perception data. The perception data may be a road image including an image of multiple lane lines.

[0114] The specific implementation of each step in procedure 605 can be found in the corresponding descriptions of the corresponding steps and units in the aforementioned lane topology relationship identification method embodiment, and will not be repeated here. Those skilled in the art will clearly understand that for the sake of convenience and brevity, the specific operating processes of the above-described devices and modules can refer to the corresponding process descriptions in the aforementioned method embodiment, and will not be repeated here.

[0115] The mobile device of an embodiment of the present application processes a road image including multiple lane lines and determines the topological relationship between two adjacent lane lines by using a nearest neighbor search to determine the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines. This allows the recognition of lane line topological relationships without the need for a high-precision map. Furthermore, since only the distances between the starting points, ending points, and at least a portion of the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines corresponding to the adjacent lane lines need to be obtained, manual labeling and other methods are not required, thereby improving the efficiency of lane line topological relationship recognition.

[0116] In this embodiment, a computer-readable storage medium is provided, storing instructions for causing a machine to execute the lane topology recognition method described herein. Specifically, a system or device equipped with a storage medium can be provided, storing software program code that implements the functions of any of the above-described embodiments, and causing a computer (or CPU or MPU) of the system or device to read and execute the program code stored in the storage medium.

[0117] In this case, the program code read from the storage medium itself can realize the function of any one of the above embodiments, so the program code and the storage medium storing the program code constitute part of this application.

[0118] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0119] In this embodiment, a computer program product is provided, including computer instructions, which instruct a computing device to perform any corresponding operation in the above-mentioned multiple method embodiments.

[0120] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0121] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium (such as CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or be implemented as computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded via a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0122] It should be noted that in the description of this application, the terms "first" and "second" are used only to facilitate the description of different components or names, and should not be understood to indicate or imply a sequential relationship, relative importance, or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include at least one of such features.

[0123] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application.

[0124] It should be noted that although the specific embodiments of the present application are described in detail in conjunction with the accompanying drawings, this should not be construed as limiting the scope of protection of the present application. Various modifications and variations that can be made by those skilled in the art without inventive effort within the scope described in the claims are still within the scope of protection of the present application.

[0125] The examples of the embodiments of the present application are intended to concisely illustrate the technical features of the embodiments of the present application so that those skilled in the art can intuitively understand the technical features of the embodiments of the present application, and are not intended to serve as improper limitations on the embodiments of the present application.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying lane line topological relationships, characterized in that: include: acquiring a road image including an image of a plurality of lane lines; Identifying a positional relationship between lane lines in the road image to determine at least one set of adjacent lane lines, wherein the adjacent lane lines include two lane lines that are adjacent to each other in the road image; Projecting lane lines in the road image onto a reference plane, and determining a start point and an end point of the projected lane lines, wherein the reference plane is parallel to a plane on which the vehicle is located, and the road image is captured by a visual sensor mounted on the vehicle; Performing a nearest neighbor search on the two projected lane lines corresponding to the adjacent lane lines to respectively determine the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines, wherein the nearest neighbor point of the starting point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the starting point of the projected lane line, and the nearest neighbor point of the ending point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the ending point of the projected lane line; A topological relationship between the two lane lines corresponding to the adjacent lane lines is determined based on a distance between at least part of the starting point, the ending point, the nearest neighbor point of the starting point, and the nearest neighbor point of the ending point of the two projected lane lines corresponding to the adjacent lane lines.

2. The method according to claim 1, characterized in that The identifying the positional relationship of lane lines in the road image to determine at least one group of adjacent lane lines includes: Performing lane line extraction on the road image to obtain a lane line extraction result, wherein the lane line extraction result includes at least one pixel point set, different pixel point sets correspond to different lane line images, and the pixel point set includes at least part of the pixel points of the corresponding lane line image; Determining a start point and an end point of the lane line in the pixel coordinate system according to coordinate information of pixel points in the pixel point set corresponding to the lane line; At least one group of adjacent lane lines is determined based on the starting point and the ending point of the lane line in the pixel coordinate system.

3. The method according to claim 2, characterized in that The u-axis and v-axis of the pixel coordinate system are located in the same plane as the road image, the positive direction of the u-axis points to the right of the road image, and the positive direction of the v-axis points to the bottom of the road image; Correspondingly, determining the starting point and the end point of the lane line in the pixel coordinate system according to the coordinate information of the pixel points in the pixel point set corresponding to the lane line includes: The pixel points in the pixel point set are sorted according to the size of the v coordinates of the pixel points in the pixel point set, and the pixel point with the smallest v coordinate in the pixel point set is determined as the end point of the lane line corresponding to the pixel point set in the pixel coordinate system, and the pixel point with the largest v coordinate in the pixel point set is determined as the starting point of the lane line corresponding to the pixel point set in the pixel coordinate system.

4. The method according to claim 2, characterized in that Determining at least one set of adjacent lane lines based on the starting point and the ending point of the lane line in the pixel coordinate system includes: Determining a lane line vector based on a start point and an end point of the lane line in the pixel coordinate system, wherein the lane line vector includes a first lane line vector to an Nth lane line vector among N lane lines included in the road image, where N is a positive integer greater than or equal to 2; At least one group of adjacent lane lines is determined based on the first lane line vector to the Nth lane line vector.

5. The method according to claim 2, characterized in that The reference plane coincides with the xOz plane in the vision sensor coordinate system, the X-axis of the vision sensor coordinate system is perpendicular to the direction of travel of the vehicle, and the positive direction of the X-axis points to the right side of the vehicle, the Y-axis of the vision sensor coordinate system is perpendicular to the direction of travel of the vehicle, and the positive direction of the Y-axis points directly below the vehicle, and the positive direction of the Z-axis of the vision sensor coordinate system points to the direction of travel of the vehicle; Projecting the lane line in the road image onto a reference plane and determining the start and end points of the projected lane line includes: According to the conversion relationship between the pixel coordinate system and the visual sensor coordinate system, a set of pixel points corresponding to the lane lines in the road image is converted to the visual sensor coordinate system; Projecting the pixel point set in the visual sensor coordinate system onto the xOz plane of the visual sensor coordinate system to obtain the projected lane line; The starting point and the end point of the projected lane line are determined according to the coordinate values ​​of the z coordinates of the pixel point set corresponding to the projected lane line in the xOz plane.

6. The method according to claim 5, characterized in that The determining the topological relationship between the two lane lines corresponding to the adjacent lane lines based on the distances between at least a portion of the starting points, the ending points, the nearest neighboring points of the starting points, and the nearest neighboring points of the ending points of the two projected lane lines corresponding to the adjacent lane lines includes: The topological relationship between the two projected lane lines corresponding to the adjacent lane lines is determined based on the first distance between the first starting point of a projected lane line corresponding to the adjacent lane line and the nearest point of the first starting point, the second distance between the nearest point of the first starting point and the second end point of another projected lane line corresponding to the adjacent lane line, the third distance between the nearest point of the first starting point and the first end point of a projected lane line corresponding to the adjacent lane line, the fourth distance between the second starting point of another projected lane line corresponding to the adjacent lane line and the nearest point of the second starting point, the fifth distance between the nearest point of the second starting point and the first end point, the sixth distance between the nearest point of the second starting point and the second end point, the seventh distance between the first end point and the nearest point of the first end point, and the eighth distance between the second end point and the nearest point of the second end point.

7. A lane line topology relationship recognition device, characterized in that: include: An acquisition module, configured to acquire a road image including an image of a plurality of lane lines; a determination module, configured to identify a positional relationship between lane lines in the road image and determine at least one set of adjacent lane lines, wherein the adjacent lane lines include two adjacent lane lines in the road image; a projection module, configured to project lane lines in the road image onto a reference plane, and determine a start point and an end point of the projected lane lines, wherein the reference plane is parallel to a plane on which the vehicle is located, and the road image is collected by a visual sensor mounted on the vehicle; a search module configured to perform a nearest neighbor search on the two projected lane lines corresponding to the adjacent lane lines, and respectively determine the nearest neighbor points of the starting points and the nearest neighbor points of the ending points of the two projected lane lines, wherein the nearest neighbor point of the starting point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the starting point of the projected lane line, and the nearest neighbor point of the ending point of one of the two projected lane lines is the point on the other projected lane line that is nearest to the ending point of the projected lane line; an identification module for determining a topological relationship between two lane lines corresponding to the adjacent lane lines based on a distance between at least a portion of the starting point, the end point, the nearest neighbor point of the starting point, and the nearest neighbor point of the end point of the two projected lane lines corresponding to the adjacent lane lines.

8. A controller comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, the communication interface and the memory communicate with each other via the communication bus, wherein: The memory is used to store computer programs; The processor is configured to execute the method steps of any one of claims 1 to 6 by running the computer program stored in the memory.

9. A mobile device, characterized in that: include: A sensor, a processor, a memory, a communication interface and a communication bus, wherein the sensor, the processor, the memory and the communication interface communicate with each other via the communication bus; The sensor is used to collect environmental data of the environment in which the mobile device is located and generate perception data; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 6 based on the perception data.

10. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images