Road boundary inspection method and apparatus, and device and medium

Abstract

A road boundary inspection method and apparatus, and a device and a medium. The method comprises: acquiring road boundary point cloud data in an ROI in real time, and determining the heights of a plurality of grids which are pre-divided in the ROI (S110); on the basis of the heights of the plurality of grids in the ROI, and a left region or right region to which the plurality of grids belong, respectively determining road boundary point sets respectively corresponding to a left side and right side of a vehicle, and generating complete road curve equations and effective road curve equations which respectively correspond to the left side and right side of the vehicle (S120); determining an inner point set on the basis of a plurality of historically updated effective road curve equations within a specified period of time, and determining a plurality of effective inner points from the inner point set on the basis of at least one of the complete road curve equations and the effective road curve equations (S130); and adding the effective inner points into the road boundary point sets, updating the effective road curve equations, and generating a road boundary inspection result (S140).

Description

Road boundary detection method, device, equipment and medium

[0001] This application claims priority to the Chinese patent application number 202311809537.8 filed with the China Patent Office on December 26, 2023. The entire contents of the above application are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of automobile intelligent technology, for example, to a road boundary detection method, device, equipment and medium. Background Art

[0003] With the development of intelligent automotive technology, more and more cars are equipped with autonomous driving systems. In road scenes, road boundaries composed of curbs, guardrails, etc. are one of the important road features and are also the focus of the perception module in autonomous driving systems.

[0004] In the related art, the lidar-based road boundary detection method generally fits the overall road boundary curve equation based on all potential points when fitting the road curve. However, road scenes are complex and changeable, and the actual road boundary may have relatively complex scenes such as gaps and intersections. Taking a typical T-shaped intersection as an example, one side is usually a continuous road boundary, while the other side has a gap of a certain length. In addition, when the road boundary is blocked by a vehicle or other object, the identified road boundary may also be missing. Therefore, it is difficult to effectively describe the actual road boundary only through the overall curve equation, which may cause misjudgment of subsequent functions and affect the safety of the autonomous driving function.

[0005] Summary of the Invention

[0006] The present application provides a road boundary detection method, device, equipment and medium, which can accurately distinguish road boundaries from road gaps and effectively improve the accuracy of road boundary detection.

[0007] According to one aspect of the present application, a road boundary detection method is provided, comprising:

[0008] Acquire road boundary point cloud data within the ROI in real time, and determine the heights of a plurality of grids pre-divided within the ROI based on the road boundary point cloud data;

[0009] Determining, based on the heights of multiple grids within the ROI region and the left or right regions to which the multiple grids belong, road boundary point sets corresponding to the left and right sides of the vehicle, respectively, and generating, based on the road boundary point sets, complete road curve equations and effective road curve equations corresponding to the left and right sides of the vehicle, respectively; wherein the effective road curve equation is determined by whether there is a road gap on the left or right side of the vehicle;

[0010] Determining an inlier point set based on a plurality of historically updated road effective curve equations within a specified time period, and determining a plurality of valid inliers in the inlier point set based on at least one of the road complete curve equation and the road effective curve equation;

[0011] The valid interior points are added to the road boundary point set, and according to the road boundary point set after the valid interior points are added, the road valid curve equation is updated to generate a road boundary detection result.

[0012] According to another aspect of the present application, a road boundary detection device is provided, comprising:

[0013] a grid height acquisition module configured to acquire road boundary point cloud data within the ROI in real time and determine the heights of a plurality of grids pre-divided within the ROI based on the road boundary point cloud data;

[0014] a curve equation generation module configured to determine, based on the heights of a plurality of grids within the ROI region and the left or right regions to which the plurality of grids belong, a set of road boundary points corresponding to the left and right sides of the vehicle, respectively, and generate, based on the road boundary point sets, a complete road curve equation and a valid road curve equation corresponding to the left and right sides of the vehicle, respectively; wherein the valid road curve equation is determined by whether there is a road gap on the left or right side of the vehicle;

[0015] a valid inlier determination module configured to determine an inlier point set based on a plurality of historically updated road valid curve equations within a specified time period, and to determine a plurality of valid inliers in the inlier point set based on at least one of the road complete curve equation and the road valid curve equation;

[0016] The road boundary detection result generating module is configured to add the valid inner point to the road boundary point set, and update the road valid curve equation according to the road boundary point set after adding the valid inner point, so as to generate a road boundary detection result.

[0017] According to another aspect of the present application, an electronic device is provided, comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the road boundary detection method described in any embodiment of the present application.

[0021] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the road boundary detection method described in any embodiment of the present application when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The following will introduce the drawings required for the description of the embodiments. The drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] FIG1 is a flow chart of a road boundary detection method provided according to an embodiment of the present application;

[0024] FIG2 is a schematic diagram of a vehicle region of interest provided according to an embodiment of the present application;

[0025] FIG3 is a flow chart of another road boundary detection method provided according to an embodiment of the present application;

[0026] FIG4 is a schematic structural diagram of a road boundary detection device provided according to an embodiment of the present application;

[0027] FIG5 is a schematic diagram of the structure of an electronic device for implementing the road boundary detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The embodiments described are some related embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units listed, but may include other steps or units that are not listed or that are inherent to these processes, methods, products or devices.

[0030] Figure 1 is a flow chart of a road boundary detection method provided by an embodiment of the present application. This embodiment is applicable to accurately distinguishing between normal road boundaries and road boundary gaps. The method can be performed by a road boundary detection device, which can be implemented in the form of hardware and / or software and can generally be configured in a vehicle computer or an autonomous driving system processor. As shown in Figure 1, the method includes:

[0031] S110 , acquiring road boundary point cloud data within the ROI in real time, and determining the heights of a plurality of grids pre-divided within the ROI based on the road boundary point cloud data.

[0032] Optionally, the point cloud data may be collected in real time by a laser radar carried by a vehicle, and the road boundary point cloud data may refer to point cloud data that can represent the road boundary.

[0033] Optionally, the ROI (Region of Interest) can be a preset area, or an area within a specified range around the vehicle. Generally, the ROI will be smaller than the maximum area that can be collected by the lidar. The ROI can be determined based on the current vehicle coordinate system.

[0034] Figure 2 is a schematic diagram of an optional vehicle ROI. As shown in Figure 2, xoy is the vehicle coordinate system, the origin o of the coordinate system can be the vehicle's center of mass or center, the x-axis can point in the direction of the vehicle's front, and the y-axis can point in a direction perpendicular to the vehicle's front. The pattern fill area is the vehicle's ROI.

[0035] Optionally, after the laser radar obtains the point cloud data at the current moment, the point cloud data within the ROI can be filtered out according to the pre-divided ROI, and then the point cloud data belonging to the interference objects in the ROI can be filtered out, and only the point cloud data that can represent the road boundary is retained, and the point cloud data that can represent the road boundary is downsampled to generate the road boundary point cloud data.

[0036] Optionally, the ROI may be divided into multiple grids according to a predetermined grid division rule. As shown in FIG2 , in the ROI shown in the pattern-filled area, the multiple grids divided by black lines are the pre-divided grids within the ROI.

[0037] Optionally, after determining multiple grids within the ROI, the road boundary point cloud data can be divided into point cloud data within each grid; the point clouds in each grid that obviously do not belong to the road boundary are deleted, and the point cloud with the highest height is determined in the remaining point clouds, and the height of the point cloud with the highest height is used as the height of the grid where the point cloud is located, thereby determining the height of each pre-divided grid within the ROI.

[0038] S120. Determine, based on the heights of multiple grids within the ROI area and the left or right areas to which the multiple grids belong, road boundary point sets corresponding to the left and right sides of the vehicle, respectively. Generate, based on the road boundary point sets, complete curve equations and effective curve equations of the road corresponding to the left and right sides of the vehicle, respectively.

[0039] The effective road curve equation is determined by whether there is a road gap on the left or right side of the vehicle.

[0040] In some embodiments, before determining the road boundary point sets corresponding to the left and right sides of the vehicle respectively based on the heights of the plurality of grids within the ROI region and the left or right regions to which the plurality of grids belong, the method may further include:

[0041] Obtaining motion data of the vehicle, and calculating the current position of the vehicle based on the motion data;

[0042] Calculating a motion trajectory equation of the vehicle based on the vehicle's historical position within a specified time period and the vehicle's current position, and determining the vehicle's motion trajectory based on the motion trajectory equation;

[0043] The grid passed by the motion trajectory is determined as the middle grid, and multiple grids located on the left side of the middle grid in the ROI are divided into a left area, and multiple grids located on the right side of the middle grid are divided into a right area.

[0044] Optionally, the motion data of the vehicle may include the linear velocity, yaw angular velocity, vehicle position of the vehicle at the last data collection moment and the time interval between each motion data collection.

[0045] Optionally, you can use the formula Calculate the position variable Δp of the vehicle on the horizontal axis from the last data collection moment to the current data collection moment x , according to the formula Calculate the position variable Δp of the vehicle on the vertical coordinate from the last data collection moment to the current data collection moment y , where v k is the linear velocity at the last data collection moment, ω k is the yaw angular velocity at the last data collection moment, and T is the time interval between each motion data collection.

[0046] In some embodiments, after determining the position variable of the vehicle, the current position of the vehicle can be calculated based on the vehicle position at the last data collection moment and the position variable of the vehicle.

[0047] Optionally, the designated time period can generally be a shorter time period, and can be determined based on the vehicle speed. For example, when the vehicle speed is 20 kilometers per hour (km / h), the designated time period can be 5 seconds, and when the vehicle speed is 80 km / h, the designated time period can be 2 seconds. However, this is only an example, and the method for selecting the designated time period can be set according to actual needs.

[0048] Optionally, the historical position can refer to the vehicle's position at each data collection moment within a specified time period. Based on each historical position and the vehicle's current position, multiple vehicle positions can be fitted to generate a trajectory equation. The vehicle's trajectory can then be determined based on the curve of the trajectory equation. In the example shown in Figure 2, the dashed line represents the vehicle's trajectory.

[0049] The size of the grid is generally much smaller than the size of the vehicle body. Therefore, after determining the middle grid that the vehicle trajectory passes through, there is no need to process the point cloud data of the middle grid. The road boundaries are generally located on the left and right sides of the vehicle.

[0050] Optionally, after dividing the left and right areas of the vehicle, the left and right sides can be searched separately starting from the middle grid, and the height differences between two adjacent grids can be compared one by one. When the height difference is higher than a certain value, there may be a road boundary such as a curb or a fence. The higher grid in the adjacent grid may fall on the road boundary. The highest point in the grid can be used as a road boundary point. After determining the grid where the road boundary point is located, stop continuing to compare the height differences of the grids outside the grid.

[0051] In some embodiments, multiple road boundary points belonging to the left area can be combined to generate a road boundary point set corresponding to the left side of the vehicle, and multiple road boundary points belonging to the right area can be combined to generate a road boundary point set corresponding to the right side of the vehicle.

[0052] Optionally, after obtaining the road boundary point set, the road boundary point sets corresponding to the left area and the right area respectively may be fitted to obtain complete curve equations of the roads on the left and right sides of the vehicle.

[0053] Optionally, after obtaining the road boundary point set, the distance difference between each two adjacent road boundary points in the road boundary point set can be calculated separately, that is, the distance difference between the road boundary points closest to each other in the road boundary point set is calculated. When the distance difference between two road boundary points is greater than a certain value, it can be determined that there is a gap between the two road boundary points. At this time, the two road boundary points can be used as endpoints on both sides of the gap.

[0054] Optionally, for a single-sided road, there may be multiple gaps, one gap, or no gap. When there is at least one gap in the single-sided road, the effective road curve equation may include multiple road segment curve equations. When there is no gap in the single-sided road, the effective road curve equation is the complete road curve equation.

[0055] Optionally, when there is a gap in the road, the road boundary point set on the vehicle side can be divided into multiple subsets by at least one gap, and the road boundary points in each subset are fitted to generate a road segment curve equation.

[0056] S130. Determine an inlier point set based on multiple historically updated road effective curve equations within a specified time period, and determine multiple valid inliers in the inlier point set based on at least one of the road complete curve equation and the road effective curve equation.

[0057] Optionally, the road effective curve equation historically updated within a specified time period may refer to the road effective curve finally determined by the road boundary detection at a historical moment. It can be understood that the road effective curve generated in step S120 is a preliminary generated road effective curve. The road effective curve determined at a certain moment may not be able to reflect the real and complete road conditions. Therefore, it can be updated by the historically confirmed road effective curve, so that the updated road effective curve equation is used as the final road effective curve.

[0058] Optionally, the interior points of the historically updated road effective curve may be determined based on the historically updated road effective curve and a set of road boundary points used to update the road effective curve.

[0059] Optionally, if the distance between a road boundary point in the road boundary point set and the road effective curve is less than a certain value, the road boundary point may be used as an interior point of the road effective curve.

[0060] Optionally, the interior point set may include all interior points of multiple historically updated road effective curve equations, and the interior point set may also be divided into an interior point set on the left side of the vehicle and an interior point set on the right side of the vehicle.

[0061] Optionally, after determining the inner point set, each inner point in the inner point set can be compared with at least one of the road complete curve equation and the road effective curve equation. When there is no road gap on the vehicle target side, it is only necessary to compare each inner point on the vehicle target side with the road complete curve equation. If the inner point falls within the inner point range of the road complete curve equation, the inner point is determined to be a valid inner point; when there is a gap on the vehicle target side, it is necessary to compare the inner point with the road complete curve equation and the road effective curve equation respectively. If the inner point falls within the inner point range of the road complete curve equation or the road effective curve equation, the inner point is determined to be a valid inner point.

[0062] S140: Add the valid inliers to the road boundary point set, and update the road valid curve equation according to the road boundary point set to which the valid inliers are added, to generate a road boundary detection result.

[0063] Optionally, the valid inner point can be added to the road boundary point set that matches the target vehicle side where the valid inner point is located, and based on the updated road boundary point set on the target vehicle side, the road effective curve equation on the target vehicle side is re-fitted and generated, and based on the regenerated road effective curve equation, the road effective curve equation generated in step S120 is updated.

[0064] Optionally, after obtaining the updated road effective curve equation, if the road effective curve equation is a complete road curve equation, a curve matching the complete road curve equation can be output as the road boundary; if the road effective curve equation is a segmented road curve equation, multiple segmented curves are used as road boundaries, and the gaps between multiple road boundaries are road gaps.

[0065] In an embodiment of the present application, by obtaining road boundary point cloud data within an ROI, the heights of multiple grids pre-divided within the ROI are determined, and road boundary point sets corresponding to the left and right sides of the vehicle are respectively determined based on the heights of the multiple grids within the ROI area and the left or right areas to which the multiple grids belong. Then, a complete road curve equation and a road effective curve equation corresponding to the left and right sides of the vehicle are generated based on the road boundary point sets. Then, an inner point set is obtained based on the historically updated road effective curve equation, and a valid inner point is determined in the inner point set. The road effective curve equation is updated based on the valid inner point, thereby generating a road boundary detection result. This method can filter out the influence of obstacles and special road sections on road boundary detection, and solves the problem of inaccurate road boundary detection results caused by fitting only the point cloud collected by the lidar in the related art. At the same time, updating the road effective boundary curve based on the valid inner point can solve the problem of the collected data being affected by the surrounding environment at a certain moment, reduce the road gap size error, and effectively improve the accuracy of road boundary detection.

[0066] FIG3 is a flow chart of another road boundary detection method provided by an embodiment of the present application. This embodiment illustrates a road boundary detection method based on the above embodiment. As shown in FIG3 , the method includes:

[0067] S210 , deleting the point cloud data of the interfering object from the initial point cloud data within the ROI acquired in real time, and performing downsampling processing on the remaining initial point cloud data to generate road boundary point cloud data within the ROI.

[0068] Optionally, the initial point cloud data may refer to all point cloud data belonging to the ROI in the point cloud data acquired by the laser radar, that is, the initial point cloud data may include all point cloud data within the vehicle ROI.

[0069] Optionally, the interfering objects may include pedestrians, vehicles, obstacles, and other objects that can be clearly distinguished from road boundaries.

[0070] Optionally, this application is to fit curves related to road boundaries, so as to determine the road boundaries based on the curve conditions. Curve fitting does not require a large amount of point cloud data, only universal or representative point clouds are required. Therefore, downsampling the remaining initial point cloud data can effectively improve the calculation speed and meet the needs of obtaining road boundary detection results in real time when the vehicle is driving.

[0071] S220 , dividing the road boundary point cloud data into pre-divided grids, and obtaining point cloud data of each grid respectively.

[0072] S230 , filtering the point clouds whose point cloud height values ​​exceed a first preset threshold value in the point cloud data of each grid, and using the height value of the highest point cloud in each grid after filtering as the height of the grid.

[0073] Optionally, the first preset threshold may be used to filter out point cloud data in each grid that clearly does not belong to the road boundary.

[0074] By screening and deleting the point cloud data of interfering objects, it is possible to delete obstacles with known relatively fixed shapes. However, in actual road scenarios, for overpasses, tunnels, etc., point cloud data may also be collected from the suspended parts of overpasses and tunnels, which may result in the presence of some high-altitude point clouds in the point cloud data of each grid. However, these point clouds cannot represent the road boundary. Therefore, point clouds with a height greater than the first preset threshold can be filtered out, and only the point clouds determined as road boundaries in each grid are retained.

[0075] S240: Starting from each middle grid, respectively calculate the height difference between each two adjacent grids in the left area and the right area, and determine a target adjacent grid whose height difference is greater than a second preset threshold, determine a road boundary point in the outer grid of the target adjacent grid, and stop calculating the height difference related to the outer grid.

[0076] Optionally, a second preset threshold can be used to determine whether each grid can serve as a road boundary. If the height difference between the target adjacent grids is greater than the second preset threshold, the grid with a higher height among the target adjacent grids can be used as the road boundary, and the point with the highest height in the grid can be used as the road boundary point.

[0077] The height difference between every two adjacent grids is searched outward from the middle grid because after a vehicle travels on the road, the road boundary is generally outside the vehicle. Therefore, the outer grid of the target adjacent grid is the grid with a higher height among the target adjacent grids.

[0078] S250 , summarizing a plurality of road boundary points in the left area and the right area respectively to determine a set of road boundary points corresponding to the left side and the right side of the vehicle, respectively.

[0079] S260 , fitting the road boundary point set corresponding to the left side of the vehicle and the road boundary point set corresponding to the right side of the vehicle respectively to generate complete curve equations of the road corresponding to the left side and the right side of the vehicle respectively.

[0080] S270. Determine whether there is a road gap on the left or right side of the vehicle. For the vehicle side with a road gap, generate a road segment curve equation corresponding to the vehicle side, and use the road segment curve equation as the effective road curve equation for the vehicle side; for the vehicle side without a road gap, use the road complete curve equation corresponding to the vehicle side as the effective road curve equation.

[0081] In some embodiments, determining whether there is a road gap on the left or right side of the vehicle and generating a road segment curve equation corresponding to the vehicle side with the road gap may include:

[0082] respectively calculating the distance between every two adjacent road boundary points in the road boundary point set, determining that adjacent road boundary points whose distances are greater than a third preset threshold are target adjacent road boundary points, and determining that there are road gaps between the target adjacent road boundary points;

[0083] According to the determined at least one road gap, a road boundary point set on the vehicle side where the road gap exists is split to generate a plurality of boundary sub-point sets;

[0084] Multiple boundary sub-point sets on the vehicle side where the road gap exists are fitted to generate the road segment curve equation corresponding to the vehicle side.

[0085] Optionally, a third preset threshold may be used to determine whether there is a gap in the road. When the distance between target adjacent road boundary points in the road boundary point set is greater than the third preset threshold, it may be determined that there is a road gap between the target adjacent road boundary points.

[0086] S280: Acquire multiple interior points of multiple historically updated road effective curve equations within a specified time period, convert the multiple interior points into the current vehicle coordinate system, and combine them to form an interior point set.

[0087] S290. Based on the inner point set, perform at least one of the following: calculate a first distance between each inner point in the inner point set located on the target side of the vehicle and the complete curve equation of the road on the target side of the vehicle, and calculate a second distance between each inner point in the inner point set located on the target side of the vehicle and each valid curve equation of the road on the target side of the vehicle.

[0088] S2100: Determine an interior point whose first distance or second distance is less than a fourth preset threshold as a target interior point, and determine the target interior point as a valid interior point on the target side of the vehicle.

[0089] Optionally, the fourth preset threshold is a preset inlier range value, which can be used to determine whether the inlier is a valid inlier.

[0090] S2110. Add multiple valid interior points on the vehicle target side to a road boundary point set that matches the vehicle target side, and update the road effective curve equation on the vehicle target side according to the road boundary point set to which the valid interior points are added.

[0091] In some embodiments, if there is a gap in the road, the effective boundary curve of the road obtained by only fitting the point cloud data once may cause deviations due to other environmental factors, and the accuracy of the identification of the road gap cannot be guaranteed. Therefore, by adding valid internal points, the scope of the road gap can be determined again.

[0092] S2120: Mark the curve where each updated road valid curve equation is located as a valid road boundary to obtain multiple valid road boundaries, and mark the gaps between the multiple valid road boundaries as road boundary gaps.

[0093] In an embodiment of the present application, by obtaining road boundary point cloud data within an ROI, the heights of multiple grids pre-divided within the ROI are determined, and road boundary point sets corresponding to the left and right sides of the vehicle are respectively determined based on the heights of the multiple grids within the ROI area and the left or right areas to which the multiple grids belong. Then, a complete road curve equation and a road effective curve equation corresponding to the left and right sides of the vehicle are generated based on the road boundary point sets. Then, an inner point set is obtained based on the historically updated road effective curve equation, and a valid inner point is determined in the inner point set. The road effective curve equation is updated based on the valid inner point, thereby generating a road boundary detection result. This method can filter out the influence of obstacles and special road sections on road boundary detection, and solves the problem of inaccurate road boundary detection results caused by fitting only the point cloud collected by the lidar in the related art. At the same time, updating the road effective boundary curve based on the valid inner point can solve the problem of the collected data being affected by the surrounding environment at a certain moment, reduce the road gap size error, and effectively improve the accuracy of road boundary detection.

[0094] FIG4 is a schematic diagram of the structure of a road boundary detection device provided in an embodiment of the present application. As shown in FIG4 , the device includes: a grid height acquisition module 310 , a curve equation generation module 320 , a valid interior point determination module 330 , and a road boundary detection result generation module 340 .

[0095] The grid height acquisition module 310 is configured to acquire the road boundary point cloud data within the ROI in real time, and determine the heights of a plurality of grids pre-divided within the ROI based on the road boundary point cloud data.

[0096] The curve equation generation module 320 is configured to determine the road boundary point sets corresponding to the left and right sides of the vehicle respectively based on the heights of multiple grids within the ROI area and the left or right areas to which the multiple grids belong, and generate the complete curve equations and effective curve equations of the road corresponding to the left and right sides of the vehicle respectively based on the road boundary point sets.

[0097] The effective road curve equation is determined by whether there is a road gap on the left or right side of the vehicle.

[0098] The valid inlier determination module 330 is configured to determine an inlier point set based on a plurality of historically updated road valid curve equations within a specified time period, and to determine a plurality of valid inliers in the inlier point set based on at least one of the road complete curve equation and the road valid curve equation.

[0099] The road boundary detection result generating module 340 is configured to add the valid inliers to the road boundary point set, and update the road valid curve equation according to the road boundary point set after adding the valid inliers to generate a road boundary detection result.

[0100] In an embodiment of the present application, by obtaining road boundary point cloud data within an ROI, the heights of multiple grids pre-divided within the ROI are determined, and road boundary point sets corresponding to the left and right sides of the vehicle are respectively determined based on the heights of the multiple grids within the ROI area and the left or right areas to which the multiple grids belong. Then, a complete road curve equation and a road effective curve equation corresponding to the left and right sides of the vehicle are generated based on the road boundary point sets. Then, an inner point set is obtained based on the historically updated road effective curve equation, and a valid inner point is determined in the inner point set. The road effective curve equation is updated based on the valid inner point, thereby generating a road boundary detection result. This method can filter out the influence of obstacles and special road sections on road boundary detection, and solves the problem of inaccurate road boundary detection results caused by fitting only the point cloud collected by the lidar in the related art. At the same time, updating the road effective boundary curve based on the valid inner point can solve the problem of the collected data being affected by the surrounding environment at a certain moment, reduce the road gap size error, and effectively improve the accuracy of road boundary detection.

[0101] Based on the above embodiments, the grid height acquisition module 310 is configured as follows:

[0102] The point cloud data of the interfering objects is deleted from the initial point cloud data within the ROI acquired in real time, and the remaining initial point cloud data is downsampled to generate the road boundary point cloud data within the ROI;

[0103] Dividing the road boundary point cloud data into pre-divided grids, and obtaining point cloud data of each grid respectively;

[0104] The point clouds whose point cloud height values ​​in the point cloud data of each grid exceed a first preset threshold are filtered, and the height value of the highest point cloud in each grid after filtering is used as the height of the grid.

[0105] On the basis of the above embodiments, a grid division module may be further included. The grid division module is configured to:

[0106] Obtaining motion data of the vehicle, and calculating the current position of the vehicle based on the motion data;

[0107] Calculating a motion trajectory equation of the vehicle based on the vehicle's historical position within a specified time period and the vehicle's current position, and determining the vehicle's motion trajectory based on the motion trajectory equation;

[0108] The grid passed by the motion trajectory is determined as the middle grid, and multiple grids located on the left side of the middle grid in the ROI are divided into a left area, and multiple grids located on the right side of the middle grid are divided into a right area.

[0109] Based on the above embodiments, the curve equation generating module 320 may include:

[0110] The height difference calculation unit is configured to start from each middle grid, calculate the height difference between each two adjacent grids in the left area and the right area, and determine a target adjacent grid whose height difference is greater than a second preset threshold, determine a road boundary point in an outer grid of the target adjacent grid, and stop the height difference calculation related to the outer grid;

[0111] A road boundary point set determination unit is configured to respectively aggregate a plurality of road boundary points in the left region and the right region to determine a road boundary point set corresponding to the left side and the right side of the vehicle, respectively;

[0112] a complete curve equation generating unit configured to respectively fit a set of road boundary points corresponding to the left side of the vehicle and a set of road boundary points corresponding to the right side of the vehicle to generate complete curve equations of the road corresponding to the left side and the right side of the vehicle respectively;

[0113] The road effective curve equation generating unit is configured to determine whether there is a road gap on the left and right sides of the vehicle. For the vehicle side with a road gap, a road segmented curve equation corresponding to the vehicle side is generated, and the road segmented curve equation is used as the road effective curve equation for the vehicle side; for the vehicle side without a road gap, the road complete curve equation corresponding to the vehicle side is used as the road effective curve equation.

[0114] Based on the above embodiments, the road effective curve equation generating unit is configured as follows:

[0115] respectively calculating the distance between every two adjacent road boundary points in the road boundary point set, determining that adjacent road boundary points whose distances are greater than a third preset threshold are target adjacent road boundary points, and determining that there are road gaps between the target adjacent road boundary points;

[0116] According to the determined at least one road gap, a road boundary point set on the vehicle side where the road gap exists is split to generate a plurality of boundary sub-point sets;

[0117] Multiple boundary sub-point sets on the vehicle side where the road gap exists are fitted to generate the road segment curve equation corresponding to the vehicle side.

[0118] Based on the above embodiments, the effective interior point determination module 330 is configured as follows:

[0119] Obtain multiple interior points of the effective road curve equations of multiple historical updates within a specified time period, convert the multiple interior points to the current vehicle coordinate system, and combine them to form an interior point set;

[0120] According to the interior point set, at least one of the following is performed: calculating a first distance between each interior point located on the vehicle target side in the interior point set and a complete curve equation of the road on the vehicle target side, and calculating a second distance between each interior point located on the vehicle target side in the interior point set and each effective curve equation of the road on the vehicle target side;

[0121] An interior point where the first distance or the second distance is smaller than a fourth preset threshold is determined as a target interior point, and the target interior point is determined as a valid interior point on the target side of the vehicle.

[0122] Based on the above embodiments, the road boundary detection result generating module 340 is configured to:

[0123] Adding multiple valid interior points on the target side of the vehicle to a road boundary point set that matches the target side of the vehicle, and updating the effective curve equation of the road on the target side of the vehicle based on the road boundary point set after adding the valid interior points;

[0124] The curve where each updated road valid curve equation is located is marked as a valid road boundary to obtain multiple valid road boundaries, and the gaps between the multiple valid road boundaries are marked as road boundary gaps.

[0125] The road boundary detection device provided in the embodiments of the present application can execute the road boundary detection method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0126] FIG5 shows a block diagram of an electronic device 10 that can be used to implement an embodiment of the present application. The electronic device can represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are for example only.

[0127] As shown in FIG5 , the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 and a random access memory (RAM) 13, that is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the ROM 12 or the computer program loaded from the storage unit 18 into the RAM 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0128] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0129] The processor 11 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the road boundary detection method described in the embodiments of the present application. That is:

[0130] Acquire road boundary point cloud data within the ROI in real time, and determine the heights of a plurality of grids pre-divided within the ROI based on the road boundary point cloud data;

[0131] Determining, based on the heights of multiple grids within the ROI region and the left or right regions to which the multiple grids belong, road boundary point sets corresponding to the left and right sides of the vehicle, respectively, and generating, based on the road boundary point sets, complete road curve equations and effective road curve equations corresponding to the left and right sides of the vehicle, respectively; wherein the effective road curve equation is determined by whether there is a road gap on the left or right side of the vehicle;

[0132] Determining an inlier point set based on a plurality of historically updated road effective curve equations within a specified time period, and determining a plurality of valid inliers in the inlier point set based on at least one of the road complete curve equation and the road effective curve equation;

[0133] The valid interior points are added to the road boundary point set, and according to the road boundary point set after the valid interior points are added, the road valid curve equation is updated to generate a road boundary detection result.

[0134] In some embodiments, the road boundary detection method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the road boundary detection method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the road boundary detection method in any other suitable manner (e.g., via firmware).

[0135] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), system-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (Electronic Programmable Read Only Memory, EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (Compact Disc-Read Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), or a monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0140] A computing system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services.

[0141] The various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the multiple steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of this application can be achieved.

Claims

1. A road boundary detection method, comprising: Obtaining in real time the road boundary point cloud data within the region of interest (ROI), and determining the heights of a plurality of pre-divided grids within the ROI according to the road boundary point cloud data; Determining respectively the road boundary point sets corresponding to the left and right sides of the vehicle according to the heights of the plurality of grids within the ROI region and the left or right regions to which the plurality of grids belong, and generating respectively the complete road curve equations and the valid road curve equations corresponding to the left and right sides of the vehicle according to the road boundary point sets; wherein, the valid road curve equation is determined by whether there is a road gap on the left or right side of the vehicle; Determining the inner point set according to a plurality of historically updated valid road curve equations within a specified time period, and determining a plurality of valid inner points within the inner point set according to at least one of the complete road curve equation and the valid road curve equation; Adding the valid inner points to the road boundary point set, and updating the valid road curve equation according to the road boundary point set after adding the valid inner points to generate a road boundary detection result.

2. The method according to claim 1, wherein, Obtaining in real time the road boundary point cloud data within the ROI, and determining the heights of a plurality of pre-divided grids within the ROI according to the road boundary point cloud data, comprising: Deleting the point cloud data of interfering objects from the initially acquired point cloud data within the ROI in real time, and performing downsampling processing on the remaining initially acquired point cloud data to generate the road boundary point cloud data within the ROI; Dividing the road boundary point cloud data according to the pre-divided grids to respectively obtain the point cloud data of each grid; Filtering the point cloud with the point cloud height value exceeding a first preset threshold in the point cloud data of each grid, and taking the height value of the highest point cloud within each grid after filtering as the height of the grid.

3. The method according to claim 1, before respectively determining the road boundary point sets corresponding to the left and right sides of the vehicle according to the heights of the plurality of grids within the ROI region and the left or right regions to which the plurality of grids belong, the method further comprises: Obtaining the motion data of the vehicle, and calculating the current position of the vehicle according to the motion data; Calculating the motion trajectory equation of the vehicle according to the historical position of the vehicle within a specified time period and the current position of the vehicle, and determining the motion trajectory of the vehicle according to the motion trajectory equation; Determining the grids passed by the motion trajectory as intermediate grids, dividing the plurality of grids on the left side of the intermediate grids in the ROI into a left region, and dividing the plurality of grids on the right side of the intermediate grids into a right region.

4. The method according to claim 3, wherein Determining respectively the road boundary point sets corresponding to the left and right sides of the vehicle according to the heights of the plurality of grids within the ROI region and the left or right regions to which the plurality of grids belong, and generating respectively the complete road curve equations and the valid road curve equations corresponding to the left and right sides of the vehicle according to the road boundary point sets, comprising: Starting from each intermediate grid, calculate the height differences between every two adjacent grids in the left region and the right region respectively, and determine the target adjacent grids with height differences greater than the second preset threshold. Determine the road boundary points among the outer grids of the target adjacent grids, and stop calculating the height differences related to the outer grids; Aggregate the multiple road boundary points in the left region and the right region respectively, and determine the road boundary point sets corresponding to the left and right sides of the vehicle; Fit the road boundary point set corresponding to the left side of the vehicle and the road boundary point set corresponding to the right side of the vehicle respectively, and generate the complete road curve equations corresponding to the left and right sides of the vehicle; Judge whether there are road gaps on the left and right sides of the vehicle. For the side of the vehicle with a road gap, generate the corresponding road segmented curve equation for this side of the vehicle, and use the road segmented curve equation as the effective road curve equation for this side of the vehicle; for the side of the vehicle without a road gap, use the complete road curve equation corresponding to this side of the vehicle as the effective road curve equation.

5. The method according to claim 4, wherein, Judge whether there are road gaps on the left and right sides of the vehicle. For the side of the vehicle with a road gap, generate the corresponding road segmented curve equation for this side of the vehicle, including: Calculate the distances between every two adjacent road boundary points in the road boundary point set respectively, and determine that the distance between adjacent road boundary points greater than the third preset threshold is the target adjacent road boundary point, and determine that there is a road gap between the target adjacent road boundary points; According to the determined at least one road gap, split the road boundary point set on the side of the vehicle with a road gap to generate multiple boundary sub-point sets; Fit the multiple boundary sub-point sets on the side of the vehicle with a road gap to generate the corresponding road segmented curve equation for this side of the vehicle.

6. The method according to claim 1, wherein Determine the inner point set according to multiple historically updated road effective curve equations within a specified time period, and determine multiple valid inner points in the inner point set according to at least one of the road complete curve equation and the road effective curve equation, including: Obtain multiple inner points of multiple historically updated road effective curve equations within a specified time period, and convert the multiple inner points to the current vehicle coordinate system and combine them to form an inner point set; According to the inner point set, perform at least one of the following: calculate the first distance between each inner point located on the target side of the vehicle in the inner point set and the road complete curve equation on the target side of the vehicle, and calculate the second distance between each inner point located on the target side of the vehicle in the inner point set and each road effective curve equation on the target side of the vehicle; Determine the inner points with the first distance or the second distance less than the fourth preset threshold as the target inner points, and determine the target inner points as the valid inner points on the target side of the vehicle.

7. The method according to claim 6, wherein Add the valid inner points to the road boundary point set, and update the road effective curve equation according to the road boundary point set after adding the valid inner points to generate a road boundary detection result, including: Add the multiple valid inner points on the target side of the vehicle to the road boundary point set matching the target side of the vehicle, and update the road effective curve equation on the target side of the vehicle according to the road boundary point set after adding the valid inner points; Mark the curve where each updated effective road curve equation is located as an effective road boundary, obtaining multiple effective road boundaries, and mark the gaps between the multiple effective road boundaries as road boundary gaps.

8. A road boundary detection device, comprising: A grid height acquisition module, configured to acquire in real time the road boundary point cloud data within a region of interest (ROI), and determine the heights of multiple grids pre-divided within the ROI according to the road boundary point cloud data; A curve equation generation module, configured to respectively determine the road boundary point sets corresponding to the left and right sides of the vehicle according to the heights of multiple grids within the ROI region and the left or right regions to which the multiple grids belong, and generate the complete road curve equations and effective road curve equations corresponding to the left and right sides of the vehicle according to the road boundary point sets; wherein, the effective road curve equation is determined by whether there is a road gap on the left or right side of the vehicle; An effective inlier determination module, configured to determine an inlier point set according to multiple historically updated effective road curve equations within a specified time period, and determine multiple effective inliers in the inlier point set according to at least one of the complete road curve equation and the effective road curve equation; A road boundary detection result generation module, configured to add the effective inliers to the road boundary point set, and update the effective road curve equation according to the road boundary point set after adding the effective inliers, and generate a road boundary detection result.

9. An electronic device, the electronic device comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the road boundary detection method according to any one of claims 1-7 of the present application.

10. A computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the road boundary detection method according to any one of claims 1-7 when executed by a processor.

Images