Road boundary detection method and device and vehicle
By sensing the static obstacle targets on the road and generating the target trajectory centerline, the problem of low road boundary detection accuracy in complex scenarios in the prior art is solved, and higher boundary detection accuracy and road planning accuracy are achieved.
Patent Information
- Application Number
- CN202311424116.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-30
- Publication Date
- 2025-05-02
AI Technical Summary
The prior art has low accuracy when detecting road boundaries in complex scenarios, making it difficult to deal with discontinuous road boundaries.
By obtaining the current image frame, we sense the stationary obstacle target on the road, and generate the target trajectory centerline based on the vehicle's turning radius and obstacle target, determine the boundary point of the road, and finally generate the road's boundary.
Improve the accuracy of boundary detection on curves, provide accurate road planning, timely avoid obstacles and avoid collisions.
Smart Images
Figure CN119919900A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a method, device and vehicle for detecting road boundaries. Background Art
[0002] Detecting road boundaries has wide application value in fields such as autonomous driving and road planning. In existing technologies, the boundary detection capabilities for various complex scenes such as curves are limited, and the accuracy of the detection results is low. Summary of the invention
[0003] In order to overcome the problems existing in the related art, the present disclosure provides a road boundary detection method, device, vehicle, and computer-readable storage medium to solve the problem that various discontinuous road boundaries cannot be detected. The technical solution of the present disclosure is as follows:
[0004] According to a first aspect of an embodiment of the present disclosure, a method for detecting road boundaries is provided, comprising: acquiring a currently acquired image frame, and sensing a stationary obstacle target on the road based on the image frame; generating a target trajectory centerline of the vehicle based on a turning radius of the vehicle and the obstacle target; determining a boundary point of the road based on the target trajectory centerline, and generating a boundary of the road based on the boundary point.
[0005] In one embodiment of the present disclosure, generating the target trajectory centerline of the vehicle based on the turning radius of the vehicle and the obstacle target includes: performing an i-th iteration on the vehicle trajectory based on the turning radius of the vehicle at the i-th iteration to obtain a predicted trajectory position i, wherein i is a natural number greater than or equal to 1; determining whether there is a second obstacle target near the predicted trajectory position i; if the second obstacle target exists near the predicted trajectory position i, performing an offset operation on the predicted trajectory position i to obtain the trajectory centerline i corresponding to the i-th iteration, and performing an i+1-th iteration until the iteration ends to generate the target trajectory centerline.
[0006] In one embodiment of the present disclosure, the iterating the vehicle trajectory for the i-th time based on the turning radius of the vehicle at the i-th iteration to obtain a predicted trajectory position i includes: obtaining the predicted trajectory position i-1 obtained at the i-1-th iteration; and determining the predicted trajectory position i based on the predicted trajectory position i-1, the turning radius at the i-th iteration, and a preset step value.
[0007] In one embodiment of the present disclosure, the biasing operation on the predicted trajectory position i to obtain the trajectory center line i corresponding to the i-th iteration includes: performing a biasing operation on the predicted trajectory position i to obtain the target predicted trajectory position i of the i-th iteration; and generating the trajectory center line i corresponding to the i-th iteration based on the target predicted trajectory position obtained from the first iteration to the i-th iteration.
[0008] In one embodiment of the present disclosure, the biasing operation on the predicted trajectory position i to obtain the target predicted trajectory position i of the i-th iteration includes: determining the relative position relationship between the second obstacle target and the predicted trajectory position i; and based on the relative position relationship, biasing the predicted trajectory position i to obtain the target predicted trajectory position i.
[0009] In one embodiment of the present disclosure, the predicted trajectory position i is offset based on the relative position relationship to obtain the target predicted trajectory position i, including: determining the lateral distance and azimuth relationship between the second obstacle target and the predicted trajectory position i; if the lateral distance is greater than a second set distance, according to the azimuth relationship, offsetting the predicted trajectory position i by a set value in a direction away from the second obstacle target to obtain the target predicted trajectory position i; if the lateral distance is less than or equal to the second set distance, determining the turning direction and turning angle of the predicted trajectory position i, and offsetting the predicted trajectory position i by increasing the turning angle under the constraint of the turning direction to obtain the target predicted trajectory position i.
[0010] In one embodiment of the present disclosure, the method further includes: if there is still a third obstacle target near the target predicted trajectory position i obtained through the bias operation, then terminating subsequent iterations of the vehicle trajectory.
[0011] In one embodiment of the present disclosure, determining the boundary points of the road based on the center line of the target trajectory includes: acquiring a first obstacle target located on both sides of the center line of the target trajectory and closest to the center line of the target trajectory; determining the first obstacle target as a candidate boundary point of the road; performing fitting based on the candidate boundary points to generate a fitting curve, and determining the fitting points on the fitting curve as the boundary points of the road.
[0012] In one embodiment of the present disclosure, after generating the boundary of the road based on the boundary points, the method further includes: obtaining first boundary points that are a first set distance away from the boundary; if the number of the first boundary points is less than the set number, deleting the boundary of the road.
[0013] In one embodiment of the present disclosure, the method further includes: if the image frame does not detect the boundary of the road, determining the historical boundary of the road generated by a historical image frame adjacent to the image frame; and predicting the boundary of the road corresponding to the image frame based on the historical boundary.
[0014] According to a second aspect of an embodiment of the present disclosure, a road boundary detection device is provided, comprising: a perception module, used to obtain a currently acquired image frame, and perceive a stationary obstacle target on the road based on the image frame; a first generation module, used to generate a target trajectory centerline of the vehicle based on a turning radius of the vehicle and the obstacle target; and a second generation module, used to determine a boundary point of the road based on the target trajectory centerline, and generate a boundary of the road based on the boundary point.
[0015] According to a third aspect of an embodiment of the present disclosure, a vehicle is provided, comprising a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the steps of the method described in the first aspect of the embodiment of the present disclosure.
[0016] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a vehicle, the steps of the method described in the first aspect of the embodiment of the present disclosure are implemented.
[0017] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, characterized in that when the computer program is executed by a vehicle, the steps of the method described in the first aspect of the embodiment of the present disclosure are implemented.
[0018] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by collecting the current image frame and perceiving and identifying the stationary obstacle targets on the road, the impact of moving or high-altitude obstacles on the road boundary can be avoided. Then, based on the turning radius of the vehicle and the obstacle target, the target trajectory centerline is generated to determine the boundary points of the road, and the boundary of the road is generated based on the boundary points, which improves the accuracy of boundary detection on the curve, facilitates accurate road planning, and timely avoids obstacles to avoid collisions.
[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0021] Figure 1It is a flowchart of a method for detecting road boundaries according to some embodiments of the present disclosure.
[0022] Figure 2 is a flowchart of another method for detecting road boundaries according to some embodiments of the present disclosure.
[0023] Figure 3 is a schematic diagram of a first obstacle target according to some embodiments of the present disclosure.
[0024] Figure 4 is a flowchart of another method for detecting road boundaries according to some embodiments of the present disclosure.
[0025] Figure 5 is a schematic diagram of a target trajectory centerline according to some embodiments of the present disclosure.
[0026] Figure 6 is a flowchart of generating a target trajectory centerline according to some embodiments of the present disclosure.
[0027] Figure 7 is a flowchart of another method for detecting road boundaries according to some embodiments of the present disclosure.
[0028] Figure 8 The present invention is a flowchart showing how to detect road boundaries according to some embodiments of the present disclosure.
[0029] Fig. 9 is a schematic diagram showing the generation of road boundaries according to some embodiments of the present disclosure.
[0030] Fig.10 It is a structural block diagram of a road boundary detection device according to some embodiments of the present disclosure.
[0031] Fig.11 It is a schematic diagram of a functional block diagram of a vehicle according to some embodiments of the present disclosure. DETAILED DESCRIPTION
[0032] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.
[0033] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0034] Figure 1 is a flow chart of a road boundary detection method according to some embodiments of the present disclosure, such as Figure 1 As shown, the process of the road boundary detection method includes but is not limited to the following steps:
[0035] S101, obtaining a currently captured image frame, and sensing a stationary obstacle target on the road based on the image frame.
[0036] It should be noted that the execution subject of the road boundary detection method in the embodiment of the present disclosure is an electronic device, and the electronic device may be a vehicle control system or a vehicle-mounted terminal, etc. The road boundary detection method in the embodiment of the present disclosure may be executed by the road boundary detection device in the embodiment of the present disclosure, and the road boundary detection device in the embodiment of the present disclosure may be configured in any electronic device to execute the road boundary detection method in the embodiment of the present disclosure.
[0037] In some implementations, the current image frame may be acquired based on the vehicle's own sensor or a sensor independent of the vehicle. Alternatively, the current image frame may be acquired based on the vehicle's own onboard radar, wherein the onboard radar may be a laser radar, a millimeter wave radar, or the like.
[0038] Optionally, the captured image frames may be sensed based on a target detection algorithm to obtain a detection result, wherein the detection result may include: the location of the obstacle, the category of the obstacle, etc. The category of the obstacle may be divided into stationary obstacles and non-stationary obstacles.
[0039] Furthermore, the stationary obstacle target on the road in the image frame can be identified based on the detection result. Optionally, the stationary obstacle target on the road can be determined based on the confidence assigned to the obstacle during target detection. The confidence reflects the reliability of the detection result. If the confidence of an obstacle is high and the category of the obstacle is a stationary obstacle, then according to the position of the obstacle, it is determined to be a stationary obstacle target on the road.
[0040] By way of example, the perceived obstacles may include but are not limited to pedestrians, road posts, traffic signs, traffic lights, motor vehicles, non-motor vehicles, etc. Among them, stationary obstacles are road posts, traffic signs and traffic lights, and non-stationary obstacles are pedestrians, motor vehicles and non-motor vehicles. Then, according to the position of the obstacle, the stationary obstacle target on the road is determined to be a road post and a traffic sign.
[0041] S102, generating a target trajectory centerline of the vehicle based on the turning radius of the vehicle and the obstacle target.
[0042] In some implementations, the vehicle trajectory can be predicted based on the turning radius of the vehicle to obtain the initial trajectory centerline. Then, it is determined whether the initial trajectory centerline conflicts with the obstacle target. If there is a conflict, the initial trajectory centerline is offset to obtain the target trajectory centerline; if there is no conflict, the initial trajectory centerline is determined to be the target trajectory centerline.
[0043] In some implementations, if the lateral distance between the obstacle target and the center line of the initial trajectory is less than or equal to a distance threshold, it can be considered that there is a conflict between the center line of the initial trajectory and the obstacle target. The lateral distance refers to the distance from the obstacle target to the center line of the initial trajectory in the horizontal direction.
[0044] In some implementations, the turning radius of the vehicle may be determined based on the vehicle speed and turning angular velocity. Alternatively, the vehicle speed and turning angular velocity may be obtained based on the vehicle's own control system. The formula for calculating the turning radius is as follows:
[0045] R=V / yawrate (1)
[0046] Among them, R represents the turning radius, V represents the vehicle speed, and yawrate represents the turning angular velocity.
[0047] S103, determining the boundary points of the road based on the center line of the target trajectory, and generating the boundary of the road based on the boundary points.
[0048] In some implementations, the boundary points of the road can be determined based on the positional relationship, distance relationship, and continuity of the obstacle target and the center line of the target trajectory. Optionally, obstacle targets located on the left and right sides of the center line of the target trajectory and with a distance less than a set threshold can be selected as candidate boundary points. The boundary points of the road are then determined based on the continuity of the candidate boundary points.
[0049] Further, the boundary points are connected to generate a continuous road boundary. Optionally, the boundary points can be connected by linear fitting based on the geometric relationship between the boundary points to obtain the road boundary. For example, the boundary points can be connected by straight line fitting, curve fitting, etc.
[0050] It should be noted that the road boundary detection method disclosed in the present invention can be applied to the automatic driving of vehicles. Obtaining accurate road boundaries is the key to achieving automatic driving. The generated road boundaries can provide accurate road edge information for the automatic driving system, helping the vehicle to perform operations such as lane keeping, lane changing, and traffic behavior prediction. The automatic driving system can make decisions and plan routes in a timely manner to avoid collisions, achieve safe and reliable automatic driving, and help promote the development and application of automatic driving technology.
[0051] In the road boundary detection method provided by the embodiment of the present disclosure, by collecting the current image frame and sensing and identifying the stationary obstacle targets on the road, the influence of moving or high-altitude obstacles on the road boundary can be avoided. Then, based on the turning radius of the vehicle and the obstacle target, the target trajectory center line is generated to determine the boundary points of the road, and the road boundary is generated based on the boundary points, which improves the accuracy of boundary detection on the curve, facilitates accurate road planning, and timely avoids obstacles to avoid collisions.
[0052] Figure 2 is a flow chart of a road boundary detection method according to some embodiments of the present disclosure, such as Figure 2 As shown, the process of the road boundary detection method includes but is not limited to the following steps:
[0053] S201, obtaining a currently captured image frame, and sensing a stationary obstacle target on the road based on the image frame.
[0054] In the embodiment of the present disclosure, the implementation method of step S201 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0055] S202, generating a target trajectory centerline of the vehicle based on the turning radius of the vehicle and the obstacle target.
[0056] In the embodiment of the present disclosure, the implementation method of step S202 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0057] S203, obtaining a first obstacle target located on both sides of the center line of the target trajectory and closest to the center line of the target trajectory.
[0058] In some implementations, based on the positional relationship and distance relationship between the target trajectory centerline and the obstacle target, the obstacle targets located on both sides of the target trajectory centerline and closest to the target trajectory centerline can be determined and used as the first obstacle target.
[0059] Optionally, the obstacle target may be regarded as being closest to the center line of the target trajectory if the horizontal distance between the obstacle target and the center line of the target trajectory is less than or equal to a set distance threshold.
[0060] For example, Figure 3 The figure shows a schematic diagram of the first obstacle target, and the center line of the target trajectory is the solid line in the figure. There are multiple obstacle targets near the center line of the target trajectory. Figure 3 The circle in represents the obstacle target, and the solid line represents the center line of the target trajectory. The obstacle targets are located on both sides of the center line of the target trajectory, but obstacle target A is farther away, so obstacle target B, obstacle target C, obstacle target D, obstacle target E, and obstacle target F are selected as the first obstacle target.
[0061] S204: Determine the first obstacle target as a candidate boundary point of the road.
[0062] S205 , performing fitting based on the candidate boundary points to generate a fitting curve, and determining the fitting points on the fitting curve as boundary points of the road.
[0063] In some implementations, the first obstacle target is used as a candidate boundary point of the road for fitting to obtain a fitting curve, thereby determining the boundary point of the road. Optionally, the candidate boundary points may be fitted based on a random sample consensus (RANSAC) algorithm. For example, a third-order fitting may be performed on the candidate boundary points based on the RANSAC algorithm to generate a fitted third-order curve.
[0064] It is understandable that, when fitting based on candidate boundary points, the candidate boundary points need to be preprocessed, such as removing abnormal candidate boundary points, to ensure the accuracy of fitting. In other words, the fitting points on the fitting curve can accurately describe the boundary of the road, so the fitting points are determined as the boundary points of the road.
[0065] S206: Generate a road boundary based on the boundary points.
[0066] In the embodiment of the present disclosure, the method for generating the boundary of the road in step S206 can be implemented by any method in the various embodiments of the present disclosure, which is not limited here and will not be repeated.
[0067] It is understandable that since there may be a tunnel on the current road, and the top of the tunnel is captured by the radar (millimeter wave radar) at a long distance, it will cause the tunnel to be misdetected and generate a false road boundary.
[0068] In some implementations, the generated boundaries may be screened to remove false road boundaries. Optionally, the road boundaries may be screened based on the number of boundary points within a first set distance.
[0069] Optionally, by obtaining first boundary points that are at a first set distance from the boundary, if the number of first boundary points is less than the set number, the boundary of the road is deleted. For example, the number of first boundary points within 15 meters near the boundary (such as left and right) is obtained, and if the number of first boundary points is less than or equal to 2, it is determined that this boundary is a false road boundary and the boundary is deleted.
[0070] In some implementations, for an image frame captured at the current moment, if the image frame does not detect a road boundary, the historical boundary of the road generated by the historical image frame adjacent to the image frame is determined, and then based on the historical boundary, the boundary of the road corresponding to the image frame is predicted.
[0071] Optionally, based on the geometry of historical boundaries, curvature changes and other information, the driving trend of the vehicle can be obtained, the characteristics of the road boundary, such as turning angles, existing obstacles and targets, etc., can be determined, and then the boundary of the road corresponding to the current image frame can be predicted.
[0072] In the road boundary detection method provided by the embodiment of the present disclosure, by collecting the current image frame and sensing and identifying the stationary obstacle targets on the road, the influence of moving or high-altitude obstacles on the road boundary can be avoided. Then, based on the turning radius of the vehicle and the obstacle target, the target trajectory centerline is generated. By obtaining the first obstacle target as a candidate boundary point and fitting the candidate boundary points to determine the boundary points of the road, the boundary of the road is generated based on the boundary points, which improves the accuracy of boundary detection on the curve, facilitates the provision of accurate road planning, and timely avoids obstacles to avoid collisions.
[0073] Figure 4 is a flow chart of a road boundary detection method according to some embodiments of the present disclosure, such as Figure 4 As shown, the process of the road boundary detection method includes but is not limited to the following steps:
[0074] S401, obtaining a currently captured image frame, and sensing a stationary obstacle target on the road based on the image frame.
[0075] In the embodiment of the present disclosure, the implementation method of step S401 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0076] S402, performing an i-th iteration on the vehicle trajectory based on the turning radius of the vehicle at the i-th iteration to obtain a predicted trajectory position i, where i is a natural number greater than or equal to 1.
[0077] In some implementations, the predicted trajectory position may be determined based on the turning radius of the vehicle. The formula for calculating the predicted trajectory position is as follows:
[0078] x=[R 2 -(yR) 2 ] 0.5 Formula (2)
[0079] Where x is the predicted trajectory position, R is the turning radius, and y is the preset step value during iteration. Formula (2) shows the scenario where the vehicle turns right. When the vehicle turns left, the above formula needs to be multiplied by -1 to obtain a negative predicted trajectory position.
[0080] In some implementations, when the vehicle trajectory is iterated to generate the predicted trajectory position, the predicted trajectory position of the current iteration can be determined based on the predicted trajectory position of the previous iteration and the offset of the predicted trajectory position. If the i-th iteration is performed, the predicted trajectory position i-1 obtained at the i-1-th iteration is obtained, and then the predicted trajectory position i is determined based on the predicted trajectory position i-1, the turning radius at the i-th iteration, and the preset step value. The formula for the predicted trajectory position i is as follows:
[0081] x[y(n)]=x[y(n-1)]+[R 2 -(y(n)-R) 2 ] 0.5 -[R 2 -(y(n-1)-R) 2 ] 0.5 Formula (3)
[0082] Where n is the number of iterations, and its value is i, i is a natural number greater than or equal to 1, x is the predicted trajectory position, R is the turning radius, and y is the preset step value. For example, if 1 is used as the preset step value for iteration, then y = 0, 1, 2, 3, etc. That is to say, based on the above formula (3), the predicted trajectory position i at the i-th iteration can be determined as x[y(i)] = x[y(i-1)] + [R 2 -(y(i)-R) 2 ] 0.5 -[R 2 -(y(i-1)-R) 2 ] 0.5 .
[0083] S403, determining whether there is a second obstacle target near the predicted trajectory position i.
[0084] In some implementations, it may be determined whether there is an obstacle target that conflicts with the predicted trajectory position i based on the lateral distance between the obstacle target and the predicted trajectory position i, and the conflicting obstacle target may be used as the second obstacle target.
[0085] Optionally, based on the distance threshold, it can be determined whether there is an obstacle target that conflicts with the predicted trajectory position i. If the lateral distance between the obstacle target and the predicted trajectory position is less than or equal to the distance threshold, the obstacle target is determined to be the second obstacle target.
[0086] S404: If there is a second obstacle target near the predicted trajectory position i, an offset operation is performed on the predicted trajectory position i to obtain the trajectory centerline i corresponding to the i-th iteration, and the i+1-th iteration is performed until the iteration is completed to generate the target trajectory centerline.
[0087] In some implementations, a bias operation may be performed on the predicted trajectory position i to obtain the target predicted trajectory position i of the i-th iteration. Starting from the first iteration, a bias operation may be performed on the predicted trajectory position in each iteration to obtain the target predicted trajectory position. Based on the target predicted trajectory positions obtained from the first iteration to the i-th iteration, a trajectory centerline i corresponding to the i-th iteration is generated.
[0088] Optionally, the target predicted trajectory position i can be obtained by determining the relative position relationship between the second obstacle target and the predicted trajectory position i, and biasing the predicted trajectory position i based on the relative position relationship. The relative position relationship refers to the lateral distance and azimuth relationship between the second obstacle target and the predicted trajectory position. The lateral distance refers to the distance in the horizontal direction between the obstacle target and the predicted trajectory position, and the azimuth relationship includes the second obstacle target being on the right side of the predicted trajectory position and the second obstacle target being on the left side of the predicted trajectory position.
[0089] In some implementations, the predicted trajectory position i is offset by determining the lateral distance and azimuth relationship between the second obstacle target and the predicted trajectory position i. If the lateral distance is greater than the second set distance, the predicted trajectory position i is offset by a set value in a direction away from the second obstacle target according to the azimuth relationship to obtain the target predicted trajectory position i.
[0090] For example, if the second obstacle target is on the left side of the predicted trajectory position i and the lateral distance is greater than the second set distance, such as 0.5 meters, the predicted trajectory position i is offset to the right by 0.6 meters to obtain the target predicted trajectory position i.
[0091] Optionally, if the lateral distance is less than or equal to the second set distance, the turning direction and turning angle of the predicted trajectory position i are determined, and the predicted trajectory position i is offset by increasing the turning angle under the constraint of the turning direction to obtain the target predicted trajectory position i. For example, Figure 5 If the lateral distance of the second obstacle target in the middle dotted frame is less than or equal to the second set distance and the turning direction is to the right, the right turning angle is increased to obtain the target predicted trajectory position.
[0092] In some implementations, if there is still a third obstacle target near the target predicted trajectory position i obtained after the bias operation, the subsequent iteration of the vehicle trajectory is terminated. The third obstacle target refers to the second obstacle target that conflicts with the target predicted trajectory position.
[0093] Furthermore, based on the trajectory center line i corresponding to the i-th iteration, the i+1-th iteration is performed until the iteration ends to generate a complete trajectory center line, that is, the target trajectory center line. The iteration ends when the number of iterations reaches the set value. For example, if the iteration is set to 5 times, the iteration ends after the 5th iteration is completed.
[0094] For example, Figure 5 What is shown in the figure is a schematic diagram of the center line of the target trajectory. The circle is the obstacle target, the dotted line is the center line of the trajectory before the offset, and the dotted frame is the second obstacle target with which there is a conflict. Since the lateral distance of the second obstacle target is equal to the second set distance and the turning direction is to the right, the turning angle to the right is increased to obtain the center line of the target trajectory, which is the solid line part.
[0095] Figure 6 What is shown is a flow chart for generating the target trajectory centerline. Based on the motion state of the vehicle, such as the turning radius, the vehicle trajectory is predicted to obtain the predicted trajectory position, and the predicted trajectory position is iterated to generate the target trajectory centerline. By judging whether there is a second obstacle target that conflicts with the predicted trajectory position near the predicted trajectory position, if so, the predicted trajectory position is offset to obtain the target predicted trajectory position. Continue to judge whether there is a third obstacle target that conflicts with the predicted trajectory position near the target predicted trajectory position. If so, the iteration of the vehicle trajectory is terminated. If not, the trajectory centerline corresponding to this iteration is generated based on the target predicted trajectory position, and subsequent iterations are performed until the iteration ends to obtain the target trajectory centerline.
[0096] S405, determining the boundary points of the road based on the center line of the target trajectory, and generating the boundary of the road based on the boundary points.
[0097] In the embodiment of the present disclosure, the implementation method of step S405 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0098] In the road boundary detection method provided by the embodiment of the present disclosure, by collecting the current image frame and sensing and identifying the stationary obstacle targets on the road, the influence of moving or high-altitude obstacles on the road boundary can be avoided. Then, based on the turning radius of the vehicle and the obstacle target, the vehicle trajectory is iterated to generate the center line of the target trajectory, and then the boundary points of the road are determined. The boundary of the road is generated based on the boundary points, which improves the accuracy of boundary detection on the curve, facilitates accurate road planning, and timely avoids obstacles to avoid collisions.
[0099] Figure 7 is a flow chart of a road boundary detection method according to some embodiments of the present disclosure, such as Figure 7 As shown, the process of the road boundary detection method includes but is not limited to the following steps:
[0100] S701, obtaining a currently captured image frame, and sensing a stationary obstacle target on the road based on the image frame.
[0101] S702, performing an i-th iteration on the vehicle trajectory based on the turning radius of the vehicle at the i-th iteration to obtain a predicted trajectory position i, where i is a natural number greater than or equal to 1.
[0102] S703, determining whether there is a second obstacle target near the predicted trajectory position i.
[0103] S704: If there is a second obstacle target near the predicted trajectory position i, an offset operation is performed on the predicted trajectory position i to obtain the trajectory centerline i corresponding to the i-th iteration, and the i+1-th iteration is performed until the iteration is completed to generate the target trajectory centerline.
[0104] S705: Acquire a first obstacle target located on both sides of the center line of the target trajectory and closest to the center line of the target trajectory.
[0105] S706: Determine the first obstacle target as a candidate boundary point of the road.
[0106] S707, performing fitting based on the candidate boundary points to generate a fitting curve, and determining the fitting points on the fitting curve as boundary points of the road.
[0107] S708: Generate a road boundary based on the boundary points.
[0108] S709: Acquire a first boundary point that is a first set distance away from the boundary.
[0109] S710: If the number of the first boundary points is less than the set number, delete the boundary of the road.
[0110] In the road boundary detection method provided by the embodiment of the present disclosure, by collecting the current image frame and sensing and identifying the stationary obstacle targets on the road, the influence of moving or high-altitude obstacles on the road boundary can be avoided. Then, based on the turning radius of the vehicle and the obstacle target, the target trajectory center line is generated to determine the boundary points of the road, and the road boundary is generated based on the boundary points, which improves the accuracy of boundary detection on the curve, facilitates accurate road planning, and timely avoids obstacles to avoid collisions.
[0111] Figure 8 The figure shows a flow chart of detecting the road boundary. Based on the vehicle speed and turning angle (yawrate), the turning radius of the vehicle is determined. And by sensing the collected image frames, the obstacle targets stationary on the road are determined, and then based on the turning radius and obstacle targets, the target trajectory centerline is generated.
[0112] Furthermore, based on the center line of the target trajectory, candidate boundary points of the road are determined, and a fitting curve is obtained by fitting the candidate boundary points. The fitting points on the fitting curve are determined as the boundary points of the road, and then the boundary of the road is generated based on the boundary points. Fig. 9 A schematic diagram of the boundary of the road generated based on the above method is shown. Fig. 9 The longest curve is the boundary of the road.
[0113] Fig.10 is a structural block diagram of a road boundary detection device according to some embodiments of the present disclosure. Fig.10 The road boundary detection device 1000 of the embodiment of the present disclosure includes a perception module 1001, a first generation module 1002, and a second generation module 1003.
[0114] The perception module 1001 is used to obtain the currently collected image frame and perceive the stationary obstacle target on the road based on the image frame.
[0115] The first generating module 1002 is used to generate a target trajectory centerline of the vehicle based on the turning radius of the vehicle and the obstacle target.
[0116] The second generating module 1003 is used to determine the boundary points of the road based on the center line of the target trajectory, and generate the boundary of the road based on the boundary points.
[0117] In one embodiment of the present disclosure, the first generation module 1002 is further used to: perform an i-th iteration on the vehicle trajectory based on the turning radius of the vehicle at the i-th iteration to obtain a predicted trajectory position i, wherein i is a natural number greater than or equal to 1; determine whether there is a second obstacle target near the predicted trajectory position i; if the second obstacle target exists near the predicted trajectory position i, perform an offset operation on the predicted trajectory position i to obtain a trajectory centerline i corresponding to the i-th iteration, and perform an i+1-th iteration until the iteration ends and the target trajectory centerline is generated.
[0118] In one embodiment of the present disclosure, the first generating module 1002 is further used to: obtain the predicted trajectory position i-1 obtained at the i-1th iteration; and determine the predicted trajectory position i based on the predicted trajectory position i-1, the turning radius at the i-th iteration and a preset step value.
[0119] In one embodiment of the present disclosure, the first generating module 1002 is further used to: perform a bias operation on the predicted trajectory position i to obtain the target predicted trajectory position i of the i-th iteration; and generate a trajectory centerline i corresponding to the i-th iteration based on the target predicted trajectory position obtained from the first iteration to the i-th iteration.
[0120] In one embodiment of the present disclosure, the first generating module 1002 is further used to: determine the relative position relationship between the second obstacle target and the predicted trajectory position i; based on the relative position relationship, bias the predicted trajectory position i to obtain the target predicted trajectory position i.
[0121] In one embodiment of the present disclosure, the first generation module 1002 is further used to: determine the lateral distance and azimuth relationship between the second obstacle target and the predicted trajectory position i; if the lateral distance is greater than a second set distance, according to the azimuth relationship, offset the predicted trajectory position i by a set value in a direction away from the second obstacle target to obtain the target predicted trajectory position i; if the lateral distance is less than or equal to the second set distance, determine the turning direction and turning angle of the predicted trajectory position i, and offset the predicted trajectory position i by increasing the turning angle under the constraint of the turning direction to obtain the target predicted trajectory position i.
[0122] In one embodiment of the present disclosure, the first generating module 1002 is further configured to terminate subsequent iterations of the vehicle trajectory if there is still a third obstacle target near the target predicted trajectory position i obtained through the bias operation.
[0123] In one embodiment of the present disclosure, the second generation module 1003 is further used to: obtain a first obstacle target located on both sides of the center line of the target trajectory and closest to the center line of the target trajectory; determine the first obstacle target as a candidate boundary point of the road; perform fitting based on the candidate boundary points to generate a fitting curve, and determine the fitting points on the fitting curve as boundary points of the road.
[0124] In one embodiment of the present disclosure, the second generation module 1003 is further used to: obtain first boundary points that are a first set distance away from the boundary; if the number of the first boundary points is less than the set number, delete the boundary of the road.
[0125] In one embodiment of the present disclosure, the second generation module 1003 is further used to: if the image frame does not detect the boundary of the road, determine the historical boundary of the road generated by the historical image frame adjacent to the image frame; based on the historical boundary, predict the boundary of the road corresponding to the image frame.
[0126] In the road boundary detection device provided by the embodiment of the present disclosure, by collecting the current image frame and sensing and identifying the stationary obstacle targets on the road, the influence of moving or high-altitude obstacles on the road boundary can be avoided. Then, based on the turning radius of the vehicle and the obstacle target, the target trajectory center line is generated to determine the boundary points of the road, and the boundary of the road is generated based on the boundary points, which improves the accuracy of boundary detection on the curve, facilitates accurate road planning, and timely avoids obstacles to avoid collisions.
[0127] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0128] Fig.11 1 is a block diagram of a vehicle 1100 according to some embodiments of the present disclosure. For example, the vehicle 1100 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle 1100 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.
[0129] Reference Fig.11 , the vehicle 1100 may include various subsystems, for example, an infotainment system 1101, a perception system 1102, a decision control system 1103, a drive system 1104, and a computing platform 1105. The vehicle 1100 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of the vehicle 1100 may be interconnected by wire or wireless means.
[0130] In some embodiments, the infotainment system 1101 may include a communication system, an entertainment system, and a navigation system.
[0131] The perception system 1102 may include several sensors for sensing information about the environment around the vehicle 1100. For example, the perception system 1102 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.
[0132] The decision control system 1103 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.
[0133] The drive system 1104 may include components that provide power movement for the vehicle 1100. In one embodiment, the drive system 1104 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air compression engine. The engine can convert energy provided by the energy source into mechanical energy.
[0134] Some or all functions of the vehicle 1100 are controlled by a computing platform 1105. The computing platform 1105 may include at least one processor 1151 and a memory 1152, and the processor 1151 may execute instructions 1153 stored in the memory 1152.
[0135] The processor 1151 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.
[0136] Memory 1152 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0137] In addition to instructions 1153 , memory 1152 may also store data, such as road maps, route information, vehicle location, direction, speed, etc. The data stored in memory 1152 may be used by computing platform 1105 .
[0138] In the embodiment of the present disclosure, the processor 1151 may execute instruction 1153 to complete all or part of the steps of the above-mentioned road boundary detection method.
[0139] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the road boundary detection method provided by the present disclosure are implemented.
[0140] In addition, the word "exemplary" is used herein to indicate serving as an example, instance, or diagram. Any aspect or design described as "exemplary" in this article is not necessarily understood to be advantageous compared to other aspects or designs. On the contrary, the use of the word exemplary is intended to present concepts in a concrete way. As used herein, the term "or" is intended to represent an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X applies A or B" is intended to represent any one of the natural inclusive arrangements. That is, if X applies A; X applies B; or X applies both A and B, "X applies A or B" is satisfied under any of the aforementioned examples. In addition, unless otherwise specified or clearly pointed to a singular form from the context, the articles "one" and "an" as used in this disclosure and the appended claims are generally understood to mean "one or more".
[0141] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art after reading and understanding the specification and drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components (e.g., elements, resources, etc.) described above, unless otherwise noted, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific functions of the described components, even if the structure is not equivalent to the disclosed structure. In addition, although the specific features of the present disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations as may be desired and conducive to any given or specific application. In addition, with respect to "including", "having", "having", "having", or variations thereof used in the specific embodiments or claims, such terms are intended to be inclusive in a manner similar to the term "comprising".
[0142] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0143] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
[0144] In the above detailed description, reference is made to the accompanying drawings, which show by way of illustration certain aspects in which the present disclosure may be practiced. In this regard, terms such as “center,” “longitudinal,” “lateral,” “length,” “width,” “thickness,” “upper,” “lower,” “front,” “back,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” “outer,” “outer,” “inner ...
[0145] Terms such as "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate directions or represent positional relationships. Since the components of the described devices can be positioned in a number of different orientations, the directional terms can be used for illustrative purposes rather than limiting. It should be understood that other aspects can be utilized and structural or logical changes can be made without departing from the concepts of the present disclosure. Therefore, the following detailed description should not be taken in a limiting sense.
[0146] It should be understood that, unless otherwise specifically noted, the features of some embodiments of the various present disclosures described herein may be combined with each other. As used herein, the term "and / or" includes any one of the related listed items and any combination of any two or more; similarly, "at least one of . . . " includes any one of the related listed items and any combination of any two or more.
[0147] It should be understood that, unless otherwise clearly specified and limited, the terms "joining", "attaching", "installing", "connecting", "connecting", "fixing" and the like used in the embodiments of the present disclosure should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly limited. For ordinary technicians in this field, the specific meanings of the above terms in this article can be understood according to specific circumstances.
[0148] In addition, the term "above" used in relation to a component, element or material layer formed "above" or located "above" a surface may be used herein to indicate that the component, element or material layer is "indirectly" positioned (e.g., placed, formed, deposited, etc.) on the surface such that one or more additional components, elements or layers are arranged between the surface and the component, element or material layer. However, the term "above" used in relation to a component, element or material layer formed "above" or located "above" a surface may also optionally have a specific meaning: the component, element or material layer is "directly" positioned (e.g., placed, formed, deposited, etc.) on the surface, such as in direct contact with the surface.
[0149] Although terms such as "first", "second" and "third" can be used herein to describe various components, parts, regions, layers or sections, these components, parts, regions, layers or sections are not limited to these terms. On the contrary, these terms are only used to distinguish one component, component, region, layer or section from another component, component, region, layer or section. Therefore, without departing from the teachings of each example, the first component, component, region, layer or section mentioned in the examples described herein may also be referred to as the second component, component, region, layer or section. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may expressly or implicitly include at least one of the features. In the description herein, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0150] It should be understood that spatially relative terms such as "above", "upper", "below", and "lower" are used herein to describe the relationship of one element to another element shown in the figures. In addition to the orientation depicted in the drawings, such spatially relative terms are also intended to include different orientations of the device in use or operation. For example, if the device in the drawings is turned over, the element described as being "above" or "upper" relative to another element will be "below" or "lower" relative to the other element. Therefore, depending on the spatial orientation of the device, the term "above" includes both upper and lower orientations. The device may have other orientations (e.g., rotated 90 degrees or in other orientations), and the spatially relative terms used herein should be interpreted accordingly.
Claims
1. A method for detecting road boundaries, characterized in that: The method comprises: Acquire the currently acquired image frame, and perceive the stationary obstacle target on the road based on the image frame; Based on the turning radius of the vehicle and the obstacle target, generating a target trajectory centerline of the vehicle; Based on the target trajectory centerline, boundary points of the road are determined, and based on the boundary points, a boundary of the road is generated.
2. The method according to claim 1, characterized in that The generating the target trajectory centerline of the vehicle based on the turning radius of the vehicle and the obstacle target includes: Performing an i-th iteration on the vehicle trajectory based on the turning radius of the vehicle at the i-th iteration to obtain a predicted trajectory position i, where i is a natural number greater than or equal to 1; Determine whether there is a second obstacle target near the predicted trajectory position i; If the second obstacle target exists near the predicted trajectory position i, an offset operation is performed on the predicted trajectory position i to obtain the trajectory center line i corresponding to the i-th iteration, and the i+1-th iteration is performed until the iteration is completed to generate the target trajectory center line.
3. The method according to claim 2, characterized in that The step of iterating the vehicle trajectory for the i-th time based on the turning radius of the vehicle at the i-th iteration to obtain a predicted trajectory position i includes: Obtaining the predicted trajectory position i-1 obtained at the i-1th iteration; The predicted trajectory position i is determined based on the predicted trajectory position i-1, the turning radius at the i-th iteration and the preset step value.
4. The method according to claim 2, characterized in that: The performing an offset operation on the predicted trajectory position i to obtain the trajectory center line i corresponding to the i-th iteration includes: Performing a bias operation on the predicted trajectory position i to obtain the target predicted trajectory position i of the i-th iteration; Based on the target predicted trajectory position obtained from the first iteration to the i-th iteration, a trajectory center line i corresponding to the i-th iteration is generated.
5. The method according to claim 2, characterized in that: The performing a bias operation on the predicted trajectory position i to obtain the target predicted trajectory position i of the i-th iteration includes: Determining a relative position relationship between the second obstacle target and the predicted trajectory position i; Based on the relative position relationship, the predicted trajectory position i is offset to obtain the target predicted trajectory position i.
6. The method according to claim 5, characterized in that The method of offsetting the predicted trajectory position i based on the relative position relationship to obtain the target predicted trajectory position i includes: Determining a lateral distance and azimuth relationship between the second obstacle target and the predicted trajectory position i; If the lateral distance is greater than a second set distance, the predicted trajectory position i is offset by a set value in a direction away from the second obstacle target according to the azimuth relationship to obtain the target predicted trajectory position i; If the lateral distance is less than or equal to the second set distance, the turning direction and turning angle of the predicted trajectory position i are determined, and the predicted trajectory position i is offset by increasing the turning angle under the constraint of the turning direction to obtain the target predicted trajectory position i.
7. The method according to any one of claims 4 to 6, characterized in that: The method further comprises: If there is still a third obstacle target near the target predicted trajectory position i obtained through the bias operation, the subsequent iteration of the vehicle trajectory is terminated.
8. The method according to any one of claims 1 to 6, characterized in that The determining the boundary point of the road based on the center line of the target trajectory includes: Obtaining first obstacle targets located on both sides of the center line of the target trajectory and closest to the center line of the target trajectory; Determining the first obstacle target as a candidate boundary point of the road; Fitting is performed based on the candidate boundary points to generate a fitting curve, and fitting points on the fitting curve are determined as boundary points of the road.
9. The method according to any one of claims 1 to 6, characterized in that: After generating the boundary of the road based on the boundary point, the method further includes: Acquire a first boundary point that is a first set distance away from the boundary; If the number of the first boundary points is less than a set number, the boundary of the road is deleted.
10. The method according to claim 9, characterized in that The method further comprises: If the image frame does not detect the boundary of the road, determining a historical boundary of the road generated by a historical image frame adjacent to the image frame; Based on the historical boundary, a boundary of the road corresponding to the image frame is predicted.
11. A road boundary detection device, characterized in that: The device comprises: A perception module, used to obtain a currently acquired image frame and perceive stationary obstacle targets on the road based on the image frame; A first generating module, configured to generate a target trajectory centerline of the vehicle based on a turning radius of the vehicle and the obstacle target; The second generating module is used to determine the boundary points of the road based on the center line of the target trajectory, and to generate the boundary of the road based on the boundary points.
12. A vehicle, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Implement the steps of the method according to any one of claims 1 to 10.
13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 10 are implemented.