Obstacle detection method, device, equipment and medium based on path constraints

Abstract

This application discloses a path-constrained obstacle detection method, apparatus, device, and medium, relating to the field of autonomous driving. The path-constrained obstacle detection method includes: generating a trajectory region on the parking trajectory based on the position and direction of the parking trajectory; concatenating multiple continuous trajectory regions into a path-constrained region of the parking trajectory; obtaining obstacles to be tracked in the scene and filtering out obstacles to be tracked outside the path-constrained region; and performing multi-frame matching on the obstacles to be tracked in the path-constrained region to identify target obstacles that are temporally continuous among the obstacles to be tracked.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and in particular to a method, device, equipment and medium for obstacle detection based on path constraints. Background Art

[0002] In autonomous parking systems, the BEV+Transformer architecture is often used to implement tasks such as 3D target detection and free space detection. However, due to the physical limitations of the camera and other issues, there are problems such as near-field perception blind spots, missed detection of special obstacles or obstacles of the same color, and invalid tracking.

[0003] For example, fisheye cameras experience high image distortion rates close to the vehicle body, leading to free-space polygon collapse or inaccurate bounding boxes for 3D obstacles. Furthermore, objects like ground locks and thin poles often have small contact points, making them susceptible to model subtraction. Ultrasonic sensors also often miss detections. Furthermore, as the distance between the camera and obstacles decreases, the obstacles often occupy a large area of ​​the image, making it difficult for the model to distinguish their true appearance. This leads to various missed detections and ultimately, collision risks.

[0004] In addition, when obstacles are falsely detected, if there are some safety backup strategies in the post-processing, the entire parking space will become narrow, thereby reducing the drivable area, and ultimately leading to lower planning efficiency. For example, commonly used planning algorithms such as heuristic optimization methods may encounter unsolvable situations. Summary of the Invention

[0005] This application mainly provides a path constraint-based obstacle detection method, device, equipment and medium to solve the problem of insufficient obstacle perception accuracy in autonomous driving scenarios.

[0006] To solve the above technical problems, the present application adopts a technical solution: providing a path constraint-based obstacle detection method, comprising: generating a trajectory region on the parking trajectory based on the position and direction of the parking trajectory; splicing multiple consecutive trajectory regions into a path constraint region of the parking trajectory; obtaining obstacles to be tracked in the scene and filtering out obstacles to be tracked outside the path constraint region; and performing multi-frame matching on the obstacles to be tracked in the path constraint region to identify temporally continuous target obstacles among the obstacles to be tracked.

[0007] In some embodiments, generating a trajectory region on the parking trajectory based on the position and direction of the parking trajectory includes: selecting a first segment of multiple continuous trajectories in the vehicle's travel direction as the parking trajectory; and constructing the trajectory region on the parking trajectory based on the distance and direction angle between a starting discrete point and an ending discrete point on the parking trajectory.

[0008] In some embodiments, stitching the multiple continuous trajectory areas into the path constraint area of ​​the parking trajectory includes: stitching the trajectory areas based on normal vectors between the discrete points and the multiple continuous trajectory areas; and expanding the stitched trajectory areas based on the length and width of the vehicle to form the path constraint area.

[0009] In some embodiments, obtaining obstacles to be tracked in the scene includes: detecting obstacles in the scene and classifying the obstacles to determine the types of the obstacles; and using obstacles classified as static as obstacles to be tracked in the scene.

[0010] In some embodiments, determining the type of the obstacle includes: in response to the obstacle having only a free space polygon detection result, marking the type of the obstacle as static; or, in response to the obstacle having only a 3D bounding box detection result, marking the type of the obstacle as dynamic.

[0011] In some embodiments, the obstacle detection method further includes: in response to the obstacle intersecting with an obstacle having an ultrasonic detection result, taking the obstacle as an obstacle to be tracked in the scene.

[0012] In some embodiments, performing multi-frame matching on the obstacles to be tracked in the path constraint area to identify target obstacles that are continuous in time sequence among the obstacles to be tracked includes: continuously acquiring multiple frames of images of the obstacles to be tracked; performing multi-target tracking and matching using the images of the obstacles to be tracked; extracting vertices of a 3D detection box and contour point sets of a free space polygon from successfully matched obstacle pairs, wherein the point sequence of the contour point sets meets the point sequence requirements of a boots algorithm library; and merging all the contour point sets to generate a convex hull, and using the convex hull as a geometric representation of the successfully matched obstacle.

[0013] To solve the above technical problems, another technical solution adopted in this application is to provide a path constraint-based obstacle detection method, comprising: a generation module for generating a trajectory area on the parking trajectory based on the position and direction of the parking trajectory; a splicing module for splicing multiple consecutive trajectory areas into a path constraint area of ​​the parking trajectory; a screening module for acquiring obstacles to be tracked in the scene and screening out obstacles to be tracked outside the path constraint area; and an identification module for performing multi-frame matching on the obstacles to be tracked in the path constraint area to identify temporally continuous target obstacles among the obstacles to be tracked.

[0014] The present application also provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the computer device executes the obstacle detection method as described above.

[0015] The present application also provides a computer-readable storage medium having instructions stored thereon, and when the instructions are executed by a processor, the obstacle detection method as described above is implemented.

[0016] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses a path-constrained obstacle detection method, device, equipment, and medium. Based on the position and direction of the parking trajectory, a trajectory area is generated on the parking trajectory; multiple continuous trajectory areas are spliced ​​into the path constraint area of ​​the parking trajectory; the obstacles to be tracked in the scene are obtained, and the obstacles to be tracked outside the path constraint area are screened out, and the tracking range of the obstacles is constrained to a limited area, reducing data processing redundancy and improving the efficiency of obstacle tracking and identification. Multi-frame matching is performed on the obstacles to be tracked in the path constraint area to identify the target obstacles that are continuous in time sequence among the obstacles to be tracked, thereby achieving effective tracking and identification of obstacles, increasing safety, and reducing collision risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0018] Figure 1 This is a flow chart of an embodiment of an obstacle detection method based on path constraints provided by the present application;

[0019] Figure 2 Yes Figure 1 Schematic diagram of the path constraint area of ​​the parking trajectory in the method shown;

[0020] Figure 3 Yes Figure 1 The flowchart of the embodiment of step 10 of the method is shown;

[0021] Figure 4 Yes Figure 1 The flowchart of the embodiment of step 20 of the method is shown;

[0022] Figure 5 Yes Figure 1 The flowchart of the embodiment of step 30 of the method is shown;

[0023] Figure 6 Yes Figure 1 The flowchart of the embodiment of step 40 of the method is shown;

[0024] Figure 7 1 is a schematic structural diagram of an embodiment of an obstacle detection device based on path constraints provided by the present application;

[0025] Figure 8 It is a structural diagram of an embodiment of a computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] The terms "first", "second" and "third" in the embodiments of the present application are only used for descriptive purposes and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally also include steps or units that are not listed, or may optionally also include other steps or units inherent to these processes, methods, products or devices.

[0028] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] See Figure 1 , Figure 1 1 is a flow chart of an embodiment of an obstacle detection method based on path constraints, the obstacle detection method comprising the following steps:

[0030] 10: Generate a trajectory area on the parking trajectory based on the position and direction of the parking trajectory.

[0031] When an autonomous vehicle performs automated parking, it generates a parking trajectory in the direction of travel. This trajectory consists of multiple continuous segments. Each segment is defined by a number of discrete points. By calculating the straight-line distance and direction angle between these points, a corresponding rectangular area is generated as the trajectory area corresponding to that segment of the parking trajectory.

[0032] See Figure 2 , Figure 2 Yes Figure 1 A schematic diagram of the path constraint area of ​​the parking trajectory in the illustrated method. During automated parking, the vehicle follows a predetermined path, which is divided into multiple, continuous, short segments, each of which is called a trajectory segment. A trajectory segment is the basic unit that constitutes the entire parking trajectory.

[0033] A trajectory is the entire path a vehicle takes from its starting parking position to its target parking position, and is composed of a series of continuous trajectory segments. For example, during parking, a vehicle plans a route from its current position to its target parking space. The current position serves as the vehicle's starting parking position, and the target parking space serves as the vehicle's target parking position. The route forms a continuous trajectory segment extending from the current position to the target parking space.

[0034] These trajectory segments are logically continuous, representing the continuous motion of the vehicle during parking. To mathematically and computationally describe trajectory segments, each segment is further subdivided into a number of points, known as discrete points. The overall parking trajectory is planned first, then, to implement and control it, it is divided into several shorter segments. Each segment is part of the trajectory, and together they complete the parking task.

[0035] Discrete points are a specific representation of a trajectory segment. By connecting these points, we can approximate the shape and orientation of the trajectory segment. Discrete points are sampled uniformly or on demand along the trajectory segment using algorithms or planning tools. The location and number of these points depend on the length and shape of the trajectory segment, as well as the required computational accuracy.

[0036] The continuous trajectory area is a series of rectangles. The length of each rectangle is related to the straight-line distance between two points, and the width is related to the vehicle width. A compensation value is added to ensure coverage of the vehicle body and safety range.

[0037] For any two adjacent discrete points on a trajectory segment, the straight-line distance is the Euclidean distance between them, which can be obtained through simple geometric calculations. This distance reflects the actual distance the vehicle traveled between the two points.

[0038] The heading angle is the angle between the tangent direction at a point on a trajectory segment and a reference direction (such as the vehicle's initial orientation or a direction in the global coordinate system). The heading angle is calculated by calculating the vector between two adjacent discrete points and using the angle formula between the vector and the reference direction. The heading angle is used to determine the direction of travel of the trajectory segment at each point.

[0039] Further, see Figure 3 , step 10 includes the following steps:

[0040] 11: Select the first segment of multiple continuous trajectories in the vehicle's driving direction (the first driving segment on the planned driving path) as the parking trajectory.

[0041] The first segment of the continuous trajectory is selected as the starting segment of the parking trajectory. The starting and ending points of this segment are defined as points A and B, respectively. The length of the rectangular area is determined by calculating the straight-line distance from point A to point B. The width of the rectangular area is then determined by combining the vehicle width and the compensation value. This method generates a rectangular area corresponding to the first segment of the parking trajectory, ensuring that it covers the vehicle's safe driving range.

[0042] 12: Constructing a trajectory area on the parking trajectory based on the distance and direction angle between the starting discrete point and the terminal discrete point on the parking trajectory.

[0043] By calculating the distance and direction angle between discrete points on each trajectory, a corresponding rectangular area is generated, ensuring that the length of each rectangle is consistent with the distance between the points, and the width is the vehicle width plus the compensation value.

[0044] For each discrete point on the trajectory segment, a rectangular area is generated based on the straight-line distance (as the length of the rectangle) and the direction angle of the point, as well as the vehicle width plus a certain compensation value (as the width of the rectangle). This rectangular area covers the possible driving range of the vehicle at that point and takes into account a certain safety margin.

[0045] Connect the rectangular areas corresponding to all discrete points on the trajectory segment to form a continuous trajectory area. This trajectory area represents the possible range of the vehicle in the entire parking trajectory and is used for subsequent obstacle tracking and obstacle avoidance planning.

[0046] 20: Combine multiple continuous trajectory regions into the path constraint region of the parking trajectory.

[0047] By splicing each trajectory area one by one, a complete path constraint area is formed to ensure coverage of the entire parking trajectory and ensure the safety and accuracy of the vehicle during the parking process.

[0048] The path constraint area guides the autonomous vehicle to adhere to the predetermined path during parking, preventing deviations and ensuring accurate and safe parking maneuvers. This area not only encompasses the vehicle's actual trajectory but also provides additional safety buffer space to effectively prevent potential collisions.

[0049] At the same time, obstacles are screened and tracked using the path constraint area. First, the obstacle is determined to be within the path constraint area. The polygons formed by the path constraint area are used to determine whether they intersect with the obstacle polygons or lines. If so, the obstacle is added to the candidate queue.

[0050] Further, see Figure 4 , step 20 comprises the following steps:

[0051] 21: According to the normal vectors between each discrete point and multiple continuous trajectory areas, the trajectory area is spliced.

[0052] By calculating the normal vectors between each discrete point, the system ensures smooth boundaries when stitching together rectangular areas, avoiding overlaps or gaps. Each trajectory segment is stitched together one by one to form a continuous and seamless path constraint area that covers the entire parking trajectory, ensuring the safety and accuracy of the vehicle during parking.

[0053] The normal vector between discrete points is the direction vector perpendicular to the vector connecting two adjacent discrete points. The normal vector is used to determine the perpendicular direction of the tangent line of the trajectory segment at each discrete point, determine the orientation of the rectangle, and ensure that the rectangle correctly covers the possible driving range of the vehicle.

[0054] 22: Expand the spliced ​​trajectory area based on the vehicle length and vehicle width to form a path constraint area.

[0055] During expansion, the vehicle's length and width are considered, and rectangular areas are added to the front, body, and rear of the vehicle to ensure that the path constraint area covers the entire vehicle, preventing missed obstacles and improving parking safety. Through precise calculations, the expanded area seamlessly connects with the original trajectory area, forming a complete path constraint area, ensuring the accuracy and safety of autonomous parking operations.

[0056] 30: Obtain the obstacles to be tracked in the scene and filter out the obstacles to be tracked outside the path constraint area.

[0057] By comparing the obstacle location with the path constraint area boundary, obstacles that are not within the path constraint area can be accurately screened out, ensuring that only potential risk points are focused on, thereby improving the pertinence and efficiency of obstacle tracking.

[0058] Some obstacles in the scene are selected as obstacles to be tracked and identified. The intersection of the trajectory polygon and the obstacle polygon or line is used to accurately screen out obstacles within the path constraint area and add them to the candidate queue to ensure the effectiveness and accuracy of the tracking objects and further optimize the safety performance of the autonomous driving parking system.

[0059] The obstacles to be tracked will be continuously tracked and matched by the system, and some of them will be selected for fusion processing, and the Kalman filter will be used for state prediction and sent to the downstream processing unit.

[0060] Further, see Figure 5 , step 30 comprises the following steps:

[0061] 31: Detect obstacles in the scene and classify them to determine the type of obstacle.

[0062] Obstacle data detected by each sensor is obtained and initially screened by calculating the geometric intersection of the trajectory polygon and the obstacle boundary (polygon / line segment). The vertices of the trajectory polygon and the vertices of the obstacle boundary are compared to determine whether they intersect or overlap.

[0063] Optionally, collision detection is performed through simple geometric judgment, the Separating Axis Theorem (SAT), or a ray casting method is used to determine whether the obstacle vertex is within the trajectory area.

[0064] A trajectory polygon is a polygonal area constructed based on the vehicle's expected trajectory in an automated parking system. This polygon covers the vehicle's possible travel range and is used to determine which obstacles may collide with the vehicle. Factors such as vehicle size, direction of travel, and safety margin are typically considered when constructing the trajectory polygon. The trajectory polygon is defined by a series of discrete points and their normal vectors, which are connected to form a rectangular or nearly rectangular area that covers the vehicle's entire travel path.

[0065] The obstacle boundary is the geometric outline of an obstacle in two- or three-dimensional space. In autonomous parking systems, obstacles can be static (such as curbs and walls) or dynamic (such as pedestrians and other vehicles). Obstacle boundaries are typically detected and determined using perception sensors (such as cameras, radar, and lidar). Obstacle boundaries can be polygons (e.g., closed shapes connected by multiple vertices) or line segments (e.g., representing the edge of a curb). Accurate obstacle boundary information is required for collision detection and trajectory planning.

[0066] Geometric intersection occurs when two geometric shapes share a common portion in space. In automated parking systems, geometric intersection is used to determine whether the trajectory polygon overlaps or potentially touches the obstacle boundary. If the trajectory polygon intersects the obstacle boundary, it means the vehicle could potentially collide with the obstacle if it continues its current trajectory. Therefore, the system requires intersection detection to ensure safe parking.

[0067] Specifically, obstacles are analyzed for their triple attributes: physical characteristics (such as size and shape), dynamic characteristics (such as velocity vector and acceleration), and semantic categories (such as vehicles, pedestrians, or non-motor vehicles).

[0068] Specifically, the types of obstacles include but are not limited to motor vehicles (static / dynamic), non-motor vehicles, pedestrians, ground obstacles, etc.

[0069] Specifically, in response to the obstacle having only a free space polygon detection result, the type of the obstacle is marked as static.

[0070] Alternatively, in response to the obstacle having only a 3D bounding box detection result, the obstacle type is marked as dynamic.

[0071] In the autonomous driving perception system, obstacle type labeling rules can be based on the detection results, including free space polygon detection results and 3D bounding box detection results.

[0072] The free space polygon detection result is used when the obstacle is only identified as a polygonal area by the free space detection algorithm (such as the non-drivable area generated by semantic segmentation). It is marked as a static obstacle by default. The basis for this judgment is that free space detection mainly targets fixed ground obstacles (such as curbs, isolation piers, etc.).

[0073] 3D bounding box detection results are used to mark obstacles as dynamic by default when bounding boxes are generated through 3D object detection (such as the bounding box output from a LiDAR point cloud or visual fusion). Dynamic characteristics are determined based on geometric properties such as the bounding box's implicit size and orientation, as well as the velocity vector calculated using time-series tracking.

[0074] If both detection results exist, 3D bounding box data is used first and multimodal fusion verification is enabled. Free space polygons can be used as a supplementary verification for 3D detection (such as confirming the location of static obstacles).

[0075] Through comprehensive analysis of the two detection results, the system can more accurately identify obstacle status, improve the robustness of the perception system, and ensure the accuracy and safety of autonomous driving decisions.

[0076] 32: Obstacles classified as static are considered as obstacles to be tracked in the scene.

[0077] Although static obstacles don't tend to move, the obstacle avoidance constraints used in path planning must be continuously tracked to prevent sudden changes or the emergence of new obstacles due to environmental changes. By updating the position of static obstacles in real time, the system can dynamically adjust the driving path to ensure safe passage. Furthermore, continuous monitoring of the status of static obstacles helps identify potential risks, further improving the reliability and stability of the autonomous driving system.

[0078] Optionally, in response to the obstacle intersecting with an obstacle having an ultrasonic detection result, the obstacle is taken as an obstacle to be tracked in the scene.

[0079] Although obstacles that intersect with ultrasonic obstacles are not directly output to the downstream system, they still need to be tracked to ensure that there is obstacle data for output when the ultrasonic obstacle is missed.

[0080] When the obstacle polygon detected by other sensors (such as LiDAR or vision) geometrically overlaps with the ultrasonic detection area, or when the Separating Axis Theorem (SAT) is used for rapid collision detection, the obstacle is determined to intersect with the ultrasonic barrier if the calculated intersection area ratio exceeds a preset threshold (e.g., ≥15%). A unique tracking ID is assigned to each eligible obstacle, its attributes are recorded, and ultrasonic ranging data is preferentially used to correct the obstacle's position.

[0081] 40: Perform multi-frame matching on the obstacles to be tracked in the path constraint area to identify the target obstacles that are continuous in time sequence among the obstacles to be tracked.

[0082] When performing multi-frame matching on obstacles to be tracked within the path constraint area, a spatiotemporal alignment mechanism is first established to uniformly convert the sensor data of consecutive frames into the vehicle coordinate system, and the convex hull of the free space polygon is simplified to reduce the complexity of subsequent matching calculations.

[0083] A free space polygon (FSP) is a geometric model used in autonomous driving systems to describe the safe driving area for a vehicle. Sensors such as lidar, cameras, and 4D millimeter-wave radar detect obstacle boundaries in the environment and infer the obstacle-free area within which the vehicle can navigate. A polygon formed by connecting a series of vertices covers the range of potential safe paths for the vehicle at the current moment or during the forecast period. For example, in a parking scenario, the FSP may be determined by both the parking space boundaries and the vehicle's dynamic constraints.

[0084] When performing multi-frame matching of obstacles within the path constraint area, the processing of free space polygons specifically includes mapping the sensor data of consecutive frames into a unified vehicle coordinate system, extracting the obstacle boundaries based on the raw sensor data through clustering and edge detection algorithms, defining the area outside the obstacle boundaries as free space, and constructing the corresponding polygonal representation. Complex free space polygons are simplified into convex polygons to reduce the computational complexity of the subsequent matching algorithm.

[0085] When performing geometric intersection detection between the simplified convex polygon and the obstacle trajectory, the computational efficiency is significantly improved due to the reduction in the number of vertices.

[0086] The core algorithm of multi-frame matching uses the Hungarian algorithm to associate obstacles between frames. The matching criteria include polygon overlapping area ratio and motion consistency. Successfully matched obstacles are assigned a continuous tracking ID and the time series trajectory points are recorded.

[0087] The motion states (position, velocity, acceleration, etc.) of successfully tracked obstacles in consecutive frames are recorded in chronological order, forming a spatiotemporal trajectory chain, which is used to record the time information of the trajectory points.

[0088] The degree of geometric space overlap is measured by calculating the ratio of the intersection area and the union area of ​​obstacle polygons (such as free space or trajectory polygons) in adjacent frames; the difference in motion vectors between the Kalman filter predicted trajectory and the current frame detection result is combined to determine whether the two conform to the motion pattern of the same target.

[0089] This example uses the Hungarian algorithm to construct a bipartite graph model, using polygon overlap ratio and motion consistency as edge weights to find the maximum matching solution with the minimum cost. A match is successful when the combined matching cost between the target obstacle and a trajectory is the lowest.

[0090] Optionally, an elimination rule is set to remove the frame from the tracking list if three consecutive frames fail to match.

[0091] This approach not only enables efficient dynamic obstacle identification, but also accurately grasps the real-time status of static obstacles, ensuring accurate path planning. Furthermore, the introduction of an elimination mechanism effectively reduces mistracking, improving the overall perception system's responsiveness and decision-making reliability.

[0092] Further, see Figure 6 , step 40 further includes the following steps:

[0093] 41: Continuously acquire multiple frames of images of the obstacle to be tracked.

[0094] Through image processing technology, the characteristic information of obstacles is extracted and compared with multi-frame data. Obstacles with successfully matched characteristic information are included in the continuous tracking list to ensure the continuity of tracking.

[0095] 42:Multi-target tracking and matching through images of obstacles to be tracked.

[0096] During multi-target tracking and matching, image feature information is integrated with spatiotemporal data to further improve matching accuracy. Successfully matched obstacles remain in the tracking list, and their trajectory information is updated. For obstacles that fail to match for multiple consecutive frames, the system removes them from the tracking list according to pre-set rules to prevent the accumulation of mistracking.

[0097] Specifically, the system extracts the 2D / 3D bounding boxes, category probability distributions, and appearance features of detected objects from consecutive video frames. It then maintains the detection set for the current frame and the set of previously tracked objects. It then constructs a cost matrix encompassing both spatial and semantic distance dimensions, using weighted fusion to form a comprehensive matching cost. The Hungarian algorithm is then used to find the optimal matching solution, ensuring a minimal global matching cost.

[0098] The target with successful matching updates the motion trajectory, the unmatched detection initializes a new trajectory, and the continuously mismatched tracking target is removed from the system.

[0099] Through this series of steps, the system can accurately identify and track obstacles in real time, providing reliable perception support for autonomous driving.

[0100] 43: Extract the vertices of the 3D detection box and the contour point set of the free space polygon from the successfully matched obstacle pairs. The point order of the contour point set meets the point order requirements of the boots algorithm library.

[0101] The eight vertices of the 3D detection box and the contour point set of the free space polygon are extracted from the successfully matched obstacle pairs, and all input polygons are merged to output the minimum circumscribed convex polygon, covering all original obstacle areas.

[0102] Obstacle pairs are formed by matching the same obstacle instances detected by adjacent frames or different sensors using a data association algorithm (such as the Hungarian algorithm).

[0103] 44: Merge all contour point sets to generate a convex hull, and use the convex hull as the geometric representation of the successfully matched obstacle, and use the successfully matched obstacle as the target obstacle.

[0104] Based on the time series, the matched obstacles are considered obstacle pairs, and their contour points are found. These contour points include bounding boxes for 3D detection and pure polygons for free space detection. These points are then fed into the Boost algorithm library, where a union operation is used to generate a new convex hull polygon, ensuring a larger area and convex hull properties.

[0105] The outline point set of an obstacle pair refers to the set of geometric feature points of two consecutive frames (such as the t-1th frame and the tth frame) that are judged to be the same obstacle in temporal multi-frame matching, including the bounding box point set of 3D detection and the pure polygon point set of free space detection.

[0106] The fused convex hull is used as the new geometric representation of the obstacle to update the obstacle state. The Kalman filter is used to predict the next frame position and narrow the search range for the next round of matching.

[0107] A new convex hull is generated by merging the outline point sets of the obstacle pair. The 3D bounding box vertices and free-space polygon vertices from the two frames are merged into a unified set. The convex hull vertices are extracted from the merged point set, and the output polygon satisfies convexity. The new convex hull must cover the entire area of ​​the original point set, so its area is usually larger than the convex hull of a single frame.

[0108] The convex hull is the smallest convex polygon that contains all points without any concave areas. The fused convex hull can more robustly represent the temporal and spatial motion trends of obstacles.

[0109] The center point, length, width and other parameters of the fused convex hull are input into the Kalman filter to predict the position and speed of the next frame; the candidate matching area of ​​the next frame is delineated based on the prediction results.

[0110] Through this obstacle tracking method, the system can effectively identify and handle dynamic and static obstacles, improving the robustness of the perception system.

[0111] Through trajectory tracking, the system accurately identifies obstacles within a limited range, reducing redundant code and improving safety. Kalman filtering predicts position and optimizes matching searches, ensuring efficient handling of both dynamic and static obstacles and enhancing the stability of the perception system. The modular design facilitates maintenance and allows for flexible adjustments as perception models are upgraded, ensuring continuous system optimization.

[0112] Optionally, based on specific services and scenarios, other areas can be selected as tracking areas. For example, in low-speed, enclosed scenarios, fixed grid partitioning and IOU matching are used to balance accuracy and real-time performance; in high-speed, dynamic scenarios, the convex hull of the millimeter-wave radar point cloud and the visual ROI area are combined with the adaptive Hungarian algorithm.

[0113] Fixed grid partitioning divides the scene into regular rectangular grid cells and manages the target position through grid index.

[0114] ‌IOU (Intersection over Union) matching is used to measure the overlap ratio of target areas in adjacent frames.

[0115] Millimeter-wave radar point cloud convex hull extracts obstacle outline vertices from radar point cloud data and constructs a minimum convex polygon to represent the target geometry. Millimeter-wave radar provides high-refresh rate point cloud data, compensating for the motion blur observed in high-speed vision.

[0116] The visual ROI (Region of Interest) is a region of interest defined based on the camera detection frame or semantic segmentation results.

[0117] Optionally, the tracking algorithm used in this application includes but is not limited to the Kalman filter introduced in the above embodiment, and the matching algorithm can also be adjusted according to actual needs, such as using Hungarian matching, or further reducing the computational complexity through greedy matching or nearest neighbor matching algorithms.

[0118] The above describes the obstacle detection method based on path constraint in the embodiment of the present invention. The following describes the obstacle detection device based on path constraint in the embodiment of the present invention. Figure 7 In one embodiment of the present invention, an obstacle detection device based on path constraints includes:

[0119] The generating module 401 is configured to generate a track area on the parking track based on the position and direction of the parking track.

[0120] The splicing module 402 is configured to splice multiple continuous trajectory regions into a path constraint region of the parking trajectory.

[0121] The screening module 403 is used to obtain obstacles to be tracked in the scene and screen out obstacles to be tracked outside the path constraint area.

[0122] The identification module 404 is configured to perform multi-frame matching on the obstacles to be tracked in the path constraint area to identify target obstacles that are continuous in time sequence among the obstacles to be tracked.

[0123] above Figure 7 The characteristic feature extraction device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the computer device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0124] Figure 8This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Computer device 500 may vary significantly depending on configuration or performance. It may include one or more processors (central processing units, CPUs) 510 (e.g., one or more processors), memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) storing application programs 533 or data 532. Memory 520 and storage media 530 may be either transient or persistent storage. The program stored in storage medium 530 may include one or more modules (not shown), each of which may include a series of instructions operating on computer device 500. Furthermore, processor 510 may be configured to communicate with storage medium 530 to execute the series of instructions stored in storage medium 530 on computer device 500.

[0125] The computer device 500 may further include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 8 The illustrated computer device structure does not limit the computer device and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0126] The present invention also provides a computer device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the path constraint-based obstacle detection method in the above-mentioned embodiments.

[0127] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to execute the steps of the path constraint-based obstacle detection method.

[0128] Different from the existing technology, this application adopts a path constraint approach to obstacle detection for automatic parking, and specifically provides a method for constructing a path constraint area. It rationally constructs a path constraint area for obstacle screening, improving obstacle tracking efficiency while ensuring safety. By optimizing the construction method of the trajectory area and combining multiple tracking algorithms such as the Kalman filter, efficient obstacle detection and fusion are achieved, further reducing the system's computational complexity and improving overall performance.

[0129] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0130] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0131] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An obstacle detection method based on path constraints, characterized in that: include: Selecting the first segment of multiple continuous trajectories in the vehicle's driving direction as the parking trajectory; Constructing a trajectory area on the parking trajectory based on the distance and direction angle between the starting discrete point and the terminal discrete point on the parking trajectory; splicing the trajectory areas according to the normal vectors between the discrete points and multiple continuous trajectory areas; Expanding the spliced ​​trajectory area based on the length and width of the vehicle to form the path constraint area; Obtain obstacles to be tracked in the scene, and filter out obstacles to be tracked outside the path constraint area; Multi-frame matching is performed on the obstacles to be tracked in the path constraint area to identify target obstacles that are continuous in time sequence among the obstacles to be tracked.

2. The obstacle detection method according to claim 1, characterized in that: Obtaining obstacles to be tracked in the scene includes: Detecting obstacles in the scene and classifying the obstacles to determine the type of the obstacles; Obstacles classified as static are regarded as obstacles to be tracked in the scene.

3. The obstacle detection method according to claim 2, wherein: Determining the type of the obstacle includes: In response to the obstacle having only a free space polygon detection result, marking the type of the obstacle as static; Alternatively, in response to the obstacle having only a 3D bounding box detection result, the type of the obstacle is marked as dynamic.

4. The obstacle detection method according to claim 2, wherein: The obstacle detection method further includes: In response to the obstacle intersecting with an obstacle having an ultrasonic detection result, the obstacle is taken as an obstacle to be tracked in the scene.

5. The obstacle detection method according to claim 1, wherein: The performing multi-frame matching on the obstacles to be tracked in the path constraint area to identify target obstacles that are continuous in time sequence among the obstacles to be tracked includes: Continuously acquiring multiple frames of images of the obstacle to be tracked; Performing multi-target tracking and matching through the image of the obstacle to be tracked; Extract the vertices of the 3D detection box and the contour point set of the free space polygon from the successfully matched obstacle pairs, where the point order of the contour point set meets the point order requirements of the boots algorithm library; All the contour point sets are merged to generate a convex hull, and the convex hull is used as a geometric representation of the successfully matched obstacle, and the successfully matched obstacle is used as the target obstacle.

6. An obstacle detection device based on path constraints, characterized in that: include: a generating module, configured to generate a trajectory area on the parking trajectory based on the position and direction of the parking trajectory; The generating module is further configured to select a first segment of a plurality of continuous trajectories in the direction of vehicle travel as a parking trajectory; and construct a trajectory region on the parking trajectory based on a distance and a direction angle between a starting discrete point and an ending discrete point on the parking trajectory; a splicing module configured to splice a plurality of continuous trajectory regions into a path constraint region of the parking trajectory; the splicing module further configured to splice the trajectory regions based on normal vectors between the discrete points and the plurality of continuous trajectory regions; and to expand the spliced ​​trajectory regions based on a vehicle length and a vehicle width to form the path constraint region; A screening module, configured to obtain obstacles to be tracked in the scene and screen out obstacles to be tracked outside the path constraint area; The identification module is configured to perform multi-frame matching on the obstacles to be tracked in the path constraint area to identify target obstacles that are continuous in time sequence among the obstacles to be tracked.

7. A computer device, characterized in that: The computer device includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the computer device to execute the obstacle detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the obstacle detection method according to any one of claims 1 to 5 is implemented.

Images