Obstacle detection method and device based on path constraint, equipment and medium

Abstract

The invention discloses an obstacle detection method and device based on path constraint, equipment and a medium, and relates to the field of automatic driving. The obstacle detection method based on path constraint comprises the following steps: generating a track area on a parking track based on the position and direction of the parking track; splicing the multiple sections of continuous track areas into a path constraint area of the parking track; acquiring a to-be-tracked obstacle in the scene, and screening out the to-be-tracked obstacle outside the path constraint area; and performing multi-frame matching on the to-be-tracked obstacles in the path constraint region to identify target obstacles with continuous time sequences in the to-be-tracked obstacles.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and particularly to an obstacle detection method, device, equipment and medium based on path constraints. Background Art

[0002] In an autonomous parking system, the BEV+Transformer architecture is commonly used to implement tasks such as 3D object detection and free space detection. However, due to problems such as the physical limitations of cameras, there are problems such as near-field perception blind spots, missed detection of special obstacles or same-color obstacles, and ineffective tracking.

[0003] For example, a fisheye camera has a high image distortion rate when it is close to the vehicle body, which can cause the collapse of the free space polygon or inaccurate bounding boxes of 3D target obstacles. At the same time, objects such as ground locks and thin rods often have relatively small grounding points, making it easy for the model to deduct. Ultrasonic sensors also often have missed detection. And as the distance between the camera and the obstacle gets closer, from the image, the obstacle often occupies a large area of the image, resulting in the model being unable to distinguish the true appearance of the obstacle, leading to various missed detections and finally collision risks.

[0004] In addition, when there are false detections of obstacles, if there are some safety fallback strategies in the post-processing, the entire parking space will become narrow, resulting in a smaller drivable area and finally lower planning efficiency. For example, common planning algorithms such as those based on heuristic optimization methods may have no solution. Summary of the Invention

[0005] This application mainly provides an obstacle detection method, device, equipment and medium based on path constraints to solve the problem of insufficient obstacle perception accuracy in the autonomous driving scenario.

[0006] To solve the above technical problems, a technical solution adopted by this application is: to provide an obstacle detection method based on path constraints, including: generating a trajectory area on the parking trajectory based on the position and direction of the parking trajectory; splicing multiple continuous trajectory areas into a path constraint area of the parking trajectory; obtaining the obstacles to be tracked in the scene and screening out the obstacles to be tracked outside the path constraint area; performing multi-frame matching on the obstacles to be tracked in the path constraint area to identify the target obstacles with continuous time series in the obstacles to be tracked.

[0007] In some embodiments, generating a trajectory area on the parking trajectory based on the position and direction of the parking trajectory includes: selecting the first segment of multiple continuous trajectories in the vehicle driving direction as the parking trajectory; constructing a trajectory area on the parking trajectory based on the distance and direction angle between the starting discrete point and the terminal discrete point on the parking trajectory.

[0008] In some embodiments, the stitching of multiple continuous trajectory regions into the path constraint region of the parking trajectory includes: stitching the trajectory regions according to the normal vectors between the discrete points and the multiple continuous trajectory regions; and expanding the stitched trajectory region based on the length and width of the vehicle to form the path constraint region.

[0009] In some embodiments, the obtaining of the obstacles to be tracked in the scene includes: detecting the obstacles in the scene and classifying the obstacles to determine the types of the obstacles; and taking the obstacles with the classification result of static as the obstacles to be tracked in the scene.

[0010] In some embodiments, the determining of the types of the obstacles includes: in response to the obstacle having only a free space polygon detection result, marking the type of the obstacle as static; or, in response to the obstacle having only a 3D bounding box detection result, marking the type of the obstacle as dynamic.

[0011] In some embodiments, the obstacle detection method further includes: in response to the obstacle intersecting with an obstacle having an ultrasonic detection result, taking the obstacle as the obstacle to be tracked in the scene.

[0012] In some embodiments, the multi-frame matching of the obstacles to be tracked in the path constraint region to identify the target obstacles with continuous time series in the obstacles to be tracked includes: continuously obtaining images of the obstacles to be tracked in multiple frames; performing multi-target tracking and matching through the images of the obstacles to be tracked; extracting the vertices of the 3D detection box and the contour point set of the free space polygon from the successfully matched obstacle pairs, where the point order of the contour point set meets the point order requirements of the boots algorithm library; merging all the contour point sets to generate a convex hull, and taking the convex hull as the geometric representation of the successfully matched obstacle.

[0013] To solve the above technical problems, another technical solution adopted by this application is: providing an obstacle detection method based on path constraints, including: a generation module, configured to generate a trajectory region on the parking trajectory based on the position and direction of the parking trajectory; a stitching module, configured to stitch multiple continuous trajectory regions into the path constraint region of the parking trajectory; a screening module, configured to obtain the obstacles to be tracked in the scene and screen out the obstacles to be tracked outside the path constraint region; and an identification module, configured to perform multi-frame matching on the obstacles to be tracked in the path constraint region to identify the target obstacles with continuous time series in the obstacles to be tracked.

[0014] The present application also provides a computer device, which includes: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor invokes the instructions in the memory to enable the computer device to execute the obstacle detection method as described above.

[0015] The present application also provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, the obstacle detection method as described above is implemented.

[0016] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses an obstacle detection method, device, equipment and medium based on path constraints. Based on the position and direction of the parking trajectory, a trajectory area is generated on the parking trajectory; multiple continuous trajectory areas are spliced into a path constraint area of the parking trajectory; the obstacles to be tracked in the scene are obtained, and the obstacles to be tracked outside the path constraint area are screened out, so as to confine the tracking range of the obstacles in a limited area, reduce data processing redundancy, and improve the efficiency of obstacle tracking and recognition. Multi-frame matching is performed on the obstacles to be tracked in the path constraint area to identify the target obstacles with continuous time series in the obstacles to be tracked, so as to realize effective tracking and recognition of the obstacles, increase safety and reduce the collision risk. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings, where: Figure 1 is a schematic flowchart of an embodiment of the obstacle detection method based on path constraints provided by the present application; Figure 2 is as Figure 1 shown in the schematic diagram of the path constraint area of the parking trajectory in the method; Figure 3 is as Figure 1 shown in the schematic flowchart of Embodiment 10 of the method steps; Figure 4 is as Figure 1 shown in the schematic flowchart of Embodiment 20 of the method steps; Figure 5 is as Figure 1 shown in the schematic flowchart of Embodiment 30 of the method steps; Figure 6 is as Figure 1 shown in the schematic flowchart of Embodiment 40 of the method steps; Figure 7 FIG. Figure 7 is a schematic structural diagram of an embodiment of an obstacle detection device based on path constraints provided by the present application; Figure 8 FIG. is a schematic structural diagram of an embodiment of a computer device in an embodiment of the present application. Detailed implementation manners

[0018] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0019] The terms "first", "second", and "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", and "third" may explicitly or implicitly include at least one of such features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes unlisted steps or units, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0020] Referring to

[0021] FIG. Figure 1 Figure 1 , Figure 1 FIG. Figure 1 is a schematic flowchart of an embodiment of an obstacle detection method based on path constraints. The obstacle detection method includes the following steps: 10: Generate a trajectory area on the parking trajectory based on the position and direction of the parking trajectory.

[0022] During the automatic parking process of an autonomous vehicle, a parking trajectory in the driving direction is generated, which consists of multiple consecutive trajectory segments. Each trajectory segment is defined by a number of discrete points. By calculating the straight-line distance and direction angle between these points, a corresponding rectangular area is generated as the trajectory area corresponding to this segment of the parking trajectory.

[0023] See Figure 2 , Figure 2 is a schematic diagram of the path constraint area of the parking trajectory in the method as Figure 1 shown. During the automatic parking process, the vehicle travels along a predetermined path, and this path is divided into multiple consecutive and shorter driving segments. Each such segment is called a trajectory segment. The trajectory segment is the basic unit that constitutes the entire parking trajectory.

[0024] The trajectory is the entire driving path of the vehicle from the starting parking position to the target parking position, consisting of a series of consecutive trajectory segments. For example, during the parking process, the vehicle will plan a driving route from the current position to the target parking space, where the current position is the starting parking position of the vehicle, and the target parking space is the target parking position of the vehicle. The driving route forms consecutive trajectory segments extending from the current position to the target parking space.

[0025] These trajectory segments are logically continuous and represent the continuous actions during the vehicle's parking process. To describe the trajectory segments mathematically and computationally, each trajectory segment is further divided into a number of points, which are called discrete points. First, there is an overall parking trajectory plan, and then in order to implement and control this trajectory, it is divided into several shorter trajectory segments. Each trajectory segment is a part of the trajectory and together they complete the parking task.

[0026] Discrete points are the specific representation of the trajectory segment. By connecting these points, we can approximately describe the shape and direction of the trajectory segment. Discrete points are obtained by uniformly or on-demand sampling on the trajectory segment through algorithms or planning tools. The positions and quantities of these points depend on the length, shape of the trajectory segment, and the computational accuracy requirements.

[0027] The continuous trajectory area is a series of rectangles. The length of each rectangle is related to the straight-line distance between two points, the width is related to the vehicle width, and a compensation value is added to ensure coverage of the vehicle body and the safety range.

[0028] For any two adjacent discrete points on the trajectory segment, the straight-line distance is the Euclidean distance between them, which can be obtained through simple geometric calculations. This distance reflects the actual distance the vehicle travels between these two points.

[0029] The direction angle refers to the angle between the tangent direction at a certain point on the trajectory segment and a certain reference direction (such as the initial orientation of the vehicle or a certain direction in the global coordinate system). By calculating the vector between two adjacent discrete points and using the angle formula between the vector and the reference direction, the direction angle can be obtained. The direction angle is used to determine the driving direction of the trajectory segment at each point.

[0030] Further, referring to Figure 3 , step 10 includes the following steps: 11: Select the first segment of multiple continuous trajectories in the vehicle driving direction (the first driving segment on the predetermined driving path) as the parking trajectory.

[0031] Select the first segment of multiple continuous trajectories as the starting segment of the parking trajectory. The starting point and the ending point of this segment are defined as point A and point B respectively. By calculating the straight-line distance from point A to point B, the length of the rectangular area is determined; then, in combination with the vehicle width and the compensation value, the width of the rectangular area is determined. In this way, the rectangular area corresponding to the first parking trajectory is generated to ensure covering the safe driving range of the vehicle.

[0032] 12: Construct the trajectory area on the parking trajectory based on the distance and direction angle between the starting discrete point and the terminal discrete point on the parking trajectory.

[0033] By calculating the distance and direction angle between discrete points on each trajectory segment, a corresponding rectangular area is generated to ensure that the length of each rectangle is consistent with the distance between points, and the width is the vehicle width plus the compensation value.

[0034] For each discrete point on the trajectory segment, according to the straight-line distance of this point (as the length of the rectangle), the direction angle, and the vehicle width plus a certain compensation value (as the width of the rectangle), a rectangular area is generated. This rectangular area covers the possible driving range of the vehicle at this point and takes into account a certain safety margin.

[0035] Connect the rectangular areas corresponding to all discrete points on the trajectory segment to form a continuous trajectory area. This trajectory area represents the possible driving range of the vehicle on the entire parking trajectory and is used for subsequent obstacle tracking and obstacle avoidance planning.

[0036] 20: Piece together multiple continuous trajectory areas into the path constraint area of the parking trajectory.

[0037] By piecing together each trajectory area one by one, a complete path constraint area is formed to ensure covering the entire parking trajectory and guarantee the safety and accuracy of the vehicle during parking.

[0038] The path constraint area is used to guide an autonomous vehicle to follow a predetermined path during parking, avoid deviating from the trajectory, and ensure the accuracy and safety of the parking operation. This area not only covers the actual driving trajectory of the vehicle but also provides an additional safety buffer space to effectively prevent potential collision risks.

[0039] Meanwhile, obstacles are screened and tracked through the path constraint area. First, it is determined whether the obstacle falls within the path constraint area. It is judged by whether the polygon formed by the path constraint area intersects with the obstacle polygon or line. If there is an intersection, it is added to the candidate queue.

[0040] Furthermore, referring to Figure 4 , step 20 includes the following steps: 21: Stitch the trajectory areas according to the normal vectors between discrete points and multiple continuous trajectory areas.

[0041] By calculating the normal vectors between discrete points, it is ensured that the boundaries are smooth when stitching rectangular areas, avoiding overlaps or gaps. Each segment of the trajectory area is stitched one by one to form a continuous and seamless path constraint area that covers the entire parking trajectory, ensuring the safety and accuracy of the vehicle during parking.

[0042] The normal vector between discrete points refers to the direction vector perpendicular to the vector connecting two adjacent discrete points. The normal vector is used to determine the perpendicular direction of the tangent direction of the trajectory segment at each discrete point, determine the orientation of the rectangle, and ensure that the rectangle can correctly cover the possible driving range of the vehicle.

[0043] 22: Expand the stitched trajectory area based on the length and width of the vehicle to form a path constraint area.

[0044] When expanding, considering the vehicle length and width, rectangular areas are supplemented at the front, body, and rear of the vehicle respectively to ensure that the path constraint area covers the entire vehicle, prevent missed detection of obstacles, and improve parking safety. Through precise calculation, it is ensured that the expanded area is seamlessly connected to the original trajectory area to form a complete path constraint area, guaranteeing the precision and safety of the autonomous parking operation.

[0045] 30: Obtain the obstacles to be tracked in the scene and filter out the obstacles to be tracked outside the path constraint area.

[0046] By comparing the obstacle positions with the boundaries of the path constraint area, the obstacles not within the path constraint area are accurately filtered out, ensuring that only potential risk points are concerned and improving the pertinence and efficiency of obstacle tracking.

[0047] Select some obstacles in the scene as obstacles to be tracked for tracking and recognition. Use the intersection judgment between the trajectory polygon and the obstacle polygon or line to accurately screen out the obstacles within the path constraint area and add them to the candidate queue to ensure the effectiveness and accuracy of the tracking object, and further optimize the safety performance of the automatic driving parking system.

[0048] The obstacles to be tracked will be continuously tracked and matched by the system, and some of them will be selected for fusion processing. The Kalman filter is used for state prediction and sent to the downstream processing unit.

[0049] Furthermore, refer to Figure 5 , step 30 includes the following steps: 31: Detect the obstacles in the scene and classify the obstacles to determine the type of the obstacles.

[0050] Obtain the obstacle data detected by each sensor, and perform a preliminary screening by calculating the geometric intersection of the trajectory polygon and the obstacle boundary (polygon / line segment). By comparing the vertices of the trajectory polygon and the vertices of the obstacle boundary, determine whether they intersect or overlap.

[0051] Optionally, perform collision detection through simple geometric judgment, the Separating Axis Theorem (SAT), or use the ray casting method to determine whether the obstacle vertices are within the trajectory area.

[0052] The trajectory polygon refers to a polygon area constructed according to the expected driving trajectory of the vehicle in the automatic driving parking system. This polygon area covers the possible driving range of the vehicle and is used to determine which obstacles may collide with the vehicle. When constructing the trajectory polygon, factors such as the vehicle's size, driving direction, and safety margin are usually considered. The trajectory polygon is determined by a series of discrete points and their normal vectors, and these points are connected to form a rectangular or approximately rectangular area to cover the entire driving path of the vehicle.

[0053] The obstacle boundary refers to the outer contour of the geometric shape of the obstacle in two-dimensional or three-dimensional space. In the automatic driving parking system, the obstacles can be static (such as curbs, walls) or dynamic (such as pedestrians, other vehicles). The boundary of the obstacle is usually detected and determined by sensing sensors (such as cameras, radars, lidars, etc.). The boundary of the obstacle can be a polygon (such as a closed shape connected by multiple vertices) or a line segment (such as the edge representing the curb). In order to perform collision detection and trajectory planning, the system needs to accurately obtain the boundary information of the obstacle.

[0054] Geometric intersection refers to the situation where two geometric shapes have a common part in space. In an automatic parking system, geometric intersection is used to determine whether the trajectory polygon overlaps or may come into contact with the obstacle boundary. If the trajectory polygon intersects the obstacle boundary, it means that the vehicle may collide with the obstacle when driving along the current trajectory. Therefore, the system needs to perform intersection detection to ensure the safety of the parking process.

[0055] Specifically, triple-attribute analysis of the physical characteristics (such as size and shape), dynamic characteristics (such as velocity vector and acceleration), and semantic categories (such as vehicles, pedestrians, or non-motor vehicles, etc.) of the obstacle is carried out.

[0056] Specifically, the types of obstacles include, but are not limited to, motor vehicles (static / dynamic), non-motor vehicles, pedestrians, ground obstacles, etc.

[0057] Specifically, in response to the obstacle having only the free space polygon detection result, mark the type of the obstacle as static.

[0058] Or, in response to the obstacle having only the 3D bounding box detection result, mark the type of the obstacle as dynamic.

[0059] In an automatic driving perception system, the obstacle type marking rule can be based on the detection result forms including the free space polygon detection result and the 3D bounding box detection result.

[0060] The free space polygon detection result is used when the obstacle is only recognized as a polygon area by the free space detection algorithm (such as the non-drivable area generated by semantic segmentation). By default, it is marked as a static obstacle. The basis for this determination is that the free space detection mainly targets ground-fixed obstacles (such as curbs and isolation piers).

[0061] The 3D bounding box detection result is used when the obstacle generates a bounding box through 3D object detection (such as the bbox output by LiDAR point cloud or visual fusion). By default, it is marked as a dynamic obstacle. Among them, the basis for judging dynamic characteristics includes geometric attributes such as the implicit size and orientation of the bounding box, and the need to cooperate with temporal tracking to calculate the velocity vector.

[0062] If both detection results exist simultaneously, give priority to using the 3D bounding box data and start multi-modal fusion verification. The free space polygon can be used as a supplementary verification for 3D detection (such as confirming the position of static obstacles).

[0063] Through the comprehensive analysis of the two detection results, the system can more accurately identify the obstacle state, improve the robustness of the perception system, and ensure the accuracy and safety of automatic driving decisions.

[0064] 32: Take the obstacles with the classification result of static as the obstacles to be tracked in the scene.

[0065] Although static obstacles have no tendency to move, it is necessary to continuously track the obstacle avoidance constraints for path planning to prevent sudden changes or the emergence of new obstacles due to environmental changes. By updating the position information of static obstacles in real time, the system can dynamically adjust the driving path to ensure the safe passage of the vehicle. At the same time, continuously monitoring the state of static obstacles helps to identify potential risks and further improve the reliability and stability of the autonomous driving system.

[0066] Optionally, in response to an obstacle intersecting with an obstacle with ultrasonic detection results, the obstacle is regarded as an obstacle to be tracked in the scene.

[0067] For an obstacle intersecting with an ultrasonic obstacle, although it is not directly output to the downstream system, it still needs to be tracked to ensure that there is obstacle data for output when the ultrasonic detection fails.

[0068] When the obstacle polygon detected by other sensors (such as LiDAR or vision) has geometric overlap with the ultrasonic detection area or uses the Separating Axis Theorem (SAT) for fast collision detection, and the calculated intersection area ratio exceeds a preset threshold (such as ≥15%), it is determined that the obstacle intersects with the ultrasonic obstacle. Assign a unique tracking ID to the eligible obstacle, record its attributes, and preferentially use the ultrasonic ranging data to correct the obstacle position.

[0069] 40: Perform multi-frame matching on the obstacles to be tracked in the path constraint area to identify the target obstacles with continuous time series among the obstacles to be tracked.

[0070] When performing multi-frame matching on the obstacles to be tracked within the path constraint area, first establish a spatio-temporal alignment mechanism, uniformly convert the sensor data of consecutive frames to the vehicle coordinate system, and perform convex hull simplification on the free space polygon to reduce the subsequent matching calculation complexity.

[0071] The Free Space Polygon is a geometric model that describes the area where a vehicle can safely drive in an autonomous driving system. By sensors such as lidar, cameras, and 4D millimeter-wave radars, the boundaries of obstacles in the environment are detected, and the obstacle-free area where the vehicle can drive is deduced inversely; a polygon formed by connecting a series of vertices that covers the potential safe path range of the vehicle at the current moment or the prediction period. For example, in a parking scenario, the Free Space Polygon may be jointly determined by the parking space boundary and vehicle dynamics constraints.

[0072] When performing multi-frame matching on obstacles within the path constraint area, the processing of the free space polygon specifically includes mapping the sensor data of consecutive frames into a unified vehicle coordinate system, extracting the obstacle boundaries based on the original sensor data through clustering and edge detection algorithms, defining the area outside the obstacle boundaries as free space, constructing a corresponding polygon representation, and simplifying the complex free space polygon into a convex polygon to reduce the computational complexity of subsequent matching algorithms.

[0073] When performing geometric intersection detection between the simplified convex polygon and the obstacle trajectory, due to the reduction in the number of vertices, the computational efficiency is significantly improved.

[0074] The core algorithm of multi-frame matching uses the Hungarian algorithm for inter-frame obstacle association. The matching criteria include the proportion of the overlapping area of the polygons and motion consistency, and assigns a continuous tracking ID to the successfully matched obstacles and records the sequential trajectory points in time series.

[0075] The sequence of points recording the motion states (such as position, velocity, acceleration, etc.) of the successfully tracked obstacles in consecutive frames in chronological order constitutes a spatio-temporal trajectory chain for recording the time information of the trajectory points.

[0076] By calculating the ratio of the intersection area to the union area of the obstacle polygons (such as free space or trajectory polygons) in adjacent frames, the degree of overlap in the geometric space is measured; combining the difference in the motion vectors between the Kalman filter predicted trajectory and the detection result of the current frame to determine whether they conform to the motion pattern of the same target.

[0077] In this embodiment, the Hungarian algorithm is used to construct a bipartite graph model, with the polygon overlap ratio and motion consistency as the edge weights, to solve the maximum matching with the minimum cost. The target obstacle is successfully matched when the comprehensive matching cost with a certain trajectory is the lowest.

[0078] Optionally, a rule for elimination is set that if the matching fails for three consecutive frames, the object is removed from the tracking list.

[0079] In this way, the system can not only efficiently and dynamically identify obstacles, but also accurately grasp the real-time state of static obstacles, ensuring the accuracy of path planning. At the same time, the introduction of the elimination mechanism effectively reduces mis-tracking and improves the response speed and decision-making reliability of the overall perception system.

[0080] Further, referring to Figure 6 , step 40 further includes the following steps: 41: Continuously obtain images of multiple frames of obstacles to be tracked.

[0081] Through image processing technology, extract the feature information of the obstacles, compare it with the multi-frame data, and include the obstacles with successfully matched feature information in the continuous tracking list to ensure the coherence of tracking.

[0082] 42: Perform multi-object tracking and matching through the images of obstacles to be tracked.

[0083] During the multi-object tracking and matching process, the image feature information and spatio-temporal data are fused to further improve the matching accuracy. The obstacles with successful matching will continue to be retained in the tracking list, and their trajectory information will be updated. For the obstacles that fail to match for multiple consecutive frames, the system will remove them from the tracking list according to the preset rules to avoid the accumulation of false tracking.

[0084] Specifically, extract the 2D / 3D bounding boxes, class probability distributions, and appearance features of the detection targets from consecutive video frames, and maintain the current frame detection set and the existing tracking target set. Construct a cost matrix, which includes two dimensions of spatial distance and semantic distance, form a comprehensive matching cost through weighted fusion, and use the Hungarian algorithm to find the optimal matching solution to ensure the minimum global matching cost.

[0085] The targets with successful matching update their motion trajectories, the detections without matching initialize new trajectories, and the tracking targets with consecutive mismatches are removed from the system.

[0086] Through this series of steps, the system can identify and track obstacles in real time and accurately, providing reliable perception support for autonomous driving.

[0087] 43: Extract the vertices of the 3D detection box and the contour point set of the free space polygon from the successfully matched obstacle pairs, and the point order of the contour point set meets the point order requirements of the boots algorithm library.

[0088] Extract the 8 vertices of the 3D detection box and the contour point set of the free space polygon from the successfully matched obstacle pairs, and merge all the input polygons. The output result is the minimum circumscribed convex polygon, covering all the original obstacle areas.

[0089] The obstacle pairs are the corresponding relationships formed by matching the same obstacle instances detected in adjacent frames or different sensors through a data association algorithm (such as the Hungarian algorithm).

[0090] 44: Merge all the contour point sets to generate a convex hull, and use the convex hull as the geometric representation of the successfully matched obstacles, and use the successfully matched obstacles as the target obstacles.

[0091] Based on the time sequence, regard the matched obstacles as obstacle pairs, and find the contour point sets of the obstacle pairs. The contour point sets include the bounding boxes of 3D detection and the pure polygons of free space detection. Input these point sets into the boost algorithm library, and use the union operation to generate a new convex hull polygon to ensure its larger area and convex hull characteristics.

[0092] The contour point set of an obstacle pair refers to the set of geometric feature points of two consecutive frames (such as the (t - 1)-th frame and the t-th frame) determined to be the same obstacle in multi-frame matching over time, including the bounding box point set detected in 3D and the pure polygon point set detected in free space.

[0093] Use the fused convex hull as the new geometric representation of the obstacle to update the obstacle state. Predict the position of the next frame through Kalman filtering to narrow the search range for the next round of matching.

[0094] Generate a new convex hull by merging the contour point sets of the obstacle pair. Merge the vertices of the 3D bounding boxes and the vertices of the free space polygons of the two frames into a unified set, extract the convex hull vertices from the merged point set, and output a polygon that satisfies convexity. The new convex hull needs to cover all regions of the original point set, so its area is usually larger than that of the single-frame convex hull.

[0095] A convex hull is the smallest convex polygon that contains all points and has no concave regions. The fused convex hull can more robustly represent the spatio-temporal motion trend of the obstacle.

[0096] Input the center point, length, width, and other parameters of the fused convex hull into the Kalman filter to predict the position and speed of the next frame; delimit the candidate matching region of the next frame according to the prediction result.

[0097] Through this obstacle tracking method, the system can effectively identify and process dynamic and static obstacles, improving the robustness of the perception system.

[0098] Through the trajectory tracking scheme, the system accurately identifies obstacles within a limited range, reduces redundant code, and improves safety. The Kalman filter predicts the position, optimizes the matching search, ensures efficient processing of dynamic and static obstacles, and enhances the stability of the perception system. The modular design is convenient for maintenance and can be flexibly adjusted as the perception model is upgraded to ensure continuous optimization of the system.

[0099] Optionally, according to specific services and scenarios, select a region composed of other methods as the tracking region. For example, in a low-speed closed scenario, use fixed grid partitioning + IOU matching to balance accuracy and real-time performance; in a high-speed dynamic scenario, fuse the convex hull of millimeter-wave radar point cloud and the visual ROI region, and cooperate with the adaptive Hungarian algorithm, etc.

[0100] Fixed grid partitioning divides the scene into regular rectangular grid cells and manages the target position through grid indexing.

[0101] IOU (Intersection over Union) matching is used to measure the overlapping ratio of the target regions in adjacent frames.

[0102] The convex hull of millimeter-wave radar point cloud extracts the contour vertices of obstacles from the radar point cloud data and constructs a minimum convex polygon to represent the geometric shape of the target. The millimeter-wave radar provides high-refresh-rate point cloud data to make up for the motion blur defect of vision at high speeds.

[0103] The visual ROI (Region of Interest) area is an area of interest delimited based on the camera detection box or the semantic segmentation result.

[0104] Optionally, the tracking algorithm adopted in this application includes, but is not limited to, introducing a Kalman filter in the above embodiments, and the matching algorithm can also be adjusted according to actual needs. For example, the Hungarian matching is adopted, or the computational complexity is further reduced through greedy matching or the nearest neighbor matching algorithm, etc.

[0105] The method for obstacle detection based on path constraints in the embodiments of the present invention has been described above. Next, the device for obstacle detection based on path constraints in the embodiments of the present invention will be described. Please refer to Figure 7 One embodiment of the device for obstacle detection based on path constraints in the embodiments of the present invention includes: A generation module 401, configured to generate a trajectory area on the parking trajectory based on the position and direction of the parking trajectory.

[0106] A splicing module 402, configured to splice multiple continuous trajectory areas into a path constraint area of the parking trajectory.

[0107] A screening module 403, configured to obtain the obstacles to be tracked in the scene and screen out the obstacles to be tracked outside the path constraint area.

[0108] An identification module 404, configured to perform multi-frame matching on the obstacles to be tracked in the path constraint area to identify the target obstacles with continuous time series among the obstacles to be tracked.

[0109] Above Figure 7 The feature extraction device in the embodiments of the present invention has been described in detail from the perspective of modular functional entities. Next, the computer device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0110] Figure 8It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. The computer device 500 may vary greatly due to different configurations or performances, and may include one or more processors (central processing units, CPUs) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 for storing application programs 533 or data 532 (for example, one or more mass storage devices). Among them, the memory 520 and the storage media 530 may be transient storage or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the computer device 500. Further, the processor 510 may be configured to communicate with the storage media 530 and execute a series of instruction operations in the storage media 530 on the computer device 500.

[0111] The computer device 500 may further include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, MacOS X, Unix, Linux, FreeBSD, and so on. Those skilled in the art can understand that Figure 8 The shown computer device structure does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0112] The present invention also provides a computer device, which includes a memory and a processor. When computer-readable instructions stored in the memory are executed by the processor, the processor is caused to execute the steps of the obstacle detection method based on path constraints in the above embodiments.

[0113] The present invention also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may also be a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the obstacle detection method based on path constraints.

[0114] Different from the prior art, the present application uses a path constraint method for obstacle detection in automatic parking, and specifically provides a method for constructing a path constraint area. The path constraint area is reasonably constructed to screen obstacles, improving the obstacle tracking efficiency on the basis of ensuring safety. By optimizing the construction method of the trajectory area and combining multiple tracking algorithms such as the Kalman filter, efficient obstacle detection and fusion are achieved, further reducing the system calculation amount and improving the overall performance.

[0115] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be described herein again.

[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0117] The above are only the embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent protection scope of the present application.

Claims

1. An obstacle detection method based on path constraints, characterized in that including: generating a trajectory area on the parking trajectory based on the position and direction of the parking trajectory; splicing multiple continuous trajectory areas into a path constraint area of the parking trajectory; acquiring the obstacles to be tracked in the scene and screening out the obstacles to be tracked outside the path constraint area; performing multi-frame matching on the obstacles to be tracked in the path constraint area to identify the target obstacles with continuous time series among the obstacles to be tracked.

2. The obstacle detection method according to claim 1, wherein The generating a trajectory area on the parking trajectory based on the position and direction of the parking trajectory includes: selecting the first segment of multiple continuous trajectories in the vehicle driving direction as the parking trajectory; constructing a trajectory area on the parking trajectory based on the distance and direction angle between the starting discrete point and the terminal discrete point on the parking trajectory.

3. The obstacle detection method according to claim 2, characterized in that The splicing multiple continuous trajectory areas into a path constraint area of the parking trajectory includes: splicing the trajectory areas according to the normal vectors between the discrete points and multiple continuous trajectory areas; expanding the spliced trajectory area based on the length and width of the vehicle to form the path constraint area.

4. The obstacle detection method according to claim 1, wherein The acquiring the obstacles to be tracked in the scene includes: detecting the obstacles in the scene and classifying the obstacles to judge the types of the obstacles; taking the obstacles with the classification result of static as the obstacles to be tracked in the scene.

5. The obstacle detection method according to claim 4, wherein, The judging the types of the obstacles includes: responding to the obstacle having only the free space polygon detection result, marking the type of the obstacle as static; or, responding to the obstacle having only the 3D bounding box detection result, marking the type of the obstacle as dynamic.

6. The obstacle detection method according to claim 4, wherein The obstacle detection method further includes: responding to the obstacle intersecting with an obstacle having an ultrasonic detection result, taking the obstacle as the obstacle to be tracked in the scene.

7. The obstacle detection method according to claim 1, wherein The performing multi-frame matching on the obstacles to be tracked in the path constraint area to identify the target obstacles with continuous time series among the obstacles to be tracked includes: continuously acquiring images of multiple frames of the obstacles to be tracked; performing multi-object tracking and matching through the images of the obstacles to be tracked; extracting the vertices of the 3D detection box and the contour point set of the free space polygon from the successfully matched obstacle pairs, and the point order of the contour point set meets the point order requirements of the boots algorithm library; merging all the contour point sets to generate a convex hull, and taking the convex hull as the geometric representation of the successfully matched obstacle, and taking the successfully matched obstacle as the target obstacle.

8. An obstacle detection device based on path constraints, characterized in that, including: a generating module for generating a trajectory area on the parking trajectory based on the position and direction of the parking trajectory; a splicing module for splicing multiple continuous trajectory areas into a path constraint area of the parking trajectory; a screening module for acquiring the obstacles to be tracked in the scene and screening out the obstacles to be tracked outside the path constraint area; an identifying module for performing multi-frame matching on the obstacles to be tracked in the path constraint area to identify the target obstacles with continuous time series among the obstacles to be tracked.

9. A computer device, characterized in that, The computer device includes: a memory and at least one processor, and instructions are stored in the memory; The at least one processor invokes the instructions in the memory to cause the computer device to execute the obstacle detection method according to any one of claims 1-7.

10. A computer-readable storage medium, on which instructions are stored, characterized in that, When the instructions are executed by the processor, the obstacle detection method according to any one of claims 1-7 is implemented.

Images