Method, device and equipment for generating thermodynamic diagram of drivable area and medium

Abstract

The invention relates to the technical field of automatic parking, and discloses a drivable area thermodynamic diagram generation method and device, equipment and a medium, and the method comprises the steps: obtaining target point cloud data collected by a laser radar; wherein the target point cloud data represents three-dimensional geometric data of the surrounding environment of the vehicle within a preset range; processing the target point cloud data to determine a non-ground point set; performing obstacle analysis and tracking based on the non-ground point set, and determining an obstacle tracking result; and carrying out risk field modeling based on the obstacle tracking result to obtain a target comprehensive risk value, and carrying out fusion processing on the target comprehensive risk value in a time sequence to generate a driving area thermodynamic diagram. According to the technical scheme provided by the invention, the accuracy of dynamic obstacle risk modeling, the adaptive ability of the time sequence fusion method and the balance between the real-time performance and the accuracy during point cloud data processing can be considered.

Description

Technical Field

[0001] This application relates to the field of automatic parking technology, and in particular to a method, apparatus, device and medium for generating a heat map of a drivable area. Background Technology

[0002] Path planning in automated parking systems relies on the accurate generation of drivable areas, which directly impacts safety, obstacle avoidance capabilities, and path smoothness. However, traditional obstacle recognition methods have limitations, especially in low-light or complex environments. Binary representations cannot effectively express the risk level and dynamic changes of obstacles, leading to inaccurate path planning. Although multimodal point cloud fusion and sector-area bird's-eye view methods have improved the accuracy of obstacle recognition and dynamic avoidance, they still have shortcomings in risk assessment and dynamic response, failing to meet the system's requirements for rapid response and accurate decision-making.

[0003] Therefore, how to balance the accuracy of dynamic obstacle risk modeling, the adaptability of temporal fusion methods, and the balance between real-time performance and accuracy in point cloud data processing are technical problems that urgently need to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for generating heat maps of drivable areas, which achieves a technical effect that balances the accuracy of dynamic obstacle risk modeling, the adaptability of temporal fusion methods, and the balance between real-time performance and accuracy in point cloud data processing.

[0005] To achieve the above objectives, the main technical solutions adopted in this application include: In a first aspect, embodiments of this application provide a method for generating a heat map of a drivable area, the method comprising: Acquire target point cloud data collected by lidar; wherein, the target point cloud data represents the three-dimensional geometric data of the vehicle's surrounding environment within a preset range; The target point cloud data is processed to determine the set of non-ground points; Obstacle analysis and tracking are performed based on the set of non-ground points to determine the obstacle tracking results; Based on the obstacle tracking results, a risk field model is performed to obtain the target comprehensive risk value. The target comprehensive risk value is then fused over time to generate a heat map of the drivable area.

[0006] This embodiment provides a method for generating drivable area heatmaps. Using target point cloud data collected by LiDAR, it accurately characterizes the three-dimensional geometric information of the vehicle's surrounding environment, providing high-precision support for subsequent obstacle analysis and path planning. In the processing, a ground segmentation algorithm is first used to extract a set of non-ground points, accurately identifying obstacles and avoiding interference from ground information. Subsequently, multi-frame data and tracking algorithms (such as Kalman filtering) are used to track and predict dynamic obstacles in real time, ensuring accurate capture of environmental changes. Furthermore, by performing risk field modeling and temporal fusion on the obstacle tracking results, an accurate drivable area heatmap can be generated, providing a reliable basis for autonomous driving decisions. The entire process effectively balances real-time performance and accuracy, ensuring the system responds quickly and makes correct decisions in complex and dynamic environments, thereby improving the safety and efficiency of the autonomous driving system.

[0007] In one embodiment, acquiring the target point cloud data collected by the lidar includes: The raw lidar data collected by the lidar is acquired, and the raw lidar data is transformed from the lidar coordinate system to the vehicle coordinate system to obtain the transformed raw lidar data. The converted raw lidar data is cropped to obtain point cloud data within a preset range; The point cloud data within the preset range is divided into cubic grids of a specified length, and the point cloud data in each cubic grid is filtered to obtain the filtered point cloud data. Outliers are removed from the filtered point cloud data to obtain the target point cloud data.

[0008] This embodiment transforms LiDAR data from the LiDAR coordinate system to the vehicle coordinate system, ensuring that the data is consistent with the actual movement of the vehicle, thus providing an accurate foundation for subsequent dynamic obstacle modeling and temporal data fusion. Next, region pruning retains only the effective point cloud data around the vehicle, reducing interference from irrelevant information and improving processing efficiency. Based on this, mesh generation and filtering operations are used to retain the point cloud data with the highest reflectivity, effectively reducing redundant data and providing clearer feature points for obstacle identification. Furthermore, outlier removal eliminates noisy data, further improving the accuracy and reliability of the point cloud data. This series of processing steps not only improves the accuracy of dynamic obstacle risk modeling but also enhances the adaptive capability of temporal data fusion, achieving a good balance between accuracy and real-time performance, thereby providing the autonomous driving system with more efficient and stable environmental perception capabilities.

[0009] In one embodiment, processing the target point cloud data to determine the set of non-ground points includes: dividing the target point cloud data into multiple grids along a specified plane to obtain a set of data points in each grid; Based on the data point set, determine the seed point set in each grid; Based on the seed point set, a plane fitting is performed to determine the ground plane; Determine the vertical distance from the target point cloud data to the ground plane, and determine the set of non-ground points based on the determination result.

[0010] This embodiment divides the target point cloud data into multiple grids, allowing each grid's data point set to be processed independently. This avoids the computational burden caused by excessively large data sets and improves processing accuracy. Next, by determining the seed point set and performing plane fitting, ground features can be accurately identified, and outliers can be effectively eliminated, further improving the accuracy of ground fitting. By calculating the vertical distance from the target point cloud data to the ground plane and combining it with dynamic threshold adjustment, dynamic obstacles can be identified, enhancing adaptability in complex environments, especially demonstrating high accuracy in detecting dynamically changing obstacles. Furthermore, the combination of gridded processing and dynamic thresholding not only ensures processing speed but also improves real-time performance, ensuring accurate responses in rapidly changing environments. Overall, this method balances accuracy, real-time performance, and adaptability, making it suitable for dynamic obstacle risk modeling and point cloud data processing in complex environments.

[0011] In one implementation, the step of performing obstacle analysis and tracking based on the set of non-ground points to determine the obstacle tracking result includes: Cluster analysis is performed on the non-ground points in the set of non-ground points to obtain the set of obstacles; Each obstacle in the current frame's obstacle set is associated with each obstacle in the previous frame's obstacle set. A cost matrix is ​​constructed to determine the matching cost between each pair of obstacles. The matching cost represents the probability that the obstacle in the current frame and the obstacle in the previous frame are the same obstacle. Determine the total matching cost based on all the aforementioned matching costs; The matching scheme with the minimum total matching cost is determined as the obstacle tracking result.

[0012] This embodiment accurately identifies and categorizes obstacles by performing cluster analysis on the set of non-ground points. Based on this, a cost matrix is ​​constructed to calculate the matching cost between obstacles in the current frame and those in the previous frame, achieving accurate obstacle association. Finally, by optimizing the matching cost, the matching scheme with the minimum total cost is determined, resulting in accurate obstacle tracking. This process not only improves the accuracy of dynamic obstacle tracking but also balances timeliness and accuracy through adaptive adjustment of the matching strategy, ensuring efficient and reliable obstacle detection and tracking in complex environments.

[0013] In one implementation, the step of performing cluster analysis on the non-ground points in the set of non-ground points to obtain the obstacle set includes: Determine the neighborhood of each non-ground point in the set of non-ground points; The core point is determined based on the number of non-ground points within the area. Based on the core points, an extended processing is performed to obtain a set of obstacles.

[0014] This embodiment analyzes the neighborhood of each non-ground point in the non-ground point set, accurately identifying and distinguishing obstacles in different areas, reducing noise interference, and laying the foundation for subsequent obstacle detection and tracking. By determining core points based on the number of non-ground points in the neighborhood, the accuracy of obstacle risk modeling can be optimized, ensuring the stability of the selected core points in space, thereby improving the reliability of tracking. Finally, based on the core points, expansion processing is performed to form a complete obstacle set, achieving accurate tracking of dynamic obstacles. The entire process strikes a balance between the adaptive capability of the temporal fusion method and real-time performance and accuracy, effectively improving the accuracy and efficiency of obstacle tracking in dynamic environments.

[0015] In one implementation, the step of modeling the risk field based on the obstacle tracking results to obtain the target's comprehensive risk value includes: Based on the obstacle tracking results, risk field modeling is performed on static obstacles to determine static risk values; Based on the obstacle tracking results, a risk field model is performed on the dynamic obstacles to determine the dynamic risk value; The static risk value and the dynamic risk value are fused together to obtain the target comprehensive risk value.

[0016] In this embodiment, static obstacles are modeled using static risk values, which flexibly adapts to the complexity of different environments and provides high-precision risk modeling. Next, dynamic obstacles are modeled using a temporal fusion method, taking into account their position and velocity, ensuring adaptability to real-time environmental changes, especially when dealing with fast-moving obstacles, providing accurate risk predictions. The fusion of static and dynamic risk values ​​generates a comprehensive risk value, providing accurate decision support for subsequent path planning. While ensuring accuracy, this approach effectively controls computational load, optimizing the balance between real-time performance and accuracy, making it suitable for scenarios requiring real-time perception and rapid response, such as autonomous driving and robot navigation.

[0017] In one implementation, the step of fusing the target comprehensive risk value over a time series to generate a drivable area heatmap includes: A weighted fusion of target comprehensive risk values ​​over a specified number of past frames in the time series is performed to obtain a fused risk value; the fused risk value is then smoothed to obtain a smoothed risk value. Based on a preset risk threshold, the smoothed risk value is probability-mapped to obtain the probability value of the drivable area. The probability values ​​of the drivable areas are mapped to a color space to generate a heatmap of the drivable areas.

[0018] This embodiment smooths out instantaneous fluctuations and extracts stable risk trends by weighted fusion of target comprehensive risk values ​​from time series data, thereby improving robustness to dynamic obstacles. Based on this, the fused risk values ​​are smoothed to eliminate high-frequency noise, ensuring more accurate and reliable risk modeling. Next, based on a preset risk threshold, the smoothed risk values ​​are probability-mapped to obtain drivable area probability values, accurately distinguishing between safe and high-risk areas. Finally, by mapping the drivable area probability values ​​to a color space, a drivable area heatmap is generated, visualizing risk areas and helping the autonomous driving system make fast and accurate path planning and obstacle avoidance decisions. This process effectively balances the accuracy of dynamic obstacle risk modeling, the adaptive capability of time-series fusion, and the real-time performance and accuracy in point cloud data processing.

[0019] Secondly, embodiments of this application provide a drivable area heat map generation device, the device comprising: a point cloud data acquisition unit, used to acquire target point cloud data collected by lidar; wherein, the target point cloud data represents three-dimensional geometric data of the vehicle's surrounding environment within a preset range; The non-ground point determination unit is used to process the target point cloud data and determine the set of non-ground points; The tracking result determination unit is used to perform obstacle analysis and tracking based on the set of non-ground points, and determine the obstacle tracking result; The driving area generation unit is used to perform risk field modeling based on the obstacle tracking results, obtain the target comprehensive risk value, and perform fusion processing on the target comprehensive risk value in a time series to generate a driving area heat map.

[0020] Thirdly, embodiments of this application provide a computer device, including: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes these computer instructions to perform the aforementioned method for generating heatmaps of drivable areas.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the above-described method for generating a heat map of a drivable area. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a method for generating a heat map of a drivable area, as provided in this application embodiment; Figure 2 A flowchart of step S1 provided in the embodiments of this application; Figure 3 A flowchart of step S3 provided in the embodiments of this application; Figure 4 A flowchart of step S5 provided in an embodiment of this application; Figure 5 A flowchart of step S51 provided in an embodiment of this application; Figure 6 A flowchart for obtaining the target comprehensive risk value provided in this application embodiment; Figure 7 A flowchart for generating a heat map of a drivable area provided in an embodiment of this application; Figure 8 A block diagram of a drivable area heat map generation device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Path planning in automated parking systems relies on the precise generation of drivable areas. This area directly impacts vehicle safety and determines obstacle avoidance capabilities and path smoothness. To ensure system performance in dynamic environments, it is essential to accurately describe road boundaries, obstacle locations, and the dynamic characteristics of obstacles, such as their speed and trends.

[0026] However, traditional obstacle recognition solutions face several challenges. Existing vision-based and LiDAR-based solutions have limitations, especially in low-light or complex environments. Furthermore, common binarization methods cannot represent the risk levels of different obstacles or handle dynamic obstacle changes. This leads to inaccurate path planning and even overly conservative or risky decisions. Current improvements, such as multimodal point cloud fusion and sector-area bird's-eye view methods, attempt to address these issues by combining different perceptual data and precisely segmenting regions. These methods improve the recognition of low-lying obstacles and environmental details, and significantly enhance the accuracy of obstacle avoidance for dynamic obstacles. However, these methods still have some limitations in representation, particularly in risk assessment and dynamic response.

[0027] Traditional drivable area maps represent obstacle locations using binarization, which, while simple, fails to effectively express the risk gradient and dynamic changes of obstacles. The edge risk of static obstacles changes abruptly, while the assessment of dynamic obstacles is often delayed. This representation method cannot meet the needs of automated parking systems for rapid response and accurate decision-making.

[0028] Therefore, how to balance the accuracy of dynamic obstacle risk modeling, the adaptability of temporal fusion methods, and the balance between real-time performance and accuracy in point cloud data processing are technical problems that urgently need to be solved.

[0029] To address the aforementioned technical problems, according to an embodiment of this application, a method for generating a drivable area heat map is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] This embodiment provides a method for generating a heat map of a drivable area. Figure 1 A flowchart illustrating a method for generating a heat map of a drivable area, as provided in this application embodiment, is shown below. Figure 1 As shown, the process includes the following steps: Step S1: Acquire target point cloud data collected by lidar; wherein, the target point cloud data represents the three-dimensional geometric data of the vehicle's surrounding environment within a preset range.

[0031] Specifically, target point cloud data acquired through LiDAR can provide high-precision 3D point cloud data of the surrounding environment. The acquired point cloud data includes the spatial location, shape, and relative relationship of the target object to its surroundings. This data can be used to accurately reconstruct a 3D geometric model of the vehicle's surroundings. Within a preset range, the target point cloud data provides a comprehensive, real-time view reflecting the distribution of roads, obstacles, pedestrians, and other target objects, providing crucial perception information for autonomous driving systems. By analyzing and processing this target point cloud data, key tasks such as environmental perception, obstacle detection, and path planning can be achieved, effectively improving vehicle safety and autonomous driving capabilities.

[0032] Step S3: Process the target point cloud data to determine the set of non-ground points.

[0033] Specifically, processing target point cloud data to determine the set of non-ground points is a crucial step in autonomous driving environmental perception. The goal is to accurately separate the non-ground components from a complex point cloud containing both ground and obstacles, providing clean data for subsequent obstacle detection and environmental modeling. This process typically involves ground segmentation algorithms, which distinguish ground points from non-ground points by fitting a ground model and calculating the vertical distance between the point cloud and the model. The set of non-ground points encompasses the 3D positional information of all potential obstacles around the vehicle (such as other vehicles, pedestrians, roadblocks, etc.). It serves as a direct input for subsequent obstacle detection, classification, and tracking algorithms, directly impacting the accuracy and response speed of the autonomous driving system's perception of the surrounding dynamic environment. This is essential for ensuring vehicle safety and achieving efficient path planning.

[0034] Step S5: Perform obstacle analysis and tracking based on the set of non-ground points to determine the obstacle tracking results.

[0035] Specifically, the system identifies individual obstacles from the target point cloud data and continuously tracks their motion. This process begins by analyzing non-ground points using a clustering algorithm, dividing the target point cloud data into independent obstacle clusters. Then, a cost matrix is ​​constructed using multi-frame data to calculate the matching cost between obstacles in the current frame and those in the previous frame. An optimization algorithm finds the matching scheme with the minimum cost, thus achieving continuous obstacle tracking. Finally, methods such as Kalman filtering are used to estimate and update the obstacle's motion state (e.g., position and velocity), yielding the obstacle tracking results. These results include not only the obstacle's current position but also its motion trend, providing real-time and accurate environmental information for the autonomous driving system. This information is crucial for safe and efficient path planning and decision-making.

[0036] Step S7: Based on the obstacle tracking results, perform risk field modeling, obtain the target comprehensive risk value, and perform time series fusion processing on the target comprehensive risk value to generate a drivable area heat map.

[0037] Specifically, firstly, risk field models are constructed for static and dynamic obstacles using obstacle tracking results. The static obstacle risk field calculates its static risk value in a two-dimensional grid using a Gaussian diffusion model, while the dynamic obstacle risk field considers its motion direction and velocity to predict its risk distribution at future time points. Subsequently, the static and dynamic risk values ​​are weighted and fused to obtain a comprehensive risk value. To reduce the impact of instantaneous noise, the comprehensive risk values ​​from multiple frames are further time-weighted and fused to make the risk field more stable. Finally, the fused comprehensive risk value is smoothed using Gaussian filtering to suppress high-frequency noise while preserving the clarity of drivable area boundaries. The smoothed comprehensive risk value is converted into drivable area probability values ​​through mapping and visualized as a heatmap, intuitively showing the drivability probability of different areas. This heatmap provides crucial information for path planning and decision-making for autonomous vehicles, helping them drive safely and efficiently in complex environments.

[0038] This embodiment provides a method for generating drivable area heatmaps. Using target point cloud data collected by LiDAR, it accurately characterizes the three-dimensional geometric information of the vehicle's surrounding environment, providing high-precision support for subsequent obstacle analysis and path planning. In the processing, a ground segmentation algorithm is first used to extract a set of non-ground points, accurately identifying obstacles and avoiding interference from ground information. Subsequently, multi-frame data and tracking algorithms (such as Kalman filtering) are used to track and predict dynamic obstacles in real time, ensuring accurate capture of environmental changes. Furthermore, by performing risk field modeling and temporal fusion on the obstacle tracking results, an accurate drivable area heatmap can be generated, providing a reliable basis for autonomous driving decisions. The entire process effectively balances real-time performance and accuracy, ensuring the system responds quickly and makes correct decisions in complex and dynamic environments, thereby improving the safety and efficiency of the autonomous driving system.

[0039] Figure 2 The flowchart for step S1 provided in the embodiments of this application may include the following steps: Step S11: Obtain the raw lidar data collected by the lidar, and transform the raw lidar data from the lidar coordinate system to the vehicle coordinate system to obtain the transformed raw lidar data.

[0040] Specifically, the raw lidar data P raw It is composed of {(x L ,y L ,z L ,I L The data consists of the spatial coordinates of a point and the reflection intensity, where L represents the lidar coordinate system and I represents the reflection intensity. The goal of converting the original lidar data from the lidar coordinate system to the vehicle coordinate system is to eliminate the influence of the vehicle's motion on the data. This step is typically achieved using rotation and translation matrices, and requires consideration of the sensor's installation position and extrinsic parameter calibration accuracy. Since the conversion from the lidar coordinate system to the vehicle coordinate system is obtained using existing technology, it will not be elaborated upon in this embodiment. The converted original lidar data is in the vehicle coordinate system. This data will serve as the basis for subsequent steps, ensuring consistency with the vehicle's own motion trajectory.

[0041] Step S13: Perform region cropping on the converted original lidar data to obtain point cloud data within a preset range.

[0042] Specifically, by setting the minimum and maximum values ​​of x, y, and z (x min ,x max ,y min ,y max ,h min ,h maxThe point cloud is cropped, retaining only the point cloud data P within this preset range. roi This removes areas other than vehicles or areas that don't need to be considered (e.g., the sky, areas far from vehicles). The cropped point cloud data P roi It only includes points within a specified spatial range, and these points are the basis for subsequent processing.

[0043] Step S15: Divide the point cloud data within the preset range into cubic grids of a specified length, and filter the point cloud data in each cubic grid to obtain the filtered point cloud data.

[0044] Specifically, the cropped point cloud data P roi Divide into sides of length r voxel A cubic grid is used. Points within each cubic grid are selected based on their reflection intensity, retaining the point P with the highest reflection intensity. voxel This approach ensures that only the most representative points are retained from each grid, facilitating more accurate subsequent analysis. The points retained from each cube grid form a new, filtered point cloud data P. voxel .

[0045] Step S17: Remove outliers from the filtered point cloud data to obtain the target point cloud data.

[0046] Specifically, for each point P in the filtered point cloud data i ∈P voxel Calculate the average distance between its K nearest neighbors (KNN search): Among them, P j For P i The j-th neighboring point.

[0047] Calculate the mean distance μ and standard deviation σ for all points: Where N is the total number of filtered point cloud data, and μ±α×σ is used as the threshold (α is a multiple of the standard deviation, such as 1.0 or 2.0): If It is then considered an outlier and removed, resulting in P. clean It refers to the target point cloud data after outlier removal.

[0048] This embodiment transforms LiDAR data from the LiDAR coordinate system to the vehicle coordinate system, ensuring that the data is consistent with the actual movement of the vehicle, thus providing an accurate foundation for subsequent dynamic obstacle modeling and temporal data fusion. Next, region pruning retains only the effective point cloud data around the vehicle, reducing interference from irrelevant information and improving processing efficiency. Based on this, mesh generation and filtering operations are used to retain the point cloud data with the highest reflectivity, effectively reducing redundant data and providing clearer feature points for obstacle identification. Furthermore, outlier removal eliminates noisy data, further improving the accuracy and reliability of the point cloud data. This series of processing steps not only improves the accuracy of dynamic obstacle risk modeling but also enhances the adaptive capability of temporal data fusion, achieving a good balance between accuracy and real-time performance, thereby providing the autonomous driving system with more efficient and stable environmental perception capabilities.

[0049] Figure 3 The flowchart for step S3 provided in the embodiments of this application may include the following steps: Step S31: Divide the target point cloud data into multiple grids along the specified plane to obtain the data point set in each grid.

[0050] Meshing simplifies point cloud data processing. Point cloud data is three-dimensional, but dividing it along a specified plane, such as the XY plane, can effectively reduce computational complexity, especially for large-scale datasets. This is achieved by calculating the mesh size Δ. x and Δ y Then, the data is distributed to different small areas according to the grid size.

[0051] Specifically, the filtered point cloud data P voxel The XY plane is divided into an n×m grid, with each grid cell having the following size: Therefore, the point set of the (a,b)th grid is: P a，b ={p i |x min +aΔx≤x i ≤x min +(a+1)Δx,y min +bΔy≤y i ≤y min +(b+1)Δy} Step S33: Based on the data point set, determine the seed point set in each grid.

[0052] Specifically, for each grid P a,b Sort the points in ascending order of z, and take the top k% as the seed point set S. a,bAnd remove highly outliers (z>μ) z +2σ z , where μ z ,σ z S represents the mean and standard deviation of the seed point height. a,b ={p i ∈P a,b │z i ≤z quantile (k%)}.

[0053] Step S35: Perform plane fitting based on the seed point set to determine the ground plane.

[0054] Specifically, by calculating the covariance matrix of the seed point set, the spatial distribution characteristics of the point cloud can be obtained. Covariance matrix C a,b This represents the distribution pattern of the point cloud, containing information about the correlation of the point cloud in three dimensions. The seed point set S is calculated. a,b Covariance matrix: in, This represents the average position of the seed point in the (a,b)th grid.

[0055] Next, by performing eigenvalue decomposition on the covariance matrix, three eigenvalues ​​and corresponding eigenvectors can be obtained. The eigenvalues ​​reflect the magnitude of the variance of the point cloud in different directions, and the eigenvector corresponding to the smallest eigenvalue represents the direction in which the point cloud distribution is minimized. This is usually the ground normal vector n. a,b For C a,b Perform eigenvalue decomposition: C a,b =V∧V T , ∧=diag(λ1,λ2,λ3),λ1≤λ2≤λ3 Take the eigenvector V1 corresponding to the smallest eigenvalue λ1 as the ground normal vector n. a,b .

[0056] The equation of the plane is: Step S37: Determine the vertical distance from the target point cloud data to the ground plane, and determine the set of non-ground points based on the determination result.

[0057] Specifically, the vertical distance from the target point cloud data to the ground plane is used to determine whether the target point cloud data belongs to the ground. The purpose of this step is to separate ground points from non-ground points (i.e., obstacle points) from the point cloud, thereby achieving dynamic obstacle detection. This is achieved by calculating p for each target point cloud data. i Vertical distance d to the fitted ground plane iThis can determine whether a point is located on the ground. If the distance from the point to the ground is less than a certain threshold, the point is considered to be on the ground. Calculate the target point cloud data p. i to the ground plane π a,b Vertical distance: Calculate the dynamic threshold: τ a,b =μ d +ασ d Where, μ d and σ d These represent the mean and standard deviation of the distance between seed points in the current grid, respectively, with α being an adjustment factor.

[0058] Based on the current distribution of points within the grid (mean μ) d and standard deviation σ d The threshold is adjusted to adapt to changes in different environments. The adjustment factor α controls the sensitivity of the threshold and usually needs to be adjusted under different environmental conditions.

[0059] The judgment rule is as follows: When the target point cloud data p i to the ground plane π a,b vertical distance d i Less than the dynamic threshold τ a,b If the target point cloud data is considered to belong to the ground, it is otherwise considered a non-ground point (e.g., an obstacle), thus obtaining the non-ground point P. obstacle .

[0060] This embodiment divides the target point cloud data into multiple grids, allowing each grid's data point set to be processed independently. This avoids the computational burden caused by excessively large data sets and improves processing accuracy. Next, by determining the seed point set and performing plane fitting, ground features can be accurately identified, and outliers can be effectively eliminated, further improving the accuracy of ground fitting. By calculating the vertical distance from the target point cloud data to the ground plane and combining it with dynamic threshold adjustment, dynamic obstacles can be identified, enhancing adaptability in complex environments, especially demonstrating high accuracy in detecting dynamically changing obstacles. Furthermore, the combination of gridded processing and dynamic thresholding not only ensures processing speed but also improves real-time performance, ensuring accurate responses in rapidly changing environments. Overall, this method balances accuracy, real-time performance, and adaptability, making it suitable for dynamic obstacle risk modeling and point cloud data processing in complex environments.

[0061] Figure 4 The flowchart for step S5 provided in the embodiments of this application may include the following steps: Step S51: Perform cluster analysis on the non-ground points in the non-ground point set to obtain the obstacle set.

[0062] Specifically, cluster analysis is performed on the non-ground points in the set of non-ground points to extract specific obstacle objects (such as vehicles, pedestrians, etc.). The neighborhood radius r and the minimum number of points min are determined. Pts The neighborhood radius *r* represents the maximum distance at which a point is considered a "neighbor". Only points with a distance less than or equal to *r* can be considered a point's neighborhood. The minimum number of points is *min*. Pts Used to define whether a cluster (i.e., obstacles) is dense enough. If a point has at least min Pts If there are 1 point, then this point is marked as the core point. For non-ground point P... obstacle Each point p in i Calculate its domain: N r (p i )={p i ||||p i -p j ||2≤r} For non-ground point P obstacle Each point p in i Find all points p that are within r of it. j These points form p i neighborhood N r (p i ).

[0063] If the number of points in the neighborhood meets the density condition, mark it as a core point and expand the cluster: |N r (p i )|≥min pts Where, if p i The core point is the starting point. All points within its neighborhood are added to the current cluster. If a point in the neighborhood is also a core point, then the neighborhood of those points is expanded to form a new cluster C. k Ultimately, clustering yields multiple independent obstacles (v). x ,v y Each obstacle (v) x ,v y ) corresponds to a cluster C k These clusters are considered valid obstacle detection results.

[0064] Step S53: Associate each obstacle in the current frame obstacle set with each obstacle in the previous frame obstacle set, and determine the matching cost between each pair of obstacles by constructing a cost matrix; wherein, the matching cost represents the probability that the obstacle in the current frame and the obstacle in the previous frame are the same obstacle.

[0065] Specifically, the matching cost c kl It is calculated using the following cost matrix: Intersection over Union (IoU) is a criterion for calculating the similarity between two obstacle bounding boxes. It is calculated by comparing obstacles in the current frame. Obstacles from the previous frame The bounding boxes are used to obtain their geometric similarity. The closer the IoU is to 1, the more similar the two obstacles are. Obstacles in the current frame The bounding box, Obstacles from the previous frame The bounding box; Obstacles from the previous frame Prediction speed, Obstacles in the current frame The observation rate; weight β1+β2=1; σ v The standard deviation of the speed difference is used to control the weights for speed similarity.

[0066] In a preferred embodiment, the Kalman filter algorithm is used for motion state estimation, and the state vector is: x k =[x,y,z,v x ,v y ,v z ] T Prediction steps: in, To predict the state, i.e., to predict the current state based on the state at the previous time step without any observed data; I3 is a 3×3 identity matrix; Δt is the time step, representing the time interval between two frames; the state transition matrix F describes the state changes from the previous time step l-1 to the current time step l. The position x is updated according to the velocity v and the time step Δt, while the velocity remains unchanged (assuming uniform motion).

[0067] Covariance prediction: P l - =FP l-1 F T +Q Among them, Pl - Let be the prediction covariance matrix, representing the uncertainty in the prediction velocity. Let Q be the process noise covariance matrix, representing the uncertainty in the system model. in, Variance is the position noise variance, representing the uncertainty of the position measurement. The variance of the velocity noise represents the uncertainty in velocity measurement.

[0068] Therefore, when calculating the matching cost, the prediction speed of obstacles in the previous frame is considered. It is obtained through the prediction step of Kalman filtering.

[0069] Update steps: H = [I3 0] Among them, K k is the Kalman gain, used to balance the reliability of predicted and observed values; the observation matrix H is used to extract position information from the state vector, ignoring velocity information; R is the observation noise covariance matrix, representing the uncertainty of the observation data; The variance of the observed noise.

[0070] Among them, z k This represents the current observation value (location information). To predict the location. The observation residual is the difference between the observed value and the predicted value.

[0071] Covariance update: Among them, P k Let be the updated covariance matrix, representing the uncertainty of the updated state.

[0072] Therefore, the observation speed of obstacles in the current frame It is obtained through the update step of Kalman filtering.

[0073] Step S55: Determine the total matching cost based on all matching costs.

[0074] Specifically, determine the total matching cost: Σ k,l c kl · Xk 1 Where, x kl∈{0,1} is the association variable, a binary variable representing whether to associate the k-th obstacle in the current frame with the l-th obstacle in the previous frame. kl =1 indicates association; x kl =0 indicates no association.

[0075] Step S57: The matching scheme with the minimum total matching cost is determined as the obstacle tracking result.

[0076] Specifically, the goal is to minimize the total matching cost: minΣ k,1 c kl ·x kl The constraints are as follows: Each obstacle in the current frame can be associated with at most one obstacle from the previous frame: Each obstacle from the previous frame can be associated with at most one obstacle in the current frame: The Kalman filter algorithm is used for motion state estimation. The state vector is: x k =[x,y,z,v x v y ,v z ] T Where x, y, z are the position coordinates of the obstacle in three-dimensional space; v x ,v y ,v z Let be the velocity of the obstacle in three directions.

[0077] This embodiment accurately identifies and categorizes obstacles by performing cluster analysis on the set of non-ground points. Based on this, a cost matrix is ​​constructed to calculate the matching cost between obstacles in the current frame and those in the previous frame, achieving accurate obstacle association. Finally, by optimizing the matching cost, the matching scheme with the minimum total cost is determined, resulting in accurate obstacle tracking. This process not only improves the accuracy of dynamic obstacle tracking but also balances timeliness and accuracy through adaptive adjustment of the matching strategy, ensuring efficient and reliable obstacle detection and tracking in complex environments.

[0078] Figure 5 The flowchart for step S51 provided in the embodiments of this application may include the following steps: Step S511: Determine the neighborhood of each non-ground point in the set of non-ground points.

[0079] Specifically, cluster analysis is performed on the non-ground points in the set of non-ground points to extract specific obstacle objects (such as vehicles, pedestrians, etc.). The neighborhood radius r and the minimum number of points min are determined. Pts The neighborhood radius *r* represents the maximum distance at which a point is considered a "neighbor". Only points with a distance less than or equal to *r* can be considered a point's neighborhood. The minimum number of points is *min*. Pts Used to define whether a cluster (i.e., obstacles) is dense enough. If a point has at least min Pts If there are 1 point, then this point is marked as the core point. For non-ground point P... obstacle Each point p in i Calculate its domain: N r (p i )={p j |||p i -p j ||2≤r For non-ground point P obstacle Each point p in i Find all points p that are within r of it. j These points form p i neighborhood N r (p i ).

[0080] Step S513: Determine the core points based on the number of non-ground points in the domain.

[0081] Specifically, if the number of points in the neighborhood satisfies the density condition, it is marked as a core point and the cluster is expanded: |N r (p i )|≥min pts Where, if p i It is the core point. Starting from the core point, all points in the neighborhood will be added to the current cluster.

[0082] Step S515: Perform extended processing based on the core points to obtain the obstacle set.

[0083] Specifically, if a point within the neighborhood is also a core point, then the neighborhood of these points will be expanded to form a new cluster C. k Ultimately, clustering yields multiple independent obstacles (v). x ,v y Each obstacle (v) x ,v y ) corresponds to a cluster C k These clusters are considered valid obstacle detection results.

[0084] This embodiment analyzes the neighborhood of each non-ground point in the non-ground point set, accurately identifying and distinguishing obstacles in different areas, reducing noise interference, and laying the foundation for subsequent obstacle detection and tracking. By determining core points based on the number of non-ground points in the neighborhood, the accuracy of obstacle risk modeling can be optimized, ensuring the stability of the selected core points in space, thereby improving the reliability of tracking. Finally, based on the core points, expansion processing is performed to form a complete obstacle set, achieving accurate tracking of dynamic obstacles. The entire process strikes a balance between the adaptive capability of the temporal fusion method and real-time performance and accuracy, effectively improving the accuracy and efficiency of obstacle tracking in dynamic environments.

[0085] Figure 6 The flowchart for obtaining the target comprehensive risk value provided in the embodiments of this application may include the following steps: Step S711, based on the obstacle tracking results, perform risk field modeling on the static obstacle to determine the static risk value.

[0086] Specifically, the target point cloud data with obstacle attributes is converted into a two-dimensional mesh using obstacle tracking results. This is the step of mapping three-dimensional obstacle data onto a plane, setting the BEV range [x]. min ,x max The resolution Δx = Δy = 0.1m is defined. The range defines the spatial boundary of the 2D grid, and the resolution controls the grid's precision (each grid cell is 0.1m × 0.1m in size). The 2D grid coordinates are represented using (e,f). The (x,y,z) coordinates of the target point cloud data are obtained through the projection formula: This formula maps each target point cloud data to grid coordinates (e, f).

[0087] Static obstacles typically refer to obstacles whose location does not change over time, such as walls or buildings. Risk field modeling for static obstacles involves calculating a Gaussian risk distribution to evaluate the static risk value at each grid location. Where Ω represents the neighborhood range, indicating the region related to the current grid; M static (u,v) represents the risk weight of static obstacles; σ s This is the static risk diffusion coefficient, used to control the spatial extent of risk expansion.

[0088] By calculating the Gaussian risk value of each grid, the risk field of a static obstacle can be obtained, reflecting the impact of the static obstacle on the surrounding area.

[0089] Step S713: Based on the obstacle tracking results, perform risk field modeling on the dynamic obstacles and determine the dynamic risk value.

[0090] Specifically, dynamic obstacles typically refer to obstacles that move over time, such as pedestrians and vehicles. Risk field modeling for dynamic obstacles takes into account the movement of the obstacles. Among them, M dynamic (u,v) represents the risk weight of the dynamic obstacle; Δt is the prediction time step, used to predict the position of the dynamic obstacle at a future time; σ d This represents the dynamic risk diffusion coefficient.

[0091] The dynamic risk value of each grid is assessed by calculating the trajectory of dynamic obstacles and their impact on the surrounding grid.

[0092] Step S715: The static risk value and the dynamic risk value are merged to obtain the target comprehensive risk value.

[0093] Specifically, after obtaining the static and dynamic risk values, they need to be fused to obtain a comprehensive risk value. This process considers the relative importance of static and dynamic obstacles and fuses their risk fields through a weighted approach: R total (e,f)=γR static (e,f)+(1-γ)R dynamic (e,f) Here, γ represents the weight of static risk, indicating the degree of influence of static obstacles on the overall risk value. 1-γ represents the weight of dynamic risk. Through this weighted summation, combining the effects of static and dynamic obstacles, the overall risk value for each grid is obtained.

[0094] Finally, the composite risk value is standardized through a normalization step: The purpose of normalization is to limit the overall risk value to the range [0,1], so that the maximum risk value is 1, which makes it easier to assess and compare the risk at different locations.

[0095] In this embodiment, static obstacles, through static risk values, can flexibly adapt to the complexity of different environments, providing high-precision risk modeling. Next, dynamic obstacles, considering their position and velocity, are modeled using a temporal fusion method to ensure adaptability to real-time environmental changes, especially when dealing with fast-moving obstacles, providing accurate risk prediction. The fusion of static and dynamic risk values ​​generates a comprehensive risk value, providing accurate decision support for subsequent path planning. While ensuring accuracy, it effectively controls computational load, optimizing the balance between real-time performance and accuracy, making it suitable for scenarios requiring real-time perception and rapid response, such as autonomous driving and robot navigation.

[0096] Figure 7 The flowchart for generating a heat map of a drivable area provided in this application embodiment may include the following steps: Step S731: The target comprehensive risk value is obtained by weighted fusion of the target comprehensive risk values ​​of a specified number of past frames in the time series.

[0097] Specifically, in dynamic environments, the overall target risk value changes over time. Therefore, it is necessary to perform weighted fusion of the overall target risk values ​​over a period of time to improve the stability of the overall data and reduce the impact of instantaneous noise. A weight is assigned to each time frame, and the weight gradually decreases over time. Specifically, the current frame (g=0) has the highest weight, while earlier frames (with larger g values) have lower weights. The weights are calculated using an exponential decay function, and the decay coefficient λ controls the rate of decay.

[0098] ω g =e -λg Where, ω k λ represents the weight of the k-th frame, with the most recent frame having the highest weight and earlier frames having lower weights; the attenuation coefficient λ controls the rate of time decay; g is the frame index, g = 0, 1, ..., G-1.

[0099] By normalizing all weights, the weighted sum of all frames is equal to 1, ensuring that the weighted fusion process will not be biased due to different weight sums.

[0100] Based on normalized weights, the comprehensive risk values ​​of the target from the past g frames are weighted and fused to obtain a fused risk value. This process can smooth out local sudden changes, making the response to time series data more stable.

[0101] Among them, R g (x,y) represents the target comprehensive risk value of the g-th frame; The risk value after fusion incorporates information from the past k frames.

[0102] Step S733: Smooth the fused risk values ​​to obtain smoothed risk values.

[0103] Specifically, smoothing is used to further remove high-frequency noise caused by sensor noise or transient errors. Gaussian filtering is applied to smooth the weighted fused risk values. Gaussian filtering can effectively suppress small-scale noise while preserving large-scale structural features, such as road boundaries.

[0104] Here, σ controls the width of the Gaussian kernel, determining the smoothing intensity; r is the filtering radius, defining the local range for smoothing in space. G(u,v) represents the contribution weight of the neighborhood point (u,v) to the center point (x,y); u represents the lateral offset (along the x-axis), and v represents the longitudinal offset (along the y-axis).

[0105] Step S735: Based on a preset risk threshold, perform probability mapping on the smoothed risk value to obtain the probability value of the drivable area.

[0106] Specifically, the smoothed risk field value is mapped to a drivable area probability value using the Sigmoid function. This mapping transforms continuous risk values ​​into easily understandable probability values, which is helpful for subsequent path planning.

[0107] Here, 'a' controls the slope of the function, and 'b' is the risk threshold, which determines the mapping offset between the risk value and the probability. By adjusting 'a' and 'b', the relationship between the risk value and the probability of a drivable area can be flexibly changed.

[0108] Step S737: Map the drivable area probability values ​​to a color space to generate a drivable area heatmap.

[0109] Specifically, the probability value P of the drivable area drivable (x, y) is mapped to a color space to generate a heatmap of drivable areas. Typically, color gradients are used to represent risk levels. For example, high-risk areas are represented by red, and low-risk areas by green. Specifically, the probability value P of a drivable area... drivable (x,y) will be mapped to RGB values ​​through a color gradient bar, R=255×(1-P) drivable (x,y)), G=255×P drivable (x,y), B=0. Thus, when P drivable When (x,y) is close to 0, the point will be displayed in red (R=255,G=0,B=0); while when P drivable When (x,y) is close to 1, the point will be displayed in green (R=0,G=255,B=0).

[0110] Preferably, the generated drivable area heatmap can be overlaid on the original point cloud data or bird's-eye view (BEV map), making the visualization of the drivable area more intuitive. This display method can help autonomous driving systems or robots clearly identify drivable areas.

[0111] This embodiment smooths out instantaneous fluctuations and extracts stable risk trends by weighted fusion of target comprehensive risk values ​​from time series data, thereby improving robustness to dynamic obstacles. Based on this, the fused risk values ​​are smoothed to eliminate high-frequency noise, ensuring more accurate and reliable risk modeling. Next, based on a preset risk threshold, the smoothed risk values ​​are probability-mapped to obtain drivable area probability values, accurately distinguishing between safe and high-risk areas. Finally, by mapping the drivable area probability values ​​to a color space, a drivable area heatmap is generated, visualizing risk areas and helping the autonomous driving system make fast and accurate path planning and obstacle avoidance decisions. This process effectively balances the accuracy of dynamic obstacle risk modeling, the adaptive capability of time-series fusion, and the real-time performance and accuracy in point cloud data processing.

[0112] Accordingly, please refer to Figure 8 A block diagram of a drivable area heat map generation device provided in this application embodiment, the terminal including: The point cloud data acquisition unit 101 is used to acquire target point cloud data collected by the lidar; wherein, the target point cloud data represents the three-dimensional geometric data of the vehicle's surrounding environment within a preset range. The non-ground point determination unit 103 is used to process the target point cloud data and determine the set of non-ground points; The tracking result determination unit 105 is used to perform obstacle analysis and tracking based on a set of non-ground points, and to determine the obstacle tracking result; The driving area generation unit 107 is used to model the risk field based on the obstacle tracking results, obtain the target comprehensive risk value, and perform fusion processing on the target comprehensive risk value in the time series to generate a drivable area heat map.

[0113] In some optional implementations, the point cloud data acquisition unit 101 includes: Obtain the raw lidar data collected by the lidar, transform the raw lidar data from the lidar coordinate system to the vehicle coordinate system, and obtain the transformed raw lidar data; The converted raw lidar data is cropped to obtain point cloud data within a preset range; The point cloud data within a preset range is divided into cubic grids of a specified length. The point cloud data in each cubic grid is then filtered to obtain the filtered point cloud data. Outliers are removed from the filtered point cloud data to obtain the target point cloud data.

[0114] In some optional implementations, the non-ground point determination unit 103 includes: The target point cloud data is divided into multiple grids along a specified plane, and the data point set in each grid is obtained; Based on the data point set, determine the seed point set in each grid; Plane fitting is performed based on the seed point set to determine the ground plane; Determine the vertical distance from the target point cloud data to the ground plane, and determine the set of non-ground points based on the determination result.

[0115] In some optional implementations, the tracking result determination unit 105 includes: Cluster analysis is performed on the non-ground points in the non-ground point set to obtain the obstacle set; Each obstacle in the current frame's obstacle set is associated with each obstacle in the previous frame's obstacle set. By constructing a cost matrix, the matching cost between each pair of obstacles is determined. The matching cost represents the probability that the obstacle in the current frame and the obstacle in the previous frame are the same obstacle. Determine the total matching cost based on all matching costs; The matching scheme with the minimum total matching cost is determined as the obstacle tracking result.

[0116] In some alternative implementations, cluster analysis is performed on the non-ground points in the non-ground point set to obtain an obstacle set, including: Determine the neighborhood of each non-ground point in the set of non-ground points; Determine the core points based on the number of non-ground points within the domain; Based on the core points, an extended processing is performed to obtain a set of obstacles.

[0117] In some optional implementations, the driving area generation unit 107 includes: Based on the obstacle tracking results, a risk field model is performed on static obstacles to determine the static risk value; Based on the obstacle tracking results, a risk field model is performed on the dynamic obstacles to determine the dynamic risk value; By integrating static and dynamic risk values, a comprehensive target risk value is obtained.

[0118] In some optional implementations, the driving area generation unit 107 includes: The target comprehensive risk value is obtained by weighted fusion of the target comprehensive risk values ​​over a specified number of past frames in the time series; the fused risk value is then smoothed to obtain the smoothed risk value. Based on a preset risk threshold, the smoothed risk value is probability-mapped to obtain the probability value of the drivable area. The probability values ​​of drivable areas are mapped to a color space to generate a heatmap of drivable areas.

[0119] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0120] In this embodiment, a drivable area heat map generation device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0121] Please see Figure 9 , Figure 9 This application provides a schematic diagram of the structure of a computer device, as shown in the embodiment of the present application. Figure 9 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9 Take a processor 10 as an example.

[0122] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0123] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0124] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0125] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0126] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0127] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0128] The apparatus and units described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0129] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0130] Those skilled in the art will understand that the embodiments of this application can be provided as methods or apparatus. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, and devices according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0136] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0137] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for generating a heat map of a drivable area, characterized in that, The method includes: Acquire target point cloud data collected by lidar; wherein, the target point cloud data represents the three-dimensional geometric data of the vehicle's surrounding environment within a preset range; The target point cloud data is processed to determine the set of non-ground points; Obstacle analysis and tracking are performed based on the set of non-ground points to determine the obstacle tracking results; Based on the obstacle tracking results, a risk field model is performed to obtain the target comprehensive risk value. The target comprehensive risk value is then fused over time to generate a heat map of the drivable area.

2. The method according to claim 1, characterized in that, The acquisition of target point cloud data collected by lidar includes: The raw lidar data collected by the lidar is acquired, and the raw lidar data is transformed from the lidar coordinate system to the vehicle coordinate system to obtain the transformed raw lidar data. The converted raw lidar data is cropped to obtain point cloud data within a preset range; The point cloud data within the preset range is divided into cubic grids of a specified length, and the point cloud data in each cubic grid is filtered to obtain the filtered point cloud data. Outliers are removed from the filtered point cloud data to obtain the target point cloud data.

3. The method according to claim 1, characterized in that, The process of processing the target point cloud data to determine the set of non-ground points includes: The target point cloud data is divided into multiple grids along a specified plane to obtain the data point set in each grid; Based on the data point set, determine the seed point set in each grid; Based on the seed point set, a plane fitting is performed to determine the ground plane; Determine the vertical distance from the target point cloud data to the ground plane, and determine the set of non-ground points based on the determination result.

4. The method according to claim 1, characterized in that, The obstacle analysis and tracking based on the set of non-ground points, and the determination of the obstacle tracking results, include: Cluster analysis is performed on the non-ground points in the set of non-ground points to obtain the set of obstacles; Each obstacle in the current frame's obstacle set is associated with each obstacle in the previous frame's obstacle set. A cost matrix is ​​constructed to determine the matching cost between each pair of obstacles. The matching cost represents the probability that the obstacle in the current frame and the obstacle in the previous frame are the same obstacle. Determine the total matching cost based on all the aforementioned matching costs; The matching scheme with the minimum total matching cost is determined as the obstacle tracking result.

5. The method according to claim 4, characterized in that, The clustering analysis of the non-ground points in the set of non-ground points yields a set of obstacles, including: Determine the neighborhood of each non-ground point in the set of non-ground points; The core point is determined based on the number of non-ground points within the area. Based on the core points, an extended processing is performed to obtain a set of obstacles.

6. The method according to claim 1, characterized in that, The step of modeling the risk field based on the obstacle tracking results to obtain the target's comprehensive risk value includes: Based on the obstacle tracking results, risk field modeling is performed on static obstacles to determine static risk values; Based on the obstacle tracking results, a risk field model is performed on the dynamic obstacles to determine the dynamic risk value; The static risk value and the dynamic risk value are fused together to obtain the target comprehensive risk value.

7. The method according to claim 1, characterized in that, The process of fusing the target comprehensive risk value over time to generate a drivable area heat map includes: A weighted fusion of target comprehensive risk values ​​over a specified number of past frames in the time series is performed to obtain a fused risk value; the fused risk value is then smoothed to obtain a smoothed risk value. Based on a preset risk threshold, the smoothed risk value is probability-mapped to obtain the probability value of the drivable area. The probability values ​​of the drivable areas are mapped to a color space to generate a heatmap of the drivable areas.

8. A device for generating a heat map of a drivable area, characterized in that, The device includes: A point cloud data acquisition unit is used to acquire target point cloud data collected by lidar; wherein, the target point cloud data represents the three-dimensional geometric data of the vehicle's surrounding environment within a preset range; The non-ground point determination unit is used to process the target point cloud data and determine the set of non-ground points; The tracking result determination unit is used to perform obstacle analysis and tracking based on the set of non-ground points, and to determine the obstacle tracking result; The driving area generation unit is used to perform risk field modeling based on the obstacle tracking results, obtain the target comprehensive risk value, and perform fusion processing on the target comprehensive risk value in a time series to generate a driving area heat map.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the method for generating a drivable area heat map according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method for generating a heat map of a drivable area as described in any one of claims 1 to 7.

Images