A method and related equipment for filtering point cloud data occupying grid height dimension

Abstract

The present application provides a method and related equipment for filtering the height dimension of a grid occupied by point cloud data. The method includes: projecting the point cloud data onto a two-dimensional grid map to obtain two-dimensional feature information of each observation grid in the two-dimensional grid map; clustering the static point cloud within each observation grid according to a set of height values ​​to obtain at least one cluster target, and obtaining the height dimension feature information of each cluster target; constructing a state feature value set of the current frame prediction grid based on the state feature information of the previous frame; performing a two-level association between the prediction grid and the observation grid, and updating or extrapolating the state feature value set based on the association result; if the observation grid is not associated with any prediction grid and meets the regeneration condition, the new grid is regenerated and its state information is initialized; and finally, invalid prediction grids are filtered out. The present application can accurately express multiple height-dimensional cluster targets within the grid, effectively improving the obstacle structure modeling capability in complex multi-height scenarios.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a method for filtering the height dimension of a grid occupied by point cloud data and related equipment. Background Art

[0002] With the development of intelligent driving systems, environmental perception technology plays a key role in autonomous vehicles. LiDAR and millimeter-wave radar are widely used in obstacle detection and map construction. LiDAR uses optical ranging principles and has high ranging accuracy, but its speed measurement capability is limited, and its performance degrades significantly in adverse weather conditions such as rain and fog. Furthermore, LiDAR's ability to detect distant targets is insufficient. In contrast, millimeter-wave radar uses high-frequency electromagnetic waves to measure speed and distance, has strong anti-interference capabilities, can operate stably in complex weather environments, and has a detection range of over 200 meters, making it more suitable for building large-scale environmental perception models.

[0003] Existing technologies, such as lidar or millimeter-wave radar, can output 3D point cloud data, which can theoretically be used to construct a 3D occupancy grid map, providing a more complete representation of the contours and edge information of ground obstacles (such as curbs, guardrails, and bushes) and overhead obstacles (such as trees, buildings, and gantries) in the environment. However, due to limitations in the processing power of onboard computing platforms, practical applications often project the point cloud onto a 2D plane, extracting only the maximum, minimum, or average height value of each 2D grid point cloud to form a 2D occupancy grid map.

[0004] However, while this simplified processing method retains some height information, it severely compresses the information dimension in the vertical direction, resulting in an inability to accurately express the obstacle structure in the height dimension, which has significant flaws. Specifically, the two-dimensional occupancy grid can only reflect whether a certain area is occupied by an obstacle, but cannot accurately determine whether the occupation occurs on the ground, in the air, or both. In other words, when multiple targets of different heights appear at the same horizontal position at the same time, due to the lack of multi-level occupancy information in the height dimension, the system cannot determine whether it affects the vehicle's passage, resulting in erroneous judgment of passability or error in obstacle recognition. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present application provides a point cloud data occupancy grid height dimension filtering method and related equipment, which is at least used to solve the problem in the existing technology that the two-dimensional occupancy grid cannot accurately express the information of multiple height obstacles in the height dimension, thereby resulting in a decrease in the accuracy of the system in obstacle type identification, spatial structure understanding and passability judgment.

[0006] In order to achieve the above objectives and other advantages, some embodiments of the present application provide the following aspects:

[0007] In a first aspect, some embodiments of the present application provide a method for filtering point cloud data occupying a grid height dimension, comprising:

[0008] Projecting the point cloud data onto a two-dimensional grid map, obtaining first two-dimensional feature information of each observation grid in the two-dimensional grid map, wherein the first two-dimensional feature information includes: average position, number of static point clouds, number of dynamic point clouds, and height value of the static point clouds;

[0009] Clustering the static point cloud within each observation grid according to the height value set to obtain at least one cluster target, and acquiring first height dimension feature information of each cluster target, wherein the first height dimension feature information includes: average height position, cluster height, and number of clustered static point clouds;

[0010] Based on the state feature information of the predicted grid in the previous frame, determining the state feature value set of each predicted grid in the current frame;

[0011] The prediction grid is associated with the observation grid in two levels, and a state feature value set of the prediction grid is updated or extrapolated based on the association result, wherein the two-level association includes: a first-level association is a grid-level matching based on two-dimensional position, and a second-level association is a matching based on height-dimensional clustering targets;

[0012] For the observation grids that are not successfully associated with any prediction grids in the current frame, grid regeneration is performed and the state feature value set of the new grid is initialized;

[0013] The validity of all prediction rasters in the current frame is judged, and the prediction rasters judged to be invalid are removed from the raster list of the current frame.

[0014] In a second aspect, some embodiments of the present application further provide an electronic device, comprising:

[0015] One or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processors execute any one of the above-described point cloud data occupancy grid height dimension filtering methods.

[0016] In a third aspect, some embodiments of the present application further provide a computer-readable storage medium having stored thereon a computer program and / or instructions, which, when executed by a processor, implements any of the above-described methods for filtering the grid height dimension occupied by point cloud data.

[0017] In a fourth aspect, some embodiments of the present application further provide a computer program product, comprising a computer program and / or instructions, which, when executed by a processor, implements any of the above-described methods for filtering the grid height dimension occupied by point cloud data.

[0018] Compared to related technologies, the solution provided in the embodiments of this application introduces a height-dimensional structured modeling mechanism while retaining the ability of a two-dimensional occupancy grid to express the horizontal position and boundary contours of obstacles. By clustering the height of static point clouds within each two-dimensional grid, this method can effectively distinguish multiple height-dimensional obstacle targets existing at the same two-dimensional location, further achieving an explicit distinction between ground targets and high-altitude targets. Therefore, this application can accurately express whether there are ground obstacles, high-altitude obstacles, or both within the occupancy grid area, effectively improving the accuracy and robustness of the environmental perception system in complex multi-height scenarios such as tunnels, elevated roads, and intersections. Moreover, compared to directly constructing a complete three-dimensional occupancy grid map, the two-dimensional projection and local height-dimensional clustering method adopted in this application does not require voxel encoding and three-dimensional spatial traversal processing for each point. The actual clustering situation can be determined by grid merging and pruning. This can significantly reduce the number of height-dimensional grids tracked, thereby reducing the overall execution time of the algorithm and improving operational efficiency, making it suitable for deployment on resource-constrained embedded platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is one of the flow charts of a method for filtering the height dimension of a grid occupied by point cloud data provided in an embodiment of the present application;

[0021] Figure 2 This is a second flow chart of a method for filtering the height dimension of a grid occupied by point cloud data provided in an embodiment of the present application;

[0022] Figure 3 This is a flow chart of performing two-level association between the prediction grid and the observation grid and performing update and extrapolation processing according to an embodiment of the present application;

[0023] Figure 4 This is a logical flow diagram of determining whether the first-level association is successful, provided by an embodiment of the present application;

[0024] Figure 5This is a logical flow diagram of determining whether the second-level association is successful, provided by an embodiment of the present application;

[0025] Figure 6 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] First embodiment

[0028] The first embodiment of the present application relates to a method for filtering the height dimension of a grid occupied by point cloud data. The method can be applied to a system, and the system can be installed in an electronic device. Figure 1 、 Figure 2 As shown, the method may include the following steps:

[0029] Step S1: Project the point cloud data onto a two-dimensional grid map to obtain the first two-dimensional feature information of each observation grid in the two-dimensional grid map. The first two-dimensional feature information includes: average position, number of static point clouds, number of dynamic point clouds, and height value of static point clouds.

[0030] Specifically, for step S1, point cloud data refers to a collection of discrete points on the surface of an object in three-dimensional space collected by sensors (such as millimeter-wave radar and lidar). Each point usually contains spatial coordinate information (x, y, z) and attributes such as reflection intensity, which are used to represent the spatial structure in the environment.

[0031] A two-dimensional grid map refers to a regular grid map formed by projecting three-dimensional point cloud data onto a horizontal plane. Each grid cell represents a fixed area on the ground and is used to indicate obstacle occupancy information in the area, facilitating subsequent processing and path planning.

[0032] The observation grid refers to the grid cells that the lidar device can observe after projecting the point cloud data onto a horizontal plane. Each grid cell records information such as the probability of an obstacle appearing in that area, its category, or its structural characteristics. It is the fundamental unit for building an environmental model. The lidar device can be a 4D millimeter-wave radar.

[0033] For example, a frame of 3D point cloud data acquired by the sensor is projected onto a horizontal 2D plane to construct a 2D grid map. Each point cloud typically contains 3D spatial coordinates (x, y, z), and optionally also includes information such as reflection intensity and velocity components. The point cloud is partitioned at a set grid resolution, and its 2D grid coordinates are calculated, forming a 2D structured grid consisting of multiple observation grids. For each observation grid, the following first 2D feature information is collected and extracted.

[0034] Mean position (mean_x, mean_y): The arithmetic mean of the x and y coordinates of all point clouds projected into the grid, representing the geometric center position of the observation grid.

[0035] Static point cloud number (spts_num): Based on the speed information of each point cloud, if the speed is lower than the set static threshold (for example, 0.1m / s), it is judged as a static point and counted in the static point cloud number to identify stable environmental structures.

[0036] Dynamic point cloud number (dpts_num): If the point cloud speed is higher than the static threshold, it is considered a dynamic point and counted in the dynamic point cloud number for identifying moving objects such as vehicles and pedestrians.

[0037] Height value of static point cloud: Extract the height z values ​​of all points identified as static points to form the height dimension analysis input of the grid for subsequent height dimension clustering operations.

[0038] Step S2: Cluster the static point cloud in each observation grid according to the height value set to obtain at least one cluster target, and obtain the first height dimension feature information of each cluster target. The first height dimension feature information includes: average height position, cluster height and the number of clustered static point clouds.

[0039] Specifically, for step S2, for each observation grid, a height-dimensional clustering operation is performed based on the set of height values ​​of the static point cloud extracted from the grid according to a preset clustering rule, thereby dividing the point cloud with continuity in height and dense spatial distribution characteristics into one or more cluster targets. This clustering rule may include similarity judgment based on whether the adjacent difference in point cloud height values ​​is less than a set clustering interval threshold. It can also be implemented through sequential traversal, sliding window segmentation, or density analysis. The specific implementation method can be flexibly selected based on system performance requirements.

[0040] After clustering, the system further extracts the first-dimensional feature information of each clustered object. This information includes the clustered object's mean height position (meanz), which describes its center position in the height dimension; the cluster height (height), which represents the vertical distribution range of the clustered object; and the number of clustered static point clouds (clspts_num), which measures the clustered object's structural stability and spatial occupancy intensity. This allows the system to introduce a hierarchical representation of the height dimension based on two-dimensional grid mapping, effectively identifying the combined occupancy of ground obstacles and high-altitude structures.

[0041] Step S3: Based on the state feature information of the prediction grid in the previous frame, determine the state feature value set of each prediction grid in the current frame.

[0042] Specifically, regarding step S3, relying solely on observation data from a single frame may result in missing or incomplete target information. Therefore, to achieve continuous tracking and dynamic prediction of obstacle states during multi-frame continuous observation, the system appropriately extrapolates and corrects the spatial position of the prediction grid based on the state information recorded by the prediction grid in the previous frame, combined with the vehicle's motion information or posture changes between two adjacent frames. Simultaneously, the state feature information contained in the prediction grid in the previous frame is inherited or updated as the state feature information of the prediction grid in the current frame, thereby constructing a set of state feature values ​​for the prediction grid in the current frame. This provides basic data support for grid association, state matching, and grid generation in subsequent steps. In other words, the prediction grid is not generated directly from the observation results of the current frame, but rather evolved from historical states. It is used to maintain and predict the state of the target obstacle between consecutive frames. This effectively ensures the continuity and stability of the grid state in the temporal dimension, avoids target tracking deviations caused by inter-frame data discontinuities, and further improves the accuracy and robustness of the entire environmental perception system.

[0043] Step S4: The predicted grid and the observed grid are associated with each other in two levels, and the state feature value set of the predicted grid is updated or extrapolated based on the association result, wherein the two-level association includes: the first level association is grid-level matching based on two-dimensional position, and the second level association is matching based on height-dimensional clustering targets.

[0044] For step S4, specifically, the two-level association includes a first-level association and a second-level association. Among them, the first-level association is a grid-level matching based on two-dimensional position, that is, it determines whether the predicted grid and the observed grid are close in position on the two-dimensional plane. If the match is successful, the corresponding state of the predicted grid is updated according to the two-dimensional features of the observed grid. If the match fails, the state is extrapolated according to the state of the previous frame. The second-level association is based on the matching of the height-dimensional clustering target. On the basis of the success of the first-level association, it further determines whether the height-dimensional structure of the predicted grid and the corresponding observed grid matches, so as to update or extrapolate the height-dimensional features of the predicted grid accordingly. It ensures the consistency of target matching of the system at the two-dimensional plane and height-dimensional levels, and effectively improves the stability of the grid state and the tracking continuity.

[0045] Step S5: Perform grid regeneration for the observation grids that are not successfully associated with any prediction grids in the current frame, and initialize the state feature value set of the regenerated grids.

[0046] Specifically, regarding step S5, after completing the double-layer association matching between the predicted grid and the observed grid, the system analyzes the observed grids in the current frame that failed to successfully associate with any predicted grids based on the set regeneration criteria to determine whether these unmatched observed grids meet the criteria for regeneration as new grids. If the regeneration criteria are met, the observed grid is marked as a new grid (with no historical prediction basis), and its state feature value set is initialized to be included in the subsequent filtering, updating, and lifecycle management of occupied grids. For observed grids that do not meet the regeneration criteria, which may be subject to interference from factors such as occasional noise, motion interference, and false detections, the system will not track or save them for the time being. This avoids immediately establishing tracking status for every newly appearing observation point or observation grid without screening, which can easily lead to misjudgments. Therefore, by setting reasonable regeneration criteria, a grid regeneration mechanism is introduced to achieve a balance between rapid response to environmental changes and limiting false detections.

[0047] Step S6: performing validity judgment on all prediction grids in the current frame, and removing prediction grids judged to be invalid from the grid list of the current frame.

[0048] For step S6, specifically, the system makes a comprehensive judgment based on the key attributes contained in the state feature value set of the grid, including but not limited to whether the dynamic occupancy probability and spatial position of the predicted grid exceed the set two-dimensional grid map boundary. If a predicted grid has obvious dynamics in the current frame, that is, its dynamic point cloud occupancy probability is greater than 0, then the predicted grid may correspond to an unstable or moving target and is not suitable for continuous tracking as a static structure. The system marks it as invalid and removes it from the grid list of the current frame; at the same time, if a predicted grid has exceeded the set grid map range after state extrapolation or update, its two-dimensional position will also be regarded as an invalid target and eliminated. Through the grid post-processing step, the system can regularly clean up failed, invalid or out-of-bounds predicted grids to ensure that only valid state information is retained in subsequent cycles, thereby improving the operating efficiency and tracking accuracy of the entire occupancy grid filtering method, avoiding the accumulation of invalid data or mismatching, and enhancing the overall stability and robustness of the perception system.

[0049] It is not difficult to find that compared with related technologies, the solution provided by the embodiments of the present application introduces a height-dimensional structured modeling mechanism while retaining the ability of the two-dimensional occupancy grid to express the horizontal position and boundary contours of obstacles. By clustering the height of static point clouds within each two-dimensional grid, this method can effectively distinguish multiple height-dimensional obstacle targets existing at the same two-dimensional location, further achieving explicit distinction between ground targets and high-altitude targets. Therefore, the present application can accurately express whether there are ground obstacles, high-altitude obstacles, or both in the occupancy grid area, effectively improving the accuracy and robustness of the environmental perception system in complex multi-height scenarios such as tunnels, elevated roads, and intersections. Moreover, compared to directly constructing a complete three-dimensional occupancy grid map, the two-dimensional projection and local height-dimensional clustering method adopted in this application does not require voxel encoding and three-dimensional space traversal processing for each point. The actual clustering situation can be determined by grid merging and pruning. This can significantly reduce the number of height-dimensional grids to track, thereby reducing the overall execution time of the algorithm and improving operational efficiency, making it suitable for deployment on resource-constrained embedded platforms.

[0050] Second embodiment

[0051] The second embodiment of the present application relates to a method for filtering the height dimension of a grid occupied by point cloud data. The second embodiment is an improvement on the first embodiment. The specific improvement is that: in the second embodiment of the present application, a specific implementation method for height-dimensional clustering of static point clouds within an observation grid is provided. That is, in step S2, the static point clouds within each observation grid are clustered according to a set of height values ​​to obtain at least one clustering target, which can further include the following steps:

[0052] Step S201: sorting all static point clouds in the observation grid according to their height values ​​from small to large;

[0053] Step S202: Calculate the height difference between adjacent height values ​​and determine whether the height difference is less than a preset height difference threshold;

[0054] Step S203: when the height difference is less than the height difference threshold, the corresponding two static point clouds are classified into the same cluster target;

[0055] Step S204: when the height difference is greater than or equal to the height difference threshold, classifying the corresponding two static point clouds into different cluster targets;

[0056] Step S205: performing height difference judgment and clustering on all static point clouds in the observation grid until all static point clouds are divided into one or more cluster targets.

[0057] In this embodiment, when processing each two-dimensional observation grid, the system will traverse the point cloud set marked as "static" in the grid and extract its height value (i.e., z-coordinate). These points are then sorted in ascending order by height value to form a height sequence. The height difference between adjacent height values ​​is calculated, and a determination is made as to whether the height difference is less than a preset height difference threshold.

[0058] For example, the system records the sorted point cloud height value sequence as , and calculate the height difference of adjacent point clouds pair by pair from beginning to end ,Then each height difference is compared with the set height difference threshold kClusterThr (e.g., kClusterThr = 0.2 meters) to determine whether the adjacent points have vertical structural continuity.

[0059] like , indicating that the two point clouds are closely distributed in the height direction and may belong to the same object surface or continuous structure, so the system classifies them as part of the current clustering target. , indicating that the current point cloud has a significant jump in height from the previous structure, that is, the structure boundaries of different height layers appear. At this time, the system ends the construction of the current cluster and initializes a new cluster target, using the current point cloud as an element in the new cluster.

[0060] Repeat the above process, traverse and cluster the entire sorted height value sequence until all static point clouds are divided into one or more cluster targets. After completing this process, the system will construct several height layers in each observation grid, and each cluster target represents an independent structure of the grid in the height dimension. After completing the height clustering of the static point cloud in the observation grid, the system further extracts the structural feature information of each cluster target in the height dimension. The extracted first height dimension feature information includes: average height position (referring to the average value of the height values ​​of all static point clouds in the cluster target); cluster height (referring to the height difference between the highest point and the lowest point inside the cluster target, which can reflect the thickness or structural range of the cluster target in the vertical direction); number of clustered static point clouds (representing the number of static points contained in the cluster target, which is a measure of the stability of the target and can also be used to filter out small noise clusters). By extracting the above three eigenvalues, the system can not only identify whether each two-dimensional grid has a multi-layer structure in the vertical direction, but also assign a clear position, height range and confidence level to each layer of obstacles, thereby improving the expression ability and tracking accuracy of the entire occupancy grid filtering method in three-dimensional space.

[0061] It is not difficult to find that in the embodiment of the present application, by setting a reasonable height difference threshold for clustering, it is possible to distinguish obstacle targets of multiple height levels within the same two-dimensional grid, such as the simultaneous existence of ground curbs and high-altitude beams. This method enables the system to no longer be limited to using simple statistical features such as maximum, minimum or average values ​​to represent height, and achieves a more detailed depiction of the real spatial structure in the grid. Moreover, the clustering process is only based on height value sorting and difference judgment, without the need to introduce complex three-dimensional clustering algorithms or preset cluster numbers. The computational complexity is low, and it is easy to deploy and run in platforms with high real-time requirements and limited resources, such as on-board processors.

[0062] Third embodiment

[0063] The third embodiment of the present application relates to a method for filtering the height dimension of a grid occupied by point cloud data. The third embodiment is an improvement on the first embodiment. The specific improvement is that: in the third embodiment of the present application, a specific implementation method for constructing a set of state feature values ​​of a prediction grid is provided. That is, step S3 can further include the following steps:

[0064] Step S301: constructing a state feature value set of a prediction grid;

[0065] Step S302: Based on the average position of the predicted grid in the previous frame and the vehicle's posture change information between the previous and next frames, the average position of the predicted grid in the current frame is extrapolated to obtain the two-dimensional position of the predicted grid in the current frame and write it into the state feature value set;

[0066] Step S303: The number of static point clouds, the number of dynamic point clouds, the number of detection frames, the static point cloud occupancy probability, the dynamic point cloud occupancy probability, and the grid maturity flag of the predicted grid in the previous frame are used as the second two-dimensional feature information of the predicted grid in the current frame and written into the state feature value set;

[0067] Step S304: The height-dimensional average position, height-dimensional cluster height, height-dimensional static point cloud occupancy probability, and height-dimensional grid maturity flag of each cluster target in the predicted grid in the previous frame are used as the second height-dimensional feature information of the predicted grid in the current frame and written into the state feature value set.

[0068] In this embodiment, the system initializes the state feature value structure in the current frame for each predicted grid object retained from the previous frame, which is used to record its two-dimensional spatial properties, dynamic statistical properties, and height-dimensional structural information. Therefore, a state feature value set is constructed for each predicted grid in the current frame. This feature value set includes the second two-dimensional feature information of the predicted grid in the plane position: the two-dimensional position information (x, y) of the predicted grid, the cumulative number of static point clouds α_static, the cumulative number of dynamic point clouds α_dynamic, the number of detection frames β, the static point cloud occupancy probability λ_static, the dynamic point cloud occupancy probability λ_dynamic, and the grid maturity flag flg_mature. It also includes the second height-dimensional feature information of the predicted grid in the height dimension: the height-dimensional average position z, the height-dimensional cluster height h, the height-dimensional static point cloud occupancy probability λ_static_z, and the height-dimensional grid maturity flag flg_zmature.

[0069] The two-dimensional position of the predicted grid in the current frame is calculated by extrapolating the average position of the predicted grid in the previous frame and the vehicle's posture change information (such as linear velocity and angular velocity) between the previous and next frames. The remaining feature information can directly inherit the state feature values ​​of the corresponding predicted grid in the previous frame if there is no special change.

[0070] For example, the two-dimensional position of the prediction grid can be calculated according to the following formula:

[0071]

[0072] in, Represents the two-dimensional position vector of the predicted grid in the current frame, is the two-dimensional position vector of the predicted grid of the previous frame, A is the rotation matrix that describes the position change relationship between the previous and next frames, is the change vector of the vehicle position coordinates between the two frames.

[0073] It is not difficult to find that in the embodiment of the present application, the detailed structure of each predicted grid in the height direction is guaranteed to be preserved, and can be used in the subsequent height-dimensional matching with the observation grid, ensuring that the system has structural consistency and tracking capabilities between the upper and lower frames during the height-dimensional modeling process.

[0074] It should be noted that the third embodiment of the present application may also be an improvement based on any one or more of the first to second embodiments.

[0075] Fourth embodiment

[0076] The fourth embodiment of the present application relates to a method for filtering the height dimension of a grid occupied by point cloud data. The fourth embodiment is an improvement based on the third embodiment. The specific improvement is that: in the fourth embodiment of the present application, a specific implementation method of the state association and update process between the prediction grid and the observation grid based on a two-level association mechanism is provided. Figure 3 As shown, step S4 may further include the following steps:

[0077] Step S401: Determine whether each prediction grid and observation grid in the current frame meet the first level association condition.

[0078] In some embodiments, reference Figure 4 As shown, step S401 may further include the following steps:

[0079] Step S4011: Based on the two-dimensional position of the predicted grid, determine whether the predicted grid falls within the valid area of ​​the current frame observation grid map;

[0080] Step S4012: If yes, further determine whether the number of static point clouds in the corresponding observation grid is greater than a preset first point cloud number threshold;

[0081] Step S4013: If the number of point clouds is greater than the first point cloud quantity threshold, the first level association is deemed successful;

[0082] Step S4014: If the number of static point clouds is less than or equal to the first point cloud number threshold, determining whether the sum of the number of static point clouds of each observation grid in the eight-connected neighborhood of the prediction grid in the observation grid map is greater than a preset second point cloud number threshold;

[0083] Step S4015: If the number of point clouds is greater than the second point cloud quantity threshold, the first-level association is deemed successful, and the joint two-dimensional feature information of each observation grid in the eight-connected neighborhood is fused to perform an update operation;

[0084] Step S4016: If none of the above conditions are met, it is determined that the first level association has failed.

[0085] In this embodiment, the valid area of ​​the observation grid map of the current frame is the area within the boundary range of the grid cells with actual observation point cloud data in the two-dimensional grid map. This area represents the spatial range that the current frame perception system can see, and is also the basis for judging whether the predicted grid can participate in the two-dimensional position grid-level matching (first-level association). If the two-dimensional position of a predicted grid falls outside the valid area, it means that it is in an area that cannot be observed by the sensor. At this time, if forced matching is performed, it may cause false association, so grid extrapolation processing is required. Grid extrapolation refers to the process of predicting the two-dimensional spatial position and / or height dimension features of the grid and continuing its existence state based on the grid status in the historical frame in the absence of direct observation data in the current frame.

[0086] Since some grids in the current frame may consist of only a few point clouds, these are often false points caused by reflection errors, boundary effects, or noise. Therefore, setting the first point cloud number threshold kSPtsThr can exclude these unreliable observation grids and prevent the predicted grid from being misassociated. If the number of static point clouds in the observation grid reaches a certain number, satisfying spts_num > kSPtsThr, it indicates that the observation in this area is stable and spatially coherent, and the observation grid can be trusted as a reference for updating the predicted grid features.

[0087] If spts_num ≤ kSPtsThr, the algorithm continues to search for observation grids within the eight-connected neighborhood of the prediction grid. The cumulative number of static point clouds, tspts_num, for the observation grids within the eight-connected neighborhood is determined to determine whether it exceeds the second point cloud number threshold, kTSPtsThr. The eight-connected neighborhood refers to the eight grid cells directly adjacent to the current prediction grid in the two-dimensional grid map, including the top, bottom, left, right, and four diagonal directions. By summing the static point cloud counts of all observation grids within the eight-connected neighborhood and performing joint feature extraction, the algorithm can leverage local neighborhood information to assist in grid association judgment and state compensation when a single grid point cloud is insufficient.

[0088] If the sum of the number of static point clouds in the eight-connected neighborhood, tspts_num, exceeds kTSPtsThr, the first-level association is still considered successful, and the two-dimensional feature information of the predicted grid is fused and updated based on the joint two-dimensional feature information of all observation grids in the eight-connected neighborhood (such as average position, cumulative number of static point clouds, cumulative number of dynamic point clouds, etc.). At the same time, the height values ​​of the static point clouds in all observation grids in the eight-connected neighborhood are accumulated and re-sorted and clustered to obtain the average height position, cluster height, and cluster static point cloud number.

[0089] If none of the above conditions are met, the first-level association is deemed to have failed. At this time, the prediction grid cannot be updated based on the observation information of the current frame, but the state extrapolation operation is turned into a further prediction of its state feature value.

[0090] Through the hierarchical judgment and neighborhood information fusion mechanism, this embodiment effectively improves the association success rate of the grid in sparse or edge areas of the point cloud, while avoiding accidental deletion or extrapolation misjudgment caused by occasional omissions, and enhances the robustness and continuity of the entire filtering method.

[0091] Step S402: If the first level association is successful, the second two-dimensional feature information of the prediction grid is updated based on the first two-dimensional feature information of the observation grid associated with it.

[0092] In this embodiment, after the predicted grid and the observed grid are successfully matched through the first association, the second two-dimensional feature information of the predicted grid can be updated based on the two-dimensional feature information of the observed grid. Specifically, the update can be performed as follows: based on the average two-dimensional position of the observed grid in the current frame, combined with the extrapolated position of the predicted grid in the previous frame, the predicted position is corrected using the gain matrix and state transition matrix of the Kalman filter, thereby obtaining the updated two-dimensional position of the predicted grid in the current frame, which can be calculated according to the following formula:

[0093]

[0094] in, is the updated predicted grid two-dimensional position vector, is the predicted position vector, is the two-dimensional position measurement value of the observation grid in the current frame, that is, the average position (meanx, meany), H is the state transfer matrix, is the Kalman gain matrix. This update formula can effectively integrate the predicted state and the current observation to improve the position update accuracy.

[0095] Subsequently, based on the number of static point clouds and dynamic point clouds of the current observation grid, the number of static point clouds and dynamic point clouds of the prediction grid are cumulatively updated, and the number of detection frames of the prediction grid is updated accordingly, which can be calculated according to the following formula:

[0096]

[0097] Among them, α_static and α_dynamic represent the current accumulated number of static point clouds and dynamic point clouds of the prediction grid, respectively. spts_num and dpts_num represent the number of static point clouds and dynamic point clouds associated with the observation grid in the current frame, respectively. β is the number of frames in which the prediction grid is observed, which is used to measure the tracking persistence of the grid.

[0098] The static point cloud occupancy probability is calculated based on the updated number of static point clouds and the number of detection frames, and the dynamic point cloud occupancy probability is calculated based on the updated number of dynamic point clouds and the number of detection frames. The calculation can be performed according to the following formula:

[0099]

[0100] Among them, λ_static and λ_dynamic represent the current static point cloud occupancy probability and dynamic point cloud occupancy probability of the predicted grid, respectively, and are important indicators for measuring the credibility of the predicted grid state.

[0101] Finally, the maturity state of the predicted grid is judged. If the number of detection frames β is greater than the preset maturity threshold kMatureThr, the grid is marked as "mature". The grid in the mature state can be used in the subsequent environmental perception module.

[0102] Step S404: Based on the success of the first-level association, it is determined whether each prediction grid and observation grid in the current frame meet the second-level association conditions.

[0103] In some embodiments, reference Figure 5 As shown, step S404 may further include the following steps:

[0104] Step S4041: extracting cluster targets of the prediction grid in the previous frame and cluster targets of the observation grid associated with the current frame;

[0105] Step S4042: Calculating the height difference between the cluster target of each prediction grid and the cluster target of the corresponding observation grid, and determining whether the height difference is less than a preset first height difference threshold;

[0106] Step S4043: If the height difference is less than the first height difference threshold, determine whether the height range of the cluster target of the prediction grid overlaps with the height range of the cluster target of the corresponding observation grid. The overlapping condition is that the height difference is less than half of the sum of the cluster heights of the two cluster targets;

[0107] Step S4044: If there is overlap, the cluster targets of the prediction grid that meet the above conditions are combined with the cluster targets of the observation grid to form target pairs, and a cost matrix is ​​constructed. The target pairs constitute the effective elements of the cost matrix, and the Hungarian algorithm is used to perform minimum cost matching on the target pairs to determine whether there is a matching relationship;

[0108] Step S4045: If the match is successful, the second level association is deemed successful;

[0109] Step S4046: If none of the above conditions are met, it is determined that the second level association has failed.

[0110] In this embodiment, for each predicted grid that has completed the first association, all height-dimensional clustered objects corresponding to the previous frame are extracted. Simultaneously, all clustered objects in the observed grid that has successfully completed the first association with the predicted grid in the current frame are extracted. For each pair of clustered objects in these two target sets, the difference between their height centers, δz, is calculated to determine whether it is less than a preset first height difference threshold, kDistanceThr.

[0111] If δz < kDistanceThr, then further judge the overlap of the height-dimensional cluster pair, that is, whether the height difference δz is less than half of the sum of the two cluster heights. If δz < (h + height) / 2, where h is the height of the cluster target of the prediction grid and height is the height of the cluster target of the observation grid.

[0112] All prediction grids that meet the above two conditions are grouped with the cluster targets of the observation grid to form target pairs, and a cost matrix for matching these target pairs is constructed. The elements of the cost matrix can be calculated based on the height mean difference, cluster height overlap, or other pre-defined structural similarity metrics. The cost matrix is ​​optimally matched using the Hungarian minimum weight matching algorithm. If a target pair is successfully matched, the second-level height dimension association between the prediction grid and the observation grid is considered successful. If no target pair is successfully matched, the height dimension association between the prediction grid and the observation grid is considered unsuccessful.

[0113] Step S405: If the second level association is successful, the second height dimension feature information of the prediction grid is updated based on the first height dimension feature information of the observation grid associated with it.

[0114] Specifically, based on the observation vector composed of the height mean and cluster height of the observed grid cluster target, and the height vector of the previous frame of the predicted grid cluster target, the state transfer matrix and gain matrix in the Kalman filter algorithm are introduced to perform height state update and obtain the updated height dimension vector in the current frame, which can be calculated according to the following formula:

[0115]

[0116] is the updated height position vector, which contains the average height position z and cluster height h of the cluster target in the current frame. is the height vector of the predicted grid of the previous frame, is the Kalman filter gain matrix, is the height-dimensional observation vector of the observation grid in the current frame, , T is the state transfer matrix.

[0117] At the same time, the occupancy probability of the static point cloud in the height dimension is updated as follows:

[0118]

[0119] in, is the static point cloud occupancy probability of the clustering target in the current frame, is the static point cloud occupancy probability of the clustering target in the previous frame, is the number of detection frames of the previous frame, clspts_num is the number of static point clouds contained in the cluster target in the current frame, and β is the number of detection frames of the current frame.

[0120] When the second-level association is successful, the first-dimensional feature information of the clustered targets in the observation grid is used to update the second-dimensional feature information of the prediction grid, correcting the prediction error and helping to accurately reflect the target's true state in the height dimension. The updated height-dimensional information serves as the basis for predictions in subsequent frames, forming a more stable and reliable temporal filtering trajectory. It can also accurately determine whether the target is a persistent obstacle, reducing the false recognition rate and false alarm rate. This is especially true in environments with multi-layered structures such as viaducts, tunnels, and intersections, enabling continuous tracking of obstacles at varying heights.

[0121] Step S403: If the first level association fails, an extrapolation operation is performed on the second two-dimensional feature information and the second high-dimensional feature information of the predicted grid based on the state feature value set of the predicted grid in the current frame.

[0122] For the extrapolated two-dimensional feature information, specifically, the extrapolation processing for the second two-dimensional feature information can be performed under the premise of satisfying the following extrapolation conditions, that is, the static point cloud occupancy probability of the predicted grid in the previous frame Greater than the set first extrapolation threshold kLambdaThr. The first extrapolation threshold kLambdaThr is not a fixed setting and is dynamically adjusted according to the height position characteristics of the prediction grid and the different maturity flags of the grid. When the height position of the prediction grid is high or its maturity state is unstable, the system can appropriately increase the set value of kLambdaThr, thereby increasing the confidence threshold of extrapolation and reducing the occurrence of erroneous extrapolation; when the prediction grid is near the ground and has been marked as mature by continuous observations, the value of kLambdaThr can be appropriately reduced to enhance the timeliness and continuity of extrapolation.

[0123] When the above extrapolation conditions are met, the system will output the two-dimensional feature information of the predicted grid, specifically: keeping the two-dimensional position predicted in the previous frame unchanged (as the predicted position of the current frame) and performing extrapolation operations on other features:

[0124] The number of detection frames β increases: β+= 1;

[0125] The static point cloud occupancy probability λ_static is calculated according to the following formula:

[0126]

[0127] in, is the number of static point clouds accumulated in the previous frame.

[0128] The dynamic point cloud occupancy probability λ_dynamic is calculated according to the following formula:

[0129]

[0130] in, The number of dynamic point clouds accumulated in the previous frame.

[0131] At the same time, the mature flag of the prediction grid remains the same as the previous frame, and no state switching occurs. If the above extrapolation conditions are not met, the state features of the prediction grid are determined to no longer have extrapolation value, and its state will be cleared. In other words, the two-dimensional feature information is considered to have disappeared in the current frame and will no longer participate in the state update or trajectory tracking process in subsequent frames.

[0132] For the extrapolation of height dimension feature information, specifically, if the predicted grid does not meet the second-level association condition at the height dimension level with the observation grid in the current frame, the system can perform the extrapolation operation of the height dimension feature based on the second height dimension feature information of the predicted grid in the previous frame. To ensure the reliability of the extrapolation result, the system sets the height dimension extrapolation condition, which requires that the static point cloud occupancy probability of the height dimension cluster target in the previous frame is greater than a second extrapolation threshold kLambdaZThr, which can also be dynamically adjusted according to the height position of the grid and its maturity state.

[0133] When the above extrapolation conditions are met, the system will output the predicted height dimension feature information of the cluster target, specifically including: keeping the predicted height position z and height value h in the previous frame unchanged, that is, the spatial structure information of the height dimension cluster target in the current frame remains consistent with the previous frame.

[0134] According to the occupancy probability of the static point cloud in the previous frame and detection frame number , calculate the static point cloud occupancy probability of the clustering target in the current frame according to the following formula :

[0135]

[0136] Among them, β is the number of detection frames of the current frame.

[0137] The maturity flag of the highly dimensional clustered target in the current frame remains consistent with that of the previous frame, ensuring that the extrapolated state continues with the original state. If the above extrapolation conditions are not met, the system determines that the highly dimensional clustered target no longer has extrapolation value in the current frame, and its state will be cleared. In other words, the highly dimensional clustered target is considered dead in the current frame and will no longer participate in the state update or trajectory tracking process in subsequent frames.

[0138] By performing extrapolation operations on the two-dimensional feature information and height-dimensional feature information of the predicted grid when matching fails, the system can effectively enhance its temporal consistency and structural integrity in multi-frame continuous detection, avoid misidentification or trajectory interruption caused by information loss, and help the system maintain continuous tracking and accurate expression of the obstacle status.

[0139] Step S406: If the second level association fails, then an extrapolation operation is performed on the height dimension feature information of the predicted grid based on the state feature value set of the predicted grid in the current frame.

[0140] It should be noted that if the predicted grid successfully matches the observed grid in two dimensions but fails to meet the second-level association condition in the height dimension, that is, if the cluster targets of any observed grid are not successfully matched, the system can perform an extrapolation operation on the second-height-dimensional feature information of each height-dimensional cluster target in the predicted grid based on the state feature value set of the predicted grid in the current frame. The specific extrapolation operation can be found in the above description of extrapolating height-dimensional feature information and will not be repeated here.

[0141] Furthermore, in some embodiments, the method further includes a step of detecting an extrapolated state of the second height dimension feature information of the prediction grid, specifically including:

[0142] When the first level association is successful, but all the second height dimension feature information of the prediction grid does not meet the second level association conditions and the extrapolation operation is not performed, or when the second two-dimensional feature information of the prediction grid has been extrapolated but the corresponding second height dimension feature information has not been extrapolated, trigger the extrapolation status detection of the second height dimension feature information of the prediction grid;

[0143] The extrapolation state detection is used to ensure that at least one second-dimensional feature information has completed extrapolation, specifically including:

[0144] When the predicted grid in the previous frame has only one high-dimensional clustered target, the extrapolation operation is directly performed on the second high-dimensional feature information of the high-dimensional clustered target;

[0145] When the predicted grid in the previous frame contains multiple height-dimensional clustered targets, the second height-dimensional feature information of the height-dimensional clustered target with the highest probability of being occupied by the height-dimensional static point cloud is selected to perform the extrapolation operation.

[0146] In this embodiment, to ensure the continuity and sustainability of the predicted grid at the height dimension level, after performing the height dimension feature information update or extrapolation operation, a step of performing an extrapolation status detection on the height dimension grid is also included. This detection step is used to deal with the following two situations: one is that the predicted grid in the current frame has passed the first-level association, but all of its height dimension feature information has failed to pass the second-level association and no extrapolation operation has been performed; the other is that the two-dimensional feature information of the predicted grid in the current frame has been extrapolated, but all of its corresponding height dimension feature information has not yet been extrapolated. In such cases, the system will trigger the extrapolation status detection operation of the height dimension feature to ensure that the height dimension feature information of at least one cluster target can be extrapolated.

[0147] The extrapolation state detection process may include the following two situations: if the prediction grid contains only one high-dimensional clustered target in the previous frame, the system directly performs an extrapolation operation on the second high-dimensional feature information of the target; if the previous frame contains multiple high-dimensional clustered targets, the system can perform an extrapolation operation based on the high-dimensional static point cloud occupancy probability. The target with the highest probability is selected for extrapolation.

[0148] By introducing a height-dimensional feature information extrapolation state detection mechanism, we can effectively prevent inconsistencies between two-dimensional and height-dimensional feature information. This is especially true when the predicted grid fails to match the current frame's observed grid in the height dimension, or when association fails due to factors such as occlusion or intermittent loss of height-dimensional clustered objects. This detection mechanism ensures that at least one height-dimensional feature is continuously extrapolated, avoiding information discontinuities in the height dimension.

[0149] Furthermore, in some embodiments, the step of performing a maturity detection on the second height dimension feature information of the prediction grid is further included, specifically comprising:

[0150] When the second two-dimensional feature information of the prediction grid is updated to a mature state, and all the second high-dimensional feature information of the prediction grid is not set to a mature state in the current frame, triggering a maturity state detection for the second high-dimensional feature information of the prediction grid;

[0151] The maturity state detection is used to ensure that at least one second high-dimensional feature information is set to a mature state, specifically including:

[0152] When the predicted grid in the previous frame has only one high-dimensional clustered target, the second two-dimensional feature information of the high-dimensional clustered target is directly set to a mature state;

[0153] When the predicted grid in the previous frame contains multiple high-dimensional clustered targets, the high-dimensional clustered target with the highest probability of being occupied by the high-dimensional static point cloud is selected and its second high-dimensional feature information is set to a mature state.

[0154] In this embodiment, in order to further improve the state consistency and environmental understanding ability of the system under multi-dimensional perception, after the two-dimensional feature information of the prediction grid is updated to a mature state, the system also includes a step of detecting the maturity state of the height-dimensional feature information in the prediction grid. This detection mechanism is mainly used to resolve the following inconsistent scenarios: that is, the two-dimensional feature information has been determined to be mature, but all the corresponding height-dimensional feature information has not yet been set to a mature state. In this scenario, there will be no high-clustered target with sufficient observation accumulation, or the height information fluctuates greatly and the number of observations is small, which cannot support its state to be judged as "mature". Such a contradictory phenomenon of "the area is very stable at the ground level, but its internal height structure is completely unreliable" is extremely unreasonable in the real physical environment.

[0155] Specifically, in the current frame, if the system determines that the number of detection frames corresponding to the second two-dimensional feature information of a certain predicted grid is greater than the maturity threshold, it is set as mature, but in all the second high-dimensional feature information corresponding to the grid, no clustering target is set as mature. At this time, the mature state detection mechanism will be triggered to ensure that at least one high-dimensional feature information also enters the mature state to maintain the logical consistency of information expression.

[0156] The mature state detection process may include the following two situations: If the predicted grid in the previous frame contains only one high-dimensional clustered target, the system directly sets the second high-dimensional feature information of the clustered target as mature. If the predicted grid in the previous frame contains multiple high-dimensional clustered targets, the system prioritizes the high-dimensional static point cloud with the highest occupancy probability. The largest target is taken as the representative target, and the height-dimensional feature information of the target is set as the mature state.

[0157] This mechanism combines the consistency constraint relationship between two-dimensional and high-dimensional feature states (that is, when a two-dimensional grid matures, at least one corresponding high-dimensional grid should be mature), which helps to improve the physical rationality of the system in highly complex structural scenarios.

[0158] It is not difficult to find that in the embodiment of the present application, by introducing a dual association mechanism of two-dimensional and height dimensions, the system can not only achieve coarse-grained matching between the predicted grid and the observed grid based on the two-dimensional position, but can also further refine the association of clustered targets at the height dimension, thereby significantly improving the accuracy and robustness of cross-frame target matching, which is particularly suitable for complex scene environments with overlapping height layers. At the same time, an extrapolation update strategy is also introduced. When the predicted grid fails to successfully associate the feature information of the observed grid in the current frame, the system can deduce its two-dimensional and height-dimensional feature information based on the previous frame state, ensuring the continuity and sustainability of the grid state, and avoiding the disappearance of targets due to short-term occlusion or observation failure.

[0159] It should be noted that the fourth embodiment of the present application may also be an improvement based on any one or more of the first to third embodiments.

[0160] Fifth embodiment

[0161] The fifth embodiment of this application relates to a method for filtering the height dimension of a grid occupied by point cloud data. The fifth embodiment is an improvement on the first embodiment. Specifically, the improvement is that: in the fifth embodiment of this application, a specific implementation method for determining a grid regeneration mechanism based on point cloud density is provided. That is, step S5 can further include the following steps:

[0162] For the observation grid that has not been successfully associated with any prediction grid in the current frame, determine whether it meets the preset regeneration conditions. If the regeneration conditions are met, the observation grid is regenerated as a new grid and the state feature value set of the new grid is initialized.

[0163] It should be noted that in the filter tracking system, the prediction grid represents the continuation of the state of the previous frame, while the observation grid is the raw observation data newly acquired in this frame. When the two cannot match, it means that there is new observation information in this frame, but there is no existing prediction object in the system that can correspond to it. If these unrelated observation grids are not processed at this time, newly appeared obstacles or important environmental information may be ignored or lost. Therefore, the purpose of the grid regeneration mechanism is to supplement the above-mentioned matching omissions. For example, some obstacles enter the radar observation field of view for the first time and no corresponding prediction grid has been formed, or some targets may not form a stable tracking trajectory in the previous frame due to occlusion or long-term stillness. There are also some situations where the correct matching cannot be made due to the tracking accuracy error of the previous frame.

[0164] Among them, the new student conditions include:

[0165] New condition 1: If the number of dynamic point clouds of the observation grid is less than or equal to the preset third point cloud number threshold and the number of static point clouds of the observation grid is greater than the preset first point cloud number threshold, then the new condition is determined to be met.

[0166] New condition 2: If new condition 1 is not met, the total number of dynamic point clouds and the total number of static point clouds of each observation grid in the eight-connected neighborhood of the predicted grid in the observation grid map are calculated. If the total number of dynamic point clouds is less than or equal to the preset fourth point cloud number threshold and the total number of static point clouds is greater than the preset second point cloud number threshold, the new condition is considered to be met;

[0167] For example, if the current observation grid meets the following conditions, the new operation is triggered:

[0168] New condition 1: If the number of dynamic point clouds dpts_num of the observation grid is less than or equal to the set third point cloud number threshold kDPtsThr, and the number of static point clouds spts_num is greater than the set first point cloud number threshold kSPtsThr, that is, dpts_num≤kDPtsThr and spts_num>kSPtsThr is satisfied.

[0169] Newborn condition 1 is based on the static strength of a single observation grid. If the number of dynamic point clouds for an observation grid is small and the number of static point clouds is large, the observation grid is directly determined to meet the newborn condition. This reduces unnecessary neighborhood traversal and calculations, and improves the response speed to stable obstacles.

[0170] New condition 2: If new condition 1 is not met, count the total number of dynamic point clouds, tdpts_num, and the total number of static point clouds, tspts_num, within the eight-connected neighborhood of the observation grid, and determine whether the total number of dynamic point clouds, tdpts_num, exceeds the fourth threshold value, kTDPtsThr. If the total number of dynamic point clouds, tdpts_num, is less than or equal to the fourth threshold value, kTDPtsThr, and the total number of static point clouds, tspts_num, is greater than the second threshold value, kTSPtsThr, then tdpts_num ≤ kTDPtsThr and tspts_num > kTSPtsThr.

[0171] The second regeneration condition is based on a joint neighborhood assessment. When a single grid cell fails the first regeneration condition, the total number of dynamic and static point clouds of the observation grids within its eight-connected neighborhood is counted. If the total number of dynamic point clouds is small and the total number of static point clouds exceeds a set threshold, the area is considered to contain a stable obstacle and regeneration is allowed. This prevents the omission of static targets due to local point cloud sparseness and enhances the system's perception of obstacle edges and weakly reflective areas.

[0172] If none of the above conditions are met, grid regeneration will not be performed.

[0173] When an observation grid is determined to be a new grid, its two-dimensional feature information and height-dimensional feature information will be initialized.

[0174] Initialization of two-dimensional feature information:

[0175]

[0176] Initialization of high-dimensional feature information:

[0177]

[0178] It is not difficult to find that in the embodiment of the present application, multiple regeneration judgment logics including regeneration condition one and regeneration condition two are set up, which can accurately identify and regenerate observation grids that have not been successfully associated with any predicted grid in the current frame under the premise of ensuring that the regenerated grid has sufficient static reliability. When faced with complex situations such as occlusion, sparse echoes, and long-distance detection in actual road environments, it can flexibly decide whether to regenerate the grid, thereby maintaining a complete closed loop of the system status update link. Specifically, on the one hand, by jointly judging the number of static point clouds and the number of dynamic point clouds, false regeneration caused by dynamic objects or temporary noise is effectively avoided, thereby improving the accuracy and stability of environmental modeling; on the other hand, even if a single grid does not meet regeneration condition one, the grid can be regenerated based on the statistical characteristics of the eight-connected neighborhood, realizing reasonable modeling of sparse obstacles or weak signal areas, and enhancing the continuity and integrity of environmental perception. In addition, a reasonable regeneration control strategy avoids the large-scale generation of invalid grids, helps to optimize the allocation of computing resources, and improves the overall real-time performance and execution efficiency of the system.

[0179] It should be noted that the fifth embodiment of the present application may also be an improvement based on any one or more of the first to fourth embodiments.

[0180] The step division of the above various methods is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0181] In addition, some embodiments of the present application further provide an electronic device. The electronic device may be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device may also be various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0182] The electronic device includes: one or more processors; and a memory storing computer program instructions, and when the computer program instructions are executed, the processor executes a point cloud data grid height dimension filtering method provided by any one or more of the above embodiments. Figure 6 An exemplary structural diagram of the electronic device is disclosed. The electronic device includes: one or more processors 1101, a memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, with each device providing some of the necessary operations. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0183] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0184] Input device 1103 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. Examples include a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, and other input devices. Output device 1104 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). The display device may include, but is not limited to, a liquid crystal display, a light emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0185] To provide user interaction, the electronic device may be a computer. The computer includes a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse) through which the user can provide input to the computer. Other types of devices may also be used to provide user interaction; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback), and input from the user may be received in any form (e.g., voice input or tactile input).

[0186] In embodiments of the present application, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implements a method for filtering the height dimension of a point cloud data grid provided in any one or more of the aforementioned embodiments. The computer-readable medium may be included in the electronic device described in the aforementioned embodiments, or it may exist independently and not be incorporated into the device. The computer-readable medium carries one or more computer-readable instructions.

[0187] The memory 1102 can be used as a non-transitory computer-readable storage medium to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor 1101 executes the non-transitory software programs, instructions, and modules stored in the memory 1102 to execute various functional applications and data processing of the server, thereby implementing the program instructions / modules corresponding to the method provided in any one or more of the above embodiments of the present application.

[0188] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely located relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0189] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above. Computer-readable media may be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0190] Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technology, compact discs, digital versatile discs or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0191] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network or a wide area network, or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0192] In the above embodiments, all or part of the steps or functions of the present invention may be implemented using software, hardware, firmware, or any combination thereof. For example, implementation may be achieved using a dedicated integrated circuit, a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of the present application may be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) may be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, a floppy disk, or the like. In addition, some steps or functions of the present application may be implemented using hardware, for example, as a circuit that cooperates with a processor to perform the various steps or functions.

[0193] The computer program product provided in the embodiments of the present application includes one or more computer programs / instructions that, when executed by a processor, fully or partially produce the processes or functions described in accordance with the embodiments of the present application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0194] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-specific system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0195] The scope of this application is defined by the appended claims rather than the foregoing description and is therefore intended to encompass within this application all changes that come within the meaning and range of equivalents of the claims. Any reference signs in the claims should not be construed as limiting the claims to which they relate. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim may also be implemented by one unit or device through software or hardware. Words such as "first" and "second" are only used to distinguish the description and do not indicate any particular order, nor should they be understood as indicating or implying relative importance.

[0196] The above descriptions are merely specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art may easily propose variations or substitutions within the technical scope disclosed in the present application, and such variations or substitutions shall be encompassed within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims, and the above descriptions shall be regarded as exemplary and non-limiting.

Claims

1. A method for filtering point cloud data occupying grid height dimension, characterized in that: include: Projecting the point cloud data onto a two-dimensional grid map, obtaining first two-dimensional feature information of each observation grid in the two-dimensional grid map, wherein the first two-dimensional feature information includes: average position, number of static point clouds, number of dynamic point clouds, and height value of the static point clouds; Clustering the static point cloud within each observation grid according to the height value set to obtain at least one cluster target, and acquiring first height dimension feature information of each cluster target, wherein the first height dimension feature information includes: average height position, cluster height, and number of clustered static point clouds; Based on the state feature information of the predicted grid in the previous frame, determining the state feature value set of each predicted grid in the current frame; The prediction grid is associated with the observation grid in two levels, and a state feature value set of the prediction grid is updated or extrapolated based on the association result, wherein the two-level association includes: a first-level association is a grid-level matching based on two-dimensional position, and a second-level association is a matching based on height-dimensional clustering targets; For the observation grids that are not successfully associated with any prediction grids in the current frame, grid regeneration is performed and the state feature value set of the new grid is initialized; The validity of all prediction grids in the current frame is judged, and the prediction grids judged to be invalid are removed from the grid list of the current frame; The number of static point clouds, the number of dynamic point clouds, the number of detection frames, the static point cloud occupancy probability, the dynamic point cloud occupancy probability, and the grid maturity flag of the predicted grid in the previous frame are used as the second two-dimensional feature information of the predicted grid in the current frame; The height-dimensional average position, height-dimensional cluster height, height-dimensional static point cloud occupancy probability, and height-dimensional grid maturity flag of each cluster target in the predicted grid in the previous frame are used as the second height-dimensional feature information of the predicted grid in the current frame; The step of performing two-level association between the prediction grid and the observation grid, and updating or extrapolating the state feature value set of the prediction grid based on the association result, includes: Determine whether each prediction grid and observation grid in the current frame meet the first-level association conditions; If the first level association is successful, updating the second two-dimensional feature information of the prediction grid based on the first two-dimensional feature information of the observation grid associated therewith; If the first level association fails, performing an extrapolation operation on the second two-dimensional feature information and the second height-dimensional feature information of the prediction grid based on the state feature value set of the prediction grid in the current frame; On the basis of the success of the first-level association, it is determined whether each prediction grid and the observation grid in the current frame meet the second-level association conditions; If the second level association is successful, then updating the second height dimension feature information of the prediction grid based on the first height dimension feature information of the observation grid associated therewith; If the second level association fails, an extrapolation operation is performed on the height dimension feature information of the prediction grid based on the state feature value set of the prediction grid in the current frame.

2. The point cloud data grid height dimension filtering method according to claim 1, characterized in that: The step of clustering the static point cloud within each observation grid according to the height value set to obtain at least one clustering target includes: Sort all static point clouds in the observation grid in ascending order of height value; Calculate the height difference between adjacent height values ​​and determine whether the height difference is less than a preset height difference threshold; When the height difference is less than the height difference threshold, the corresponding two static point clouds are classified into the same cluster target; When the height difference is greater than or equal to the height difference threshold, classifying the corresponding two static point clouds into different clustering targets; Height difference judgment and clustering are performed on all static point clouds in the observation grid until all static point clouds are divided into one or more cluster targets.

3. The point cloud data grid height dimension filtering method according to claim 1, characterized in that: The step of determining a state feature value set of each prediction grid in the current frame based on the state feature information of the prediction grid in the previous frame includes: Construct a set of state feature values ​​of the prediction grid; Based on the average position of the predicted grid in the previous frame and the posture change information of the vehicle between the previous and next frames, the average position of the predicted grid in the current frame is extrapolated to obtain the two-dimensional position of the predicted grid in the current frame and written into the state feature value set; The second two-dimensional feature information and the second high-dimensional feature information are written into the state feature value set.

4. The point cloud data grid height dimension filtering method according to claim 3, characterized in that: Determine whether each prediction grid and observation grid in the current frame meet the first-level association conditions, including: Based on the two-dimensional position of the predicted grid, determining whether the predicted grid falls within a valid area of ​​the current frame observation grid map; If so, further determining whether the number of static point clouds in the corresponding observation grid is greater than a preset first point cloud number threshold; If it is greater than the first point cloud quantity threshold, the first level association is deemed successful; If it is less than or equal to the first point cloud quantity threshold, determining whether the sum of the static point cloud quantities of the observation grids in the eight-connected neighborhood of the prediction grid in the observation grid map is greater than a preset second point cloud quantity threshold; If it is greater than the second point cloud quantity threshold, the first-level association is deemed successful, and the joint two-dimensional feature information of each observation grid in the eight-connected neighborhood is fused to perform an update operation; If none of the above conditions are met, the first-level association is deemed to have failed; On the basis of the success of the first-level association, it is determined whether each prediction grid and observation grid in the current frame meet the second-level association conditions, including: Extract the cluster targets of the predicted grid in the previous frame and the cluster targets in the observation grid associated with the current frame; Calculating the height difference between the cluster target of each prediction grid and the cluster target of the corresponding observation grid, and determining whether the height difference is less than a preset first height difference threshold; If it is less than the first height difference threshold, determine whether the height range of the cluster target of the prediction grid overlaps with the height range of the cluster target of the corresponding observation grid, and the overlapping condition is that the height difference is less than half of the sum of the cluster heights of the two cluster targets; If there is overlap, the cluster targets of the prediction grid that meet the above conditions are combined with the cluster targets of the observation grid to form a target pair, and a cost matrix is ​​constructed. The target pairs constitute the effective elements of the cost matrix, and the Hungarian algorithm is used to perform minimum cost matching on the target pairs to determine whether there is a matching relationship. If the match is successful, the second-level association is considered successful. If none of the above conditions are met, the second-level association is deemed to have failed.

5. The point cloud data grid height dimension filtering method according to claim 4, characterized in that: The method further includes the step of detecting the extrapolated state of the second height dimension feature information of the prediction grid, specifically including: When the first level association is successful, but all the second height dimension feature information of the prediction grid does not meet the second level association conditions and the extrapolation operation is not performed, or when the second two-dimensional feature information of the prediction grid has been extrapolated but the corresponding second height dimension feature information has not been extrapolated, trigger the extrapolation status detection of the second height dimension feature information of the prediction grid; The extrapolation state detection is used to ensure that at least one second high-dimensional feature information has completed extrapolation, specifically including: When the predicted grid in the previous frame has only one high-dimensional clustered target, directly performing an extrapolation operation on the second high-dimensional feature information of the high-dimensional clustered target; When the predicted grid in the previous frame contains multiple high-dimensional clustered objects, the second high-dimensional feature information of the high-dimensional clustered object with the highest probability of being occupied by the high-dimensional static point cloud is selected to perform an extrapolation operation.

6. The point cloud data grid height dimension filtering method according to claim 4, characterized in that: The method further includes the step of detecting a mature state of the second height dimension feature information of the prediction grid, specifically comprising: When the second two-dimensional feature information of the prediction grid is updated to a mature state, and all the second high-dimensional feature information of the prediction grid is not set to a mature state in the current frame, triggering a maturity state detection for the second high-dimensional feature information of the prediction grid; The maturity state detection is used to ensure that at least one second high-dimensional feature information is set to a mature state, specifically including: When the predicted grid in the previous frame has only one high-dimensional clustered target, directly setting the second high-dimensional feature information of the high-dimensional clustered target to a mature state; When the predicted grid in the previous frame contains multiple high-dimensional clustered objects, the high-dimensional clustered object with the highest probability of being occupied by the high-dimensional static point cloud is selected and its second high-dimensional feature information is set to a mature state.

7. The point cloud data grid height dimension filtering method according to claim 1, characterized in that: The step of performing grid regeneration for the observation grid that has not been successfully associated with any prediction grid in the current frame and initializing the state feature value set of the regenerated grid includes: For an observation grid that has not been successfully associated with any prediction grid in the current frame, determine whether a preset regeneration condition is met. If the regeneration condition is met, regenerate the observation grid as a new grid and initialize the state feature value set of the new grid; The new conditions include: New condition 1: If the number of dynamic point clouds of the observation grid is less than or equal to a preset third point cloud number threshold and the number of static point clouds of the observation grid is greater than a preset first point cloud number threshold, then the new condition is determined to be met; New condition 2: If the new condition 1 is not met, the total number of dynamic point clouds and the total number of static point clouds of each observation grid in the eight-connected neighborhood of the prediction grid in the observation grid map are calculated. If the total number of dynamic point clouds is less than or equal to the fourth preset point cloud number threshold and the total number of static point clouds is greater than the second preset point cloud number threshold, then the new condition is determined to be met; If none of the above conditions are met, grid regeneration will not be performed.

8. An electronic device, characterized in that: The electronic device comprises: One or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor performs the point cloud data grid height dimension filtering method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that: When the computer program and / or instructions are executed by a processor, the method for filtering the grid height dimension occupied by point cloud data according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instruction is executed by a processor, the method for filtering the grid height dimension occupied by point cloud data as described in any one of claims 1 to 7 is implemented.

Images