Point cloud data occupied grid height dimension filtering method and related equipment

Abstract

The invention provides a point cloud data occupied grid height dimension filtering method and related equipment, and the method comprises the steps: projecting point cloud data into a two-dimensional grid map, and obtaining the two-dimensional feature information of each observation grid in the two-dimensional grid map; clustering the static point clouds in each observation grid according to the height value set to obtain at least one clustering target, and obtaining height dimension feature information of each clustering target; constructing a state feature value set of the prediction grid of the current frame based on the state feature information of the previous frame; performing two-stage association on the prediction grid and the observation grid, and updating or extrapolating a state feature value set based on an association result; if the observation grid is not associated with any prediction grid and meets a new condition, determining that the new grid is a new grid and initializing the state information of the new grid; and finally, invalid prediction grids are screened out. According to the method, the multiple height dimension clustering targets in the occupied grid can be accurately expressed, and the obstacle structure modeling capability in a complex multi-height scene is effectively improved.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and particularly to a method for filtering the height dimension of occupancy grids of point cloud data and related devices. Background Art

[0002] With the development of intelligent driving systems, environmental perception technology plays a key role in autonomous vehicles. Among them, lidar and millimeter-wave radar are widely used in obstacle detection and map construction. Lidar uses the principle of optical ranging and has high ranging accuracy, but its speed measurement ability is limited, and its performance drops significantly under harsh weather conditions such as rain and fog. At the same time, lidar has insufficient detection ability for distant targets. In contrast, millimeter-wave radar can use high-frequency electromagnetic waves to measure speed and distance, has strong anti-interference ability, can operate stably in complex weather environments, and has a detection range of more than 200 meters, making it more suitable for constructing a large-scale environmental perception model.

[0003] In the prior art, lidar or millimeter-wave radar can output three-dimensional point cloud data, which can theoretically be used to construct a three-dimensional occupancy grid map to more completely reflect the contour and edge information of ground obstacles (such as curbs, guardrails, bushes, etc.) and high-altitude obstacles (such as trees, buildings, gantries, etc.) in the environment. However, limited by the processing power of in-vehicle computing platforms, in practical applications, the method of projecting point clouds onto a two-dimensional plane is often adopted, and only the maximum value, minimum value, or average height value of the point cloud in each two-dimensional grid is extracted, and then a two-dimensional occupancy grid map is formed.

[0004] However, although this simplified processing method retains some height information, it severely compresses the information dimension in the vertical direction, resulting in the inability to accurately express the obstacle structure in the height dimension, and there are significant defects. Specifically, the two-dimensional occupancy grid can only reflect whether a certain area is occupied by an obstacle, but cannot accurately judge whether the occupancy occurs on the ground, at high altitude, or both. That is to say, when there are multiple targets at different heights appearing at the same horizontal position, due to the lack of multi-level occupancy information in the height dimension, the system cannot judge whether it affects the vehicle's passage, resulting in incorrect judgment of passability or incorrect identification of obstacles. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, this application provides a method for filtering the height dimension of occupancy grids of point cloud data and related devices, at least to solve the problem that the two-dimensional occupancy grid in the prior art cannot accurately express the information of multiple height obstacles in the height dimension, resulting in a decrease in the accuracy of the system in obstacle type recognition, spatial structure understanding, and passability judgment.

[0006] To achieve the above objectives and other advantages, some embodiments of this application provide the following aspects: In a first aspect, some embodiments of the present application provide a method for filtering the height dimension of occupancy grids of point cloud data, including: Project the point cloud data onto a two-dimensional grid map, and obtain first two-dimensional feature information of each observed grid in the two-dimensional grid map, where the first two-dimensional feature information includes: average position, number of static point clouds, number of dynamic point clouds, and height values of static point clouds; Cluster the static point clouds in each of the observed grids according to the set of height values to obtain at least one clustering target, and obtain first height dimension feature information of each of the clustering targets, where the first height dimension feature information includes: average height position, clustering height, and number of clustering static point clouds; Based on the state feature information of the predicted grids in the previous frame, determine a set of state feature values for each predicted grid in the current frame; Perform two-level association between the predicted grids and the observed grids, and update or extrapolate the set of state feature values of the predicted grids based on the association result, where the two-level association includes: the first-level association is grid-level matching based on two-dimensional positions, and the second-level association is matching based on height dimension clustering targets; Perform grid birth for the observed grids in the current frame that have not been successfully associated with any predicted grids, and initialize the set of state feature values of the newly born grids; Judge the validity of all predicted grids in the current frame, and remove the predicted grids judged to be invalid from the grid list of the current frame.

[0007] In a second aspect, some embodiments of the present application further provide an electronic device, where the electronic device includes: One or more processors; and a memory storing computer program instructions, where the computer program instructions, when executed, cause the processor to execute the method for filtering the height dimension of occupancy grids of point cloud data as described in any one of the above.

[0008] In a third aspect, some embodiments of the present application further provide a computer-readable storage medium, on which computer programs and / or instructions are stored, and when the computer programs and / or instructions are executed by a processor, the method for filtering the height dimension of occupancy grids of point cloud data as described in any one of the above is implemented.

[0009] In a fourth aspect, some embodiments of the present application further provide a computer program product, including computer programs and / or instructions, and when the computer programs / instructions are executed by a processor, the method for filtering the height dimension of occupancy grids of point cloud data as described in any one of the above is implemented.

[0010] Compared with related technologies, in the solution provided by the embodiments of the present application, on the basis of retaining the expression ability of the two-dimensional occupancy grid for the horizontal position and boundary contour of obstacles, a structured modeling mechanism for the height dimension is introduced. Through the height clustering process of the static point cloud in each two-dimensional grid, this method can effectively distinguish multiple height-dimensional obstacle targets existing at the same two-dimensional position, and further realizes the explicit distinction between ground targets and high-altitude targets. Therefore, the present application can accurately express whether there are ground obstacles, high-altitude obstacles, or both in the occupancy grid area, effectively improving the accuracy and robustness of the environmental perception system in complex multi-height scenarios such as tunnels, viaducts, and interchanges. Moreover, compared with directly constructing a complete three-dimensional occupancy grid map, the two-dimensional projection and local height-dimensional clustering method adopted by the present application does not require voxel encoding and three-dimensional space traversal processing for each point. Its actual clustering situation can be determined by grid merging and pruning, which can significantly reduce the number of height-dimensional grid tracking, thereby reducing the overall execution time of the algorithm and improving the operation efficiency, and is suitable for deployment on resource-constrained embedded platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other implementation manners can also be obtained based on these drawings.

[0012] Figure 1 is one of the schematic flowcharts of a method for filtering the height dimension of a point cloud data occupancy grid provided by an embodiment of the present application; Figure 2 is the second schematic flowchart of a method for filtering the height dimension of a point cloud data occupancy grid provided by an embodiment of the present application; Figure 3 is the schematic flowchart of the process of performing two-level association between the prediction grid and the observation grid and performing update extrapolation processing provided by an embodiment of the present application; Figure 4 is the schematic logical flowchart of determining whether the first-level association is successful provided by an embodiment of the present application; Figure 5 is the schematic logical flowchart of determining whether the second-level association is successful provided by an embodiment of the present application; Figure 6 is the schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0014] First Embodiment

[0015] The first embodiment of this application relates to a method for filtering the height dimension of occupancy grids of point cloud data. This method can be applied to a system, and the system can be installed in an electronic device. Referring to Figure 1 、 Figure 2 As shown, this method may include the following steps: Step S1: Project the point cloud data onto a two-dimensional grid map to obtain the first two-dimensional feature information of each observed grid in the two-dimensional grid map. The first two-dimensional feature information includes: average position, number of static point clouds, number of dynamic point clouds, and height values of static point clouds.

[0016] Regarding step S1, specifically, point cloud data refers to a set of discrete points on the surface of an object in a three-dimensional space collected by sensors (such as millimeter-wave radar, lidar). Each point usually contains spatial coordinate information (x, y, z) and attributes such as reflection intensity, and is used to represent the spatial structure in the environment.

[0017] A two-dimensional grid map refers to a regular grid map formed by projecting the point cloud data in a three-dimensional space onto a horizontal plane. Each grid cell represents a fixed area on the ground and is used to represent the occupancy information of obstacles in that area, facilitating subsequent processing and path planning.

[0018] An observed grid refers to a grid cell that can be observed by a lidar device after the point cloud data is projected onto a horizontal plane. Each grid cell records the state information such as the probability, category, or structural characteristics of the appearance of obstacles in that area and is the basic unit for constructing an environmental model. This lidar device can be a 4D millimeter-wave radar.

[0019] Exemplarily, a frame of three-dimensional point cloud data collected by a sensor is projected onto a horizontal two-dimensional plane to construct a two-dimensional grid map. Each point cloud contains typical fields of three-dimensional spatial coordinates (x, y, z). Optionally, it also contains information such as reflection intensity and velocity components. The point cloud is divided at a set grid resolution, and its two-dimensional grid coordinates are calculated to form a two-dimensional structured grid containing multiple observed grids. For each observed grid, the following first two-dimensional feature information is statistically extracted.

[0020] Average position (mean_x, mean_y): The arithmetic mean of the x and y coordinates of the point cloud where all projections fall within this grid, representing the geometric center position of this observation grid.

[0021] Number of static point clouds (spts_num): Based on the velocity information of each point cloud, if the velocity is lower than the set static threshold (e.g., 0.1 m / s), it is judged as a static point and included in the number of static point clouds to identify stable environmental structures.

[0022] Number of dynamic point clouds (dpts_num): If the point cloud velocity is higher than the static threshold, it is regarded as a dynamic point and included in the number of dynamic point clouds, used to identify moving objects such as vehicles and pedestrians.

[0023] Height value of the static point cloud: Extract the z values of the height values of all points identified as static points, constituting the input for the height dimension analysis of this grid, used for subsequent height dimension clustering operations.

[0024] Step S2: Cluster the static point clouds within each observation grid according to the set of height values to obtain at least one clustering target, and obtain the first height dimension feature information of each clustering target. The first height dimension feature information includes: average height position, clustering height, and number of clustering static point clouds.

[0025] Regarding step S2, specifically, for each observation grid, based on the set of height values of the static point cloud extracted from this grid, perform height dimension clustering operations according to the preset clustering rules, so as to divide the point clouds with continuous height and spatially dense distribution characteristics into one or more clustering targets. The clustering rules can include similarity judgment based on whether the adjacent difference of the point cloud height value is less than the set clustering interval threshold, or can be achieved through sequential traversal, sliding window segmentation, or density analysis, etc. The specific implementation method can be flexibly selected according to the system performance requirements.

[0026] After clustering, for each clustering target, further extract its first height dimension feature information. The first height dimension feature information includes: the average height position meanz of this clustering target, used to describe its central position in the height dimension; the clustering height height, used to represent the vertical distribution range of this clustering target; and the number of clustering static point clouds clspts_num, used to measure the structural stability and spatial occupancy intensity of this clustering target. This enables the system to introduce a height dimension hierarchical expression on the basis of two-dimensional grid mapping, and can effectively identify the composite occupancy situation of ground obstacles and high-altitude structures.

[0027] Step S3: Based on the state feature information of the predicted grid in the previous frame, determine the set of state feature values of each predicted grid in the current frame.

[0028] Regarding step S3, specifically, relying solely on the observation data of a single frame may result in the loss or incompleteness of target information. Therefore, in order to achieve continuous tracking and dynamic prediction of the obstacle state during multi-frame continuous observation, the system extrapolates and corrects the spatial position of the prediction grid appropriately based on the state information recorded in the previous frame's prediction grid, combined with the motion information or pose change of the ego-vehicle between adjacent two frames. Meanwhile, the state feature information contained in the previous frame's prediction grid is inherited or updated as the state feature information of the current frame's prediction grid, thereby constructing the state feature value set of the current frame's prediction grid, providing basic data support for grid association, state matching, and grid birth in subsequent steps. That is to say, the prediction grid is not generated directly from the observation results of the current frame, but evolved from historical states, used to achieve the state maintenance and predictive tracking of the target obstacle between consecutive frames, which can effectively ensure the continuity and stability of the grid state in the time dimension, avoid target tracking deviations caused by discontinuous inter-frame data, and further improve the accuracy and robustness of the entire environmental perception system.

[0029] Step S4: Perform two-level association between the prediction grid and the observation grid, and update or extrapolate the state feature value set of the prediction grid based on the association result. Among them, the two-level association includes: the first-level association is grid-level matching based on two-dimensional position, and the second-level association is matching based on height-dimensional clustering targets.

[0030] Regarding step S4, specifically, this two-level association includes the first-level association and the second-level association. Among them, the first-level association is grid-level matching based on two-dimensional position, that is, to determine whether the positions of the prediction grid and the observation grid are close in the two-dimensional plane. If the matching is successful, the corresponding state of the prediction grid is updated according to the two-dimensional features of the observation grid. If the matching fails, state extrapolation is performed according to the state of the previous frame. The second-level association is matching based on height-dimensional clustering targets. On the basis of the successful first-level association, it is further determined whether the height-dimensional structures of the prediction grid and the corresponding observation grid match, so as to update or extrapolate the height-dimensional features of the prediction grid. This ensures the consistency of target matching at the two-dimensional plane and height-dimensional levels of the system, effectively improving the stability and tracking continuity of the grid state.

[0031] Step S5: Perform grid birth for the observation grids in the current frame that have not been successfully associated with any prediction grids, and initialize the state feature value set of the newly born grids.

[0032] Regarding step S5, specifically, after completing the double-layer association matching of the predicted grid and the observed grid, for the observed grids in the current frame that fail to be successfully associated with any predicted grid, the system will analyze these unmatched observed grids based on the set newborn determination conditions to determine whether they have the basis to be newborn as new grids. If the newborn conditions are met, the observed grid will be marked as a newborn grid (without historical prediction basis), and its set of state feature values will be initialized to be incorporated into the subsequent filtering, updating, and life cycle management of the occupancy grid. For the observed grids that do not meet the newborn conditions, there may be factors such as accidental noise, motion interference, and false detections interfering. The system will neither track nor save them temporarily, thus avoiding immediately establishing a tracking state for every newly emerged observation point or observed grid without screening, which is likely to cause misjudgment. Therefore, by setting reasonable newborn conditions, a grid newborn mechanism is introduced to balance the rapid response to environmental changes and the control of false detections.

[0033] Step S6: Judge the validity of all predicted grids in the current frame, and remove the predicted grids judged to be invalid from the grid list of the current frame.

[0034] Regarding step S6, specifically, the system makes a comprehensive judgment based on the key attributes included in the set of state feature values of the grid, including but not limited to the dynamic occupancy probability of the predicted grid and whether the spatial position exceeds the set two-dimensional grid map boundary. If a predicted grid has obvious dynamics in the current frame, that is, its dynamic point cloud occupancy probability is greater than 0, then this predicted grid may correspond to an unstable or moving target and is not suitable for continuous tracking as a static structure. The system will mark it as invalid and remove it from the grid list of the current frame; at the same time, if a predicted grid's two-dimensional position exceeds the set grid map range after state extrapolation or update, it will also be regarded as an invalid target and excluded. Through the grid post-processing step, the system can regularly clean up the invalid, ineffective, or out-of-bounds predicted grids, ensuring that only valid state information is retained in subsequent cycles, improving the operation efficiency and tracking accuracy of the entire occupancy grid filtering method, avoiding the accumulation of invalid data or mis-matching, and enhancing the overall stability and robustness of the perception system.

[0035] It is not difficult to find that, compared with the related technologies, in the solution provided by the embodiments of the present application, on the basis of retaining the expression ability of the two-dimensional occupancy grid for the horizontal position and boundary contour of obstacles, a structured modeling mechanism in the height dimension is introduced. Through the height clustering process of the static point cloud in each two-dimensional grid, this method can effectively distinguish multiple obstacle targets in the height dimension existing at the same two-dimensional position, and further realizes the explicit distinction between ground targets and high-altitude targets. Therefore, the present application can accurately express whether there are ground obstacles, high-altitude obstacles, or both in the occupancy grid area, effectively improving the accuracy and robustness of the environmental perception system in complex multi-height scenarios such as tunnels, elevated roads, and interchanges. Moreover, compared with directly constructing a complete three-dimensional occupancy grid map, the two-dimensional projection and local height-dimension clustering method adopted by the present application does not require voxel encoding and three-dimensional space traversal processing for each point. Its actual clustering situation can be determined by grid merging and pruning, which can significantly reduce the number of height-dimension grid tracking, thereby reducing the overall execution time of the algorithm and improving the operation efficiency, and is suitable for deployment on resource-constrained embedded platforms.

[0036] Second Embodiment The second embodiment of the present application relates to a method for filtering the height dimension of the occupancy grid of point cloud data. The second embodiment is an improvement based on the first embodiment. The specific improvement lies in: in the second embodiment of the present application, a specific implementation manner for height dimension clustering processing of the static point cloud in the observation grid is provided. That is, in step S2, the static point cloud in each observation grid is clustered according to the height value set to obtain at least one clustering target, which may further include the following steps: Step S201: Sort all the static point clouds in the observation grid in ascending order of height values; Step S202: Calculate the height difference between adjacent height values, and determine whether the height difference is less than a preset height difference threshold; Step S203: When the height difference is less than the height difference threshold, classify the corresponding two static point clouds into the same clustering target; Step S204: When the height difference is greater than or equal to the height difference threshold, classify the corresponding two static point clouds into different clustering targets; Step S205: Perform height difference judgment and clustering division on all the static point clouds in the observation grid until all the static point clouds are divided into one or more clustering targets.

[0037] In this embodiment, when processing each two-dimensional observation grid, the system will traverse the set of point clouds marked as "static" in the grid and extract their height values (i.e., the z - coordinate). Then sort these points in ascending order according to the height values to form a height sequence. Calculate the height difference between adjacent height values and determine whether this height difference is less than a preset height - difference threshold.

[0038] Exemplarily, the system records the sorted sequence of point - cloud height values as and calculates the height differences between adjacent point - clouds pair - by - pair from the beginning to the end . Subsequently, compare each height difference with the set height - difference threshold kClusterThr (for example, kClusterThr = 0.2 m) to determine whether adjacent points have vertical - structure continuity.

[0039] If , it indicates that these two point - clouds are closely distributed in the height direction and may belong to the same object surface or continuous structure. Therefore, the system classifies them as components of the current clustering target. If , it indicates that there is an obvious jump in the height direction of the current point - cloud compared with the previous structure, that is, it represents the structural boundary of different height layers. At this time, the system ends the construction of the current clustering and initializes a new clustering target, taking the current point - cloud as an element in the new clustering.

[0040] Repeat the above process to traverse and cluster the entire sorted height - value sequence until all static point - clouds are divided into one or more clustering targets. After the system completes this process, it constructs several height layers in each observation grid, and each clustering target represents an independent structure in the height dimension of the grid. After completing the height - clustering of the static point - clouds in the observation grid, the system further extracts the structural - feature information in the height dimension for each clustering target. The first height - dimension feature information extracted includes: the average height position (referring to the average value of the height values of all static point - clouds within the clustering target); the clustering height (referring to the height difference between the highest point and the lowest point within the clustering target, which can reflect the thickness or structural range of the clustering target in the vertical direction); the number of clustering static point - clouds (indicating the number of static points contained in the clustering target, which is a measure of the stability of the target and can also be used to filter out small noise - like clusters). By extracting these three feature values, the system can not only identify whether there are multi - layer structures in the vertical direction for each two - dimensional grid, but also assign clear positions, height ranges, and confidence levels to each layer of obstacles, thereby improving the expression ability and tracking accuracy of the entire occupancy - grid filtering method in three - dimensional space.

[0041] It is not difficult to find that in the embodiments of the present application, by setting a reasonable height difference threshold for clustering, it is possible to distinguish obstacle targets at multiple height levels within the same two-dimensional grid, such as a ground curb and a high-altitude crossbeam existing simultaneously. This method enables the system to no longer be limited to using simple statistical features such as maximum value, minimum value, or average value to represent height, and realizes a more refined description of the real space structure in the grid. Moreover, this clustering process is only based on height value sorting and difference judgment, without the need to introduce complex three-dimensional clustering algorithms or preset the number of clusters, with low computational complexity, and is easy to be deployed and run on platforms with high real-time requirements and limited resources such as vehicle-mounted processors.

[0042] Third Embodiment The third embodiment of the present application relates to a method for filtering the height dimension of point cloud data occupancy grids. The third embodiment is an improvement based on the first embodiment. The specific improvement lies in that: in the third embodiment of the present application, a specific implementation method for constructing a set of state feature values of the predicted grid is provided. That is, step S3 may further include the following steps: Step S301: Construct a set of state feature values of the predicted grid; Step S302: Based on the average position of the predicted grid in the previous frame, combined with the pose change information of the vehicle itself between the front and rear frames, extrapolate the coordinates of the average position of the predicted grid in the current frame to obtain the two-dimensional position of the predicted grid in the current frame and write it into the set of state feature values; Step S303: Take the number of static point clouds, the number of dynamic point clouds, the number of detection frames, the occupancy probability of static point clouds, the occupancy probability of dynamic point clouds, and the grid maturity flag bit of the predicted grid in the previous frame as the second two-dimensional feature information of the predicted grid in the current frame and write it into the set of state feature values; Step S304: Take the average position in the height dimension, the clustering height in the height dimension, the occupancy probability of static point clouds in the height dimension, and the grid maturity flag bit in the height dimension of each clustering target of the predicted grid in the previous frame as the second height dimension feature information of the predicted grid in the current frame and write it into the set of state feature values.

[0043] In this embodiment, for each predicted grid object retained in the previous frame, the system initializes its state feature value structure in the current frame to record its two-dimensional spatial attributes, dynamic statistical attributes, and height-dimensional structure information. Therefore, for each predicted grid in the constructed current frame, the set of state feature values includes the second two-dimensional feature information of the predicted grid in the planar position: the two-dimensional position information (x, y) of the predicted grid, the cumulative static point cloud quantity α_static, the cumulative dynamic point cloud quantity α_dynamic, the number of detection frames β, the static point cloud occupancy probability λ_static, the dynamic point cloud occupancy probability λ_dynamic, the grid maturity flag bit flg_mature, and the second height-dimensional feature information of the predicted grid in the height dimension: the average position z in the height dimension, the clustering height h in the height dimension, the static point cloud occupancy probability λ_static_z in the height dimension, and the grid maturity flag bit flg_zmature in the height dimension.

[0044] Among them, the two-dimensional position of the predicted grid in the current frame is extrapolated and calculated based on the average position of the predicted grid in the previous frame, combined with the pose change information (such as linear velocity and angular velocity) of the vehicle itself between the two frames. In the case of no special changes, the remaining feature information can directly inherit the state feature values of the corresponding predicted grid in the previous frame.

[0045] Exemplarily, the two-dimensional position of the predicted grid can be calculated according to the following formula:

[0046] Among them, represents the two-dimensional position vector of the predicted grid in the current frame, is the two-dimensional position vector of the predicted grid in the previous frame, A is the rotation matrix describing the position change relationship between the two frames, is the change vector of the vehicle position coordinates between the two frames.

[0047] It is not difficult to find that in the embodiment of the present application, the detailed structure of each predicted grid in the height direction is ensured to be retained and can be used in subsequent height-dimensional matching with the observed grid, ensuring the structural consistency and tracking ability between the upper and lower frames in the height-dimensional modeling process of the system.

[0048] It should be noted that the third embodiment of the present application can also be an improvement based on any one or more of the first embodiment to the second embodiment.

[0049] Fourth Embodiment The fourth embodiment of this application relates to a method for filtering the height dimension of occupancy grids in point cloud data. The fourth embodiment is an improvement based on the third embodiment. The specific improvement lies in: in the fourth embodiment of this application, a specific implementation manner of the state association and update process between the predicted grid and the observed grid based on a two-level association mechanism is provided. Refer to Figure 3 As shown, that is, step S4 may further include the following steps: Step S401: Determine whether each predicted grid and observed grid in the current frame satisfies the first-level association condition.

[0050] In some embodiments, refer to Figure 4 As shown, step S401 may further include the following steps: Step S4011: Based on the two-dimensional position of the predicted grid, determine whether the predicted grid falls into the valid area of the observed grid map of the current frame; Step S4012: If so, further determine whether the number of static point clouds in the corresponding observed grid is greater than a preset first point cloud number threshold; Step S4013: If it is greater than the first point cloud number threshold, it is determined that the first-level association is successful; Step S4014: If it is less than or equal to the first point cloud number threshold, determine whether the sum of the static point cloud numbers of the observed grids in the eight-connected neighborhood of the predicted grid in the observed grid map is greater than a preset second point cloud number threshold; Step S4015: If it is greater than the second point cloud number threshold, it is determined that the first-level association is successful, and the combined two-dimensional feature information of the observed grids in the eight-connected neighborhood is fused for update operations; Step S4016: If none of the above conditions are met, it is determined that the first-level association fails.

[0051] In this embodiment, the valid area of the observed grid map of the current frame is the area within the boundary formed by the grid cells with actual observed point cloud data in this two-dimensional grid map. This area represents the spatial range that the current frame perception system can see, and it is also the basis for judging whether the predicted grid can participate in the two-dimensional position grid-level matching (first-level association). If the two-dimensional position of a certain predicted grid falls outside this valid area, it means that it is in an area that the sensor cannot observe. At this time, if forced matching is carried out, it may cause mis-association. Therefore, grid extrapolation processing is required. Grid extrapolation refers to the process of predicting the two-dimensional spatial position and / or height dimension characteristics of the grid and continuing its existence state based on the grid state in the historical frame when there is a lack of direct observation data in the current frame.

[0052] Since there may be some grids in the current frame composed of only a very small number of point clouds, which are often spurious points caused by reflection errors, boundary effects, or noise. Therefore, setting the first point cloud number threshold kSPtsThr can exclude these unreliable observed grids and avoid misassociating the predicted grids. When the number of static point clouds in the observed grid reaches a certain number, satisfying spts_num > kSPtsThr, it indicates that the observation in this area has stability and spatial coherence, and this observed grid can be trusted as a reference to update the features of the predicted grid.

[0053] In the case of spts_num ≤ kSPtsThr, continue to try to find the observed grid in the eight-connected neighborhood of the predicted grid, accumulate the number of static point clouds tspts_num in the observed grids in the eight-connected neighborhood, and determine whether it exceeds the second point cloud number threshold kTSPtsThr. The eight-connected neighborhood refers to the eight grid cells directly adjacent to the current predicted grid in the two-dimensional grid map, including the upper, lower, left, right, and four diagonal directions. By statistically summing the number of static point clouds in all observed grids in this eight-connected neighborhood and performing joint feature extraction, in the case of insufficient point clouds in a single grid, the local neighborhood information is used to assist in completing the grid association judgment and state compensation.

[0054] If the sum of the number of static point clouds tspts_num in the eight-connected neighborhood exceeds kTSPtsThr, it is still determined that the first-level association is successful, and based on the joint two-dimensional feature information (such as average position, cumulative number of static point clouds, cumulative number of dynamic point clouds, etc.) of all observed grids in the eight-connected neighborhood, the two-dimensional feature information of the predicted grid is fused and updated. At the same time, accumulate the height values of the static point clouds in all observed grids in the eight-connected neighborhood, reorder and cluster them to obtain the average position in the height dimension, the clustering height, and the clustering number of static point clouds.

[0055] If the above conditions are not all satisfied, it is determined that the first-level association fails. At this time, the predicted grid cannot be updated based on the observation information of the current frame, but instead, it is transferred to the state extrapolation operation to further predict its state characteristic value.

[0056] Through this hierarchical judgment and neighborhood information fusion mechanism, this embodiment effectively improves the association success rate of grids in sparse or edge areas of point clouds, and at the same time avoids misdeletion or extrapolation misjudgment caused by occasional missing, enhancing the robustness and continuity of the entire filtering method.

[0057] Step S402: If the first-level association is successful, update the second two-dimensional feature information of the predicted grid based on the first two-dimensional feature information of the associated observed grid.

[0058] In this embodiment, after the prediction grid and the observation grid are successfully matched by the first association, the second two-dimensional feature information of the prediction grid can be updated based on the two-dimensional feature information of the observation grid. Specifically, the update can be performed in the following manner: According to the average two-dimensional position of the observation grid in the current frame, combined with the extrapolated position of the prediction grid in the previous frame, the prediction position is corrected using the gain matrix and the state transition matrix of the Kalman filter, so as to obtain the updated two-dimensional position of the prediction grid in the current frame. The calculation can be performed according to the following formula:

[0059] Wherein, is the updated two-dimensional position vector of the prediction grid, is the prediction position vector, is the two-dimensional position measurement value of the observation grid in the current frame, that is, the average position (meanx, meany), H is the state transition matrix, is the Kalman gain matrix. Through this update formula, the prediction state and the current observation can be effectively fused to improve the position update accuracy.

[0060] Subsequently, based on the number of static point clouds and the number of dynamic point clouds of the current observation grid, the number of static point clouds and the number of dynamic point clouds of the prediction grid are cumulatively updated, and the detection frame number of the prediction grid is updated accordingly. The calculation can be performed according to the following formula:

[0061] Wherein, α_static and α_dynamic respectively represent the currently accumulated number of static point clouds and the number of dynamic point clouds of the prediction grid, spts_num and dpts_num respectively represent the number of static point clouds and the number of dynamic point clouds of the associated observation grid in the current frame, and β is the number of frames in which the prediction grid is observed, which is used to measure the tracking persistence of the grid.

[0062] Calculate the static point cloud occupancy probability based on the updated number of static point clouds and the detection frame number, and calculate the dynamic point cloud occupancy probability based on the updated number of dynamic point clouds and the detection frame number. The calculation can be performed according to the following formula:

[0063] Wherein, λ_static and λ_dynamic respectively represent the current static point cloud occupancy probability and the dynamic point cloud occupancy probability of the prediction grid, which are important measurement indicators for the state credibility of the prediction grid.

[0064] Finally, judge the mature state of the prediction grid. If its detection frame number β is greater than the preset mature threshold kMatureThr, mark the grid as "mature". The grid in the mature state can be used for the subsequent environmental perception module.

[0065] Step S404: On the basis of successful first-level association, determine whether each predicted grid in the current frame and the observed grid meet the second-level association conditions.

[0066] In some embodiments, with reference to Figure 5 as shown, step S404 may further include the following steps: Step S4041: Extract the clustering targets of the predicted grid in the previous frame and the clustering targets in the observed grid associated with the current frame; Step S4042: Calculate the height difference between the clustering targets of each predicted grid and the clustering targets of the corresponding observed grid, and determine whether the height difference is less than a preset first height difference threshold; Step S4043: If it is less than the first height difference threshold, determine whether there is an overlap in the height ranges of the clustering targets of the predicted grid and the corresponding observed grid. The overlap condition is that the height difference is less than half of the sum of the clustering heights of the two clustering targets; Step S4044: If there is an overlap, form a target pair from the clustering targets of the predicted grid and the observed grid that meet the above conditions, construct a cost matrix, make the target pair the valid elements of the cost matrix, and use the Hungarian algorithm to perform minimum cost matching on the target pair to determine whether there is a matching relationship; Step S4045: If the matching is successful, it is determined that the second-level association is successful; Step S4046: If none of the above conditions are met, it is determined that the second-level association fails.

[0067] In this embodiment, for each predicted grid that has completed the first association, extract all the height-dimensional clustering targets corresponding to the previous frame, and at the same time extract all the clustering targets in the observed grid that is successfully first-associated with the predicted grid in the current frame. For each pair of clustering targets in the above two target sets, calculate the difference δz between their height centers, and determine whether it is less than a preset first height difference threshold kDistanceThr.

[0068] If δz < kDistanceThr, then further determine the overlap situation of this height-dimensional clustering pair, that is, whether the height difference δz is less than half of the sum of the two clustering heights, if δz < (h + height) / 2, where h is the height of the clustering target of the predicted grid and height is the height of the clustering target of the observed grid.

[0069] Form target pairs between all the predicted grids that meet the above two conditions and the clustering targets of the observed grids, and construct a cost matrix for matching the target pairs. The element values of the cost matrix can be calculated based on the difference in height means, the degree of clustering height overlap, or other preset structural similarity measurement methods. Use the Hungarian minimum weight matching algorithm to perform optimal matching on the cost matrix. If a matching relationship is successfully established for the target pair, it is determined that the second-level height dimension association between the predicted grid and the observed grid is successful; if there is no successfully matched target pair, it is determined that the height dimension association between the predicted grid and the observed grid fails.

[0070] Step S405: If the second-level association is successful, update the second height dimension feature information of the predicted grid based on the first height dimension feature information of the associated observed grid.

[0071] Specifically, based on the observed vector composed of the height mean and clustering height of the observed grid clustering target, and the height vector of the previous frame of the predicted grid clustering target, introduce the state transition matrix and gain matrix in the Kalman filter algorithm, perform height state update, and obtain the updated height dimension vector in the current frame, which can be calculated according to the following formula:

[0072] is the updated height position vector, including the average height position z and clustering height h of the clustering target in the current frame, is the height vector of the previous frame of the predicted grid, is the Kalman filter gain matrix, is the height dimension observed vector of the observed grid in the current frame, , T is the state transition matrix.

[0073] At the same time, update the occupancy probability of the static point cloud in the height dimension, and the formula is as follows:

[0074] Among them, is the occupancy probability of the static point cloud of the clustering target in the current frame, is the occupancy probability of the static point cloud of the clustering target in the previous frame, is the number of detection frames in the previous frame, clspts_num is the number of static point clouds included in this clustering target in the current frame, and β is the number of detection frames in the current frame.

[0075] When the second-level association is successful, by using the first height-dimensional feature information of the clustering target in the observation grid to update the second height-dimensional feature information of the prediction grid and correct the prediction error, it helps to accurately reflect the true state of the target in the height dimension. The updated height-dimensional information can be used as the prediction basis for subsequent frames to form a more stable and reliable temporal filtering trajectory, and can also accurately determine whether the target is a continuously existing obstacle, reducing the misrecognition rate and false alarm rate. Especially in an environment with multi-level structures such as viaducts, tunnels, and interchanges, continuous tracking of obstacles at different heights can be achieved.

[0076] Step S403: If the first-level association fails, based on the set of state feature values of the prediction grid in the current frame, perform extrapolation operations on both the second two-dimensional feature information and the second height-dimensional feature information of the prediction grid simultaneously.

[0077] For the extrapolated two-dimensional feature information, specifically, the extrapolation process for the second two-dimensional feature information can be performed on the premise of meeting the following extrapolation conditions, that is, the static point cloud occupancy probability of this prediction grid in the previous frame is greater than the set first extrapolation threshold kLambdaThr. The first extrapolation threshold kLambdaThr is not set fixedly and is dynamically adjusted according to the height position characteristics of the prediction grid and the different grid maturity flag bits. When the height position of the prediction grid is relatively high or its maturity state is unstable, the system can appropriately increase the set value of kLambdaThr, thereby raising the confidence threshold of extrapolation and reducing the occurrence of incorrect extrapolations; while when the prediction grid is in a near-ground position and has been continuously observed and marked as a mature state, the value of kLambdaThr can be appropriately reduced to enhance the timeliness and continuity of extrapolation.

[0078] When the above extrapolation conditions are met, the system will output the two-dimensional feature information of this prediction grid, specifically including: keeping the predicted two-dimensional position in the previous frame unchanged (as the predicted position in the current frame), and performing extrapolation operations on other features: The number of detection frames β increases: β += 1; The static point cloud occupancy probability λ_static is calculated according to the following formula:

[0079] where is the cumulative number of static point clouds in the previous frame.

[0080] The dynamic point cloud occupancy probability λ_dynamic is calculated according to the following formula:

[0081] where is the cumulative number of dynamic point clouds in the previous frame.

[0082] Meanwhile, the maturity flag of the predicted grid remains the same as that of the previous frame without state switching. If the above extrapolation conditions are not met, it is determined that the state characteristics of the predicted grid no longer have extrapolation value, and its state will be cleared, that is, it is regarded as the disappearance of two-dimensional feature information in the current frame and will no longer participate in the state update or trajectory tracking process of subsequent frames.

[0083] For the extrapolated height dimension feature information, specifically, if the predicted grid does not meet the second-level association condition with the observed grid in the current frame at the height dimension level, the system can perform the extrapolation operation of the height dimension feature based on the second height dimension feature information of the predicted grid in the previous frame. To ensure the reliability of the extrapolation result, the system sets the height dimension extrapolation condition, that is, it is required that the static point cloud occupancy probability of the height dimension clustering target in the previous frame is greater than the second extrapolation threshold kLambdaZThr, and this threshold can also be dynamically adjusted according to the height position of the grid and its maturity state.

[0084] When the above extrapolation conditions are met, the system will output the predicted height dimension feature information of the clustering target, specifically including: keeping the predicted height position z and height value h in the previous frame unchanged, that is, the spatial structure information of the height dimension clustering target in the current frame is consistent with that in the previous frame.

[0085] According to the static point cloud occupancy probability of the previous frame and the number of detection frames , calculate the static point cloud occupancy probability of the clustering target in the current frame according to the following formula :

[0086] where β is the number of detection frames in the current frame.

[0087] The maturity flag of the height dimension clustering target in the current frame is consistent with that in the previous frame, ensuring the continuity of the extrapolation state and the original state. If the above extrapolation conditions are not met, the system determines that the height dimension clustering target no longer has extrapolation value in the current frame, and its state will be cleared, that is, it is regarded as the disappearance of the height dimension clustering target in the current frame and will no longer participate in the state update or trajectory tracking process of subsequent frames.

[0088] By performing the extrapolation operation on the two-dimensional feature information and height dimension feature information of the predicted grid in the case of matching failure, the time consistency and structural integrity of the system in multi-frame continuous detection can be effectively enhanced, avoiding misidentification or trajectory interruption caused by information loss, which is beneficial for the system to continuously track and accurately express the state of obstacles.

[0089] Step S406: If the second-level association fails, extrapolate the height-dimensional feature information of the predicted grid based on the set of state feature values of the predicted grid in the current frame.

[0090] It should be noted that if the predicted grid and the observed grid are successfully matched in the two-dimensional position, but do not meet the second-level association condition at the height dimension level, that is, the clustering target of any observed grid cannot be successfully matched, the system can perform an extrapolation operation on the second height-dimensional feature information of each height-dimensional clustering target in the predicted grid based on the set of state feature values of the predicted grid in the current frame. For the specific extrapolation operation, please refer to the above description of extrapolating height-dimensional feature information, which will not be elaborated here.

[0091] Furthermore, in some embodiments, there is also a step of detecting the extrapolation state of the second height-dimensional feature information of the predicted grid, specifically including: When the first-level association is successful, and all the second height-dimensional feature information of the predicted grid does not meet the second-level association condition and no extrapolation operation has been performed, or when the second two-dimensional feature information of the predicted grid has been extrapolated but the corresponding second height-dimensional feature information has not been extrapolated, trigger the detection of the extrapolation state of the second height-dimensional feature information of the predicted grid; The extrapolation state detection is used to ensure that at least one second height-dimensional feature information is extrapolated, specifically including: When there is only one height-dimensional clustering target in the predicted grid in the previous frame, directly perform an extrapolation operation on the second height-dimensional feature information of the height-dimensional clustering target; When there are multiple height-dimensional clustering targets in the predicted grid in the previous frame, select the second height-dimensional feature information of the height-dimensional clustering target with the largest occupancy probability of the height-dimensional static point cloud to perform an extrapolation operation.

[0092] In this embodiment, in order to ensure the continuity and sustainability of the predicted grid at the height dimension level, after performing the update or extrapolation operation of the height-dimensional feature information, there is also a step of detecting the extrapolation state of the height-dimensional grid. This detection step is used to handle the following two situations: one is that the predicted grid in the current frame has passed the first-level association, but all its height-dimensional feature information fails to pass the second-level association and no extrapolation operation has been performed; the other is that the two-dimensional feature information of the predicted grid in the current frame has been extrapolated, but all its corresponding height-dimensional feature information has not been extrapolated. In such cases, the system will trigger the extrapolation state detection operation of the height-dimensional feature to ensure that the height-dimensional feature information of at least one clustering target can be extrapolated.

[0093] The extrapolation state detection process may include the following two scenarios: If the predicted grid contains only one height - dimension clustering target in the previous frame, the system directly performs an extrapolation operation on the second height - dimension feature information of this target; if there are multiple height - dimension clustering targets in the previous frame, the system may select the target with the highest probability according to the size of the height - dimension static point cloud occupancy probability and perform an extrapolation operation on the selected target.

[0094] By introducing the extrapolation state detection mechanism for height - dimension feature information, the problem of inconsistency between two - dimension feature information and height - dimension feature information can be effectively prevented. Especially when the predicted grid fails to establish a matching relationship with the current - frame observation grid at the height - dimension level, or when the association fails due to factors such as occlusion or intermittent loss of height - dimension clustering targets, this detection mechanism can ensure that at least one height - dimension feature information is continuously extrapolated, avoiding information breakage of the grid in the height - dimension.

[0095] Furthermore, in some embodiments, it further includes the step of performing a maturity state detection on the second height - dimension feature information of the predicted grid, specifically including: When the second two - dimension feature information of the predicted grid is updated to the mature state, while all the second height - dimension feature information of the predicted grid is not set to the mature state in the current frame, trigger the maturity state detection on the second height - dimension feature information of the predicted grid; The maturity state detection is used to ensure that at least one second height - dimension feature information is set to the mature state, specifically including: When there is only one height - dimension clustering target in the predicted grid in the previous frame, directly set the second two - dimension feature information of the height - dimension clustering target to the mature state; When there are multiple height - dimension clustering targets in the predicted grid in the previous frame, select the height - dimension clustering target with the highest height - dimension static point cloud occupancy probability and set its second height - dimension feature information to the mature state.

[0096] In this embodiment, to further improve the state consistency and environmental understanding ability of the system under multi - dimensional perception, after the two - dimension feature information of the predicted grid is updated to the mature state, the system further includes the step of performing a maturity state detection on the height - dimension feature information in this predicted grid. This detection mechanism is mainly used to solve the following inconsistent scenarios: that is, the scenario where the two - dimension feature information has been determined to be mature, while all its corresponding height - dimension feature information has not been set to the mature state. In this scenario, there may be no height clustering target with sufficient observation accumulation, or the height information fluctuates greatly and the number of observations is small, making it impossible to support its state to be judged as "mature". Such a contradictory phenomenon of "the area is very stable at the ground level, but its internal height structure is completely untrustworthy" is extremely unreasonable in the real physical environment.

[0097] Specifically, in the current frame, if the system determines that the number of detection frames corresponding to the second two-dimensional feature information of a certain predicted grid is greater than the maturity threshold and is thus set as mature, but among all the second height-dimensional feature information corresponding to this grid, no clustering target is set as mature, then this mature state detection mechanism will be triggered to ensure that at least one height-dimensional feature information also enters the mature state to maintain the logical consistency of information expression.

[0098] The mature state detection process may include the following two situations: If the predicted grid in the previous frame contains only one height-dimensional clustering target, the system directly sets the second height-dimensional feature information of this clustering target as the mature state. If the predicted grid in the previous frame contains multiple height-dimensional clustering targets, the system preferentially selects the target with the highest occupancy probability of the height-dimensional static point cloud as the representative target and sets the height-dimensional feature information of this target as the mature state.

[0099] This mechanism combines the consistency constraint relationship between the two-dimensional and height-dimensional feature states (that is, when the two-dimensional grid is mature, at least one of the corresponding height-dimensional grids should be mature), which helps to improve the physical rationality expression of the system in highly complex structure scenarios.

[0100] It is not difficult to find that in the embodiments of the present application, by introducing the dual correlation mechanism of two-dimensional and height-dimensional, the system can not only achieve a coarse-grained matching between the predicted grid and the observed grid based on the two-dimensional position, but also further perform fine-grained association of the clustering targets at the height-dimensional level, thereby significantly improving the accuracy and robustness of cross-frame target matching, and is particularly suitable for complex scenario environments with overlapping height layers. At the same time, an extrapolation update strategy is also introduced. When the predicted grid fails to successfully associate with the feature information of the observed grid in the current frame, the system can deduce its two-dimensional and height-dimensional feature information based on the previous frame state to ensure the continuity and sustainability of the grid state and avoid the disappearance of targets caused by short-term occlusion or observation failure.

[0101] It should be noted that the fourth embodiment of the present application can also be an improvement based on any one or more of the first to third embodiments.

[0102] The Fifth Embodiment The fifth embodiment of the present application relates to a method for filtering the height dimension of a point cloud data occupancy grid. The fifth embodiment is an improvement based on the first embodiment. The specific improvement lies in: In the fifth embodiment of the present application, a specific implementation manner of a grid newborn mechanism based on point cloud density determination is provided. That is, step S5 can further include the following steps: For an observation grid in the current frame that has not been successfully associated with any prediction grid, determine whether it meets the preset newborn condition. If the newborn condition is met, the observation grid is newborned into a newborn grid, and the set of state feature values of the newborn grid is initialized.

[0103] It should be noted that in the filtering and tracking system, the prediction grid represents the continuation of the state of the previous frame, while the observation grid is the original observation data newly obtained in this frame. When the two cannot be matched, it means that there is new observation information in this frame, but there is no existing prediction object in the system that can correspond to it. If these unassociated observation grids are not processed at this time, new obstacles or important environmental information may be ignored or lost. Therefore, the purpose of the grid newborn mechanism is to supplement the above-mentioned matching omissions. For example, some obstacles enter the radar observation field of view for the first time and there is no corresponding prediction grid yet, or some targets may not form a stable tracking trajectory in the previous frame due to occlusion or long-term stillness, and some are due to the tracking accuracy error in the previous frame resulting in incorrect matching.

[0104] Among them, the newborn conditions include: Newborn condition one: If the number of dynamic point clouds of the observation grid is less than or equal to the preset third point cloud number threshold and the number of static point clouds of the observation grid is greater than the preset first point cloud number threshold, it is considered that the newborn condition is met.

[0105] Newborn condition two: In the case where newborn condition one is not met, calculate the total number of dynamic point clouds and the total number of static point clouds of each observation grid in the eight-connected neighborhood of the prediction grid in the observation grid map. If the total number of dynamic point clouds is less than or equal to the preset fourth point cloud number threshold and the total number of static point clouds is greater than the preset second point cloud number threshold, it is considered that the newborn condition is met; Exemplarily, if the current observation grid meets the following conditions, the newborn operation is triggered: Newborn condition one: If the number of dynamic point clouds dpts_num of the observation grid is less than or equal to the set third point cloud number threshold kDPtsThr, and the number of static point clouds spts_num is greater than the set first point cloud number threshold kSPtsThr, that is, dpts_num ≤ kDPtsThr and spts_num > kSPtsThr.

[0106] Newborn condition one is judged based on the static intensity of a single observation grid. If the number of dynamic point clouds of the observation grid is small and the number of static point clouds is large, it is directly determined that the observation grid meets the newborn condition. This reduces unnecessary neighborhood traversal and calculation, and improves the response speed to stable obstacles.

[0107] Newborn Condition 2: If Newborn Condition 1 is not satisfied, then count the total number of dynamic point clouds tdpts_num and the total number of static point clouds tspts_num in the eight-connected neighborhood of the observed grid, and determine whether the total number of dynamic point clouds tdpts_num exceeds the set fourth point cloud number threshold kTDPtsThr. If the total number of dynamic point clouds tdpts_num is less than or equal to the fourth point cloud number threshold kTDPtsThr, and the total number of static point clouds tspts_num is greater than the set second point cloud number threshold kTSPtsThr, that is, tdpts_num ≤ kTDPtsThr and tspts_num > kTSPtsThr are satisfied.

[0108] Newborn Condition 2 is based on joint neighborhood judgment. When a single grid does not satisfy Newborn Condition 1, count the total number of dynamic point clouds and the total number of static point clouds of the observed grids in its eight-connected neighborhood. If the total number of dynamic point clouds is small and the total number of static point clouds exceeds the set threshold, it is considered that there are stable obstacles in the area and newborn is allowed. This avoids missing static targets due to sparse local point clouds and enhances the system's perception ability for obstacle edges and weak reflection areas.

[0109] If none of the above conditions are satisfied, grid newborn is not executed.

[0110] When a certain observed grid is determined to be a newborn grid, its two-dimensional feature information and height-dimensional feature information will be initialized.

[0111] Initialization of two-dimensional feature information:

[0112] Initialization of height-dimensional feature information:

[0113] It is not difficult to find that in the embodiments of the present application, a multiple new generation judgment logic including the first new generation condition and the second new generation condition is set, which can accurately identify and newly generate the observation grids in the current frame that have not been successfully associated with any predicted grids on the premise of ensuring sufficient static reliability of the new grids. When facing complex situations such as occlusion, sparse echoes, and long-distance detection in the actual road environment, it can flexibly decide whether to newly generate grids, so as to maintain the complete closed-loop of the system state update link. Specifically, on the one hand, through the joint judgment of the static point cloud quantity and the dynamic point cloud quantity, the false new generation caused by dynamic objects or temporary noises is effectively avoided, thereby improving the accuracy and stability of environmental modeling; on the other hand, even if a single grid does not meet the first new generation condition, the grid new generation can also be carried out based on the statistical characteristics of the eight-connected neighborhood, realizing the reasonable modeling of sparse obstacles or weak signal areas, and enhancing the continuity and integrity of environmental perception. In addition, the reasonable new generation control strategy avoids the large generation of invalid grids, helps to optimize the allocation of computing resources, and improves the overall real-time performance and execution efficiency of the system.

[0114] It should be noted that the fifth embodiment of the present application can also be an improvement based on any one or more of the first to fourth embodiments.

[0115] The step division of the above various methods is only for clear description. When implemented, they can be combined into one step or some steps can be split into multiple steps. As long as the same logical relationship is included, they are all within the protection scope of the present application; adding insignificant modifications to the algorithm or process or introducing insignificant designs, but not changing the core design of its algorithm and process are all within the protection scope of this application.

[0116] In addition, some embodiments of the present application also provide an electronic device. The electronic device can be various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and so on. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0117] The electronic device includes: one or more processors; and a memory storing computer program instructions, which when executed cause the processors to execute a method for filtering the height dimension of the occupancy grid of point cloud data as provided in any one or more of the above embodiments. Figure 6An exemplary structural diagram of the electronic device is disclosed. The electronic device includes: one or more processors 1101, a memory 1102, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. Each component is interconnected using different buses and can be mounted on a common motherboard or otherwise installed as required. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory for displaying graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories if needed. Similarly, multiple electronic devices can be connected, with each device providing part of the necessary operations. Among them, the components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0118] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103, and the output device 1104 can be connected by a bus or other means. Figure 6 Taking connection by bus as an example.

[0119] The input device 1103 can receive input digital or character information and generate key signal inputs related to the user settings and function controls of the electronic device, such as input devices like touchscreens, keypads, mice, trackpads, touchpads, pointing sticks, one or more mouse buttons, trackballs, joysticks, etc. The output device 1104 can include a display device, an auxiliary lighting device (such as an LED), and a haptic feedback device (such as a vibration motor), etc. The display device can include, but is not limited to, liquid crystal displays, light-emitting diode displays, and plasma displays. In some embodiments, the display device can be a touchscreen.

[0120] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device for displaying information to the user (such as a cathode ray tube or an LCD monitor); and a keyboard and a pointing device (such as a mouse), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (such as visual feedback, auditory feedback); and input from the user can be received in any form (such as voice input or tactile input).

[0121] In the embodiments of the present application, a computer program / instructions is stored on a computer-readable medium. When the computer program / instructions is executed by a processor, it implements a method for filtering the height dimension of a point cloud data occupancy grid provided by any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist alone without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions.

[0122] The memory 1102 can be used as a non-transitory computer-readable storage medium, and can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules. By running the non-transitory software programs, instructions, and modules stored in the memory 1102, the processor 1101 executes various functional applications and data processing of the server, so as to implement the program instructions / modules corresponding to the methods provided by any one or more of the above embodiments in the embodiments of the present application.

[0123] The memory 1102 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the electronic device. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 1102 may optionally include a memory remotely set relative to the processor 1101, and these remote memories may be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0124] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0125] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, compact disc read-only memory, digital versatile disc or other optical storage, magnetic cassette tapes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0126] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network or a wide area network, or it can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0127] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. For example, application-specific integrated circuits, general-purpose computers, or any other similar hardware devices can be used. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, or a floppy disk and similar devices. Additionally, some steps or functions of this application can be implemented by hardware, for example, as a circuit that cooperates with the processor to execute each step or function.

[0128] The computer program product provided by the embodiments of the present application includes one or more computer programs / instructions. When the computer programs / instructions are executed by a processor, they wholly or partially generate the processes or functions described in the embodiments of the present application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrating one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc.

[0129] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0130] The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes that fall within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be construed as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the apparatus claims may also be implemented by one unit or device through software or hardware. The words "first", "second", etc. are only used for distinguishing descriptions and do not represent any specific order, nor can they be understood as indicating or implying relative importance.

[0131] As described above, the foregoing are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily make changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for filtering the grid height dimension occupied by point cloud data, characterized in that: include: Projecting the point cloud data into a two-dimensional grid map, obtaining first two-dimensional feature information of each observation grid in the two-dimensional grid map, wherein the first two-dimensional feature information includes: average position, number of static point clouds, number of dynamic point clouds, and height value of static point clouds; Clustering the static point cloud in each observation grid according to the height value set to obtain at least one cluster target, and acquiring first height dimension feature information of each cluster target, wherein the first height dimension feature information includes: average height position, cluster height, and number of clustered static point clouds; Based on the state feature information of the prediction grid in the previous frame, determine the state feature value set of each prediction grid in the current frame; The prediction grid is associated with the observation grid in two levels, and the state feature value set of the prediction grid is updated or extrapolated based on the association result, wherein the two-level association includes: the first level association is a grid-level matching based on a two-dimensional position, and the second level association is a matching based on a height-dimensional clustering target; For the observation grids that have not been successfully associated with any prediction grids in the current frame, grid regeneration is performed, and the state feature value set of the new grid is initialized; The validity of all prediction grids in the current frame is judged, and the prediction grids judged to be invalid are removed from the grid list of the current frame.

2. The point cloud data grid height dimension filtering method according to claim 1, characterized in that: The step of clustering the static point cloud in each observation grid according to the height value set to obtain at least one clustering target comprises: Sort all static point clouds in the observation grid in ascending order according to their height values; Calculate the height difference between adjacent height values, and determine whether the height difference is less than a preset height difference threshold; When the height difference is less than the height difference threshold, the corresponding two static point clouds are classified into the same cluster target; When the height difference is greater than or equal to the height difference threshold, classifying the corresponding two static point clouds into different clustering targets; Height difference judgment and clustering are performed on all static point clouds in the observation grid until all static point clouds are divided into one or more cluster targets.

3. The point cloud data grid height dimension filtering method according to claim 1, characterized in that: The step of determining a state feature value set of each prediction grid in the current frame based on the state feature information of the prediction grid in the previous frame comprises: Construct a set of state feature values ​​of the prediction grid; Based on the average position of the predicted grid in the previous frame and combined with the posture change information of the vehicle between the previous and next frames, the average position of the predicted grid in the current frame is extrapolated to obtain the two-dimensional position of the predicted grid in the current frame and write it into the state feature value set; The number of static point clouds, the number of dynamic point clouds, the number of detection frames, the static point cloud occupancy probability, the dynamic point cloud occupancy probability and the grid maturity flag of the predicted grid in the previous frame are used as the second two-dimensional feature information of the predicted grid in the current frame and written into the state feature value set; The height-dimensional average position, height-dimensional cluster height, height-dimensional static point cloud occupancy probability and height-dimensional grid maturity flag of each cluster target of the predicted grid in the previous frame are used as the second height-dimensional feature information of the predicted grid in the current frame and written into the state feature value set.

4. The point cloud data grid height dimension filtering method according to claim 3, characterized in that: The step of performing two-level association between the prediction grid and the observation grid, and updating or extrapolating the state feature value set of the prediction grid based on the association result, comprises: Determine whether each predicted grid and observed grid in the current frame meet the first-level association conditions, including: Based on the two-dimensional position of the prediction grid, determining whether the prediction grid falls within a valid area of ​​the current frame observation grid map; If so, further determine whether the number of static point clouds in the corresponding observation grid is greater than a preset first point cloud number threshold; If it is greater than the first point cloud quantity threshold, the first level association is deemed successful; If it is less than or equal to the first point cloud quantity threshold, then determining whether the sum of the static point cloud quantities of each observation grid in the eight-connected neighborhood of the prediction grid in the observation grid map is greater than a preset second point cloud quantity threshold; If it is greater than the second point cloud quantity threshold, the first level association is deemed successful, and the joint two-dimensional feature information of each observation grid of the eight-connected neighborhood is fused to perform an update operation; If none of the above conditions are met, the first-level association is deemed to have failed; If the first level association is successful, the second two-dimensional feature information of the prediction grid is updated based on the first two-dimensional feature information of the observation grid associated with the prediction grid; If the first level association fails, based on the state feature value set of the prediction grid in the current frame, simultaneously extrapolating the second two-dimensional feature information and the second high-dimensional feature information of the prediction grid; On the basis of the success of the first level association, it is determined whether each predicted grid and the observed grid in the current frame meet the second level association conditions, including: Extract the cluster targets of the predicted grid in the previous frame and the cluster targets in the associated observation grid in the current frame; Calculate the height difference between the cluster target of each prediction grid and the cluster target of the corresponding observation grid, and determine whether the height difference is less than a preset first height difference threshold; If it is less than the first height difference threshold, determine whether there is an overlap between the height ranges of the cluster targets of the prediction grid and the corresponding cluster targets of the observation grid, where the overlapping condition is that the height difference is less than half of the sum of the cluster heights of the two cluster targets; If there is overlap, the clustering targets of the prediction grid that meet the above conditions and the clustering targets of the observation grid are formed into target pairs, and a cost matrix is ​​constructed. The target pairs constitute the effective elements of the cost matrix. The Hungarian algorithm is used to perform minimum cost matching on the target pairs to determine whether there is a matching relationship. If the match is successful, the second-level association is considered to be successful. If none of the above conditions are met, the second-level association is deemed to have failed; If the second level association is successful, then based on the first height dimension feature information of the observation grid associated therewith, the second height dimension feature information of the prediction grid is updated; If the second level association fails, an extrapolation operation is performed on the height dimension feature information of the prediction grid based on the state feature value set of the prediction grid in the current frame.

5. The point cloud data grid height dimension filtering method according to claim 4, characterized in that: The step of detecting the extrapolated state of the second height dimension feature information of the prediction grid is also included, specifically including: When the first level association is successful, but all the second height dimension feature information of the prediction grid does not meet the second level association condition and the extrapolation operation is not performed, or when the second two-dimensional feature information of the prediction grid has been extrapolated but the corresponding second height dimension feature information has not been extrapolated, triggering the extrapolation state detection of the second height dimension feature information of the prediction grid; The extrapolation state detection is used to ensure that at least one second high-dimensional feature information has completed extrapolation, specifically including: When the predicted grid in the previous frame has only one high-dimensional clustered target, directly performing an extrapolation operation on the second high-dimensional feature information of the high-dimensional clustered target; When the predicted grid in the previous frame contains multiple high-dimensional clustered targets, the second high-dimensional feature information of the high-dimensional clustered target with the highest probability of being occupied by the high-dimensional static point cloud is selected to perform an extrapolation operation.

6. The point cloud data grid height dimension filtering method according to claim 4, characterized in that: The step of detecting the maturity state of the second height dimension feature information of the prediction grid is also included, specifically including: When the second two-dimensional feature information of the prediction grid is updated to a mature state, and all the second high-dimensional feature information of the prediction grid is not set to a mature state in the current frame, triggering a mature state detection for the second high-dimensional feature information of the prediction grid; The mature state detection is used to ensure that at least one second high-dimensional feature information is set to a mature state, specifically including: When the predicted grid in the previous frame has only one high-dimensional clustered target, directly setting the second two-dimensional feature information of the high-dimensional clustered target to a mature state; When the predicted grid in the previous frame contains multiple high-dimensional clustered targets, the high-dimensional clustered target with the highest probability of being occupied by the high-dimensional static point cloud is selected and its second high-dimensional feature information is set to a mature state.

7. The point cloud data grid height dimension filtering method according to claim 1, characterized in that: The step of performing grid regeneration for the observation grid that is not successfully associated with any prediction grid in the current frame and initializing the state feature value set of the new grid includes: For an observation grid that has not been successfully associated with any prediction grid in the current frame, determine whether a preset regeneration condition is met; if the regeneration condition is met, regenerate the observation grid as a regenerated grid, and initialize a state feature value set of the regenerated grid; The new conditions include: New condition 1: if the number of dynamic point clouds of the observation grid is less than or equal to the preset third point cloud number threshold and the number of static point clouds of the observation grid is greater than the preset first point cloud number threshold, then the new condition is determined to be met; New condition 2: When the new condition 1 is not met, the total number of dynamic point clouds and the total number of static point clouds of each observation grid in the eight-connected neighborhood of the prediction grid in the observation grid map are calculated. If the total number of dynamic point clouds is less than or equal to the preset fourth point cloud number threshold and the total number of static point clouds is greater than the preset second point cloud number threshold, the new condition is determined to be met; If none of the above conditions are met, grid regeneration will not be performed.

8. An electronic device, characterized in that: The electronic device comprises: One or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the point cloud data occupancy grid height dimension filtering method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that: When the computer program and / or instruction is executed by a processor, the point cloud data occupancy grid height dimension filtering method as described in any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or instruction is executed by a processor, the method for filtering the grid height dimension occupied by point cloud data as described in any one of claims 1 to 7 is implemented.

Images