Dynamic obstacle point cloud filtering method and device

Abstract

The embodiment of the invention discloses a dynamic obstacle point cloud filtering method. The method comprises the following steps: acquiring a frame image containing a target area and inertial measurement data; using an iterative error Kalman filtering algorithm to obtain second point cloud data and pose information of the frame image; determining and obtaining third point cloud data under the global coordinate system by using a registration algorithm; dividing the spatial range covered by the third point cloud data into grid units, and counting the number of point clouds and the height variance of the point clouds in each grid unit; if the number of the point clouds exceeds a first threshold value and an absolute value of a variance difference value between the number of the point clouds and the number of historical frame point clouds exceeds a second threshold value, determining that the grid is a dynamic grid; if the dynamic confidence is lower than a set threshold value, the dynamic grids are marked as static grids again; and obtaining a point cloud of the dynamic obstacle through a spatial region growth algorithm, and filtering the point cloud of the dynamic obstacle to obtain static point cloud data of the target region. According to the invention, effective filtering of the dynamic obstacle point cloud is realized.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and device for extracting a dynamic obstacle point cloud. Background Art

[0002] In autonomous driving and robot navigation, LiDAR is often used to obtain three-dimensional point cloud data of the surrounding environment in real time. Point cloud data contains static obstacles (such as buildings, roads, fixed facilities, etc.) and dynamic obstacles (such as pedestrians, vehicles, animals, etc.). In a dynamic environment, how to efficiently and accurately remove dynamic obstacles has become an important issue in point cloud processing.

[0003] Existing methods usually rely on the separation of static environment modeling and dynamic obstacle detection, but they still have difficulties in real-time processing and application in complex environments. Therefore, how to accurately and in real time remove dynamic obstacles from point cloud data is an important technical problem that needs to be solved. Summary of the invention

[0004] The present application provides a method and device for filtering a dynamic obstacle point cloud, which can effectively filter out a dynamic obstacle point cloud.

[0005] This application provides the following solutions:

[0006] According to a first aspect, a method for filtering out a dynamic obstacle point cloud is provided, the method comprising: acquiring more than one frame image containing a target area and inertial measurement data corresponding to each of the frame images; generating first point cloud data for each of the frame images; using an iterative error Kalman filter algorithm, fusing the first point cloud data with the inertial measurement data to obtain second point cloud data and the pose information of the frame image; using a registration algorithm, determining a pose transformation matrix of the frame image relative to a reference frame according to the pose information; using the pose transformation matrix, converting the second point cloud data from a local coordinate system to a global coordinate system for representation, and obtaining third point cloud data in the global coordinate system; in the global coordinate system, dividing the spatial range covered by the third point cloud data into more than one grid units, and counting the number of point clouds and the point cloud height variance in each of the grid units; and judging whether the grid unit is a point cloud according to the point cloud height variance using a plane fitting algorithm. If it is not a ground grid unit, the third point cloud data judged as a ground grid unit is filtered out; if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold, the grid unit is marked as a dynamic grid; the historical frame information of the dynamic grid is obtained, and the historical frame information is the number of point clouds and the point cloud height variance of one or more frame images before the current frame image corresponding to the current dynamic grid in the dynamic grid; the historical frame information is processed by a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than the set threshold, the mark of the dynamic grid is modified to a static grid; for the grid unit marked as a dynamic grid, the point cloud of the dynamic obstacle is obtained by a spatial region growing algorithm, and the point cloud of the dynamic obstacle is filtered out from the third point cloud data to obtain the static point cloud data of the target area.

[0007] According to an achievable method in an embodiment of the present application, the use of a registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame based on the pose information includes: acquiring point cloud data of the reference frame; generating an initial pose transformation matrix of the second point cloud data and the reference frame point cloud data using a point-to-point iterative closest point algorithm; and optimizing the initial pose transformation matrix using a gradient descent algorithm to obtain the pose transformation matrix.

[0008] According to an achievable method in an embodiment of the present application, in the global coordinate system, dividing the spatial range covered by the third point cloud data into more than one grid units includes: generating a three-dimensional bounding box for the third point cloud data in the global coordinate system, non-uniformly dividing the bounding box along three mutually perpendicular coordinate axis directions in the three-dimensional coordinate system, and adaptively adjusting the grid resolution according to the distribution characteristics of the third point cloud data; assigning a multidimensional index to each of the grid units, and assigning the third point cloud data to the corresponding grid units.

[0009] According to an achievable method in an embodiment of the present application, the use of time series analysis to process the historical frame information and calculate the dynamic confidence of the dynamic grid includes: constructing a point cloud quantity time series and a point cloud height variance time series based on the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information; calculating the rate of change, stability and change trend of the point cloud quantity and height variance in the dynamic grid based on the point cloud quantity time series and the point cloud height variance time series; defining a dynamic confidence formula based on the change rate, stability and change trend, and calculating the dynamic confidence based on the dynamic confidence formula.

[0010] According to an achievable method in an embodiment of the present application, the plane fitting algorithm is used to determine whether the grid unit is a ground grid unit based on the point cloud height variance, and the third point cloud data determined to be a ground grid unit is filtered out, including: based on the point cloud data in each of the grid cells, a plane model is fitted using a random sampling consistency algorithm to obtain a fitting plane; the vertical distance between each point in the third point cloud data and the fitting plane is calculated, and whether it is a ground point is determined based on a distance threshold, and points that meet the distance threshold condition are removed from the point cloud data; wherein the distance threshold is dynamically adjusted based on the point cloud height variance.

[0011] According to an achievable method in an embodiment of the present application, for the grid unit marked as a dynamic grid, a point cloud of a dynamic obstacle is obtained through a spatial region growing algorithm, and the point cloud of the dynamic obstacle is filtered out from the third point cloud data, including: from the dynamic grid, an initial seed point cloud is selected according to a preset indicator, and a region is expanded from the initial seed point cloud according to a distance metric and a point cloud density function, and adjacent point clouds are included in the expanded region to form a continuous point cloud region; and a clustering algorithm is used to divide the continuous point cloud region into multiple independent regions, and the point cloud of the dynamic obstacle is filtered out from the independent regions.

[0012] According to an achievable method in an embodiment of the present application, defining a dynamic confidence formula based on the change rate, stability and change trend includes: dividing into multiple different time intervals; obtaining timestamp information of the historical frame information, and dividing the historical frame information into corresponding time intervals based on the timestamp information; setting different weights for the change rate, stability and change trend in different time intervals based on the importance differences of the number of point clouds and height variance in different time intervals, and determining the dynamic confidence formula based on the weights.

[0013] According to a second aspect, a dynamic obstacle point cloud filtering device is provided, the device comprising: a data acquisition unit, configured to acquire more than one frame image containing a target area and inertial measurement data corresponding to each of the frame images; a data fusion unit, configured to generate first point cloud data for each of the frame images; using an iterative error Kalman filter algorithm, the first point cloud data is fused with the inertial measurement data to obtain second point cloud data and the pose information of the frame image; a coordinate conversion unit, configured to use a registration algorithm to determine the pose transformation matrix of the frame image relative to a reference frame according to the pose information; using the pose transformation matrix, the second point cloud data is converted from a local coordinate system to a global coordinate system for representation, and third point cloud data in the global coordinate system is obtained; a grid data statistics unit, configured to divide the spatial range covered by the third point cloud data into more than one grid units in the global coordinate system, and count the number of point clouds and the point cloud height variance in each of the grid units; a ground point cloud filtering unit, configured to use the point cloud height variance to obtain the ground point cloud data; A plane fitting algorithm is used to determine whether the grid unit is a ground grid unit, and the third point cloud data in the grid unit determined to be a ground grid unit is filtered out; a dynamic grid determination unit is configured to mark the grid unit as a dynamic grid if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold; historical frame information of the dynamic grid is obtained, and the historical frame information is the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the frame image corresponding to the current dynamic grid; the historical frame information is processed by a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than a set threshold, the mark of the dynamic grid is modified to a static grid; a dynamic point cloud filtering unit is configured to obtain a point cloud of a dynamic obstacle for the grid unit marked as a dynamic grid through a spatial region growth algorithm, and filter the point cloud of the dynamic obstacle from the third point cloud data to obtain static point cloud data of the target area.

[0014] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0015] According to a fourth aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, and the program instructions, when read and executed by the one or more processors, execute the steps of any one of the methods described in the first aspect.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0017] (1) This method generates the first point cloud data by acquiring multiple frame images containing the target area and their corresponding inertial measurement data, and obtains the second point cloud data and the pose information of the frame image by fusing them with the inertial measurement data through the iterative error Kalman filter algorithm. The registration algorithm is used to further determine the pose transformation matrix and convert the data to the global coordinate system. Through grid processing and statistical analysis, dynamic grids are identified, and the dynamic confidence is calculated in combination with time series analysis, so as to accurately identify and filter out the point cloud of dynamic obstacles. The advantage of this method is that it improves the detection accuracy of dynamic obstacles and reduces interference in environmental modeling.

[0018] (2) This application uses a point-to-point iterative closest point algorithm and a gradient descent optimization algorithm to calculate and optimize the pose transformation matrix, which further enhances the spatial consistency and accuracy of the point cloud data, significantly reduces errors in the data alignment process, and improves the stability and accuracy of subsequent processing.

[0019] (3) When performing grid division in the global coordinate system, the present application uses three-dimensional bounding boxes and non-uniform resolution adjustment technology to more accurately reflect the distribution characteristics of point cloud data. This method makes the analysis of each grid unit more detailed, improves the detection and resolution of dynamic grids, and further enhances the distinction between static and dynamic obstacles.

[0020] (4) This application calculates the dynamic confidence of the dynamic grid through time series analysis. This method can effectively evaluate the dynamic behavior of each grid unit and reduce the misjudgment rate. The introduction of dynamic confidence makes the determination of the dynamic grid more scientific and avoids the problem of misjudgment or missed detection caused by environmental changes.

[0021] (5) The plane fitting method based on the random sampling consistency algorithm of this application can accurately identify the ground grid and remove the ground points. This method dynamically adjusts the distance threshold to adapt to different environmental conditions, ensuring the accurate removal of ground points, thereby improving the filtering efficiency and point cloud data quality.

[0022] (6) This application uses a spatial region growing algorithm to filter out point clouds from dynamic grids. This method can effectively separate dynamic obstacles from complex point clouds. A clustering algorithm is used to subdivide the point cloud area to ensure that dynamic obstacles can be accurately detected and filtered out. This technology can significantly improve the real-time and reliability of environmental perception systems.

[0023] (7) This application improves the sensitivity to the dynamic changes of dynamic grids by defining the dynamic confidence formula and adjusting the weight of the time interval. Combining the timestamp information of historical data for analysis makes the dynamic confidence closer to the changes in the actual environment, which helps to more accurately identify and process dynamic grids and improves the overall performance of the system.

[0024] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A system architecture diagram applicable to the embodiments of the present application;

[0027] Figure 2 A flow chart of a method for filtering a dynamic obstacle point cloud provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of the process of the dynamic obstacle point cloud filtering method provided in an embodiment of the present application;

[0029] Figure 4 A structural block diagram of a dynamic obstacle point cloud filtering device provided in an embodiment of the present application;

[0030] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0032] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0033] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0034] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0035] There are already some technical methods for dynamic obstacle point cloud filtering, most of which rely on motion detection, model prediction and other technologies. However, these methods require additional computing resources and complex algorithm design, and the accuracy of point cloud filtering is low in complex dynamic scenes.

[0036] In view of this, the present application provides a new idea. In order to facilitate the understanding of the present application, the system architecture on which the present application is based is first described. Figure 1 An exemplary system architecture to which the embodiments of the present application can be applied is shown. Figure 1 As shown in , the system architecture may include: a user device and a dynamic obstacle point cloud filtering device located on the server side.

[0037] The user can input frame images and inertial measurement data through the user device, and send them to the dynamic obstacle point cloud filtering device on the server side through the user device. The dynamic obstacle point cloud filtering device can use the method provided in the embodiment of the present application to perform dynamic obstacle point cloud filtering on the point cloud data in the frame image to obtain static point cloud data of the target area. The server side can send the static point cloud data of the target area to the user terminal in response to the request of the user terminal, and the user terminal uses the three-dimensional real scene model for rendering to obtain a two-dimensional image, a three-dimensional image, a VR (virtual reality) scene or an AR (augmented reality) scene, etc.

[0038] Among them, user equipment may include but is not limited to: smart mobile terminals, smart home devices, wearable devices, PC (Personal Computer), etc. Among them, smart mobile devices may include such as mobile phones, tablet computers, laptops, PDAs (Personal Digital Assistants), Internet cars, etc. Smart home devices may include smart TVs, smart refrigerators, etc. Wearable devices may include such as smart watches, smart glasses, virtual reality devices, augmented reality devices, mixed reality devices (i.e., devices that can support virtual reality and augmented reality), etc.

[0039] The dynamic obstacle point cloud filtering device can be set up as an independent server, a server group, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system to solve the problems of difficult management and weak service scalability in traditional physical hosts and virtual private servers (VPS). Figure 1 In addition to the shown architecture, the dynamic obstacle point cloud filtering device can also be set on a computer terminal with strong computing power.

[0040] It should be understood that Figure 1 The user equipment and the dynamic obstacle point cloud filtering device in the figure are only illustrative. According to the implementation requirements, there may be any number of user equipment and dynamic obstacle point cloud filtering devices.

[0041] Figure 2 The flowchart of the dynamic obstacle point cloud filtering method provided in the embodiment of the present application can be Figure 1 The dynamic obstacle point cloud filtering device in the system shown is executed. Figure 2 As shown in , the method may include the following steps:

[0042] Step 201: Obtain one or more frame images containing a target area and inertial measurement data corresponding to each of the frame images.

[0043] Step 202: For each of the frame images, generate first point cloud data; use an iterative error Kalman filter algorithm to fuse the first point cloud data with the inertial measurement data to obtain second point cloud data and the pose information of the frame image.

[0044] Step 203: Using a registration algorithm, determine the pose transformation matrix of the frame image relative to the reference frame according to the pose information; using the pose transformation matrix, transform the second point cloud data from the local coordinate system to the global coordinate system for representation, and obtain the third point cloud data in the global coordinate system.

[0045] Step 204: In the global coordinate system, the spatial range covered by the third point cloud data is divided into one or more grid units, and the number of point clouds and the point cloud height variance in each grid unit are counted.

[0046] Step 205: According to the point cloud height variance, a plane fitting algorithm is used to determine whether the grid unit is a ground grid unit, and the third point cloud data determined to be a ground grid unit is filtered out.

[0047] Step 206: If the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold, the grid unit is marked as a dynamic grid; historical frame information of the dynamic grid is obtained, the historical frame information being the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the frame image corresponding to the current dynamic grid; the historical frame information is processed using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than a set threshold, the mark of the dynamic grid is changed to a static grid.

[0048] Step 207: For the grid cells marked as dynamic grids, a point cloud of a dynamic obstacle is obtained by using a spatial region growing algorithm, and the point cloud of the dynamic obstacle is filtered out from the third point cloud data to obtain static point cloud data of the target area.

[0049] As can be seen from the above process, this method generates the first point cloud data by acquiring multiple frame images containing the target area and their corresponding inertial measurement data, and obtains the second point cloud data and the pose information of the frame image by fusing iterative error Kalman filter algorithm with the inertial measurement data. The registration algorithm is used to further determine the pose transformation matrix and convert the data to the global coordinate system. Through gridding processing and statistical analysis, dynamic grids are identified, and the dynamic confidence is calculated in combination with time series analysis, so as to accurately identify and filter out the point cloud of dynamic obstacles. The advantage of this method is that it improves the detection accuracy of dynamic obstacles and reduces interference in environmental modeling. By fusing multiple frames of images and inertial measurement data, combined with time series analysis and spatial region growth algorithm, the accuracy of dynamic obstacle detection is improved.

[0050] The following is a detailed description of each step in the above process and the effects that can be further produced in combination with the embodiments. It should be noted that the "first" and "second" and other limitations involved in the present disclosure do not have limitations in terms of size, order and quantity, and are only used to distinguish in name, for example, "first point cloud data" and "second point cloud data" are used to distinguish two groups of point cloud data.

[0051] First, the above step 201, namely "obtaining more than one frame image including the target area and inertial measurement data corresponding to each frame image", is described in detail in conjunction with the embodiment.

[0052] In this application, the target area is the area where dynamic point cloud filtering needs to be performed, and the frame image contains the target area. The frame image is a continuous image frame, which can represent the viewing angle information in a specific area or scene. Preferably, a laser radar can be used to obtain frame images. The laser radar device completes a scan within a fixed time interval to generate a frame of data. Different laser radar devices have different scanning speeds, and the scanning frequency (the number of frames scanned per second) can range from tens to hundreds of frames.

[0053] Inertial measurement data comes from an inertial measurement unit (IMU), which is an electronic device used to measure and report the motion state of an object in three-dimensional space. It usually includes an accelerometer and a gyroscope, and sometimes a magnetometer. Inertial measurement data contains the acceleration, angular velocity, and other motion parameters of the device at a specific point in time. By obtaining this data, the frame image can be matched with its corresponding motion information, thereby achieving higher-precision point cloud data processing. After obtaining the inertial measurement data, the inertial measurement data can be preprocessed, such as denoising, low-pass filtering, and time synchronization, to improve the accuracy of the inertial measurement data.

[0054] The above step 202, namely "generating first point cloud data for each of the frame images; fusing the first point cloud data with the inertial measurement data using an iterative error Kalman filter algorithm to obtain second point cloud data and the pose information of the frame image" is described in detail below in conjunction with an embodiment.

[0055] In this application, the first point cloud data is the original point cloud data collected by the laser radar, which is generated by the laser radar device by scanning a specific environmental area. Each frame image represents the spatial data obtained by the laser radar device at a certain point in time, including the distance and direction information of the laser beam reflected from the target area.

[0056] The iterative error Kalman filter algorithm is a recursive filtering method for fusing multi-source data, and is particularly suitable for processing noise and errors in dynamic systems. The iterative error Kalman filter algorithm can reduce the estimation error by predicting and updating the system state, and gradually approach the true value in multiple iterations. In this application, it is used to fuse the first point cloud data generated by the lidar with the inertial measurement data of the IMU to improve the spatial accuracy of the point cloud data and the estimation accuracy of the device posture.

[0057] Figure 3 The process diagram of the dynamic obstacle point cloud filtering method provided in the embodiment of the present application is that after the first point cloud data is fused with the IMU data through the iterative error Kalman filtering algorithm, an improved second point cloud data is generated, which is more accurate in space and conforms to the actual position and posture of the device. At the same time, the state information output by the filtering algorithm provides the pose information of the frame image, including the position and posture of the device. Specifically, the acceleration and angular velocity data of the IMU can be used to predict the position and posture of the device at the current time point. After that, the predicted pose is compared with the first point cloud data, and the pose estimation is corrected through error calculation and filter update steps to ensure that the output pose information is more in line with the actual situation. And the filter state is optimized through multiple iterations, so that the pose estimation and point cloud data are improved in accuracy. Finally, the second point cloud data output by the filter contains the corrected point cloud information, reflecting the fused environment space. The pose information generated at the same time includes the device position and posture of the frame image during the acquisition process, such as position coordinates and orientation angles, etc.

[0058] The following is a detailed description of step 203, namely, "using a registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame based on the pose information; using the pose transformation matrix to transform the second point cloud data from the local coordinate system to the global coordinate system for representation, and obtaining the third point cloud data in the global coordinate system" in conjunction with the embodiment.

[0059] The registration algorithm is a method for spatially aligning multi-frame point cloud data. According to the pose information of the frame image, the registration algorithm can calculate the pose transformation matrix between the current frame image and the reference frame. The reference frame refers to a specific frame point cloud or coordinate system used as a reference in a series of point cloud data processing processes. The data of other frames are aligned with the reference frame through pose estimation to ensure the uniformity of the point cloud data in space. The reference frame can be a selected static frame or point cloud data at a certain moment in a dynamic environment, usually represented in the form of a global coordinate system. The present application can be implemented by a variety of registration algorithms, such as the Iterative Closest Point (ICP) algorithm and the Normalized Distribution Transform (NDT) algorithm, which can calculate the relative displacement and rotation relationship between the two frames of data by matching similar feature points in the point cloud data. Specifically, feature points (such as edge points and plane points) can be extracted from the second point cloud data and the reference frame point cloud for matching calculations. The ICP algorithm is used to match the feature points through the nearest neighbor search and calculate the initial transformation matrix; the high-precision posture transformation matrix is ​​further obtained through error optimization (such as the least square method). Finally, the posture transformation matrix describing the frame image relative to the reference frame is obtained.

[0060] The pose transformation matrix is ​​a matrix that describes the motion of a rigid body. It contains a rotation matrix and a translation vector, which is used to represent the change in the position and orientation of a frame of data in space relative to the reference frame. The pose transformation matrix is ​​generally in the form of a 4×4 homogeneous matrix, in which the rotation part defines the attitude change and the translation part defines the position change.

[0061] As an implementable method, the present application can use a point-to-point iterative closest point algorithm to obtain a pose transformation matrix. Specifically, obtain the point cloud data of the reference frame; use a point-to-point iterative closest point algorithm to calculate the initial pose transformation matrix of the second point cloud data and the reference frame point cloud data; use a gradient descent algorithm to optimize the initial pose transformation matrix to obtain the pose transformation matrix.

[0062] Among them, the point-to-point iterative closest point (ICP) algorithm is first used to match the closest point pairs in the two groups of point clouds, construct an error function to describe the geometric deviation between the two groups of point clouds, and iteratively optimize the error function to obtain the initial pose transformation matrix. The initial pose transformation matrix includes a translation matrix and a plane rotation matrix, which are used to preliminarily align the two groups of point clouds. Then, the gradient descent algorithm is used to optimize the above initial pose transformation matrix. Specifically, the optimization objective function is constructed according to the output result of the initial pose transformation matrix. The objective function is usually the sum of the squares of the point-to-point distances between the two groups of point clouds. The parameters in the pose transformation matrix (including rotation angle and translation) are continuously adjusted by the gradient descent algorithm, and the value of the objective function is iteratively optimized until the preset error threshold or the number of iterations is reached, and finally the optimized pose transformation matrix is ​​obtained. The optimized pose transformation matrix can accurately describe the pose relationship of the current frame point cloud data relative to the reference frame point cloud data, and serve as the core parameter when the subsequent point cloud data is transformed to the global coordinate system.

[0063] The local coordinate system is a reference coordinate system with the lidar sensor as the origin, while the global coordinate system is a unified coordinate system that describes the entire environment. Through the pose transformation matrix, the point cloud data in the local coordinate system can be converted to the global coordinate system for representation, ensuring the consistency of the spatial position between each frame of data. Specifically, each point of the second point cloud data is transformed through the pose transformation matrix, and the formula is:

[0064] P global =T·P local (1)

[0065] Among them, P local is the point cloud coordinate in the local coordinate system, T is the pose transformation matrix, P global is the point cloud coordinate in the global coordinate system.

[0066] The point cloud data after coordinate transformation is uniformly represented in the global coordinate system, and the data of all frames can be seamlessly merged to form the third point cloud data. This data meets the requirements of environmental modeling in terms of spatial consistency and accuracy.

[0067] The above step 204, namely "dividing the spatial range covered by the third point cloud data into one or more grid units in the global coordinate system, and counting the number of point clouds and the point cloud height variance in each of the grid units" is described in detail below in conjunction with the embodiments.

[0068] In the process of dynamic obstacle detection, in order to accurately analyze the point cloud data in the global coordinate system, the spatial range of the point cloud data needs to be divided. Specifically, the point cloud data in the global coordinate system will cover a certain three-dimensional spatial range, and this range usually contains all the point clouds in the target area. By dividing the covered spatial range into multiple grid cells, the point cloud data can be locally grouped to more effectively analyze the distribution characteristics of the point cloud in different spatial areas. Each grid cell is a three-dimensional cube area that is used to contain the point cloud data in the area.

[0069] Among them, the division of grid units can be carried out in a variety of ways, for example, uniform division along three mutually perpendicular coordinate axes in the three-dimensional coordinate system according to a fixed resolution. Preferably, the present application can perform non-uniform division, generate a three-dimensional bounding box for the third point cloud data of the global coordinate system, perform non-uniform division on the bounding box along three mutually perpendicular coordinate axes in the three-dimensional coordinate system, and adaptively adjust the grid resolution according to the distribution characteristics of the third point cloud data; assign a multidimensional index to each of the grid units, and assign the third point cloud data to the corresponding grid unit.

[0070] Specifically, a three-dimensional bounding box is generated for the third point cloud data in the global coordinate system, and the spatial range of the entire point cloud can be wrapped with the smallest three-dimensional rectangular box to ensure that all point cloud data are contained in the bounding box. The boundaries of the bounding box are determined by the maximum and minimum coordinate values ​​of the point cloud data, and extend along the X, Y and Z axes of the three-dimensional coordinate system, respectively, thereby defining the spatial distribution range of the point cloud in the global coordinate system.

[0071] Based on the three-dimensional bounding box, the bounding box can be divided into multiple grid units by performing non-uniform division along the three mutually perpendicular coordinate axes of X, Y and Z. The non-uniform division process is based on the distribution characteristics of the point cloud data. For example, in areas with high point cloud density, smaller grid units are used to improve resolution; while in areas with sparse point clouds, larger grid units can be used to reduce the amount of calculation. By analyzing the density and distribution characteristics of the point cloud, the resolution of the grid can be adaptively adjusted to make the division more targeted and efficient.

[0072] Each generated grid cell is assigned a unique multi-dimensional index, which can represent the position of the grid cell in the three-dimensional coordinate system. For example, a triple (i, j, k) is used to represent a grid cell, where i, j, and k are the serial numbers of the grid cells in the X, Y, and Z axis directions, respectively. Then, each point in the third point cloud data is assigned to the corresponding grid cell according to its coordinate value. Specifically, by judging the coordinates of the point, the specific grid cell in which it falls within the bounding box is determined, thereby mapping the point cloud data to a three-dimensional grid structure.

[0073] After the division is completed, the point cloud in each grid unit is statistically analyzed. The statistical indicators include the number of point clouds in each grid unit and the point cloud height variance. The number of point clouds indicates the number of points contained in the grid unit, reflecting the local density characteristics of the point cloud; the point cloud height variance indicates the degree of distribution dispersion of the point cloud in the height direction within the grid, which is used to judge the uniformity and hierarchy of the point cloud distribution. These statistical information can lay the foundation for the subsequent dynamic obstacle extraction.

[0074] Among them, the point cloud height variance The calculation formula is:

[0075]

[0076] Among them, h(p j ) is point p j Height value, N(g i ) is the number of point clouds, is the grid g i The mean height of all points in The calculation formula is:

[0077]

[0078] The above step 205, namely "determining whether the grid unit is a ground grid unit by using a plane fitting algorithm according to the point cloud height variance, and filtering out the third point cloud data determined to be a ground grid unit" is described in detail below in conjunction with an embodiment.

[0079] First, for the spatial range of the global coordinate system covered by the third point cloud data, potential ground grid cells are preliminarily screened based on the characteristics of the point cloud height variance. The point cloud height variance reflects the uniformity of the point cloud distribution in the vertical direction (Z axis) within the grid cell. When the height variance is small, it means that the point cloud has a small change in the height direction and has a flat characteristic close to the ground. By setting a height variance threshold, grid cells with a height variance below the threshold are marked as candidate ground grids.

[0080] Then, in the candidate ground grid, the plane fitting algorithm is applied to further determine whether it is a ground grid unit. Specifically, the plane fitting algorithm builds a plane model and calculates the distance from the point cloud to the plane by fitting the spatial distribution of the point cloud data in the grid unit. The fitting plane can be solved by the least squares method, and its formula is:

[0081] ax+by+cz+d=0 (4)

[0082] Among them, a, b, c are the normal vector parameters of the plane, and d is the offset. The point cloud data in the grid cell is fitted using the least squares method, and the vertical distance from each point to the fitted plane is calculated. If the mean or maximum distance from all point clouds to the fitted plane is lower than the preset threshold, the grid cell can be considered to meet the ground characteristics.

[0083] For point cloud data judged as ground grid cells, they are further filtered out to avoid interference of ground point clouds with subsequent processing (such as obstacle identification or environment modeling). Specifically, these ground point clouds are removed from the third point cloud dataset, thereby retaining non-ground point clouds for higher-precision spatial analysis and processing.

[0084] As an implementable method, the present application can use a random sampling consistency algorithm to fit a plane model based on the point cloud data in each of the grid units to obtain a fitting plane; calculate the vertical distance between each point in the third point cloud data and the fitting plane, determine whether it is a ground point based on a distance threshold, and remove points that meet the distance threshold condition from the point cloud data; wherein the distance threshold is dynamically adjusted based on the point cloud height variance.

[0085] Specifically, first, for the point cloud data in each grid cell, the distribution characteristics of the point cloud in the vertical direction (Z axis) in the grid cell are evaluated according to the point cloud height variance. When the point cloud height variance is lower than the preset standard, the grid cells with potential ground characteristics are preliminarily screened out as candidates for subsequent processing.

[0086] In the candidate grid cells, the random sampling consensus algorithm is applied to fit the plane model to the point cloud data. The random sampling consensus algorithm randomly selects a subset of the point cloud data as the fitting sample, iteratively constructs possible plane models, and calculates the number of inliers of the model, that is, the number of point clouds that satisfy the plane model within a certain tolerance range. By maximizing the number of inliers, the optimal plane model is determined to describe the overall plane characteristics of the grid cell.

[0087] After obtaining the fitted plane, calculate the perpendicular distance between each point and the plane. The perpendicular distance is defined as the shortest distance from the point to the plane, and its formula is:

[0088]

[0089] Among them, a, b, c are the normal vector parameters of the fitted plane, x, y, z are the point cloud coordinates, and d is the offset of the plane.

[0090] The distance threshold is dynamically adjusted according to the point cloud height variance to identify ground points. Specifically, when the height variance is small, it means that the height variation range of the point cloud is small, and the distance threshold can be set to a lower value to improve the accuracy of ground point recognition; when the height variance is large, it means that the height distribution of the point cloud is more dispersed, and the distance threshold can be appropriately relaxed to cover more point clouds that may belong to the ground.

[0091] Finally, for the point cloud that meets the distance threshold condition, that is, the point cloud whose vertical distance is less than the dynamically adjusted threshold, it is judged as a ground point and removed from the third point cloud data. The non-ground point cloud data is retained for further analysis, such as dynamic obstacle detection or environment modeling.

[0092] The following is a detailed description of step 206 in conjunction with an embodiment, namely, "if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold, marking the grid unit as a dynamic grid; obtaining historical frame information of the dynamic grid, the historical frame information being the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the frame image corresponding to the current dynamic grid; processing the historical frame information using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than a set threshold, changing the mark of the dynamic grid to a static grid."

[0093] When determining whether a grid unit is a dynamic grid, the present application first makes a preliminary judgment on whether the grid unit is a dynamic grid by using a first threshold and a second threshold, and then further judges the dynamic grid by calculating the dynamic confidence.

[0094] Among them, if the number of point clouds in the current frame exceeds the first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds the second threshold, the grid unit may contain dynamic objects, such as moving vehicles or pedestrians. At this time, the grid unit is marked as a dynamic grid to indicate that it has greater variability or instability. The historical frame of the present application may refer to the previous frame data of the current frame. When calculating the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame, it can only be judged whether the absolute value of the variance difference between the number of point clouds in the current frame and the previous frame exceeds the second threshold; the historical frame of the present application may also refer to the previous multiple frames of data of the current frame, and the absolute value of the variance difference between the number of point clouds in the current frame and the previous multiple frames is calculated respectively. When the multiple absolute values ​​obtained all exceed the second threshold, it is determined that the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds the second threshold.

[0095] In order to further evaluate the actual dynamics of the dynamic grid, the historical frame information of the dynamic grid is extracted from the historical frame. The historical frame information refers to the number of point clouds and the point cloud height variance contained in the previous one or more frames of the current dynamic grid in the time series. Through these historical data, the change trend of the dynamic grid in the time dimension can be analyzed to provide a basis for subsequent dynamic judgment.

[0096] By processing the historical frame information of the dynamic grid using time series analysis methods, its changing patterns in the time dimension can be captured. Time series analysis can include algorithms such as moving average, exponentially weighted average or autoregressive analysis. These analyses help calculate the dynamic confidence of the dynamic grid, that is, the probability that it has dynamic characteristics. Dynamic confidence is usually a quantitative indicator used to indicate the degree of dynamicity of the grid unit.

[0097] As an implementable method, the present application uses time series analysis to process historical frame information, and calculates the dynamic confidence of the dynamic grid, including: constructing a point cloud quantity time series and a point cloud height variance time series based on the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information; calculating the rate of change, stability and change trend of the point cloud quantity and height variance in the dynamic grid based on the point cloud quantity time series and the point cloud height variance time series; defining a dynamic confidence formula based on the change rate, stability and change trend, and calculating the dynamic confidence based on the dynamic confidence formula.

[0098] Specifically, first, two time series are constructed, namely, the time series of point cloud quantity and the time series of point cloud height variance, according to the number of point clouds and the point cloud height variance recorded in the historical frame information corresponding to the dynamic grid. These two time series represent the changes in the number of point clouds and the height variance in the dynamic grid in continuous time frames.

[0099] Next, based on the time series of point cloud quantity and point cloud height variance, the change rate, stability and change trend of the point cloud quantity and height variance of the dynamic grid are calculated respectively. Among them, the change rate represents the change amplitude of the adjacent frame data in the time series, which is used to quantify the degree of fluctuation of the point cloud quantity or height variance; stability describes whether the point cloud quantity or height variance fluctuates within a certain range by analyzing the variance or standard deviation of the time series; the change trend uses trend analysis methods (such as linear regression or polynomial fitting) to determine whether the time series has a significant growth or decline trend.

[0100] Based on these calculation results, a dynamic confidence formula is defined. The dynamic confidence formula can be constructed by comprehensively considering the change rate, stability and change trend through a weighted method. For example, the dynamic confidence can be expressed as:

[0101] D=w1·R+w2·(1-S)+w3·T (6)

[0102] Among them, R represents the rate of change, S represents stability, T represents the trend of change, and w1, w2, and w3 are weight coefficients used to balance the impact of different factors on the dynamic confidence. According to this formula, the dynamic confidence of the dynamic grid is calculated to obtain a quantitative value.

[0103] Furthermore, the present application can also divide multiple different time intervals; obtain the timestamp information of the historical frame information, and divide the historical frame information into corresponding time intervals according to the timestamp information; according to the importance difference of the number of point clouds and height variance in different time intervals, set different weights for the change rate, stability and change trend in different time intervals, and determine the dynamic confidence formula according to the weights.

[0104] Specifically, first, the timestamp information is extracted according to the historical frame information of the dynamic grid, and the historical frame data is divided into multiple time intervals according to the time dimension. The division of time intervals can be flexibly set according to specific application scenarios, such as evenly dividing into the nearest short time interval, medium time interval and distant time interval. The grouping rules can also be dynamically adjusted according to task requirements to balance the impact of real-time and historical data on dynamic confidence calculation.

[0105] In each time interval, three key characteristics are calculated for the historical changes in the number of point clouds and height variance: rate of change, stability, and trend of change. Since different time intervals may contribute differently to dynamic judgment, it is necessary to set weights for the characteristics of each interval based on the importance difference. For example, data from the most recent time interval has a stronger impact on dynamic judgment and can be given a higher weight; data from a more distant time interval may provide auxiliary information for trend judgment and have a relatively lower weight. The weight value can be set empirically or optimized based on actual scenarios.

[0106] Based on the above analysis, the dynamic confidence formula is defined, and the dynamic confidence is calculated by comprehensively adding the weighted values ​​of the change rate, stability and change trend of each time interval. The formula can be expressed as:

[0107]

[0108] Among them, C d is the dynamic confidence; i represents the time interval; n is the total number of time intervals; ω i is the weight of the time interval; R i , S i , T i They respectively represent the rate of change, stability and change trend in the i-th time interval; α, β, γ are the weight coefficients of the characteristic items, which are set according to specific needs.

[0109] If the dynamic confidence obtained by time series analysis is lower than the pre-set threshold, it indicates that the dynamic grid is not dynamic enough to continue to be considered as a dynamic state, for example, it may be a misjudgment caused by sensor noise or other abnormal conditions. At this time, the dynamic grid can be re-labeled as a static grid, thereby reducing misjudgments of dynamic scenes and optimizing the processing accuracy of point cloud data.

[0110] Preferably, when making a dynamic grid determination, the present application can also determine whether the grid is a newly added scene. If the historical frame information of the grid shows that the grid has never been occupied, the grid is considered to belong to a new scene, not a dynamic obstacle, and the grid is marked as a newly added grid. Among them, whether the grid is occupied can be determined by judging the number of point clouds of the grid in the historical data. If the number of point clouds is zero or lower than a preset threshold, the grid is considered to be unoccupied.

[0111] The above step 207, i.e., "for the grid cells marked as dynamic grids, obtaining the point cloud of dynamic obstacles through a spatial region growing algorithm, and filtering the point cloud of the dynamic obstacles from the third point cloud data to obtain static point cloud data of the target area," is described in detail below in conjunction with an embodiment.

[0112] In the present application, firstly, based on the position of the dynamic grid, the point cloud within it is used as the initial seed point cloud to execute the spatial region growing algorithm. The region growing algorithm is a commonly used clustering method, which mainly adds adjacent points with similar characteristics to the same set in a recursive or iterative manner. In a specific implementation, it is possible to determine whether the point clouds belong to the same region based on the spatial distance between the point clouds and the similarity of the normal vectors. For example, a spatial distance threshold and a normal vector angle threshold are set, and only when both are met, the point will be included in the current region. This method can divide the point cloud data within the dynamic grid into multiple connected regions, which are further used to extract a complete dynamic obstacle point cloud.

[0113] Secondly, according to the dynamic characteristics of the dynamic grid (such as areas with high dynamic confidence), the identified connected areas are marked as dynamic obstacle point cloud data. Dynamic obstacles usually have significant spatial variation characteristics, such as vehicles, people, animals, etc. These point cloud data represent dynamic objects that change over time in the target area and need to be filtered out when modeling the environment.

[0114] Then, the extracted dynamic obstacle point cloud is removed from the third point cloud data in the global coordinate system to eliminate the interference of dynamic objects on the environment modeling. In a specific implementation, the point cloud data marked as dynamic obstacles can be removed or shielded point by point through point cloud indexing or coordinate matching, thereby retaining the static point cloud data.

[0115] Finally, the static point cloud data of the target area is obtained. These static point cloud data are not disturbed by dynamic objects and can more accurately reflect the fixed environmental characteristics of the target area, providing a basis for further applications such as 3D environment modeling, path planning or object detection.

[0116] As an implementable method, the present application filters out point clouds of dynamic obstacles for the dynamic grid through a spatial region growing algorithm, including: selecting an initial seed point cloud from the dynamic grid according to preset indicators, expanding the region from the initial seed point cloud according to a distance metric and a point cloud density function, incorporating adjacent point clouds into the expanded region to form a continuous point cloud region; using a clustering algorithm to divide the continuous point cloud region into multiple independent regions, and filtering out point clouds of dynamic obstacles from the independent regions.

[0117] Specifically, the selection of the initial seed point cloud is based on preset indicators, which may include characteristics such as spatial distribution, local density, curvature or height of the point cloud. For example, a point cloud with high point density and certain connectivity in spatial distribution is selected as the initial seed point cloud to ensure the stability and accuracy of regional expansion.

[0118] Next, the region is expanded from the initial seed point cloud using the spatial region growing algorithm. The region expansion process is based on two key criteria: distance metric and point cloud density function. The distance metric is used to evaluate the spatial distance between the seed point cloud and its neighboring point clouds, usually using the Euclidean distance calculation to ensure that the expanded point cloud has sufficient spatial similarity with the seed point cloud. The point cloud density function is used to determine the local density characteristics of the neighboring point clouds to avoid including isolated point clouds or discretely distributed noise points into the expanded region. Through gradual expansion, the neighboring point clouds that meet the conditions are included in the expanded region to form a continuous point cloud region.

[0119] After the continuous point cloud area is generated, the clustering algorithm is applied to divide it into regions. The clustering algorithm divides the point cloud area into multiple independent regions by analyzing the spatial distribution characteristics of the points in the point cloud area. Commonly used clustering algorithms include the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm and the K-Means algorithm based on k-means. Taking the DBSCAN algorithm as an example, by setting the density threshold and the minimum point number threshold, the point cloud is divided into several independent regions with good connectivity, and isolated point clouds are automatically filtered out.

[0120] Finally, the point cloud of dynamic obstacles is filtered out according to the segmentation results. Combining the spatial characteristics of dynamic obstacles (such as height range, density distribution or motion trajectory) and the regional segmentation results, the identified dynamic obstacle point cloud is removed from the continuous point cloud area, and the static point cloud data is retained for subsequent processing.

[0121] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0122] According to an embodiment of another aspect, a dynamic obstacle point cloud filtering device is provided. Figure 4 A schematic block diagram of a dynamic obstacle point cloud filtering device according to an embodiment is shown. The device is arranged at Figure 1 The server side of the architecture shown in Figure 1. Figure 4 As shown, the device 400 includes:

[0123] The data acquisition unit 401 is configured to acquire one or more frame images including a target area and inertial measurement data corresponding to each frame image.

[0124] The data fusion unit 402 is configured to generate first point cloud data for each of the frame images; and fuse the first point cloud data with the inertial measurement data using an iterative error Kalman filter algorithm to obtain second point cloud data and the pose information of the frame image.

[0125] The coordinate conversion unit 403 is configured to use a registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame according to the pose information; use the pose transformation matrix to convert the second point cloud data from the local coordinate system to the global coordinate system for representation, and obtain the third point cloud data in the global coordinate system.

[0126] The grid data statistics unit 404 is configured to divide the spatial range covered by the third point cloud data into more than one grid units in the global coordinate system, and count the number of point clouds and the point cloud height variance in each of the grid units.

[0127] The ground point cloud filtering unit 405 is configured to determine whether the grid unit is a ground grid unit by using a plane fitting algorithm according to the point cloud height variance, and to filter out the third point cloud data in the grid unit determined to be a ground grid unit.

[0128] The dynamic grid judgment unit 406 is configured to mark the grid unit as a dynamic grid if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold; obtain historical frame information of the dynamic grid, the historical frame information being the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the frame image corresponding to the current dynamic grid; process the historical frame information using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than a set threshold, modify the mark of the dynamic grid to a static grid.

[0129] The dynamic point cloud filtering unit 407 is configured to obtain the point cloud of the dynamic obstacle by a spatial region growing algorithm for the grid unit marked as a dynamic grid, and filter the point cloud of the dynamic obstacle from the third point cloud data to obtain the static point cloud data of the target area.

[0130] As one of the feasible ways, when the coordinate conversion unit 403 uses the registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame according to the pose information, it can be configured as follows: acquiring the point cloud data of the reference frame; generating the initial pose transformation matrix of the second point cloud data and the reference frame point cloud data by using a point-to-point iterative closest point algorithm; and optimizing the initial pose transformation matrix by using a gradient descent algorithm to obtain the pose transformation matrix.

[0131] As one of the feasible ways, the grid data statistics unit 404 can be configured as follows when dividing the spatial range covered by the third point cloud data into more than one grid units in the global coordinate system: 3. Generate a three-dimensional bounding box for the third point cloud data in the global coordinate system, divide the bounding box non-uniformly along three mutually perpendicular coordinate axis directions in the three-dimensional coordinate system, and adaptively adjust the grid resolution according to the distribution characteristics of the third point cloud data; assign a multi-dimensional index to each of the grid cells, and assign the third point cloud data to the corresponding grid cells.

[0132] As one of the feasible ways, the dynamic grid judgment unit 406 can be configured as follows when processing the historical frame information using time series analysis to calculate the dynamic confidence of the dynamic grid: constructing a point cloud quantity time series and a point cloud height variance time series according to the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information; calculating the rate of change, stability and change trend of the point cloud quantity and height variance in the dynamic grid according to the point cloud quantity time series and the point cloud height variance time series; defining a dynamic confidence formula according to the change rate, stability and change trend, and calculating the dynamic confidence according to the dynamic confidence formula.

[0133] As one of the feasible ways, the ground point cloud filtering unit 405 can be configured as follows when determining whether the grid unit is a ground grid unit based on the point cloud height variance and using a plane fitting algorithm, and filtering out the third point cloud data determined to be a ground grid unit: based on the point cloud data in each of the grid cells, a random sampling consistency algorithm is used to perform plane model fitting to obtain a fitting plane; the vertical distance between each point in the third point cloud data and the fitting plane is calculated, and whether it is a ground point is determined based on a distance threshold, and points that meet the distance threshold condition are removed from the point cloud data; wherein the distance threshold is dynamically adjusted based on the point cloud height variance.

[0134] As one of the achievable methods, the dynamic point cloud filtering unit 407, for the grid unit marked as a dynamic grid, obtains the point cloud of the dynamic obstacle through the spatial region growth algorithm, and filters the point cloud of the dynamic obstacle from the third point cloud data. It can be configured as follows: from the dynamic grid, an initial seed point cloud is selected according to a preset indicator, and a region is expanded from the initial seed point cloud according to a distance metric and a point cloud density function, and adjacent point clouds are included in the expanded region to form a continuous point cloud region; the continuous point cloud region is divided into a plurality of independent regions by using a clustering algorithm, and the point cloud of the dynamic obstacle is filtered out from the independent regions.

[0135] As one of the feasible ways, the dynamic grid judgment unit 406 can be configured to: divide into multiple different time intervals when defining the dynamic confidence formula according to the change rate, stability and change trend; obtain the timestamp information of the historical frame information, and divide the historical frame information into corresponding time intervals according to the timestamp information; set different weights for the change rate, stability and change trend in different time intervals according to the importance difference of the number of point clouds and height variance in different time intervals, and determine the dynamic confidence formula according to the weights.

[0136] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0138] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0139] And an electronic device, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, and the program instructions, when read and executed by the one or more processors, execute the steps of any of the methods described in the aforementioned method embodiments.

[0140] The present application also provides a computer program product, including a computer program, which implements the steps of any one of the methods in the aforementioned method embodiments when executed by a processor.

[0141] in, Figure 5 The architecture of the electronic device is shown as an example, which may include a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, and a memory 520. The processor 510, the video display adapter 511, the disk drive 512, the input / output interface 513, the network interface 514, and the memory 520 may be communicatively connected via a communication bus 530.

[0142] The processor 510 may be implemented by a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in this application.

[0143] The memory 520 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 520 can store an operating system 521 for controlling the operation of the electronic device 500, and a basic input and output system (BIOS) 522 for controlling the low-level operation of the electronic device 500. In addition, a web browser 523, a data storage management system 524, and a dynamic obstacle point cloud filtering device 525, etc. can also be stored. The above-mentioned dynamic obstacle point cloud filtering device 525 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 520 and is called and executed by the processor 510.

[0144] The input / output interface 513 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0145] The network interface 514 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0146] The bus 530 comprises a pathway for transmitting information between the various components of the device (eg, the processor 510, the video display adapter 511, the disk drive 512, the input / output interface 513, the network interface 514, and the memory 520).

[0147] It should be noted that, although the above device only shows a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, a memory 520, a bus 530, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0148] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a computer program product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0149] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for filtering a dynamic obstacle point cloud, characterized in that: The method comprises: Acquire more than one frame image including the target area and inertial measurement data corresponding to each of the frame images; For each of the frame images, first point cloud data is generated; the first point cloud data is fused with the inertial measurement data using an iterative error Kalman filter algorithm to obtain second point cloud data and position information of the frame image; Determine, by using a registration algorithm, a posture transformation matrix of the frame image relative to a reference frame according to the posture information; transform the second point cloud data from a local coordinate system to a global coordinate system for representation by using the posture transformation matrix, and obtain third point cloud data in the global coordinate system; In the global coordinate system, the spatial range covered by the third point cloud data is divided into one or more grid units, and the number of point clouds and the point cloud height variance in each of the grid units are counted; According to the point cloud height variance, using a plane fitting algorithm to determine whether the grid unit is a ground grid unit, and filtering out the third point cloud data in the ground grid unit; If the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold, the grid unit is marked as a dynamic grid; historical frame information of the dynamic grid is obtained, and the historical frame information is the number of point clouds and the point cloud height variance of one or more frame images before the current frame image corresponding to the current dynamic grid in the dynamic grid; The historical frame information is processed using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than a set threshold, the mark of the dynamic grid is modified to a static grid; For the grid cells marked as dynamic grids, a point cloud of a dynamic obstacle is obtained by using a spatial region growing algorithm, and the point cloud of the dynamic obstacle is filtered out from the third point cloud data to obtain static point cloud data of the target area.

2. According to the method of claim 1, the step of using a registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame according to the pose information comprises: Acquiring point cloud data of the reference frame; Generate an initial pose transformation matrix of the second point cloud data and the reference frame point cloud data by using a point-to-point iterative closest point algorithm; The initial posture transformation matrix is ​​optimized using a gradient descent algorithm to obtain the posture transformation matrix.

3. The method according to claim 1, wherein in the global coordinate system, dividing the spatial range covered by the third point cloud data into more than one grid units comprises: Generate a three-dimensional bounding box for the third point cloud data in the global coordinate system, divide the bounding box non-uniformly along three mutually perpendicular coordinate axis directions in the three-dimensional coordinate system, and adaptively adjust the grid resolution according to the distribution characteristics of the third point cloud data; A multi-dimensional index is allocated to each of the grid units, and the third point cloud data is allocated to the corresponding grid unit.

4. According to the method of claim 1, the processing of the historical frame information by time series analysis to calculate the dynamic confidence of the dynamic grid comprises: Constructing a time series of point cloud quantity and a time series of point cloud height variance according to the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information; Calculate the change rate, stability and change trend of the point cloud quantity and height variance in the dynamic grid according to the point cloud quantity time series and the point cloud height variance time series; A dynamic confidence formula is defined according to the change rate, stability and change trend, and the dynamic confidence is calculated according to the dynamic confidence formula.

5. The method according to claim 1, wherein judging whether the grid unit is a ground grid unit by using a plane fitting algorithm according to the point cloud height variance, and filtering out the third point cloud data judged to be a ground grid unit comprises: Based on the point cloud data in each of the grid cells, a random sampling consistency algorithm is used to perform plane model fitting to obtain a fitting plane; Calculate the vertical distance between each point in the third point cloud data and the fitting plane, determine whether it is a ground point according to a distance threshold, and remove the points that meet the distance threshold condition from the point cloud data; wherein the distance threshold is dynamically adjusted based on the point cloud height variance.

6. The method according to claim 1, wherein for the grid cells marked as dynamic grids, obtaining a point cloud of a dynamic obstacle by using a spatial region growing algorithm, and filtering the point cloud of the dynamic obstacle from the third point cloud data comprises: From the dynamic grid, an initial seed point cloud is selected according to a preset index, and a region is expanded from the initial seed point cloud according to a distance metric and a point cloud density function, and adjacent point clouds are included in the expanded region to form a continuous point cloud region; The continuous point cloud region is divided into a plurality of independent regions by using a clustering algorithm, and point clouds of dynamic obstacles are filtered out from the independent regions.

7. The method according to claim 4, characterized in that The dynamic confidence formula defined according to the change rate, stability and change trend includes: Acquire timestamp information of the historical frame information, and divide the historical frame information into corresponding time intervals according to the timestamp information; According to the difference in importance of the number of point clouds and height variance in different time intervals, different weights are set for the change rate, stability and change trend in different time intervals, and the dynamic confidence formula is determined according to the weights.

8. A dynamic obstacle point cloud filtering device, characterized in that: The device comprises: A data acquisition unit, configured to acquire one or more frame images containing a target area and inertial measurement data corresponding to each of the frame images; A data fusion unit is configured to generate first point cloud data for each frame image; Using an iterative error Kalman filter algorithm, the first point cloud data is fused with the inertial measurement data to obtain second point cloud data and position information of the frame image; A coordinate conversion unit is configured to use a registration algorithm to determine a pose transformation matrix of the frame image relative to a reference frame according to the pose information, and use the pose transformation matrix to convert the second point cloud data from a local coordinate system to a global coordinate system for representation, and obtain third point cloud data in the global coordinate system; A grid data statistics unit is configured to divide the spatial range covered by the third point cloud data into more than one grid units in the global coordinate system, and count the number of point clouds and the point cloud height variance in each of the grid units; a ground point cloud filtering unit configured to determine whether the grid unit is a ground grid unit by using a plane fitting algorithm according to the point cloud height variance, and to filter out the third point cloud data in the grid unit determined to be a ground grid unit; The dynamic grid judgment unit is configured to mark the grid unit as a dynamic grid if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold; obtain the historical frame information of the dynamic grid, the historical frame information being the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the current frame image corresponding to the current dynamic grid; The historical frame information is processed using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than a set threshold, the mark of the dynamic grid is modified to a static grid; The dynamic point cloud filtering unit is configured to obtain the point cloud of the dynamic obstacle by a spatial region growing algorithm for the grid unit marked as a dynamic grid, and filter the point cloud of the dynamic obstacle from the third point cloud data to obtain the static point cloud data of the target area.

9. An electronic device, characterized in that: include: one or more processors; And a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the steps of the method described in any one of claims 1 to 7 are executed.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images