Dynamic scene real-time reconstruction method, navigation method, device, equipment and medium

Abstract

The invention discloses a dynamic scene real-time reconstruction method, a navigation method, a navigation device, equipment and a medium. The reconstruction method comprises the following steps: collecting point cloud data of a dynamic scene through a laser radar; performing grid reconstruction on the point cloud data to obtain single-frame grid data; determining a single-frame distance field based on the single-frame grid data, and updating a global distance field based on the single-frame distance field; determining a scene space mark based on the updated global distance field, and removing a dynamic occupied space based on the global distance field and the single-frame distance field before updating to obtain a static space mark corresponding to the dynamic space; and constructing a scene model based on the static space mark. According to the method, single-frame reconstruction is carried out on the point cloud data of the dynamic scene acquired by the laser radar, then the single-frame distance fields are determined based on the single-frame reconstruction data, and the single-frame distance fields are fused to carry out multi-frame reconstruction, so that the point cloud data captured by the laser radar are fully utilized, and the accuracy of multi-frame reconstruction is improved. Discrete points scanned by the laser radar are converted into continuous areas for surface reconstruction.

Description

Technical Field

[0001] The present application relates to the technical field of computer graphics, and in particular to a real-time reconstruction method, navigation method, device, equipment and medium for dynamic scenes. Background Art

[0002] As one of the most widely used three-dimensional spatial sensors in recent years, LiDAR (Light Detection and Ranging) has gained significant attention in outdoor scene acquisition. Compared with visual solutions, LiDAR alone has been proven to be more effective in environments with weak textures and low light levels, and can improve the reconstruction of outdoor scenes. However, in the existing scene reconstruction methods using point cloud data collected by LiDAR, since real-time LiDAR scanning cannot directly present spatial shape content, it is impossible to timely convert the discrete points scanned by LiDAR into continuous areas for surface reconstruction and three-dimensional mapping, and thus cannot guarantee the timeliness of scene reconstruction.

[0003] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present application is to provide a real-time reconstruction method, navigation method, device, equipment and medium for dynamic scenes in view of the deficiencies in the prior art.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for real-time reconstruction of a dynamic scene, wherein the method for real-time reconstruction of a dynamic scene specifically includes: Collect point cloud data of dynamic scenes through LiDAR; Reconstructing the point cloud data into a grid to obtain single-frame grid data corresponding to the point cloud data; Determine a single-frame distance field based on the laser radar line of sight based on the single-frame grid data, and update a global distance field based on the single-frame distance field, wherein the global distance field is initialized based on the single-frame distance field corresponding to the first frame point cloud data; Determine a spatial marker of the dynamic scene based on the updated global distance field, and remove the dynamic occupied space in the free space in the spatial marker based on the global distance field before updating and the single-frame distance field, so as to obtain a static scene free space corresponding to the dynamic scene; A scene model that does not include dynamic objects and corresponds to the dynamic scene is constructed based on the free space of the static scene.

[0006] The real-time reconstruction method of a dynamic scene, wherein the step of reconstructing the point cloud data into a grid to obtain single-frame grid data corresponding to the point cloud data specifically comprises: Constructing a single-frame reconstruction grid of the point cloud data; Obtaining a normal vector of each triangle in the single-frame reconstructed mesh, and determining a confidence weight of each triangular mesh based on the normal vector of each triangle; Acquire multiple triangles corresponding to each point in the point cloud data, and determine the normal vector of each point in the point cloud data based on the acquired normal vectors of the triangles and the confidence weight; Based on the normal vector of each point in the point cloud data and the single-frame reconstruction network, single-frame mesh data corresponding to the point cloud data is determined.

[0007] The method for real-time reconstruction of dynamic scenes, wherein the step of constructing a single-frame reconstruction grid of the point cloud data specifically comprises: Dividing the point cloud data into a plurality of non-overlapping sub-point cloud data; Perform radial transformation on each sub-point cloud data to obtain a reverse point set of each sub-point cloud data; The convex hull of the union of the reverse point set of each sub-point cloud data and the viewpoint of the point cloud data is calculated to obtain a single-frame reconstruction grid of the point cloud data.

[0008] The method for real-time reconstruction of dynamic scenes, wherein the step of determining a single-frame distance field based on a laser radar line of sight based on the single-frame grid data specifically comprises: Calculating a bounding box of the point cloud data, and constructing an octree voxel set based on the bounding box; Obtaining a ray from the viewpoint of the point cloud data to each point in the point cloud data, obtaining all voxels in the octree voxel set that pass through the laser radar line of sight, and obtaining the laser radar line of sight and the single-frame grid intersection in the single-frame grid data, wherein the voxel is initialized based on the single-frame grid data, and the centroid, normal vector, and confidence weight of the triangle are recorded in the voxel; The single-frame distance from each voxel to the intersection of the single-frame grid is calculated, and based on the single-frame distance of the voxel, the perception space corresponding to the point cloud data is determined to be marked as free space, edge space and unknown space to obtain a single-frame distance field based on the laser radar line of sight.

[0009] The method for real-time reconstruction of dynamic scenes, wherein the updating of the global distance field based on the single-frame distance field specifically comprises: Initialize the global distance field based on the single-frame distance field of the first frame of point cloud data; Look up voxels in a single frame's distance field in the global distance field; For the found voxel, updating the global distance of the voxel in the global distance field based on the single-frame distance of the voxel in the single-frame distance field; For the voxel that is not found, the voxel is added to the global distance field, and the single-frame distance of the voxel in the single-frame distance field is used as its global distance in the global distance field.

[0010] The method for real-time reconstruction of dynamic scenes, wherein the removing of the dynamic occupied space in the free space in the space marker based on the global distance field before updating and the single-frame distance field to obtain the static scene free space corresponding to the dynamic scene specifically comprises: Comparing the voxels in the single-frame distance field with the voxels in the global distance field before updating to select the dynamic occupied space; The selected dynamic occupied space is separated from the free space in the space mark to obtain the static scene free space corresponding to the dynamic scene.

[0011] A second aspect of the present application provides a navigation method, using the above-mentioned dynamic scene real-time reconstruction method, the navigation method specifically includes: Collecting point cloud data through the laser radar carried by the unmanned equipment, and reconstructing the scene model through the dynamic scene real-time reconstruction method; A free space in the scene model is extracted, and a waypoint of the unmanned device is determined based on the free space to form a navigation path of the unmanned device.

[0012] The third aspect of the present application provides a dynamic scene real-time reconstruction device, wherein the dynamic scene real-time reconstruction device specifically includes: The acquisition module is used to collect point cloud data of dynamic scenes through lidar; A single-frame reconstruction module is used to reconstruct the grid of the point cloud data to obtain single-frame grid data corresponding to the point cloud data; A multi-frame reconstruction module is used to determine a single-frame distance field based on a laser radar line of sight based on the single-frame grid data, update a global distance field based on the single-frame distance field, determine a spatial marker of the dynamic scene based on the updated global distance field, remove the dynamic occupied space in the free space in the spatial marker based on the global distance field before updating and the single-frame distance field to obtain a static scene free space corresponding to the dynamic scene, and construct a scene model corresponding to the dynamic scene that does not contain dynamic objects based on the static scene free space, wherein the global distance field is initialized based on the single-frame distance field corresponding to the first frame of point cloud data.

[0013] A fourth aspect of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the above-described methods for real-time reconstruction of dynamic scenes.

[0014] A fifth aspect of the present application provides a terminal device, comprising: a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in any of the above-mentioned methods for real-time reconstruction of dynamic scenes are implemented.

[0015] Beneficial effects: Compared with the prior art, the present application provides a method, a navigation method, a device, an equipment and a medium for real-time reconstruction of a dynamic scene, wherein the reconstruction method comprises collecting point cloud data of a dynamic scene through a laser radar; performing grid reconstruction on the point cloud data to obtain single-frame grid data corresponding to the point cloud data; determining a single-frame distance field based on the laser radar line of sight based on the single-frame grid data, and updating a global distance field based on the single-frame distance field; determining a spatial marker of the dynamic scene based on the updated global distance field, and removing the dynamic occupied space in the free space in the spatial marker based on the global distance field before the update and the single-frame distance field to obtain a static scene free space corresponding to the dynamic scene; and constructing a scene model corresponding to the dynamic scene that does not contain dynamic objects based on the static scene free space. The present application first performs single-frame reconstruction on the point cloud data of the dynamic scene collected by the laser radar, and then fuses the single-frame distance field corresponding to the single-frame reconstruction data and the global distance field obtained by fusing all forward point cloud data to perform multi-frame reconstruction, thereby making full use of the point cloud data captured by the laser radar, and instantly converting the discrete points scanned by the laser radar into continuous areas for surface reconstruction, thereby ensuring the timeliness of dynamic scene reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of a method for real-time reconstruction of a dynamic scene provided in an embodiment of the present application.

[0018] Figure 2 A comparison diagram of the normal vector estimated by the embodiment of the present application and the normal vector estimated by the existing method.

[0019] Figure 3 Comparison diagram of the occupied voxels with and without dynamic removal.

[0020] Figure 4 This is an example image of the reconstructed scene model.

[0021] Figure 5 An example diagram of a navigation path.

[0022] Figure 6 This is a principle block diagram of the device for real-time reconstruction of dynamic scenes provided in an embodiment of the present application.

[0023] Figure 7 A functional block diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application provide a method for real-time reconstruction of a dynamic scene, a navigation method, an apparatus, a device and a medium. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0025] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0026] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0027] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0028] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0029] This embodiment provides a method for real-time reconstruction of a dynamic scene. Figure 1 As shown, the method includes: S10. Collect point cloud data of dynamic scenes through laser radar.

[0030] Specifically, the point cloud data is obtained by scanning the dynamic scene at the current moment, for example, the laser radar is mounted on a robot, and the robot is used to scan the dynamic scene to obtain the point cloud data of the dynamic scene. The laser radar scans the dynamic scene by rotating the laser beam to capture the three-dimensional point cloud data, wherein the three-dimensional point cloud data is a point cloud with a ring structure, and the number of rings of the three-dimensional point cloud data corresponds to the number of laser beams emitted by the laser radar.

[0031] S20, reconstructing the grid of the point cloud data to obtain single-frame grid data corresponding to the point cloud data; Specifically, the single-frame mesh data is obtained by performing network reconstruction on the point cloud data, and is used to describe the internal space and boundary of the point cloud data, wherein the single-frame mesh data includes a single-frame reconstruction network (i.e., a closed surface model) and a normal vector. In other words, when performing mesh reconstruction on the point cloud data, it is necessary to construct a single-frame reconstruction network of the point cloud data and a normal vector of each point in the point cloud data.

[0032] Based on this, the mesh reconstruction of the point cloud data to obtain the single-frame mesh data corresponding to the point cloud data specifically includes: S21, constructing a single-frame reconstruction grid of the point cloud data; S22, obtaining a normal vector of each triangle in the single-frame reconstructed mesh, and determining a confidence weight of each triangular mesh based on the normal vector of each triangle; S23, obtaining multiple triangles corresponding to each point in the point cloud data, and determining the normal vector of each point in the point cloud data based on the normal vectors of the obtained triangles and the confidence weights.

[0033] Specifically, in step S21, the surface of the point cloud data is first reconstructed to obtain a single-frame reconstructed mesh, that is, a closed surface model of the point cloud data is obtained. The single-frame reconstructed mesh can be obtained by performing surface reconstruction on the point cloud data, that is, after acquiring the point cloud data, the surface of the point cloud data can be reconstructed to obtain a single-frame reconstructed mesh of the point cloud data. When performing surface reconstruction on the point cloud data, the reverse point set of the point cloud data can be directly obtained, and then the surface reconstruction is performed by calculating the convex hull of the union of the reverse point set and the viewpoint to obtain the single-frame reconstructed mesh. The acquisition process of the reverse point set can be: For point cloud data Each point in ,point The coordinates of are located at the viewpoint (i.e. the lidar position) In the local coordinate system centered at Through the The odometer reading of the frame is obtained, represents the three-dimensional coordinates of the viewpoint. Then, the viewpoint Place it at the center of the sphere and align the point cloud data Perform radial transformation , the point cloud data can be Convert to inverse point set , where the reverse point set The reverse point The expression is: , in, Representing point clouds The reverse point, represents the viewpoint of the LiDAR, which can be obtained from the odometer reading, represents radial transformation, Represents the kernel function.

[0034] Further, in the embodiment of the present application, Point Cloud The origin of the local coordinate system simplifies the expression of the reverse point to: .

[0035] Kernel Function You can use the mirror mode, that is, the kernel function It can be expressed as: , in, represents the scaling factor, such as wait.

[0036] After the reverse point set is calculated, the correspondence between the point cloud data and the reverse point set can be based on the union of the reverse point set and the viewpoint. The convex hull of the point cloud data is used to establish a network connection to construct a single-frame reconstruction grid of the point cloud data.

[0037] Furthermore, due to the negative nonlinearity of the convex hull calculation, the reconstruction process will slow down as the number of points increases, thereby affecting the real-time reconstruction speed of dynamic scenes. Based on this, in a typical implementation, when constructing a single-frame reconstruction grid of the point cloud data, the point cloud data can be divided into a number of non-overlapping sub-point cloud data, and the surface of each sub-point cloud data is reconstructed in parallel to obtain a single-frame reconstruction grid of the point cloud data. Specifically, the construction of the single-frame reconstruction grid of the point cloud data specifically includes: Dividing the point cloud data into a plurality of non-overlapping sub-point cloud data; Perform radial transformation on each sub-point cloud data to obtain a reverse point set of each sub-point cloud data; The convex hull of the union of the reverse point set of each sub-point cloud data and the viewpoint of the point cloud data is calculated to obtain a single-frame reconstruction grid of the point cloud data.

[0038] Specifically, each of the plurality of non-overlapping sub-point cloud data is included in the point cloud data, and the plurality of non-overlapping sub-point cloud data constitute the point cloud data. Divide into Non-overlapping sub-point cloud data ,in, ， Each Covering horizontal angle range After dividing into several non-overlapping sub-point cloud data, the reverse point set of each sub-point cloud data is determined by radial transformation, wherein the process of obtaining the reverse point set corresponding to each sub-point cloud data is the same as the process of obtaining the reverse point set of the above point cloud data, which will not be repeated here.

[0039] After obtaining the reverse point set corresponding to each sub-point cloud data, the convex hull of the union of the reverse point set corresponding to each sub-point cloud data and the viewpoint can be calculated (i.e., calculating convex hull of each non-overlapping sub-point cloud data), and then establish a network connection of the point cloud data based on the convex hull of each non-overlapping sub-point cloud data to obtain a single-frame reconstruction grid of the point cloud data. When calculating the convex hull of each non-overlapping sub-point cloud data, parallel calculation can be performed through parallel threads, which can further improve the construction speed of the single-frame reconstruction grid of the point cloud data, and further improve the real-time reconstruction speed of the dynamic scene.

[0040] Further, in step S22 and step S23, the point cloud data is estimated for normal vectors to obtain the normal vectors of each point in the point cloud data. When estimating the normal vectors, the normal vectors of each triangle in the single-frame reconstruction mesh may be first obtained, and then a confidence weight may be assigned to each triangle, and then the normal vectors of each point may be calculated based on the normal vectors and confidence weights of the triangles corresponding to each point. The triangles are determined when constructing the single-frame reconstruction network.

[0041] Since the LiDAR captures point cloud data by emitting a laser beam to measure the reflection position and flight time, when the surface in the dynamic scene is parallel to the laser beam, the LiDAR cannot accurately capture the shape and position of the surface in the dynamic scene, so all reconstructed surfaces close to the plane laser beam are unreliable and need to be eliminated. It can be seen that when the triangle is perpendicular to the laser beam of the LiDAR (that is, the triangle faces the LiDAR), the confidence of the LiDAR is high. Conversely, when the triangle is parallel to the laser beam of the LiDAR, the confidence of the LiDAR is 0. Based on this, the confidence weight of the triangle can be determined by the relationship between the point in the point cloud data contained in the triangle and the viewpoint. For example, the confidence weight of the triangle can be expressed as: , in, represents the confidence weight of the triangle, represents the normal vector of the triangle, represents the center point of the triangle, Indicates viewpoint.

[0042] After obtaining the confidence weight of each triangle, obtain multiple triangles corresponding to each point in the point cloud data (such as selecting a triangle containing the point in the point cloud data, and then obtaining the adjacent triangles of the triangle to obtain the triangle corresponding to the point in the point cloud data, etc.). After obtaining the triangle corresponding to each point in the point cloud data, determine the normal vector of each point in the point cloud data by weighting the normal vectors of the triangles corresponding to each point in the point cloud data. The calculation formula for the normal vector of each point in the point cloud data can be: , in, Represents a point in point cloud data The normal vector of Indicates The normal vector of the triangle, Indicates The confidence weight of the triangle, Indicate point The number of corresponding triangles.

[0043] The embodiment of the present application can significantly reduce the noise in the normal vector estimation by using the normal vector of the triangle and the confidence weight to estimate the normal vector of the point in the point cloud data. In addition, the normal vector estimation method in this application is compared with the normal estimation performance of PUMA, VDBFusion and SHINE-Mapping methods on a set of highly uneven point clouds captured by a single frame: top: synthetic scene, bottom: real scene, the accuracy of the estimated normals is evaluated using their cosine similarity with the ground truth normals, and the comparison results are shown in Figure 1. Figure 2 As shown. Figure 2 It can be seen that the normal vector estimation method in this application determines the normal vector more accurately.

[0044] S30, determining a single-frame distance field based on the laser radar line of sight based on the single-frame grid data, and updating the global distance field based on the single-frame distance field.

[0045] Specifically, the single-frame distance field reflects the free space in the single-frame network data, where the free space refers to the area without any objects, and the laser beam can penetrate the free space and return to the lidar. To this end, when determining the single-frame distance field based on the lidar line of sight, the environment can be discretized into voxels, and the voxels can be initialized using the single-frame grid data, so that each voxel represents a local surface on a boundary grid, which can be expressed in three-dimensional barycentric coordinates. pt ( q ), normal vector nt ( q ) and confidence weights wt ( q ) to approximate the point cloud data (i.e., each voxel records the centroid, normal vector, and confidence weight of a triangle (i.e., a triangular patch)); the distance from each voxel to the object surface is calculated along the direction of the laser radar scanning line and recorded in the voxel to obtain a single-frame distance field corresponding to the point cloud data.

[0046] Exemplarily, the determining of the single-frame distance field based on the laser radar line of sight based on the single-frame grid data specifically includes: Calculating a bounding box of the point cloud data, and constructing an octree voxel set based on the bounding box; Obtaining a ray from the viewpoint of the point cloud data to each point in the point cloud data, obtaining all voxels in the octree voxel set that pass through the laser radar line of sight, and obtaining the laser radar line of sight and the single-frame grid intersection in the single-frame grid data, wherein the voxel is initialized based on the single-frame grid data, and the centroid, normal vector, and confidence weight of the triangle are recorded in the voxel; The single-frame distance from each voxel to the intersection of the single-frame grid is calculated, and based on the single-frame distance of the voxel, the perception space corresponding to the point cloud data is determined to be marked as free space, edge space and unknown space to obtain a single-frame distance field based on the laser radar line of sight.

[0047] Specifically, in the single-frame distance field based on the laser radar line of sight (denoted as LoS (Line of Sight) distance field), the object surface is located on the 0-value isosurface, the voxels between the object surface and the laser radar have a positive LoS single-frame distance, and the voxels behind the object surface have a negative single-frame distance, where the free space represents the interior of the single-frame reconstruction), the edge space represents the vicinity of the single-frame reconstructed surface, and the unknown space represents the reconstructed external space. To this end, the bounding box of the point cloud data is calculated, and an octree voxel set is constructed based on the bounding box (that is, the environment in the single-frame grid data is discretized into voxels), and then a ray is projected from the viewpoint of the laser radar to the center of each voxel in the octree voxel set. (i.e., the ray from the viewpoint of the point cloud data to the voxel), the ray As the voxel corresponds to the lidar line of sight.

[0048] After obtaining the laser radar line of sight, the grid intersection of the laser radar line of sight and the single-frame reconstruction network in the single-frame grid data is calculated, and then the single-frame distance of the voxel is determined based on the grid intersection. The expression of the single-frame distance can be: , in, Indicates The single-frame distance field of the voxel centered at Indicates the viewpoint, Represents the grid intersection of the lidar line of sight and the single-frame reconstruction network in the single-frame grid data, represents the size of the voxel, Indicates the maximum value operation, through It ensures that negative single-frame distances are truncated to The minimum value of .

[0049] In addition, due to visibility limitations, a single-frame point cloud data can only provide partial free space information in the scene. In order to accurately represent the complete free space of the entire scene, after obtaining the single-frame distance field, the single-frame distance field can be fused with the existing global distance field to obtain the free space of the dynamic scene. Based on this, after obtaining the single-frame distance field, the single-frame distance field will be fused into the existing global distance field, and the global distance field is initialized by the first-frame single-frame distance field, that is, the global distance field is a single-frame distance field that fuses all point cloud data collected before the point cloud data, so that the global distance field corresponding to the point cloud data fuses the single-frame distance fields of all point cloud data from the first frame point cloud data to the point cloud data, thereby more accurately representing the complete free space of the dynamic scene.

[0050] Exemplarily, fusing the single-frame distance fields to obtain the global distance field specifically includes: updating the global distance field based on the single-frame distance fields specifically includes: Look up voxels in a single frame's distance field in the global distance field; For the found voxel, updating the global distance of the voxel in the global distance field based on the single-frame distance of the voxel in the single-frame distance field; For the voxel that is not found, the voxel is added to the global distance field, and the single-frame distance of the voxel in the single-frame distance field is used as its global distance in the global distance field.

[0051] Specifically, updating the global distance of the voxel in the global distance field based on the single-frame distance of the voxel in the single-frame distance field refers to weighting the single-frame distance of the voxel in the single-frame distance field and the global distance of the voxel in the existing global distance field (i.e., the global distance field before updating) to obtain the updated global distance corresponding to the voxel, wherein the global weight value of the global distance and the single-frame weight value of the single-frame distance can both be preset, and the global weight value and the single-frame weight value can be the same, for example, the global weight value and the single-frame weight value are both 1, etc. Of course, in actual applications, the global weight value and the single-frame weight value can also be set according to implementation requirements, and no specific limitation is made here.

[0052] Furthermore, after obtaining the global weight value and the single-frame weight value, the single-frame distance and the existing global distance can be weighted based on the global weight value and the single-frame weight value to obtain an updated global distance field. The global distance can be expressed as: , in, represents the updated global distance, Represents the single frame distance, represents the global distance before updating, represents the global weight value, Represents the weight value of a single frame.

[0053] The present application updates the global distance field by fusing the single-frame distance field into the global distance field. As the unmanned equipment equipped with a lidar continuously explores the scene and continuously updates the global distance field, the global distance field is gradually refined, thereby more accurately reconstructing the dynamic scene and more accurately representing the complete free space of the dynamic field nodes, providing more accurate scene information for navigation based on the real-time reconstruction model of the dynamic scene.

[0054] S40, determining a spatial marker of the dynamic scene based on the updated global distance field, and removing the dynamic occupied space in the free space in the spatial marker based on the global distance field before updating and the single-frame distance field to obtain a static scene free space corresponding to the dynamic scene; Specifically, the global distance field helps detect dynamic objects in dynamic scenes. In scenes with only static objects, the value of the LoS distance field increases monotonically as the lidar explores the scene from multiple perspectives, reflecting the expanding free space. However, when a dynamic object enters the previously marked free space, the value of the LoS distance field decreases. Therefore, the point where the dynamic object is located can be identified by comparing the single-frame distance field with the global distance field before updating, and it will be excluded when reconstructing the static scene in real time. Based on this, the method of removing the dynamic occupied space in the free space in the spatial mark based on the global distance field before updating and the single-frame distance field to obtain the static scene free space corresponding to the dynamic scene specifically includes: Comparing the voxels in the single-frame distance field with the voxels in the global distance field before updating to select the dynamic occupied space; The selected dynamic occupied space is separated from the free space in the space mark to obtain the static scene free space corresponding to the dynamic scene.

[0055] Specifically, after obtaining the updated global distance field, the voxels whose global distance in the updated global distance field is less than the preset distance threshold can be regarded as occupied voxels, and then the single-frame distance of each occupied voxel in the single-frame distance field is compared with the global distance in the global distance field before the update. If the global distance in the global distance field before the update is greater than the preset distance threshold, and the single-frame distance is less than or equal to the preset distance threshold, the occupied voxel is regarded as a dynamic occupied voxel to obtain the dynamic occupied space. Based on this, the dynamic occupied voxel needs to meet the following conditions: , in, represents the updated global distance, Represents the single frame distance, Represents the global distance before updating.

[0056] The present application can avoid significant errors caused by points of dynamic objects in the real-time reconstruction process by excluding dynamic occupied voxels when reconstructing static scenes in real time, for example, Figure 3 The reconstructed model after removing the dynamically occupied voxels and the reconstructed model without removing the occupied voxels are shown. Figure 3 It can be seen that the reconstructed model after removing the dynamically occupied voxels can more accurately represent the static scene.

[0057] S50: constructing a scene model corresponding to the dynamic scene and excluding dynamic objects based on the static scene free space.

[0058] Specifically, after acquiring the static scene free space, all points on the static surfaces in the static scene free space are fused to reconstruct a high-quality dynamic scene. For example, a voxel-based signed distance function (SDF) framework is used for real-time online modeling to reconstruct a scene model that does not contain dynamic objects in real time. The embodiment of the present application can avoid significant errors caused by the points of dynamic objects in the real-time reconstruction process by excluding dynamic objects when reconstructing static scenes in real time. For example, Figure 4 The robot's real-time autonomous exploration and scene reconstruction are shown. During the robot's autonomous navigation and exploration, the surface model of the dynamic scene is gradually formed and Figure 4 In the scenes 2 and 3 shown, the method provided by the embodiments of the present application effectively maintains the smoothness and accuracy of the model even when there are obstacles in the scenes, highlighting its ability to handle different scenarios and meet the challenges of robot applications.

[0059] In summary, this embodiment provides a method for real-time reconstruction of dynamic scenes, which collects point cloud data of dynamic scenes through laser radar; reconstructs the point cloud data into grids to obtain single-frame grid data corresponding to the point cloud data; determines a single-frame distance field based on the laser radar line of sight based on the single-frame grid data, and updates the global distance field based on the single-frame distance field; determines the spatial mark of the dynamic scene based on the updated global distance field, and removes the dynamic occupied space in the free space in the spatial mark based on the global distance field before updating and the single-frame distance field to obtain the static scene free space corresponding to the dynamic scene; and constructs a scene model corresponding to the dynamic scene that does not contain dynamic objects based on the static scene free space. This application first reconstructs the point cloud data of the dynamic scene collected by the laser radar into a single frame, and then fuses the single-frame distance field corresponding to the single-frame reconstruction data and the global distance field obtained by fusing all forward point cloud data to perform multi-frame reconstruction, which makes full use of the point cloud data captured by the laser radar, and instantly converts the discrete points scanned by the laser radar into continuous areas for surface reconstruction, thereby ensuring the timeliness of dynamic scene reconstruction and improving the accuracy of dynamic scene reconstruction.

[0060] Based on the above-mentioned dynamic scene real-time reconstruction method, this embodiment provides a navigation method, using the above-mentioned dynamic scene real-time reconstruction method, the navigation method specifically includes: Collecting point cloud data through the laser radar carried by the unmanned equipment, and reconstructing the scene model through the dynamic scene real-time reconstruction method; A free space in the scene model is extracted, and a waypoint of the unmanned device is determined based on the free space to form a navigation path of the unmanned device.

[0061] Specifically, the unmanned equipment may be a drone, a robot, etc. The unmanned equipment is equipped with a laser radar, which collects point cloud data of the scene through the laser radar, reconstructs the scene based on the collected point cloud data, and plans the navigation path based on the free space in the reconstructed dynamic scene, so as to ensure that the planned path avoids both dynamic obstacles and static obstacles. For example, Figure 5 The robot is shown to extract free space in real time in an application scenario and move from an initial position to an unknown target position. Figure 5 It can be seen that the robot gradually selects waypoints by utilizing free space, plans an open path to reach the goal, and finally reaches the end smoothly without knowing anything about the scene.

[0062] Based on the above-mentioned dynamic scene real-time reconstruction method, this embodiment provides a dynamic scene real-time reconstruction device, such as Figure 6 As shown, the dynamic scene real-time reconstruction device specifically includes: The acquisition module 100 is used to acquire point cloud data of dynamic scenes through a laser radar; The single-frame reconstruction module 200 is used to perform mesh reconstruction on the point cloud data to obtain single-frame mesh data corresponding to the point cloud data; The multi-frame reconstruction module 300 is used to determine a single-frame distance field based on a laser radar line of sight based on the single-frame grid data, update a global distance field based on the single-frame distance field, determine a spatial marker of the dynamic scene based on the updated global distance field, remove the dynamically occupied space in the free space in the spatial marker based on the global distance field before updating and the single-frame distance field to obtain a static scene free space corresponding to the dynamic scene, and construct a scene model corresponding to the dynamic scene that does not contain dynamic objects based on the static scene free space, wherein the global distance field is initialized based on the single-frame distance field corresponding to the first frame of point cloud data.

[0063] Based on the above-mentioned dynamic scene real-time reconstruction method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the dynamic scene real-time reconstruction method as described in the above-mentioned embodiment.

[0064] Based on the above dynamic scene real-time reconstruction method, the present application also provides a terminal device, such as Figure 7As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, the display screen 21, the memory 22, and the communications interface 23 may communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communications interface 23 may transmit information. The processor 20 may call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0065] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0066] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0067] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0068] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for real-time reconstruction of a dynamic scene, characterized in that: The dynamic scene real-time reconstruction method specifically includes: Collect point cloud data of dynamic scenes through LiDAR; Reconstructing the point cloud data into a grid to obtain single-frame grid data corresponding to the point cloud data; Determine a single-frame distance field based on the laser radar line of sight based on the single-frame grid data, and update a global distance field based on the single-frame distance field, wherein the global distance field is initialized based on the single-frame distance field corresponding to the first frame point cloud data; Determine a spatial marker of the dynamic scene based on the updated global distance field, and remove the dynamic occupied space in the free space in the spatial marker based on the global distance field before updating and the single-frame distance field, so as to obtain a static scene free space corresponding to the dynamic scene; A scene model that does not include dynamic objects and corresponds to the dynamic scene is constructed based on the free space of the static scene.

2. The method for real-time reconstruction of dynamic scenes according to claim 1, characterized in that: The grid reconstruction of the point cloud data to obtain single-frame grid data corresponding to the point cloud data specifically includes: Constructing a single-frame reconstruction grid of the point cloud data; Obtaining a normal vector of each triangle in the single-frame reconstructed mesh, and determining a confidence weight of each triangular mesh based on the normal vector of each triangle; A plurality of triangles corresponding to each point in the point cloud data are obtained, and a normal vector of each point in the point cloud data is determined based on the normal vectors of the obtained triangles and the confidence weights.

3. The method for real-time reconstruction of dynamic scenes according to claim 2, characterized in that: The constructing of a single-frame reconstruction grid of the point cloud data specifically includes: Dividing the point cloud data into a plurality of non-overlapping sub-point cloud data; Perform radial transformation on each sub-point cloud data to obtain a reverse point set of each sub-point cloud data; The convex hull of the union of the reverse point set of each sub-point cloud data and the viewpoint of the point cloud data is calculated to obtain a single-frame reconstruction grid of the point cloud data.

4. The method for real-time reconstruction of dynamic scenes according to claim 1, characterized in that: The determining of the single-frame distance field based on the laser radar line of sight based on the single-frame grid data specifically includes: Calculating a bounding box of the point cloud data, and constructing an octree voxel set based on the bounding box; Obtaining a ray from the viewpoint of the point cloud data to each point in the point cloud data, obtaining all voxels in the octree voxel set that pass through the laser radar line of sight, and obtaining the laser radar line of sight and the single-frame grid intersection in the single-frame grid data, wherein the voxel is initialized based on the single-frame grid data, and the centroid, normal vector, and confidence weight of the triangle are recorded in the voxel; The single-frame distance from each voxel to the intersection of the single-frame grid is calculated, and based on the single-frame distance of the voxel, the perception space corresponding to the point cloud data is determined to be marked as free space, edge space and unknown space to obtain a single-frame distance field based on the laser radar line of sight.

5. The method for real-time reconstruction of dynamic scenes according to claim 1, characterized in that: The updating of the global distance field based on the single-frame distance field specifically includes: Look up voxels in a single frame's distance field in the global distance field; For the found voxel, updating the global distance of the voxel in the global distance field based on the single-frame distance of the voxel in the single-frame distance field; For the voxel that is not found, the voxel is added to the global distance field, and the single-frame distance of the voxel in the single-frame distance field is used as its global distance in the global distance field.

6. The method for real-time reconstruction of dynamic scenes according to claim 1, characterized in that: The removing the dynamic occupied space in the free space in the space marker based on the global distance field before updating and the single-frame distance field to obtain the static scene free space corresponding to the dynamic scene specifically includes: Comparing the voxels in the single-frame distance field with the voxels in the global distance field before updating to select the dynamic occupied space; The selected dynamic occupied space is separated from the free space in the space mark to obtain the static scene free space corresponding to the dynamic scene.

7. A navigation method, characterized in that: Using the dynamic scene real-time reconstruction method according to any one of claims 1 to 6, the navigation method specifically comprises: Collecting point cloud data through the laser radar carried by the unmanned equipment, and reconstructing the scene model through the dynamic scene real-time reconstruction method; A free space in the scene model is extracted, and a waypoint of the unmanned device is determined based on the free space to form a navigation path of the unmanned device.

8. A dynamic scene real-time reconstruction device, characterized in that: The dynamic scene real-time reconstruction device specifically comprises: The acquisition module is used to collect point cloud data of dynamic scenes through lidar; A single-frame reconstruction module is used to reconstruct the grid of the point cloud data to obtain single-frame grid data corresponding to the point cloud data; A multi-frame reconstruction module is used to determine a single-frame distance field based on a laser radar line of sight based on the single-frame grid data, and update a global distance field based on the single-frame distance field, determine a spatial marker of the dynamic scene based on the updated global distance field, remove the dynamic occupied space in the free space in the spatial marker based on the global distance field before updating and the single-frame distance field to obtain a static scene free space corresponding to the dynamic scene, and construct a scene model corresponding to the dynamic scene that does not contain dynamic objects based on the static scene free space, wherein the global distance field is initialized based on the single-frame distance field corresponding to the first frame of point cloud data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the real-time reconstruction method of dynamic scenes as described in any one of claims 1-6.

10. A terminal device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the real-time reconstruction method of a dynamic scene as described in any one of claims 1 to 6 are implemented.

Images