A dense point cloud mapping method and system based on score driving and voxel management, a computer device and a storage medium

Abstract

A kind of dense point cloud mapping method based on score driving and voxel management, comprising: obtaining original video, extracting key frame from original video;The key frame is parsed to obtain the dense point cloud containing color information, the dense point cloud is transformed into world coordinate system;The target space is voxelized and processed to construct sparse voxel hash map;The basic voxel hit by dense point cloud is used as target voxel, and the voxel data of target voxel is modified;The single-frame observation score of target voxel is calculated, and the cumulative score of target voxel is calculated based on the forgetting strategy and linear superposition strategy;Based on cumulative score, stable voxel is screened out from basic voxel and frozen, and the remaining basic voxel is dynamically updated;Based on stable voxel, path voxel sequence is generated, and path voxel sequence is merged;Based on sparse voxel hash map, the final map is generated and released.The present application can significantly improve the accuracy and robustness of map construction while ensuring real-time.

Description

Technical Field

[0001] This invention relates to the field of visual simultaneous localization and mapping technology, specifically to a dense point cloud mapping method, system, computer device, and storage medium based on score-driven and voxel-managed mapping. Background Technology

[0002] Dense point cloud mapping based on vision and multi-sensor fusion is a fundamental technology for autonomous navigation of UAVs, unmanned vehicles, and robots. Existing methods mostly use voxel grids, truncated signed distance functions (TSDF), or octomaps for data organization and fusion. However, in long-term operation and complex dynamic scenarios, these schemes usually process observations with fixed resolution and static fusion weights, making it difficult to adaptively select point clouds based on multi-frame geometric consistency and observation quality. This easily leads to long-distance noise accumulation, occlusion misjudgment, and map pollution. At the same time, existing systems do not make sufficient use of "empty observations" (unobserved occupancy) and lack mechanisms to incorporate empty path evidence into updates and conflict arbitration, making it difficult to suppress false filling and drift in a timely manner. In terms of efficiency, real-time maintenance of large-scale dense data puts a burden on computing power and memory, and there is a lack of lightweight scoring and incremental voxel management strategies for online operation. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a dense point cloud mapping method, system, computer device, and storage medium based on score-driven and voxel management. This method enables adaptive fusion of observation data, dynamic management of voxel states, and coordinated control of spatial blocking and write protection in complex dynamic environments, significantly improving the accuracy and robustness of map construction while ensuring real-time performance.

[0004] To achieve the above objectives, the specific solution adopted by this invention is as follows: a dense point cloud mapping method based on score-driven and voxel management, comprising: Acquire the original video and extract keyframes from it; The keyframes are parsed to obtain a dense point cloud containing color information, and the dense point cloud is transformed into the world coordinate system. The target space is voxelized to construct a sparse voxel hash map, which includes multiple basic voxels. The basic voxels hit by the dense point cloud are used as target voxels, and the voxel data of the target voxels are modified. Calculate the single-frame observation score of the target voxel, and calculate the cumulative score of the target voxel based on the forgetting strategy and the linear superposition strategy; Stable voxels are selected from the basic voxels based on cumulative scores and then frozen, while the remaining basic voxels are dynamically updated. Path voxel sequences are generated based on stable voxels, and the path voxel sequences are merged. The final map is generated and published based on the sparse voxel hash map.

[0005] As a further optimization of the above-mentioned dense point cloud mapping method based on score-driven and voxel management, the method for determining target voxels includes: Calculate the distance between the voxel center of the basic voxel and the center shooting position of the original video; The basic voxel is determined to be hit by the dense point cloud based on the distance value and the adaptively generated voxel activation threshold, and the basic voxel that is hit is identified as the target voxel.

[0006] As a further optimization of the above-mentioned dense point cloud mapping method based on score-driven and voxel management: the single-frame observation score includes a base part and a reward part, wherein the base part is calculated based on the distance between the voxel center of the target voxel and the center shooting position of the original video and the shooting angle of the original video, and the reward part is calculated based on the coverage gain, motion continuity and temporal freshness within the keyframe sliding window.

[0007] As a further optimization of the above-mentioned dense point cloud mapping method based on score-driven and voxel management: the method for calculating the cumulative score of the target voxel based on the forgetting strategy and the linear superposition strategy is as follows: ; in, Basic voxels In the The cumulative score of each keyframe. Forgetting coefficient, The score is for a single frame observation. voxels In the The cumulative score of each keyframe.

[0008] As a further optimization of the above-mentioned dense point cloud mapping method based on score-driven and voxel management, the method for selecting stable voxels from basic voxels based on cumulative scores is as follows: Basic voxels Cumulative score Compared with the preset release threshold and stability threshold Continuing with the comparison, if Then the basic voxels If it is determined to be in an unreleased state, Then the basic voxels It is determined to be an unstable state, if Then the basic voxels The state is determined to be stable, and the basic voxels of the stable state are... As a stable voxel.

[0009] As a further optimization of the above-mentioned dense point cloud mapping method based on score-driven and voxel management: For basic voxels in an unstable state... Perform delayed freezing, and perform immediate freezing on stable voxels.

[0010] As a further optimization of the above-mentioned dense point cloud mapping method based on score-driven and voxel management: the content of voxel data modification and the content of dynamic updating of basic voxels both include representative position, representative color and status.

[0011] A dense point cloud mapping system based on score-driven and voxel management is provided to implement the aforementioned dense point cloud mapping method based on score-driven and voxel management. The system includes: The data access and registration module is used to acquire the original video, extract keyframes from the original video, parse the keyframes to obtain a dense point cloud containing color information, and transform the dense point cloud to the world coordinate system. The data processing module is used to perform voxelization on the target space to construct a sparse voxel hash map and to process the basic voxels in the sparse voxel hash map based on keyframes. The map publishing module is used to generate and publish the final map based on the sparse voxel hash map.

[0012] Computer equipment, including: Memory, used to store computer programs; A processor is used to read and execute the computer program to implement the above-described dense point cloud mapping method based on score-driven and voxel management.

[0013] A storage medium for storing a computer program that, when executed, implements the aforementioned dense point cloud mapping method based on score-driven and voxel-managed principles.

[0014] Beneficial effects: This invention can achieve adaptive fusion of observation data, dynamic management of voxel states, and coordinated control of spatial blocking and write protection in complex dynamic environments, significantly improving the accuracy and robustness of map construction while ensuring real-time performance. Attached Figure Description

[0015] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a detailed flowchart of the method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figure 1 and Figure 2 As shown, the present invention first provides a dense point cloud mapping method based on score-driven and voxel management, including S1 to S8.

[0018] S1. Acquire the raw video captured by the camera and extract keyframes from the raw video. The method for extracting keyframes is a mature existing technology in this field and will not be described in detail here.

[0019] S2. Analyze the keyframes to obtain a dense point cloud containing color information, and transform the dense point cloud to the world coordinate system. The dense point cloud can be represented as... , This provides detailed data in the camera coordinate system, where (Xc, Yc, Zc) are the camera coordinates and (r, g, b) are the color information. This is then combined with the pose transformation matrix. It can transform dense point clouds into the world coordinate system, and the pose transformation matrix is ​​used. for: ; in, It is a special Euclidean group, representing rigid body transformations in three-dimensional space. Let be a 3×3 rotation matrix, representing the rotation from the camera coordinate system to the world coordinate system. It is a 3×1 translation vector, representing the translation from the camera coordinate system to the world coordinate system. Dense point cloud After transformation, it can be denoted as ,Right now And there are This means preserving the color information as is and only transforming the coordinates.

[0020] Furthermore, it can be utilized This represents all the 3D coordinates and color information contained in the dense point cloud after coordinate transformation. Based on this, when the world coordinate system is selected as the camera coordinate system of the first keyframe, the pose transformation matrix of the dense point cloud in the first keyframe during transformation is: ,in It is a 4×4 identity matrix, so that the dense point cloud of the first keyframe remains unchanged during the transformation.

[0021] S3. Perform voxelization on the target space to construct a sparse voxel hash map, which includes multiple basic voxels. Specifically, set the voxel side lengths. By voxelizing the world coordinate system, a sparse voxel hash map is obtained. If the origin of the world coordinate system is... For any point in a dense point cloud The voxel index is ,and ; Where i is the voxel index of the point in the x-direction. Let r be the x-coordinate of the origin of the world coordinate system, r be the side length of the voxel, and j be the voxel index of that point in the y-direction. Let k be the coordinate of the origin of the world coordinate system in the y-direction, and k be the voxel index of that point in the z-direction. It is the coordinate of the origin of the world coordinate system in the z-direction.

[0022] The voxel center corresponding to this point is .

[0023] This invention is based on sparse hash voxel maps, which allocate storage space only for observed voxels, thereby reducing storage space occupancy and ensuring reproducibility.

[0024] S4. The basic voxels hit by the dense point cloud are taken as target voxels, and the voxel data of the target voxels are modified. More specifically, the method for determining the target voxels includes S41 to S42.

[0025] S41. Calculate the distance between the voxel center of the basic voxel and the center shooting position of the original video. The method for calculating the distance is as follows: .

[0026] S42. Based on the distance value and the adaptively generated voxel activation threshold, determine whether a basic voxel is hit by the dense point cloud, and identify the hit basic voxel as the target voxel. The voxel activation threshold includes a proximity threshold. and far threshold ,and Based on this, the method for determining whether a basic voxel is hit by a dense point cloud is as follows: ; in, This indicates that, given the current distance value d, there are at least [number] points in the dense point cloud. A single point hits the element. The maximum activation threshold, The minimum activation threshold is defined as follows: If the number of points hitting voxels in the dense point cloud corresponding to the keyframe is denoted as... Then in At this point, the voxels enter the subsequent processing steps, in which... It is an indicator function, and when the condition is true... The value is 1, otherwise The value is 0, if In the current keyframe, no further processing is performed on the voxels. In one embodiment of the invention, The value is 10. The value is 3. The value is 2m. The value is 8m.

[0027] S5. Calculate the single-frame observation score of the target voxel, and calculate the cumulative score of the target voxel based on the forgetting strategy and the linear superposition strategy. More specifically, the single-frame observation score includes a base component and a reward component. The base component is calculated based on the distance between the voxel center of the target voxel and the center shooting position of the original video, as well as the shooting angle of the original video. The reward component is calculated based on the coverage gain, motion continuity, and temporal freshness within the keyframe sliding window. The specific calculation methods for the base component and the reward component are as follows.

[0028] The basic components include distance score and viewpoint score, at camera coordinates of That is, assuming the center shooting position coordinates of the original video are accurate, the distance score is: ; in, This is the upper limit of the measuring range; in one embodiment of the present invention, the value is 12m. Perspective score: ; The viewpoint score can also be normalized and mapped to the interval [0.5, 1]. The specific method is as follows: .

[0029] Furthermore, the final calculation method for the basic part is as follows: ; in, Weights are assigned to the distance score and the viewpoint score, respectively, using preset non-negative constants. In one embodiment of the invention, The value is 0.4. The value is 0.6.

[0030] In the reward section, the coverage score calculated based on coverage gain is as follows: ; in, To adjust the sliding window size, For temporary window counts, it refers to the number of keyframes that hit a voxel within the sliding window. This represents the number of keyframes that hit voxels within the previous sliding window. This is an indicator function for whether the current keyframe hits a voxel. For voxels whether the first The instructions for keyframe hits are explained.

[0031] In the reward section, the methods for calculating the continuity score based on motion continuity include: First, calculate ; Next, calculate the translation increment. ,in, This represents the translation increment of the j-th keyframe, i.e., the translation vector. The model; Then, the rotation increment is calculated. ,in, Rotation matrix traces; Next, calculate the average amount of exercise. ,in, It is a continuous time constant.

[0032] In the rewards section, the method for calculating the freshness score based on time freshness is as follows: ; in, Let t be the keyframe index where the voxel last generated a valid score, and t be the current keyframe index. This is an index scale constant used to limit the decay rate. In one embodiment of the invention, The value is 10.

[0033] Furthermore, the reward component also includes a distance density coupling score based on distance calculation, calculated as follows: ; ; ; in, The number of voxels hit in the dense point cloud for a keyframe, with an upper limit of [value]. , , which are constants and numerical stability terms that match the resolution of the sensor. In one embodiment of the invention, The value is 20. The value is 5.

[0034] Based on the scores mentioned above, the reward portion is calculated as follows: ; in, , as well as These are all weighting coefficients, used to balance the proportion of each part.

[0035] Furthermore, the method for calculating the total score corresponding to the voxel and keyframe t is as follows: ; in, This is a proportionality coefficient used to adjust the weight of the reward component. In one embodiment of the invention, The value is 1.2.

[0036] Based on the total score mentioned above, the method for calculating the cumulative score of the target voxel using the forgetting strategy and the linear superposition strategy is as follows: .

[0037] in, Basic voxels In the The cumulative score of each keyframe. Forgetting coefficient, The score is for a single frame observation. voxels In the The cumulative score of each keyframe.

[0038] S6. After selecting stable voxels from the basic voxels based on cumulative scores, freeze them, and dynamically update the remaining basic voxels. More specifically, the method for selecting stable voxels from the basic voxels based on cumulative scores is as follows: Basic voxels Cumulative score Compared with the preset release threshold and stability threshold Continuing with the comparison, if Then the basic voxels If it is determined to be in an unreleased state, it will only continue to accumulate points; if Then the basic voxels If it is determined to be in an unstable state, it can be published and the score can continue to be updated; if Then the basic voxels The state is determined to be stable, and the basic voxels of the stable state are... As a stable voxel. Basic voxel. When first entering a stable state, it is a basic voxel. Write stability flags and write-protect flags to prevent their representative values ​​and integrals from being overwritten by subsequent observations.

[0039] Furthermore, for basic voxels in an unstable state Perform delayed freezing, and freeze stable voxels instantly. Specifically, freeze the voxels after accumulating the score for the current keyframe. After updating and determining the status, some basic voxels are... Freezing can be performed using freeze events, which are represented as follows: ,in To record the camera center at the time of the freeze event, For this basic voxel The representative position, The type is freeze, and the freeze type is either delayed freeze or immediate freeze.

[0040] When the basic voxel The system has not yet entered a stable state, and the hit confidence of the dense point cloud in the current keyframe meets the requirements, i.e. At that time, the freeze event will occur on the basic voxels. It will be frozen once it reaches a stable state in the future.

[0041] When the basic voxels have entered a stable state and the hit confidence of the dense point cloud in the current keyframe meets the requirements, i.e. If necessary, the basic voxels are immediately frozen. Freezing voxels can eliminate and prevent noise points or subsequent misobservations from contaminating confirmed areas, thereby ensuring the stability and consistency of the spatial structure of dense point clouds.

[0042] Furthermore, the content modified in the voxel data, as well as the content dynamically updated in the basic voxel, includes the representative position, representative color, and state. Details are as follows.

[0043] The method for updating the representative position is as follows: in the basic voxel It is effectively hit by keyframe t, that is At that time, for basic voxels The representative color and coverage statistics are updated all at once, while voxels that were not hit retain their previous statistics. The update method can be represented as: .

[0044] The method for updating the representative color is as follows: For basic voxels Let the representative color of the current keyframe be... The previous representative color was , fusion count is The maximum fusion limit is Then, the update is performed using a finite number of weighted averages. The update method can be expressed as: ,when At this point, color blending stops to suppress over-smoothing and lighting drift. In one embodiment of the invention, The value is 5.

[0045] Furthermore, to avoid the coverage window count score lagging behind the overall score by one frame, this invention employs a two-stage processing method for coverage statistics: temporary update for scoring and formal fixation after state update. The temporary update first constructs a temporary coverage window count when a basic voxel is hit in the current keyframe. After the cumulative score calculation and voxel state update in the current keyframe, the temporary update result is converted into a formal update, and the historical index set is updated, clipped to the window size according to the keyframe index, thus providing each basic voxel with an independent index window.

[0046] S7. Generate a path voxel sequence based on stable voxels, and merge the path voxel sequences. Specifically, first determine the freeze path based on the aforementioned freeze events. The freeze path is determined by the camera center. To voxel representation position The connection points can be determined using an algorithm based on three-dimensional discrete numerical stepping, which is existing technology and will not be elaborated upon here. Subsequently, for each voxel on the freezing path... If a voxel meets the conditions of not being frozen, not being stable, and not being write-protected, it will be frozen. Once frozen, the voxel will no longer accept the insertion of new point clouds; that is, it will be set to a non-writable state.

[0047] S8. Generate and publish the final map based on the sparse voxel hash map.

[0048] The present invention further provides a dense point cloud mapping system based on score-driven and voxel management, which is used to implement the above-mentioned dense point cloud mapping method based on score-driven and voxel management. The system includes a data access and registration module, a data processing module, and a map publishing module.

[0049] The data access and registration module is used to acquire the original video, extract keyframes from the original video, parse the keyframes to obtain a dense point cloud containing color information, and transform the dense point cloud into the world coordinate system.

[0050] The data processing module is used to perform voxelization on the target space to construct a sparse voxel hash map and to process the basic voxels in the sparse voxel hash map based on keyframes.

[0051] The map publishing module is used to generate and publish the final map based on the sparse voxel hash map.

[0052] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or modules may be electrical, mechanical, or other forms.

[0053] The present invention also provides a computer device, including a memory and a processor.

[0054] Memory is used to store computer programs.

[0055] The processor is used to read and execute computer programs to implement the above-mentioned dense point cloud mapping method based on score-driven and voxel management.

[0056] Finally, the present invention provides a storage medium for storing a computer program that, when executed, implements the above-described dense point cloud mapping method based on score-driven and voxel-management.

[0057] The memory, as a carrier of resources, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it can include the operating system, computer programs, etc., and the storage method can be temporary or permanent storage. The operating system is used to manage and control the various hardware devices and computer programs on the electronic device, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the adaptive emotion regulation method based on personalized reconfigurable music disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The processor can be a general-purpose processor product based on architectures such as x86, IA64, RISC, MIPS, and ARM.

[0058] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dense point cloud mapping method based on score-driven and voxel-management, characterized in that, include: Acquire the original video and extract keyframes from it; The keyframes are parsed to obtain a dense point cloud containing color information, and the dense point cloud is transformed into the world coordinate system. The target space is voxelized to construct a sparse voxel hash map, which includes multiple basic voxels. The basic voxels hit by the dense point cloud are used as target voxels, and the voxel data of the target voxels are modified. Calculate the single-frame observation score of the target voxel, and calculate the cumulative score of the target voxel based on the forgetting strategy and the linear superposition strategy; Stable voxels are selected from the basic voxels based on cumulative scores and then frozen, while the remaining basic voxels are dynamically updated. Path voxel sequences are generated based on stable voxels, and the path voxel sequences are merged. The final map is generated and published based on the sparse voxel hash map.

2. The dense point cloud mapping method based on score-driven and voxel management as described in claim 1, characterized in that, Methods for determining target voxels include: Calculate the distance between the voxel center of the basic voxel and the center shooting position of the original video; The basic voxel is determined to be hit by the dense point cloud based on the distance value and the adaptively generated voxel activation threshold, and the basic voxel that is hit is identified as the target voxel.

3. The dense point cloud mapping method based on score-driven and voxel management as described in claim 1, characterized in that, The single-frame observation score includes a base component and a reward component. The base component is calculated based on the distance between the voxel center of the target voxel and the center shooting position of the original video, as well as the shooting angle of the original video. The reward component is calculated based on the coverage gain, motion continuity, and temporal freshness within the keyframe sliding window.

4. The dense point cloud mapping method based on score-driven and voxel management as described in claim 1, characterized in that, The method for calculating the cumulative score of the target voxel based on the forgetting strategy and the linear superposition strategy is as follows: ； in, Basic voxels In the The cumulative score of each keyframe. Forgetting coefficient, The score is for a single frame observation. voxels In the The cumulative score of each keyframe.

5. The dense point cloud mapping method based on score-driven and voxel management as described in claim 1, characterized in that, The method for selecting stable voxels from basic voxels based on cumulative scores is as follows: Basic voxels Cumulative score Compared with the preset release threshold and stability threshold Continuing with the comparison, if Then the basic voxels If it is determined to be in an unreleased state, Then the basic voxels It is determined to be an unstable state, if Then the basic voxels The state is determined to be stable, and the basic voxels of the stable state are... As a stable voxel.

6. The dense point cloud mapping method based on score-driven and voxel management as described in claim 5, characterized in that, For basic voxels in an unstable state Perform delayed freezing, and perform immediate freezing on stable voxels.

7. The dense point cloud mapping method based on score-driven and voxel-management as described in claim 1, characterized in that, The content of voxel data modification and the content of dynamic updates to basic voxels both include representative position, representative color, and status.

8. A dense point cloud mapping system based on score-driven and voxel-management, characterized in that, The system is used to implement a dense point cloud mapping method based on score-driven and voxel management as described in any one of claims 1-7, the system comprising: The data access and registration module is used to acquire the original video, extract keyframes from the original video, parse the keyframes to obtain a dense point cloud containing color information, and transform the dense point cloud to the world coordinate system. The data processing module is used to perform voxelization on the target space to construct a sparse voxel hash map and to process the basic voxels in the sparse voxel hash map based on keyframes. The map publishing module is used to generate and publish the final map based on the sparse voxel hash map.

9. A computer device, characterized in that, include: Memory, used to store computer programs; A processor for reading and executing the computer program to implement a dense point cloud mapping method based on score-driven and voxel-managed methods as described in any one of claims 1-7.

10. A storage medium, characterized in that, Used to store a computer program that, when executed, implements a dense point cloud mapping method based on score-driven and voxel-managed methods as described in any one of claims 1-7.

Images