Point cloud registration method and device, operation machine and medium
By performing dimensionality reduction projection and feature point sampling fusion on point clouds in mines or mound scenes, the problem of pose error accumulation between point clouds is solved, and a high-precision three-dimensional reconstruction effect is achieved.
Patent Information
- Application Number
- CN202411831122.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-05-06
AI Technical Summary
In the existing technology, in low-space feature scenarios such as mines or mounds, the accumulation of pose errors between point clouds leads to poor three-dimensional reconstruction results, and the IMU and GPS methods have cumulative drift problems.
By dimensionality reduction projection on two consecutive frames of three-dimensional point clouds, the two-dimensional projection images are extracted, feature points are matched and filtered, the dimension-raising mapping is obtained to obtain the three-dimensional feature point set, and sampling and fusion are performed in the area determined by the feature point set, and the pose transfer matrix is obtained for point cloud registration.
Low-drift point cloud scene reconstruction is achieved, the three-dimensional reconstruction accuracy of low-profile spatial structures such as mines or mounds is improved, the calculation amount is reduced, and the feature point description ability is enhanced.
Smart Images
Figure CN119941808A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of three-dimensional scene reconstruction and the technical field of intelligent construction sites, and specifically to a point cloud registration method, device, operating machinery and medium. Background Art
[0002] In the construction of smart construction sites, it involves field modeling of the working conditions of operating machinery, etc., and this modeling process requires the use of sensors to perform three-dimensional reconstruction of actual engineering scenes such as mines and earth piles. The existing technology mainly uses sensors such as laser radar to perform spatial three-dimensional reconstruction of scenes, which uses multi-line radar to obtain a radial sparse description of the scene. Among them, the point cloud field of the radar is centered on the radar itself. If a point cloud description of the entire construction site or mine is required, continuous point cloud frames need to be registered. However, unlike urban scenes with many point cloud details, the point cloud features of mines or earth pile scenes are not clear, and even only a single surface can be obtained without forming a spatial scene, that is, an application scene with low spatial features. The existing laser radar positioning and three-dimensional reconstruction algorithms, such as LOAM (Lidar Odometry and Mapping), rely on the three-dimensional feature points of the point cloud to perform pose transformation calculations, which can achieve good results in urban scenes, but for application scenes with low spatial features such as earth piles and mines, it will lead to the accumulation of pose errors between continuous point cloud frames, and the three-dimensional reconstruction of the entire scene cannot be completed. In addition, the use of methods such as IMU (Inertial Measurement Unit) or GPS (Global Positioning System) has cumulative fixed drift, which also cannot achieve good scene reconstruction effects. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a point cloud registration method, device, operating machine and medium to at least partially solve the above-mentioned technical problems.
[0004] In order to achieve the above-mentioned objectives, the first aspect of the present application provides a point cloud registration method, including: performing dimensionality reduction projection on the initial three-dimensional point cloud between two consecutive frames of a specified scene to obtain corresponding two consecutive frames of two-dimensional projection images; performing inter-frame image feature point matching and matching point pair screening between the two consecutive frames of two-dimensional projection images to obtain correctly matched feature point pairs; performing dimensionality increase mapping on the correctly matched feature point pairs to obtain corresponding three-dimensional feature point sets; sampling the initial three-dimensional point cloud within the point cloud sampling area determined by the three-dimensional feature point set to obtain corresponding sampled point clouds, and merging the corresponding three-dimensional feature point set with the sampled point cloud to obtain a fused feature point cloud; and obtaining a pose transfer matrix between the fused feature point clouds between two consecutive frames for point cloud registration.
[0005] In the embodiment of the present application, the dimensionality reduction projection includes: performing perspective projection in any direction to obtain a two-dimensional projection image of the corresponding direction view; or performing perspective projection in multiple directions to obtain a combined multi-view image. Figure 2 dimensional projection image.
[0006] In an embodiment of the present application, the point cloud registration method also includes: in the process of the dimensionality reduction projection, for each frame of the two-dimensional projection image, determining a mapping matrix between the two-dimensional projection image and the corresponding initial three-dimensional point cloud, wherein the mapping matrix is configured to perform the dimensionality increase mapping.
[0007] In an embodiment of the present application, determining the mapping matrix includes: determining the index relationship between the feature points on the two-dimensional projection image and the corresponding three-dimensional point cloud according to the surface priority projection principle and the point cloud marks projected from the initial three-dimensional point cloud to the corresponding two-dimensional projection image; and determining the mapping matrix according to the index relationship.
[0008] In an embodiment of the present application, the matching point pair screening includes: for the initial matching point pairs obtained by the feature point matching, selecting the correct matching feature point pairs therein based on one or more preset constraint conditions.
[0009] In an embodiment of the present application, the constraint conditions include: a feature point slope constraint condition, which is configured to limit the area used to select the correct matching feature point pairs based on the slope of the feature point matching line formed by each point pair in the initial matching point pair, wherein the number of feature point pairs that meet the preset matching line slope in the selected area is the largest; and / or a matching line length constraint condition, which is configured to limit the area used to select the correct matching feature point pairs based on the length of the feature point matching line formed by each point pair in the initial matching point pair, wherein the number of feature point pairs that meet the preset matching line length range in the selected area is the largest.
[0010] In an embodiment of the present application, the point cloud sampling area is determined based on the extreme value distribution of each feature point in the three-dimensional feature point set.
[0011] A second aspect of the present application provides a point cloud registration device, comprising: a memory configured to store instructions; and a processor configured to call the instructions from the memory and to implement any of the above-mentioned point cloud registration methods when executing the instructions.
[0012] The third aspect of the present application provides an operating machine, comprising: a point cloud acquisition device, used to obtain an initial three-dimensional point cloud for a specified scene; and any of the above-mentioned point cloud registration devices, used to perform point cloud registration on the initial three-dimensional point cloud.
[0013] A fourth aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute any of the above-mentioned point cloud registration methods.
[0014] Through the above technical solution, the embodiment of the present application uses dimensionality reduction projection to extract the projection image information of the point cloud, and then uses threshold sampling to extract the effective structural information of the point cloud. The combination of the two can achieve low-drift point cloud scene reconstruction.
[0015] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:
[0017] Figure 1 A schematic diagram of a process of a point cloud registration method according to an embodiment of the present application is schematically shown;
[0018] Figure 2 An example dimensionality reduction projection diagram according to an embodiment of the present application is schematically shown;
[0019] Figure 3 An exemplary dimension-raising mapping diagram according to an embodiment of the present application is schematically shown;
[0020] Figure 4 Schematically shows an example process diagram of threshold-limited sampling and point cloud fusion based on extreme value distribution of feature points according to an embodiment of the present application; and
[0021] Figure 5 The structural block diagram of a point cloud registration device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0023] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. In the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, which should be considered as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.
[0024] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0025] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0026] Figure 1 A flow chart of a point cloud registration method according to an embodiment of the present application is schematically shown, wherein the point cloud registration in the embodiment of the present application refers to solving the pose transfer matrix of two consecutive frames of point clouds in the same coordinate system, and using the pose transfer matrix to integrate multi-view point clouds into a specified coordinate system, thereby finally completing the three-dimensional reconstruction of the scene.
[0027] like Figure 1 As shown, an embodiment of the present application provides a point cloud registration method, which may include the following steps S110-S150.
[0028] Step S110 , performing dimensionality reduction projection on the initial three-dimensional point cloud between two consecutive frames of the specified scene to obtain corresponding two consecutive frames of two-dimensional projection images.
[0029] In a preferred embodiment of the present application, the dimensionality reduction projection includes: performing perspective projection in any direction to obtain a two-dimensional projection image of the corresponding direction view; or performing perspective projection in multiple directions to obtain a combined multi-view image. Figure 2The direction is for projection, for example, the front and back, up and down, left and right directions relative to the scene object, forming a corresponding front view, side view and top view.
[0030] For example, the laser radar is installed on the vehicle to obtain the point cloud information of the specified scene, and then the point cloud between two consecutive frames is perspective projected, so that the radar point cloud is projected onto the plane image, and the corresponding two consecutive frames of two-dimensional projection images are obtained. Due to the uncertainty of the point cloud distribution, the centroid alignment method can be used to project the point cloud centroid to the center of the image. Calculated as:
[0031]
[0032] Among them, the centroid is projected to the image center (mid_x, mid_y), the projection scaling factor is (scale), and the projection of multiple views includes the front view, side view and top view.
[0033] Among them, the coordinates of the front view point cloud (x, y, z) and the projection image coordinates (x img ,y img ) is:
[0034]
[0035] Among them, the side view point cloud coordinates (x, y, z) and the projection image coordinates (x img ,y img ) is:
[0036]
[0037] Among them, the top view point cloud coordinates (x, y, x) and the projection image coordinates (x img ,y img ) is:
[0038]
[0039] In this way, it is only necessary to adjust the projection matrix to perform dimensionality reduction projection of the point cloud of any view. The combination of the above three views is exemplary, and the embodiments of the present application are not limited to this.
[0040] It should be noted that in the above-mentioned dimensionality reduction projection process, a mapping matrix between the two-dimensional projection image and the corresponding initial three-dimensional point cloud, that is, a two-dimensional-three-dimensional mapping matrix, can be determined for each frame of the two-dimensional projection image. The mapping matrix can be configured for the dimensionality increase mapping involved in the subsequent step S130.
[0041] In a preferred embodiment of the present application, determining the mapping matrix includes: determining the index relationship between the feature points on the two-dimensional projection image and the corresponding three-dimensional point cloud according to the surface first projection principle and the point cloud marks on the corresponding two-dimensional projection image projected by the initial three-dimensional point cloud; and determining the mapping matrix according to the index relationship.
[0042] For example, Figure 2 An example dimensionality reduction projection diagram according to an embodiment of the present application is schematically shown, which belongs to an orthographic projection, wherein line S1 represents the projection of the point cloud centroid to the image center (mid_x, mid_y). Figure 2 In the projection process shown, a mapping matrix is created synchronously for each view. The mapping matrix is a matrix of the same size as the projected image, which is used to store the point cloud labels projected onto the image, that is, its ordinal number. In the initial three-dimensional point cloud data, the corresponding three-dimensional point cloud can be found according to the index indicated by the ordinal number. When projecting according to the projection matrix, when multiple points correspond to the same pixel, such as line S2 and line S3, the surface priority principle is adopted, and points with smaller depth are projected first. After the projection is completed, the front view image of the point cloud is obtained. The two-dimensional-three-dimensional mapping matrix created in this process will be used for dimensional mapping in subsequent steps, which will be described in detail below and will not be repeated here.
[0043] Step S120 , performing inter-frame image feature point matching and matching point pair screening between the two consecutive frames of two-dimensional projection images to obtain correct matching feature point pairs.
[0044] For the feature point matching, for example, after the projection is completed, SIFT (Scale Invariant Feature Transform) feature description is performed on the multi-view projection images between two consecutive frames to obtain initial matching point pairs between the two point clouds.
[0045] However, there are still many mismatches, so matching point pair screening is required. In a preferred embodiment of the present application, the matching point pair screening includes: for the initial matching point pairs obtained by the feature point matching, selecting the correct matching feature point pairs based on one or more preset constraints.
[0046] The constraint condition may also be referred to as a constraint strategy, which is intended to filter out mismatched point pairs. In a more preferred embodiment, the constraint condition may include the following feature point slope constraint condition and matching line length constraint condition.
[0047] First, the feature point slope constraint is configured as follows: the slope of the feature point matching line formed by each point pair in the initial matching point pair is used to limit the area used to select the correct matching feature point pairs, wherein the number of feature point pairs that meet the preset matching line slope in the selected area is the largest.
[0048] It should be noted that the recognition of feature points is local in nature, but the scene itself has its own spatial continuity, and there is also spatial continuity between the captured frames. This continuity determines that the correct feature point matching lines are often within a continuous slope range. Therefore, in the example, according to the distribution relationship of this continuous slope range, the slope line from -1 to 1 can be divided into V parts, and s is the ordinal number of V parts (value 0-V). The number of feature point pairs belonging to this slope in each area of the n pairs of matching point pairs is counted. If they belong to this area, the statistics are added by 1, otherwise they are added by 0. Finally, the distribution area k with the largest number of feature point pairs is selected, and the feature point pairs within the k slope interval are obtained. According to the idea of this example, the total number of feature point pairs that meet the slope conditions of each interval, that is, the number of slope distributions C in each area s , which can be described as follows:
[0049]
[0050] Among them, i represents the ordinal number of the matching point pair.
[0051] Second, the matching line length constraint is configured as follows: the length of the feature point matching line formed by each point pair in the initial matching point pair is used to limit the area used to select the correct matching feature point pairs, wherein the number of feature point pairs that meet the preset matching line length range in the selected area is the largest.
[0052] For example, the difference between the horizontal coordinates of the two feature points of the feature point pair can be represented as the length of the matching line. The average value outlier method is used, and the outlier ratio range is set to α. The lengths of all feature points are accumulated and averaged, and the feature point pairs whose ratio relative to the average value exceeds the set range are filtered out. s is the total number of matching point pairs that satisfy the slope condition of each interval. According to the idea of this example, C s It can be described as follows:
[0053]
[0054] Where i represents the ordinal number of the matching point pair; len(i) represents the length average ratio calculation function. For len(i), 1 represents the same length as the average value, and α represents the acceptable deviation ratio.
[0055] A smaller number of correctly matched feature point pairs can be obtained through the above two constraints. Here, "correct" means that the corresponding feature point pairs can accurately reflect the spatial characteristics of the specified scene. In addition, the correctly matched feature points can be obtained by screening only through the first constraint, or only through the second constraint, or through both the first and second constraints, and the order of using the two constraints is not limited. According to actual needs, other constraints can also be selected to obtain correctly matched feature point pairs.
[0056] Step S130, performing dimensional mapping on the correctly matched feature point pairs to obtain a corresponding three-dimensional feature point set.
[0057] Continuing from the above, the correctly matched feature point pairs obtained in step S120 are combined with the mapping matrix obtained in step S110, and the mapping matrix is accessed through the two-dimensional coordinates of the feature points to obtain a three-dimensional feature point set. Figure 3 The following schematic diagram shows an example of a dimension-upgraded mapping diagram according to an embodiment of the present application, which describes the specific mapping extraction process. The black filled box is the image center, which corresponds to the point cloud centroid; the two orange filled boxes represent the coordinates of a pair of correctly matched feature points in the projected image, which, through the dimension-upgraded mapping, ultimately obtain a three-dimensional feature point set formed by the correctly matched feature point pairs represented by "point cloud_1" and "point cloud_2".
[0058] Step S140, sampling the initial three-dimensional point cloud within the point cloud sampling area determined by the three-dimensional feature point set to obtain a corresponding sampling point cloud, and fusing the corresponding three-dimensional feature point set with the sampling point cloud to obtain a fused feature point cloud.
[0059] For example, after extracting image features, matching and filtering, the three-dimensional feature point set is relatively accurate, such as Figure 3 The "Point Cloud_1" and "Point Cloud_2" shown, but their number is small, only the image features of the point cloud projection are combined, and the three-dimensional feature point set obtained thereby also retains this feature, which is accurate but not robust enough. Therefore, the embodiment of the present application considers further threshold-limited sampling of the initial three-dimensional point cloud so that the corresponding sampled point cloud is merged with the obtained three-dimensional feature point set to enrich the point cloud information. That is, the initial three-dimensional point cloud is sampled within the threshold range corresponding to the point cloud sampling area determined by the three-dimensional feature point set. It is easy to know that when the three-dimensional feature point set can provide accurate spatial features, the point cloud within the point cloud sampling area defined by it is also accurate.
[0060] The point cloud sampling area may be determined based on the extreme value distribution of each feature point in the three-dimensional feature point set. For example, Figure 4The exemplary process diagram of threshold-limited sampling and point cloud fusion based on the extreme value distribution of feature points according to an embodiment of the present application is schematically shown. The extreme values of the distribution of the three-dimensional feature points extracted by the dimensionality-upgrading mapping can be determined, including the maximum and minimum values on the three axes of xyz ( Figure 4 The six extreme values can determine a cubic space region. Within the cubic space region, the point cloud is sampled, and the point cloud outside this region is not sampled. Thus, the threshold-limited sampling and point cloud fusion corresponding to this example can be described as follows:
[0061] P sample =p i ∈P|x min ≤x i ≤x max ,y min ≤y i ≤y max , z min ≤z i ≤
[0062] z max , i mod n = 0 (9)
[0063] P fused =P sample ∪P feature (10)
[0064] Among them, P sample represents the sampled point cloud, P feature Represents the point cloud corresponding to the three-dimensional feature point set, P fused Represents the obtained fused feature point cloud ( Figure 4 “Select Point Cloud 1” and “Select Point Cloud 2” shown in min 、x max ,y min ,y max 、z min and z max Indicates the corresponding six extreme values.
[0065] It should be noted that by combining the fusion results of feature points on multiple views, an overall accurate description of the point cloud can be formed as follows:
[0066] P fused =P Front ∪P Top ∪P Side ∪P Ot erPossibleView (11)
[0067] Among them, P Front , P Top , P Sideand P Ot erPossibleView Respectively represent the fused feature point clouds corresponding to the front view, top view, side view and other possible direction views.
[0068] Step S150, obtaining a pose transfer matrix between the fused feature point clouds between two consecutive frames for point cloud registration.
[0069] For example, for the fusion of feature point clouds, ICP (Iterative Point Calculation) is used. e The pose transfer matrix is calculated by the iterative nearest neighbor method. The calculated transfer matrix may include a rotation matrix and a translation matrix, thereby realizing the pose transfer of the point cloud between two frames through six degrees of freedom. The specific calculation process is expressed by the following formula, for example:
[0070] Q=RP+T (12)
[0071] (q x \q y \q z )=(r 11 r 12 r 13 \r 21 r 22 r 23 \r 31 r 32 r 33 )(p x \p y \p z )+
[0072] (t x \t y \t z ) (13)
[0073] R x =(1 0 0\0 cos(θ x ) -sin(θ x )\0 sin(θ x ) cos(θ x )) (14)
[0074] R y =(cos(θ y ) 0 sin(θ y )\0 1 0\-sin(θ y ) 0 cos(θ y )) (15)
[0075] R z =(cos(θ z ) -sin(θz ) 0\sin(θ z ) cos(θ z ) 0\0 0 1) (16)
[0076] R=R x *Ry*Rz (17)
[0077] Where R represents the rotation matrix, which is described as (r 11 r 12 r 13 \r 21 r 22 r 23 \r 31 r 32 r 33 ), which is used to determine the three rotation angles θ x ,θ y ,θ z ; T represents the translation matrix, which is described as (t x \t y \t z ), used to achieve the three translation parameters t x ,t y and t z To control the translation of the point cloud. Through R and T, the rigid body pose transformation between two consecutive point clouds can be realized, where P is the initial point cloud (also called the source point cloud), which is described as (p x \p y \p z ); Q is the transformed point cloud (also called target point cloud), which is described as (q x \q y \q z ).
[0078] In summary, the point cloud registration method of the embodiment of the present application is adopted. In the example, the process of realizing point cloud registration may include: first, perspective projecting the point clouds between the two frames, creating a two-dimensional-to-three-dimensional mapping matrix during the projection process, and obtaining a multi-view projection image of the point cloud; secondly, using feature point detection to describe and match the feature points of the projection image, adding a constraint strategy to filter the feature point pairs, and mapping the filtered feature point pairs through a mapping matrix to obtain a three-dimensional feature point set at multiple angles; then, calculating the maximum and minimum values of the three-dimensional feature point set in three dimensions, and using threshold restrictions on multiple views to sample the original point cloud, the mapped feature point cloud is fused with the sampled point cloud to obtain a fused feature point cloud; finally, the two parts of the point cloud are registered using a registration method, and the rotation matrix and translation matrix are calculated to perform a series of three-dimensional scene reconstruction.
[0079] Through this example, the point cloud registration method of the embodiment of the present application has at least the following advantages:
[0080] 1) The embodiment of the present application uses dimensionality reduction projection to extract the projection image information of the point cloud, and then uses threshold sampling to extract the effective structural information of the point cloud. The combination of the two can achieve low-drift point cloud scene reconstruction. Among them, the three-dimensional point cloud is projected into a two-dimensional image using dimensionality reduction projection, which makes the calculation of feature points simpler and requires less calculation. In addition, through the method of threshold-limited sampling, the plane distribution characteristics and spatial structure characteristics of the point cloud are combined at the same time, which improves the accuracy of pose calculation, and is suitable for the reconstruction of low-feature spatial structure scenes such as earthworks or mines.
[0081] 2) The embodiment of the present application can perform multi-view projection during the dimensionality reduction projection process, without limiting the description angle and the number of views, and can increase the description capability of feature points.
[0082] 3) The embodiment of the present application uses the surface priority principle in the dimensionality reduction projection process, retains the mapping matrix so that it can be subsequently upgraded to three-dimensional feature points, and can better describe the point cloud scene.
[0083] Figure 5 The structure block diagram of a point cloud registration device according to an embodiment of the present application is schematically shown. Figure 5 As shown, the device may include: a memory configured to store instructions; and a processor configured to call the instructions from the memory and implement the above-mentioned point cloud registration method when executing the instructions.
[0084] The point cloud registration device may be a built-in controller of a laser radar or the like, or a remote controller. In addition, other implementation details and effects of the point cloud registration device may also refer to the above-mentioned embodiment of the point cloud registration method, which will not be described in detail here.
[0085] An embodiment of the present application also provides a working machine, including: a point cloud acquisition device, used to obtain an initial three-dimensional point cloud for a specified scene; and the above-mentioned point cloud registration device, used to perform point cloud registration on the initial three-dimensional point cloud.
[0086] The point cloud acquisition device may be a laser radar, a depth camera, etc. Moreover, the operating machinery may be, for example, earth-moving machinery and mining machinery such as an excavator, a loader, a mining dump truck, and a crusher.
[0087] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned point cloud registration method.
[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0092] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0093] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0094] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0095] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0096] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A point cloud registration method, characterized in that: include: Perform dimensionality reduction projection on the initial three-dimensional point cloud between two consecutive frames of the specified scene to obtain the corresponding two consecutive frames of two-dimensional projection images; Performing inter-frame image feature point matching and matching point pair screening between the two consecutive frames of two-dimensional projection images to obtain correctly matched feature point pairs; Performing dimensional mapping on the correctly matched feature point pairs to obtain a corresponding three-dimensional feature point set; In the point cloud sampling area determined by the three-dimensional feature point set, the initial three-dimensional point cloud is sampled to obtain a corresponding sampling point cloud, and the corresponding three-dimensional feature point set is fused with the sampling point cloud to obtain a fused feature point cloud; as well as The pose transfer matrix between the fused feature point clouds between two consecutive frames is obtained for point cloud registration.
2. The point cloud registration method according to claim 1, characterized in that: The dimension reduction projection comprises: Perform perspective projection in any direction to obtain a two-dimensional projection image of the corresponding direction view; or Perspective projections in multiple directions are performed to obtain a combined multi-view two-dimensional projection image.
3. The point cloud registration method according to claim 1, characterized in that: The point cloud registration method further comprises: In the process of the dimensionality reduction projection, for each frame of the two-dimensional projection image, a mapping matrix between the two-dimensional projection image and the corresponding initial three-dimensional point cloud is determined, wherein the mapping matrix is configured to perform the dimensionality increase mapping.
4. The point cloud registration method according to claim 3, characterized in that: Determining the mapping matrix includes: Determine an index relationship between feature points on the two-dimensional projection image and the corresponding three-dimensional point cloud according to a surface priority projection principle and point cloud labels of the initial three-dimensional point cloud projected onto the corresponding two-dimensional projection image; and The mapping matrix is determined according to the index relationship.
5. The point cloud registration method according to claim 1, characterized in that: The matching point pair screening includes: For the initial matching point pairs obtained by the feature point matching, a correct matching feature point pair is selected based on one or more preset constraint conditions.
6. The point cloud registration method according to claim 5, characterized in that: The constraints include: The feature point slope constraint condition is configured to: define an area for selecting the correct matching feature point pairs according to the slope of the feature point matching line formed by each point pair in the initial matching point pairs, wherein the number of feature point pairs satisfying the preset matching line slope in the selected area is the largest; and / or The matching line length constraint condition is configured as follows: based on the length of the feature point matching line formed by each point pair in the initial matching point pair, an area for selecting the correct matching feature point pair is limited, wherein the number of feature point pairs that meet the preset matching line length range is the largest in the selected area.
7. The point cloud registration method according to claim 1, characterized in that: The point cloud sampling area is determined based on the extreme value distribution of each feature point in the three-dimensional feature point set.
8. A point cloud registration device, characterized in that: include: a memory configured to store instructions; as well as A processor is configured to call the instructions from the memory and implement the point cloud registration method according to any one of claims 1 to 7 when executing the instructions.
9. A working machine, characterized in that: include: Point cloud acquisition equipment, used to obtain the initial three-dimensional point cloud for a specified scene; as well as The point cloud registration device according to claim 8 is used to perform point cloud registration on the initial three-dimensional point cloud.
10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for causing a machine to execute the point cloud registration method according to any one of claims 1 to 7.