A precise registration method for unordered multi-view 3D point clouds based on recursive primitive shape generation

Abstract

The embodiments of the present application relate to the field of computer vision technology and disclose a method for precise registration of unordered multi-view three-dimensional point clouds based on recursive meta-shape generation, comprising: S1, merging all initial point cloud views into an unupdated meta-shape; S2, entering the iterative process of calling the meta-shape-multi-view update and determining the number of iterations included in this call; S3, updating the unupdated meta-shape; S4, updating all initial point cloud views; S5, merging all updated point cloud views into a new meta-shape; S6, repeating S3 to S5 until the maximum number of iterations corresponding to this call is reached; S7, taking the updated meta-shape after this call, still using all initial point cloud views as the unupdated point cloud views, starting a new call, and repeating S2 to S6 until the maximum number of calls is reached; S8, outputting all updated point cloud views after the last call. This method achieves stable error convergence and avoids falling into local optimality.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for generating an unordered multi-view image based on recursive element shapes. Figure 3 dimensional point cloud precise registration method. Background Art

[0002] Compared with two-dimensional image data, three-dimensional point cloud data is directly sampled by three-dimensional sensors (such as lidar) and has the following advantages: First, with such a direct three-dimensional representation method of point coordinates, the sampling point group can express the three-dimensional shape and geometric relationship of objects in space. Two-dimensional images can only express the projected shape of objects through limited optical information and cannot restore the three-dimensional geometric form of objects; second, it has strong adaptability in complex environments. Even in poor light or severe weather conditions such as storms, high-quality three-dimensional point cloud data can still be obtained through lidar, while the imaging quality of two-dimensional images obtained under the same conditions is greatly reduced.

[0003] Disordered Multi-View Figure 3 The 3D point cloud registration technology is a technology that aligns or stitches point clouds sampled from multiple perspectives to reconstruct a complete point cloud of an object or scene under the condition of unknown LiDAR camera pose and sampling sequence. Because this technology does not require information such as LiDAR pose and sampling sequence, it can more efficiently and simply stitch and reconstruct 3D point clouds. Figure 3 Dimensional point cloud registration technology is widely used in many application scenarios such as digital protection of cultural relics, autonomous driving, and remote sensing stereo mapping, and has high practical value.

[0004] Disordered Multi-View Figure 3 Dimensional point cloud registration technology can be divided into two different technical routes, namely traditional methods based on spanning trees and graphs, and methods based on deep learning. Traditional methods based on spanning trees and graphs mainly evaluate the overlap score between views, use nodes to represent each point cloud view, establish a spanning tree or graph structure for all point cloud views to characterize the relationship between point cloud views, and then perform registration in a certain order. This method relies on the effect of two-view registration in the composition process and subsequent registration steps. There are problems such as being affected by low-overlap views, resulting in poor registration results and long composition time. Deep learning-based methods use networks to extract and match local and global features of point clouds. Although they have good registration results due to the network's high-dimensional feature extraction and attention mechanisms, they require training with a large amount of data for specific scenes. When the scene scale changes or the geometric distribution differs from the training set, the registration success rate is greatly reduced. How to improve the generalization of deep learning-based methods has become an urgent problem that needs to be improved. Summary of the Invention

[0005] In view of this, the embodiment of the present application proposes an unordered multi-view image processing method based on recursive element shape generation. Figure 3 The 3D point cloud registration method recursively calls the meta-shape-multi-view iterative process, and timely widens the distance between the meta-shape and the point cloud view while gradually updating the meta-shape and the point cloud view, thereby achieving stable error convergence and avoiding falling into the local optimum, thereby effectively improving the disordered multi-view Figure 3 The success rate and registration accuracy of 3D point cloud registration.

[0006] To achieve the above objectives, the embodiment of the present application proposes a disordered multi-view image processing method based on recursive element shape generation. Figure 3 The method for precise registration of dimensional point clouds comprises the following steps: S1, merging all initial point cloud views into an unupdated meta-shape; wherein the initial point cloud view is a roughly aligned point cloud view after coarse registration; S2, entering the iterative process of calling meta-shape-multi-view update, and determining the number of iterative rounds of meta-shape-multi-view update included in this call according to the accumulated number of calls; S3, randomly sampling the unupdated meta-shape, selecting seed points, and updating the meta-shape based on the selected seed points to obtain the updated meta-shape; S4, performing point cloud registration on all initial point cloud views and the updated meta-shape in parallel, and All initial point cloud views are aligned to the updated meta-shape to complete the update of the point cloud view of this round; S5, all updated point cloud views are merged into a new meta-shape to complete the update of the meta-shape of this round, thus completing the complete meta-shape-multi-view update of this round; S6, repeat S3 to S5 until the maximum number of iterations corresponding to this call is reached; S7, take the meta-shape updated in this call, still use all initial point cloud views as the unupdated point cloud views, start a new call, repeat S2 to S6 until the maximum number of calls is reached; S8, output all point cloud views updated in the last call to complete the disordered multi-view Figure 3 dimensional point cloud precise registration.

[0007] Optionally, S2 specifically includes: S21, entering the iterative process of calling meta-shape-multi-view update, and calculating the number of iterative rounds of meta-shape-multi-view update included in this call based on the initial number of iterations, the accumulated number of calls and the incremental step of the number of iterations; S22, if this call is the first call, use the unupdated meta-shape to enter this call, if this call is not the first call, use the meta-shape updated in the last call to enter this call; S23, after entering the meta-shape-multi-view update iteration, if this round of iteration is the first round of iteration, use all initial point cloud views to enter this round of iteration, if this round of iteration is not the first round of iteration, use all point cloud views updated in the last round of iteration to enter this round of iteration.

[0008] Optionally, S21 is specifically implemented by the following formula: Among them, Iinit represents the initial iteration number, j represents the cumulative number of calls including this call, ΔI represents the incremental step size of the iteration number, Indicates the number of iterations of the meta-shape-multi-view update included in this call.

[0009] Optionally, S3 specifically includes: S31, based on the number of iterations of the current iteration, calculating the number of sampling points corresponding to the current iteration, and randomly sampling the element shapes used in the current iteration according to the number of sampling points corresponding to the current iteration, and selecting seed points; the formula for calculating the number of sampling points corresponding to the current iteration is: Among them, i represents the number of iterations in this round, N all Represents the total number of points in all point cloud views, λ is the preset hyperparameter that controls the sampling ratio, Indicates the number of sampling points corresponding to the current iteration calculated; S32, in the meta-shape used in the current iteration, a kNN nearest neighbor search is performed on each selected seed point, and k i k nearest neighbors, i =k init +(i-1)×Δk,k init represents the initial k value, Δk represents the increment step of the nearest neighbor point; S33, based on the k selected for each seed point i The nearest neighbor points constitute the local point cloud P local , the local point cloud P local The centroid of the updated seed point is the updated set of seed points, which is the updated element shape P meta .

[0010] Optionally, S4 specifically includes: S41, parallelly combining all point cloud views used in this round of iteration with the updated primitive shape P meta , use the ICP algorithm to perform point cloud precision registration, align all point cloud views to the updated primitive shape, and obtain the updated rotation parameters and translation parameters of each point cloud view; S42, adjust the posture of each point cloud view based on the updated rotation parameters and translation parameters of each point cloud view, and complete the update of this round of point cloud views.

[0011] Optionally, the updated rotation and translation parameters of the point cloud view are expressed by the following formula:

[0012]

[0013] Among them, R represents the rotation parameter, t represents the translation parameter, and p z Represents a point z in the point cloud view, N represents the total number of points in the point cloud view, p meta Represents the updated primitive shape P of point z meta The corresponding points in .

[0014] Optionally, the maximum number of calls is 30.

[0015] To achieve the above purpose, the embodiment of the present application also proposes a disordered multi-view based on recursive element shape generation. Figure 3 The invention discloses a point cloud precise registration system, comprising: a preparation module M1, used to merge all initial point cloud views into an unupdated meta-shape, wherein the initial point cloud view is a roughly aligned point cloud view after rough registration; an iteration number determination module M2, used to enter the iterative process of calling meta-shape-multi-view update, and determine the number of iteration rounds of meta-shape-multi-view update included in this call according to the accumulated number of calls; a meta-shape first update module M3, used to randomly sample the unupdated meta-shape, select seed points, and update the meta-shape based on the selected seed points to obtain the updated meta-shape; a point cloud view update module M4, used to perform point cloud registration on all initial point cloud views and the updated meta-shape in parallel, register all initial point cloud views to the updated meta-shape, and complete the update of the point cloud view of this round; a meta-shape second ... The update module M5 is used to merge all the updated point cloud views into a new meta-shape, complete the meta-shape update of this round, and thus complete the meta-shape-multi-view update of this round; the first discrimination module M6 is used to determine whether the maximum number of iterations corresponding to this call has been reached. If so, it enters the second discrimination module M7, otherwise, it returns to the meta-shape first update module M3; the second discrimination module M7 is used to determine whether the maximum number of calls has been reached. If so, it enters the final output module M9, otherwise, it enters the recursive call module M8; the recursive call module M8 is used to take the meta-shape updated by this call, still using all the initial point cloud views as the unupdated point cloud views, start a new call, and return to the iteration round determination module M2; the final output module M9 is used to output all the point cloud views updated by the last call, completing the disordered multi-view Figure 3 dimensional point cloud precise registration.

[0016] To achieve the above-mentioned purpose, an embodiment of the present application further proposes an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned disordered multi-view image generation based on recursive element shapes. Figure 3 dimensional point cloud precise registration method.

[0017] To achieve the above object, the embodiment of the present application further proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the above-mentioned disordered multi-view image generation based on recursive element shapes. Figure 3 dimensional point cloud precise registration method.

[0018] The embodiment of the present application provides a disordered multi-view image based on recursive element shape generation. Figure 3 The 3D point cloud fine registration method merges all the roughly aligned initial point cloud views after coarse registration into unupdated meta-shapes, and indirectly eliminates the errors between all point cloud views through gradually growing meta-shapes, which increases the overlap rate between views and registration objects during point cloud registration, thereby improving the success rate and registration accuracy of view point cloud registration. Even when faced with a low overlap rate between views, it can still eliminate the errors between point cloud views relatively stably, does not rely on auxiliary structures such as spanning trees or compositions, and has strong robustness and universality. The scheme of large-loop recursive calling and small-loop meta-shape-multi-view iterative updating is adopted. Through the small-loop meta-shape-multi-view iterative updating, the errors of all point cloud views are continuously eliminated, while the large-loop recursive calling only updates the meta-shapes, which timely widens the distance between the point cloud views and the meta-shapes, avoiding the error of all point cloud views. Figure 1 The vertical element shape registration falls into the dilemma of local optimality, which effectively improves the disordered multi-view Figure 3 In addition, the entire point cloud registration process does not involve neural networks and does not require any learning-based training steps, which effectively controls the computational cost and achieves a good balance between benefits and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the following is a brief introduction to the drawings required for use in the embodiments of the present application or the description of the related technologies. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 An embodiment of the present invention provides an unordered multi-view image generated based on recursive element shapes. Figure 3 Flowchart of the 3D point cloud precise registration method;

[0021] Figure 2 are all point cloud views with residuals of the input provided in one embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a meta-shape gradually growing with iterations provided in one embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a point cloud view error gradually eliminated with iterations provided in one embodiment of the present application;

[0024] Figure 5This is an output error-eliminated multi-view point cloud rendering provided in one embodiment of the present application;

[0025] Figure 6 This is a comparison diagram of all point cloud views of a satellite simulation data set input and a multi-view point cloud rendering of an error-eliminated satellite simulation data set output provided in one embodiment of the present application;

[0026] Figure 7 Another embodiment of the present invention provides an unordered multi-view image generated based on recursive element shapes. Figure 3 Schematic diagram of the structure of the 3D point cloud precise registration system;

[0027] Figure 8 This is a structural diagram of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0029] In order to solve the problems existing in the traditional methods based on spanning trees and graphs and the methods based on deep learning, an embodiment of the present application proposes an unordered multi-view method based on recursive meta-shape generation. Figure 3 The method for precise registration of dimensional point clouds is applied to electronic devices, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described by taking the server as an example. The following is an example of a disordered multi-view image based on recursive element shape generation proposed in this embodiment. Figure 3 The following content is only for the convenience of understanding the implementation details and is not necessary for the implementation of this solution.

[0030] This embodiment proposes a disordered multi-view based on recursive element shape generation Figure 3 The specific process of the dimensional point cloud precise registration method can be as follows: Figure 1 Shown, including:

[0031] S1, merge all initial point cloud views into unupdated primitive shapes, where the initial point cloud views are roughly aligned point cloud views after coarse registration.

[0032] In the specific implementation, the server first needs to obtain all the initial point cloud views. The initial point cloud views are roughly aligned point cloud views after rough registration, but all the initial point cloud views still have residuals, disordered multi-view Figure 3 The ultimate goal of 3D point cloud registration is to eliminate these residuals. After obtaining all the initial point cloud views, the server needs to merge all the initial point cloud views into an unupdated primitive shape, which is an imperfect and rough primitive shape.

[0033] In one example, all point cloud views with residuals in the input can be Figure 2 As shown, the server merges it into an unupdated metashape

[0034] S2, entering the iterative process of calling the meta-shape-multi-view update, and determining the number of iterative rounds of the meta-shape-multi-view update included in this call according to the accumulated number of calls.

[0035] In the specific implementation, after the server obtains the unupdated metashape, it can enter the iterative process of calling the metashape-multi-view update. This first requires determining the number of iterative rounds of metashape-multi-view update included in this call based on the cumulative number of calls. As the number of calls increases, the number of iterative rounds of metashape-multi-view update also increases.

[0036] In one example, S2 specifically includes:

[0037] S21, entering the iterative process of calling the meta-shape-multi-view update, and calculating the number of meta-shape-multi-view update iterations included in this call based on the initial iteration number, the accumulated number of calls and the incremental step of the iteration number.

[0038] The number of iterations corresponding to this call can be calculated using the following formula:

[0039]

[0040] Among them, I init represents the initial number of iterations, j represents the cumulative number of calls including this call, ΔI represents the incremental step size of the number of iterations, Indicates the number of iterations of the meta-shape-multi-view update included in this call.

[0041] I init It can be set to 5, ΔI can be set to 5. For the first call, a total of 5 iterations of meta-shape-multi-view update are required. For the second call, a total of 10 iterations of meta-shape-multi-view update are required.

[0042] S22, if this is the first call, then use the unupdated meta-shape to enter this call; if this is not the first call, then use the meta-shape updated in the previous call to enter this call.

[0043] For the first call, it is naturally necessary to use the unupdated meta-shape to enter the first call, and for the j-th call, it is necessary to use the meta-shape updated in the j-1-th call to enter the j-th call.

[0044] S23, after entering the meta-shape-multi-view update iteration, if this round of iteration is the first round of iteration, all initial point cloud views are used to enter this round of iteration; if this round of iteration is not the first round of iteration, all point cloud views updated in the previous round of iteration are used to enter this round of iteration.

[0045] After entering a called meta-shape-multi-view update iteration, for the first round of iteration, it is naturally necessary to use all initial point cloud views to enter the first round of iteration, and for the i-th round of iteration, it is necessary to use all point cloud views updated after the i-1th round of iteration to enter the i-th round of iteration.

[0046] S3, randomly sampling the unupdated meta-shapes, selecting seed points, and updating the meta-shapes based on the selected seed points to obtain updated meta-shapes.

[0047] In the specific implementation, after the server enters the first round of iteration, it first updates the unupdated meta-shapes. This requires random sampling of the unupdated meta-shapes, selecting seed points, and updating the meta-shapes based on the selected seed points to obtain updated meta-shapes.

[0048] In one example, S3 specifically includes:

[0049] S31, based on the number of iterations of this round, calculating the number of sampling points corresponding to this round, and randomly sampling the element shapes used in this round according to the number of sampling points corresponding to this round, to select seed points.

[0050] The formula for calculating the number of sampling points corresponding to this round of iteration is:

[0051]

[0052] Among them, i represents the number of iterations in this round, N all Represents the total number of points in all point cloud views, λ is the preset hyperparameter that controls the sampling ratio, Indicates the calculated number of sampling points corresponding to this round of iteration.

[0053] Generally speaking, λ can be set to 200.

[0054] S32, in the meta-shape used in this round of iteration, a kNN nearest neighbor search is performed on each selected seed point, and k i k nearest neighbors, i =k init +(i-1)×Δk,k init represents the initial k value, and Δk represents the incremental step size of the nearest neighbor point.

[0055] Generally speaking, k init It can be set to 3 and Δk can be set to 2. That is to say, in the first round of iteration, each seed point needs to select 3 nearest neighbor points, and in the second round of iteration, each seed point needs to select 5 nearest neighbor points.

[0056] S33, based on the k selected for each seed point i The nearest neighbor points constitute the local point cloud P local , the local point cloud P local The centroid of the updated seed point is the updated set of seed points, which is the updated element shape P meta .

[0057]

[0058] S4, perform point cloud registration on all initial point cloud views and the updated primitive shape in parallel, register all initial point cloud views to the updated primitive shape, and complete the update of this round of point cloud views.

[0059] In the specific implementation, for the first round of iteration, the server performs point cloud registration on all initial point cloud views with the updated meta-shape in parallel, taking each initial point cloud view as the source point cloud and the updated meta-shape as the target point cloud, and registers all initial point cloud views to the updated meta-shape to complete the update of the point cloud view in this round.

[0060] In one example, S4 specifically includes:

[0061] S41, parallelize all point cloud views used in this round of iteration with the updated primitive shape P meta ,The ICP algorithm is used to perform point cloud precise ,registration, all point cloud views are registered to the updated ,metashape, and the updated rotation and translation parameters of each point cloud ,view are obtained.

[0062] Among them, the ICP algorithm is used with a matching distance screening threshold of 4 times the point cloud resolution, the closest matching, and a maximum number of iterations of no more than 20.

[0063] S42 , adjusting the position of each point cloud view based on the updated rotation parameters and translation parameters of each point cloud view, to complete the update of this round of point cloud views.

[0064] The updated rotation and translation parameters of the point cloud view can be expressed by the following formula:

[0065]

[0066] Among them, R represents the rotation parameter, t represents the translation parameter, and p z Represents a point z in the point cloud view, N represents the total number of points in the point cloud view, p meta Represents the updated primitive shape P of point z meta The corresponding points in .

[0067] S5: Merge all updated point cloud views into a new meta-shape, completing the meta-shape update of this round. This completes the meta-shape-multi-view update of this round.

[0068] In the specific implementation, after completing the update of the point cloud view in this round, the server needs to update the meta-shape for the second time, that is, merge all the updated point cloud views into a new meta-shape to complete the update of the meta-shape in this round, thus completing the complete meta-shape-multi-view update in this round.

[0069] S6: Repeat S3 to S5 until the maximum number of iterations corresponding to this call is reached.

[0070] In the specific implementation, after the server completes a round of iteration, it determines whether the accumulated number of iteration rounds has reached the maximum number of iteration rounds corresponding to this call. If it has reached it, the call is completed. Otherwise, it returns to S3 and repeats S3 to S5 until the maximum number of iteration rounds corresponding to this call is reached.

[0071] In one example, Figure 3 、 Figure 4 As shown in Figure 2, as the meta-shape-multi-view update is performed over multiple iterations, the meta-shape gradually grows with each iteration, and the errors between all point cloud views are gradually eliminated with each iteration.

[0072] S7, take the updated primitive shape of this call, still use all the initial point cloud views as the unupdated point cloud views, start a new call, and repeat S2 to S6 until the maximum number of calls is reached.

[0073] In the specific implementation, after the current call is completed, the server needs to determine whether the cumulative number of calls has reached the preset maximum number of calls (usually set to 30 times). If it has reached it, the recursive call ends. Otherwise, it returns to S2, takes the updated primitive shape of this call, and still uses all the initial point cloud views as the unupdated point cloud views. A new call is started, and S2 to S6 are repeated until the maximum number of calls is reached.

[0074] S8, output all point cloud views after the last call update, completing the disordered multi-view Figure 3dimensional point cloud precise registration.

[0075] In the specific implementation, after the recursive call ends, the server outputs all the point cloud views updated after the last call (such as Figure 5 As shown), the disordered multi-view is now complete. Figure 3 dimensional point cloud precise registration.

[0076] In an example, the method proposed in this embodiment is applied to the actual multi-view point cloud precise registration, and the effect is as follows: Figure 6 As shown, from the performance of the method proposed in this embodiment on the satellite simulation data set, it can be seen that the method proposed in this embodiment can eliminate the residuals between the multiple input point cloud views and obtain a high-precision multi-view point cloud through alignment.

[0077] In this embodiment, all roughly aligned initial point cloud views after coarse registration are merged into unupdated meta-shapes, and the errors between all point cloud views are indirectly eliminated through gradually growing meta-shapes, which increases the overlap rate between views and registration objects during point cloud registration, thereby improving the success rate and registration accuracy of view point cloud registration. Even when faced with a low overlap rate between views, the errors between point cloud views can still be eliminated relatively stably, without relying on auxiliary structures such as spanning trees or compositions, and have strong robustness and universality. A scheme of large-loop recursive calls and small-loop meta-shape-multi-view iterative updates is adopted. Through the small-loop meta-shape-multi-view iterative updates, errors are continuously eliminated for all point cloud views. The large-loop recursive calls only update the meta-shapes, which timely widens the distance between the point cloud views and the meta-shapes, avoiding the problem of all point cloud views being out of sync. Figure 1 The vertical element shape registration falls into the dilemma of local optimality, which effectively improves the disordered multi-view Figure 3 In addition, the entire point cloud registration process does not involve neural networks and does not require any learning-based training steps, which effectively controls the computational cost and achieves a good balance between benefits and efficiency.

[0078] The step division of the above various methods is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0079] Another embodiment of the present application proposes an unordered multi-view method based on recursive primitive shape generation. Figure 3 dimensional point cloud precise registration system, the following is a disordered multi-view based on recursive element shape generation proposed in this embodiment Figure 3The following content is only provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0080] The proposed method of this embodiment is based on the disordered multi-view generation of recursive element shapes. Figure 3 The specific structure of the 3D point cloud precise registration system can be as follows: Figure 7 As shown, it includes: a preparation module M1, an iteration round determination module M2, a meta-shape first update module M3, a point cloud view update module M4, a meta-shape second update module M5, a first discrimination module M6, a second discrimination module M7, a recursive call module M8 and a final output module M9.

[0081] The preparation module M1 is used to merge all initial point cloud views into non-updated primitive shapes, wherein the initial point cloud views are roughly aligned point cloud views after coarse registration.

[0082] The iteration round number determination module M2 is used to enter the iterative process of calling the meta-shape-multi-view update and determine the number of iteration rounds of the meta-shape-multi-view update included in this call according to the accumulated number of calls.

[0083] The first meta-shape updating module M3 is used to randomly sample the unupdated meta-shapes, select seed points, and update the meta-shapes based on the selected seed points to obtain updated meta-shapes.

[0084] The point cloud view updating module M4 is used to perform point cloud registration on all initial point cloud views and the updated primitive shapes in parallel, register all initial point cloud views to the updated primitive shapes, and complete the update of this round of point cloud views.

[0085] The meta-shape second updating module M5 is used to merge all updated point cloud views into a new meta-shape, completing the meta-shape update of this round, thus completing the meta-shape-multi-view update of this round.

[0086] The first determination module M6 is used to determine whether the maximum number of iterations corresponding to this call has been reached. If so, it enters the second determination module M7; otherwise, it returns to the first meta-shape update module M3.

[0087] The second determination module M7 is used to determine whether the maximum number of calls has been reached. If so, it enters the final output module M9; otherwise, it enters the recursive call module M8.

[0088] The module M8 is recursively called to obtain the updated primitive shape of this call, still using all initial point cloud views as the unupdated point cloud views, start a new call, and return to the iteration number determination module M2.

[0089] The final output module M9 is used to output all point cloud views after the last call update, completing the disordered multi-view Figure 3 dimensional point cloud precise registration.

[0090] It is worth mentioning that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovation of this application, this embodiment does not include units that are not closely related to solving the technical problem proposed by this application. However, this does not mean that other units do not exist in this embodiment.

[0091] It is not difficult to find that this embodiment is a system embodiment corresponding to the above-mentioned method embodiment, and this embodiment can be implemented in conjunction with the above-mentioned method embodiment. The relevant technical details and technical effects mentioned in the above-mentioned embodiments are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above-mentioned embodiments.

[0092] Another embodiment of the present application provides an electronic device, the structure of which is as follows: Figure 8 As shown, it includes: at least one processor C1; and a memory C2 in communication with the at least one processor C1; wherein the memory C2 stores instructions that can be executed by the at least one processor C1, and the instructions are executed by the at least one processor C1 to enable the at least one processor C1 to perform the disordered multi-view image generation based on recursive element shapes described in the above-mentioned method embodiments. Figure 3 dimensional point cloud precise registration method.

[0093] The memory and processor can be connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are well known in the art and therefore will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by the processor is transmitted on a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0094] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0095] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the computer program can implement the above-mentioned method embodiments of generating an unordered multi-view image based on recursive element shapes. Figure 3 dimensional point cloud precise registration method.

[0096] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0097] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for precise registration of unordered multi-view 3D point clouds based on recursive primitive shape generation, characterized in that: include: S1, merge all initial point cloud views into an unupdated primitive shape; the initial point cloud view is the roughly aligned point cloud view after coarse registration; S2, enter the iterative process of calling meta-shape-multi-view update, and determine the number of iterative rounds of meta-shape-multi-view update included in this call based on the accumulated number of calls; S3, randomly sampling the unupdated meta-shapes, selecting seed points, and updating the meta-shapes based on the selected seed points to obtain updated meta-shapes; S4, perform point cloud registration on all initial point cloud views and the updated primitive shape in parallel, register all initial point cloud views to the updated primitive shape, and complete the update of this round of point cloud views; S5: Merge all updated point cloud views into a new meta-shape, completing the meta-shape update of this round. This completes the meta-shape-multi-view update of this round. S6, repeat S3 to S5 until the maximum number of iterations corresponding to this call is reached; S7, taking the updated primitive shape of this call, still using all the initial point cloud views as the unupdated point cloud views, starting a new call, and repeating S2 to S6 until the maximum number of calls is reached; S8, output all the updated point cloud views after the last call, and complete the precise registration of the unordered multi-view 3D point cloud; S3 specifically includes: S31, based on the number of iterations of this round, calculate the number of sampling points corresponding to this round, and randomly sample the element shapes used in this round according to the number of sampling points corresponding to this round, and select seed points; the formula for calculating the number of sampling points corresponding to this round is: ； in, Indicates the number of iterations in this round, Indicates the total number of points in all point cloud views, is a preset hyperparameter for controlling the sampling ratio. Indicates the calculated number of sampling points corresponding to this round of iteration; S32, in the meta-shape used in this round of iteration, a kNN nearest neighbor search is performed on each selected seed point to select nearest neighbor points, , Indicates initial value, Indicates the incremental step size of the nearest neighbor point; S33, based on the selection of each seed point The nearest neighbor points form a local point cloud , the local point cloud The centroid point is used as the updated seed point, and the set of updated seed points is the updated meta-shape ; S4 specifically includes: S41, parallelize all point cloud views used in this iteration with the updated primitive shape , use the ICP algorithm to perform point cloud precise registration, align all point cloud views to the updated primitive shape, and obtain the updated rotation and translation parameters of each point cloud view; S42, adjusting the position of each point cloud view based on the updated rotation parameters and translation parameters of each point cloud view, completing the update of this round of point cloud views; The updated rotation and translation parameters of the point cloud view are expressed by the following formula: ； in, represents the rotation parameter, represents the translation parameter, Represents a point in the point cloud view , Indicates the total number of points in the point cloud view. Indicates a point In the updated meta shape The corresponding points in .

2. The method for precise registration of unordered multi-view 3D point clouds based on recursive primitive shape generation according to claim 1, characterized in that: S2 specifically includes: S21, entering the iterative process of calling the meta-shape-multi-view update, and calculating the number of meta-shape-multi-view update iterations included in this call based on the initial iteration number, the accumulated number of calls, and the iteration increment step; S22, if this is the first call, then use the unupdated meta-shape to enter this call; if this is not the first call, then use the meta-shape updated in the previous call to enter this call; S23, after entering the meta-shape-multi-view update iteration, if this round of iteration is the first round of iteration, all initial point cloud views are used to enter this round of iteration; if this round of iteration is not the first round of iteration, all point cloud views updated in the previous round of iteration are used to enter this round of iteration.

3. The method for precise registration of unordered multi-view 3D point clouds based on recursive primitive shape generation according to claim 2, wherein: S21 is specifically implemented through the following formula: ； in, represents the initial number of iterations, Indicates the cumulative number of calls including this call. Indicates the incremental step size of the number of iterations, Indicates the number of iterations of the meta-shape-multi-view update included in this call.

4. A method for precise registration of unordered multi-view 3D point clouds based on recursive primitive shape generation according to any one of claims 1 to 3, characterized in that: The maximum number of calls is 30.

5. A system for precise registration of unordered multi-view 3D point clouds based on recursive primitive shape generation, characterized in that: include: A preparation module M1 is used to merge all initial point cloud views into non-updated primitive shapes, wherein the initial point cloud views are roughly aligned point cloud views after coarse registration; The iteration round number determination module M2 is used to enter the iterative process of calling the meta-shape-multi-view update and determine the number of iterative rounds of the meta-shape-multi-view update included in this call according to the accumulated number of calls; a first meta-shape updating module M3, configured to randomly sample the unupdated meta-shapes, select seed points, and update the meta-shapes based on the selected seed points to obtain updated meta-shapes; The point cloud view update module M4 is used to perform point cloud registration on all initial point cloud views and the updated primitive shapes in parallel, register all initial point cloud views to the updated primitive shapes, and complete the point cloud view update of this round; The meta-shape second updating module M5 is used to merge all updated point cloud views into a new meta-shape, completing the meta-shape update of this round, thus completing the meta-shape-multi-view update of this round; The first judgment module M6 is used to judge whether the maximum number of iterations corresponding to this call has been reached. If so, it enters the second judgment module M7; otherwise, it returns to the first meta-shape update module M3; The second judgment module M7 is used to determine whether the maximum number of calls has been reached. If so, it enters the final output module M9; otherwise, it enters the recursive call module M8; Recursively call module M8 to obtain the updated primitive shape of this call, still use all initial point cloud views as the unupdated point cloud views, start a new call, and return to the iteration number determination module M2; The final output module M9 is used to output all the updated point cloud views after the last call, completing the precise registration of the disordered multi-view 3D point cloud; Randomly sample the unupdated meta-shapes, select seed points, and update the meta-shapes based on the selected seed points to obtain updated meta-shapes, specifically including: Based on the number of iterations in this round, the number of sampling points corresponding to this round is calculated, and the element shapes used in this round are randomly sampled according to the number of sampling points corresponding to this round, and the seed points are selected. The formula for calculating the number of sampling points corresponding to this round is: ； in, Indicates the number of iterations in this round, Indicates the total number of points in all point cloud views, is a preset hyperparameter for controlling the sampling ratio. Indicates the calculated number of sampling points corresponding to this round of iteration; In the meta-shape used in this round of iteration, a kNN nearest neighbor search is performed on each selected seed point to select nearest neighbor points, , Indicates initial value, Indicates the incremental step size of the nearest neighbor point; Based on each seed point The nearest neighbor points form a local point cloud , the local point cloud The centroid point is used as the updated seed point, and the set of updated seed points is the updated meta-shape ; Parallel point cloud registration is performed on all initial point cloud views with the updated primitive shapes. All initial point cloud views are registered to the updated primitive shapes to complete the update of this round of point cloud views. Specifically, the following steps are performed: In parallel, all point cloud views used in this iteration are combined with the updated primitive shape , use the ICP algorithm to perform point cloud precise registration, align all point cloud views to the updated primitive shape, and obtain the updated rotation and translation parameters of each point cloud view; Adjust the position of each point cloud view based on the updated rotation and translation parameters of each point cloud view to complete the update of this round of point cloud views; The updated rotation and translation parameters of the point cloud view are expressed by the following formula: ； in, represents the rotation parameter, represents the translation parameter, Represents a point in the point cloud view , Indicates the total number of points in the point cloud view. Indicates a point In the updated meta shape The corresponding points in .

6. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the unordered multi-view three-dimensional point cloud precise registration method based on recursive primitive shape generation as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can implement the method for precise registration of unordered multi-view three-dimensional point clouds based on recursive primitive shape generation according to any one of claims 1 to 4.

Images