Spatial registration method and device based on feature point cloud, equipment and medium

Abstract

The invention discloses a spatial registration method, device and equipment based on feature point clouds and a medium, and the method comprises the steps: carrying out the coarse registration and alignment of a data point cloud and a model point cloud based on the feature data point cloud coordinates of a preset feature point in an optical positioning system and the feature model point cloud coordinates of the preset feature point in an image space; calculating a coarse registration alignment error of the data point cloud and the model point cloud and a coarse registration error weight based on a coarse registration alignment result; based on the coarse registration alignment error and the coarse registration error weight, performing initial fine registration alignment on the coarse registration data point cloud and the model point cloud in the coarse registration alignment result; and performing iterative optimization on the initial fine registration alignment result to obtain a final fine registration result of the optical positioning system. According to the technical scheme, the stability of spatial registration and the accuracy of a registration result can be improved.

Description

Technical Field

[0001] The present application belongs to the field of spatial registration technology, and specifically relates to a spatial registration method, device, equipment and medium based on feature point cloud. Background Art

[0002] Spatial registration is the operation of accurately registering the positions of the same points on the target to be located and tracked in two different spaces. When tracking a target through an optical positioning system, the accuracy of the optical positioning system and the image space registration directly affects the accuracy of target processing and navigation and the processing results.

[0003] In the prior art, the method of spatial alignment is mainly to stick multiple markers on the surface of the target to be tracked, determine the position of the marker in the image space by performing image recognition on the captured image of the target to be tracked, obtain the position of the marker in the optical positioning coordinate system through the optical positioning system, and perform spatial alignment based on the position of the marker in the optical positioning system and the position in the image space.

[0004] However, due to the limited surface area of ​​the target to be processed, the existing technology can only perform spatial registration based on a small number of identification points. When the identification falls off, spatial registration cannot be continued. At the same time, the existing technology cannot verify the registration results with deviations, and there are problems such as unstable registration process and inaccurate registration results. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a spatial registration method, device, equipment and medium based on feature point cloud, which solves the problems of unstable registration methods and inaccurate registration results using the existing technology. By performing coarse registration and alignment on the data point cloud and the model point cloud, the coarse registration alignment error and the coarse registration error weight are calculated, the coarse registration data point cloud and the model point cloud are initially finely registered and aligned, and the initial fine registration alignment results are iteratively optimized to obtain the final fine registration result of the optical positioning system. The purpose of spatial registration based on point cloud coordinates can be achieved, the stability of the registration and the registration success rate are improved, and the registration results can be error verified and iteratively optimized to improve the accuracy of the registration results.

[0006] In a first aspect, an embodiment of the present application provides a spatial registration method based on a feature point cloud, the method comprising:

[0007] Based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the data point cloud and the model point cloud are roughly registered and aligned;

[0008] Calculating a coarse registration alignment error between the data point cloud and the model point cloud and a coarse registration error weight based on the coarse registration alignment result;

[0009] Based on the coarse registration alignment error and the coarse registration error weight, performing initial fine registration alignment on the coarse registration data point cloud and the model point cloud in the coarse registration alignment result;

[0010] The initial fine registration alignment result is iteratively optimized to obtain the final fine registration result of the optical positioning system.

[0011] Optionally, the calculating the coarse registration alignment error of the data point cloud and the model point cloud and the coarse registration error weight based on the coarse registration alignment result includes:

[0012] Based on a preset matching rule, the coarse registration data point cloud in the coarse registration alignment result is matched with the model point cloud to obtain a coarse registration matching point cloud pair;

[0013] Calculating a coarse registration alignment error based on the coarse registration data point cloud coordinates and the model point cloud coordinates in the coarse registration matching point cloud pair;

[0014] A coarse registration error weight is calculated based on the coarse registration alignment error and a preset error weight algorithm.

[0015] Optionally, the formula of the preset error weight algorithm is:

[0016]

[0017] Among them, w i is the error weight corresponding to the rough registration alignment error of the i-th matching point cloud pair, p i is the point cloud coordinates of the roughly registered data in the matching point cloud pair, q c(i) are the coordinates of the model point cloud in the matching point cloud, R is the rotation matrix between the optical positioning system and the image space in the coarse alignment result, t is the translation matrix between the optical positioning system and the image space in the coarse alignment result, and α is the error weight calculation coefficient.

[0018] Optionally, before performing initial fine registration and alignment on the coarse registration data point cloud in the coarse registration and alignment result and the model point cloud, the method further includes:

[0019] Identifying the number of neighborhood coarsely registered data point clouds within a first preset radius of the coarsely registered data point cloud;

[0020] Determine a noise threshold of the coarse registration data point cloud based on the number of the neighborhood coarse registration data point cloud;

[0021] Performing point cloud filtering on the coarse registration data point cloud based on the coarse registration data point cloud noise threshold to complete the initialization of the coarse registration data point cloud;

[0022] as well as,

[0023] Identify the number of neighborhood model point clouds within a second preset radius of the model point cloud;

[0024] Determine a model point cloud noise threshold based on the number of neighborhood model point clouds;

[0025] Point cloud filtering is performed on the model point cloud based on the model point cloud noise threshold to complete model point cloud initialization.

[0026] Optionally, performing initial fine registration and alignment on the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result based on the coarse registration and alignment error and the coarse registration and alignment error weight includes:

[0027] Based on the coarse registration alignment error, the coarse registration error weight and a preset fine registration conversion algorithm, an initial fine registration conversion relationship between the coarse registration data point cloud and the model point cloud is calculated;

[0028] Based on the initial fine registration transformation relationship, the coarse registration data point cloud is subjected to an initial fine registration transformation to obtain an initial fine registration alignment result.

[0029] Optionally, the iterative optimization of the initial fine registration alignment result to obtain the final fine registration result of the optical positioning system includes:

[0030] Performing point cloud matching on the initial fine registration data point cloud in the initial fine registration alignment result and the model point cloud based on a preset matching rule to obtain an initial fine registration matching point cloud pair;

[0031] Based on the initial fine registration matching point cloud, the initial fine registration data point cloud coordinates and the model point cloud coordinates are centered, calculating the initial fine registration alignment error;

[0032] When the initial fine registration alignment error does not meet a preset error threshold, calculating an initial fine registration alignment error weight based on the initial fine registration alignment error;

[0033] Based on the initial fine registration alignment error and the initial fine registration error weight, updating the initial fine registration transformation relationship;

[0034] The initial fine registration result is iterated based on the initial fine registration transformation relationship update result until the fine registration error meets a preset error threshold or the number of iterations meets a preset iteration threshold, thereby obtaining a final fine registration result of the optical positioning system.

[0035] Optionally, the coarse registration and alignment of the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space includes:

[0036] Extracting feature data point cloud coordinates and feature model point cloud coordinates of preset feature points in the optical positioning system and the image space respectively, wherein the number of the preset feature points is at least three;

[0037] Calculate the coarse registration transformation relationship between the optical positioning system and the image space based on a preset coarse registration algorithm, feature data point cloud coordinates of the same preset feature points, and feature model point cloud coordinates;

[0038] The data point cloud is converted to the image space based on the coarse registration transformation relationship to obtain a coarse registration alignment result of the data point cloud and the model point cloud.

[0039] Optionally, before roughly aligning the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the method further includes:

[0040] Acquire a two-dimensional image of the registration target, and perform three-dimensional reconstruction based on the two-dimensional image;

[0041] Extracting point cloud from the three-dimensional model to obtain a model point cloud of the registered target in the image space;

[0042] as well as,

[0043] Obtaining the optical positioning coordinates of the collected point cloud of the registered target;

[0044] A data point cloud of the registered target under the optical positioning system is determined based on the optical positioning coordinates.

[0045] Optionally, after obtaining the final fine registration result of the optical positioning system, the method further includes:

[0046] Obtain the change of the optical positioning coordinates of the registered target under the optical positioning system;

[0047] Based on the change of the optical positioning coordinates and the final precise registration result, the position and posture of the registered target in the image space are updated.

[0048] In a second aspect, an embodiment of the present application provides a spatial registration device based on a feature point cloud, the device comprising:

[0049] A coarse registration module, used for coarsely registering and aligning the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space;

[0050] A coarse registration error calculation module, used to calculate the coarse registration alignment error of the data point cloud and the model point cloud, and the coarse registration error weight based on the coarse registration alignment result;

[0051] A fine registration module, configured to perform initial fine registration and alignment on the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result based on the coarse registration and alignment error and the coarse registration and alignment error weight;

[0052] The fine registration iteration module is used to iteratively optimize the initial fine registration alignment result to obtain the final fine registration result of the optical positioning system.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method of the first aspect.

[0054] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0055] In the fifth aspect, an embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium, and at least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the method of the first aspect.

[0056] In an embodiment of the present application, based on the feature data point cloud coordinates of preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the data point cloud and the model point cloud are coarsely registered and aligned; based on the coarse registration and alignment result, the coarse registration alignment error and the coarse registration error weight of the data point cloud and the model point cloud are calculated; based on the coarse registration and alignment error and the coarse registration error weight, the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result are initially finely registered and aligned; the initial finely registered and aligned result is iteratively optimized to obtain the final fine registration result of the optical positioning system. The above-mentioned spatial registration method based on feature point cloud is used to solve the problems of unstable registration method and inaccurate registration results existing in the prior art. The data point cloud and the model point cloud are coarsely registered and aligned to calculate the coarse registration alignment error and the coarse registration error weight, and the coarse registration data point cloud and the model point cloud are initially finely registered and aligned. The initial fine registration alignment result is iteratively optimized to obtain the final fine registration result of the optical positioning system. The purpose of spatial registration based on point cloud coordinates can be achieved, the stability of registration and the registration success rate are improved, and the registration result can be error verified and iteratively optimized to improve the accuracy of the registration result. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of a spatial registration method based on feature point cloud provided in an embodiment of the present application;

[0058] Figure 2 It is a schematic diagram of the point cloud of the femur model provided in this application;

[0059] Figure 3 is a flow chart of calculating the rough registration alignment error provided by an embodiment of the present application;

[0060] Figure 4 is a flow chart of iterative optimization alignment results provided by an embodiment of the present application;

[0061] Figure 5 It is a structural block diagram of a spatial registration device based on feature point cloud provided in an embodiment of the present application;

[0062] Figure 6 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0065] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0066] In conjunction with the accompanying drawings, a method, device, equipment and medium for spatial registration based on feature point cloud provided by an embodiment of the present application are described in detail through specific embodiments and their application scenarios.

[0067] Figure 1 This is a flow chart of a spatial registration method based on feature point cloud provided in an embodiment of the present application.

[0068] like Figure 1 As shown, the specific steps include:

[0069] S101, based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the data point cloud and the model point cloud are roughly aligned.

[0070] First, the application scenario of this solution can be a scenario where the processing process of the target to be processed is navigated based on the image information, especially a scenario where the target to be processed is tracked in real time through an optical positioning system. Based on the above application scenarios, it can be understood that the executor of this solution can be an electronic device with data processing and point cloud registration capabilities, such as: smart terminals such as mobile phones and computers.

[0071] Among them, the preset feature points can be points that are pre-collected or determined and can identify the three-dimensional features of the registered target. Taking the human face as an example, the preset feature points can be points at the corners of the eyes, the tip of the nose, and the corners of the mouth. That is, the preset feature points can be identified and found in various spaces. The number of preset feature points can be set according to the coarse registration algorithm and the coarse registration accuracy requirements. The optical positioning system can be a system for collecting the coordinates of the reflective mark under the optical positioning system. The image space can be a three-dimensional image space obtained by photographing the registered target.

[0072] The data point cloud may be a point cloud obtained by collecting the surface of the registered target under the optical positioning system. The feature data point cloud coordinates may be the coordinates of the data point cloud at the feature point position in the optical positioning system coordinate system. The model point cloud may be a point cloud obtained by collecting the three-dimensional model surface of the registered target in the image space. The feature model point cloud coordinates may be the point cloud coordinates of the model point cloud at the feature point position in the image space coordinate system. Both the data point cloud coordinates and the model point cloud coordinates are three-dimensional coordinates. Coarse registration and alignment may be an operation of roughly aligning the data point cloud under the optical positioning system with the model point cloud under the image space. Coarse registration and alignment may be calculating a coarse registration transformation relationship based on the data point cloud coordinates of the feature points under the optical positioning system and the model point cloud coordinates under the image space, and converting the data point cloud to the image space through the coarse registration transformation relationship. The coarse registration transformation relationship may be a transformation matrix for roughly converting the point cloud coordinates under the optical positioning system coordinate system to the image space coordinate system. The coarse registration transformation relationship may include a rotation and translation matrix between the optical positioning system coordinate system and the image space coordinate system. The purpose of coarse registration is to roughly align the data point cloud and the model point cloud when the transformation matrix between the optical positioning system and the image space is completely unknown.

[0073] In one embodiment, since the point clouds of the same preset feature point in different spaces have different point cloud coordinates, the conversion relationship between the feature data point cloud coordinates and the feature model point cloud coordinates, i.e., the coarse registration conversion relationship, can be calculated based on the feature data point cloud coordinates of the same preset feature point in the optical positioning system and the feature model point cloud coordinates in the image space. The coarse registration conversion relationship is used to convert all the data point cloud coordinates in the optical positioning system to the image space coordinate system to perform coarse registration and alignment of the data point cloud and the model point cloud.

[0074] In one embodiment, optionally, based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the data point cloud and the model point cloud are roughly aligned, including:

[0075] Extracting feature data point cloud coordinates and feature model point cloud coordinates of preset feature points in the optical positioning system and the image space respectively, wherein the number of preset feature points is at least three;

[0076] Calculate the coarse registration transformation relationship between the optical positioning system and the image space based on the preset coarse registration algorithm, the feature data point cloud coordinates of the same preset feature points and the feature model point cloud coordinates;

[0077] The data point cloud is converted to the image space based on the coarse registration transformation relationship to obtain a coarse registration alignment result of the data point cloud and the model point cloud.

[0078] The preset rough registration algorithm may be an algorithm for calculating the conversion relationship between different spatial coordinate systems according to the three-dimensional coordinates of different spatial coordinate systems, for example, SVD (Singular Value Decomposition) algorithm.

[0079] In one embodiment, a plurality of feature points for rough registration can be selected in advance on the registration target, and the feature data point cloud coordinates of the point cloud at each preset feature point position collected in advance in the optical positioning system coordinate system and the feature model point cloud coordinates in the image space coordinate system are read respectively. In order to ensure that the transformation relationship of the three-dimensional space can be calculated, the number of preset feature points selected is at least three, preferably four. Based on the feature data point cloud coordinates and feature model point cloud coordinates of the same preset feature point, the preset rough registration algorithm is used to calculate the rough registration transformation relationship between the optical positioning system and the image space. According to the rough registration transformation relationship, the data point cloud is converted from the coordinates under the optical positioning system to the image space, and the rough registration alignment result of the data point cloud and the model point cloud is obtained.

[0080] This solution can reduce the amount of calculation of the coarse registration transformation relationship and improve the calculation efficiency of the coarse registration transformation relationship by extracting the feature data point cloud coordinates and feature model point cloud coordinates of preset feature points under the optical positioning system and the image space respectively, and calculating the coarse registration transformation relationship between the optical positioning system and the image space based on the preset coarse registration algorithm and the point cloud coordinates.

[0081] In one embodiment, optionally, before calculating the coarse registration transformation relationship of the optical positioning system based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the method further includes:

[0082] Obtain a two-dimensional image of the registered target, and perform three-dimensional reconstruction based on the two-dimensional image;

[0083] Extract point cloud from the 3D model to obtain the model point cloud of the registered target in the image space;

[0084] as well as,

[0085] Obtaining the optical positioning coordinates of the collected point cloud of the registered target;

[0086] The data point cloud of the registered target under the optical positioning system is determined based on the optical positioning coordinates.

[0087] The two-dimensional image may be a planar image obtained by photographing the registered target through an image acquisition device. The three-dimensional reconstruction may be an operation of constructing a three-dimensional model of the registered target based on the stacking of multiple two-dimensional images. The model point cloud may be a data set used to form a three-dimensional model of the registered target in the image space. The model point cloud may be obtained by performing point cloud extraction on the three-dimensional model of the registered target.

[0088] In one embodiment, the multi-layer two-dimensional images of the registered target can be acquired through an image acquisition device, and the registered target can be reconstructed in three dimensions in the image space by stacking and extracting the contours of the multi-layer two-dimensional images. For example, a series of two-dimensional images of the entire registered target can be acquired through a CT scanning device. The series of two-dimensional images can be multi-layer two-dimensional slice images of the entire registered target. The image quality can be improved by pre-processing the series of two-dimensional images such as threshold extraction, denoising and contrast enhancement. The registered target part is separated from the overall image by using image segmentation technology, and the segmented two-dimensional image data is reconstructed in three dimensions by using a moving cube three-dimensional reconstruction algorithm. During the reconstruction process, adjacent two-dimensional slice images need to be interpolated to obtain a smoother three-dimensional structure, and the three-dimensional model is displayed in a three-dimensional form by image rendering and visualization technology.

[0089] The morphological processing technology is used to extract the two-dimensional pixel points of the boundary of the registered target in the two-dimensional image, and the two-dimensional pixel points are converted into a three-dimensional image coordinate system for point cloud extraction of the three-dimensional model to obtain the model point cloud of the registered target in the image space.

[0090] In one embodiment, a reflective mark can be set on the probe in advance to construct a probe coordinate system. The three-dimensional coordinates of the probe tip in the probe coordinate system are obtained by needle tip calibration, and the collected point cloud is obtained by the contact between the probe tip and the registered target surface. The optical coordinates of the reflective mark are collected in real time by an optical locator to obtain the optical positioning coordinates of the collected point cloud in the optical positioning system coordinate system, and the optical positioning coordinates are the coordinates of the data point cloud of the registered target under the optical positioning system.

[0091] This solution performs three-dimensional reconstruction based on the two-dimensional image of the registered target, extracts point cloud from the three-dimensional model to obtain the model point cloud in the image space, and determines the data point cloud of the registered target under the optical positioning system through the optical positioning coordinates of the point cloud collected by the registered target, thereby improving the efficiency of point cloud extraction of the registered target in different spaces.

[0092] S102, calculating a coarse registration error between the data point cloud and the model point cloud, and a coarse registration error weight based on the coarse registration result.

[0093] The coarse registration alignment error may be the error between the data point cloud in the image space and the corresponding model point cloud after the data point cloud is converted from the optical positioning system to the image space. The coarse registration error weight may be the degree of influence of the coarse registration alignment error of each point cloud on the subsequent registration transformation relationship calculation. The larger the coarse registration alignment error, the smaller the corresponding error weight is, so as to reduce the influence of the error on the subsequent registration transformation relationship calculation.

[0094] Figure 2 It is a schematic diagram of the point cloud of the femur model provided in this application.

[0095] like Figure 2 As shown in the figure, the white point cloud is the model point cloud extracted based on the three-dimensional model of the femur, and the circled black points are the positions of the preset feature points on the model point cloud. Figure 2 It can be seen that each preset feature point position includes multiple model point clouds.

[0096] Since the point cloud coordinates in the coarse registration process are the coordinates of the corresponding point clouds in the two spaces selected in the optical positioning system and the image space according to the feature point positions of the registered target, and there are multiple point clouds at the same feature point positions, the point clouds selected in the two spaces are likely not completely corresponding, that is, the coordinates of the corresponding point clouds in the two spaces may not be different coordinates of the same point cloud in the two spaces. At this time, there will be a coarse registration alignment error between the data point cloud and the model point cloud.

[0097] In one embodiment, the rough registration data point cloud coordinates of each data point cloud in the image space coordinate system are obtained according to the rough registration alignment result, and the difference between each rough registration data point cloud coordinate and the original coordinate of the corresponding model point cloud in the image space coordinate system is calculated to obtain the rough registration alignment error, and the rough registration error weight corresponding to each rough registration alignment error is calculated according to the rough registration alignment error. According to the rough registration alignment result, the model point cloud image in the image space can be matched with the data point cloud image under the optical positioning system, and the matched image can be divided into regions, and the average value of the rough registration data point cloud coordinates within the same region and the average value of the original coordinates of the model point cloud in the region are calculated, and the difference between the corresponding converted coordinate average and the original coordinate average in each region is calculated to obtain multiple rough registration alignment errors, and the proportion of each rough registration alignment error in all rough registration alignment errors is calculated, and the error weight of each rough registration alignment error is determined according to the principle that the larger the rough registration alignment error value, the smaller the corresponding error weight.

[0098] In one embodiment, optionally, before performing initial fine registration and alignment on the coarse registration data point cloud in the coarse registration and alignment result and the model point cloud, the method further comprises:

[0099] Identifying the number of neighborhood coarsely registered data point clouds within a first preset radius of the coarsely registered data point cloud;

[0100] Determine a noise threshold of the coarse registration data point cloud based on the number of the neighborhood coarse registration data point cloud;

[0101] Performing point cloud filtering on the coarse registration data point cloud based on the coarse registration data point cloud noise threshold to complete the initialization of the coarse registration data point cloud;

[0102] as well as,

[0103] Identify the number of neighborhood model point clouds within a second preset radius of the model point cloud;

[0104] Determine a model point cloud noise threshold based on the number of neighborhood model point clouds;

[0105] Point cloud filtering is performed on the model point cloud based on the model point cloud noise threshold to complete model point cloud initialization.

[0106] Among them, the first preset radius can be a sphere radius pre-set with each data point cloud as the center of the sphere, which is used to filter out the irregularly distributed noise points in the data point cloud after rough registration. The number of neighborhood rough registration data point clouds can be the number of data point clouds within the first preset radius sphere of each point cloud in each rough registration data point cloud. The rough registration data point cloud noise threshold can be the mean and standard deviation of the number of domain data point clouds within the first preset radius of all rough registration data point clouds. The rough registration data point cloud initialization can be an operation of filtering out the noise points in the rough registration data point cloud. Under the second preset radius, it can be a sphere radius pre-set with each model point cloud as the center of the sphere, which is used to filter out the irregularly distributed noise points in the model point cloud. The number of neighborhood model point clouds can be the number of model point clouds within the sphere of the second preset radius of each point cloud in each model point cloud. The model point cloud noise threshold can be the mean and standard deviation of the number of domain model point clouds within the second preset radius of all model point clouds. The model point cloud initialization can be an operation of filtering out the noise points in the model point cloud.

[0107] In one embodiment, scattered and irregularly distributed point clouds can be regarded as noise points, and the noise points far away from the correct point cloud can be filtered out by using an adaptive threshold radius filtering algorithm. By setting an appropriate radius, the number of other points in the sphere neighborhood of a given radius of each point in the data point cloud is calculated, and the mean and standard deviation of the number of neighborhood points of all data points are counted. Point clouds with fewer neighborhood points than the mean and / or standard deviation are regarded as noise points and filtered out.

[0108] In one embodiment, the number of data point clouds within a first preset radius can be determined by identifying the data point cloud image after rough registration, a data point cloud noise threshold can be calculated based on the number of data point clouds, and relatively scattered point clouds in the data point cloud can be filtered out based on the data point cloud noise threshold to complete data point cloud initialization.

[0109] In one embodiment, the number of model point clouds at a second preset radius can be determined by identifying the model point cloud image, the model point cloud noise threshold can be calculated based on the number of model point clouds, and the more scattered point clouds in the model point cloud can be filtered out based on the model point cloud noise threshold to complete the model point cloud initialization.

[0110] This scheme, by counting the number of point clouds with a preset radius, calculating the point cloud noise threshold for point cloud filtering, and completing point cloud initialization, can achieve the purpose of filtering out point cloud noise that may interfere with spatial registration, thereby improving the point cloud quality and facilitating the accuracy of subsequent spatial registration results.

[0111] S103: performing initial fine registration and alignment on the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result based on the coarse registration and alignment error and the coarse registration and alignment error weight.

[0112] Among them, the initial fine registration and alignment can be the operation of first matching and aligning the coarse registration data point cloud with the model point cloud in the image space. The difference between coarse registration and alignment and fine registration and alignment is that coarse registration and alignment is based on the point cloud coordinates of a preset number of feature points in different spaces, and converts the data point cloud from the optical positioning system to the image space; fine registration is specifically based on the data point cloud converted by coarse registration and the corresponding model point cloud in the image space. The operation of aligning the data point cloud and the corresponding model point cloud is to make the data point cloud continue to approach the model point cloud.

[0113] In one embodiment, the initial fine registration transformation relationship between the entire coarse registration data point cloud and the entire model point cloud in the coarse registration alignment result can be calculated based on the coarse registration alignment error and the coarse registration error weight, and the coordinates of the coarse registration data point cloud are transformed in the image space according to the initial fine registration transformation relationship to obtain the initial fine registration alignment result of the coarse registration data point cloud and the model point cloud.

[0114] Optionally, performing initial fine registration and alignment on the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result based on the coarse registration and alignment error and the coarse registration and alignment error weight includes:

[0115] Based on the coarse registration alignment error, the coarse registration error weight and a preset fine registration conversion algorithm, an initial fine registration conversion relationship between the coarse registration data point cloud and the model point cloud is calculated;

[0116] Based on the initial fine registration transformation relationship, the coarse registration data point cloud is subjected to an initial fine registration transformation to obtain an initial fine registration alignment result.

[0117] In one embodiment, the preset fine registration conversion algorithm can be calculated by the following formula:

[0118]

[0119] Among them, w i is the coarse registration error weight of the i-th fine registration data point cloud, ||(Rp i +t)-q c(i) || 2 is the coarse registration alignment error between the i-th fine registration data point cloud and the corresponding model point cloud, R'p i +t' is the coordinate of the i-th precise registration data point cloud in the image space, q c(i)The coordinates of the model point cloud corresponding to the i-th precision registration data point cloud in the image space, R' is the rotation matrix between the optical positioning system and the image space, t' is the translation matrix between the optical positioning system and the image space, R k is the rotation matrix of the kth precise registration, t k is the translation matrix of the kth fine registration.

[0120] The overall error weighted sum of the coarse registration data point cloud can be calculated based on the coarse registration alignment error and coarse registration error weight of each coarse registration data point cloud and the corresponding model point cloud, and the initial fine registration conversion relationship is calculated based on the overall error weighted sum and the algorithm formula of the above-mentioned preset fine registration conversion algorithm. According to the initial fine registration conversion relationship, the coordinates of the coarse registration data point cloud in the image space are subjected to initial fine registration conversion to obtain the initial fine registration alignment result.

[0121] This scheme calculates the initial fine registration conversion relationship between the coarse registration data point cloud and the model point cloud through the coarse registration alignment error, the coarse registration error weight and the preset fine registration conversion algorithm, performs the initial fine registration conversion on the coarse registration data point cloud, and obtains the initial fine registration alignment result, which can improve the efficiency and accuracy of the initial fine registration alignment of the coarse registration data point cloud.

[0122] S104, iteratively optimizing the initial fine registration alignment result to obtain the final fine registration result of the optical positioning system.

[0123] The iterative optimization may be an operation of cyclically optimizing the coordinates of the data point cloud in the image space so as to gradually bring the data point cloud closer to the corresponding model point cloud.

[0124] In one embodiment, the model point cloud corresponding to each data point cloud can be determined by calculating the Euclidean distance between each data point cloud coordinate and all model point cloud coordinates in the initial fine registration alignment result, and the error value of each point cloud pair and the root mean square error of all point cloud pairs can be calculated by the error calculation formula. It is identified whether the root mean square error is less than the preset error threshold. If not, the initial fine registration error weight is calculated based on the error value of each point cloud pair of the data point cloud and the model point cloud in the obtained initial fine registration alignment result, and the conversion relationship of the next fine registration is calculated based on the error value and the initial fine registration error weight to perform the next iterative calculation of fine registration. The above steps are repeated until the root mean square error of all point cloud pairs in the fine registration result is less than the error threshold or the number of iterations reaches the preset iteration number threshold, and the registration result between the data point cloud coordinates and the model point cloud coordinates after the last iteration is used as the final fine registration result of the optical positioning system.

[0125] In one embodiment, optionally, after obtaining the fine registration result of the optical positioning system, the method further includes:

[0126] Obtain the change of the optical positioning coordinates of the registered target under the optical positioning system;

[0127] Based on the changes in the optical positioning coordinates and the final precise registration results, the position and posture of the registered target in the image space are updated.

[0128] The optical positioning coordinate change condition may be whether the positioning coordinate of the registered target under the optical positioning system has changed and the positioning coordinate after the change.

[0129] In one embodiment, since the registered target may move or the processing area may be offset, a target bracket may be fixed on the target to be positioned, and a reflective mark is provided on the target bracket for real-time tracking by the optical positioning system during the processing of the registered target. The target bracket does not need to be placed inside the target to be positioned, nor does it need to scan a two-dimensional image together with the target to be positioned, so it is more convenient and reliable, and the position and posture changes of the target bracket are consistent with those of the registered target. The optical positioning system can track the position and posture changes of the target bracket in real time, determine the changes in the optical positioning coordinates of the registered target based on the position and posture changes of the target bracket, and update the position and posture changes of the registered target to the image space based on the changes in the optical positioning coordinates and the above-mentioned precise registration results. At this time, when the registered target moves, there is no need to perform registration again, and the position and posture of the registered target in the image space can be automatically updated.

[0130] After the target bracket (i.e., the registered target) is registered with the image space, the conversion relationship between the target bracket coordinate system and the image space coordinate system can be considered to be fixed. When the registered target changes its posture, the optical positioning system can be used to track the change in the posture of the target bracket in the optical positioning system coordinate system in real time. According to the change in the posture of the target bracket and the conversion relationship between the target bracket coordinate system and the image space coordinate system, the conversion relationship between the optical positioning system coordinate system and the image space coordinate system is updated, so that the change in the posture of the target bracket (i.e., the registered target) can be reflected in real time in the image space coordinate system, thereby realizing real-time positioning and tracking of the registered target.

[0131] This scheme, by obtaining the changes in the optical positioning coordinates of the registered target under the optical positioning system and updating the position and posture of the registered target in the image space according to the precise registration results, can achieve the purpose of automatically updating the registration transformation relationship according to the real-time position and posture changes of the registered target, thereby improving the real-time and effectiveness of the registered target processing guidance.

[0132] The technical solution provided in the embodiment of the present application performs coarse registration and alignment of the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space; calculates the coarse registration and alignment errors of the data point cloud and the model point cloud, as well as the coarse registration and alignment error weight based on the coarse registration and alignment results; performs initial fine registration and alignment of the coarsely registered data point cloud and the model point cloud in the coarse registration and alignment results with the coarse registration and alignment error based on the coarse registration and alignment error and the coarse registration and alignment error weight; and iteratively optimizes the initial fine registration and alignment results to obtain the final fine registration results of the optical positioning system. The above-mentioned spatial registration method based on feature point cloud is used to solve the problems of unstable registration method and inaccurate registration results existing in the prior art. The data point cloud and the model point cloud are coarsely registered and aligned to calculate the coarse registration alignment error and the coarse registration error weight, and the coarse registration data point cloud and the model point cloud are initially finely registered and aligned. The initial fine registration alignment result is iteratively optimized to obtain the final fine registration result of the optical positioning system. The purpose of spatial registration based on point cloud coordinates can be achieved, the stability of registration and the registration success rate are improved, and the registration result can be error verified and iteratively optimized to improve the accuracy of the registration result.

[0133] Figure 3 is a flow chart of calculating the rough registration alignment error provided by the embodiment of the present application. Figure 3 As shown, the specific steps include:

[0134] S301, performing point cloud matching on the coarse registration data point cloud in the coarse registration alignment result and the model point cloud based on a preset matching rule to obtain a coarse registration matching point cloud pair.

[0135] The preset matching rule may be a pre-set principle for finding the point cloud coordinates of the same point on the registered target in different spatial coordinate systems. For example, the minimum Euclidean distance principle. The matching point cloud pair may be a data point cloud and a model point cloud pair with the minimum Euclidean distance.

[0136] In one embodiment, the distance between each coarsely registered data point cloud and each model point cloud can be calculated according to the coordinates of the data point cloud and the coordinates of the model point cloud in the coarse registration alignment result. At this time, the coarsely registered data point cloud and the model point cloud are point clouds in the image space after coarse registration. According to the preset matching rules and the distance between the point clouds, the coarsely registered data point cloud and the model point cloud are matched to obtain a coarsely registered matching point cloud pair.

[0137] For example: Assume that the coordinates of the coarse registration data point cloud are P = {p1, p2, ..., p n}, the coordinates of the model point cloud are Q = {q1,q2,...,q m}, the corresponding points of the data point cloud and the model point cloud can be assigned according to the principle of minimum Euclidean distance. That is, for any point p in the data point cloud i For example, its corresponding point in the model point cloud is q ck(i) , where c k(i) =argmin||(R k-1 p i +t k-1 )-q c(i) || 2 , R k-1 represents the coarse registration rotation matrix, t k-1 represents the coarse registration translation matrix, c k(i) Represents the minimum Euclidean distance between the i-th coarsely registered data point cloud and the corresponding model point cloud.

[0138] S302, calculating a coarse registration alignment error based on the coarse registration data point cloud coordinates and the model point cloud coordinates in the coarse registration matching point cloud pair.

[0139] The roughly registered data point cloud coordinates may be the coordinates of the data point cloud after the roughly registration, that is, the coordinates in the image space coordinate system. The model point cloud coordinates may be the original coordinates of the model point cloud in the image space coordinate system.

[0140] In one embodiment, the difference between the rough registration data point cloud coordinates and the model point cloud coordinates in each matching point cloud pair can be calculated respectively, and the difference can be used as the rough registration alignment error of the corresponding matching point. The error calculation formula can also be used: i =||(Rp i +t)-q c(i) || 2 Calculate the coarse registration alignment error of the i-th coarsely registered matching point cloud pair.

[0141] S303: Calculate a coarse registration error weight based on the coarse registration alignment error and a preset error weight algorithm.

[0142] The preset error weight algorithm may be an algorithm for calculating the influence of each coarse registration alignment error on the subsequent fine registration transformation relationship calculation.

[0143] In one embodiment, the error weights corresponding to the coarse registration alignment errors are calculated respectively according to the coarse registration alignment errors and a preset error weight algorithm.

[0144] In one embodiment, optionally, the formula of the preset error weight algorithm is:

[0145]

[0146] Among them, w iis the error weight corresponding to the rough registration alignment error of the i-th matching point cloud pair, p i is the point cloud coordinates of the roughly registered data in the matching point cloud pair, q c(i) are the coordinates of the model point cloud in the matching point cloud, R is the rotation matrix between the optical positioning system and the image space in the coarse alignment result, t is the translation matrix between the optical positioning system and the image space in the coarse alignment result, and α is the error weight calculation coefficient.

[0147] In one embodiment, the formula of the preset error weight algorithm is:

[0148]

[0149] Among them, w i is the error weight corresponding to the coarse registration alignment error of the i-th matching point cloud pair, and 0≤w i ≤1,w i is the diagonal element of the weight matrix W, the weight matrix W is a diagonal matrix, p i To match the point cloud to roughly register the data point cloud coordinates, q c(i) The matching point cloud is aligned with the model point cloud coordinates, R is the rotation matrix between the optical positioning system and the image space in the coarse alignment result, t is the translation matrix between the optical positioning system and the image space in the coarse alignment result, and α is the error weight calculation coefficient.

[0150] This solution calculates the error weight by using a preset error weight algorithm formula, which can achieve the purpose of associating the error value of the point cloud registration coordinates with the corresponding error weight, and can ensure that the calculation result is less than or equal to 1, thereby simplifying the calculation steps and improving the calculation efficiency.

[0151] The technical solution provided in the embodiment of the present application, by performing point cloud matching of the coarse registration data point cloud in the coarse registration alignment result with the model point cloud, matching the coarse registration data point cloud coordinates and the model point cloud coordinates in the coarse registration point cloud, calculating the coarse registration alignment error, and calculating the coarse registration error weight based on a preset error weight algorithm, can achieve the purpose of calculating the coarse registration error of the data point cloud after coarse registration and determining the error weight based on the error result, which is beneficial to the accuracy of the subsequent calculation of the fine registration transformation relationship.

[0152] Figure 4 Flowchart of iterative optimization alignment results provided by the embodiment of the present application. Figure 4 As shown, the specific steps include:

[0153] S401, performing point cloud matching on the initial fine registration data point cloud in the initial fine registration alignment result and the model point cloud based on a preset matching rule to obtain an initial fine registration matching point cloud pair.

[0154] In one embodiment, the Euclidean distance between the coordinates of each initial fine registration data point cloud and the coordinates of all model point clouds in the initial fine registration alignment result can be calculated, and the model point cloud with the smallest Euclidean distance is used as the initial fine registration matching point cloud pair of the initial fine registration data point cloud.

[0155] S402, calculating an initial fine registration alignment error based on the initial fine registration data point cloud coordinates and the model point cloud coordinates in the initial fine registration matching point cloud.

[0156] In one embodiment, the initial fine registration error of each initial fine registration matching point cloud pair can be calculated by using the above error calculation formula and the distance between the initial fine registration data point cloud coordinates and the corresponding model point cloud coordinates in the initial fine registration matching point cloud pair, and the overall error of the initial fine registration alignment can be obtained by calculating the root mean square of the initial fine registration errors of each initial fine registration matching point cloud pair.

[0157] S403: When the initial fine registration alignment error does not meet a preset error threshold, calculate an initial fine registration alignment error weight based on the initial fine registration alignment error.

[0158] In one embodiment, when the overall error of the initial fine registration alignment is less than a preset error threshold, it means that the initial fine registration result between the data point cloud and the model point cloud in the image space is not accurate enough, and the initial fine registration alignment error weight can be calculated based on the initial fine registration alignment error and the above-mentioned preset error weight algorithm.

[0159] S404: Update the initial fine registration transformation relationship based on the initial fine registration alignment error and the initial fine registration error weight.

[0160] In one embodiment, the conversion relationship for the next fine registration may be calculated according to a preset fine registration conversion algorithm, an initial fine registration alignment error, and an initial fine registration error weight to update the initial fine registration conversion relationship.

[0161] S405, iterating the initial fine registration result based on the initial fine registration transformation relationship update result until the fine registration error meets a preset error threshold or the number of iterations meets a preset iteration threshold, to obtain a final fine registration result of the optical positioning system.

[0162] The preset iteration number threshold may be a preset maximum iteration number of the transformation relationship, and the preset iteration number threshold may control the number of calculations while ensuring the calculation accuracy of the transformation relationship, thereby avoiding invalid calculations.

[0163] In one embodiment, the initial fine registration data point cloud coordinates in the initial fine registration alignment result can be iteratively transformed according to the initial fine registration transformation relationship update result, and the matching model point cloud pairs and the overall error calculation can be re-searched based on the transformed results until the overall error obtained in the last iteration meets the preset error threshold or the number of iterations meets the preset iteration number threshold.

[0164] The technical solution provided in the embodiment of the present application performs point cloud matching between the initial fine registration data point cloud and the model point cloud to obtain an initial fine registration matching point cloud pair, updates the initial fine registration transformation relationship based on the initial fine registration alignment error and the initial fine registration error weight, and iterates the initial fine registration alignment result until the fine registration alignment error meets a preset error threshold or the number of iterations meets a preset iteration number threshold, thereby obtaining the final fine registration result of the optical positioning system, which can achieve the purpose of iterative optimization of the fine registration results and improve the accuracy of fine registration result evaluation and iteration.

[0165] Figure 5 is a structural block diagram of a spatial registration device based on feature point cloud provided in an embodiment of the present application. Figure 5 As shown, specifically including the following:

[0166] A coarse registration module 501 is used to perform coarse registration and alignment on the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space;

[0167] A coarse registration error calculation module 502, used to calculate the coarse registration alignment error of the data point cloud and the model point cloud, and the coarse registration error weight based on the coarse registration alignment result;

[0168] A fine registration module 503 is used to perform initial fine registration and alignment on the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result based on the coarse registration and alignment error and the coarse registration and alignment error weight;

[0169] The fine registration iteration module 504 is used to iteratively optimize the initial fine registration alignment result to obtain the final fine registration result of the optical positioning system.

[0170] Optionally, the coarse registration error calculation module 502 is specifically used for:

[0171] Based on a preset matching rule, the coarse registration data point cloud in the coarse registration alignment result is matched with the model point cloud to obtain a coarse registration matching point cloud pair;

[0172] Calculating a coarse registration alignment error based on the coarse registration data point cloud coordinates and the model point cloud coordinates in the coarse registration matching point cloud pair;

[0173] A coarse registration error weight is calculated based on the coarse registration alignment error and a preset error weight algorithm.

[0174] Optionally, the formula of the preset error weight algorithm is:

[0175]

[0176] Among them, w i is the error weight corresponding to the rough registration alignment error of the i-th matching point cloud pair, p i is the point cloud coordinates of the roughly registered data in the matching point cloud pair, q c(i) are the coordinates of the model point cloud in the matching point cloud, R is the rotation matrix between the optical positioning system and the image space in the coarse alignment result, t is the translation matrix between the optical positioning system and the image space in the coarse alignment result, and α is the error weight calculation coefficient.

[0177] Optionally, the fine registration module 503 is further used for:

[0178] Identifying the number of neighborhood coarsely registered data point clouds within a first preset radius of the coarsely registered data point cloud;

[0179] Determine a noise threshold of the coarse registration data point cloud based on the number of the neighborhood coarse registration data point cloud;

[0180] Performing point cloud filtering on the coarse registration data point cloud based on the coarse registration data point cloud noise threshold to complete the initialization of the coarse registration data point cloud;

[0181] as well as,

[0182] Identify the number of neighborhood model point clouds within a second preset radius of the model point cloud;

[0183] Determine a model point cloud noise threshold based on the number of neighborhood model point clouds;

[0184] Point cloud filtering is performed on the model point cloud based on the model point cloud noise threshold to complete model point cloud initialization.

[0185] Optionally, the fine registration module 503 is specifically used for:

[0186] Based on the coarse registration alignment error, the coarse registration error weight and a preset fine registration conversion algorithm, an initial fine registration conversion relationship between the coarse registration data point cloud and the model point cloud is calculated;

[0187] Based on the initial fine registration transformation relationship, the coarse registration data point cloud is subjected to an initial fine registration transformation to obtain an initial fine registration alignment result.

[0188] Optionally, the fine registration iteration module 504 is specifically used for:

[0189] Performing point cloud matching on the initial fine registration data point cloud in the initial fine registration alignment result and the model point cloud based on a preset matching rule to obtain an initial fine registration matching point cloud pair;

[0190] Based on the initial fine registration matching point cloud, the initial fine registration data point cloud coordinates and the model point cloud coordinates are centered, calculating the initial fine registration alignment error;

[0191] When the initial fine registration alignment error does not meet a preset error threshold, calculating an initial fine registration alignment error weight based on the initial fine registration alignment error;

[0192] Based on the initial fine registration alignment error and the initial fine registration error weight, updating the initial fine registration transformation relationship;

[0193] The initial fine registration result is iterated based on the initial fine registration transformation relationship update result until the fine registration error meets a preset error threshold or the number of iterations meets a preset iteration threshold, thereby obtaining a final fine registration result of the optical positioning system.

[0194] Optionally, the coarse registration module 501 is specifically used for:

[0195] Extracting feature data point cloud coordinates and feature model point cloud coordinates of preset feature points in the optical positioning system and the image space respectively, wherein the number of the preset feature points is at least three;

[0196] Calculate the coarse registration transformation relationship between the optical positioning system and the image space based on a preset coarse registration algorithm, feature data point cloud coordinates of the same preset feature points, and feature model point cloud coordinates;

[0197] The data point cloud is converted to the image space based on the coarse registration transformation relationship to obtain a coarse registration alignment result of the data point cloud and the model point cloud.

[0198] Optionally, the coarse registration module 501 is further used for:

[0199] Acquire a two-dimensional image of the registration target, and perform three-dimensional reconstruction based on the two-dimensional image;

[0200] Extracting point cloud from the three-dimensional model to obtain a model point cloud of the registered target in the image space;

[0201] as well as,

[0202] Obtaining the optical positioning coordinates of the collected point cloud of the registered target;

[0203] A data point cloud of the registered target under the optical positioning system is determined based on the optical positioning coordinates.

[0204] Optionally, the fine registration iteration module 504 is further used for:

[0205] Obtain the change of the optical positioning coordinates of the registered target under the optical positioning system;

[0206] Based on the change of the optical positioning coordinates and the final precise registration result, the position and posture of the registered target in the image space are updated.

[0207] The technical solution provided in the embodiment of the present application includes a coarse registration module, which is used to perform coarse registration and alignment of the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space; a coarse registration error calculation module, which is used to calculate the coarse registration alignment error and the coarse registration error weight of the data point cloud and the model point cloud based on the coarse registration alignment result; a fine registration module, which is used to perform initial fine registration and alignment of the coarse registration data point cloud and the model point cloud in the coarse registration alignment result based on the coarse registration alignment error and the coarse registration error weight; and a fine registration iteration module, which is used to iteratively optimize the initial fine registration alignment result to obtain the final fine registration result of the optical positioning system. The above-mentioned spatial registration device based on feature point cloud is used to solve the problems of unstable registration method and inaccurate registration result existing in the prior art. The data point cloud and the model point cloud are coarsely registered and aligned to calculate the coarse registration alignment error and the coarse registration error weight, the coarse registration data point cloud and the model point cloud are initially finely registered and aligned, and the initial fine registration alignment result is iteratively optimized to obtain the final fine registration result of the optical positioning system. The purpose of spatial registration based on point cloud coordinates can be achieved, the stability of registration and the registration success rate are improved, and the registration result can be error verified and iteratively optimized to improve the accuracy of the registration result.

[0208] A spatial registration device based on a feature point cloud in an embodiment of the present application can be configured in a component, an integrated circuit, or a chip in a terminal. It can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0209] The spatial registration device based on feature point cloud in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0210] A spatial registration device based on feature point cloud provided in an embodiment of the present application can implement each process implemented in the above-mentioned method embodiments. To avoid repetition, it will not be described again here.

[0211] like Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a program or instruction stored in the memory 602 and executable on the processor 601. When the program or instruction is executed by the processor 601, each process of the above-mentioned embodiment of a spatial registration method based on a feature point cloud is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0212] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0213] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned embodiment of a spatial registration method based on a feature point cloud is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0214] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0215] The present application embodiment further provides a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute a spatial registration method based on a feature point cloud recorded in the present application embodiment. The program product can be implemented in any combination of one or more readable media.

[0216] It should be noted that, in this article, the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or system including the element. In addition, it should be pointed out that the scope of the method and system in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0217] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0218] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0219] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A spatial registration method based on feature point cloud, characterized in that: The method comprises: Based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the data point cloud and the model point cloud are roughly registered and aligned; Calculating a coarse registration alignment error between the data point cloud and the model point cloud and a coarse registration error weight based on the coarse registration alignment result; Based on the coarse registration alignment error and the coarse registration error weight, performing initial fine registration alignment on the coarse registration data point cloud and the model point cloud in the coarse registration alignment result; The initial fine registration alignment result is iteratively optimized to obtain the final fine registration result of the optical positioning system.

2. The spatial registration method based on feature point cloud according to claim 1, characterized in that: The calculating the coarse registration alignment error of the data point cloud and the model point cloud and the coarse registration error weight based on the coarse registration alignment result includes: Based on a preset matching rule, the coarse registration data point cloud in the coarse registration alignment result is matched with the model point cloud to obtain a coarse registration matching point cloud pair; Calculating a coarse registration alignment error based on the coarse registration data point cloud coordinates and the model point cloud coordinates in the coarse registration matching point cloud pair; A coarse registration error weight is calculated based on the coarse registration alignment error and a preset error weight algorithm.

3. The spatial registration method based on feature point cloud according to claim 2, characterized in that: The formula of the preset error weight algorithm is: Among them, w i is the error weight corresponding to the rough registration alignment error of the i-th matching point cloud pair, p i is the point cloud coordinate of the roughly registered data in the matching point cloud pair, q c(i) are the coordinates of the model point cloud in the matching point cloud, R is the rotation matrix between the optical positioning system and the image space in the coarse alignment result, t is the translation matrix between the optical positioning system and the image space in the coarse alignment result, and α is the error weight calculation coefficient.

4. The spatial registration method based on feature point cloud according to claim 1, characterized in that: Before performing initial fine registration and alignment on the coarse registration data point cloud in the coarse registration and alignment result and the model point cloud, the method further comprises: Identifying the number of neighborhood coarsely registered data point clouds within a first preset radius of the coarsely registered data point cloud; Determine a noise threshold of the coarse registration data point cloud based on the number of the neighborhood coarse registration data point cloud; Performing point cloud filtering on the coarse registration data point cloud based on the coarse registration data point cloud noise threshold to complete the initialization of the coarse registration data point cloud; as well as, Identify the number of neighborhood model point clouds within a second preset radius of the model point cloud; Determine a model point cloud noise threshold based on the number of neighborhood model point clouds; Point cloud filtering is performed on the model point cloud based on the model point cloud noise threshold to complete model point cloud initialization.

5. The spatial registration method based on feature point cloud according to claim 1, characterized in that: The performing initial fine registration and alignment on the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result based on the coarse registration and alignment error and the coarse registration and alignment error weight comprises: Based on the coarse registration alignment error, the coarse registration error weight and a preset fine registration conversion algorithm, an initial fine registration conversion relationship between the coarse registration data point cloud and the model point cloud is calculated; Based on the initial fine registration transformation relationship, the coarse registration data point cloud is subjected to an initial fine registration transformation to obtain an initial fine registration alignment result.

6. The spatial registration method based on feature point cloud according to claim 1, characterized in that: The iterative optimization of the initial fine registration alignment result to obtain the final fine registration result of the optical positioning system includes: Performing point cloud matching on the initial fine registration data point cloud in the initial fine registration alignment result and the model point cloud based on a preset matching rule to obtain an initial fine registration matching point cloud pair; Based on the initial fine registration matching point cloud, the initial fine registration data point cloud coordinates and the model point cloud coordinates are centered, calculating the initial fine registration alignment error; When the initial fine registration alignment error does not meet a preset error threshold, calculating an initial fine registration alignment error weight based on the initial fine registration alignment error; Based on the initial fine registration alignment error and the initial fine registration error weight, updating the initial fine registration transformation relationship; The initial fine registration result is iterated based on the initial fine registration transformation relationship update result until the fine registration error meets a preset error threshold or the number of iterations meets a preset iteration threshold, thereby obtaining a final fine registration result of the optical positioning system.

7. The spatial registration method based on feature point cloud according to claim 1, characterized in that: The method of roughly registering and aligning the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space includes: Extracting feature data point cloud coordinates and feature model point cloud coordinates of preset feature points in the optical positioning system and the image space respectively, wherein the number of the preset feature points is at least three; Calculate the coarse registration transformation relationship between the optical positioning system and the image space based on a preset coarse registration algorithm, feature data point cloud coordinates of the same preset feature points, and feature model point cloud coordinates; The data point cloud is converted to the image space based on the coarse registration transformation relationship to obtain a coarse registration alignment result of the data point cloud and the model point cloud.

8. The spatial registration method based on feature point cloud according to claim 1, characterized in that: Before roughly aligning the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space, the method further includes: Acquire a two-dimensional image of the registration target, and perform three-dimensional reconstruction based on the two-dimensional image; Extracting point cloud from the three-dimensional model to obtain a model point cloud of the registered target in the image space; as well as, Obtaining the optical positioning coordinates of the collected point cloud of the registered target; A data point cloud of the registered target under the optical positioning system is determined based on the optical positioning coordinates.

9. The spatial registration method based on feature point cloud according to claim 1, characterized in that: After obtaining the final fine registration result of the optical positioning system, the method further includes: Obtain the change of the optical positioning coordinates of the registered target under the optical positioning system; Based on the change of the optical positioning coordinates and the final precise registration result, the position and posture of the registered target in the image space are updated.

10. A spatial registration device based on feature point cloud, characterized in that: The device comprises: A coarse registration module, used for coarsely registering and aligning the data point cloud and the model point cloud based on the feature data point cloud coordinates of the preset feature points in the optical positioning system and the feature model point cloud coordinates of the preset feature points in the image space; A coarse registration error calculation module, used to calculate the coarse registration alignment error of the data point cloud and the model point cloud, and the coarse registration error weight based on the coarse registration alignment result; A fine registration module, configured to perform initial fine registration and alignment on the coarse registration data point cloud and the model point cloud in the coarse registration and alignment result based on the coarse registration and alignment error and the coarse registration and alignment error weight; The fine registration iteration module is used to iteratively optimize the initial fine registration alignment result to obtain the final fine registration result of the optical positioning system.

11. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein when the program or instruction is executed by the processor, the steps of a spatial registration method based on a feature point cloud are implemented as described in any one of claims 1 to 9.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of a spatial registration method based on a feature point cloud as described in any one of claims 1 to 9 are implemented.

Images