Point cloud registration method, device, electronic device and computer readable storage medium

Abstract

The present application discloses a point cloud registration method, device, electronic device and computer-readable storage medium, which relate to the field of image processing. The method includes: taking the point in the source point cloud data that is closest to the first point of interest as the first key point, and taking the point in the target point cloud data that is closest to the second point of interest as the second key point, wherein the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data; according to the coordinate difference information between the first key point and the second key point, updating the coordinate information of the first key point, and taking the updated first key point as the third key point; registering the source point cloud data and the target point cloud data according to the second key point and the third key point to obtain a registration result. The present application solves the technical problem of poor registration flexibility caused by only selecting interest points on the target image for registration in the prior art.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a point cloud registration method, device, electronic device and computer-readable storage medium. Background Art

[0002] Point cloud registration plays a vital role in 3D reconstruction, autonomous driving, robot navigation, and computer vision. The main task of point cloud registration is to align two or more point cloud data to obtain a unified coordinate system, thereby achieving more accurate 3D model construction.

[0003] However, most traditional point cloud registration methods only support the selection of interest points on the target image for registration, and cannot simultaneously select points on the source image (floating image). This limitation results in a lack of flexibility and accuracy in the registration process, especially when it is necessary to process point clouds containing a large amount of redundant data or noise, resulting in low efficiency and insufficient robustness.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The present application provides a point cloud registration method, device, electronic device and computer-readable storage medium to at least solve the technical problem of poor registration flexibility caused by only selecting interest points on the target image for registration in the prior art.

[0006] According to one aspect of the present application, a point cloud registration method is provided, including: taking the point in the source point cloud data that is closest to the first point of interest as the first key point, and taking the point in the target point cloud data that is closest to the second point of interest as the second key point, wherein the source point cloud data and the target point cloud data are two point cloud data that need to be registered, the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data; according to the coordinate difference information between the first key point and the second key point, updating the coordinate information of the first key point, and taking the updated first key point as the third key point; and registering the source point cloud data and the target point cloud data according to the second key point and the third key point to obtain a registration result.

[0007] Optionally, in the point cloud registration method of the present application, the coordinate information of the first key point is updated according to the coordinate difference information between the first key point and the second key point, including: adjusting the source point cloud data to be aligned with the target point cloud data according to the coordinate difference information, and using the adjusted source point cloud data as the first point cloud data; after adjusting the source point cloud data to the first point cloud data, updating the coordinate information of the first key point according to the coordinate difference information.

[0008] Optionally, in the point cloud registration method of the present application, the source point cloud data and the target point cloud data are registered according to the second key point and the third key point to obtain a registration result, including: determining a first area in the target point cloud data according to the second key point, wherein the second key point is located within the first area; determining a second area in the first point cloud data according to the third key point, wherein the third key point is located within the second area; and registering the source point cloud data and the target point cloud data based on the first area and the second area to obtain a registration result, wherein the first area corresponds to the target point cloud data and the second area corresponds to the source point cloud data.

[0009] Optionally, in the point cloud registration method of the present application, determining a first area in the target point cloud data according to the second key point; determining a second area in the first point cloud data according to the third key point, includes: calculating the distance between each point in the target point cloud data except the second key point and the second key point; determining X first target points from the target point cloud data according to the distance between each point in the target point cloud data except the second key point and the second key point, wherein X is an integer greater than 1, and the distance between the first target point and the second key point is less than the distance between the non-first target point and the second key point; taking the area covered by the X first target points in the target point cloud data as the first area; calculating the distance between each point in the first point cloud data except the third key point and the third key point; determining Y second target points from the first point cloud data according to the distance between each point in the first point cloud data except the third key point and the third key point, wherein Y is an integer greater than 1, and the distance between the second target point and the third key point is less than the distance between the non-second target point and the third key point; taking the area covered by the Y second target points in the first point cloud data as the second area.

[0010] Optionally, in the point cloud registration method of the present application, according to the distance between each point except the second key point in the target point cloud data and the second key point, X first target points are determined from the target point cloud data; according to the distance between each point except the third key point in the first point cloud data and the third key point, Y second target points are determined from the first point cloud data, including: constructing a first point cloud tree based on the target point cloud data, wherein the first point cloud tree is a data structure in the form of a binary tree, which is used to represent the target point cloud data in the form of a binary tree; by calculating the first point cloud tree The method of calculating the Euclidean distance between the point in the second point cloud tree and the second key point, and using the nearest neighbor search algorithm to determine the X points in the target point cloud data that are closest to the second key point, so as to obtain X first target points; a second point cloud tree is constructed based on the first point cloud data, wherein the second point cloud tree is a data structure in the form of a binary tree, and is used to represent the first point cloud data in the form of a binary tree; by calculating the Euclidean distance between the point in the second point cloud tree and the third key point, and using the nearest neighbor search algorithm to determine the Y points in the first point cloud data that are closest to the third key point, so as to obtain Y second target points.

[0011] Optionally, in the point cloud registration method of the present application, the source point cloud data and the target point cloud data are registered based on the first area and the second area to obtain a registration result, including: replacing the target point cloud data with the first area; replacing the source point cloud data with the second area; performing a registration operation on the first area and the second area to obtain a registration result.

[0012] Optionally, in the point cloud registration method of the present application, a registration operation is performed on the first area and the second area to obtain a registration result, including: performing a rotation operation on the first area and / or the second area in at least one directional dimension, and generating a rotation matrix and a displacement, wherein the rotation matrix is ​​used to characterize the rotation angle of the first area and / or the second area in each directional dimension during the registration operation; the displacement amount characterizes the displacement information of the first area and / or the second area in each directional dimension during the registration operation; and the rotation matrix and the displacement are used as the registration result.

[0013] According to another aspect of the present application, a point cloud registration device is also provided, wherein the point cloud registration device includes: a first processing unit, used to take the point in the source point cloud data that is closest to the first point of interest as the first key point, and take the point in the target point cloud data that is closest to the second point of interest as the second key point, wherein the source point cloud data and the target point cloud data are two point cloud data that need to be registered, the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data; a second processing unit, used to update the coordinate information of the first key point according to the coordinate difference information between the first key point and the second key point, and use the updated first key point as the third key point; a third processing unit, used to register the source point cloud data and the target point cloud data according to the second key point and the third key point to obtain a registration result.

[0014] According to another aspect of the present application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program runs, the device where the computer-readable storage medium is located executes the above-mentioned point cloud registration method.

[0015] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned point cloud registration method.

[0016] In the present application, firstly, the point in the source point cloud data that is closest to the first point of interest is taken as the first key point, and the point in the target point cloud data that is closest to the second point of interest is taken as the second key point, wherein the source point cloud data and the target point cloud data are two point cloud data that need to be registered, the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data. Then, according to the coordinate difference information between the first key point and the second key point, the coordinate information of the first key point is updated, and the updated first key point is taken as the third key point. Finally, the source point cloud data and the target point cloud data are registered according to the second key point and the third key point to obtain the registration result.

[0017] From the above content, it can be seen that according to the technical solution of the present application, users can be allowed to select points of interest on the source point cloud data and the target point cloud data respectively. For example, a first point of interest is selected as a registration reference point for the source point cloud data; and a second point of interest is selected as a registration reference point for the target point cloud data. The selection of bidirectional interest points can make the registration process more flexible and accurate, thereby solving the technical problem of poor registration flexibility caused by only selecting interest points on the target image for registration in the prior art.

[0018] In addition, since the selected interest points may not correspond to real points in the point cloud data, the present application also uses the point in the source point cloud data that is closest to the first interest point as the first key point, and uses the point in the target point cloud data that is closest to the second interest point as the second key point, and subsequently completes the alignment of the source point cloud data and the target point cloud data mainly based on the first key point and the second key point, thereby improving the accuracy of the alignment and avoiding the technical problem of large alignment errors caused by the selected interest points not being real points in the point cloud data.

[0019] In addition, the present application also updates the coordinate information of the first key point based on the coordinate difference information between the first key point and the second key point, and uses the updated first key point as the third key point. Finally, the source point cloud data and the target point cloud data are aligned according to the second key point and the third key point to obtain the alignment result. This process is equivalent to achieving preliminary rigid alignment based on the first key point and the second key point, providing a more accurate initial alignment. Finally, the source point cloud data and the target point cloud data are aligned based on the aligned third key point and the second key point, which can improve the overall alignment accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 is a flow chart of an optional point cloud registration method according to an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of an optional point cloud processing system according to an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of an optional point cloud registration device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] It should also be noted that the information and data collected by this application are authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation portals for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0027] According to an embodiment of the present application, an embodiment of a point cloud registration method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] It should be noted that a point cloud processing system can be used as the executor of the point cloud registration method of the embodiment of the present application. It can be understood that the point cloud registration method provided in the embodiment of the present application can also be executed by other systems or devices, and the embodiment of the present application does not make any specific limitations on this.

[0029] Figure 1 is a flow chart of an optional point cloud registration method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0030] In step S101, a point in the source point cloud data that is closest to the first point of interest is taken as a first key point, and a point in the target point cloud data that is closest to the second point of interest is taken as a second key point.

[0031] In step S101, source point cloud data and target point cloud data are two point cloud data that need to be registered, the first interest point is a registration reference point set for the source point cloud data, and the second interest point is a registration reference point set for the target point cloud data.

[0032] Alternatively, 3D point cloud data is a commonly used data representation in computer vision, 3D modeling, scanning and measurement technology, which consists of a large number of points distributed in 3D space. Each point contains three coordinate values ​​(usually X, Y, Z), which describe the specific position of the point in space.

[0033] Optionally, the source point cloud data and the target point cloud data in the present application are two point cloud data that need to be registered, wherein both the source point cloud data and the target point cloud data are point cloud data in three-dimensional form.

[0034] Optionally, in the embodiment of the present application, the source point cloud data may be recorded as ; The target point cloud data is recorded as .

[0035] Optionally, for the source point cloud data , a point of interest can be determined randomly or in a specified manner, namely the first point of interest .

[0036] Optionally, for the target point cloud data , a point of interest can be determined randomly or in a specified manner, namely the second point of interest .

[0037] It should be noted that traditional point cloud registration methods usually only support the selection of interest points on the target point cloud data, and cannot select points on the source point cloud data (floating image). This limitation makes the registration process lack flexibility and accuracy, especially in application scenarios that require fine alignment.

[0038] However, this application adopts a two-way point of interest selection method, allowing users to select points of interest on the source point cloud data and the target point cloud data respectively, thereby making the alignment process more flexible and accurate, meeting the needs of different application scenarios.

[0039] In addition, it should be noted that since the points of interest selected by the user may not be actual points in the point cloud data, in this case, the registration accuracy of the point cloud data cannot be effectively guaranteed. In order to avoid this problem, this application adopts a method of determining the key points mapped by the points of interest in the point cloud data, as follows:

[0040] First, in the source point cloud data Identify the first point of interest Euclidean distance to the nearest point , point The key point mapped as the first interest point in the source point cloud data is first , as shown in formula (1):

[0041]

[0042] Secondly, in the target point cloud data Identify and second point of interest Euclidean distance to the nearest point , point The second key point mapped as the second interest point in the target point cloud data , as shown in formula (2):

[0043]

[0044] It should be noted that Euclidean distance is a common method for measuring the distance between two points in multidimensional space. Euclidean distance is based on the distance definition in the rectangular coordinate system. It determines the straight-line distance between two points by calculating the square root of the sum of the difference in the coordinate values ​​of the two points, reflecting the direct interval between the two points in space.

[0045] In addition to using the Euclidean distance, other distances (such as straight-line distance) can also be used to determine the key points corresponding to the interest points.

[0046] By determining the key points mapped to the points of interest in the point cloud data, the purpose of adaptively mapping the points of interest determined by the user to real points or the closest points in the three-dimensional point cloud data is achieved, and replacing the input points of interest as key points for post-processing of the point cloud data, thereby solving the problem of deviation in point cloud preprocessing caused by directly using the points of interest for point cloud registration. The point cloud data is processed based on the mapped key points, achieving the technical effect of providing registration accuracy.

[0047] Step S102: updating the coordinate information of the first key point according to the coordinate difference information between the first key point and the second key point, and using the updated first key point as the third key point.

[0048] Optionally, after determining the first key point and the second key point, the point cloud processing system may obtain coordinate difference information between the first key point and the second key point. ,in, It can also be used as the coordinate difference information between the source point cloud data and the target point cloud data.

[0049] in, The calculation formula can refer to the following formula (3):

[0050]

[0051] Optionally, based on the coordinate difference information between the first key point and the second key point The point cloud processing system can update the coordinate information of the first key point, wherein the purpose of the update is to achieve preliminary alignment of the source point cloud data and the target point cloud data, and to achieve preliminary alignment of the first key point and the second key point.

[0052] It should be noted that many traditional registration algorithms (such as the ICP algorithm) have high requirements for the initial alignment position. If the initial alignment is not accurate enough, the algorithm may fall into a local optimal solution, which ultimately affects the registration accuracy, increasing the complexity and risk of registration failure. However, in the embodiment of the present application, by selecting points of interest on the source point cloud data and the target point cloud data, respectively, and performing preliminary rigid registration based on the two key points corresponding to the two points of interest, the problem of high requirements for the initial alignment position in the traditional method is solved, providing a more accurate initial alignment and reducing the risk of registration failure.

[0053] Step S103, registering the source point cloud data and the target point cloud data according to the second key point and the third key point to obtain a registration result.

[0054] Optionally, the point cloud processing system may directly perform overall registration of the source point cloud data and the target point cloud data based on the second key point and the third key point, that is, register the overall source point cloud data and the overall target point cloud data.

[0055] Optionally, the point cloud processing system can also determine a partial area in the target point cloud data as a first area of ​​interest based on the second key point, and determine a partial area in the source point cloud data as a second area of ​​interest based on the third key point, and then align the first area of ​​interest and the second area of ​​interest, and use the alignment result as the alignment result of the source point cloud data and the target point cloud data. This method can avoid feature matching and optimization based on the entire point cloud, and avoid processing a large amount of redundant information, thereby improving the alignment speed and the point cloud computing speed, and achieving the technical effect of precise alignment only for the area of ​​interest.

[0056] Based on the contents of the above steps S101 to S103, it can be known that in the present application, the point in the source point cloud data that is closest to the first point of interest is first used as the first key point, and the point in the target point cloud data that is closest to the second point of interest is used as the second key point, wherein the source point cloud data and the target point cloud data are two point cloud data that need to be registered, the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data. Then, according to the coordinate difference information between the first key point and the second key point, the coordinate information of the first key point is updated, and the updated first key point is used as the third key point. Finally, the source point cloud data and the target point cloud data are registered according to the second key point and the third key point to obtain the registration result.

[0057] From the above content, it can be seen that according to the technical solution of the present application, users can be allowed to select points of interest on the source point cloud data and the target point cloud data respectively. For example, a first point of interest is selected as a registration reference point for the source point cloud data; and a second point of interest is selected as a registration reference point for the target point cloud data. The selection of bidirectional interest points can make the registration process more flexible and accurate, thereby solving the technical problem of poor registration flexibility caused by only selecting interest points on the target image for registration in the prior art.

[0058] In addition, since the selected interest points may not correspond to real points in the point cloud data, the present application also uses the point in the source point cloud data that is closest to the first interest point as the first key point, and uses the point in the target point cloud data that is closest to the second interest point as the second key point, and subsequently completes the alignment of the source point cloud data and the target point cloud data mainly based on the first key point and the second key point, thereby improving the accuracy of the alignment and avoiding the technical problem of large alignment errors caused by the selected interest points not being real points in the point cloud data.

[0059] In addition, the present application also updates the coordinate information of the first key point based on the coordinate difference information between the first key point and the second key point, and uses the updated first key point as the third key point. Finally, the source point cloud data and the target point cloud data are aligned according to the second key point and the third key point to obtain the alignment result. This process is equivalent to achieving preliminary rigid alignment based on the first key point and the second key point, providing a more accurate initial alignment. Finally, the source point cloud data and the target point cloud data are aligned based on the aligned third key point and the second key point, which can improve the overall alignment accuracy.

[0060] In an optional embodiment, the coordinate information of the first key point is updated according to the coordinate difference information between the first key point and the second key point, including: adjusting the source point cloud data to be aligned with the target point cloud data according to the coordinate difference information, and using the adjusted source point cloud data as the first point cloud data; after adjusting the source point cloud data to the first point cloud data, updating the coordinate information of the first key point according to the coordinate difference information.

[0061] Optionally, after obtaining the coordinate difference information between the first key point and the second key point Afterwards, the point cloud processing system can convert the source point cloud data into Initial alignment to target point cloud data , get the initially aligned source point cloud data , refer to the following formula (4):

[0062]

[0063] Finally, the source point cloud data With target point cloud data After the initial alignment, the first key point corresponding to the source point cloud data needs to be A preliminary alignment is also performed with the second key point to obtain the source point cloud data The corresponding third key point , as shown in formula (5):

[0064]

[0065] It should be noted that by selecting interest points on the source point cloud data and the target point cloud data respectively, and performing preliminary rigid registration based on the two key points corresponding to these two interest points, the problem of high requirements for the initial alignment position of the traditional method is solved, a more accurate initial alignment is provided, and the risk of registration failure is reduced.

[0066] In an optional embodiment, the source point cloud data and the target point cloud data are registered according to the second key point and the third key point to obtain a registration result, including: determining a first area in the target point cloud data according to the second key point, wherein the second key point is located in the first area; determining a second area in the first point cloud data according to the third key point, wherein the third key point is located in the second area; and registering the source point cloud data and the target point cloud data based on the first area and the second area to obtain a registration result, wherein the first area corresponds to the target point cloud data and the second area corresponds to the source point cloud data.

[0067] Optionally, after obtaining the second key point and the third key point, a partial area in the target point cloud data can be determined as the first region of interest (i.e., the first region) based on the second key point, and a partial area in the first point cloud data (i.e., the source point cloud data aligned with the target point cloud data) can be determined as the second region of interest (i.e., the second region) based on the third key point. Then, the first region of interest and the second region of interest are registered, and the registration result is used as the registration result of the source point cloud data and the target point cloud data. This method can avoid feature matching and optimization based on the entire point cloud, and avoid processing a large amount of redundant information, thereby improving the registration speed and the point cloud computing speed, and achieving the technical effect of accurate registration only for the region of interest.

[0068] In an optional embodiment, determining a first area in the target point cloud data according to the second key point; and determining a second area in the first point cloud data according to the third key point include the following steps:

[0069] Exemplarily, when determining the first area in the target point cloud data according to the second key point, it includes: calculating the distance between each point in the target point cloud data except the second key point and the second key point; determining X first target points from the target point cloud data according to the distance between each point in the target point cloud data except the second key point and the second key point, wherein X is an integer greater than 1, and the distance between the first target point and the second key point is less than the distance between the non-first target point and the second key point; and taking the area covered by the X first target points in the target point cloud data as the first area.

[0070] Optionally, the point cloud processing system may sequentially calculate the distance between each point in the target point cloud except the second key point and the second key point, for example, the Euclidean distance, and then select X points closest to the second key point as X first target points. Finally, the point cloud processing system may use the area covered by the target point cloud data of the X first target points as the first area, thereby determining the area of ​​highest user interest from the target point cloud data.

[0071] Exemplarily, when determining the second area in the first point cloud data (i.e., the source point cloud data aligned with the target point cloud data) based on the third key point, it includes: calculating the distance between each point in the first point cloud data except the third key point and the third key point; determining Y second target points from the first point cloud data based on the distance between each point in the first point cloud data except the third key point and the third key point, wherein Y is an integer greater than 1, and the distance between the second target point and the third key point is less than the distance between the non-second target point and the third key point; and taking the area covered by the Y second target points in the first point cloud data as the second area.

[0072] Optionally, the point cloud processing system can calculate the distance between each point in the first point cloud data except the third key point and the third key point in turn, for example, the Euclidean distance, and then select Y points closest to the third key point as Y second target points. Finally, the point cloud processing system can use the area covered by the first point cloud data with the Y second target points as the second area, thereby determining the area with the highest user interest in the first point cloud data, which is equivalent to determining the area with the highest user interest from the source point cloud data.

[0073] It should be noted that traditional point cloud registration methods usually rely on global feature matching and optimization algorithms. When processing large-scale point cloud data, the computational complexity is high and the efficiency is low, making it difficult to meet the needs of real-time applications. In addition, point cloud data often contains a large amount of noise and redundant information. Traditional methods have poor robustness when processing these data and are easily affected by noise, resulting in inaccurate registration results. Moreover, most existing technologies cannot meet users' needs for refined registration of specific areas and lack flexibility. Users cannot specify areas of interest for registration according to actual needs, which limits the scope of application and effect of the technology.

[0074] However, by adopting the technical solution of the present application, by selecting points of interest, determining the key points corresponding to the points of interest in the point cloud data, and generating regions of interest for registration based on the diffusion of key points, the amount of registration calculations is significantly reduced, the processing efficiency is improved, and it is suitable for large-scale point cloud data processing. In addition, by focusing on the region of interest, the present application reduces the interference of noise and redundant information, and enhances the robustness and accuracy of the registration results. Finally, the present application also allows users to select points of interest on the source point cloud data and the target point cloud data according to specific application requirements and perform registration based on this, thereby achieving a more refined and customized point cloud registration and meeting the user's needs for precise alignment of specific areas.

[0075] In an optional embodiment, X first target points are determined from the target point cloud data according to the distance between each point except the second key point in the target point cloud data and the second key point, including: constructing a first point cloud tree based on the target point cloud data, wherein the first point cloud tree is a data structure in the form of a binary tree, and is used to represent the target point cloud data in the form of a binary tree; by calculating the Euclidean distance between the points in the first point cloud tree and the second key point, a nearest neighbor search algorithm is used to determine the X points in the target point cloud data that are closest to the second key point, thereby obtaining the X first target points.

[0076] In an optional embodiment, Y second target points are determined from the first point cloud data based on the distance between each point in the first point cloud data except the third key point and the third key point, including: constructing a second point cloud tree based on the first point cloud data, wherein the second point cloud tree is a data structure in the form of a binary tree, used to represent the first point cloud data in the form of a binary tree; by calculating the Euclidean distance between the points in the second point cloud tree and the third key point, a nearest neighbor search algorithm is used to determine the Y points in the first point cloud data that are closest to the third key point, thereby obtaining the Y second target points.

[0077] Optionally, in order to reduce noise and redundant information in point cloud data and improve computing efficiency, the present application performs an efficient nearest neighbor search based on key points in combination with a KD (K-dimensional tree) tree to obtain the region of interest corresponding to the point cloud data, and the size of the region of interest can be set by the user, which is more flexible.

[0078] First, through the second key point mentioned above And the third key point , respectively calculate its And the first point cloud data The number of points that make up the region of interest and , as shown in formula (6).

[0079]

[0080] in, and It can be determined by the user according to needs.

[0081] Secondly, construct the target point cloud data respectively And the first point cloud data The KD tree is recorded as the first point cloud tree and the second point cloud tree .

[0082] Finally, the nearest neighbor search algorithm based on Euclidean distance searches for the key points in the two KD trees respectively. and The nearest Points and Points , respectively, to form specific regions of interest based on key points and , as shown in formula (7):

[0083]

[0084] in, and They respectively represent the target point cloud data and the initially aligned source point cloud data (first point cloud data). and Respectively represent the target point cloud data Constructed KD tree and first point cloud data Constructed KD tree. Indicates the key points found based on the Euclidean distance in the KD tree Recent A collection of points; Indicates the key points found based on the Euclidean distance in the KD tree Recent A collection of points. Indicates that Keypoint-based specific area of ​​interest. Indicates that Keypoint-based specific area of ​​interest.

[0085] In an optional embodiment, the source point cloud data and the target point cloud data are registered based on the first area and the second area to obtain a registration result, including: replacing the target point cloud data with the first area; replacing the source point cloud data with the second area; performing a registration operation on the first area and the second area to obtain a registration result.

[0086] Optionally, after obtaining the region of interest corresponding to each point cloud data, the point cloud processing system can perform accurate registration based on the region of interest, including: first, converting the target point cloud data Replace with the above based on the second key point The first area determined Secondly, the source point cloud data Replace with the above based on the third key point The second area determined Finally, the first area and the second region , input into any registration model, and the registration model completes the registration of the two regions.

[0087] It should be noted that by selecting points of interest and diffusing them to form regions of interest, the amount of computation for point cloud registration is significantly reduced, the processing speed is improved, and it is suitable for large-scale point cloud data processing. In addition, focusing on the region of interest for point cloud registration reduces the interference of noise and redundant information, and improves the robustness and accuracy of the registration results.

[0088] In an optional embodiment, a registration operation is performed on the first region and the second region to obtain a registration result, including: performing a rotation operation on the first region and / or the second region in at least one directional dimension, and generating a rotation matrix and a displacement, wherein the rotation matrix is ​​used to characterize the rotation angle of the first region and / or the second region in each directional dimension during the registration operation; the displacement amount characterizes the displacement information of the first region and / or the second region in each directional dimension during the registration operation; and the rotation matrix and the displacement amount are used as the registration result.

[0089] Optionally, when registering the first region and the second region, it is necessary to calculate and obtain the rotation matrix R and the displacement T. Finally, since the preliminary alignment step of the source point cloud data and the target point cloud data is involved, it is necessary to determine the final displacement , as shown in formula (8):

[0090]

[0091] From the above content, it can be seen that this application allows users to select points of interest on the source point cloud data and the target point cloud data respectively, and supports the preliminary alignment of the source point cloud data to the target point cloud data based on the points of interest, and then generates regions of interest based on the diffusion of the points of interest, and finally performs fine registration based on the regions of interest. This application significantly improves the flexibility and accuracy of registration, reduces the amount of calculation, enhances robustness, meets the customization needs of users, and has a wider application prospect.

[0092] In an optional embodiment, Figure 2 is a schematic diagram of an optional point cloud processing system according to an embodiment of the present application, such as Figure 2 As shown, the point cloud processing system may include: a data reading module, a point selection acquisition module, a preliminary alignment module, an area of ​​interest acquisition module, and a rigid registration module.

[0093] Optionally, the reading module is used to obtain source point cloud data and target point cloud data.

[0094] Optionally, the selection point acquisition module is used to determine a first point of interest for the source point cloud data, and determine a first key point corresponding to the first point of interest in the source point cloud data; determine a second point of interest for the target point cloud data, and determine a second key point corresponding to the second point of interest in the target point cloud data.

[0095] Optionally, the preliminary alignment module is used to update the coordinate information of the first key point according to the coordinate difference information between the first key point and the second key point, and use the updated first key point as the third key point, and realize the preliminary alignment of the source point cloud data and the target point cloud data.

[0096] Optionally, the area of ​​interest acquisition module is used to determine a first area in the target point cloud data based on a second key point, wherein the second key point is located in the first area; and determine a second area in the first point cloud data based on a third key point, wherein the third key point is located in the second area.

[0097] Optionally, a rigid registration module is used to register the source point cloud data and the target point cloud data according to the first area and the second area to obtain a registration result, wherein the first area corresponds to the target point cloud data and the second area corresponds to the source point cloud data.

[0098] In an optional embodiment, an application scenario of the point cloud registration method in the embodiment of the present application can be applied to an optical surface scanning system, wherein the optical surface scanning system is an important auxiliary tool in the field of radiotherapy, which uses optical technologies such as laser and structured light to accurately detect and monitor the patient's body surface information, and combines the registration algorithm to complete real-time monitoring of the patient's position, respiratory synchronization monitoring and non-invasive positioning, thereby achieving the purpose of improving the treatment effect and the patient's comfort. In the actual application process, the user needs to observe the patient's position under some tricky angles and tricky parts, but the traditional registration method cannot select points on the source point cloud data and the target point cloud data at the same time, so that the effective alignment of the source point cloud data and the target point cloud data cannot be achieved, resulting in a lack of flexibility and accuracy in the registration process. The present application allows users to select points of interest on the source point cloud data and the target point cloud data respectively, which can make the registration process more flexible and accurate, solve technical problems that cannot be solved by the prior art, and achieve technical effects that cannot be achieved by the prior art.

[0099] It should be noted that the application of the above-mentioned optical surface scanning system is only an example. Those skilled in the art can understand that, except for the field of radiotherapy technology, any other scenario that requires point cloud processing or point cloud registration is applicable to the technical solution of the embodiment of the present application.

[0100] According to an embodiment of the present application, an embodiment of a point cloud registration device is also provided. Figure 3is a schematic diagram of an optional point cloud registration device according to an embodiment of the present application, such as Figure 3 As shown, the point cloud registration device includes: a first processing unit 301, a second processing unit 302, and a third processing unit 303.

[0101] Optionally, the first processing unit 301 is used to take the point in the source point cloud data that is closest to the first point of interest as the first key point, and take the point in the target point cloud data that is closest to the second point of interest as the second key point, wherein the source point cloud data and the target point cloud data are two point cloud data that need to be registered, the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data; the second processing unit 302 is used to update the coordinate information of the first key point according to the coordinate difference information between the first key point and the second key point, and take the updated first key point as the third key point; the third processing unit 303 is used to align the source point cloud data and the target point cloud data according to the second key point and the third key point to obtain a registration result.

[0102] Optionally, the second processing unit 302 includes: a first processing subunit and a second processing subunit. The first processing subunit is used to adjust the source point cloud data to be aligned with the target point cloud data according to the coordinate difference information, and use the adjusted source point cloud data as the first point cloud data; the second processing subunit is used to update the first key coordinate information according to the coordinate difference information after adjusting the source point cloud data to the first point cloud data.

[0103] Optionally, the third processing unit 303 includes: a first determining subunit, a second determining subunit, and a registration subunit. The first determining subunit is used to determine a first area in the target point cloud data according to the second key point, wherein the second key point is located in the first area; the second determining subunit is used to determine a second area in the first point cloud data according to the third key point, wherein the third key point is located in the second area; the registration subunit is used to register the source point cloud data and the target point cloud data according to the first area and the second area to obtain a registration result, wherein the first area corresponds to the target point cloud data, and the second area corresponds to the source point cloud data.

[0104] Optionally, the third processing unit 303 includes: a first calculating subunit, a third determining subunit, a third processing subunit, a second calculating subunit, a fourth determining subunit, and a fourth processing subunit. The first calculation subunit is used to calculate the distance between each point in the target point cloud data except the second key point and the second key point; the third determination subunit is used to determine X first target points from the target point cloud data according to the distance between each point in the target point cloud data except the second key point and the second key point, wherein X is an integer greater than 1, and the distance between the first target point and the second key point is less than the distance between the non-first target point and the second key point; the third processing subunit is used to take the area covered by the X first target points in the target point cloud data as the first area; the second calculation subunit is used to calculate the distance between each point in the first point cloud data except the third key point and the third key point; the fourth determination subunit is used to determine Y second target points from the first point cloud data according to the distance between each point in the first point cloud data except the third key point and the third key point, wherein Y is an integer greater than 1, and the distance between the second target point and the third key point is less than the distance between the non-second target point and the third key point; the fourth processing subunit is used to take the area covered by the Y second target points in the first point cloud data as the second area.

[0105] Optionally, the third processing unit 303 includes: a fifth processing subunit, a sixth processing subunit, a seventh processing subunit and an eighth processing subunit. The fifth processing subunit is used to construct a first point cloud tree based on the target point cloud data, wherein the first point cloud tree is a data structure in the form of a binary tree, and is used to represent the target point cloud data in the form of a binary tree; the sixth processing subunit is used to determine the X points closest to the second key point in the target point cloud data by calculating the Euclidean distance between the points in the first point cloud tree and the second key point, and obtain X first target points by using a nearest neighbor search algorithm; the seventh processing subunit is used to construct a second point cloud tree based on the first point cloud data, wherein the second point cloud tree is a data structure in the form of a binary tree, and is used to represent the first point cloud data in the form of a binary tree; the eighth processing subunit is used to determine the Y points closest to the third key point in the first point cloud data by calculating the Euclidean distance between the points in the second point cloud tree and the third key point, and obtain Y second target points by using a nearest neighbor search algorithm.

[0106] Optionally, the registration subunit includes: a first processing module, a second processing module and a third processing module. The first processing module is used to replace the target point cloud data with the first area; the second processing module is used to replace the source point cloud data with the second area; the third processing module is used to perform a registration operation on the first area and the second area to obtain a registration result.

[0107] Optionally, the third processing module includes: a first processing submodule and a second processing submodule. The first processing submodule is used to perform a rotation operation on the first region and / or the second region in at least one directional dimension, and generate a rotation matrix and a displacement, wherein the rotation matrix is ​​used to represent the rotation angle of the first region and / or the second region in each directional dimension during the registration operation; the displacement represents the displacement information of the first region and / or the second region in each directional dimension during the registration operation; and the second processing submodule is used to use the rotation matrix and the displacement as the registration result.

[0108] According to another aspect of the present application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned point cloud registration method.

[0109] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned point cloud registration method.

[0110] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0111] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0114] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0116] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A point cloud registration method, characterized in that: include: The point in the source point cloud data that is closest to the first point of interest is used as the first key point, and the point in the target point cloud data that is closest to the second point of interest is used as the second key point, wherein the source point cloud data and the target point cloud data are two point cloud data that need to be registered, the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data; According to the coordinate difference information between the first key point and the second key point, updating the coordinate information of the first key point, and using the updated first key point as the third key point; Registering the source point cloud data and the target point cloud data according to the second key point and the third key point to obtain a registration result; Wherein, the source point cloud data and the target point cloud data are registered according to the second key point and the third key point to obtain a registration result, including: adjusting the source point cloud data to be aligned with the target point cloud data according to the coordinate difference information, and using the adjusted source point cloud data as the first point cloud data; determining a first area in the target point cloud data according to the second key point, wherein the second key point is located in the first area; determining a second area in the first point cloud data according to the third key point, wherein the third key point is located in the second area; registering the source point cloud data and the target point cloud data according to the first area and the second area to obtain a registration result, wherein the first area corresponds to the target point cloud data, and the second area corresponds to the source point cloud data; Wherein, determining the first area in the target point cloud data according to the second key point includes: calculating the distance between each point in the target point cloud data except the second key point and the second key point; determining X first target points from the target point cloud data according to the distance between each point in the target point cloud data except the second key point and the second key point, wherein X is an integer greater than 1, and the distance between the first target point and the second key point is less than the distance between the non-first target point and the second key point; and taking the area covered by the X first target points in the target point cloud data as the first area.

2. The point cloud registration method according to claim 1, characterized in that: Updating the coordinate information of the first key point according to the coordinate difference information between the first key point and the second key point includes: After the source point cloud data is adjusted to the first point cloud data, the first key coordinate information is updated according to the coordinate difference information.

3. The point cloud registration method according to claim 1, characterized in that: Determining a second area in the first point cloud data according to the third key point includes: Calculating the distance between each point in the first point cloud data except the third key point and the third key point; Determine Y second target points from the first point cloud data according to the distance between each point in the first point cloud data except the third key point and the third key point, wherein Y is an integer greater than 1, and the distance between the second target point and the third key point is less than the distance between a non-second target point and the third key point; The area covered by the Y second target points in the first point cloud data is taken as the second area.

4. The point cloud registration method according to claim 3, characterized in that: Determine X first target points from the target point cloud data according to the distance between each point in the target point cloud data except the second key point and the second key point; determine Y second target points from the first point cloud data according to the distance between each point in the first point cloud data except the third key point and the third key point, including: Constructing a first point cloud tree based on the target point cloud data, wherein the first point cloud tree is a data structure in the form of a binary tree, and is used to represent the target point cloud data in the form of the binary tree; By calculating the Euclidean distance between the points in the first point cloud tree and the second key point, a nearest neighbor search algorithm is used to determine the X points in the target point cloud data that are closest to the second key point, thereby obtaining the X first target points; Constructing a second point cloud tree based on the first point cloud data, wherein the second point cloud tree is a data structure in the form of a binary tree, and is used to represent the first point cloud data in the form of the binary tree; By calculating the Euclidean distance between the points in the second point cloud tree and the third key point, a nearest neighbor search algorithm is used to determine the Y points in the first point cloud data that are closest to the third key point, and Y second target points are obtained.

5. The point cloud registration method according to claim 1, characterized in that: Registering the source point cloud data and the target point cloud data according to the first area and the second area to obtain a registration result, including: Replacing the target point cloud data with the first area; replacing the source point cloud data with the second area; Performing a registration operation on the first region and the second region to obtain the registration result.

6. The point cloud registration method according to claim 5, characterized in that: Performing a registration operation on the first region and the second region to obtain the registration result includes: Performing a rotation operation on the first region and / or the second region in at least one directional dimension, and generating a rotation matrix and a displacement, wherein the rotation matrix is ​​used to characterize the rotation angle of the first region and / or the second region in each directional dimension during the registration operation; the displacement amount characterizes the displacement information of the first region and / or the second region in each directional dimension during the registration operation; The rotation matrix and the displacement are used as the registration result.

7. A point cloud registration device, characterized in that: include: A first processing unit is used to use a point in the source point cloud data that is closest to the first point of interest as a first key point, and use a point in the target point cloud data that is closest to the second point of interest as a second key point, wherein the source point cloud data and the target point cloud data are two point cloud data that need to be registered, the first point of interest is a registration reference point set for the source point cloud data, and the second point of interest is a registration reference point set for the target point cloud data; A second processing unit, configured to update the coordinate information of the first key point according to the coordinate difference information between the first key point and the second key point, and use the updated first key point as the third key point; A third processing unit, configured to register the source point cloud data and the target point cloud data according to the second key point and the third key point to obtain a registration result; Wherein, the third processing unit includes: a first determining subunit, used for adjusting the source point cloud data to be aligned with the target point cloud data according to the coordinate difference information, and using the adjusted source point cloud data as the first point cloud data; determining a first area in the target point cloud data according to the second key point, wherein the second key point is located in the first area; a second determining subunit, used for determining a second area in the first point cloud data according to the third key point, wherein the third key point is located in the second area; a registration subunit, used for registering the source point cloud data and the target point cloud data according to the first area and the second area to obtain a registration result, wherein the first area corresponds to the target point cloud data, and the second area corresponds to the source point cloud data; The third processing unit also includes: a first calculating subunit, used to calculate the distance between each point in the target point cloud data except the second key point and the second key point; a third determining subunit, used to determine X first target points from the target point cloud data according to the distance between each point in the target point cloud data except the second key point and the second key point, wherein X is an integer greater than 1, and the distance between the first target point and the second key point is less than the distance between the non-first target point and the second key point; the third processing subunit is used to take the area covered by the X first target points in the target point cloud data as the first area.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the point cloud registration method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the point cloud registration method described in any one of claims 1 to 6.

Images