Three-dimensional point cloud registration method and device, computer device and storage medium

Abstract

The application relates to the field of three-dimensional point cloud registration, and discloses a three-dimensional point cloud registration method, a three-dimensional point cloud registration device, computer equipment and a storage medium, the method comprising the following steps: acquiring a first source point cloud and a target point cloud; obtaining each sub-point cloud based on a center point of a boundary box of the first source point cloud, and selecting a second source point cloud from the sub-point clouds; determining a third source point cloud based on the distance from the vertex of the second source point cloud to a fitting plane and the second source point cloud; and determining a six-degree-of-freedom value based on the third source point cloud and the target point cloud, so that the six-dimensional deviation values of the patient pose can be calculated based on the first source point cloud and the target point cloud. The automatic registration process reduces the manual operation of doctors, reduces the workload and the risk of human error, and improves the accuracy and consistency of the positioning. Through efficient registration and positioning adjustment, the time for patient positioning is shortened, the uncomfortable experience caused by long-time fixed posture is reduced, and the comfort of the treatment process is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional point cloud registration, and particularly to a three-dimensional point cloud registration method and device, computer equipment and a storage medium. BACKGROUND

[0002] Radiotherapy is an important means of treating tumors at present, and its goal is to concentrate high-precision radiation dose on tumor sites, maximize the elimination of tumor cells, and at the same time, protect the surrounding normal tissues as much as possible. In order to achieve this goal, the patient needs to be accurately positioned before receiving radiotherapy to ensure that each treatment can accurately aim at the tumor. This positioning process usually requires the doctor to manually adjust the patient according to the image data, which not only consumes time and effort, but also may bring adverse experiences to the patient, such as discomfort caused by long-time fixed posture, etc. Three-dimensional point cloud matching emerges as the times require, but the current three-dimensional point cloud matching is not accurate enough. SUMMARY

[0003] Therefore, it is necessary to propose a three-dimensional point cloud registration method, device, computer equipment and storage medium in view of the technical problem that the existing three-dimensional point cloud matching is not accurate enough.

[0004] In a first aspect, a three-dimensional point cloud registration method is provided, and the method comprises:

[0005] obtaining a first source point cloud and a target point cloud;

[0006] dividing the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud;

[0007] determining a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0008] determining a 6-degree-of-freedom value based on the third source point cloud and the target point cloud.

[0009] In a second aspect, a three-dimensional point cloud registration device is provided, and the device comprises:

[0010] an acquisition module configured to obtain a first source point cloud and a target point cloud;

[0011] a selection module configured to divide the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and select a second source point cloud from each sub-point cloud;

[0012] The first determining module is configured to determine a third source point cloud based on distances from vertices of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud.

[0013] The second determining module is configured to determine a 6-DOF value based on the third source point cloud and the target point cloud.

[0014] In a third aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the three-dimensional point cloud registration method when running the computer program.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the three-dimensional point cloud registration method when executed by a processor.

[0016] The three-dimensional point cloud registration method provided by the application can calculate the 6-dimensional deviation value of the patient pose based on the first source point cloud and the target point cloud. The automatic registration process reduces the manual operation of the doctor, reduces the workload and the risk of human error, improves the accuracy and consistency of the positioning, shortens the patient positioning time through efficient registration and positioning adjustment, reduces the discomfort experience caused by long-time fixed posture, and improves the comfort of the treatment process. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] In the drawings:

[0019] Figure 1 It is an application environment diagram of the three-dimensional point cloud registration method in an embodiment;

[0020] Figure 2 It is a flowchart of the three-dimensional point cloud registration method in an embodiment;

[0021] Figure 3 a structural block diagram of a three-dimensional point cloud registration device in one embodiment;

[0022] Figure 4 a structural block diagram of a computer device in one embodiment;

[0023] Figure 5 a structural block diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application; the description and claims of this application as well as the above abstract are intended to cover all alternatives, modifications, equivalents and equivalents thereof falling within the scope of the application. The terms "comprising", "having", "including", and "containing" used herein are meant to be interpreted in a non-limiting manner.

[0025] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all directed to the same embodiment, or to a single alternative implementation.

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort fall within the scope of the present application.

[0027] The three-dimensional point cloud registration method provided by the embodiments of the present application can be applied to, for example, Figure 1In an application environment, the client 110 communicates with the server 120 through a network. The server 120 can receive the first source point cloud and the target point cloud through the client 110, then divide the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and select the second source point cloud in each sub-point cloud, and then determine the third source point cloud based on the distance from the vertex of the second source point cloud to the fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud, and finally determine the 6-DOF value based on the third source point cloud and the target point cloud, which can calculate the 6-dimensional deviation value of the patient's pose based on the first source point cloud and the target point cloud. The automatic registration process reduces the doctor's manual operation, reduces the workload and the risk of human error, and improves the accuracy and consistency of the positioning. Through efficient registration and positioning adjustment, the patient's positioning time is shortened, the discomfort experience caused by long-time fixed posture is reduced, and the comfort of the treatment process is improved. The client 110 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers. The application will be described in detail through specific embodiments.

[0028] Please refer to Figure 2 , as shown in Figure 2 A flowchart of a three-dimensional point cloud registration method provided by an embodiment of the application, comprising the following steps:

[0029] Step S101: obtaining a first source point cloud and a target point cloud;

[0030] The first source point cloud is generated based on the vertex serial number of the region of interest, and the vertex serial number of the region of interest is obtained by the target user selecting the point cloud file. The target point cloud is pre-acquired. For example, the first source point cloud can be real-time acquired from the patient's chest, and the target point cloud can be pre-acquired from the patient's chest.

[0031] As an example, vtk is used to realize the three-dimensional rendering of the point cloud and the selection operation of the region of interest. The user can select the sketch mode and the observation mode through the key operation. The sketch mode can select a rectangular box and a free figure tool, and then use the mouse to click and push to sketch, and finally confirm the selection or deletion through the key. The sketch area will not select or delete the vertex behind the current view angle, realizing the sketch function of what you see is what you get. The observation mode can perform various operations on the point cloud, including zoom in and out, translation, rotation. It can also be switched to mesh surface display. The tool input is the point cloud file, and the output is the vertex serial number in the selected region of interest.

[0032] Step S102: dividing the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud;

[0033] In an embodiment, the first source point cloud is divided based on a spatial coordinate system constructed based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud; and a sub-point cloud with the largest number of vertices is selected as the second source point cloud from each sub-point cloud.

[0034] For example, the xyz coordinate system in which the center point of the bounding box of the first source point cloud P1 is located generates three planes, i.e., an XOY plane in which an x-axis and a y-axis are located, an XOZ plane in which the x-axis and a z-axis are located, and a YOZ plane in which the y-axis and the z-axis are located. The first source point cloud P1 is segmented using the three planes, and a sub-point cloud with the largest number of points is selected as the second source point cloud P2.

[0035] Step S103: determining a third source point cloud based on the distance from the vertices of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud.

[0036] For example, after determining the distance from the vertices of the second source point cloud to the fitting plane, the vertices of the second source point cloud with a distance less than a preset value are selected as the third source point cloud.

[0037] Step S104: determining a 6-DOF value based on the third source point cloud and a target point cloud.

[0038] For example, a registration algorithm is used to calculate a rotation matrix and a translation vector of the third source point cloud to the target point cloud, and the 6-DOF value is calculated based on the rotation matrix and the translation vector of the third source point cloud to the target point cloud.

[0039] In an embodiment, the step of determining the 6-DOF value based on the third source point cloud and the target point cloud comprises:

[0040] Step S1041: calculating a first rotation matrix and a first translation vector based on the third source point cloud and the target point cloud.

[0041] Step S1042: performing a rigid change on the third source point cloud based on the first rotation matrix and the first translation vector to obtain a fourth source point cloud.

[0042] In an embodiment, the first rotation matrix R1 and the first translation vector T1 of the third source point cloud P3 relative to the target point cloud X are used to perform a rigid change on the third source point cloud P3 to obtain a fourth source point cloud P4, and the change formula is P4=R1 P3+T1.

[0043] Step S1043: based on the fourth source point cloud and the target point cloud, a second rotation matrix and a second translation vector are calculated;

[0044] Step S1044: based on the second rotation matrix and the second translation vector, the fourth source point cloud is rigidly changed to obtain a fifth source point cloud;

[0045] In an embodiment, the second rotation matrix R2 and the second translation vector T2 of the fourth source point cloud P4 relative to the target point cloud X are used to rigidly change the fourth source point cloud P4 to obtain the fifth source point cloud P5, and the change formula is P5 = R2P4 + T2. The three-dimensional point cloud registration method provided by the application can be used to calculate the 6-dimensional deviation value of the patient's pose.

[0046] Step S1045: based on the third source point cloud and the fifth source point cloud, a third rotation matrix and a third translation vector are determined, and based on the third rotation matrix and the third translation vector, a 6-degree-of-freedom value is determined.

[0047] In an embodiment, the transformation from the third source point cloud P3 to the fifth source point cloud P5 is calculated as the transformation from the third source point cloud P3 to the target point cloud X. The formula is: X = RP3 + T, where the third rotation matrix R = R2R1, the third translation vector T = R2T1 + T2, R2 is the second rotation matrix, R1 is the first rotation matrix, T1 is the first translation vector, and T2 is the second translation vector.

[0048] It should be noted that the registration algorithm uses the ICP point-to-plane algorithm, and the main steps of the algorithm are nearest point search and solving linear equations. GPU acceleration through cuda programming can reduce the time of a large number of repetitive calculations, thereby improving time efficiency. The nearest point search finds the nearest point q i in the target point cloud for each point p i in the source point cloud by constructing a KDTree of the target point cloud. For each pair of matched points (p i, q i), the normal vector n i of the target point cloud is used to construct a linear equation system. The rotation matrix R and the translation vector T are obtained by solving the linear equation.

[0049] Referring to Figure 3 In an embodiment, a three-dimensional point cloud registration device is provided, and the device comprises: an acquisition module 10 configured to acquire a first source point cloud and a target point cloud;

[0050] A selection module 20 is configured to divide the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and select a second source point cloud from each sub-point cloud;

[0051] The first determining module 30 is configured to determine a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0052] The second determining module 40 is configured to determine the 6-DOF value based on the third source point cloud and the target point cloud.

[0053] The selecting module 20 is further configured to divide the first source point cloud based on a space coordinate system constructed based on a center point of the bounding box of the first source point cloud to obtain each sub-point cloud.

[0054] The selecting module 20 is further configured to select a sub-point cloud with the largest number of vertices from each of the sub-point clouds as the second source point cloud.

[0055] The second determining module 40 is further configured to calculate a first rotation matrix and a first translation vector based on the third source point cloud and the target point cloud.

[0056] The third source point cloud is rigidly changed based on the first rotation matrix and the first translation vector to obtain a fourth source point cloud.

[0057] A second rotation matrix and a second translation vector are calculated based on the fourth source point cloud and the target point cloud.

[0058] The fourth source point cloud is rigidly changed based on the second rotation matrix and the second translation vector to obtain a fifth source point cloud.

[0059] A third rotation matrix and a third translation vector are determined based on the third source point cloud and the fifth source point cloud, and the 6-DOF value is determined based on the third rotation matrix and the third translation vector.

[0060] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is configured to communicate with an external client through a network connection. The computer program is executed by the processor to implement the functions or steps of a three-dimensional point cloud registration method server side.

[0061] In one embodiment, a computer device is provided, which can be a client, and an internal structure diagram of the computer device can be as shown in Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of a three-dimensional point cloud registration method client side.

[0062] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0063] Obtain a first source point cloud and a target point cloud;

[0064] Divide the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and select a second source point cloud from each sub-point cloud;

[0065] Determine a third source point cloud based on the distance from the vertex of the second source point cloud to the fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0066] Determine a 6-degree-of-freedom value based on the third source point cloud and the target point cloud.

[0067] In one embodiment, a computer readable storage medium is provided, which stores a computer program, wherein the computer program is executed by a processor to implement the following steps:

[0068] Obtain a first source point cloud and a target point cloud;

[0069] Divide the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and select a second source point cloud from each sub-point cloud;

[0070] Determine a third source point cloud based on the distance from the vertex of the second source point cloud to the fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0071] Determine a 6-degree-of-freedom value based on the third source point cloud and the target point cloud.

[0072] It should be noted that the functions or steps described above with respect to the computer readable storage medium or the computer device can correspond to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0073] A person of ordinary skill in the art can understand that all or part of the processes in the foregoing method embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a nonvolatile computer readable storage medium. When the computer program is executed, the processes of the foregoing embodiments of the method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided in the present application can include nonvolatile and / or volatile memory. The nonvolatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synch link) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified. In actual applications, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0075] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features. Such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of registering three-dimensional point clouds, characterized by, The three-dimensional point cloud registration method comprises: obtaining a first source point cloud and a target point cloud; based on the center point of the bounding box of the first source point cloud, the first source point cloud is divided to obtain each sub-point cloud, and a second source point cloud is selected from each sub-point cloud, wherein the center point of the bounding box of the first source point cloud P1 generates three planes in the xyz coordinate system, which are XOY plane, XOZ plane and YOZ plane, respectively, the first source point cloud P1 is segmented using the three planes, and the sub-point cloud with the largest number of points is selected as the second source point cloud P2; based on the distance from the vertex of the second source point cloud to the fitting plane and the second source point cloud, a third source point cloud is determined, wherein the fitting plane is generated based on the second source point cloud; based on the third source point cloud and the target point cloud, a 6-DOF value is determined; the step of determining the 6-DOF value based on the third source point cloud and the target point cloud comprises: based on the third source point cloud and the target point cloud, a first rotation matrix and a first translation vector are calculated; based on the first rotation matrix and the first translation vector, the third source point cloud is rigidly changed to obtain a fourth source point cloud; based on the fourth source point cloud and the target point cloud, a second rotation matrix and a second translation vector are calculated; based on the second rotation matrix and the second translation vector, the fourth source point cloud is rigidly changed to obtain a fifth source point cloud; based on the third source point cloud and the fifth source point cloud, a third rotation matrix and a third translation vector are determined, and based on the third rotation matrix and the third translation vector, the 6-DOF value is determined, wherein P4=R1P3+T1, P5=R2P4+T2, R=R2R1, T=R2T1+T2, P5 is the fifth source point cloud, P4 is the fourth source point cloud, P3 is the third source point cloud, R1 is the first rotation matrix, T1 is the first translation vector, T2 is the second translation vector, R2 is the second rotation matrix, R is the third rotation matrix, and T is the third translation vector.

2. A three-dimensional point cloud registration apparatus, characterized by, The three-dimensional point cloud registration device for performing the three-dimensional point cloud registration method of claim 1 comprises: an acquisition module for acquiring a first source point cloud and a target point cloud; a selection module for dividing the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud; a first determination module for determining a third source point cloud based on the distance from the vertex of the second source point cloud to the fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud; a second determination module for determining a 6-DOF value based on the third source point cloud and the target point cloud.

3. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the three-dimensional point cloud registration method of claim 1.

4. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 3. The computer program is executed by the processor to implement the steps of the three-dimensional point cloud registration method of claim 1.

Images