Three-dimensional point cloud registration method and apparatus, computer device, and storage medium

Abstract

The present application relates to the technical field of three-dimensional point cloud registration, and discloses a three-dimensional point cloud registration method and apparatus, a computer device, and a storage medium. The method comprises: acquiring a first source point cloud and a target point cloud; obtaining sub-point clouds on the basis of the center point of a bounding box of the first source point cloud, and selecting a second source point cloud from among the sub-point clouds; determining a third source point cloud on the basis of the distances from vertexes of the second source point cloud to a fitting plane and the second source point cloud; and determining six degrees-of-freedom values on the basis of the third source point cloud and the target point cloud. The deviation values of the patient pose in six degrees of freedom can be calculated on the basis of the first source point cloud and the target point cloud. The automated registration process reduces manual operations by doctors, reduces the workload, reduces the risk of human errors, and improves the accuracy and consistency of positioning. By means of efficient registration and positioning adjustment, the patient positioning time is shortened, the uncomfortable experience caused by maintaining fixed postures for long times is reduced, and the comfort in treatment processes is improved.

Description

Three-dimensional point cloud registration method and device, computer equipment and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional point cloud registration, and particularly relates to a three-dimensional point cloud registration method, device, computer equipment and storage medium. BACKGROUND

[0002] Radiotherapy is an important means of treating tumors at present, and the goal is to concentrate high-precision radiation dose on tumor sites, maximize the elimination of tumor cells, and at the same time, protect the surrounding normal tissues as much as possible. In order to achieve this goal, the patient needs to be accurately positioned before receiving radiotherapy to ensure that each treatment can accurately aim at the tumor. This positioning process usually requires the doctor to manually adjust the patient according to the image data, which not only consumes time and effort, but also may bring adverse experiences to the patient, such as discomfort caused by long-time fixed posture, etc. Three-dimensional point cloud matching emerges as the times require, but the current three-dimensional point cloud matching is not accurate enough.

[0003] SUMMARY

[0004] Therefore, it is necessary to propose a three-dimensional point cloud registration method, device, computer equipment and storage medium in view of the technical problem that the existing three-dimensional point cloud matching is not accurate enough.

[0005] In a first aspect, a three-dimensional point cloud registration method is provided, and the method comprises:

[0006] obtaining a first source point cloud and a target point cloud;

[0007] dividing the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud;

[0008] determining a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0009] determining a 6-DOF value based on the third source point cloud and the target point cloud.

[0010] In a second aspect, a three-dimensional point cloud registration device is provided, and the device comprises:

[0011] an acquisition module configured to acquire a first source point cloud and a target point cloud;

[0012] a selection module configured to divide the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and select a second source point cloud from each sub-point cloud;

[0013] The first determining module is configured to determine a third source point cloud based on distances from vertices of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud.

[0014] The second determining module is configured to determine a 6-DOF value based on the third source point cloud and the target point cloud.

[0015] In a third aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the three-dimensional point cloud registration method when executing the computer program.

[0016] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the three-dimensional point cloud registration method when executed by a processor.

[0017] The three-dimensional point cloud registration method provided by the application can calculate the 6-dimensional deviation values of the patient pose based on the first source point cloud and the target point cloud. The automatic registration process reduces the manual operation of the doctor, reduces the workload and the risk of human error, and improves the accuracy and consistency of the positioning. Through efficient registration and positioning adjustment, the time for patient positioning is shortened, the discomfort experience caused by long-time fixed posture is reduced, and the comfort of the treatment process is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0019] In the formula, the first source point cloud is represented as P, the target point cloud is represented as T, the second source point cloud is represented as P', the third source point cloud is represented as P'', the fitting plane is represented as P'', the 6-DOF value is represented as T', and the 6-DOF value is represented as T''.

[0020] Fig. 1 is an application environment diagram of the three-dimensional point cloud registration method in an embodiment;

[0021] Fig. 2 is a flowchart of the three-dimensional point cloud registration method in an embodiment;

[0022] FIG. 3 is a structural block diagram of a three-dimensional point cloud registration device in one embodiment;

[0023] FIG. 4 is a structural block diagram of a computer device in one embodiment;

[0024] FIG. 5 is a structural block diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application; the description and claims herein and the above description of drawings herein utilize terms such as "including" and "having" and variations thereof that are intended to be broad and encompass the terms "consisting of" and "consisting essentially of." The terms "first," "second," and the like, as used herein do not necessarily have an ordinal meaning.

[0026] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all directed to the same embodiment, or to a single alternative embodiment.

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0028] The three-dimensional point cloud registration method provided by the embodiment of the application can be applied in the application environment shown in FIG. 1, wherein the client 110 communicates with the server 120 through a network. The server 120 can receive a first source point cloud and a target point cloud through the client 110, then divide the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub point cloud, select a second source point cloud from each sub point cloud, determine a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud, and finally determine a 6-degree-of-freedom value based on the third source point cloud and the target point cloud, so that the 6-dimensional deviation value of the patient pose can be calculated based on the first source point cloud and the target point cloud. The automatic registration process reduces the manual operation of the doctor, reduces the workload and the risk of human error, and improves the accuracy and consistency of the positioning. Through efficient registration and positioning adjustment, the time for patient positioning is shortened, the discomfort experience caused by long-time fixed posture is reduced, and the comfort of the treatment process is improved. The client 110 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers. The application will be described in detail through specific embodiments.

[0029] Referring to FIG. 2, FIG. 2 is a flowchart of a three-dimensional point cloud registration method provided by an embodiment of the application, which includes the following steps:

[0030] Step S101: Obtain a first source point cloud and a target point cloud.

[0031] The first source point cloud is generated based on a vertex sequence number of a region of interest, the vertex sequence number of the region of interest is obtained by a target user selecting a point cloud file, and the target point cloud is pre-collected. For example, the first source point cloud can be collected in real time on the chest of a patient, and the target point cloud can be collected in advance on the chest of the patient.

[0032] As an example, vtk is used to realize the three-dimensional rendering of the point cloud and the selection operation of the region of interest. The user can select a sketch mode and an observation mode through a key operation. The sketch mode can select a rectangular frame and a free-form tool, then the mouse is clicked and pushed to sketch, and finally the selection or deletion is confirmed through a key. The sketch region does not select or delete the vertex behind the current view angle, so that the sketch function of what you see is what you get is realized. The observation mode can perform various operations on the point cloud, including zoom-in and zoom-out operations, translation operations and rotation operations. The mesh surface can also be switched to be displayed. The tool input is a point cloud file, and the output is a vertex sequence number in the selected region of interest.

[0033] Step S102: dividing the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud;

[0034] In an embodiment, the first source point cloud is divided based on a spatial coordinate system constructed based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud; and a sub-point cloud with the largest number of vertices is selected as the second source point cloud from each sub-point cloud.

[0035] For example, an xyz coordinate system in which the center point of the bounding box of the first source point cloud P1 is located generates three planes, i.e., an XOY plane in which an x-axis and a y-axis are located, an XOZ plane in which the x-axis and a z-axis are located, and a YOZ plane in which the y-axis and the z-axis are located. The first source point cloud P1 is segmented using the three planes, and a sub-point cloud with the largest number of points is selected as the second source point cloud P2.

[0036] Step S103: determining a third source point cloud based on the distance from the vertices of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud.

[0037] For example, after the distance from the vertices of the second source point cloud to the fitting plane is determined, the vertices of the second source point cloud with a distance less than a preset value are selected as the third source point cloud.

[0038] Step S104: determining a 6-DOF value based on the third source point cloud and a target point cloud.

[0039] For example, a registration algorithm is used to calculate a rotation matrix and a translation vector of the third source point cloud to the target point cloud, and the 6-DOF value is calculated based on the rotation matrix and the translation vector of the third source point cloud to the target point cloud.

[0040] In an embodiment, the step of determining the 6-DOF value based on the third source point cloud and the target point cloud comprises:

[0041] Step S1041: calculating a first rotation matrix and a first translation vector based on the third source point cloud and the target point cloud.

[0042] Step S1042: performing a rigid change on the third source point cloud based on the first rotation matrix and the first translation vector to obtain a fourth source point cloud.

[0043] In an embodiment, the first rotation matrix R1 and the first translation vector T1 of the third source point cloud P3 relative to the target point cloud X are used to perform a rigid change on the third source point cloud P3 to obtain a fourth source point cloud P4, and the change formula is P4=R1P3+T1.

[0044] Step S1043: based on the fourth source point cloud and the target point cloud, a second rotation matrix and a second translation vector are calculated;

[0045] Step S1044: based on the second rotation matrix and the second translation vector, the fourth source point cloud is rigidly changed to obtain a fifth source point cloud;

[0046] In an embodiment, the second rotation matrix R2 and the second translation vector T2 of the fourth source point cloud P4 relative to the target point cloud X are used to rigidly change the fourth source point cloud P4 to obtain the fifth source point cloud P5, and the change formula is P5=R2P4+T2. The three-dimensional point cloud registration method provided by the application can be used to calculate the six-dimensional deviation values of the patient's pose.

[0047] Step S1045: based on the third source point cloud and the fifth source point cloud, a third rotation matrix and a third translation vector are determined, and based on the third rotation matrix and the third translation vector, a six-degree-of-freedom value is determined.

[0048] In an embodiment, the transformation from the third source point cloud P3 to the fifth source point cloud P5 is calculated as the transformation from the third source point cloud P3 to the target point cloud X. The formula is: X=RP3+T, wherein the third rotation matrix R=R2R1, the third translation vector T=R2T1+T2, R2 is the second rotation matrix, R1 is the first rotation matrix, T1 is the first translation vector, and T2 is the second translation vector.

[0049] It should be noted that the registration algorithm uses the ICP point-to-plane algorithm, and the main steps of the algorithm are nearest point search and solving linear equations. GPU acceleration through cuda programming can reduce the time of a large number of repetitive calculations, thereby improving time efficiency. The nearest point search finds the nearest point qi in the target point cloud for each point pi in the source point cloud by constructing a KDTree of the target point cloud. For each pair of matched points (pi, qi), the normal vector ni of the target point cloud is used to construct a linear equation system. The rotation matrix R and the translation vector T are obtained by solving the linear equation.

[0050] Referring to FIG. 3, in an embodiment, a three-dimensional point cloud registration device is provided, which comprises: an acquisition module 10 configured to acquire a first source point cloud and a target point cloud;

[0051] A selection module 20 is configured to divide the first source point cloud based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud, and select a second source point cloud from each sub-point cloud;

[0052] A first determination module 30 is configured to determine a third source point cloud based on the distance from the vertex of the second source point cloud to the fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0053] The second determining module 40 is configured to determine the 6-DOF value based on the third source point cloud and the target point cloud.

[0054] The selecting module 20 is further configured to divide the first source point cloud based on a space coordinate system constructed based on the center point of the bounding box of the first source point cloud to obtain each sub-point cloud.

[0055] The sub-point cloud with the largest number of vertices is selected as the second source point cloud.

[0056] The second determining module 40 is further configured to calculate a first rotation matrix and a first translation vector based on the third source point cloud and the target point cloud.

[0057] The third source point cloud is rigidly changed based on the first rotation matrix and the first translation vector to obtain a fourth source point cloud.

[0058] A second rotation matrix and a second translation vector are calculated based on the fourth source point cloud and the target point cloud.

[0059] The fourth source point cloud is rigidly changed based on the second rotation matrix and the second translation vector to obtain a fifth source point cloud.

[0060] A third rotation matrix and a third translation vector are determined based on the third source point cloud and the fifth source point cloud, and the 6-DOF value is determined based on the third rotation matrix and the third translation vector.

[0061] In an embodiment, a computer device can be provided, which can be a server. An internal structure diagram of the computer device can be as shown in FIG. 4. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external client through a network connection. The computer program is executed by the processor to implement functions or steps of a three-dimensional point cloud registration method server side.

[0062] In an embodiment, a computer device is provided, which can be a client, and an internal structure diagram thereof can be as shown in FIG. 5. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with an external server through a network connection. The computer program is executed by the processor to implement the functions or steps of a three-dimensional point cloud registration method on the client side.

[0063] In an embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0064] obtaining a first source point cloud and a target point cloud;

[0065] dividing the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud;

[0066] determining a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0067] determining a 6-DOF value based on the third source point cloud and the target point cloud.

[0068] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0069] obtaining a first source point cloud and a target point cloud;

[0070] dividing the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud;

[0071] determining a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud;

[0072] determining a 6-DOF value based on the third source point cloud and the target point cloud.

[0073] It should be noted that the functions or steps described above with respect to the computer readable storage medium or the computer device can correspond to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0074] A person of ordinary skill in the art can understand that all or part of the processes in the foregoing method embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a nonvolatile computer readable storage medium. When the computer program is executed, the processes of the foregoing embodiments of the method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided in the present application can include nonvolatile and / or volatile memory. The nonvolatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified. In actual applications, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0076] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of registering three-dimensional point clouds, characterized by, The three-dimensional point cloud registration method comprises: obtaining a first source point cloud and a target point cloud; dividing the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud; determining a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud; determining a 6-DOF value based on the third source point cloud and the target point cloud.

2. The method of claim 1, wherein, The step of dividing the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and selecting a second source point cloud from each sub-point cloud, comprises: dividing the first source point cloud based on a space coordinate system constructed by the center point of the bounding box of the first source point cloud to obtain each sub-point cloud; selecting a sub-point cloud with the most vertices as the second source point cloud from each sub-point cloud.

3. The method of claim 1, wherein, The step of determining a 6-DOF value based on the third source point cloud and the target point cloud, comprises: calculating a first rotation matrix and a first translation vector based on the third source point cloud and the target point cloud; performing a rigid change on the third source point cloud based on the first rotation matrix and the first translation vector to obtain a fourth source point cloud; calculating a second rotation matrix and a second translation vector based on the fourth source point cloud and the target point cloud; performing a rigid change on the fourth source point cloud based on the second rotation matrix and the second translation vector to obtain a fifth source point cloud; determining a third rotation matrix and a third translation vector based on the third source point cloud and the fifth source point cloud, and determining the 6-DOF value based on the third rotation matrix and the third translation vector.

4. A three-dimensional point cloud registration apparatus, characterized by, The three-dimensional point cloud registration device comprises: an acquisition module configured to obtain a first source point cloud and a target point cloud; a selection module configured to divide the first source point cloud based on a center point of a bounding box of the first source point cloud to obtain each sub-point cloud, and select a second source point cloud from each sub-point cloud; a first determination module configured to determine a third source point cloud based on a distance from a vertex of the second source point cloud to a fitting plane and the second source point cloud, wherein the fitting plane is generated based on the second source point cloud; a second determination module configured to determine a 6-DOF value based on the third source point cloud and the target point cloud.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the three-dimensional point cloud registration method according to any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 5. The computer program is executed by the processor to implement the steps of the three-dimensional point cloud registration method according to any one of claims 1 to 3.

Images