Point cloud registration hardware acceleration method and system for orthopedic surgery navigation

Abstract

The invention discloses a point cloud registration hardware acceleration method and system for orthopedic surgery navigation, relates to the technical field of registration space registration in a medical surgery, and solves the problems of high point cloud registration calculation cost, long consumed time and low registration precision of a space registration algorithm in an existing orthopedic surgery navigation surgery. According to the scheme, the method comprises the following steps: acquiring actually registered bone point cloud data in an electromagnetic positioning system; importing the bone point cloud data into an FPGA (Field Programmable Gate Array); performing point cloud coarse registration on the bone point cloud data based on a DSP module to obtain an initial transformation matrix; performing point cloud precise registration based on the initial transformation matrix to obtain an optimal transformation matrix; wherein K-D Tree point cloud data topological structure construction and nearest neighbor point search are executed by a tree topological structure data module and a nearest neighbor point search module in the FPGA respectively, and other steps are executed by an embedded CPU in the FPGA; coarse registration and fine registration are combined with FPGA acceleration, so that the cost is reduced, the point registration speed is increased, and the registration precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of registration space alignment in medical surgery, and more specifically, to a point cloud registration hardware acceleration method and system for orthopedic surgery navigation. Background Art

[0002] Orthopedic surgery navigation systems can provide functions such as surgical planning and positioning measurement for doctors with the treatment concepts and application technologies of intelligence, precision, and minimally invasive. They show great application value and broad application prospects in orthopedic clinical surgeries. However, there are deficiencies in the space registration technology of domestic surgery navigation systems, and currently, the mainstream domestic navigation systems are all monopolized by foreign countries. The mainstream domestic navigation systems are optical positioning systems, which have relatively high accuracy and good synchronization stability. However, optical surgery navigation has problems such as high price cost and complex surgical operations due to light occlusion. Since the optical surgery navigation system needs to ensure no occlusion between the infrared camera and the tracking target during the operation, it brings inconvenience to the doctor's operation. Moreover, once the reference target points fixed on the patient's bony structure move during the operation, the navigation accuracy will deviate, thus affecting the intraoperative registration time and accuracy efficiency.

[0003] Space registration is one of the core technologies of orthopedic robot systems, and its goal is to obtain the rigid transformation relationship between the patient's physical space and the three-dimensional virtual model image space. The existing intraoperative registration space alignment technologies are divided into paired marker registration methods based on medical markers and unpaired marker registration methods based on anatomical feature structural features. The former implants medical markers (such as medical titanium and steel screws) into the patient's bones so that these markers can be clearly displayed in the image space and are easy to expose and collect in the patient space, thus ensuring a relatively high point cloud sample collection accuracy. However, the implantation of markers will additionally increase the patient's pain and psychological burden, and it is very likely that the markers will move slightly during the operation, and even in some cases, they cannot be implanted smoothly. The latter collects the point cloud data of the anatomical structure on the surface of the patient's bone, and then registers these two groups of point clouds through a space registration algorithm. In actual surgeries, the exposed range of the affected bone part is relatively limited, the field of view is small, and the local structure of the bone tissue is relatively complex. The point cloud samples collected in the patient space without implanting markers are difficult to find the virtual bone model surface point cloud samples corresponding to their positions one by one in the image space. The number of point cloud samples of the patient's bone surface collected during the operation is very limited. For the case without implanting markers, the existing point cloud registration algorithms are prone to long registration time and the accuracy cannot meet the requirements of medical surgeries for TKA surgeries.

[0004] The ICP (Iterative Closest Point) algorithm is currently the most commonly used algorithm for solving the 3D point cloud registration problem. The ICP algorithm requires a large amount of time for calculation. After verification, it is found that the nearest neighbor point search process occupies 72% of the calculation time of this algorithm. Most of the existing registration algorithms use the CPU or GPU for calculation, which is prone to problems such as high equipment cost price, long calculation time, large latency, or excessive resource computing power in actual applications, and cannot be directly applied to the intraoperative registration spatial registration in medical TKA surgery, especially in the case of large point cloud computing volume, high complexity, and high requirements for real-time performance and accuracy. The traditional ICP point cloud registration algorithm is calculated on the CPU, and the CPU performs serial calculation on the point cloud data, unable to achieve pipeline parallel processing, resulting in long registration time and unable to meet the requirements of total knee arthroplasty surgery.

[0005] It can be seen that in view of the current situation of intraoperative spatial registration technology for orthopedic surgery navigation systems, how to reduce costs, accelerate the registration time of point cloud registration algorithms, and improve registration accuracy are problems that need to be solved urgently by those in this technical field. Summary of the Invention

[0006] The purpose of this application is to provide a point cloud registration hardware acceleration method and system for orthopedic surgery navigation, to solve the problems of large point cloud registration calculation cost, long time consumption, and low registration accuracy existing in the existing intraoperative spatial registration algorithms for orthopedic surgery navigation; this application uses an electromagnetic positioning system to collect registration point information, adopts a point cloud registration algorithm combining rough registration and fine registration, and uses FPGA to accelerate the calculation of the point cloud registration algorithm, reducing the point cloud registration cost, accelerating the point cloud registration speed, and improving the registration accuracy.

[0007] This application first provides a point cloud registration hardware acceleration method for orthopedic surgery navigation, including: S1, obtaining the actual registered bone point cloud data in the electromagnetic positioning system; S2, importing the bone point cloud data into the DDR memory of the FPGA; S3, performing rough point cloud registration on the bone point cloud data based on the DSP module of the FPGA. Rough point cloud registration refers to introducing PCA to calculate the main axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; S4, performing fine point cloud registration based on the initial transformation matrix. Fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud; among them, the construction of the K-D Tree point cloud data topological structure and the nearest neighbor point search in the ICP registration algorithm are respectively executed by the tree topological structure data module and the nearest neighbor point search module at the PL end of the FPGA, and the remaining steps are executed by the embedded CPU at the PS end of the FPGA. The constructed K-D Tree topological structure of the target point cloud data and the target point cloud closest to the source point cloud searched are stored in the DDR memory of the FPGA.

[0008] In a possible implementation, the tree topology data module includes: a sorting module, configured to repeatedly sort the input point cloud data in the order of three dimensions of x, y, and z based on a buffer and a comparison unit, and select the median in the ordered point sequence as a tree node to complete the construction of the tree topology; an ordered point sequence caching module, configured to store the ordered point sequence after each single sorting in each round based on the BRAM in the FPGA.

[0009] In a possible implementation, the sorting module includes: N data input ports and one data output port, where N is an even number; the N data input ports are connected to N buffers, every two buffers are connected to a comparison unit, each comparison unit is connected to a buffer, and the buffers and the comparison units are alternately connected until the last buffer is connected to the data output port.

[0010] In a possible implementation, in the ordered point sequence caching module, both the number and the depth of the BRAMs are multiples of powers of 2.

[0011] In a possible implementation, the nearest neighbor point search module includes: a target point cloud data structure storage module, configured to store the constructed target point cloud K-D Tree structure, the target point cloud K-D Tree structure is divided into multiple sub-regions, and each sub-region contains multiple target point cloud points; a sub-region flag module, configured to determine the sub-region for the input source point cloud point to perform the nearest point search; a distance calculation module, configured to calculate the Euclidean distance between the source point cloud point and the target point cloud points in the sub-region.

[0012] In a possible implementation, the target point cloud data structure storage module is composed of n BRAMs, and the x, y, and z information of all the target point cloud points in the same sub-region is stored at the same set of addresses of the n BRAMs.

[0013] In a possible implementation, the sub-region flag module includes: multiple comparators and multiple two-to-one selectors; the comparators are configured to compare the x, y, and z of the input source point cloud point with the tree nodes of the target point cloud data K-D Tree topology; the two-to-one selectors are configured to output 1 or 0 according to the comparison results of the comparators, and the outputs of the two-to-one selectors are concatenated from high to low to obtain the sub-region flag for the source point cloud point to perform the nearest point search, and the sub-region flag is used to distinguish different sub-regions.

[0014] In a possible implementation, the distance calculation module is specifically configured to calculate the Euclidean distance between the input source point cloud point and each target point cloud point in the corresponding sub-region based on the DSP module in the FPGA, obtain the target point cloud point closest to the source point cloud point, and store it in the DDR memory of the FPGA.

[0015] The present application also provides a point cloud registration hardware acceleration system for orthopedic surgery navigation, which is used to implement the point cloud registration hardware acceleration method for orthopedic surgery navigation, including: a data acquisition unit for acquiring the bone point cloud data actually registered in the electromagnetic positioning system; a data import unit for importing the bone point cloud data into the DDR memory of the FPGA; a rough registration unit for performing rough point cloud registration on the bone point cloud data based on the DSP module of the FPGA. The rough point cloud registration refers to introducing PCA to calculate the main axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; a fine registration acceleration unit for performing fine point cloud registration based on the initial transformation matrix. The fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud. Among them, the construction of the K-D Tree point cloud data topology structure and the nearest neighbor point search in the ICP registration algorithm are respectively executed by the tree topology structure data module and the nearest neighbor point search module at the PL end of the FPGA, and the remaining steps are executed by the embedded CPU at the PS end of the FPGA. The constructed K-D Tree topology structure of the target point cloud data and the target point cloud closest to the source point cloud searched are stored in the DDR memory of the FPGA.

[0016] The present application also provides a computer program product, which, when running on a mobile terminal, enables the mobile terminal to execute the point cloud registration hardware acceleration method for orthopedic surgery navigation.

[0017] Compared with the prior art, the present application has the following beneficial effects: aiming at the problems of high cost and complex surgical operations caused by light occlusion in optical surgery navigation, an electromagnetic positioning navigation system is adopted to obtain the bone point cloud data of the actual registered patient. The electromagnetic surgical navigation is relatively low in price and there is no occlusion situation, reducing the complexity of the surgical registration process and the equipment cost; aiming at the problems of long registration time and low registration accuracy existing in the ICP registration algorithm, a point cloud registration process combining rough registration and fine registration is adopted. PCA is introduced to calculate the main axis direction of the point cloud data, and rough registration is completed to obtain the initial transformation matrix, and the fine registration adopts the ICP registration algorithm; aiming at the problem of long time-consuming nearest neighbor point search process in the ICP registration algorithm, the FPGA hardware is introduced to give play to the advantages of low cost and high-speed parallelism of the FPGA, and realize the hardware acceleration of the nearest neighbor point search for ICP fine registration. Description of the Drawings

[0018] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, form a part of this application, and do not constitute a limitation to the embodiments of the present invention. In the drawings:

[0019] Figure 1 It is a flowchart of the point cloud registration hardware acceleration method for orthopedic surgery navigation provided by the embodiment of the present application;

[0020] Figure 2The overall hardware acceleration structure diagram provided by the embodiments of the present application;

[0021] Figure 3 The principle example diagram of the data construction process of the tree topology structure provided by the embodiments of the present application;

[0022] Figure 4 The circuit diagram of the tree topology structure data module provided by the embodiments of the present application;

[0023] Figure 5 The principle example diagram of the nearest neighbor point search provided by the embodiments of the present application;

[0024] Figure 6 The composition diagram of the nearest neighbor point search module provided by the embodiments of the present application;

[0025] Figure 7 The mapping relationship diagram of the sub-region flag and BRAM address provided by the embodiments of the present application;

[0026] Figure 8 The circuit diagram of the sub-region flag module provided by the embodiments of the present application;

[0027] Figure 9 The working mechanism diagram of the sub-region flag module provided by the embodiments of the present application;

[0028] Figure 10 The registration effect diagram after performing the point cloud registration hardware acceleration method provided by the embodiments of the present application;

[0029] Figure 11 The structure diagram of the point cloud registration hardware acceleration system for orthopedic surgery navigation provided by the embodiments of the present application. Detailed implementation manners

[0030] In the following, the term "comprise" or "may comprise" that may be used in various embodiments of the present application indicates the presence of the claimed functions, operations or elements, and does not limit the addition of one or more functions, operations or elements. Further, as used in various embodiments of the present application, the terms "comprise", "have" and their cognates are only intended to indicate the presence of specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be construed as precluding the existence or addition of the possibility of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0031] It should be noted that: If a description states that a component is "connected" to another component or "linked" to another component, the first component can be directly connected to the second component, and a third component can be "connected" between the first component and the second component. Conversely, when a component is "directly connected" to another component or "directly linked" to another component, it can be understood that there is no third component between the first component and the second component.

[0032] The terms used in the various embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit the various embodiments of this application. As used herein, the singular form is intended to also include the plural form unless the context clearly indicates otherwise. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the various embodiments of this application belong. The terms (such as those defined in a general use dictionary) will be interpreted as having the same meaning as their contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of this application.

[0033] To make the objectives, technical solutions, and advantages of this application clearer and more understandable, the following further elaborates on this application in combination with embodiments and drawings. The illustrative embodiments of this application and their descriptions are only used to explain this application and do not serve as a limitation to this application.

[0034] Please refer to Figure 1 as shown Figure 1 is a flowchart of a point cloud registration hardware acceleration method for orthopedic surgery navigation provided by an embodiment of this application. The method includes: S1, obtaining the bone point cloud data actually registered in the electromagnetic positioning system; S2, importing the bone point cloud data into the DDR memory of the FPGA; S3, performing rough point cloud registration on the bone point cloud data based on the DSP module of the FPGA. Rough point cloud registration refers to introducing PCA to calculate the main axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; S4, performing fine point cloud registration based on the initial transformation matrix. Fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud. Among them, the construction of the K-D Tree point cloud data topological structure and the nearest neighbor point search in the ICP registration algorithm are respectively executed by the tree topological structure data module and the nearest neighbor point search module at the PL end of the FPGA, and the remaining steps are executed by the embedded CPU at the PS end of the FPGA. The constructed K-D Tree topological structure of the target point cloud data and the searched target point cloud closest to the source point cloud are stored in the DDR memory of the FPGA.

[0035] Specifically, please refer to Figure 2As shown Figure 2 This is the overall hardware acceleration structure diagram provided by the embodiment of the present application. In the point cloud registration hardware acceleration method, step S1 is to obtain the actually registered bone point cloud data from the electromagnetic positioning system and store it in the SD card in the form of a txt file; step S2 can use the embedded CPU on the PS side of the FPGA to read the txt point cloud data from the SD card and store it in the DDR memory of the FPGA. The DDR memory can use the relatively common DDR3, which supports higher transmission rates and lower power consumption; step S3 is to call the DSP (Digital Signal Processor) module of the FPGA to complete the rough registration process of the point cloud, that is: introduce PCA to calculate the main axis direction of the point cloud data, realize the initial registration of the source point cloud and the target point cloud, and obtain the initial transformation matrix, which is the iterative basis for the subsequent ICP fine registration; step S4 is to perform hardware acceleration on the ICP registration algorithm, in which the construction of the K-DTree point cloud data topology structure and the nearest neighbor point search process are hardware accelerated, and the remaining ICP registration steps are executed on the embedded CPU on the PS side. The above-mentioned hardware acceleration is performed by the hardware acceleration module composed of the PS side and the PL side of the FPGA: the tree topology structure data module and the nearest neighbor point search module. The DDR memory is used to store the constructed K-DTree topology structure of the target point cloud data and the calculation result of the nearest target point corresponding to the source point cloud. Here, the PS side and the PL side communicate through the AXI protocol.

[0036] The improvement of the present application lies in that, aiming at the problems of high cost and complex surgical operations caused by light occlusion in optical surgical navigation, an electromagnetic positioning navigation system is used to obtain the actually registered patient bone point cloud data. The electromagnetic surgical navigation is relatively low in price and there is no occlusion situation, reducing the complexity of the surgical registration process and the equipment cost; aiming at the problems of long registration time and low registration accuracy existing in the ICP registration algorithm, a point cloud registration process combining rough registration and fine registration is adopted, introducing PCA to calculate the main axis direction of the point cloud data, completing the rough registration to obtain the initial transformation matrix, and the fine registration adopts the ICP registration algorithm; aiming at the problem of long time-consuming nearest neighbor point search process in the ICP registration algorithm, FPGA hardware is introduced to give play to the advantages of low cost and high-speed parallelism of the FPGA, and realize the hardware acceleration of the nearest neighbor point search for ICP fine registration.

[0037] Step S1 is to obtain the actually registered bone point cloud data in the electromagnetic positioning system. It should be noted that the maximum number of registered points collected by the electromagnetic positioning system is 32,768 points. For example, for total knee arthroplasty, in order to be closer to the actual application, the source point cloud data obtained from the surgical registration can be subjected to downsampling preprocessing, and the same is true for the processing of the target point cloud. When the amount of point cloud data does not exceed 32,768 points, the method provided by the present application is used for hardware acceleration.

[0038] Step S4 is to accelerate the ICP registration process based on FPGA hardware. The ICP registration algorithm is a classic point cloud registration method used to calculate the optimal rigid body transformation (rotation and translation) between two point clouds to align them as closely as possible.

[0039] For the construction of the K-D Tree point cloud data topological structure in the ICP registration algorithm, this step requires sorting the point cloud data multiple times and calculating tree nodes. The construction of the K-D Tree first sorts n data for the first time, then selects the median as the tree node, and then divides into the left and right subspaces at the second layer. It is necessary to sort the n / 2 data in both subspaces respectively, and select the corresponding median as the tree nodes of the left and right subspaces, and so on. This sorting process consumes a lot of time. Therefore, this application proposes to use a hardware acceleration module (tree topological structure data module) to implement merge sorting to find the tree nodes and then construct the data of the tree topological structure.

[0040] Merge sorting is divided into two steps: recursive decomposition and merging. Recursive decomposition is to divide the sequence to be sorted into two parts, usually dividing the sequence into left and right subsequences from the middle position. Continue to recursively decompose these two subsequences until each subsequence contains only one element. Merging is to compare and merge two ordered subsequences into an ordered sequence. The space complexity of merge sorting is O(n), and the time complexity is O(nlogn). It is relatively easy to implement using FPGA hardware and the sorting is stable.

[0041] Please refer to Figure 3 as shown in Figure 3 the principle example diagram of the data construction process of the tree topological structure provided by the embodiment of this application. The input point cloud data is repeatedly sorted by merging the data of the selected dimension in the order of the three dimensions x, y, and z, and then the median is searched in the ordered sequence as the tree node, and the tree nodes are calculated iteratively to construct the tree topological structure of the data. Taking Figure 3 as an example: at the first layer, the tree node for the x dimension is obtained by sorting as 4, the left subsequence is (2, 7, 6), (4, 1, 2), (3, 9, 12), and the right subsequence is (9, 15, 21), (14, 10, 7). At the second layer, the tree nodes of the left and right subsequences on both sides are obtained by sorting as 7 for the left subsequence and 15 for the right subsequence in the y dimension, and then the left and right subsequences of each are constructed. Thus, the data is constructed into a tree topological structure.

[0042] In a possible implementation, the tree topology data module includes: a sorting module, which is used to repeatedly sort the input point cloud data in the order of the three dimensions of x, y, and z based on a buffer (Buffer) and a comparison unit (Ce), and select the median in the ordered point sequence as the tree node to complete the construction of the tree topology; an ordered point sequence caching module, which is used to store the ordered point sequence after each single sorting in each round based on the BRAM in the FPGA.

[0043] Specifically, in the tree topology data module of the FPGA hardware design, it is divided into a sorting module and an ordered point sequence caching module. The sorting module implements the sorting function and repeatedly constructs the tree topology for the input point cloud data in the order of the three dimensions of x, y, and z; the sorting module only sorts a certain dimension of x, y, or z of the point cloud data each time, and selects the median in the ordered point sequence as the tree node to complete the data construction. The ordered point sequence caching module uses the BRAM of the FPGA to store the point cloud data in the hardware acceleration module, specifically for storing the ordered point sequence after each single sorting in each round.

[0044] Further, the sorting module includes: N data input ports and one data output port, where N is an even number; the N data input ports are connected to N buffers, every two buffers are connected to a comparison unit, each comparison unit is connected to a buffer, and the buffers and the comparison units are alternately connected until the last buffer is connected to the data output port.

[0045] Specifically, according to the FPGA data storage characteristics and the point cloud data volume characteristics, it is set that the maximum number of point clouds that the sorting module allows to sort is 32768 points. The sorting module can be set with 64 data input ports, which are composed of a buffer (Buffer) and a comparison unit (Ce). Please refer to Figure 4 as shown Figure 4 This is the circuit diagram of the tree topology data module provided by the embodiment of the present application. When the sorting module sorts the initially unordered point cloud data for the first time, in the first round of sorting, multiple sorting processes with 64 data inputs will be performed to obtain 512 ordered point sequences with a sequence length of 64. In the second round of sorting, each time 64 ordered point sequences with a sequence length of 64 are input into the module for sorting to obtain 8 ordered point sequences with a sequence length of 4096. In the third round, only one sorting is required to obtain an ordered point sequence with a length of 32768, and the corresponding median is found as the tree node, thereby completing the tree structure construction of the left subsequence and the right subsequence in the first x dimension. The data construction of each subsequent layer of the tree structure is completed according to the above method. After the tree topology calculation is completed, the tree nodes and topological data structures of each layer are written back to the DDR memory.

[0046] Figure 4When the tree topology data module circuit shown is performing round sorting, in the first CLK, the 64 Buffer1s in the first column in the circuit complete pairwise comparison. In the second CLK, the results of the pairwise comparison are written into the 32 Buffer2s in the second column, and so on. Starting from the 12th CLK, the minimum value among the entire 64 input values is output. Every 2 CLKs, a minimum value is output in sequence and stored in the specified BRAM addresses in sequence, so as to finally output an ordered point sequence with a length of 64.

[0047] It can be understood that the clock required for the tree topology data module circuit is relatively less. Therefore, using an FPGA can effectively accelerate the sorting process of the tree topology compared to a CPU. For example, if the number of input ports of the single-sorting point data is 64 input terminals, the sorting process of up to 32,768 points with the maximum number of points can be completed through at most 3 rounds of sorting. In this way, when the number of target point clouds is large, the sorting module can still normally complete the acceleration task.

[0048] Furthermore, in the ordered point sequence cache module, both the number and depth of the BRAMs are multiples of 2 to the power of.

[0049] Specifically, the ordered point sequence cache module can be composed of 32 BRAMs with an address length of 1024 and a data bit width of 48 bits, 8 BRAMs with an address length of 4096 and a data bit width of 48 bits, and multiple multiplexers. For example, in the first round of sorting mentioned above, 512 ordered point sequences with a length of 64 are obtained. After the single sorting is completed, each ordered point sequence with a length of 64 is stored in the BRAM with a depth of 1024 in sequence according to the single-sorting order. There are 16 ordered point sequences with a length of 64 in each BRAM. Finally, all 512 ordered point sequences are stored in 32 BRAMs with a depth of 1024; in the second round of sorting, according to the previously stored address order, a single ordered point sequence with a length of 64 is read out from 32 BRAMs with a depth of 1024 in sequence and input to the 64 input terminals of the sorting module for sorting. After the single sorting is completed, an ordered point sequence with a length of 4096 is obtained. A total of 8 single sortings are performed to obtain 8 ordered point sequences with a length of 4096, and these 8 ordered point sequences are stored in the corresponding 8 BRAMs with a depth of 4096 in sequence.

[0050] It should be noted that, in addition to the above 32 BRAMs with an address length of 1024 and a data bit width of 48 bits, and 8 BRAMs with an address length of 4096 and a data bit width of 48 bits in the ordered point sequence cache module, when there are sufficient FPGA hardware storage resources, it can also be changed to other BRAM quantities and depths that are multiples of 2. The bit width of 48 bits is set for storage convenience based on the fact that each point in the point cloud has three-dimensional information of x, y, and z, and each dimension is stored with 16 bits.

[0051] For the nearest neighbor point search in the ICP registration algorithm, the calculation mechanism of the nearest neighbor point search module is as follows: The constructed target point cloud K-D Tree structure is divided into a sub-regions according to the subsequence of the tree topology structure, and each sub-region contains b target point cloud points. When a source point cloud point enters this module to search for the nearest neighbor point in the target point cloud, it first determines the entered sub-region through the sub-region flag module, and then calculates and compares the Euclidean distance with each of the b target point cloud points in this sub-region to determine its nearest point. Finally, the nearest point data is returned and stored in the DDR memory.

[0052] Please refer to Figure 5 shown in Figure 5 the example diagram of the nearest neighbor point search principle provided by the embodiment of this application. Assume that the nearest point search is performed on the input source point cloud point (8, 11, 3). Compare the value of the x dimension of this point with the first-layer tree node, and get 8 > 4, then enter the right sub-region; then compare the value of the y dimension of this point with the second-layer tree node, and get 11 < 15, then enter the left sub-region, that is, sub-region 1. Then the sub-region flag of this source point cloud point is 01, and subsequently, the Euclidean distance will be calculated and compared with all the target point cloud points in sub-region 1.

[0053] In a possible implementation manner, the nearest neighbor point search module includes: a target point cloud data structure storage module for storing the constructed target point cloud K-D Tree structure, which is divided into multiple sub-regions, and each sub-region contains multiple target point cloud points; a sub-region flag module for determining the sub-region where the input source point cloud point performs the nearest point search; and a distance calculation module for calculating the Euclidean distance between the source point cloud point and the target point cloud points in the sub-region.

[0054] Specifically, please refer to Figure 6 shown in Figure 6This is the composition diagram of the nearest neighbor point search module provided by the embodiments of this application. The nearest neighbor point search module consists of three major modules: the sub-region flag module, the target point cloud data structure storage module, and the distance calculation module. The sub-region flag module is used to determine which sub-region in the K-D Tree topological structure of the target point cloud data the input source point cloud points need to enter for nearest point search. The target point cloud data structure storage module is used to store the constructed K-D Tree topological structure of the target point cloud data. The distance calculation module is used to calculate the Euclidean distance between the source point cloud points and each target point cloud point in the corresponding sub-region.

[0055] Furthermore, the target point cloud data structure storage module consists of n BRAMs, and the same set of addresses of the n BRAMs stores the x, y, and z information of all target point cloud points in the same sub-region.

[0056] Specifically, the target point cloud data structure storage module stores the target point cloud data with a K-D Tree topological structure in the BRAM in the DDR memory. The K-D Tree topological structure of the target point cloud data is divided into multiple sub-regions according to the subsequence of the tree topological structure, and each sub-region contains multiple target point cloud points. In order to determine in which sub-region the input source point cloud points perform nearest point search, a sub-region flag is set. In this application, the addresses of the n BRAMs are corresponded to the sub-region flag, so the two are mapped. This application uses the storage space of 3 BRAM addresses to store 1 48-bit point data. 1 48-bit point data includes three-dimensional information of x, y, and z, and each dimension occupies 16 bits. All b target point cloud points in the same sub-region are stored at the same set of addresses of the n BRAMs. Please refer to Figure 7 as shown Figure 7 This is the sub-region flag and BRAM address mapping relationship diagram provided by the embodiments of this application. For example: all points in sub-region 0 are sequentially stored at addresses 0, 1, and 2 of each BRAM among the n BRAMs. The x, y, and z information of the first point in sub-region 0 is stored at addresses 0, 1, and 2 of BRAM1, and the x, y, and z information of the second point in sub-region 0 is stored at addresses 0, 1, and 2 of BRAM2, and so on to complete the storage of the target point cloud point information in one sub-region. Then, store the target point cloud point information in sub-region 1. The x, y, and z information of the first point in sub-region 1 is stored at addresses 3, 4, and 5 of BRAM1, and the x, y, and z information of the second point in sub-region 1 is stored at addresses 3, 4, and 5 of BRAM2, and so on. Finally, complete the storage of all sub-region points of the target point cloud in the corresponding n BRAM addresses.

[0057] Further, the sub-region flag module includes: a plurality of comparators and a plurality of two-to-one selectors; the comparators are used to compare the x, y, and z of the input source point cloud points with the tree nodes of the K-D Tree topological structure of the target point cloud data; the two-to-one selectors are used to output 1 or 0 according to the comparison results of the comparators, and the outputs of the two-to-one selectors are concatenated from high to low to obtain the sub-region flag for the nearest point search of the source point cloud points, and the sub-region flag is used to distinguish different sub-regions.

[0058] Specifically, please refer to Figure 8 as shown in Figure 8 the circuit diagram of the sub-region flag module provided by the embodiment of the present application. The sub-region flag module is used to calculate which sub-region of the K-D Tree topological structure of the target point cloud data the input source point cloud points enter for searching. Please refer to Figure 9 as shown in Figure 9 the working mechanism diagram of the sub-region flag module provided by the embodiment of the present application. Taking the 4 sub-regions obtained from the sorting results of Figure 5 two layers as an example, the input source point cloud point is (8, 11, 3). The tree node in the x dimension of the first layer is 4, and the left and right tree nodes in the y dimension of the second layer are 7 and 15. First, compare the value in the x dimension of the input source point cloud points with the tree node of the first layer, 8 > 4, the comparator 1_1 outputs 0, the high bit Add_r[1] = 0, and the D0 end of the two-to-one selector is selected. Then compare the value in the y dimension of the source point cloud points with the left and right tree nodes of the second layer, 11 > 7, the comparator 2_1 outputs 0. 15 > 11, the comparator 2_2 outputs 1. Since the selected end of the two-to-one multiplexer is the D0 port, Add_r[0] is 1. Concatenating and calculating from high to low to obtain the corresponding sub-region flag as 01, that is, sub-region 1. Finally, the source point cloud point (8, 11, 3) enters sub-region 1 for the nearest point search.

[0059] After being calculated by the sub-region flag module, the input source point cloud points are calculated with the tree nodes step by step, and finally an m-bit sub-region flag is obtained by concatenating from high to low, which can represent 0 to (2m - 1) sub-regions. Finally, multiplying the sub-region flag by 3 can obtain the read initial address of the BRAM as shown in Figure 9 Thus, it is possible to enter the corresponding BRAM to read all the target point cloud point information in the corresponding sub-region, calculate and compare the distance between the source point cloud points and the target point cloud points, and search for the nearest point of the source point cloud points among the target point cloud points.

[0060] Further, the distance calculation module is specifically used to calculate the Euclidean distance between the input source point cloud points and each target point cloud point in the corresponding sub-region based on the DSP module in the FPGA, obtain the target point cloud point closest to the source point cloud points and store it in the DDR memory of the FPGA.

[0061] Specifically, after determining the sub-region flag, the distance calculation module reads all the target point cloud point information in the selected sub-region at the corresponding BRAM address. Each target point cloud point is subjected to Euclidean calculation with the input source point cloud point to find its nearest neighbor point. Finally, the calculation results are returned and stored in the DDR memory.

[0062] This module uses the DSP resources in the FPGA to complete the calculation. The DSP of the FPGA contains calculation units such as multipliers, adders, and subtracters, which can be used to calculate the Euclidean distance between two points. Since the Euclidean distance involves a square root operation and the FPGA cannot perform this calculation comparison, the square of the Euclidean distance is used to compare the distances between two points.

[0063] The subtracter is used to calculate the differences in the three dimensions x, y, and z of two points in sequence, and the calculation results are input into the DSP to calculate the Euclidean distance d value. In order to retain the nearest point information for subsequent storage in the DDR memory, the nearest point information and the distance d value are bit-concatenated (the high 48 bits are the target point information, and the low 24 bits are the Euclidean distance). Subsequently, only the low-bit Euclidean distance is compared in a pipeline form. The corresponding number of nearest points can be obtained within a certain period. Finally, the calculated nearest points are returned and stored in the DDR memory to complete the hardware acceleration process of the nearest neighbor point search.

[0064] It can be understood that the point cloud registration hardware acceleration method for orthopedic surgery navigation space registration proposed by the present invention, on the one hand, overcomes the problems in the prior art that the cost of optical surgery navigation is high and the overall hardware device cost is high due to most registration algorithms using CPU or GPU for calculation. By using an electromagnetic positioning system to collect registration point information and an FPGA to perform registration algorithm calculations, the hardware device development cost is significantly reduced. On the other hand, by adopting a registration method that combines PCA coarse registration and ICP fine registration, while ensuring the registration accuracy required for medical surgery, and then utilizing the parallel processing advantage of the FPGA pipeline, the construction of the K-D Tree topological data structure and the nearest neighbor point search process in ICP registration are executed on the FPGA for hardware acceleration, thereby reducing the entire registration time by several times. Ultimately, the registration achieves the effects of higher accuracy and less time consumption.

[0065] It should be noted that in the present invention, the point cloud point data, the FPGA storage module, or the number of data input ports should preferably be an exponential multiple of 2, so that the FPGA can process data more quickly.

[0066] In addition, the embodiments of the present application also provide a specific experimental example to execute the point cloud registration hardware acceleration method as Figure 1 described.

[0067] To meet the hardware acceleration conditions, the point cloud data and target point cloud data in total knee arthroplasty are first downsampled. After downsampling, there are 16,384 target point cloud points and 4,096 source point cloud points. The sorting module in the tree topology data module in step S4 sets 64 data input ports. The ordered point sequence cache module uses 32 BRAMs with an address length of 1,024 and a data bit width of 48 bits, and 8 BRAMs with an address length of 4,096 and a data bit width of 48 bits. The nearest neighbor point search module is divided into a sub-regions, and each region contains b target point cloud points, where a = 256 and b = 128. The target point cloud data structure storage module consists of n BRAMs, where n = 128, and can store a maximum point cloud quantity scale of 32,768 points. The m-bit sub-region flag calculated by the sub-region flag module, where m = 8, corresponds to 256 sub-regions.

[0068] Please refer to Figure 10 as shown in Figure 10 the registration effect diagram after applying the point cloud registration hardware acceleration method provided by the embodiment of the present application. Among them, the green is the target point cloud, and the red is the registered point cloud after the source point cloud is transformed by the transformation matrix. The number of target point clouds is 16,384 points, and the source point cloud data obtained by surgical registration is 4,096 points. The average distance error of the registration accuracy is 1.089 mm, and the time consumed is 58.7 s. The registration time before acceleration is 153.51 s, which is 2.6 times the registration time after acceleration.

[0069] It can be seen that by applying the point cloud registration hardware acceleration method of the present application, it is possible to successfully reduce the time consumed by registration while meeting the registration accuracy; and as the scale of the point cloud expands, the registration time can be reduced in a multiple form.

[0070] Please refer to Figure 11 as shown in Figure 11 the structural diagram of the point cloud registration hardware acceleration system for orthopedic surgical navigation provided by the embodiment of the present application. The system is used to implement as Figure 1The point cloud registration hardware acceleration method for orthopaedic surgery navigation shown in the figure includes: a data acquisition unit for acquiring the bone point cloud data actually registered in the electromagnetic positioning system; a data import unit for importing the bone point cloud data into the DDR memory of the FPGA; a rough registration unit for performing rough point cloud registration on the bone point cloud data based on the DSP module of the FPGA. The rough point cloud registration refers to introducing PCA to calculate the main axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; a fine registration acceleration unit for performing fine point cloud registration based on the initial transformation matrix. The fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud. Among them, the construction of the K-D Tree point cloud data topological structure and the nearest neighbor point search in the ICP registration algorithm are respectively executed by the tree topological structure data module and the nearest neighbor point search module at the PL end of the FPGA, and the remaining steps are executed by the embedded CPU at the PS end of the FPGA. The constructed K-D Tree topological structure of the target point cloud data and the target point cloud closest to the source point cloud searched are stored in the DDR memory of the FPGA.

[0071] It can be understood that the point cloud registration hardware acceleration system for orthopaedic surgery navigation of the present application is used to implement the Figure 1 point cloud registration hardware acceleration method, corresponding one by one to the method, and having corresponding technical effects, so it will not be elaborated here.

[0072] The embodiment of the present application also provides a computer program product. When the computer program product runs on a mobile terminal, it enables the mobile terminal to execute the Figure 1 point cloud registration hardware acceleration method for orthopaedic surgery navigation shown in the figure.

[0073] The specific embodiments described above have further elaborated on the purpose, technical solution and beneficial effects of the present invention. It should be understood that the above description is only the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A point cloud registration hardware acceleration method for orthopedic surgery navigation, characterized in that: include: S1, obtaining bone point cloud data actually registered in the electromagnetic positioning system; S2, import the bone point cloud data into the DDR memory of FPGA; S3, performing point cloud rough registration on the bone point cloud data based on the FPGA DSP module, wherein the point cloud rough registration refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; S4, performing point cloud precise registration based on the initial transformation matrix, wherein the point cloud precise registration refers to executing an ICP registration algorithm to obtain an optimal transformation matrix of a source point cloud and a target point cloud; Among them, the KD Tree point cloud data topology structure construction and nearest neighbor search in the ICP registration algorithm are executed by the tree topology structure data module and the nearest neighbor search module on the PL side of the FPGA respectively, and the remaining steps are executed by the embedded CPU on the PS side of the FPGA. The constructed target point cloud data KD Tree topology structure and the searched target point cloud closest to the source point cloud are stored in the DDR memory of the FPGA.

2. The point cloud registration hardware acceleration method for orthopedic surgery navigation according to claim 1, characterized in that: The tree topology data module includes: A sorting module is used to repeatedly sort the input point cloud data in the order of x, y, and z dimensions based on the buffer and the comparison unit to obtain an ordered point sequence, and select the median in the ordered point sequence as a tree node to complete the construction of the tree topology structure; The ordered point sequence cache module is used to store the ordered point sequence after a single sorting is completed in each round of sorting based on the BRAM in the FPGA.

3. The point cloud registration hardware acceleration method for orthopedic surgery navigation according to claim 2, characterized in that: The sorting module comprises: N data input ports and one data output port, wherein N is an even number; N data input ports are connected to N buffers, every two buffers are connected to a comparison unit, each comparison unit is connected to a buffer, and the buffers are connected to the comparison units alternately until the last buffer is connected to the data output port.

4. The point cloud registration hardware acceleration method for orthopedic surgery navigation according to claim 2, characterized in that: In the ordered point sequence cache module, the number and depth of BRAMs are both exponential times of 2.

5. The point cloud registration hardware acceleration method for orthopedic surgery navigation according to claim 1, characterized in that: The nearest neighbor search module comprises: A target point cloud data structure storage module is used to store the constructed target point cloud data KD Tree topological structure, wherein the target point cloud data KD Tree topological structure is divided into a plurality of sub-regions according to a sub-sequence of the tree topological structure, and each sub-region contains a plurality of target point cloud points; The sub-region marking module is used to determine the sub-region of the input source point cloud point for the nearest point search; The distance calculation module is used to calculate the Euclidean distance between the source point cloud point and the target point cloud point in the sub-region.

6. The point cloud registration hardware acceleration method for orthopedic surgery navigation according to claim 5, characterized in that: The target point cloud data structure storage module is composed of n BRAMs, and the same group address of the n BRAMs stores the x, y, and z information of all target point cloud points in the same sub-area.

7. The point cloud registration hardware acceleration method for orthopedic surgery navigation according to claim 5, characterized in that: The sub-region marking module includes: a plurality of comparators and a plurality of two-to-one selectors; The comparator is used to compare the x, y, and z of the input source point cloud point with the tree nodes of the target point cloud data KD Tree topological structure; The two-to-one selector is used to output 1 or 0 according to the comparison result of the comparator. The output of the two-to-one selector is spliced ​​from high to low to obtain a sub-region flag for searching the nearest point of the source point cloud point. The sub-region flag is used to distinguish different sub-regions.

8. The point cloud registration hardware acceleration method for orthopedic surgery navigation according to claim 5, characterized in that: The distance calculation module is specifically used to calculate the Euclidean distance between the input source point cloud point and each target point cloud point in the corresponding sub-area based on the DSP module in the FPGA, obtain the target point cloud point closest to the source point cloud point and store it in the DDR memory of the FPGA.

9. Point cloud registration hardware acceleration system for orthopedic surgery navigation, characterized in that: The method for executing the point cloud registration hardware acceleration method according to any one of claims 1 to 8 comprises: A data acquisition unit, used for acquiring bone point cloud data actually registered in the electromagnetic positioning system; A data import unit is used to import bone point cloud data into the DDR memory of FPGA; A coarse registration unit, used for performing coarse registration of the bone point cloud data based on the DSP module of the FPGA, wherein the coarse registration of the point cloud refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain an initial transformation matrix of the source point cloud and the target point cloud; A precision registration acceleration unit is used to perform precision registration of point clouds based on the initial transformation matrix. Precise registration of point clouds refers to executing an ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud. The K-DTree point cloud data topology structure construction and the nearest neighbor search in the ICP registration algorithm are respectively executed by the tree topology structure data module and the nearest neighbor search module on the PL side of the FPGA, and the remaining steps are executed by the embedded CPU on the PS side of the FPGA. The constructed target point cloud data KD Tree topology structure and the searched target point cloud closest to the source point cloud are stored in the DDR memory of the FPGA.

10. A computer program product, characterized in that When the computer program product runs on a mobile terminal, the mobile terminal executes the point cloud registration hardware acceleration method as described in any one of claims 1-8.

Images