Point cloud registration hardware acceleration method and system for orthopedic surgery navigation

Abstract

The application discloses a point cloud registration hardware acceleration method and system for orthopedic surgery navigation, relates to the field of registration space registration technology in medical surgery, and solves the problems of high point cloud registration calculation cost, long time consumption and low registration accuracy of the existing orthopedic surgery navigation space registration algorithm, and the scheme points are as follows: acquiring actual registered bone point cloud data in an electromagnetic positioning system; importing the bone point cloud data into an FPGA; performing point cloud coarse registration on the bone point cloud data based on a DSP module to obtain an initial transformation matrix; performing point cloud fine registration based on the initial transformation matrix to obtain an optimal transformation matrix; wherein K-D tree point cloud data topology structure construction and nearest neighbor point search are respectively performed by a tree topology structure data module and a nearest neighbor point search module in the FPGA, and the remaining steps are performed by an embedded CPU in the FPGA; coarse registration and fine registration are combined with FPGA acceleration, so that the cost is reduced, the point registration speed is accelerated, and the registration accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of registration spatial registration technology in medical surgery, and more specifically, to a point cloud registration hardware acceleration method and system for orthopedic surgical navigation. Background Technology

[0002] Orthopedic surgical navigation systems, with their intelligent, precise, and minimally invasive treatment concepts and application technologies, provide surgeons with functions such as surgical planning and positioning measurements, demonstrating significant application value and broad prospects in orthopedic clinical surgery. However, domestic surgical navigation systems suffer from deficiencies in spatial registration technology, and the mainstream navigation systems in China are currently monopolized by foreign companies. The mainstream navigation systems in China are optical positioning systems, which offer relatively high accuracy and good synchronization stability. However, optical surgical navigation suffers from high costs and complications arising from light obstruction during surgery. Because optical surgical navigation systems require unobstructed views between the infrared camera and the tracking target during surgery, it inconveniences the surgeon. Furthermore, if the reference target point fixed to the patient's bony structure moves during surgery, the navigation accuracy will shift, affecting intraoperative registration time and precision efficiency.

[0003] Spatial registration is one of the core technologies of orthopedic robotic systems. Its goal is to obtain a rigid transformation relationship between the patient's physical space and its three-dimensional virtual model image space. Existing intraoperative registration spatial registration techniques are divided into paired marker registration methods based on medical markers and unpaired marker registration methods based on anatomical structural features. The former implants medical markers (such as medical titanium or steel screws) into the patient's bone so that these markers can be clearly displayed in the image space and are easily exposed and acquired in the patient space, thus ensuring high accuracy of point cloud sample acquisition. However, the implantation of markers will increase the patient's pain and psychological burden, and slight movement of the markers may occur during the operation, or even failure to be successfully implanted in some cases. The latter acquires point cloud data of the anatomical structure on the surface of the patient's bone and then registers the two sets of point clouds using a spatial registration algorithm. In actual surgery, the exposed area of ​​the affected bone is limited, the field of view is small, and the local structure of bone tissue is complex. Without implanted markers, it is difficult to find virtual bone model surface point cloud samples in the image space that correspond one-to-one with their positions in the point cloud samples acquired in the patient space. The number of point cloud samples collected from the patient's bone surface during surgery is very limited. For TKA surgery without implanted markers, existing point cloud registration algorithms are prone to long registration times and the accuracy cannot meet the requirements of medical surgery.

[0004] The Iterative Closest Point (ICP) algorithm is currently the most commonly used algorithm for solving the 3D point cloud registration problem. The ICP algorithm requires a significant amount of computation time; verification has shown that the nearest neighbor search process accounts for 72% of the algorithm's computation time. Most existing registration algorithms use CPUs or GPUs for computation, which in practical applications often results in high equipment costs, long computation times, high latency, or excessive resource consumption, making them unsuitable for direct application in intraoperative registration space registration during total knee arthroplasty (TKA) surgery, especially when point cloud data is large, complex, and requires high real-time performance and accuracy. Traditional ICP point cloud registration algorithms are computed on CPUs, which perform serial computation on point cloud data, failing to achieve pipelined parallel processing, thus resulting in long registration times and not meeting the requirements of total knee arthroplasty.

[0005] It is evident that, given the current state of intraoperative spatial registration technology in orthopedic surgical navigation systems, reducing costs, accelerating the registration time of point cloud registration algorithms, and improving registration accuracy are pressing issues that researchers in this field need to address. Summary of the Invention

[0006] The purpose of this application is to provide a hardware acceleration method and system for point cloud registration for orthopedic surgical navigation, which solves the problems of high computational cost, long time consumption and low registration accuracy of existing intraoperative spatial registration algorithms for orthopedic surgical navigation. This application uses an electromagnetic positioning system to collect registration point information, adopts a point cloud registration algorithm that combines coarse and fine registration, and uses FPGA to accelerate the calculation of the point cloud registration algorithm, thereby reducing the cost of point cloud registration, speeding up the point cloud registration, and improving the registration accuracy.

[0007] This application first provides a hardware acceleration method for point cloud registration for orthopedic surgical navigation, including: S1, acquiring bone point cloud data actually registered in the electromagnetic positioning system; S2, importing the bone point cloud data into the DDR memory of the FPGA; S3, performing coarse point cloud registration on the bone point cloud data based on the FPGA's DSP module, where coarse point cloud registration refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; S4, performing fine point cloud registration based on the initial transformation matrix, where fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud; wherein, the KD Tree point cloud data topology construction and nearest neighbor search in the ICP registration algorithm are executed by the tree topology data module and the nearest neighbor search module on the PL end of the FPGA, respectively, and the remaining steps are executed by the embedded CPU on the PS end of the FPGA, and the constructed target point cloud data KD Tree topology and the target point cloud closest to the source point cloud are stored in the FPGA's DDR memory.

[0008] In one possible implementation, the tree topology data module includes: a sorting module, used to repeatedly sort the input point cloud data according to the three dimensions of x, y, and z based on a buffer and a comparison unit, and select the median of the ordered point sequence as a tree node to complete the construction of the tree topology; and an ordered point sequence caching module, used to store the ordered point sequence after each round of sorting based on the BRAM in the FPGA.

[0009] In one possible implementation, the sorting module includes: N data input ports and one data output port, where N is an even number; the N data input ports are connected to N buffers, every two buffers are connected to a comparison unit, and each comparison unit is connected to a buffer, with the buffers and comparison units connected alternately until the last buffer is connected to the data output port.

[0010] In one possible implementation, the number and depth of BRAMs in the ordered point sequence cache module are both exponential multiples of 2.

[0011] In one possible implementation, the nearest neighbor search module includes: a target point cloud data structure storage module for storing a constructed target point cloud KD Tree structure, wherein the target point cloud KD Tree structure is divided into multiple sub-regions, each sub-region containing multiple target point cloud points; a sub-region labeling module for determining the sub-region for nearest neighbor search of the input source point cloud point; and a distance calculation module for calculating the Euclidean distance between the source point cloud point and the target point cloud point in the sub-region.

[0012] In one possible implementation, the target point cloud data structure storage module consists of n BRAMs, and the same set of addresses in the n BRAMs stores the x, y, and z information of all target point cloud points within the same sub-region.

[0013] In one possible implementation, the sub-region labeling module includes: multiple comparators and multiple binary selectors; the comparators are used to compare the x, y, and z values ​​of the input source point cloud points with the tree nodes of the KD Tree topology structure of the target point cloud data; the binary selectors are used to output 1 or 0 according to the comparison result of the comparators, and the outputs of the binary selectors are concatenated from high bits to low bits to obtain the sub-region label for the source point cloud points to perform nearest point search, and the sub-region label is used to distinguish different sub-regions.

[0014] In one possible implementation, the distance calculation module is specifically used to calculate the Euclidean distance between the input source point cloud and each target point cloud in the corresponding sub-region based on the DSP module in the FPGA, to obtain the target point cloud closest to the source point cloud and store it in the DDR memory of the FPGA.

[0015] This application also provides a hardware acceleration system for point cloud registration for orthopedic surgical navigation, which implements a hardware acceleration method for point cloud registration for orthopedic surgical navigation. The system includes: a data acquisition unit for acquiring bone point cloud data actually registered in the electromagnetic positioning system; a data import unit for importing the bone point cloud data into the DDR memory of the FPGA; a coarse registration unit for performing coarse point cloud registration on the bone point cloud data based on the FPGA's DSP module. Coarse point cloud registration refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain the initial transformation matrix of the source and target point clouds; and a fine registration acceleration unit for performing fine point cloud registration based on the initial transformation matrix. Fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source and target point clouds. In the ICP registration algorithm, the KD Tree point cloud data topology construction and nearest neighbor search are executed by the tree topology data module and nearest neighbor search module on the PL side of the FPGA, respectively. The remaining steps are executed by the embedded CPU on the PS side of the FPGA. The constructed target point cloud data KD... The tree topology and the target point cloud that is closest to the source point cloud are stored in the FPGA's DDR memory.

[0016] This application also provides a computer program product that, when run on a mobile terminal, causes the mobile terminal to execute a point cloud registration hardware acceleration method for orthopedic surgical navigation.

[0017] Compared with existing technologies, this application has the following advantages: Addressing the problems of high cost and complex surgical procedures caused by light occlusion in optical surgical navigation, an electromagnetic positioning and navigation system is used to acquire actual registered patient bone point cloud data. Electromagnetic surgical navigation is relatively inexpensive and does not suffer from occlusion, reducing the complexity of the surgical registration process and equipment costs. Addressing the problems of long registration time and low registration accuracy in the ICP registration algorithm, a point cloud registration process combining coarse and fine registration is adopted. PCA is introduced to calculate the principal axis direction of the point cloud data to complete coarse registration and obtain the initial transformation matrix. The fine registration criterion uses the ICP registration algorithm. Addressing the long nearest neighbor search process in the ICP registration algorithm, FPGA hardware is introduced, leveraging the advantages of low cost and high-speed parallelism of FPGA to achieve hardware acceleration of the nearest neighbor search in ICP fine registration. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0019] Figure 1 A flowchart of a point cloud registration hardware acceleration method for orthopedic surgical navigation provided in an embodiment of this application;

[0020] Figure 2This is a diagram illustrating the overall hardware acceleration structure provided in the embodiments of this application.

[0021] Figure 3 A schematic diagram illustrating the principle of data construction of a tree topology provided in this application embodiment;

[0022] Figure 4 A circuit diagram of a tree topology data module provided in an embodiment of this application;

[0023] Figure 5 Example diagram of the nearest neighbor search principle provided in the embodiments of this application;

[0024] Figure 6 A diagram illustrating the configuration of the nearest neighbor search module provided in this application embodiment;

[0025] Figure 7 A sub-region identifier and BRAM address mapping diagram provided for embodiments of this application;

[0026] Figure 8 A circuit diagram of a sub-region marker module provided in an embodiment of this application;

[0027] Figure 9 A diagram illustrating the working mechanism of the sub-region marker module provided in this application embodiment;

[0028] Figure 10 The registration effect diagram is provided by the hardware acceleration method for point cloud registration in the embodiments of this application.

[0029] Figure 11 This is a structural diagram of a point cloud registration hardware acceleration system for orthopedic surgical navigation provided in an embodiment of this application. Detailed Implementation

[0030] In the following, the terms “comprising” or “may include” as used in the various embodiments of this application indicate the presence of the claimed function, operation, or element, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in the various embodiments of this application, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of features, numbers, steps, operations, elements, components, or combinations of the foregoing.

[0031] It should be noted that if a description refers to "connecting" a component to another component or "connecting" it to another component, then the first component can be directly connected to the second component, and a third component can be "connected" between the first and second components. Conversely, when a component is "directly connected" to another component or "directly connected" to another component, it can be understood that there is no third component between the first and second components.

[0032] The terminology used in the various embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments of this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. The terms (such as those defined in a generally used dictionary) are to be interpreted as having the same meaning as in the context of the relevant technical field and are not to be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0034] Please see Figure 1 As shown, Figure 1 This is a flowchart illustrating a hardware acceleration method for point cloud registration in orthopedic surgical navigation provided in an embodiment of this application. The method includes: S1, acquiring bone point cloud data actually registered in an electromagnetic positioning system; S2, importing the bone point cloud data into the DDR memory of an FPGA; S3, performing coarse point cloud registration on the bone point cloud data based on the FPGA's DSP module, where coarse point cloud registration refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; S4, performing fine point cloud registration based on the initial transformation matrix, where fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud; wherein, the KD Tree point cloud data topology construction and nearest neighbor search in the ICP registration algorithm are executed by the tree topology data module and the nearest neighbor search module on the PL side of the FPGA, respectively, and the remaining steps are executed by the embedded CPU on the PS side of the FPGA, and the constructed target point cloud data KD Tree topology and the searched target point cloud closest to the source point cloud are stored in the FPGA's DDR memory.

[0035] Specifically, please see Figure 2As shown, Figure 2 This is a diagram illustrating the overall hardware acceleration structure provided in this application embodiment. In the point cloud registration hardware acceleration method, step S1 involves acquiring the actual registered bone point cloud data from the electromagnetic positioning system and storing it as a txt file on an SD card. Step S2 utilizes the embedded CPU on the PS side of the FPGA to read the txt point cloud data from the SD card and store it in the FPGA's DDR memory. The DDR memory can be the commonly used DDR3, supporting higher transmission rates and lower power consumption. Step S3 involves calling the FPGA's DSP (Digital Signal Processor) module to complete the coarse point cloud registration process, i.e., introducing PCA to calculate the principal axis direction of the point cloud data to achieve initial registration of the source and target point clouds, obtaining an initial transformation matrix. This initial transformation matrix is ​​the basis for subsequent ICP fine registration iterations. Step S4 involves hardware acceleration of the ICP registration algorithm, specifically hardware acceleration of the K-DTree point cloud data topology construction and nearest neighbor search process, while the remaining ICP registration steps are executed by the embedded CPU on the PS side. The aforementioned hardware acceleration is performed by a hardware acceleration module composed of the FPGA's PS and PL sides: a tree topology data module and a nearest neighbor search module. DDR memory is used to store the constructed target point cloud data K-DTree topology and the cloud computing results of the nearest target point corresponding to the source point cloud. The PS and PL terminals communicate via the AXI protocol.

[0036] The improvements in this application are as follows: To address the issues of high cost and complex surgical procedures caused by light occlusion in optical surgical navigation, an electromagnetic positioning and navigation system is used to acquire actual registered patient bone point cloud data. Electromagnetic surgical navigation is relatively inexpensive and does not suffer from occlusion, reducing the complexity of the surgical registration process and equipment costs. To address the issues of long registration time and low registration accuracy in the ICP registration algorithm, a point cloud registration process combining coarse and fine registration is adopted. PCA is introduced to calculate the principal axis direction of the point cloud data to complete coarse registration and obtain the initial transformation matrix. The fine registration criterion uses the ICP registration algorithm. To address the long nearest neighbor search process in the ICP registration algorithm, FPGA hardware is introduced, leveraging the advantages of low cost and high-speed parallelism of FPGAs to achieve hardware acceleration of the nearest neighbor search in ICP fine registration.

[0037] Step S1 involves acquiring the actual registered bone point cloud data in the electromagnetic positioning system. It should be noted that the maximum number of registered points collected using the electromagnetic positioning system is 32,768. For total knee replacement surgery, to better reflect real-world applications, the source point cloud data obtained from surgical registration can undergo downsampling preprocessing; the processing of the target point cloud is similar. Hardware acceleration is performed using the method provided in this application when the point cloud data volume does not exceed 32,768 points.

[0038] Step S4 accelerates the ICP registration process using FPGA hardware. The ICP registration algorithm is a classic point cloud registration method used to calculate the optimal rigid body transformation (rotation and translation) between two point clouds to align them as closely as possible.

[0039] For the KD Tree point cloud data topology construction in the ICP registration algorithm, this step requires multiple sorting and tree node calculations of the point cloud data. KD Tree construction involves first sorting n data points, then selecting the median as the tree node, and then dividing the data into left and right subspaces at the second level. This requires sorting n / 2 data points in each subspace separately, selecting the corresponding median as the tree node for the left and right subspaces, and so on. This sorting process is time-consuming. Therefore, this application proposes using a hardware acceleration module (tree topology data module) to perform a merge sort method to sort the data, find the tree nodes, and then construct the tree topology data.

[0040] Merge sort consists of two steps: recursive decomposition and merging. Recursive decomposition divides the sequence to be sorted into two parts, typically splitting it into two subsequences from the middle. These subsequences are then recursively decomposed until each subsequence contains only one element. Merging compares the two sorted subsequences and combines them into a single sorted sequence. Merge sort has a space complexity of O(n) and a time complexity of O(nlogn), making it relatively easy to implement using FPGA hardware and ensuring stable sorting.

[0041] Please see Figure 3 As shown, Figure 3 This diagram illustrates the principle of the data construction process for the tree topology structure provided in this application embodiment. The input point cloud data is repeatedly merged and sorted along the selected dimensions (x, y, z). The median is then found as the tree node in the ordered sequence, and the tree nodes are iteratively calculated to construct the tree topology structure. Figure 3 For example: In the first layer, sorting yields a tree node of 4 in the x-axis, with left subsequences of (2,7,6), (4,1,2), and (3,9,12), and right subsequences of (9,15,21) and (14,10,7). In the second layer, sorting the subsequences on both sides yields a tree node of 7 in the left subsequence and 15 in the right subsequence in the y-axis, and then constructing their respective left and right subsequences. This builds a tree topology from the data.

[0042] In one possible implementation, the tree topology data module includes: a sorting module, used to repeatedly sort the input point cloud data according to the three dimensions of x, y, and z based on a buffer and a comparison unit (Ce), and select the median of the ordered point sequence as the tree node to complete the construction of the tree topology; and an ordered point sequence caching module, used to store the ordered point sequence after each round of sorting based on the BRAM in the FPGA.

[0043] Specifically, the tree topology data module in the FPGA hardware design is divided into a sorting module and an ordered point sequence caching module. The sorting module performs the sorting function, repeatedly constructing a tree topology from the input point cloud data according to the x, y, and z dimensions. The sorting module sorts only one dimension (x, y, z) of the point cloud data at a time, selecting the median of the ordered point sequence as the tree node to complete the data construction. The ordered point sequence caching module uses the FPGA's BRAM to store the point cloud data from the hardware acceleration module, specifically storing the ordered point sequence after each round of sorting.

[0044] Furthermore, the sorting module includes: N data input ports and one data output port, where N is an even number; the N data input ports are connected to N buffers, every two buffers are connected to a comparison unit, and each comparison unit is connected to a buffer, with buffers and comparison units connected alternately until the last buffer is connected to the data output port.

[0045] Specifically, based on the characteristics of FPGA data storage and the volume of point cloud data, the sorting module is set to sort a maximum of 32,768 points. The sorting module can be configured with 64 data input ports, consisting of buffers and comparison units (Ce). Please refer to [link to relevant documentation]. Figure 4 As shown, Figure 4 This is a circuit diagram of the tree topology data module provided in this embodiment. When the sorting module sorts the initial unordered point cloud data, the first round of sorting involves multiple sorting processes with 64 data inputs, resulting in 512 ordered point sequences of length 64. In the second round, each of the 64 ordered point sequences of length 64 is input into the module for sorting, resulting in 8 ordered point sequences of length 4096. The third round only requires one sorting process to obtain an ordered point sequence of length 32768. The corresponding median is then used as the tree node, thus completing the first construction of the tree structure for the left and right subsequences along the x-axis. The data construction for each subsequent layer of the tree structure is completed using the above method. After the tree topology calculation is completed, the tree nodes and topology data structure for each layer are written back to DDR memory.

[0046] Figure 4When the tree topology data module circuit shown performs round-by-round sorting, the first CLK completes the pairwise comparison of the 64 Buffer1s in the first column of the circuit. The second CLK writes the results of the pairwise comparisons into the 32 Buffer2s in the second column, and so on. Starting from the 12th CLK, the minimum value among the 64 input values ​​is output. Every 2 CLKs, a minimum value is output and stored in the specified BRAM address, thus finally outputting an ordered point sequence of length 64.

[0047] Understandably, the data module circuit using this tree topology structure requires relatively fewer clock cycles, thus allowing FPGAs to effectively speed up the sorting process compared to CPUs. For example, with 64 input ports per sorting cycle, a maximum of 32,768 points can be sorted in a maximum of 3 rounds. This ensures that even with a large number of target point clouds, the sorting module can still perform its accelerated task effectively.

[0048] Furthermore, in the ordered point sequence cache module, the number and depth of BRAMs are both exponential multiples of 2.

[0049] Specifically, the ordered point sequence cache module can consist of 32 BRAMs with an address length of 1024 and a data width of 48 bits, 8 BRAMs with an address length of 4096 and a data width of 48 bits, and multiple multiplexers. For example, in the first round of sorting mentioned earlier, 512 ordered point sequences of length 64 are obtained. After a single sorting, each ordered point sequence of length 64 is stored sequentially into a BRAM of depth 1024 according to the single sorting order, with each BRAM containing 16 ordered point sequences of length 64. Finally, all 512 ordered point sequences are stored into the 32 BRAMs of depth 1024. In the second round of sorting, according to the previously stored address order, a single ordered point sequence of length 64 is read from each of the 32 BRAMs of depth 1024 and input into the 64 input terminals of the sorting module for sorting. After a single sorting operation, an ordered point sequence of length 4096 is obtained. A total of 8 single sorting operations are performed to obtain 8 ordered point sequences of length 4096. These 8 ordered point sequences are then stored sequentially into 8 corresponding BRAMs of depth 4096.

[0050] It should be noted that, in addition to the aforementioned 32 BRAMs with an address length of 1024 and a data width of 48 bits, and 8 BRAMs with an address length of 4096 and a data width of 48 bits, the number and depth of BRAMs in the ordered point sequence cache module can be changed to other exponential multiples of 2, provided there are sufficient storage resources in the FPGA hardware. The 48-bit bit width is chosen because a point in the point cloud has three dimensions (x, y, z), with each dimension stored using 16 bits; this is for ease of storage.

[0051] For the nearest neighbor search in the ICP registration algorithm, the computational mechanism of the nearest neighbor search module is as follows: The constructed target point cloud KD Tree structure is divided into 'a' sub-regions according to the subsequence of the tree topology, and each sub-region contains 'b' target point cloud points. When a source point enters this module to search for its nearest neighbor in the target point cloud, it first determines the entered sub-region through the sub-region marking module, and then compares its Euclidean distance with each of the 'b' target points in that sub-region to determine the nearest point. Finally, the nearest point data is returned and stored in DDR memory.

[0052] Please see Figure 5 As shown, Figure 5 This diagram illustrates the principle of nearest neighbor search provided in this application. Assume a nearest neighbor search is performed on the input source point cloud (8, 11, 3). The x-dimensional value of this point is compared with the first-level tree nodes; 8 > 4, so it enters the right sub-region. Then, the y-dimensional value of this point is compared with the second-level tree nodes; 11 < 15, so it enters the left sub-region, i.e., sub-region 1. The sub-region of this source point cloud is then marked as 01, and subsequent distances will be calculated and compared with all target point clouds in sub-region 1.

[0053] In one possible implementation, the nearest neighbor search module includes: a target point cloud data structure storage module for storing a constructed target point cloud KD Tree structure, wherein the target point cloud KD Tree structure is divided into multiple sub-regions, each sub-region containing multiple target point cloud points; a sub-region labeling module for determining the sub-region for nearest neighbor search of the input source point cloud point; and a distance calculation module for calculating the Euclidean distance between the source point cloud point and the target point cloud point in the sub-region.

[0054] Specifically, please see Figure 6 As shown, Figure 6This diagram illustrates the structure of the nearest neighbor search module provided in this embodiment. The nearest neighbor search module consists of three main modules: a sub-region labeling module, a target point cloud data structure storage module, and a distance calculation module. The sub-region labeling module determines which sub-region within the KD Tree topology of the target point cloud data the input source point needs to enter for nearest neighbor search. The target point cloud data structure storage module stores the constructed KD Tree topology of the target point cloud data. The distance calculation module calculates the Euclidean distance between the source point and each target point in the corresponding sub-region.

[0055] Furthermore, the target point cloud data structure storage module consists of n BRAMs, and the same set of addresses in the n BRAMs stores the x, y, and z information of all target point cloud points within the same sub-region.

[0056] Specifically, the target point cloud data structure storage module stores target point cloud data with a KD Tree topology in BRAM within DDR memory. The KD Tree topology of the target point cloud data is divided into multiple sub-regions according to the sub-sequences of the tree topology, and each sub-region contains multiple target point cloud points. To determine which sub-region the input source point cloud point belongs to for nearest point lookup, a sub-region flag is set. This application uses n BRAM addresses to map to the sub-region flags, thus mapping the two. This application uses 3 BRAM address storage spaces to store one 48-bit point data. One 48-bit point data includes three dimensions: x, y, and z, each occupying 16 bits. All b target point cloud points in the same sub-region are stored in the same set of addresses across the n BRAMs. Please refer to [link to relevant documentation]. Figure 7 As shown, Figure 7 This diagram illustrates the mapping relationship between sub-region markers and BRAM addresses provided in this application embodiment. For example, all points in sub-region 0 are sequentially stored in addresses 0, 1, and 2 of each of the n BRAMs. Addresses 0, 1, and 2 of BRAM1 store the x, y, and z information of the first point in sub-region 0, and addresses 0, 1, and 2 of BRAM2 store the x, y, and z information of the second point in sub-region 0. This process is repeated to complete the storage of target point cloud information for one sub-region. Next, the target point cloud information for sub-region 1 is stored. Addresses 3, 4, and 5 of BRAM1 store the x, y, and z information of the first point in sub-region 1, and addresses 3, 4, and 5 of BRAM2 store the x, y, and z information of the second point in sub-region 1. This process is repeated to complete the storage of all sub-region points of the target point cloud in the corresponding n BRAM addresses.

[0057] Furthermore, the sub-region labeling module includes: multiple comparators and multiple binary selectors; the comparators are used to compare the x, y, and z values ​​of the input source point cloud points with the tree nodes of the KD Tree topology structure of the target point cloud data; the binary selectors are used to output 1 or 0 based on the comparison result of the comparators, and the outputs of the binary selectors are concatenated from high to low bits to obtain the sub-region label for the source point cloud points to perform nearest point search, and the sub-region label is used to distinguish different sub-regions.

[0058] Specifically, please see Figure 8 As shown, Figure 8 This is a circuit diagram of a sub-region labeling module provided in an embodiment of this application. The sub-region labeling module is used to calculate which sub-region of the KD Tree topology of the target point cloud data the input source point cloud points enter for searching. Please refer to... Figure 9 As shown, Figure 9 This is a diagram illustrating the working mechanism of the sub-region marker module provided in an embodiment of this application. Figure 5 Taking the four sub-regions obtained from the two-level sorting results as an example, the input source point cloud is (8,11,3). The tree node in the x-dimensional of the first layer is 4, and the tree nodes on the left and right sides in the y-dimensional of the second layer are 7 and 15. First, the value in the x-dimensional of the input source point cloud is compared with the tree node in the first layer. 8>4, comparator 1_1 outputs 0, the high-order bit Add_r[1]=0, and the D0 end of the two-to-one selector is selected. Then, the value in the y-dimensional of the source point cloud is compared with the tree nodes on the left and right sides in the second layer. 11>7, comparator 2_1 outputs 0. 15>11, comparator 2_2 outputs 1. Since the selected end of the two-to-one multiplexer is the D0 port, Add_r[0] is 1. The corresponding sub-region flag is 01, i.e., sub-region 1, obtained by splicing from the high-order bit to the low-order bit. Finally, the source point cloud (8,11,3) enters sub-region 1 to search for the nearest point.

[0059] After calculation by the sub-region flag module, the input source point cloud points and tree nodes are calculated level by level. Based on the concatenation of high-bit to low-bit values, an m-bit sub-region flag is finally obtained, which can represent 0 to (2m-1) sub-regions. Finally, multiplying this sub-region flag by 3 yields the result as shown below. Figure 9 The initial address of the BRAM shown is used to read the data. This allows entry into the corresponding BRAM to read all target point cloud information in the corresponding sub-region, calculate and compare the distance between the source point cloud and the target point cloud, and find the closest point of the source point cloud in the target point cloud.

[0060] Furthermore, the distance calculation module is specifically used to calculate the Euclidean distance between the input source point cloud and each target point cloud in the corresponding sub-region based on the DSP module in the FPGA, to obtain the target point cloud closest to the source point cloud and store it in the DDR memory of the FPGA.

[0061] Specifically, after determining the sub-region marker, the distance calculation module reads all target point cloud information from the selected sub-region in the corresponding BRAM address. For each target point cloud, Euclidean algorithm is performed between it and the input source point cloud to find its nearest neighbor. Finally, the calculation result is returned and stored in DDR memory.

[0062] This module utilizes the DSP resources within the FPGA to perform calculations. The FPGA's DSP contains computational units such as multipliers, adders, and subtractors, which can be used to calculate the Euclidean distance between two points. Since the Euclidean distance involves a square root operation, the FPGA cannot perform this calculation and comparison; therefore, the square of the Euclidean distance is used to compare the distance between the two points.

[0063] The subtractor sequentially calculates the differences in the three dimensions (x, y, z) of two points, and the results are input into the DSP to calculate the Euclidean distance d. To retain the nearest point information in DDR memory, the nearest point information and the distance d are bit-concatenated (the high 48 bits represent the target point information, and the low 24 bits represent the Euclidean distance). Subsequent pipelined comparisons are performed only on the low-order Euclidean distance bits. A corresponding number of nearest points can be obtained within a certain period. Finally, the calculated nearest points are returned to the DDR memory, completing the hardware-accelerated nearest neighbor search process.

[0064] Understandably, the point cloud registration hardware acceleration method for spatial registration in orthopedic surgical navigation proposed in this invention overcomes the high cost of optical surgical navigation and the high overall hardware cost caused by the CPU or GPU computation of registration algorithms in existing technologies. It uses an electromagnetic positioning system to collect registration point information and an FPGA to perform registration algorithm calculations, thus significantly reducing hardware development costs. Furthermore, by combining PCA coarse registration and ICP fine registration, the method ensures the required registration accuracy for medical surgery. Leveraging the pipelined parallel processing advantages of FPGA, the KD Tree topology data structure construction and nearest neighbor search process in ICP registration are executed on the FPGA for hardware acceleration, resulting in a significant reduction in overall registration time. Ultimately, this achieves higher accuracy and less time consumption in registration.

[0065] It should be noted that in this invention, the number of point cloud data and FPGA storage modules or data input ports should be as much as possible as a power of 2, so that the FPGA can process the data faster.

[0066] Furthermore, this application embodiment also provides specific experimental examples such as... Figure 1 A hardware acceleration method for point cloud registration.

[0067] To meet hardware acceleration requirements, the point cloud data and target point cloud data from the total knee replacement surgery are first downsampled, resulting in 16,384 target point cloud points and 4,096 source point cloud points. The sorting module in the tree topology data module of step S4 has 64 data input ports. The ordered point sequence cache module uses 32 BRAMs with an address length of 1024 and a data width of 48 bits, and 8 BRAMs with an address length of 4096 and a data width of 48 bits. The nearest neighbor search module is divided into *a* sub-regions, each containing *b* target point cloud points, where *a* = 256 and *b* = 128. The target point cloud data structure storage module consists of *n* BRAMs, where *n* = 128, and can store a maximum point cloud size of 32,768 points. The sub-region labeling module calculates *m* sub-region labels, where *m* = 8, corresponding to 256 sub-regions.

[0068] Please see Figure 10 As shown, Figure 10 This image shows the registration effect after implementing the hardware acceleration method for point cloud registration provided in this embodiment. Green represents the target point cloud, and red represents the registered point cloud after transformation by the source point cloud using the transformation matrix. The target point cloud contains 16384 points, and the source point cloud data obtained from the surgical registration contains 4096 points. The average distance error in the registration accuracy is 1.089 mm, and the time consumed is 58.7 seconds. The registration time before acceleration is 153.51 seconds, which is 2.6 times the registration time after acceleration.

[0069] It is evident that by applying the hardware acceleration method for point cloud registration in this application, the registration time can be successfully reduced while maintaining the required registration accuracy; and the registration time can be reduced exponentially as the size of the point cloud increases.

[0070] Please see Figure 11 As shown, Figure 11 This is a structural diagram of a point cloud registration hardware acceleration system for orthopedic surgical navigation provided in an embodiment of this application. The system is used to implement, for example... Figure 1The hardware acceleration method for point cloud registration for orthopedic surgical navigation shown includes: a data acquisition unit for acquiring bone point cloud data actually registered in the electromagnetic positioning system; a data import unit for importing bone point cloud data into the DDR memory of the FPGA; a coarse registration unit for performing coarse point cloud registration on the bone point cloud data based on the FPGA's DSP module, where coarse point cloud registration refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud; and a fine registration acceleration unit for performing fine point cloud registration based on the initial transformation matrix, where fine point cloud registration refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud. In the ICP registration algorithm, the KD Tree point cloud data topology construction and nearest neighbor search are executed by the tree topology data module and nearest neighbor search module at the PL end of the FPGA, respectively, while the remaining steps are executed by the embedded CPU at the PS end of the FPGA. The constructed target point cloud KD Tree topology and the target point cloud closest to the source point cloud are stored in the FPGA's DDR memory.

[0071] It is understood that the point cloud registration hardware acceleration system for orthopedic surgical navigation in this application is used to achieve, for example... Figure 1 The point cloud registration hardware acceleration method corresponds one-to-one with the method and has the corresponding technical effect, so it will not be elaborated further.

[0072] This application also provides a computer program product that, when run on a mobile terminal, causes the mobile terminal to perform actions such as... Figure 1 The illustrated point cloud registration hardware acceleration method for orthopedic surgical navigation is shown.

[0073] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A hardware acceleration method for point cloud registration for orthopedic surgical navigation, characterized in that, include: S1. Obtain the actual registered bone point cloud data in the electromagnetic positioning system; S2. Import the bone point cloud data into the FPGA's DDR memory; S3. The FPGA-based DSP module performs coarse registration of the bone point cloud data. The coarse registration refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud. S4. Perform fine registration of the point cloud based on the initial transformation matrix. Fine registration of the point cloud refers to executing the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud. In the ICP registration algorithm, the construction of the KD Tree point cloud data topology and the search for the nearest neighbor are executed by the tree topology data module and the nearest neighbor search module on the PL side of the FPGA, respectively. The remaining steps are executed by the embedded CPU on the PS side of the FPGA. The constructed target point cloud data KD Tree topology and the target point cloud that is closest to the source point cloud are stored in the DDR memory of the FPGA.

2. The point cloud registration hardware acceleration method for orthopedic surgical navigation according to claim 1, characterized in that, The tree topology data module includes: The sorting module is used to repeatedly sort the input point cloud data according to the three dimensions of x, y, and z based on the buffer and comparison unit to obtain an ordered point sequence. The median of the ordered point sequence is selected as the tree node to complete the construction of the tree topology. The ordered point sequence cache module is used to store the ordered point sequence after each round of sorting in the BRAM of the FPGA.

3. The point cloud registration hardware acceleration method for orthopedic surgical navigation according to claim 2, characterized in that, The sorting module includes: N data input ports and one data output port, where N is an even number; N data input ports are connected to N buffers. Every two buffers are connected to a comparison unit, and each comparison unit is connected to a buffer. The buffers and comparison units are connected alternately until the last buffer is connected to the data output port.

4. The point cloud registration hardware acceleration method for orthopedic surgical navigation according to claim 2, characterized in that, In the ordered point sequence cache module, the number and depth of BRAMs are both exponential multiples of 2.

5. The point cloud registration hardware acceleration method for orthopedic surgical navigation according to claim 1, characterized in that, The nearest neighbor search module includes: The target point cloud data structure storage module is used to store the constructed target point cloud data KD Tree topology structure. The target point cloud data KD Tree topology structure is divided into multiple sub-regions according to the sub-sequence of the tree topology structure, and each sub-region contains multiple target point cloud points. The sub-region labeling module is used to determine the sub-region for nearest point search of the input source point cloud; The distance calculation module is used to calculate the Euclidean distance between the source point cloud and the target point cloud in the sub-region.

6. The point cloud registration hardware acceleration method for orthopedic surgical navigation according to claim 5, characterized in that, The target point cloud data structure storage module consists of n BRAMs, and the same set of addresses of the n BRAMs stores the x, y, and z information of all target point cloud points in the same sub-region.

7. The point cloud registration hardware acceleration method for orthopedic surgical navigation according to claim 5, characterized in that, The sub-region marking module includes: multiple comparators and multiple two-to-one selectors; The comparator is used to compare the x, y, and z values ​​of the input source point cloud points with the tree nodes of the KD Tree topology structure of the target point cloud data. The two-to-one selector is used to output 1 or 0 according to the comparison result of the comparator. The output of the two-to-one selector is concatenated from the high bit to the low bit to obtain the sub-region flag for the source point cloud to find the nearest point. The sub-region flag is used to distinguish different sub-regions.

8. The point cloud registration hardware acceleration method for orthopedic surgical navigation according to claim 5, characterized in that, The distance calculation module is specifically used to calculate the Euclidean distance between the input source point cloud and each target point cloud in the corresponding sub-region based on the DSP module in the FPGA, to obtain the target point cloud closest to the source point cloud and store it in the DDR memory of the FPGA.

9. A point cloud registration hardware acceleration system for orthopedic surgical navigation, characterized in that, For performing the point cloud registration hardware acceleration method as described in any one of claims 1-8, including: The data acquisition unit is used to acquire the actual registered bone point cloud data in the electromagnetic positioning system; The data import unit is used to import bone point cloud data into the FPGA's DDR memory; The coarse registration unit is used by the FPGA-based DSP module to perform coarse registration of the bone point cloud data. The coarse registration refers to introducing PCA to calculate the principal axis direction of the bone point cloud data to obtain the initial transformation matrix of the source point cloud and the target point cloud. The fine registration acceleration unit is used to perform fine registration of point clouds based on the initial transformation matrix. The fine registration of point clouds refers to the execution of the ICP registration algorithm to obtain the optimal transformation matrix of the source point cloud and the target point cloud. In the ICP registration algorithm, the construction of the K-DTree point cloud data topology structure and the search for nearest neighbors are executed by the tree topology data module and the search for nearest neighbors on the PL side of the FPGA, respectively. The remaining steps are executed by the embedded CPU on the PS side of the FPGA. The constructed target point cloud data KD Tree topology structure and the target point cloud that is closest to the source point cloud are stored in the DDR memory of the FPGA.

10. A computer program product, characterized in that, When the computer program product is run on a mobile terminal, the mobile terminal performs the point cloud registration hardware acceleration method as described in any one of claims 1-8.

Images