Three-dimensional tomographic image stitching method, system and terminal device

Abstract

This invention provides a method, system, and terminal device for stitching three-dimensional tomographic images. The method includes: acquiring first and second three-dimensional tomographic image data; acquiring the stitching direction of the first and second three-dimensional tomographic image data; acquiring a first registration region of the first three-dimensional tomographic image data and a second registration region of the second three-dimensional tomographic image data; registering the first registration region to the second registration region based on an initial registration matrix to obtain a first registration matrix; obtaining a rotation matrix acting on the first three-dimensional tomographic image data based on the initial registration matrix and the first registration matrix; applying the rotation matrix to the first three-dimensional tomographic image data to obtain third three-dimensional tomographic image data and a third registration region; and fusing and stitching the third and second three-dimensional tomographic image data and the third and second registration regions to obtain a fused and stitched three-dimensional image of the affected area. This method achieves three-dimensional tomographic image stitching and improves the computational efficiency of tomographic image stitching.

Description

Technical Field

[0001] This invention relates to the field of image stitching, and more particularly to a method, system, and terminal device for stitching three-dimensional tomographic images. Background Technology

[0002] With the continuous development of medical conditions and technology, 3D images can help doctors observe and analyze patients' anatomical structures more intuitively, providing a more comprehensive perspective to help doctors discover details that are difficult to observe in 2D images during diagnosis. People have raised new requirements for 3D medical images, such as precision and digitization. To solve the problem of stitching tomographic image data, current solutions mainly include extracting a specific layer of tomographic image data from a 3D tomographic image, then registering it using 2D registration methods, and then connecting the upper and lower layers of that layer; or using point cloud information from a 3D meshed image for registration to achieve accurate repositioning and stitching. However, extracting a specific layer of tomographic image data from a 3D tomographic image only considers rotation and translation around the Z-axis; and using point cloud information from a 3D meshed image for registration involves cumbersome calculations, requires special tools to process the point cloud data, does not follow the DICOM protocol commonly used in the medical field, and requires additional tools for clinical use. Summary of the Invention

[0003] The purpose of this invention is to provide a three-dimensional tomographic image stitching method, system, and terminal device, which can stitch together multiple sets of three-dimensional tomographic data to obtain a complete human body data, enabling the fusion of multi-layer tomographic image data, simplifying the image registration calculation process, and facilitating doctors to observe the overall structure of the human body or the stress on implants.

[0004] The technical solution provided by this invention is as follows:

[0005] This invention provides a method for stitching three-dimensional tomographic images, comprising:

[0006] From a database of three-dimensional tomographic images obtained from different parts of the target object, acquire the first and second three-dimensional tomographic image data of a certain part that needs to be stitched together.

[0007] In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is obtained.

[0008] Acquire the first registration region of the first three-dimensional tomographic image data in the stitching direction and the second registration region of the second three-dimensional tomographic image data in the stitching direction.

[0009] Based on the initial registration matrix, the first registration region is registered to the second registration region to obtain the first registration matrix.

[0010] Based on the initial registration matrix and the first registration matrix, the rotation matrix acting on the first three-dimensional tomographic image data is obtained.

[0011] Applying the rotation matrix to the first three-dimensional tomographic image data yields the third three-dimensional tomographic image data and the third registration region.

[0012] The third 3D tomographic image data, the third registration region, the second registration region, and the second 3D tomographic image data are fused and stitched together to obtain a fused and stitched 3D image of the part.

[0013] Furthermore, based on the initial registration matrix and the first registration matrix, a rotation matrix acting on the first three-dimensional tomographic image data is obtained, including:

[0014] Comparative analysis was performed on the SSIM values ​​of the initial registration matrix and the first registration matrix applied to the first registration region and the second registration region.

[0015] The registration matrix corresponding to the larger SSIM value among the two SSIM values ​​is used as the base matrix.

[0016] Based on the displacement of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration region and the second registration region, as well as the fundamental matrix, the rotation matrix is ​​obtained.

[0017] Furthermore, in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is obtained, including:

[0018] Based on the scanning directions of the first and second three-dimensional tomographic image data, the stitching direction of the first and second three-dimensional tomographic image data is set.

[0019] Furthermore, in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is obtained, including:

[0020] The first three-dimensional tomographic images in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic images in each direction of the second three-dimensional tomographic image data are registered pairwise to obtain multiple registration matrices.

[0021] Analyze the SSIM index of each of the multiple registration regions, and take the registration direction of the region corresponding to the maximum value of the SSIM index as the stitching direction.

[0022] Further, acquiring the first registration region of the first three-dimensional tomographic image data in the stitching direction and the second registration region of the second three-dimensional tomographic image data in the stitching direction includes:

[0023] Based on the stitching direction, the number of overlapping images in the first and second three-dimensional tomographic image data is obtained.

[0024] The first registration region is obtained from the first three-dimensional tomographic image data based on the number of overlapping images, and the second registration region is obtained from the second three-dimensional tomographic image data.

[0025] Furthermore, before registering the first and second registration regions based on the initial registration matrix to obtain the first registration matrix, the process also includes:

[0026] The target phantom used in the scanning geometric correction is used to register the three-dimensional tomographic image data of the target phantom to obtain an initial registration matrix.

[0027] Furthermore, before registering the first and second registration regions based on the initial registration matrix to obtain the first registration matrix, the process also includes:

[0028] The first registration region and the second registration region are downsampled by a preset factor to obtain the first downsampled registration region and the second downsampled registration region.

[0029] The downsampling transformation matrix is ​​obtained by registering the first downsampling registration region to the second downsampling registration region.

[0030] The initial registration matrix is ​​obtained based on the downsampling transformation matrix and the preset downsampling factor.

[0031] Furthermore, the formula for calculating the rotation matrix is:

[0032]

[0033] Where T is the rotation matrix, T max The basic matrix, t offset It refers to the displacement of the first three-dimensional tomographic image data, the second three-dimensional tomographic image data to the first registration region, and the second registration region.

[0034] t offset The calculation formula is:

[0035]

[0036] Where n is the number of registration layers.

[0037] Furthermore, the third 3D tomographic image data, the third registration region, the second registration region, and the second 3D tomographic image data are fused and stitched together to obtain a fused and stitched 3D image of the affected area, including:

[0038] The third registration region and the second registration region are merged to obtain the registration fused region.

[0039] The third 3D tomographic image data (with the third registration region removed), the registration fusion region, and the second 3D tomographic image data (with the second registration region removed) are fused and stitched together to obtain the fused and stitched 3D image of the part.

[0040] Furthermore, the third registration region and the second registration region are merged to obtain the registration fusion region, including:

[0041] Its nonlinear fusion formula is:

[0042]

[0043] Among them, b' n N is the registration and fusion region, and N is the third registration region b'. 1n The total number of sheets, where n is the nth sheet in the total number of sheets, b' 1n For the third registration region, b 2n This is the second registration region.

[0044] Furthermore, the third registration region and the second registration region are merged to obtain the registration fusion region, including:

[0045] Its linear fusion formula is:

[0046] b' n =0.5*b' 1n +0.5*b 2n ;

[0047] Among them, b' n To register and integrate the region, b' 1n For the third registration region, b 2n This is the second registration region.

[0048] The present invention also provides a terminal device, comprising:

[0049] A processor is used for stitching three-dimensional tomographic image data, which calls instructions stored in a memory. When the instructions are executed, the processor performs a three-dimensional tomographic image stitching method.

[0050] The present invention also provides a three-dimensional tomographic image stitching system, comprising:

[0051] A terminal scanning device is used to scan different parts of a target object and send the obtained three-dimensional tomographic image data to the terminal device.

[0052] A terminal device, including a processor, is configured to invoke instructions stored in a memory, which, when executed, cause the processor to perform a tomographic image stitching method.

[0053] The three-dimensional tomographic image stitching method, system, and terminal device provided by this invention, based on the initial registration matrix, registers the first registration region to the second registration region to obtain the first registration matrix. The rotation matrix and the third registration region are obtained from the first registration matrix. The third registration region and the second registration region are fused to obtain the registration fusion region. This improves the computational efficiency of tomographic image stitching and can stitch three-dimensional tomographic image data from different parts into a complete three-dimensional human body data, realizing the fusion of multi-layer tomographic image data, which is convenient for doctors to observe the overall structure of the human body or the stress of implants. Attached Figure Description

[0054] The preferred embodiments will be described below in a clear and easy-to-understand manner, with reference to the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of a three-dimensional tomographic image stitching method, system, and terminal device.

[0055] Figure 1 This is a schematic diagram of the structure of an embodiment of a three-dimensional tomographic image stitching system of the present invention;

[0056] Figure 2 This is a flowchart of an embodiment of a three-dimensional tomographic image stitching method according to the present invention;

[0057] Figure 3 This is a flowchart of another embodiment of a three-dimensional tomographic image stitching method of the present invention;

[0058] Figure 4 This is a flowchart of another embodiment of a three-dimensional tomographic image stitching method of the present invention;

[0059] Figure 5 This is a schematic diagram of a tomographic image of the present invention;

[0060] Figure 6 This is a schematic diagram of another tomographic image of the present invention;

[0061] Figure 7 This is a schematic diagram of the left-right stitching of tomographic images according to the present invention;

[0062] Figure 8 This is the first three-dimensional tomographic image of the tomographic image of the present invention;

[0063] Figure 9 This is the second three-dimensional tomographic image of the tomographic image of the present invention;

[0064] Figure 10 This is a schematic diagram of stitching together tomographic images from top to bottom according to the present invention;

[0065] Figure 11 This is the first registration region of the tomographic image of the present invention;

[0066] Figure 12 This is the second registration region of the tomographic image of the present invention;

[0067] Figure 13 This is the first registration region after downsampling of the tomographic image in this invention;

[0068] Figure 14 This is the second registration region after downsampling of the tomographic image in this invention;

[0069] Figure 15 This is the first registration region after the tomographic image is downsampled and the transformation matrix is ​​applied;

[0070] Figure 16 This invention provides a third three-dimensional tomographic image with a rotation matrix applied.

[0071] Figure 17 This is the registration area of ​​the third three-dimensional tomographic image in the tomographic image of this invention;

[0072] Figure 18 It is the registration fusion region after fusing the registration region of the third three-dimensional tomographic image and the second registration region in the tomographic image of the present invention. Detailed Implementation

[0073] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0074] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or sets.

[0075] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0076] Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0078] Three-dimensional tomographic image stitching is a medical imaging technique that involves combining a series of two-dimensional tomographic image data (such as CT or MRI scans) into a three-dimensional model. Key technical aspects of three-dimensional tomographic image stitching include: image acquisition, image preprocessing, image registration, and image reconstruction.

[0079] Image acquisition involves obtaining a series of continuous two-dimensional tomographic images of the patient through CT (computed tomography) or MRI (magnetic resonance imaging) scans. Image preprocessing includes denoising, contrast enhancement, and edge enhancement to improve image quality. Image registration involves registering all two-dimensional images to a common coordinate system to ensure spatial alignment between images, which involves transformations such as translation and rotation. Image reconstruction uses three-dimensional reconstruction algorithms, such as voxel reconstruction or surface reconstruction, to convert the registered two-dimensional images into a three-dimensional model.

[0080] In the current technology, in order to solve the problem of tomographic image stitching, the main solutions are to extract the tomographic image data of a certain layer of the three-dimensional tomographic image data, register it using the two-dimensional registration method, and then connect the upper and lower layers of this layer; or to use the point cloud information of the three-dimensional mesh image for registration, so as to achieve accurate image restoration and stitching.

[0081] However, when using two-dimensional registration, this method only considers rotation and translation around the Z-axis. Therefore, it is necessary when the patient cannot move during the imaging process, and it also requires high precision from the imaging equipment. When using point cloud information for registration, the medical tomographic image data may contain a large number of artifacts and noise, and point cloud computing is time-consuming and affects the results. For these reasons, this method is limited to bone registration. In CBCT (cone-beam computed tomography) images, bones and soft tissues cannot be completely distinguished, making this approach difficult to promote.

[0082] Therefore, this invention provides a three-dimensional tomographic image stitching system. By acquiring two sets of three-dimensional tomographic image data, two registration regions are obtained from these two sets of data. An initial registration matrix is ​​obtained under rigid registration. The overlapping region is registered based on the initial matrix. Then, the two registered parts are fused using a nonlinear or linear fusion method to obtain a stitched three-dimensional image of the target area. The image stitched by this three-dimensional tomographic image stitching system has improved accuracy compared to two-dimensional registration methods, and improved speed and applicability compared to point cloud computing. It enables the fusion of multi-layer tomographic image data, facilitating doctors' observation of the overall structure of the human body or the stress on implants.

[0083] The three-dimensional tomographic image data stitching system of this invention includes a terminal scanning device and a terminal device. It can acquire images based on a mobile three-dimensional C-arm device. The mobile three-dimensional C-arm can provide multi-angle surgical diagnostic information to assist doctors in intraoperative assessments, such as fracture reduction and the size and position of implanted screws. The terminal scanning device captures images of body parts obtained through X-ray scanning via its scanning panel. During scanning, the scanning panel keeps the subject centered on the X-ray beam, reducing motion artifacts.

[0084] When the scanning device scans a patient, the machine scans different parts of the patient's body from different directions. After multiple scans, the images are sent to a terminal device, which can be a dedicated imaging terminal device. The imaging terminal device can receive the scanned images and analyze and stitch them together based on the three-dimensional tomographic image stitching method of this invention to obtain a complete three-dimensional image of the target object.

[0085] The three-dimensional tomographic image data stitching system of the present invention provides real-time high-definition perspective images and digital image sequence photography through a terminal scanning device. The obtained images are sent to a dedicated imaging terminal device. The imaging terminal device uses a three-dimensional tomographic image stitching method to register and stitch three-dimensional tomographic images of any direction and size. Multiple sets of three-dimensional tomographic data are stitched together to obtain a complete human body data. This not only realizes the fusion of multi-layer tomographic image data, but also simplifies the image registration calculation process, making it convenient for doctors to observe the overall structure of the human body for analysis and diagnosis.

[0086] The following description is in conjunction with the accompanying drawings:

[0087] In one embodiment of the present invention, such as Figure 1 As shown, a three-dimensional tomographic image stitching system 100 includes:

[0088] The terminal scanning device 110 is used to scan different parts of the target object and send the obtained three-dimensional tomographic image data to the terminal device 120.

[0089] Terminal device 120 includes a processor for calling instructions stored in a memory, wherein when the instructions are executed, the processor performs a three-dimensional tomographic image stitching method.

[0090] Specifically, the 3D tomographic image stitching system 100 is an advanced technology system specifically designed for processing and analyzing medical image data. The 3D tomographic image stitching system 100 includes a terminal scanning device 110 and a terminal device 120.

[0091] The terminal scanning device 110 can be a mobile 3D C-arm, an advanced medical imaging device typically used in operating rooms to provide high-quality real-time images. This device is designed to move between different operating rooms to accommodate varying surgical needs. A mobile 3D C-arm typically includes: a C-arm structure for stabilizing the device and allowing it to move in space; a flat panel detector for receiving X-rays after they pass through the patient's body and converting them into digital images; and a X-ray source for generating X-rays to image the patient.

[0092] A flat panel detector scans different parts of the patient from different directions, acquiring three-dimensional tomographic images of those parts. The acquired images contain overlapping portions between each pair of parts. The images scanned by the flat panel detector in terminal scanning device 110 are sent to terminal device 120 for analysis and processing.

[0093] Terminal device 120 can be a dedicated image terminal device, a 3D reconstruction workstation, etc. It is responsible for receiving the 3D tomographic images scanned by the flat panel detector in terminal scanning device 110, fusing and stitching multiple sets of 3D tomographic images through a 3D tomographic image stitching method, and displaying the obtained complete 3D human body image on the display panel of terminal device 120.

[0094] Under existing technologies, 3D reconstruction requires processing a large amount of image data, resulting in high computational load, long processing time, and a lack of multi-view image information processing. The 3D tomographic image stitching system of this invention obtains high-resolution perspective images and digitized image sequences through scanning. Based on these multi-view image data, 3D reconstruction is performed to construct a more complete 3D image.

[0095] The method for stitching three-dimensional tomographic images is described below with reference to the accompanying drawings.

[0096] In one embodiment of the present invention, please refer to Figure 2 This illustration shows a flowchart of a three-dimensional tomographic image stitching method provided in some embodiments of this disclosure. The method is executed by a terminal device 120 (i.e., an image terminal device) and includes at least the following steps:

[0097] S100 obtains the first and second three-dimensional tomographic image data of a certain part that needs to be stitched together from the three-dimensional tomographic image database obtained from different parts of the scanned target object.

[0098] S200 obtains the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data.

[0099] S300 acquires the first registration area of ​​the first three-dimensional tomographic image data in the stitching direction and the second registration area of ​​the second three-dimensional tomographic image data in the stitching direction.

[0100] S400 registers the first registration region to the second registration region based on the initial registration matrix to obtain the first registration matrix.

[0101] S500 obtains the rotation matrix acting on the first three-dimensional tomographic image data based on the initial registration matrix and the first registration matrix.

[0102] S600 applies a rotation matrix to the first three-dimensional tomographic image data to obtain the third three-dimensional tomographic image data and the third registration region.

[0103] S700 fuses and stitches together the third three-dimensional tomographic image data, the third registration region, the second registration region, and the second three-dimensional tomographic image data to obtain a three-dimensional image of the fused and stitched part.

[0104] Specifically, the terminal scanning device 110 (also known as the mobile three-dimensional C-arm) can scan human body information from multiple angles to assist doctors in preoperative, intraoperative, and postoperative assessments. During the scanning process, it can keep the subject in the center of the X-ray beam, reducing motion artifacts. The terminal scanning device 110 obtains real-time high-definition fluoroscopic images and digital image sequences through its flat panel detector.

[0105] The terminal device receives the projection scanned by the flat panel detector of the terminal scanning device, and reconstructs three-dimensional tomographic image data B1, B2, ... B based on the projection. N The acquired images contain some overlap between each pair of images. From the three-dimensional tomographic image data B1, B2, ... B... N The system acquires the first and second 3D tomographic image data that need to be stitched together. For example, to obtain complete 3D human body data (when the left and right sides are not fully captured), the person stands still, and the machine captures images of the left half of the body A1, A2, ... A from top to bottom. N Shoot the right half of the body from top to bottom: B1, B2, ... B N .like Figure 5 The image shown is a schematic diagram of the left half, A1, of a tomographic image. Figure 6The diagram shown is a schematic representation of the right half (B1) of a tomographic image. Figure 7 The diagram shows the left half A1 and the right half B1 joined together. It can be seen that each body segment is first joined from left to right: A1 is joined to B1 to obtain C1, and A2 is joined to B2 to obtain C2. Figure 8 The diagram shown is a schematic representation of the upper half of a tomographic image, C1. Figure 9 The diagram shown is a schematic representation of the lower half of a tomographic image, C2. Figure 10 The diagram shows a composite image of the upper half (C1) and lower half (C2) of a tomographic image. If C1 is stitched to C2 from top to bottom, C1 represents the first three-dimensional tomographic image data, C2 represents the second three-dimensional tomographic image data, and so on.

[0106] Step S200 involves obtaining the stitching direction of the first and second 3D tomographic image data in various directions. There are multiple methods for this. The scanning direction of the scanning terminal device can be set as the stitching direction, or the stitching direction can be obtained by analyzing the maximum SSIM exponent of each registration matrix in multiple registration matrices.

[0107] Specifically, for example, when scanning a human body, the scanning direction of the flat panel detector of the mobile three-dimensional C-arm is the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data.

[0108] If there are different scanning scenarios, such as stitching together tomographic images taken before surgery and at two different time periods, the scanning direction of the images is unclear. In this case, it is necessary to perform pairwise registration of the first three-dimensional tomographic images in each direction (such as up, down, left, right, front, and back) and the second three-dimensional tomographic images in each direction (such as up, down, left, right, front, and back) in the first three-dimensional tomographic image data and the second three-dimensional tomographic images in each direction (such as up, down, left, right, front, and back) in the second three-dimensional tomographic image data to obtain multiple registration matrices. Then, the SSIM index of each registration region in the multiple registration regions is analyzed, and the registration direction in the registration region corresponding to the maximum value of the SSIM index is taken as the stitching direction.

[0109] After obtaining the stitching direction of the 3D tomographic images, the first and second registration regions along the stitching direction are obtained based on the first and second 3D tomographic image data. The 3D data has six faces (e.g., data of size 512*512*512). B1 is the first 3D tomographic image data, and B2 is the second 3D tomographic image data. The first image containing soft tissue is located in B1 and B2 from the center of each of the six faces. Figure 11 and Figure 12The image shows the first registration region b1 and the second registration region b2. Starting from the first image, the number of images to be registered is taken to obtain the first registration region b1 (top, bottom, left, right, front, and back) and the second registration region b2 (top, bottom, left, right, front, and back), for a total of 12 sets of data. The registration region needs to be an image where the non-zero part is greater than 80%. If the position where the non-zero part is greater than 80% is the 20th image, then several images are taken from the 20th image towards the center to form the data to be registered.

[0110] The number of registration frames is usually set manually and needs to be greater than the number of overlap frames. Assuming the height of the planar detector is H1 and the distance moved between the two images is D1, then the height of the overlapped image is H1-D1, the number of reconstructed images in the scanning direction is M, and the number of overlap frames is (H1-D1) / H1*M.

[0111] Based on the initial registration matrix T0 obtained by rigid registration, the first registration region b1 and the second registration region b2 are registered to obtain the first registration matrix T1. For example, if the first registration region is b1 and the second registration region is b2, and the stitching direction is from b1 to b2, the first registration region b1 is registered to the second registration region b2 based on the initial registration matrix T0 to obtain the first registration matrix T1.

[0112] Based on the initial registration matrix T0 and the first registration matrix T1, the rotation matrix T acting on the first three-dimensional tomographic image data B1 is obtained. For example... Figure 16 As shown, applying the rotation matrix T to the first three-dimensional tomographic image data B1 yields the third three-dimensional tomographic image data B'1. However, for faster calculation speed, when mechanical precision is high and human cooperation is good, the initial registration matrix T0 can be used directly instead of the rotation matrix T for calculation.

[0113] The third 3D tomographic image data B'1, the third registration region b'1, the second registration region b2, and the second 3D tomographic image data B2 are fused and stitched together to obtain a fused 3D image of the affected area. Specifically, the third registration region b'1 and the second registration region b2 are fused to obtain the fused registration region b', where, as... Figure 17 The image shown is the third registration region b'1 of the third three-dimensional tomographic image B'1, as follows: Figure 18 The image shows the registration fusion region b' after the third registration region b'1 and the second registration region b2 are fused. The third three-dimensional tomographic image data B'1 is fused by removing the third registration region b'1 and the registration fusion region b', and the second three-dimensional tomographic image data B2 is fused by removing the second registration region b2, to obtain the three-dimensional image of the fused part.

[0114] When fusing and stitching three-dimensional tomographic image data, the above step S700 also includes: nonlinear fusion or linear fusion.

[0115] Specifically, assuming that the third registration region b'1 is nonlinearly fused to the second registration region b2, and the third registration region b'1 has N layers, the nonlinear fusion formula is:

[0116]

[0117] Among them, b' n N is the registration and fusion region, and N is the third registration region b'. 1n The total number of sheets, where n is the nth sheet in the total number of sheets, b' 1n For the third registration region, b 2n This is the second registration region.

[0118] Another method of fusion and splicing is linear, which will be explained below.

[0119] Specifically, assuming that the third registration region b'1 is linearly fused to the second registration region b2, for example, by summing 50% of each registration region, the linear fusion formula is:

[0120] b' n =0.5*b' 1n +0.5*b 2n ;

[0121] Among them, b' n To register and integrate the region, b' 1n For the third registration region, b 2n This is the second registration region.

[0122] The three-dimensional tomographic image stitching method of this application extracts the overlapping regions of three-dimensional tomographic data, registers the overlapping regions based on the initial matrix, and then fuses the tomographic images through a nonlinear fusion method or a linear fusion method, thereby improving the computational efficiency of image registration and realizing the fusion of multi-layer three-dimensional tomographic image data.

[0123] In one embodiment of the present invention, before registering the first registration region b1 to the second registration region b2 to obtain the first registration matrix T1, it is necessary to obtain an initial registration matrix T0. There are two main ways to obtain the initial registration matrix T0. One of these methods will now be described, such as... Figure 3 As shown, step S300 above may include:

[0124] S310 downsamples the first registration region and the second registration region by a preset factor to obtain the first downsampled registration region and the second downsampled registration region.

[0125] The S320 first downsampling registration region is registered to the second downsampling registration region to obtain the downsampling transformation matrix.

[0126] S330 obtains the initial registration matrix based on the downsampling transformation matrix and the preset downsampling factor.

[0127] Specifically, the registration regions b1 and b2 are downsampled by a factor of X to obtain b. down1 and b down2 ,like Figure 13 and Figure 14 The image shows the result of downsampling the first registration region b1 and the second registration region b2. down1 and b down2 .

[0128] If the splicing direction is from the first registration region b1 to the second registration region b2, then b down1 To b down2 Registration yields the downsampling transformation matrix t1. Theoretically, the resulting downsampling transformation matrix t1 acts on b. down1 The result on will be the same as b down2 Overlap. For example... Figure 15 The image shows the first registration region after downsampling, where the transformation matrix t1 has been applied.

[0129] Assuming the downsampling transformation matrix t1 is a 4x4 rotation matrix, its form is:

[0130]

[0131] Where R is the 3*3 first rotation matrix, and t is the displacement matrix of the first downsampling registration region and the second downsampling registration region.

[0132] The formula for calculating the initial registration matrix T0 is:

[0133]

[0134] Where R is the 3*3 first rotation matrix, t is the displacement matrix of the first downsampling registration region and the second downsampling registration region, and X is the preset downsampling factor.

[0135] In this embodiment, there is another method for obtaining the initial registration matrix, which includes: registering the three-dimensional tomographic image data of the target phantom used during scanning geometric correction to obtain the initial registration matrix T0.

[0136] Specifically, an initial registration matrix T0 is obtained using rigid registration (the initial registration matrix is ​​obtained using a phantom during geometric correction). For a mobile 3D C-arm scanning device, a phantom used for geometric correction can be scanned first. Then, the 3D data of the scanned phantom is registered (at this time, the initial registration matrix T0 is 0), and the phantom registration matrix is ​​saved; this matrix is ​​T0. When scanning a patient, the patient is scanned along the track used when scanning the phantom, and the initial registration matrix T0 obtained from the phantom registration is used as the initial matrix for registration of different parts of the patient.

[0137] This invention uses a geometric phantom (such as an object with known geometry and dimensions) to obtain the initial registration matrix T0. Utilizing the precise dimensions and shape of the phantom as a reference helps improve registration accuracy and simplifies the initialization steps of the registration algorithm. The initial registration matrix can serve as the starting point for the algorithm, reducing the number of iterations, accelerating the registration process, avoiding the algorithm from searching in incorrect registration regions, and improving the algorithm's stability. Furthermore, it can provide a stable reference under different imaging conditions, enhancing the adaptability of the registration algorithm to different image features.

[0138] In one embodiment of the present invention, when obtaining the rotation matrix T acting on the first three-dimensional tomographic image data, it is also necessary to analyze the SSIM exponent value, such as... Figure 4 As shown, step S500 above also includes:

[0139] S510 compares and analyzes the SSIM values ​​of the initial registration matrix and the first registration matrix after they are applied to the first registration region and the second registration region, respectively.

[0140] S520 uses the registration matrix corresponding to the larger SSIM value of the two SSIM values ​​as the base matrix.

[0141] S530 obtains the rotation matrix based on the displacement of the first three-dimensional tomographic image data, the second three-dimensional tomographic image data to the first registration region, the second registration region, and the fundamental matrix.

[0142] Specifically, the SSIM index, or Structural Similarity Index, is a metric for measuring the visual similarity between two images. It considers not only brightness and contrast but also structural information, thus providing an image quality evaluation method that better aligns with human visual perception. The SSIM index ranges from -1 to 1, where 1 indicates that the two images are identical, 0 indicates no similarity, and -1 indicates that the two images are completely different. In practical applications, SSIM values ​​are typically between 0 and 1, meaning that the higher the similarity between the images, the closer the SSIM value is to 1.

[0143] By analyzing the SSIM exponents of the initial registration matrix T0 and the first registration matrix T1, the registration matrix corresponding to the larger SSIM exponent is taken as the base matrix T. max To obtain the rotation matrix T.

[0144] The formula for calculating the rotation matrix T is as follows:

[0145]

[0146] Among them, T max The basic matrix, t offset It refers to the displacement of the first three-dimensional tomographic image data, the second three-dimensional tomographic image data to the first registration region, and the second registration region.

[0147] t offset The calculation formula is:

[0148]

[0149] Where n is the number of registration layers.

[0150] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for stitching three-dimensional tomographic images, characterized in that, include: From a database of three-dimensional tomographic images obtained from different parts of the target object, obtain the first three-dimensional tomographic image data and the second three-dimensional tomographic image data of a certain part that need to be stitched together; In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is obtained; Acquire the first registration region of the first three-dimensional tomographic image data in the stitching direction and the second registration region of the second three-dimensional tomographic image data in the stitching direction; Based on the initial registration matrix, the first registration region is registered to the second registration region to obtain the first registration matrix; Based on the initial registration matrix and the first registration matrix, a rotation matrix is ​​obtained that acts on the first three-dimensional tomographic image data. The rotation matrix is ​​applied to the first three-dimensional tomographic image data to obtain the third three-dimensional tomographic image data and the third registration region. The third three-dimensional tomographic image data, the third registration region, the second registration region, and the second three-dimensional tomographic image data are fused and stitched together to obtain a three-dimensional image of the part that has been fused and stitched together. The step of obtaining the rotation matrix acting on the first three-dimensional tomographic image data based on the initial registration matrix and the first registration matrix includes: Comparative analysis of the SSIM values ​​of the initial registration matrix and the first registration matrix applied to the first registration region and the second registration region; The registration matrix corresponding to the larger SSIM value among the two SSIM values ​​is used as the base matrix; The rotation matrix is ​​obtained based on the displacement of the first three-dimensional tomographic image data, the second three-dimensional tomographic image data to the first registration region, the second registration region, and the basic matrix.

2. The three-dimensional tomographic image stitching method according to claim 1, characterized in that, In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is obtained, including: Based on the scanning directions of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is set.

3. The three-dimensional tomographic image stitching method according to claim 1, characterized in that, In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is obtained, including: The first three-dimensional tomographic images in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic images in each direction of the second three-dimensional tomographic image data are registered pairwise to obtain multiple registration matrices. Analyze the SSIM index of each of the multiple registration regions, and take the registration direction in the registration region corresponding to the maximum value of the SSIM index as the stitching direction.

4. The three-dimensional tomographic image stitching method according to claim 1, characterized in that, The step of acquiring the first registration region of the first three-dimensional tomographic image data in the stitching direction and the second registration region of the second three-dimensional tomographic image data in the stitching direction includes: Based on the stitching direction, the number of overlapping images in the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is obtained. Based on the number of overlapping images, a first registration region is obtained from the first three-dimensional tomographic image data, and a second registration region is obtained from the second three-dimensional tomographic image data.

5. The three-dimensional tomographic image stitching method according to claim 1, characterized in that, Before registering the first registration region and the second registration region based on the initial registration matrix to obtain the first registration matrix, the method further includes: The target phantom used in scanning geometric correction is used to register the three-dimensional tomographic image data of the target phantom to obtain the initial registration matrix.

6. The three-dimensional tomographic image stitching method according to claim 1, characterized in that, Before registering the first registration region and the second registration region based on the initial registration matrix to obtain the first registration matrix, the method further includes: The first registration region and the second registration region are downsampled by a preset factor to obtain the first downsampled registration region and the second downsampled registration region; The first downsampling registration region is registered to the second downsampling registration region to obtain the downsampling transformation matrix; The initial registration matrix is ​​obtained based on the downsampling transformation matrix and the preset downsampling factor.

7. The three-dimensional tomographic image stitching method according to claim 1, characterized in that: The formula for calculating the rotation matrix is: Where T is the rotation matrix, T max The basic matrix, t offset It refers to the displacement of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration region and the second registration region; The t offset The calculation formula is: Where n is the number of registration layers.

8. The three-dimensional tomographic image stitching method according to any one of claims 1 to 7, characterized in that, The process of fusing and stitching the third three-dimensional tomographic image data, the third registration region, the second registration region, and the second three-dimensional tomographic image data to obtain a fused and stitched three-dimensional image of the affected area includes: The third registration region and the second registration region are merged to obtain a registration fusion region; The third three-dimensional tomographic image data with the third registration region removed, the registration fusion region, and the second three-dimensional tomographic image data with the second registration region removed are fused and stitched together to obtain a three-dimensional image of the part that has been fused and stitched together.

9. The three-dimensional tomographic image stitching method according to claim 8, characterized in that, The method of fusing the third registration region and the second registration region to obtain a registration fusion region includes: Its nonlinear fusion formula is: Among them, b' n N is the registration and fusion region, and N is the third registration region b'. 1n The total number of sheets, where n is the nth sheet in the total number of sheets, b' 1n For the third registration region, b 2n This is the second registration region.

10. The three-dimensional tomographic image stitching method according to claim 8, characterized in that, The method of fusing the third registration region and the second registration region to obtain a registration fusion region includes: Its linear fusion formula is: b' n =0.5*b' 1n +0.5*b 2n ; Wherein, the b' n To register and integrate the region, b' 1n For the third registration region, b 2n This is the second registration region.

11. A terminal device, characterized in that, A method for stitching three-dimensional tomographic image data includes a processor for calling instructions stored in a memory, wherein when the instructions are executed, the processor performs the three-dimensional tomographic image stitching method as described in any one of claims 1-10.

12. A three-dimensional tomographic image stitching system, characterized in that, Include: The terminal scanning device is used to scan different parts of the target object and send the obtained three-dimensional tomographic image data to the terminal device. The terminal device includes a processor for calling instructions stored in a memory, wherein when the instructions are executed, the processor performs the three-dimensional tomographic image stitching method as described in any one of claims 1-10.

Images