Three-dimensional cross-sectional image splicing method and system and terminal equipment

Abstract

The invention provides a three-dimensional cross-sectional image splicing method and system and terminal equipment. The method comprises the following steps: acquiring first and second three-dimensional cross-sectional image data; acquiring a splicing direction of the first and second three-dimensional cross-sectional image data; acquiring a first registration area of the first three-dimensional cross-sectional image data and a second registration area of the second three-dimensional cross-sectional image data; registering the first registration area to the second registration area based on the initial registration matrix to obtain a first registration matrix; obtaining a rotation matrix acting on the first three-dimensional cross-sectional image data based on the initial registration matrix and the first registration matrix; acting the rotation matrix on the first three-dimensional cross-sectional image data to obtain third three-dimensional cross-sectional image data and a third registration area; and fusing and splicing the third three-dimensional cross-sectional image data, the second three-dimensional cross-sectional image data, the third registration area and the second registration area to obtain a fused and spliced part three-dimensional image. Three-dimensional cross-sectional image splicing is realized, and the calculation efficiency of cross-sectional image splicing is improved.

Description

Technical Field

[0001] The present invention relates to the field of image stitching, and in particular to a three-dimensional tomographic image stitching method, system and terminal device. Background Art

[0002] With the continuous development of medical conditions and technology, three-dimensional images can help doctors observe and analyze the patient's anatomical structure more intuitively, and provide a more comprehensive perspective to help doctors find details that are difficult to observe in two-dimensional images during diagnosis. People have put forward new requirements for three-dimensional medical images, such as precision and digitization. In order to solve the problem of splicing tomographic image data, the current solution mainly includes extracting a certain layer of tomographic image data from the three-dimensional tomographic image data, and then registering it using the two-dimensional registration method, and then connecting the upper and lower layers of the layer; using the point cloud information of the three-dimensional grid image for registration to achieve accurate resetting and splicing. However, when extracting a certain layer of tomographic image data from the three-dimensional tomographic image data, only the rotation and translation around the Z axis are considered; when using the point cloud information of the three-dimensional grid image for registration, the calculation process is cumbersome, and the point cloud data needs special tools to process, which does not follow the common DICOM protocol in the medical field, and requires additional tools to be used in clinical practice. Summary of the invention

[0003] The purpose of the present invention is to provide a three-dimensional tomographic image stitching method, system and terminal device, so as to realize the stitching of multiple sets of three-dimensional tomographic data to obtain a complete human body data, so as to enable the fusion of multi-layer tomographic image data and simplify the image registration calculation process, so as to facilitate doctors to observe the overall structure of the human body or the stress condition of the implant.

[0004] The technical solution provided by the present invention is as follows:

[0005] The present invention provides a three-dimensional tomographic image stitching method, comprising:

[0006] From a three-dimensional tomographic image database obtained by scanning different parts of the target object, first three-dimensional tomographic image data and second three-dimensional tomographic image data of a certain part that need to be spliced ​​are obtained.

[0007] In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, a splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is acquired.

[0008] A first registration region of the first three-dimensional tomographic image data in the stitching direction and a second registration region of the second three-dimensional tomographic image data in the stitching direction are acquired.

[0009] The first registration region is registered to the second registration region based on the initial registration matrix to obtain a first registration matrix.

[0010] A rotation matrix acting on the first three-dimensional tomographic image data is obtained based on the initial registration matrix and the first registration matrix.

[0011] The rotation matrix is ​​applied to the first three-dimensional tomographic image data to obtain third three-dimensional tomographic image data and a third registration region.

[0012] The third three-dimensional tomographic image data, the third registration region, the second registration region and the second three-dimensional tomographic image data are fused and spliced ​​to obtain a three-dimensional image of the part after fusion and splicing.

[0013] Furthermore, a rotation matrix acting on the first three-dimensional tomographic image data is obtained based on the initial registration matrix and the first registration matrix, including:

[0014] Compare and analyze the SSIM values ​​of the first registration area after the initial registration matrix and the first registration matrix act on the first registration area and the second registration area respectively.

[0015] The registration matrix corresponding to the larger SSIM value of the two SSIM values ​​is used as the basic matrix.

[0016] A rotation matrix is ​​obtained based on the displacement amounts of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration region and the second registration region, and the basic matrix.

[0017] Furthermore, in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, obtaining a splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data includes:

[0018] The splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is set according to the scanning directions of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data.

[0019] Furthermore, in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, obtaining a splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data includes:

[0020] The first three-dimensional tomographic images in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic images in each direction of the second three-dimensional tomographic image data are registered in pairs to obtain a plurality of registration matrices.

[0021] The SSIM index of each registration area in the multiple registration areas is analyzed, and the registration direction in the registration area corresponding to the maximum value of the SSIM index is taken as the splicing direction.

[0022] Further, obtaining a first registration region of the first three-dimensional tomographic image data in a stitching direction and a second registration region of the second three-dimensional tomographic image data in a stitching direction includes:

[0023] Based on the stitching direction, the number of overlapping images of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is acquired in the first three-dimensional tomographic image data and the second three-dimensional tomographic image data.

[0024] A first registration region is obtained from the first three-dimensional tomographic image data based on the number of overlapping images, and a second registration region is obtained from the second three-dimensional tomographic image data.

[0025] Furthermore, before registering the first registration region and the second registration region based on the initial registration matrix to obtain the first registration matrix, the method further includes:

[0026] The target phantom used in the geometric correction is scanned, and the three-dimensional tomographic image data of the target phantom is registered to obtain an initial registration matrix.

[0027] Furthermore, before registering the first registration region and the second registration region based on the initial registration matrix to obtain the first registration matrix, the method further includes:

[0028] The first registration region and the second registration region are downsampled by a preset multiple to obtain a first downsampled registration region and a second downsampled registration region.

[0029] The first down-sampling registration region is registered to the second down-sampling registration region to obtain a down-sampling transformation matrix.

[0030] The initial registration matrix is ​​obtained based on the downsampling transformation matrix and the downsampling preset multiple.

[0031] Furthermore, the calculation formula of the rotation matrix is:

[0032]

[0033] Where T is the rotation matrix, T max is the basic matrix, t offset It is the displacement amount of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration area and the second registration area.

[0034] t offset The calculation formula is:

[0035]

[0036] Where n is the number of registration layers.

[0037] Further, the third three-dimensional tomographic image data, the third registration region, the second registration region and the second three-dimensional tomographic image data are fused and spliced ​​to obtain a three-dimensional image of the part after fusion and splicing, including:

[0038] The third registration region and the second registration region are fused to obtain a registration fusion region.

[0039] The third three-dimensional tomographic image data without the third registration area, the registration fusion area, and the second three-dimensional tomographic image data without the second registration area are fused and spliced ​​to obtain a three-dimensional image of the part after fusion and splicing.

[0040] Furthermore, the third registration region and the second registration region are fused to obtain a registration fusion region, including:

[0041] The nonlinear fusion formula is:

[0042]

[0043] Among them, b' n is the registration fusion area, N is the third registration area b' 1n The total number of sheets, n is the nth sheet in the total number of sheets, b' 1n is the third registration area, b 2n is the second registration area.

[0044] Furthermore, the third registration region and the second registration region are fused to obtain a registration fusion region, including:

[0045] Its linear fusion formula is:

[0046] b' n =0.5*b' 1n +0.5*b 2n ;

[0047] Among them, b' n is the registration fusion area, b' 1n is the third registration area, b 2n is the second registration area.

[0048] The present invention also provides a terminal device, comprising:

[0049] The invention is used for three-dimensional tomographic image data stitching, comprising a processor for calling instructions stored in a memory, wherein when the instructions are executed, the processor executes a three-dimensional tomographic image stitching method.

[0050] The present invention also provides a three-dimensional tomographic image stitching system, comprising:

[0051] The terminal scanning device is used to scan different parts of the target object and send the scanned three-dimensional tomographic image data to the terminal device.

[0052] The terminal device includes a processor for calling instructions stored in a memory, and when the instructions are executed, the processor executes a tomographic image stitching method.

[0053] By using a three-dimensional tomographic image stitching method, system and terminal device provided by the present invention, a first registration matrix is ​​obtained by aligning the first registration area with the second registration area based on the initial registration matrix, a rotation matrix and a third registration area are obtained by analyzing the first registration matrix, and the third registration area and the second registration area are fused to obtain a registration fusion area, thereby improving the calculation efficiency of tomographic image stitching, being able to stitch three-dimensional tomographic image data of different parts into a complete three-dimensional data of the human body, realizing the fusion of multi-layer tomographic image data, and facilitating doctors to observe the overall structure of the human body or the stress condition of the implant. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The following will explain the preferred implementation mode in a clear and understandable manner with reference to the accompanying drawings, and further explain the above-mentioned characteristics, technical features, advantages and implementation methods of a three-dimensional tomographic image stitching method, system and terminal device.

[0055] Figure 1 It is a structural schematic diagram of an embodiment of a three-dimensional tomographic image stitching system of the present invention;

[0056] Figure 2 is a flow chart of an embodiment of a three-dimensional tomographic image stitching method of the present invention;

[0057] Figure 3 is a flow chart of another embodiment of a three-dimensional tomographic image stitching method of the present invention;

[0058] Figure 4 is a flow chart of another embodiment of a three-dimensional tomographic image stitching method of the present invention;

[0059] Figure 5 is a schematic diagram of a tomographic image of the present invention;

[0060] Figure 6 is a schematic diagram of another tomographic image of the present invention;

[0061] Figure 7 It is a schematic diagram of left and right splicing of tomographic images of the present invention;

[0062] Figure 8 is a first three-dimensional tomographic image of the tomographic image of the present invention;

[0063] Fig. 9 is a second three-dimensional tomographic image of the tomographic image of the present invention;

[0064] Fig.10 It is a schematic diagram of the upper and lower splicing of the tomographic images of the present invention;

[0065] Fig.11 is the first registration region of the tomographic image of the present invention;

[0066] Fig.12 is the second registration region of the tomographic image of the present invention;

[0067] Fig.13 It is the first registration area after the tomographic image is downsampled in the present invention;

[0068] Fig.14 It is the second registration area after the tomographic image is downsampled in the present invention;

[0069] Fig.15 It is the first registration area of ​​the present invention to which the transformation matrix is ​​applied after the tomographic image is downsampled;

[0070] Fig.16 A third three-dimensional tomographic image obtained by applying a rotation matrix to the tomographic image of the present invention;

[0071] Fig.17 is a registration region of a third three-dimensional tomographic image in the tomographic image of the present invention;

[0072] Fig.18 It is a registered fusion area after the registration area of ​​the third three-dimensional tomographic image in the tomographic image of the present invention is fused with the second registration area. DETAILED DESCRIPTION

[0073] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0074] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0075] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0076] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0078] 3D tomographic image data stitching is a medical imaging technology that involves combining a series of 2D tomographic image data (such as CT or MRI scan images) into a 3D stereoscopic model. The key technical points of 3D tomographic image data stitching are: image acquisition, image preprocessing, image registration, and image reconstruction.

[0079] Among them, image acquisition obtains a series of continuous two-dimensional tomographic image data of the patient through CT (computed tomography) or MRI (magnetic resonance imaging) scanning; image preprocessing includes denoising, contrast enhancement, edge enhancement, etc. to improve image quality; image registration aligns all two-dimensional images to a common coordinate system to ensure spatial alignment between images, involving image translation, rotation and other transformations; image reconstruction uses three-dimensional reconstruction algorithms, such as voxel reconstruction or surface reconstruction, to convert the registered two-dimensional image into a three-dimensional model.

[0080] Under the existing technology, in order to solve the problem of tomographic image stitching, the current solution mainly includes extracting a layer of tomographic image data from the three-dimensional tomographic image data, aligning it using the two-dimensional alignment method, and then connecting the upper and lower layers of this layer; using the point cloud information of the three-dimensional grid image for alignment to achieve accurate image restoration and stitching.

[0081] However, when using the two-dimensional registration method for registration, this method only considers rotation and translation around the Z axis. Therefore, when the patient is required to be immobile during shooting, the accuracy of the shooting equipment is required to be high. When using point cloud information for registration, the medical tomographic image data may contain a large amount of artifacts and noise, and point cloud computing takes a long time and affects the effect. For the above reasons, this method is limited to bone registration, and in CBCT (cone beam computed tomography) images, bones and soft tissues cannot be completely distinguished, making this solution difficult to promote.

[0082] Therefore, the present invention provides a three-dimensional tomographic image stitching system, which obtains two groups of three-dimensional tomographic image data, obtains two registration areas from the two groups of three-dimensional tomographic image data, obtains an initial registration matrix under rigid registration, registers the overlapping area based on the initial matrix, and then fuses the two registered parts through a nonlinear fusion method or a linear fusion method, thereby obtaining a three-dimensional image of the stitched target part. The images stitched by the three-dimensional tomographic image stitching system based on the present invention have improved accuracy compared to the two-dimensional registration method, improved speed and application range compared to point cloud computing, and realize the fusion of multi-layer tomographic image data, which is convenient for doctors to observe the overall structure of the human body or the stress condition of implants.

[0083] The three-dimensional tomographic image data stitching system of the present invention includes a terminal scanning device and a terminal device, which can be based on a mobile three-dimensional C-arm device to scan and acquire images. The mobile three-dimensional C-arm can provide multi-angle surgical diagnostic information to assist doctors in intraoperative evaluation and judgment, such as fracture reduction and the size and position of implanted screws. The terminal scanning device captures the image of the body part obtained by X-ray scanning through its scanning panel. During the scanning process, the scanning panel can keep the subject in the center of the X-ray beam to reduce motion artifacts.

[0084] When the scanning device scans the patient, the machine scans different parts of the patient's body from different directions, and the images after multiple scans are sent to the terminal device. The terminal device can be a dedicated imaging terminal device. The imaging terminal device can receive the scanned images, analyze and stitch them based on the three-dimensional tomographic image stitching method of the present invention to obtain a complete three-dimensional image of the target object.

[0085] The 3D tomographic image data stitching system of the present invention provides real-time high-definition perspective images and digital image sequence photography through a terminal scanning device, and sends the obtained images to a dedicated imaging terminal device. The imaging terminal device realizes the registration and stitching of 3D tomographic images of any direction and size based on the 3D tomographic image stitching method, and splices multiple groups of 3D tomographic data to obtain a complete human body data, which not only realizes the fusion of multi-layer tomographic image data, but also simplifies the image registration calculation process, making it convenient for doctors to observe the overall structure of the human body for analysis and diagnosis.

[0086] The following is a description with reference to the accompanying drawings:

[0087] In one embodiment of the present invention, Figure 1 As shown, a three-dimensional tomographic image stitching system 100 includes:

[0088] The terminal scanning device 110 is used to scan different parts of the target object and send the scanned three-dimensional tomographic image data to the terminal device 120 .

[0089] The terminal device 120 includes a processor, which is used to call instructions stored in a memory. When the instructions are executed, the processor executes a three-dimensional tomographic image stitching method.

[0090] Specifically, the three-dimensional tomographic image stitching system 100 is an advanced technical system specially designed for processing and analyzing medical image data. The three-dimensional tomographic image stitching system 100 includes a terminal scanning device 110 and a terminal device 120 .

[0091] The terminal scanning device 110 may be a mobile three-dimensional C-arm, which is an advanced medical imaging device that is usually used in operating rooms to provide high-quality real-time images. The design of this device allows it to be moved in different operating rooms to meet different surgical needs. The mobile three-dimensional C-arm usually includes: a C-arm structure for stabilizing the device and allowing it to move in space; a flat-panel detector for receiving radiation after passing through the patient's body and converting it into a digital image; and a radiation source for generating radiation to image the patient.

[0092] The flat panel detector scans different parts of the patient from different directions to obtain three-dimensional tomographic images of different parts, and the images obtained contain overlapping parts. The images scanned by the flat panel detector in the terminal scanning device 110 are sent to the terminal device 120 for analysis and processing.

[0093] The terminal device 120 can be a dedicated imaging terminal device, a three-dimensional reconstruction workstation, etc., which is responsible for receiving the three-dimensional tomographic images scanned by the flat-panel detector in the terminal scanning device 110, fusing and splicing multiple groups of three-dimensional tomographic images through a three-dimensional tomographic image splicing method, and displaying the obtained complete three-dimensional image of the human body on the display panel of the terminal device 120.

[0094] Under the existing technology, 3D reconstruction requires processing a large amount of image data, resulting in large amount of calculation, long processing time, and no image information processing from multiple perspectives. Based on the 3D tomographic image stitching system of the present invention, high-definition perspective images and digital image sequences are obtained by scanning, and 3D reconstruction is performed based on these multi-perspective image data to construct a more complete 3D image.

[0095] The following describes a three-dimensional tomographic image stitching method in conjunction with the accompanying drawings.

[0096] In one embodiment of the present invention, please refer to Figure 2 , which shows a flow chart of a three-dimensional tomographic image stitching method provided in some embodiments of the present disclosure. The method is executed by the terminal device 120 (i.e., the imaging terminal device), and includes at least the following steps:

[0097] S100 obtains first three-dimensional tomographic image data and second three-dimensional tomographic image data of a certain part that need to be spliced ​​from a three-dimensional tomographic image database obtained by scanning different parts of the target object.

[0098] S200 : Acquire a splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data in each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data.

[0099] S300 acquires a first registration region of the first three-dimensional tomographic image data in a stitching direction and a second registration region of the second three-dimensional tomographic image data in the stitching direction.

[0100] S400 aligns the first registration region to the second registration region based on the initial registration matrix to obtain a first registration matrix.

[0101] S500 obtains a rotation matrix acting on the first three-dimensional tomographic image data based on the initial registration matrix and the first registration matrix.

[0102] S600: Apply the rotation matrix to the first three-dimensional tomographic image data to obtain third three-dimensional tomographic image data and a third registration region.

[0103] S700: The third three-dimensional tomographic image data, the third registration region, the second registration region and the second three-dimensional tomographic image data are fused and spliced ​​to obtain a fused and spliced ​​three-dimensional image of the part.

[0104] Specifically, the terminal scanning device 110 (also known as a mobile three-dimensional C-arm) can scan human body information at multiple angles to assist doctors in preoperative, intraoperative, and postoperative evaluation and judgment. During the scanning process, the subject can be kept in the center of the radiation beam to reduce motion artifacts. Real-time high-definition perspective images and digital image sequence images can be obtained through the flat-panel detector scanning of the terminal scanning device 110.

[0105] The terminal device receives the projection scanned by the flat panel detector of the terminal scanning device, and reconstructs the three-dimensional tomographic image data B1, B2, ... B based on the projection. N , wherein the acquired images contain some overlapping parts. N For example, if you want to obtain a complete three-dimensional human body data (when the left and right sides are not fully photographed), the person stands still, and the machine photographs the left half of the body from top to bottom A1, A2, ...A N , shoot the right half of the body from top to bottom B1, B2, ... B N .like Figure 5 A schematic diagram of the left half A1 of a tomographic image is shown in FIG. Figure 6A schematic diagram of the right half B1 of a tomographic image is shown in FIG. Figure 7 The left half A1 and the right half B1 are spliced ​​left and right. It can be seen that each body section is first spliced ​​left and right. A1 is spliced ​​to B1 to get C1, and A2 is spliced ​​to B2 to get C2. Figure 8 A schematic diagram of the upper half C1 of a tomographic image is shown in FIG. Fig. 9 A schematic diagram of the lower half C2 of a tomographic image is shown in FIG. Fig.10 The figure shows a schematic diagram of the upper half C1 of the tomographic image and the lower half C2 of the tomographic image after being spliced ​​up and down. If C1 is spliced ​​to C2 from top to bottom, C1 is the first three-dimensional tomographic image data, and C2 is the second three-dimensional tomographic image data, and so on.

[0106] In step S200, there may be multiple directions for obtaining the stitching of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data in various directions of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data. The scanning direction of the scanning terminal device may be set as the stitching direction, and the maximum value of the SSIM index of each registration matrix in multiple registration matrices may be analyzed to obtain the stitching direction.

[0107] Specifically, for example, when scanning a human body, when the flat-panel detector of the mobile three-dimensional C-arm scans the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, the scanning direction of the flat-panel detector is the stitching direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data.

[0108] If there are scanning situations in different scenarios, such as stitching tomographic images taken in two time periods before the operation and during the operation, and the scanning direction of the image is unclear, it is necessary to pairwise register the first three-dimensional tomographic images in each direction (such as up, down, left, right, front, and back) in the first three-dimensional tomographic image data and the second three-dimensional tomographic images in each direction (such as up, down, left, right, front, and back) in the second three-dimensional tomographic image data to obtain multiple registration matrices; analyze the SSIM index of each registration area in the multiple registration areas, and take the registration direction in the registration area corresponding to the maximum value of the SSIM index as the stitching direction.

[0109] After obtaining the stitching direction of the three-dimensional tomographic image, the first registration area and the second registration area in the stitching direction are obtained based on the first three-dimensional tomographic image data and the second three-dimensional tomographic image data. The three-dimensional data has 6 surfaces (for example, data of size 512*512*512), B1 is the first three-dimensional tomographic image data, and B2 is the second three-dimensional tomographic image data. B1 and B2 are respectively used to find the first image with soft tissue from the center of the 6 surfaces, such as Fig.11 and Fig.12The first registration area b1 and the second registration area b2 are shown. The first registration area b1 (up, down, left, right, front, and back) and the second registration area b2 (up, down, left, right, front, and back) are obtained by taking the registration images from the first image. There are 12 sets of data in total. The registration area needs to be an image with a non-zero portion greater than 80%. If the first image with a non-zero portion greater than 80% is the 20th image, then several images are taken from the 20th image toward the center to form the data to be registered.

[0110] The number of registration images is usually set manually and needs to be greater than the number of overlap images. Assuming the height of the plane detector is H1 and the distance moved between two shots is D1, the height of the overlap image is H1-D1, the number of reconstructed images in the scanning direction is M, and the number of overlap images is (H1-D1) / H1*M.

[0111] According to the initial registration matrix T0 obtained by rigid registration, the first registration area b1 and the second registration area b2 are registered to obtain the first registration matrix T1. For example, the first registration area is b1, the second registration area is b2, and the stitching direction is from b1 to b2. Based on the initial registration matrix T0, the first registration area b1 is registered to the second registration area b2 to obtain the first registration matrix T1.

[0112] Based on the initial registration matrix T0 and the first registration matrix T1, a rotation matrix T acting on the first three-dimensional tomographic image data B1 is obtained. Fig.16 As shown, the rotation matrix T is applied to the first three-dimensional tomographic image data B1 to obtain the third three-dimensional tomographic image data B'1. However, in order to achieve a faster calculation speed, when the mechanical precision is high and the human body cooperates well, the initial registration matrix T0 can be directly used instead of the rotation matrix T for calculation.

[0113] The third three-dimensional tomographic image data B'1, the third registration region b'1, the second registration region b2 and the second three-dimensional tomographic image data B2 are fused and spliced ​​to obtain a fused and spliced ​​three-dimensional image of the part. Specifically, the third registration region b'1 and the second registration region b2 are fused to obtain the registration fusion region b', where Fig.17 The third registration area b'1 of the third three-dimensional tomographic image B'1 is shown as follows. Fig.18 The figure shows the registered fusion area b' after the third registration area b'1 is fused with the second registration area b2; the third three-dimensional tomographic image data B'1 is stripped of the third registration area b'1 and the registered fusion area b', and the second three-dimensional tomographic image data B2 is stripped of the second registration area b2 for fusion and splicing to obtain a three-dimensional image of the fused and spliced ​​part.

[0114] When the three-dimensional tomographic image data is fused and spliced, the above step S700 also includes nonlinear fusion or linear fusion.

[0115] Specifically, assuming that the third registration region b'1 is nonlinearly fused to the second registration region b2, and the third registration region b'1 has N layers, the nonlinear fusion formula is:

[0116]

[0117] Among them, b' n is the registration fusion area, N is the third registration area b' 1n The total number of sheets, n is the nth sheet in the total number of sheets, b' 1n is the third registration area, b 2n is the second registration area.

[0118] Another fusion splicing method is linear, which is now explained.

[0119] Specifically, assuming that the third registration area b'1 is linearly fused to the second registration area b2, for example, 50% of each registration area is summed, the linear fusion formula is:

[0120] b' n =0.5*b' 1n +0.5*b 2n ;

[0121] Among them, b' n is the registration fusion area, b' 1n is the third registration area, b 2n is the second registration area.

[0122] Through the three-dimensional tomographic image stitching method of the present application, the overlapping area of ​​the three-dimensional tomographic data is extracted, and the overlapping area is registered based on the initial matrix, and then the tomographic images are fused through a nonlinear fusion method or a linear fusion method, thereby improving the computational efficiency of the image during registration and realizing the fusion of multi-layer three-dimensional tomographic image data.

[0123] In one embodiment of the present invention, before the first registration area b1 is registered with the second registration area b2 to obtain the first registration matrix T1, an initial registration matrix T0 needs to be obtained. There are mainly two ways to obtain the initial registration matrix T0. Now, one of the initial registration matrix methods is described. Figure 3 As shown, the above step S300 may include:

[0124] S310 downsamples the first registration region and the second registration region by a preset multiple to obtain a first downsampled registration region and a second downsampled registration region.

[0125] S320: The first down-sampling registration region is registered with the second down-sampling registration region to obtain a down-sampling transformation matrix.

[0126] S330 obtains the initial registration matrix based on the downsampling transformation matrix and the downsampling preset multiple.

[0127] Specifically, the first registration area b1 and the second registration area b2 are downsampled by X times to obtain b down1 and b down2 ,like Fig.13 and Fig.14 The first registration area b1 and the second registration area b2 are downsampled to obtain b down1 and b down2 .

[0128] If the stitching direction is from the first registration area b1 to the second registration area b2, then b down1 To b down2 Theoretically, the downsampling transformation matrix t1 obtained at this time acts on b down1 The result above will be the same as b down2 Overlap. Fig.15 Shown is the first registration region after downsampling and application of the transformation matrix t1.

[0129] Assuming that the downsampling transformation matrix t1 is a 4*4 rotation matrix, its form is:

[0130]

[0131] Wherein, R is the 3*3 first rotation matrix, and t is the displacement matrix of the first downsampling registration area and the second downsampling registration area.

[0132] Then the calculation formula of the initial registration matrix T0 is:

[0133]

[0134] Wherein, R is a 3*3 first rotation matrix, t is a displacement matrix of the first downsampling registration area and the second downsampling registration area, and X is a preset downsampling multiple.

[0135] In this embodiment, there is another method for obtaining the initial registration matrix, including: scanning the target phantom used in geometric correction, registering the three-dimensional tomographic image data of the target phantom, and obtaining the initial registration matrix T0.

[0136] Specifically, the initial registration matrix T0 is obtained by rigid registration (the initial registration matrix is ​​obtained by using a phantom during geometric correction). For a mobile 3D C-arm scanning device, a phantom used for geometric correction can be scanned first, and then the 3D data of the scanned phantom is registered (the initial registration matrix T0 is 0 at this time), and the matrix of the phantom registration is saved, which is T0. When scanning a patient, the patient is scanned along the track when the phantom is scanned, and the initial registration matrix T0 of the phantom registration is used as the initial matrix to register different parts of the patient.

[0137] The present invention uses a geometric phantom (such as an object of known geometric shape and size) to obtain an initial registration matrix T0, and uses the precise size and shape of the geometric phantom as a reference to help improve the accuracy of the registration and simplify the initialization step of the registration algorithm. The initial registration matrix can be used as the starting point of the algorithm, reduce the number of iterations, speed up the registration process, avoid the algorithm from searching in the wrong registration area, and improve the stability of the algorithm. In addition, it can also provide a stable reference under different imaging conditions, enhancing the adaptability of the registration algorithm to different image features.

[0138] In one embodiment of the present invention, when obtaining the rotation matrix T acting on the first three-dimensional tomographic image data, it is also necessary to analyze the SSIM index value, such as Figure 4 As shown, the above step S500 also includes:

[0139] S510 compares and analyzes the SSIM values ​​of the initial registration matrix and the first registration matrix respectively acting on the first registration area and the second registration area.

[0140] S520 uses the registration matrix corresponding to the larger SSIM value of the two SSIM values ​​as the basic matrix.

[0141] S530: Obtain a rotation matrix based on the displacements of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration region and the second registration region, and the basic matrix.

[0142] Specifically, the SSIM index, or Structural Similarity Index, is an indicator that measures the visual similarity between two images. It not only takes into account the brightness and contrast of the image, but also the structural information of the image, thus providing an image quality evaluation method that is more in line with human visual perception. The value range of the SSIM index is -1 to 1, where 1 means that the two images are exactly the same, 0 means that the two images have no similarity, and -1 means that the two images are completely different. In practical applications, the SSIM value is usually between 0 and 1, indicating that the higher the similarity of the images, the closer the SSIM value is to 1.

[0143] By analyzing the SSIM index of the initial registration matrix T0 and the SSIM index of the first registration matrix T1, the registration matrix corresponding to the larger SSIM index of the two SSIM indices is used as the basic matrix T max To obtain the rotation matrix T.

[0144] The calculation formula of the rotation matrix T is:

[0145]

[0146] Among them, T max is the basic matrix, t offset It is the displacement amount of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration area and the second registration area.

[0147] t offset The calculation formula is:

[0148]

[0149] Where n is the number of registration layers.

[0150] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A three-dimensional tomographic image stitching method, characterized in that: include: Acquire first three-dimensional tomographic image data and second three-dimensional tomographic image data of a certain part that need to be spliced ​​from a three-dimensional tomographic image database obtained by scanning different parts of the target object; In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, acquiring a splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data; Acquire a first registration region of the first three-dimensional tomographic image data in the stitching direction and a second registration region of the second three-dimensional tomographic image data in the stitching direction; Based on the initial registration matrix, the first registration region is registered to the second registration region to obtain a first registration matrix; Obtaining a rotation matrix acting on the first three-dimensional tomographic image data based on the initial registration matrix and the first registration matrix; Applying the rotation matrix to the first three-dimensional tomographic image data to obtain third three-dimensional tomographic image data and a third registration region; The third three-dimensional tomographic image data, the third registration area, the second registration area and the second three-dimensional tomographic image data are fused and spliced ​​to obtain a three-dimensional image of the part after fusion and splicing.

2. The three-dimensional tomographic image stitching method according to claim 1, characterized in that: The step of obtaining a rotation matrix acting on the first three-dimensional tomographic image data based on the initial registration matrix and the first registration matrix includes: Comparatively analyzing the SSIM values ​​of the initial registration matrix and the first registration matrix respectively acting on the first registration area and the second registration area; The registration matrix corresponding to the larger SSIM value of the two SSIM values ​​is used as the basic matrix; The rotation matrix is ​​obtained based on the displacement of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration area and the second registration area, and the basic matrix.

3. The three-dimensional tomographic image stitching method according to claim 1, characterized in that: In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, obtaining a splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data includes: A splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data is set according to the scanning directions of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data.

4. The three-dimensional tomographic image stitching method according to claim 1, characterized in that: In each direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data, obtaining a splicing direction of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data includes: Performing pairwise registration of first three-dimensional tomographic images in each direction of the first three-dimensional tomographic image data and second three-dimensional tomographic images in each direction of the second three-dimensional tomographic image data to obtain a plurality of registration matrices; The SSIM index of each registration area in the multiple registration areas is analyzed, and the registration direction in the registration area corresponding to the maximum value of the SSIM index is taken as the splicing direction.

5. The three-dimensional tomographic image stitching method according to claim 1, characterized in that: The acquiring of a first registration region of the first three-dimensional tomographic image data in the stitching direction and a second registration region of the second three-dimensional tomographic image data in the stitching direction comprises: Based on the stitching direction, acquiring the number of overlapping images of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data in the first three-dimensional tomographic image data and the second three-dimensional tomographic image data; A first registration region is acquired from the first three-dimensional tomographic image data based on the number of overlapping images, and a second registration region is acquired from the second three-dimensional tomographic image data.

6. The three-dimensional tomographic image stitching method according to claim 1, characterized in that: Before registering the first registration region and the second registration region based on the initial registration matrix to obtain the first registration matrix, the method further includes: The target phantom used in the geometric correction is scanned, and the three-dimensional tomographic image data of the target phantom is registered to obtain the initial registration matrix.

7. The three-dimensional tomographic image stitching method according to claim 1, characterized in that: Before registering the first registration region and the second registration region based on the initial registration matrix to obtain the first registration matrix, the method further includes: Downsampling the first registration region and the second registration region by a preset multiple to obtain a first downsampled registration region and a second downsampled registration region; The first down-sampling registration region is registered with the second down-sampling registration region to obtain a down-sampling transformation matrix; The initial registration matrix is ​​obtained based on the downsampling transformation matrix and the downsampling preset multiple.

8. The three-dimensional tomographic image stitching method according to claim 2, characterized in that: The calculation formula of the rotation matrix is: Where T is the rotation matrix, T max is the basic matrix, t offset is the displacement of the first three-dimensional tomographic image data and the second three-dimensional tomographic image data to the first registration area and the second registration area; The offset The calculation formula is: Where n is the number of registration layers.

9. The three-dimensional tomographic image stitching method according to any one of claims 1 to 8, characterized in that: The step of fusing and splicing the third three-dimensional tomographic image data, the third registration region, the second registration region and the second three-dimensional tomographic image data to obtain a three-dimensional image of the part after fusion and splicing includes: Fusing the third registration region and the second registration region to obtain a registration fusion region; The third three-dimensional tomographic image data without the third registration area, the registration fusion area, and the second three-dimensional tomographic image data without the second registration area are fused and spliced ​​to obtain a three-dimensional image of the part after fusion and splicing.

10. The three-dimensional tomographic image stitching method according to claim 9, characterized in that: The step of fusing the third registration region and the second registration region to obtain a registration fusion region includes: The nonlinear fusion formula is: Among them, b' n is the registration fusion area, N is the third registration area b' 1n The total number of sheets, n is the nth sheet in the total number of sheets, b' 1n is the third registration area, b 2n is the second registration area.

11. The three-dimensional tomographic image stitching method according to claim 9, characterized in that: The step of fusing the third registration region and the second registration region to obtain a registration fusion region includes: Its linear fusion formula is: b' n =0.5*b' 1n +0.5*b 2n ; Among them, the b' n is the registration fusion area, b' 1n is the third registration area, b 2n is the second registration area.

12. A terminal device, characterized in that: Used for three-dimensional tomographic image data stitching, comprising a processor, used to call instructions stored in a memory, when the instructions are executed, the processor executes the three-dimensional tomographic image stitching method according to any one of claims 1-11.

13. A three-dimensional tomographic image stitching system, characterized in that: Include: The terminal scanning device is used to scan different parts of the target object and send the scanned three-dimensional tomographic image data to the terminal device; The terminal device includes a processor for calling instructions stored in a memory, and when the instructions are executed, the processor executes the three-dimensional tomographic image stitching method according to any one of claims 1 to 11.

Images