Three-dimensional image arbitrary splicing method and system

Abstract

The invention relates to the technical field of image processing, in particular to a three-dimensional image arbitrary splicing method and system. The method comprises the following steps: acquiring a plurality of three-dimensional images with known pose matrixes; selecting a reference image and calculating a relative pose matrix of other images relative to the reference image; transforming all the images to a reference coordinate system by using the relative pose matrix; resampling the transformed image by using the direction matrix and the pixel pitch of the reference image to unify spatial parameters; calculating the overall size of the spliced image according to the original points and the sizes of all the resampled images; and for each pixel position in the spliced image, calculating the minimum distance from the pixel position to the boundary of each input image, carrying out normalized weighted fusion according to the minimum distance, and finally generating a seamlessly spliced three-dimensional image. The method effectively eliminates splicing artifacts, is particularly suitable for large-view three-dimensional medical image splicing supporting rotation transformation, and has the advantages of being high in processing efficiency, good in splicing quality and wide in application range.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for arbitrary stitching of three-dimensional images. Background Technology

[0002] 3D image stitching is a key technology in computer vision and medical image processing. Its core objective is to register and fuse multiple 3D image data acquired from different viewpoints or positions into a complete, continuous, large-field-of-view 3D model. Compared to 2D image stitching, 3D stitching requires handling much larger voxel data and more complex spatial transformation relationships, facing significant challenges in data alignment, coordinate system unification, and seamless fusion. This process typically involves two core steps: registration and fusion. For a long time, research in this field and the vast majority of patented technologies have focused on improving the accuracy and robustness of registration, i.e., how to accurately calculate the rotation matrix and translation vector between two volume data sets to achieve precise spatial alignment.

[0003] However, in many cutting-edge applications, such as precision-controlled micro-CT scanning, multimodal preoperative or intraoperative medical image scanning linked by physical positioning devices, and industrial non-destructive testing based on high-precision sensors, the precise rigid transformation poses between various 3D images are often known in advance. This shifts the technical challenge from the registration stage to the fusion stage. Existing fusion methods for volume data generally have significant shortcomings when dealing with such known poses, especially rigid transformations involving significant rotation. Conventional methods, such as direct voxel overlay or linear averaging, while simple and easy to implement, blur high-frequency details of the image and produce step-like artifacts and gray-level discontinuities at overlapping boundaries due to partial volume effects and interpolation errors. Spatial distance-based weighted fusion methods fail to effectively compensate for the differences in anisotropic resolution of different data sources caused by large-angle rotation, resulting in a decrease in resolution of the fusion model in certain directions. While frequency domain fusion methods are highly efficient, they are extremely sensitive to the accuracy of rigid transformation parameters, especially rotation; small errors can easily cause ringing effects, and they struggle to handle complex spatially variable point spread functions. Feature-based selective fusion methods suffer from poor feature selection reliability and are prone to introducing discontinuous block artifacts when dealing with complex internal structures and low contrast in volumetric data. Existing techniques, such as using the Sigmoid function for boundary fusion or affine transformations for registration, while offering improvements in certain aspects, still fail to achieve ideal results in detail preservation, anisotropic resolution compensation, and high-quality seamless fusion under complex transformations, especially when facing the specific problem of known precise rigid transformations. Furthermore, pursuing high-quality fusion often comes with significant computational overhead, making it difficult to meet the efficiency requirements for processing large volumetric data. Therefore, there is an urgent need for a novel 3D image stitching method that can fully utilize known precise pose constraints and effectively solve the fusion challenges caused by rotation.

[0004] Therefore, the existing technology still needs further development. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a method and system for arbitrary stitching of three-dimensional images to solve the problems existing in the prior art.

[0006] To achieve the above-mentioned technical objectives, according to a first aspect of the present invention, the present invention provides a method for arbitrary stitching of three-dimensional images, comprising: S100. Obtain multiple three-dimensional images and the pose matrix corresponding to each three-dimensional image. Based on the pose matrix, transform each three-dimensional image to a reference coordinate system to obtain the transformed image. S200. Resample the transformed image to make all images have the same orientation matrix and pixel spacing. Determine the size of the stitched three-dimensional image based on the origin and size of the transformed image. S300. For each pixel position in the stitched 3D image, calculate the fusion weight based on the spatial relationship between the pixel position and the transformed image, and perform weighted fusion to generate the stitched 3D image.

[0007] Specifically, after acquiring multiple 3D images and the pose matrix corresponding to each 3D image, the process further includes: Each 3D image is cropped to remove unnecessary parts of the image edges.

[0008] Specifically, the cropping process includes setting cropping parameters, which are used to define the cropping amount in each direction of the image.

[0009] Specifically, the transformation of each 3D image to the reference coordinate system based on the pose matrix includes: Select one of the 3D images as a reference image, calculate the relative pose matrix of each 3D image relative to the reference image, and use the relative pose matrix to transform each 3D image into the coordinate system of the reference image.

[0010] Specifically, the relative pose matrix is ​​obtained by multiplying the inverse of the pose matrix of the reference image by the pose matrix of the other images.

[0011] Specifically, the resampling step uses the orientation matrix and pixel spacing of the reference image to resample the transformed image.

[0012] Specifically, determining the size of the stitched 3D image includes: The minimum bounding box of the stitched image is calculated based on the origin and size of all transformed images to determine the dimensions.

[0013] Specifically, the calculation of the fusion weights includes: For each pixel location, calculate its distance to each transformed image edge, and determine the weight value of each transformed image at that pixel location based on the distance.

[0014] Specifically, the weight value is calculated in the following way: For each transformed image, obtain the minimum distance from the pixel position to each edge of the image, normalize the minimum distance of each image, and obtain the weight corresponding to the image.

[0015] Specifically, the weighted fusion uses the weight values ​​to perform a weighted average of the pixel values ​​of the corresponding image.

[0016] Specifically, the method further includes: The stitched 3D image is divided into blocks, and a copy-fill or fusion strategy is used for different blocks.

[0017] Specifically, the block processing is based on the overlapping and non-overlapping regions of the transformed image.

[0018] Specifically, the three-dimensional image is medical imaging data, including CBCT or MRI images.

[0019] Specifically, the pose matrix includes rotation and translation transformation parameters.

[0020] Specifically, the method is applicable to 3D image stitching with no overlapping areas.

[0021] According to a second aspect of the present invention, a three-dimensional image arbitrary stitching system is provided, comprising: The image acquisition module is used to acquire multiple 3D images and the pose matrix corresponding to each 3D image. The image transformation module is used to transform each three-dimensional image to a reference coordinate system based on the pose matrix; The resampling module is used to resample the transformed image so that all images have the same orientation matrix and pixel spacing; The fusion module is used to determine the size of the stitched 3D image based on the origin and size of the transformed image, and to calculate the fusion weight for each pixel position to perform image fusion.

[0022] Beneficial effects: The three-dimensional image arbitrary stitching method and system provided by this invention effectively overcomes many shortcomings of existing technologies through a series of innovative technical means, bringing significant technological progress and beneficial effects. First, this invention maximizes the use of the known, precise rigid transformation pose of the three-dimensional images from various viewpoints—a strong constraint condition—fundamentally avoiding the complex and time-consuming automatic registration calculation process. By directly transforming the image to a unified reference coordinate system based on the known pose matrix and resampling using the orientation matrix and pixel spacing of the reference image, it not only greatly improves processing efficiency, making it particularly suitable for real-time requirements during surgery, but also ensures the absolute accuracy of the spatial transformation, laying a solid foundation for subsequent high-quality fusion and solving the fundamental problem of fusion failure or quality degradation caused by registration errors.

[0023] Secondly, the adaptive weighted fusion strategy based on spatial distance adopted in this invention has the advantages of clear principle and high computational efficiency. This method dynamically allocates fusion weights by calculating the minimum distance from each pixel position in the stitched image to the boundaries of each input image and performing normalization processing. This mechanism can intelligently distinguish between overlapping and non-overlapping regions: in non-overlapping regions, the weights naturally tend to uniquely cover the image in that region, achieving complete data preservation; in overlapping regions, the weights smoothly transition according to the relative position of the pixel to the boundaries of each image, making the fusion boundary natural and smooth. This completely eliminates common problems in traditional methods such as staircase artifacts, grayscale jumps, and block artifacts, significantly improving the geometric fidelity and visual consistency of the final stitched image.

[0024] Furthermore, the technical solution of this invention demonstrates excellent versatility and robustness. The method itself does not rely on overlapping areas between input images; even if there are no overlapping parts between images, effective stitching can be achieved through pose information, greatly expanding its application scenarios. Simultaneously, by introducing optional image cropping preprocessing and block processing strategies, the system can flexibly handle input data of varying quality and optimize computational resource allocation, effectively processing large volumes of 3D data and balancing processing speed and result quality. Finally, this invention is specifically optimized for the stitching needs of medical imaging such as CBCT and MRI, successfully addressing the clinical pain point that mobile C-arms and other devices cannot fully observe the required anatomical structures due to their limited field of view. By generating large-scale, high-precision 3D panoramic images, it provides doctors with more comprehensive intraoperative navigation information, directly contributing to improved surgical success rates and operational efficiency, and possessing significant clinical application value. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the operation process of CBCT three-dimensional image stitching provided in a specific embodiment of the present invention; Figure 2 This is a schematic diagram of the multi-image stitching algorithm provided in a specific embodiment of the present invention; Figure 3 This is a schematic diagram of the multi-graph fusion and recombination process provided in a specific embodiment of the present invention; Figure 4 This is a schematic diagram of a long image that meets the requirements after being fused and spliced, provided in a specific embodiment of the present invention; Figure 5 This is a schematic diagram illustrating different blocks employing copy-fill or fusion strategies in specific embodiments of the present invention; Figure 6 This is a schematic diagram of a fusion algorithm for the fusion region provided in a specific embodiment of the present invention; Figure 7 This is a schematic diagram of the overall process of the three-dimensional image arbitrary stitching method provided in a specific embodiment of the present invention; Figure 8 This is a schematic diagram of the system composition of the three-dimensional image arbitrary stitching system provided in a specific embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments in this application, other similar embodiments obtained by those skilled in the art without creative effort should all fall within the scope of protection of this application. Furthermore, directional terms mentioned in the following embodiments, such as "up," "down," "left," and "right," are only for reference to the directions in the accompanying drawings; therefore, the directional terms used are for illustrative purposes and not for limiting the invention.

[0027] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0028] Please see Figure 1-7 First of all, it should be noted that Figure 1 The process of CBCT 3D image stitching is demonstrated: First, the user selects a multi-image CBCT stitching protocol using a mobile C-arm scanner; then, the C-arm is correctly positioned to capture the first CBCT image and record its pose; the system displays the stitching queue. The C-arm is moved to the next capture area (which may not overlap with the previous area), and the next CBCT image is captured and its pose recorded; the image and pose are then imported into the stitching queue. The system checks if the capture length meets clinical requirements; finally, the user clicks "End Acquisition," and the system automatically performs fusion stitching, outputting a long image that meets the requirements, such as... Figure 4 As shown, Figure 4 The simultaneous display of four different views from the same stitched 3D data volume demonstrates that the data processed by the method of this invention is a complete 3D volume capable of multi-planar reconstruction (MPR). Figure 4The top right image is a sagittal plane view, showing a lateral section of the bone. The bottom right image is a coronal plane view, showing an anteroposterior section of the bone. The top left image is an axial plane or transverse plane view, showing a horizontal section of the bone. The orientation labels "A" (anterior), "P" (posterior), "R" (right), and "L" (left) on the images indicate that the images have been unified to the correct anatomical coordinate system. All views include a ruler (labeled "10cm"), demonstrating that the stitched images have accurate physical dimension calibration, maintaining the geometric accuracy of the original images, and ensuring the continuity and consistency of the bone structure in the four images. Although this is stitched together from multiple small-field CBCT scans, the spinal bone structure shown in the images is complete, natural, and coherent, without obvious misalignment or brightness jumps. This intuitively demonstrates the effectiveness of the pose matrix-based transformation, resampling, and weighted fusion algorithm of this invention, successfully solving the field-of-view limitation problem and generating a large-scale, diagnostically usable three-dimensional image. Figure 2 and Figure 5 The algorithm for multi-image stitching is demonstrated: taking two CBCT images as an example, the algorithm takes two CBCT images and their corresponding pose matrices (such as M1 and M2) as input, optionally performs image cropping, selects a reference image (such as image 1), calculates the relative pose matrix, transforms the image to the reference coordinate system, then resamples, and finally performs fusion and recombination. Figure 3 The process of multi-image fusion and reconstruction is demonstrated: inputting a resampled image, calculating the size of the stitched image, performing block rendering, and applying a weighted fusion algorithm to the fusion region, specifically as follows... Figure 6 As shown.

[0029] Furthermore, the present invention provides a method for arbitrary stitching of three-dimensional images, comprising: S100. Obtain multiple three-dimensional images and the pose matrix corresponding to each three-dimensional image. Based on the pose matrix, transform each three-dimensional image to a reference coordinate system to obtain the transformed image.

[0030] It should be further explained that the method specifically includes: Multiple 3D images (such as CBCT images) and their pose matrices are acquired. The pose matrices are recorded by sensors or positioning devices on a C-arm and include rotation and translation parameters to ensure accurate rigid transformation. When transforming to a reference coordinate system, a rigid transformation is performed using the pose matrices, specifically calculated through relative pose matrix calculations.

[0031] S200. Resample the transformed image to make all images have the same orientation matrix and pixel spacing. Determine the size of the stitched three-dimensional image based on the origin and size of the transformed image.

[0032] It should be further explained that the resampling step uses the direction matrix and pixel spacing of the reference image to resample all images to unify the coordinate system. When determining the stitching size, the minimum bounding box is calculated based on the origin and size of all transformed images.

[0033] S300. For each pixel position in the stitched 3D image, calculate the fusion weight based on the spatial relationship between the pixel position and the transformed image, and perform weighted fusion to generate the stitched 3D image.

[0034] It should be further explained that during the fusion process, a distance-based weight is calculated for each pixel location, and a weighted average fusion is performed.

[0035] Understandably, this method addresses the limited field of view issue of C-arm strobing by automatically stitching together images, thereby expanding the field of view and improving the surgeon's success rate and efficiency during surgery. Utilizing a known pose matrix avoids complex registration processes and particularly supports transformations with rotation, reducing stitching artifacts and improving image geometric fidelity and signal consistency. This method is applicable to stitching non-overlapping regions, enhancing its practicality.

[0036] Specifically, after acquiring multiple 3D images and the pose matrix corresponding to each 3D image, the process further includes: Each 3D image is cropped to remove unnecessary parts of the image edges.

[0037] It should be further noted that after acquiring the CBCT image, the image can be optionally cropped to remove edge noise or invalid regions. The cropping parameters include cropping amounts in six directions: left, right, top, bottom, front, and back, denoted as c1, c2, c3, c4, c5, and c6. The cropped image size is calculated using the following formula: Where w0, h0, and d0 are the width, height, and depth of the original image, respectively; and w1, h1, and d1 are the width, height, and depth after cropping. c1 to c6 represent the cropping amount, with an optimal value of 50 for each. This value is chosen empirically to effectively remove edge artifacts while retaining valid data, balancing image quality and computational efficiency. It is understandable that a cropping amount of 50 pixels in CBCT images can typically cover edge noise areas without excessive information loss, making it suitable for most medical imaging scenarios.

[0038] Understandably, cropping removes unnecessary parts of the image edges (such as scan boundary effects or noise), improving the accuracy and fusion quality of subsequent stitching and reducing outlier interference at fusion boundaries.

[0039] Specifically, the cropping process includes setting cropping parameters, which are used to define the cropping amount in each direction of the image.

[0040] It should be further noted that the cropping parameters c1 to c6 correspond to the cropping amounts in six directions: left, right, top, bottom, front, and back. These parameters are configurable and can be adjusted by the user according to the image characteristics. In a preferred embodiment, c1 to c6 are all set to 50, but those skilled in the art can choose other values ​​according to the actual application (such as different modalities of images). For example, for MRI images, the cropping amount may be adjusted to 30 to retain more detail. Parameter settings are implemented through a user interface or configuration file.

[0041] Understandably, this method adapts to different image sources and qualities through configurable cropping parameters, enhancing flexibility and robustness and ensuring highly targeted cropping processing.

[0042] Specifically, the transformation of each 3D image to the reference coordinate system based on the pose matrix includes: Select one of the 3D images as a reference image, calculate the relative pose matrix of each 3D image relative to the reference image, and use the relative pose matrix to transform each 3D image into the coordinate system of the reference image.

[0043] It should be further explained that the transformation steps specifically include: Choose a reference image (e.g., image 1), whose pose matrix is ​​M1. When calculating the relative pose matrix, for the reference image, its relative pose matrix Mr1 is an identity matrix E4; for other images (e.g., image 2), the relative pose matrix Mr2 is calculated using the following formula: in, M1 is the inverse matrix, and M2 is the pose matrix of image number 2. Then, the image is transformed to the reference coordinate system using the relative pose matrix, with the transformation formula as follows: Where It is the transformed image, Mr is the relative pose matrix, and I is the original image. Physical meaning: Mr represents the rigid transformation matrix from the image coordinate system to the reference coordinate system, I is the original image data, and It is the transformed image data.

[0044] Understandably, by transforming relative poses, all images are unified into the same coordinate system, simplifying subsequent processing, ensuring spatial consistency, and especially supporting rotation transformations, thus improving stitching accuracy.

[0045] Specifically, the relative pose matrix is ​​obtained by multiplying the inverse of the pose matrix of the reference image by the pose matrix of the other images.

[0046] It should be further explained that the calculation formula is as described in the above scheme: M1 is the inverse of the pose matrix of the reference image, representing the inverse transformation of the coordinate system; M2 is the pose matrix of other images. The multiplication operation yields the relative transformation, ensuring mathematical correctness.

[0047] Understandably, this matrix operation is efficient and accurate, utilizing known poses, avoiding iterative registration, reducing computational overhead, and is particularly suitable for real-time or intraoperative applications.

[0048] Specifically, the resampling step uses the orientation matrix and pixel spacing of the reference image to resample the transformed image.

[0049] It should be further explained that this invention uses the direction matrix Direction1 and pixel spacing Spacing1 of a reference image (such as image 1) to resample the transformed images (such as It1 and It2). The resampling algorithm can use linear interpolation or cubic spline interpolation, with linear interpolation being preferred to balance accuracy and speed. The direction matrix defines the spatial orientation of the image, and the pixel spacing defines the voxel size. After resampling, images Ir1 and Ir2 have the same direction matrix and pixel spacing.

[0050] Understandably, resampling unifies the image's sampling grid, eliminates resolution differences caused by different acquisition parameters, ensures spatial alignment during fusion, and reduces interpolation errors.

[0051] Specifically, determining the size of the stitched 3D image includes: The minimum bounding box of the stitched image is calculated based on the origin and size of all transformed images to determine the dimensions.

[0052] It should be further explained that the method specifically includes: calculating the origin (the starting point of the image in the reference coordinate system) and size (width, height, depth) of all transformed images (such as Ir1 and Ir2). The minimum bounding box is obtained by taking the minimum and maximum values ​​of the origins of all images, thereby determining the size (wl, hl, dl) of the stitched image. For example, wl = max(right boundary of image 1, right boundary of image 2) - min(left boundary of image 1, left boundary of image 2).

[0053] Understandably, the minimum bounding box ensures that the stitched image fully covers all data, avoids data truncation, and optimizes memory usage.

[0054] Specifically, the calculation of the fusion weights includes: For each pixel location, calculate its distance to each transformed image edge, and determine the weight value of each transformed image at that pixel location based on the distance.

[0055] Further explanation is needed; please refer to the following: Figure 6 The method specifically includes: For each pixel position p(x,y,z) in the stitched image, calculate its distance to the edge of each transformed image (e.g., Ir1 and Ir2). The distance calculation involves the six edges of the image (left, right, top, bottom, front, and back), denoted as p(x,y,z). Let the distance of a pixel location on the i-th image be the distance from the j-th edge. Then, weights are determined based on these distances.

[0056] Furthermore, Figure 6 The two stacked transparent cuboids represent two transformed 3D images (e.g., Ir1 and Ir2), with pixel p located inside the upper cuboid (representing image Ir2). The distances from point p to each boundary of Ir2 are calculated, as indicated by the labels d in the diagram. 2_1 (to the upper boundary), d 2_2 (to the left boundary), d 2_3 (to the lower boundary), d 2_4 (To the right boundary), and the distance to the front and rear boundaries (not directly shown in the figure, but the principle is the same). Similarly, it is also necessary to calculate the distance d from point p to each boundary of the lower cuboid (representing image Ir1). 1_j These distances d i_j It is the basis for weight calculation.

[0057] Understandably, distance-based weight calculation is simple and efficient, ensuring a smooth transition at the fusion boundary and reducing step artifacts.

[0058] Specifically, the weight value is calculated in the following way: For each transformed image, obtain the minimum distance from the pixel position to each edge of the image, normalize the minimum distance of each image, and obtain the weight corresponding to the image.

[0059] It should be further explained that the method specifically includes: For each image i, first calculate the minimum distance from the pixel location to each edge of the image. : in It is the distance to the j-th edge. Then, it is normalized to obtain the weight. : in, It is the sum of the minimum distances of all images. The final fused pixel value p(x,y,z) is calculated by weighted averaging: in: It is the contribution weight of image i. It is the pixel value of image i.

[0060] Understandably, this weighting calculation emphasizes the contribution of the image's central region, reduces boundary errors, and ensures seamless fusion, making it particularly suitable for overlapping areas.

[0061] Specifically, the weighted fusion uses the weight values ​​to perform a weighted average of the pixel values ​​of the corresponding image.

[0062] It should be further explained that the method specifically includes: weighting p1 and p2 using w1 and w2. The weighted average can be extended to multiple images, and the formula is p = Σ( ).

[0063] Understandably, weighted averaging is simple to implement, maintains the smoothness of the image signal, and avoids fusion artifacts.

[0064] Specifically, the method further includes: The stitched 3D image is divided into blocks, and a copy-fill or fusion strategy is used for different blocks.

[0065] It should be further explained that the method specifically includes: The stitched long image is divided into blocks based on overlapping and non-overlapping regions. Non-overlapping regions are directly copied and filled with the corresponding image data, while overlapping regions are fused using a fusion strategy (such as weighted fusion). The block size is configurable, preferably 64x64x64 voxels to balance processing efficiency and memory usage. This size is understandably efficient on most hardware, reducing cache misses.

[0066] Understandably, block processing allows for parallel computing, which improves the processing speed of large volumes of data and meets real-time requirements.

[0067] Specifically, the block processing is based on the overlapping and non-overlapping regions of the transformed image.

[0068] It should be further explained that the method specifically includes: Overlapping regions are determined by calculating the intersection of image boundaries, while non-overlapping regions are unique. After segmentation, non-overlapping regions are directly copied, while overlapping regions are merged.

[0069] Understandably, the processing strategy was optimized, unnecessary calculations were reduced, and efficiency was improved.

[0070] Specifically, the three-dimensional image is medical imaging data, including CBCT or MRI images.

[0071] It should be further noted that the method specifically includes: the three-dimensional image being a medical image, such as CBCT or MRI. CBCT images are derived from C-arm scans, and MRI images are derived from magnetic resonance imaging. These images typically contain voxel structure and pose information.

[0072] Understandably, this method optimizes medical imaging, solves the problem of limited field of view in clinical practice, and improves diagnostic and treatment outcomes.

[0073] Specifically, the pose matrix includes rotation and translation transformation parameters.

[0074] It should be further explained that the method specifically includes: the pose matrix is ​​a 4x4 rigid transformation matrix, containing a 3x3 rotation matrix and a 3x1 translation vector. For example, M1 represents the pose of the C-arm.

[0075] Understandably, it supports complex transformations, ensuring the accuracy of spatial transformations.

[0076] Specifically, the method is applicable to 3D image stitching with no overlapping areas.

[0077] It should be further explained that the method specifically includes: images can be non-overlapping and are directly stitched together using a pose matrix. In non-overlapping areas, the fusion weights are automatically adjusted, and non-overlapping areas are directly copied.

[0078] Understandably, this expands the application scenarios and improves the versatility of the method.

[0079] Please see Figure 8 The present invention provides another embodiment, which provides a three-dimensional image arbitrary stitching system, the three-dimensional image arbitrary stitching system comprising: The image acquisition module 100 is used to acquire multiple three-dimensional images and the pose matrix corresponding to each three-dimensional image; Image transformation module 200 is used to transform each three-dimensional image to a reference coordinate system based on the pose matrix; The resampling module 300 is used to resample the transformed image so that all images have the same orientation matrix and pixel spacing; The fusion module 400 is used to determine the size of the stitched three-dimensional image based on the origin and size of the transformed image, and to calculate the fusion weight for each pixel position to perform image fusion.

[0080] It should be further explained that the image acquisition module 100 acquires CBCT images and poses via a C-arm interface; the image transformation module 200 calculates the relative pose matrix and performs transformations; the resampling module 300 resamples using reference image parameters; and the fusion module 400 calculates size and weights to achieve fusion. The system can be deployed in a computer or embedded device, and the software is implemented using Python or C++.

[0081] Understandably, this invention systematizes the process, improves automation and practicality, and supports real-time intraoperative stitching.

[0082] In a preferred embodiment, this application also provides an electronic device, the electronic device comprising: The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the described three-dimensional image arbitrary stitching method. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the method of the present invention.

[0083] This invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0084] Those skilled in the art will understand that the method steps of this invention can be performed by a computer program instructing related hardware, such as a computer device or processor, to perform the steps of this invention when executed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0085] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0086] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for arbitrarily stitching three-dimensional images, characterized in that, include: S100. Obtain multiple three-dimensional images and the pose matrix corresponding to each three-dimensional image. Based on the pose matrix, transform each three-dimensional image to a reference coordinate system to obtain the transformed image. S200. Resample the transformed image to make all images have the same orientation matrix and pixel spacing. Determine the size of the stitched three-dimensional image based on the origin and size of the transformed image. S300. For each pixel position in the stitched 3D image, calculate the fusion weight based on the spatial relationship between the pixel position and the transformed image, and perform weighted fusion to generate the stitched 3D image.

2. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, After acquiring multiple 3D images and the pose matrix corresponding to each 3D image, the method further includes: Each 3D image is cropped to remove unnecessary parts of the image edges.

3. The method for arbitrary stitching of three-dimensional images according to claim 2, characterized in that, The cropping process includes setting cropping parameters, which define the cropping amount in each direction of the image.

4. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, The process of transforming each 3D image to the reference coordinate system based on the pose matrix includes: Select one of the 3D images as a reference image, calculate the relative pose matrix of each 3D image relative to the reference image, and use the relative pose matrix to transform each 3D image into the coordinate system of the reference image.

5. The method for arbitrary stitching of three-dimensional images according to claim 4, characterized in that, The relative pose matrix is ​​obtained by multiplying the inverse of the pose matrix of the reference image by the pose matrices of the other images.

6. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, The resampling step uses the orientation matrix and pixel spacing of the reference image to resample the transformed image.

7. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, Determining the dimensions of the stitched 3D image includes: The minimum bounding box of the stitched image is calculated based on the origin and size of all transformed images to determine the dimensions.

8. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, The calculation of the fusion weights includes: For each pixel location, calculate its distance to each transformed image edge, and determine the weight value of each transformed image at that pixel location based on the distance.

9. The method for arbitrary stitching of three-dimensional images according to claim 8, characterized in that, The weight value is calculated in the following way: For each transformed image, obtain the minimum distance from the pixel position to each edge of the image, normalize the minimum distance of each image, and obtain the weight corresponding to the image.

10. The method for arbitrary stitching of three-dimensional images according to claim 9, characterized in that, The weighted fusion uses the weight values ​​to perform a weighted average of the pixel values ​​of the corresponding image.

11. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, The method further includes: The stitched 3D image is divided into blocks, and a copy-fill or fusion strategy is used for different blocks.

12. The method for arbitrary stitching of three-dimensional images according to claim 11, characterized in that, The block processing is based on the overlapping and non-overlapping regions of the transformed image.

13. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, The three-dimensional images are medical imaging data, including CBCT or MRI images.

14. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, The pose matrix includes rotation and translation transformation parameters.

15. The method for arbitrary stitching of three-dimensional images according to claim 1, characterized in that, The method is applicable to 3D image stitching with no overlapping areas.

16. A three-dimensional image arbitrary stitching system, characterized in that, include: The image acquisition module is used to acquire multiple 3D images and the pose matrix corresponding to each 3D image. The image transformation module is used to transform each three-dimensional image to a reference coordinate system based on the pose matrix; The resampling module is used to resample the transformed image so that all images have the same orientation matrix and pixel spacing; The fusion module is used to determine the size of the stitched 3D image based on the origin and size of the transformed image, and to calculate the fusion weight for each pixel position to perform image fusion.

Images