Medical image interlayer interpolation method and system based on latent space transformer

Abstract

This invention provides a method and system for inter-slice interpolation of medical images based on a latent space Transformer, comprising: acquiring adjacent preceding and following medical image slices and physical inter-slice distances; mapping the slices to a latent space using a pre-trained generative model with frozen parameters; then, combining the physical inter-slice distance information with latent space features and inputting it into a sequence-to-sequence network based on an attention mechanism to predict the latent space features of intermediate layers; finally, reconstructing the intermediate layer medical image using the decoder of the generative model. The sequence-to-sequence network is trained using a composite loss function including a latent space contrastive learning loss, which effectively suppresses blurring of the generated image by avoiding negative sample features that have undergone low-frequency degradation. This application can efficiently generate high-fidelity, high-resolution intermediate layer images.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically, to a method and system for inter-layer interpolation of medical images based on latent space Transformer. Background Technology

[0002] In clinical medical practice, to shorten scanning time and reduce the pressure on data storage and transmission, three-dimensional medical images, such as magnetic resonance imaging (MRI) or computed tomography (CT), often employ thicker interslice spacing during acquisition. While this acquisition method is efficient, it comes at the cost of unsampled tissue structure information between layers, resulting in insufficient image resolution in the interslice direction. When fine three-dimensional reconstruction or multiplanar reconstruction analysis is required, this lack of information can cause step-like artifacts in the image, reducing the continuity of anatomical structures and thus affecting diagnostic accuracy.

[0003] To address this issue, various inter-layer interpolation techniques have been proposed. Traditional interpolation methods, such as linear interpolation and cubic spline interpolation, while computationally simple and fast, are essentially mathematical fittings based on pixel grayscale values. They cannot recover complex anatomical structures and high-frequency texture details, resulting in often blurry images that fail to meet the detail requirements of clinical diagnosis. Subsequently, deep learning-based methods, particularly image domain reconstruction methods based on convolutional neural networks, have improved the visual quality of generated images to some extent. However, these methods still have significant drawbacks: First, directly modeling and interpolating images in high-dimensional pixel space involves enormous computational demands, requiring high levels of hardware resources such as graphics processing units (GPUs), hindering rapid deployment and application. Second, some existing deep learning methods often ignore the differences in the actual physical distance between slices under different scanning protocols, treating all interpolation tasks as equivalent. This leads to poor model adaptability to different scanning parameters and insufficient generalization ability. Finally, existing methods tend to produce overly smoothed or non-realistic images when generating intermediate layer images, losing crucial edge and detail information for diagnosis, resulting in blurry generated images.

[0004] Therefore, there is a market need for a medical image interlayer interpolation method that can combine interlayer geometric relationships, latent space compact representation, and multi-constraint joint training methods to generate structurally consistent and textured intermediate layers with low computational overhead. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a medical image inter-slice interpolation method and system based on latent space Transformer, which aims to solve the technical problems of high computational load, failure to effectively utilize physical inter-slice spacing information, and easy blurring and loss of detail in the generated images.

[0006] A medical image slice interpolation method based on latent space Transformer provided by the present invention includes: Obtain adjacent preceding and following medical image slices, as well as the physical interlayer spacing between the preceding and following medical image slices; A pre-trained and frozen parameter generative model is used to map the previous layer medical image slice and the next layer medical image slice to the latent space to obtain the previous latent space features and the next latent space features, respectively. The physical layer spacing information is combined with the previous latent space feature and the next latent space feature, and the combined result is input into a sequence-to-sequence network based on an attention mechanism to predict the intermediate latent space feature located between the previous medical image slice and the next medical image slice. The decoder of the generative model is used to reconstruct the intermediate latent space features into an intermediate medical image. The sequence-to-sequence network is pre-trained using a composite loss function that includes pixel-level reconstruction loss and latent space contrastive learning loss. During the training process, the latent space contrastive learning loss is used to make the predicted latent space features of the intermediate layer close to the latent space features corresponding to the real intermediate layer and far away from the preset degradation features in the feature space.

[0007] Preferably, the pre-trained and frozen parameter generative model is a variational autoencoder.

[0008] Preferably, the sequence-to-sequence network is a Transformer architecture that includes an encoding module and a decoding module; The encoding module is used to process the previous latent space features and the next latent space features to generate context information; The decoding module interacts with the context information based on a set of learnable query tokens to predict the latent space features of the intermediate layer.

[0009] Preferably, the step of combining the physical layer spacing information with the previous latent space feature and the subsequent latent space feature includes: The preceding latent space feature and the following latent space feature are divided into blocks to obtain multiple latent space blocks; A ternary embedding is applied to the multiple latent space blocks, which includes spatial location embedding shared across the entire sequence, slice type embedding that distinguishes the source of the slice, and physical interlayer spacing embedding generated based on measured physical distance.

[0010] Preferably, the preset degradation features are obtained by performing low-frequency degradation processing on real intermediate-layer medical images and then mapping them to the latent space.

[0011] Preferably, the composite loss function further includes perceptual loss and multi-scale adaptive wavelet loss.

[0012] Preferably, the low-frequency degradation processing is selected from at least one of Gaussian blur, Fourier low-pass filtering, motion blur, disk defocus blur, and anisotropic Gaussian blur.

[0013] According to one embodiment of the present invention, a medical image slice interpolation system for implementing the method is characterized in that it comprises: The image acquisition module is used to acquire metadata including the original two-dimensional slice sequence and interlayer spacing, scanning direction, and pixel spacing.

[0014] The data access module is used to receive medical image data and parse the image sequence order and physical interslice spacing.

[0015] The computing processing module includes a CPU and a GPU. The CPU is responsible for data scheduling, preprocessing, DICOM parsing, and task management; the GPU is responsible for performing tensor operations, including latent space encoding, Transformer attention matrix calculation, and latent space decoding. The storage module is used to store input images, model parameters, latent space features, intermediate calculation results, and generated interpolated images; Preferably, the sequence-to-sequence network is a Transformer architecture that includes an encoding module and a decoding module; The encoding module is used to process the previous latent space features and the next latent space features to generate context information; The decoding module interacts with the context information based on a set of learnable query tokens to predict the latent space features of the intermediate layer.

[0016] Preferably, combining the physical layer spacing information with the previous latent space feature and the subsequent latent space feature includes: The preceding latent space feature and the following latent space feature are divided into blocks to obtain multiple latent space blocks; A ternary embedding is applied to the multiple latent space blocks, which includes spatial location embedding shared across the entire sequence, slice type embedding that distinguishes the source of the slice, and physical interlayer spacing embedding generated based on measured physical distance.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention reduces computational complexity and hardware resource consumption by performing core interpolation calculations in a low-dimensional latent space, rather than in a high-dimensional pixel space, achieving high efficiency and low cost, making the model easier to deploy and apply on conventional hardware.

[0018] 2. This invention employs a ternary embedded latent space encoder-decoder Transformer, which generates an intermediate layer by jointly encoding the preceding and following slices and decoding the query token. This effectively models the long-distance spatial dependencies between slices, better captures and reconstructs complex anatomical structural changes, and the generated intermediate layer image maintains a high degree of consistency with the context structure, achieving high-fidelity reconstruction.

[0019] 3. This invention introduces physical interlayer spacing embedding, enabling the model to sense and adapt to changes in interlayer spacing under different scanning protocols, thereby improving the model's generalization ability and practicality in real clinical environments and giving it strong adaptability.

[0020] 4. This invention employs latent space contrastive learning, using real intermediate layers as positive samples and low-frequency degraded images as negative samples, which can suppress blurred reconstruction and improve the clarity of tissue boundaries and texture details.

[0021] 5. This invention integrates training objectives such as pixel-level loss, perceptual loss, multi-scale adaptive wavelet loss, and latent space contrastive learning loss, and constrains the generated images from multiple dimensions such as pixel accuracy, structural similarity, high-frequency details, and anti-blurring ability, ensuring that the final output images achieve a comprehensive quality improvement in diagnostic usability. Attached Figure Description

[0022] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the algorithm flow of the medical image slice interpolation method provided in the embodiment of the present invention during the training phase.

[0023] Figure 2 This is a schematic diagram of the structure of the ternary embedded latent space encoder / decoder Transformer provided in an embodiment of the present invention.

[0024] Figure 3 A flowchart of the training phase of the method provided in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of a medical imaging inter-slice interpolation system provided in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the algorithm flow of the medical image slice interpolation method provided in the embodiment of the present invention during the prediction stage.

[0027] Figure 6 A flowchart of the prediction phase of the method provided in the embodiments of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0029] This invention maps medical image slices to a low-dimensional latent space using a frozen Variational Autoencoder (VAE). Within this latent space, a ternary embedding latent space encoder-decoder Transformer is used to jointly model adjacent slices. Training is performed using pixel reconstruction loss, perceptual loss, wavelet high-frequency loss, and latent space contrastive learning loss, thereby improving the structural fidelity and detail recovery capability of inter-slice interpolation in medical images. This method is applicable to super-resolution reconstruction of intermediate thin slices synthesized from adjacent thick slices in 3D medical image sequences such as magnetic resonance imaging (MRI) and computed tomography (CT). Specifically, this method can be applied to cranial CT scan image sequences, aiming to improve their resolution in the inter-slice (i.e., Z-axis) direction, thus providing higher-quality image data for accurate 3D diagnosis and treatment planning in fields such as cerebrovascular diseases, jaw cysts, and odontogenic lesions.

[0030] Example 1 The medical image slice interpolation method based on latent space Transformer provided by the present invention includes a model training phase and a model prediction phase, specifically including the following: First, the model is trained, such as... Figure 1 and Figure 3 As shown.

[0031] Step S201: Acquire training data. Obtain training samples from medical image acquisition equipment or image archiving systems, for example, selecting training samples from a dataset of oral cone-beam computed tomography (CBCT) images containing various cases and scanning parameters. The training samples are multi-layered two-dimensional slices of the same examination sequence, consisting of three spatially consecutive axial slices: a preceding slice, an intermediate slice, and a following slice. For example, slices from layers a, b, and m of the sequence can be selected, such as... Figure 1 As shown, where, and As input to the model (corresponding to the previous and next slices in the graph), and This serves as the target for supervised learning (corresponding to the middle layer slices in the diagram). Simultaneously, it is necessary to record and obtain the physical layer spacing between the input slices. It should be noted that all sliced ​​images undergo preprocessing operations such as normalization before being input into the model.

[0032] Step S201: Latent space encoding is performed on the acquired slice images. In one embodiment of this application, a generative model pre-trained on a large medical image dataset (such as publicly available datasets like LIDC-IDRI) is used, specifically a variational autoencoder (VAE). This VAE contains an encoder and a decoder. Throughout the training and prediction process of this application, the network parameters of the encoder and decoder of this VAE are frozen and no longer updated. Figure 1 As shown, the acquired front / back layer slices and intermediate layer slices are input into the frozen VAE encoder. The encoder's function is to map the high-dimensional pixel-domain image, which contains a large amount of redundant information, into a low-dimensional, compact, and semantically rich latent space. After encoding, the front layer slice... The next layer slice and intermediate layer slices Correspondingly, it is converted into the previous latent space feature. Features of the latter hidden space and intermediate layer latent space features This process can be represented as: , ,

[0033] in, This refers to the encoder section of the VAE. These are the latent space features of the previous layer, the next layer, and the real intermediate layer, respectively. This is a preset latent space feature scaling factor used to adjust the numerical range of latent space features, which is beneficial for the stable training of the subsequent network. During training, the VAE parameters are kept frozen, and only the interlayer interpolation network is trained, so that the model focuses on learning the structural changes between medical images without destroying the learned general image latent space representation.

[0034] Step S203: Perform latent space segmentation and ternary embedding. This process is completed by the latent space encoder / decoder Transformer. Figure 2 The diagram shows the internal structure of the latent space encoder / decoder Transformer. First, the previous latent space features obtained in step S202 are... and the features of the last hidden space (Typically a two-dimensional or three-dimensional tensor), it is divided into multiple non-overlapping latent space blocks of the same size in spatial dimensions, i.e. Figure 2 Let the hidden space block corresponding to the previous slice and the hidden space block corresponding to the next slice be denoted as . The block size is Each slice is then divided into N latent space blocks, as shown in the following formula:

[0035] Then, each latent space block is transformed into a one-dimensional vector, called a token, through a linear projection layer (such as a fully connected layer), as shown in the following formula:

[0036] in, and These represent the first and second slices, respectively. One token, Indicates the first Hidden space block extraction operation. This represents a linear mapping of block embeddings. The output of the latent space block after linear projection is called a token, which is then serialized in the Transformer.

[0037] Next, to enable the model to understand the contextual information of each token, a ternary embedding is applied to each token. Specifically, the ternary embedding includes: Spatial location embedding: For each token, its two-dimensional location information in the original latent space feature map is added, enabling the model to perceive the spatial layout of the anatomical structure. This embedding is shared for tokens from the preceding and following slices.

[0038] Slice type embedding: For all slices originating from the previous layer The token is added to a learnable type embedding vector. and for all slices originating from the next layer The token is added to another different learnable type of embedding vector. This allows the model to clearly distinguish the origin of each token.

[0039] Interlayer spacing embedding: The physical interlayer spacing (e.g., 0.8 mm) obtained in step S201 is normalized to obtain a scalar value. Then it is coupled with a learnable interlayer spacing embedding vector. Multiply, we get This vector is added to all tokens on one side (e.g., the next layer).

[0040] In summary, the ternary embedding formula applied to consecutive slice tokens at the same spatial location is as follows:

[0041]

[0042] In the formula, These represent the first and second slices, respectively. The result of applying ternary embedding to each token is jointly encoded in this way, enabling the model to simultaneously identify spatial location, slice origin, and actual physical acquisition spacing.

[0043] After ternary embedding, all tokens from the two layers of slices are concatenated into a single token sequence:

[0044] In the formula, the subscripts a and b represent the types of the previous and next layer slices, respectively, and N represents the total number of hidden space blocks, i.e., hidden space tokens.

[0045] Step S204: Transformer Context Modeling. The embedded token sequence is input into the Transformer encoding module. This module consists of multiple stacked self-attention layers and feedforward network layers. It processes the entire input token sequence, capturing the complex spatial dependencies within and between preceding and following slices through a self-attention mechanism, ultimately generating contextual memory.

[0046] In the formula, Represents contextual memory, This refers to the Transformer encoding module.

[0047] Step S205: Query Decoding and Latent Variable Prediction. In the Transformer decoding module, a set of learnable query tokens is first introduced. Their number is equal to the number of intermediate hidden space blocks to be predicted. These query tokens This can be viewed as a set of query vectors or probes used to guide the generation process. Regarding the query token... It will also incorporate a shared spatial location embedding, as well as a new slice type embedding representing the target intermediate layer. The formula is as follows:

[0048] In the formula, This represents the result of applying spatial location embedding and slice type embedding to the i-th query token. This represents the i-th query token. Indicates spatial location embedding, Then, the Transformer decoding module uses a cross-attention mechanism to extract the context memory generated by the Transformer encoding module. Extracting the information needed to construct the intermediate layer image :

[0049] In the formula, This represents the set of all query tokens after being subjected to spatial location embedding and slice type embedding, the number of which is equal to the number of intermediate latent space blocks to be predicted. This refers to the Transformer decoding module.

[0050] After multiple layers of decoding, the final token sequence output by the decoding module is the predicted intermediate latent space block. Recombining these blocks according to their spatial positions and performing a reverse linear projection yields the predicted intermediate latent space features. :

[0051] Among them U This represents the inverse embedding operation from the token to the latent space tensor. The latent space output does not use a restricted activation function to maintain the range of latent space feature values ​​consistent with the true VAE latent space.

[0052] Step S206: Image Reconstruction and Output. The intermediate latent space features predicted in step S205 are then used for image reconstruction and output. After reverse scaling (i.e., dividing by a coefficient) After that, the data is input into the frozen VAE decoder. The VAE decoder performs the opposite operation to the encoder, reconstructing the low-dimensional latent space features back into the high-dimensional pixel space, thereby generating the final predicted intermediate layer slice, denoted as... This process can be represented as:

[0053] in For the frozen VAE decoder section, This is for generating intermediate layer medical images.

[0054] Step S207: Multi-constraint loss calculation and parameter update. To ensure the generated image... Compared with the real intermediate layer slice (denoted as To maintain consistency across multiple dimensions, this embodiment employs a composite loss function for end-to-end model training, which is used to update the latent space encoder / decoder Transformer and the projection head parameters related to contrastive learning. This composite loss function... It consists of four weighted parts:

[0055] in, These are the weighting coefficients for each loss, used to balance different optimization objectives. For pixel-level L1 loss, In order to perceive loss, For multi-scale adaptive wavelet loss, The latent space contrastive learning loss.

[0056] Specifically, pixel-level reconstruction loss ( L1 loss is used to calculate and generate images. With real images The sum of absolute differences at each pixel:

[0057] This loss is used to constrain the consistency of pixel intensity between the generated image and the real intermediate layer, ensuring the accuracy of the basic grayscale values.

[0058] Perceived loss ( ): By pre-training deep convolutional networks (such as VGG-19) on large image sets (such as ImageNet), images are extracted and generated separately. and real images In the feature maps of multiple intermediate layers, the sum of the L2 norm distances between these feature maps is calculated:

[0059] in, Indicates the first Layer-aware features are used to improve the structural and texture perception consistency of the reconstructed image. This loss better ensures the structural similarity between the two in human visual perception, making the generated image appear more natural.

[0060] Multi-scale adaptive wavelet loss ( ): For generated images and real images A multi-level discrete wavelet transform is performed to decompose the wavelet into subbands of different scales and orientations. The loss function primarily calculates the high-frequency subbands (such as horizontal subbands). ,vertical diagonal Differences in details, the first Layer loss for:

[0061] in, Indicates the first Layer A small wavelet band, These are weights adaptively generated based on the error magnitude. Let... For a custom number of wavelet decomposition levels, the multi-level wavelet loss is:

[0062] This loss is used to enhance margins, tissue boundaries, and local high-frequency details. In oral cone-beam computed tomography (CBCT) images, this helps to clearly reconstruct high-frequency structures such as enamel margins, root canal walls, and trabeculae.

[0063] Latent space contrastive learning loss ( As a key aspect of this embodiment, this loss is designed to suppress blurring in the generated image. During training, the real intermediate layer image is mapped to the latent space after low-frequency degradation.

[0064] in, This is a low-frequency degradation function. In each iteration, the degradation function randomly selects one of the following: Gaussian blur, Fourier low-pass filter, motion blur, disk defocus blur, and anisotropic Gaussian blur.

[0065] Constructing predictive latent space features As anchor points, the true intermediate layer latent space features For positive samples, the intermediate latent space features are mapped after low-frequency degradation. These are negative samples. Through a learnable shared projection head... The unit spherical representation is obtained:

[0066]

[0067]

[0068] For the first in a batch For each sample, the InfoNCE loss is:

[0069] in, For temperature coefficient, For cosine similarity, , and They represent the first The unit spherical representation of the latent space variables predicted for the nth sample, the th The unit spherical representation of the latent space variables corresponding to the true intermediate layer of each sample and after the [number]th [sample] [layer]. The loss function represents the unit spherical representation of the latent space variables corresponding to the true intermediate layer after degradation. This loss makes the predicted latent space features closer to the true intermediate layer latent space features, while moving away from the low-frequency degraded representation, thereby reducing oversmoothing.

[0070] Using the backpropagation algorithm, based on the total loss Calculate the gradient and update the parameters of the latent space encoder / decoder Transformer and its associated projector. Repeat the above steps until the model converges.

[0071] Once training is complete, the model can be used for the prediction phase. Please refer to [link / reference]. Figure 5 and Figure 6 The diagram illustrates the algorithm flowchart and step flowchart for the prediction stage. In the prediction stage (step S301), any two adjacent original slices to be interpolated are input. In step S302, these two slices are fed into the latent space via a VAE encoder with fixed parameters. Then, in step S303, their latent space features are ternary embedded and input into a latent space encoder-decoder Transformer with fixed parameters for context fusion. In step S304, the Transformer decoder predicts the intermediate latent variable located precisely between the two slices. Finally, in step S305, this feature is used for image reconstruction via a VAE decoder with fixed parameters, generating a new intermediate layer image and outputting it. By repeating this process for each pair of adjacent slices in the original sequence, all newly generated slices can be inserted into the original sequence, halving the effective interlayer spacing (e.g., from 0.4 mm to 0.2 mm), thereby doubling the interlayer resolution.

[0072] Furthermore, taking oral CT or cone-beam CT (CBCT) images as an example, this embodiment is used to generate an intermediate layer in a three-dimensional tomographic sequence of the dentition region based on axial slices of two adjacent layers, thereby improving the inter-layer directional resolution.

[0073] First, acquire CT tomographic images of the patient's oral and maxillofacial region. Let the slice spacing between adjacent slices in the original sequence be... ,For example The acquired DICOM or other medical image format data is parsed, and the two-dimensional slices are sorted according to spatial location or slice number. Meta-information such as inter-slice spacing and pixel spacing for each slice is extracted. Subsequently, each slice is normalized to a uniform grayscale range and adjusted to a fixed size according to network input requirements, for example... .

[0074] During the training phase, three slices with a continuous spatial relationship were selected from the same oral CT sequence:

[0075] in, and As input, a slice of context. This serves as the actual intermediate layer supervision label. The corresponding input layer span is:

[0076] The target location to be predicted is located between two input slices, i.e., a distance of [distance] from each of the two adjacent slices. .

[0077] Then, slice the input. , and the real middle layer Input the frozen VAE encoder to obtain the corresponding latent space features:

[0078] in For frozen VAE encoders, This represents the latent space scaling factor. Taking an oral CT slice as an example, after encoding, a size of [size missing] can be obtained. The latent space features are obtained. Since the VAE remains frozen, only the inter-layer interpolation Transformer and the contrastive learning projector parameters are updated during training.

[0079] Then, and The network is divided into multiple latent space blocks and mapped to a sequence of tokens. Each token is embedded with spatial location embedding, slice type embedding, and inter-slice embedding. The inter-slice embedding is used to represent the actual physical distance between two input slices, enabling the network to perceive inter-slice changes under different scanning protocols.

[0080] The Transformer encoding module jointly models the token sequences from the two slices to obtain a latent space memory containing contextual structural information such as the crown, root, alveolar bone, and jawbone boundaries. The Transformer decoding module uses a set of learnable query tokens to extract intermediate layer structural information from the contextual memory through cross-attention, generating predicted intermediate layer latent space features.

[0081] in This represents the ternary embedding latent space encoder / decoder Transformer to be trained. This represents the latent space features of the predicted intermediate slice.

[0082] The training loss employs a multi-constraint joint training approach, comprising pixel-level L1 loss, perceptual loss, multi-scale wavelet high-frequency loss, and latent space contrastive learning loss. For oral CT images, wavelet loss enhances the recovery of high-frequency structures such as enamel margins, root canal walls, and alveolar cortex. Latent space contrastive learning uses real intermediate latent space features as positive samples and low-frequency degraded intermediate latent space features as negative samples, ensuring the model's generated results are far removed from blurred low-frequency solutions, thereby reducing excessive smoothing of tooth boundaries and trabecular bone structures.

[0083] After training is complete, during the prediction phase, only any two adjacent original slices from the oral CT sequence need to be input. Figure 5 , Figure 6 ):

[0084] The original interlayer spacing of the two is The system first encodes the two slices into latent space features. Then trained Predict latent space features located in the middle of the two:

[0085] Finally, intermediate-layer oral CT slices are generated by freezing the decoder:

[0086] After inserting the generated intermediate layer into the original sequence, the original sequence is:

[0087] It becomes:

[0088] Therefore, the interlayer spacing changed from the original Upgraded to:

[0089] This means that the inter-slice resolution is doubled. For example, when the original oral CT slice spacing is... When interpolated using this method, the spacing between adjacent slices becomes When the original interlayer spacing is When interpolated, the equivalent interlayer spacing becomes .

[0090] Through the above process, the sampling density of oral CT sequences in the inter-slice direction can be increased without rescanning the patient or increasing the radiation dose. This makes the root morphology, periapical structure, alveolar bone boundary, jawbone fine structure, and pre-implantation assessment area more continuous in three-dimensional reconstruction, thereby improving the effects of multi-planar reconstruction, curved surface reconstruction, and three-dimensional visualization.

[0091] Example 2 The present invention also provides a medical image inter-slice interpolation system based on latent space Transformer. The medical image inter-slice interpolation system based on latent space Transformer can be implemented by executing the process steps of the medical image inter-slice interpolation method based on latent space Transformer. That is, those skilled in the art can understand the medical image inter-slice interpolation method based on latent space Transformer as a preferred embodiment of the medical image inter-slice interpolation system based on latent space Transformer.

[0092] According to the present invention, a medical image slice interpolation system based on latent space Transformer is provided, such as... Figure 4 As shown, it includes: Image acquisition module: including MRI, CT or other tomographic imaging equipment, used to acquire raw two-dimensional slice sequences and metadata such as slice spacing, scanning direction, pixel spacing, etc.

[0093] Data access module: including DICOM receiving interface, PACS / RIS communication interface or local image import interface, used to receive medical image data and parse the image sequence order and physical inter-slice spacing.

[0094] The computational processing module includes a CPU and a GPU (or other dedicated AI accelerator card). The CPU is responsible for data scheduling, preprocessing, DICOM parsing, and task management; the GPU is responsible for performing tensor operations such as latent space encoding, Transformer attention matrix calculation, and latent space decoding.

[0095] Storage module: Includes main memory, video memory and non-volatile memory, used to store input images, model parameters, latent space features, intermediate calculation results and generated interpolated images.

[0096] Display and output module: including medical monitor, PACS return interface or 3D reconstruction workstation, used to display, store or output generated high-resolution medical image sequences.

[0097] The medical image acquisition module completes rapid scanning with a large inter-slice gap; the data access module sends adjacent slices and their inter-slice gaps to the computation processing module; the computation processing module first calls the frozen VAE to compress the slices into the latent space, and then the Transformer performs inter-slice relationship modeling in the latent space, where the multi-head attention operation is completed in parallel by the matrix multiplication unit of the GPU / NPU; finally, the decoder generates intermediate layers, which together with the original slices form the inter-slice enhanced medical image sequence.

[0098] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0099] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for inter-slice interpolation of medical images based on latent space Transformer, characterized in that, include: Obtain adjacent preceding and following medical image slices, as well as the physical interlayer spacing between the preceding and following medical image slices; A pre-trained and frozen parameter generative model is used to map the previous layer medical image slice and the next layer medical image slice to the latent space to obtain the previous latent space features and the next latent space features, respectively. The physical layer spacing information is combined with the previous latent space feature and the next latent space feature, and the combined result is input into a sequence-to-sequence network based on an attention mechanism to predict the intermediate latent space feature located between the previous medical image slice and the next medical image slice. The decoder of the generative model is used to reconstruct the intermediate latent space features into an intermediate medical image. The sequence-to-sequence network is pre-trained using a composite loss function that includes pixel-level reconstruction loss and latent space contrastive learning loss. During the training process, the latent space contrastive learning loss is used to make the predicted latent space features of the intermediate layer close to the latent space features corresponding to the real intermediate layer and far away from the preset degradation features in the feature space.

2. The medical image slice interpolation method based on latent space Transformer according to claim 1, characterized in that, The pre-trained and frozen parameter generative model is a variational autoencoder.

3. The medical image slice interpolation method based on latent space Transformer according to claim 1, characterized in that, The sequence-to-sequence network is a Transformer architecture that includes an encoding module and a decoding module; The encoding module is used to process the previous latent space features and the next latent space features to generate context information; The decoding module interacts with the context information based on a set of learnable query tokens to predict the latent space features of the intermediate layer.

4. The medical image slice interpolation method based on latent space Transformer according to claim 1, characterized in that, The step of combining the physical layer spacing information with the previous latent space feature and the subsequent latent space feature includes: The preceding latent space feature and the following latent space feature are divided into blocks to obtain multiple latent space blocks; A ternary embedding is applied to the multiple latent space blocks, which includes spatial location embedding shared across the entire sequence, slice type embedding that distinguishes the source of the slice, and physical interlayer spacing embedding generated based on measured physical distance.

5. The medical image slice interpolation method based on latent space Transformer according to claim 1, characterized in that, The preset degradation features are obtained by performing low-frequency degradation processing on real intermediate-layer medical images and then mapping them to the latent space.

6. The medical image slice interpolation method based on latent space Transformer according to claim 1, characterized in that, The composite loss function further includes perceptual loss and multi-scale adaptive wavelet loss.

7. The medical image slice interpolation method based on latent space Transformer according to claim 5, characterized in that, The low-frequency degradation processing is selected from at least one of Gaussian blur, Fourier low-pass filtering, motion blur, disk defocus blur and anisotropic Gaussian blur.

8. A medical image inter-slice interpolation system for implementing the method of any one of claims 1 to 8, characterized in that, include: The image acquisition module is used to acquire metadata including the original two-dimensional slice sequence and interlayer spacing, scanning direction, and pixel spacing; The data access module is used to receive medical image data and parse the image sequence order and physical interslice spacing; The computing processing module includes a CPU and a GPU. The CPU is responsible for data scheduling, preprocessing, DICOM parsing, and task management; the GPU is responsible for performing tensor operations, including latent space encoding, Transformer attention matrix calculation, and latent space decoding. The storage module is used to store input images, model parameters, latent space features, intermediate calculation results, and generated interpolated images.

9. The medical image inter-slice interpolation system according to claim 8, characterized in that, The sequence-to-sequence network is a Transformer architecture that includes an encoding module and a decoding module; The encoding module is used to process the previous latent space features and the next latent space features to generate context information; The decoding module interacts with the context information based on a set of learnable query tokens to predict the latent space features of the intermediate layer.

10. The medical image inter-slice interpolation system according to claim 8, characterized in that, Combining the physical layer spacing information with the previous latent space feature and the subsequent latent space feature includes: The preceding latent space feature and the following latent space feature are divided into blocks to obtain multiple latent space blocks; A ternary embedding is applied to the multiple latent space blocks, which includes spatial location embedding shared across the entire sequence, slice type embedding that distinguishes the source of the slice, and physical interlayer spacing embedding generated based on measured physical distance.

Images