Ultrasonic-CT image conversion method and system

Abstract

The invention discloses an ultrasonic-CT image conversion method and system, and relates to the field of ultrasonic engineering.The method comprises the steps that paired multi-source image sample data are obtained and preprocessed, and preprocessed multi-source image sample data are obtained; constructing an initial image conversion model by using the multi-scale estimation framework and the conditional wavelet guide terminal, and updating and optimizing the initial image conversion model by using an adaptive moment estimation optimization algorithm to obtain a target image conversion model; and inputting the preprocessed multi-source image sample data into a target image conversion model for denoising conversion to obtain a multi-source image sample data image. According to the method, the conditional wavelet guide module and the multi-scale estimation frame are integrated, high-frequency features of edges and textures of the US image are injected, and structural guidance is provided, so that the detail definition of the generated image is improved, the multi-scale estimation frame is fused with prediction results under multiple resolutions, artifacts are inhibited, and the overall consistency and quality of the generated image are enhanced.

Description

Technical Field

[0001] This invention relates to the field of ultrasound engineering, and more specifically, to an ultrasound-CT image conversion method and system. Background Technology

[0002] Ultrasound (US) imaging is widely used in industrial and medical testing due to its advantages such as being radiation-free, highly accurate, and low-cost. However, ultrasound images also have inherent limitations, including low signal-to-noise ratio, relatively insufficient resolution, and image quality that is heavily dependent on the operator. Computed tomography (CT) images provide rich density resolution and structural information, and are also widely used in industry and medicine, but they have disadvantages such as radiation exposure and high cost. In addition, CT equipment requires dedicated rooms for placement and maintenance, resulting in high operating costs. Since US and CT images can reflect different characteristics of the object being measured, researching how to generate high-quality CT images from ultrasound images—i.e., cross-modal image conversion—can reduce costs and has certain application value.

[0003] In recent years, image-to-image conversion techniques in deep learning have seen some existing methods attempting to convert US-CT images using models such as Generative Adversarial Networks (GANs). For example, Chinese patent CN202311074593.1 discloses a liver US-CT medical image conversion method based on a CycleGAN network. These methods are characterized by using unpaired image data for training, reducing the difficulty of data acquisition. However, their drawbacks are also significant: due to the lack of precise, one-to-one pixel-level supervision information, the model has excessive freedom in learning the conversion mapping, leading to structural distortion, loss of detail, or unrealistic artifacts in the generated CT images, making it difficult to detect the required structural accuracy.

[0004] Furthermore, GAN-type models rely on adversarial games between the generator and the discriminator for training. This process itself has inherent flaws, and the training process is unstable. It is prone to problems such as mode collapse or gradient vanishing, which makes it difficult for the model to converge to the optimal state, thus affecting the quality and stability of the final generated image.

[0005] There are currently no effective solutions to the problems in the relevant technologies. Summary of the Invention

[0006] In view of the problems in the related technologies, the present invention proposes an ultrasound-CT image conversion method and system to overcome the above-mentioned technical problems existing in the existing related technologies.

[0007] Therefore, the specific technical solution adopted by the present invention is as follows:

[0008] In a first aspect, the present invention proposes an ultrasound-CT image conversion method, the method comprising:

[0009] S1. Obtain the paired multi-source image sample data and perform preprocessing to obtain preprocessed multi-source image sample data;

[0010] S2. Based on the multi-scale estimation framework and conditional wavelet-guided terminal, an initial image conversion model is constructed, and the initial image conversion model is updated and optimized using the adaptive moment estimation optimization algorithm to obtain the target image conversion model;

[0011] S3. Input the preprocessed multi-source image sample data into the target image conversion model for denoising conversion to obtain the multi-source image sample data image.

[0012] Furthermore, the paired multi-source image sample data is acquired and preprocessed to obtain preprocessed multi-source image sample data, including:

[0013] S11. Based on the fixed box containing the sample, and using a preset parallel ultrasound scanning terminal and a standard computed tomography scanning device, the sample is scanned along the same axis and path to obtain two-dimensional ultrasound slice images and two-dimensional computed tomography slice images respectively.

[0014] S12. For the acquired original ultrasound and computed tomography (CT) image sequences, and with the CT images as fixed images and the ultrasound images as floating images, spatial registration of the ultrasound and CT images is performed using affine transformation to ensure that the ultrasound and CT images represent the same anatomical structure in spatial location.

[0015] S13. After spatial registration, the ultrasound and computed tomography data are preprocessed by resampling and center cropping to obtain preprocessed paired ultrasound and computed tomography images.

[0016] Furthermore, the spatially registered ultrasound and computed tomography (CT) data are preprocessed by resampling and center cropping to obtain preprocessed paired ultrasound and CT images, which also include:

[0017] S131. Randomly shuffle the preprocessed paired ultrasound and computed tomography image samples as a whole.

[0018] S132. Based on the shuffled samples, a corresponding number of samples are taken out as the training set by rounding down according to a preset first ratio; based on the remaining samples, a corresponding number of samples are taken out as the validation set according to a preset second ratio; based on the remaining samples, the test set is used to achieve the division of the training set, validation set and test set.

[0019] Furthermore, the initial image transformation model is constructed based on a multi-scale estimation framework and a conditional wavelet-guided terminal, including:

[0020] S21. Construct an initial image conversion model based on a multi-scale estimation framework, a conditional wavelet-guided terminal, and a cross-modal image segmentation engine;

[0021] S22. The multi-scale estimation framework includes a cross-modal image segmentation engine, and utilizes distribution, parallel processing and fusion strategies to invoke the core engine;

[0022] S23. Using a multi-scale estimation framework, receive the tensors of the stitched multimodal input time steps and record the original spatial dimension of the input tensors. According to the preset list of scale factors, the image is downsampled cyclically.

[0023] S24. Input the downsampled tensor and time step into the cross-modal image segmentation engine simultaneously for forward propagation to obtain the noise prediction map corresponding to the current scale; and upsample the noise prediction maps at each scale to restore them to the original spatial dimension of the record.

[0024] S25. Based on the noise prediction tensor that restores the original size, stack the tensors using the scale dimension and calculate the average value to obtain the final noise prediction tensor. Then, split the input tensor of the multi-scale estimation framework into the conditional image and the noisy target image.

[0025] S26. Input the conditional image into the conditional wavelet guiding terminal, and decompose the conditional image using discrete wavelet transform to obtain the first frequency component and the second frequency component.

[0026] S27. The second frequency component obtained from each level of wavelet decomposition is stitched together along the channel dimension, and the feature map is matched with the number of channels of the core cross-modal image segmentation encoder based on the projection of the convolutional layer.

[0027] Furthermore, the second frequency components obtained from each level of wavelet decomposition are concatenated along the channel dimension, and the feature map, based on the projection of the convolutional layer and matching the number of channels of the core cross-modal image segmentation encoder, also includes:

[0028] S271. Input the projected second frequency feature map into the core cross-modal image segmentation encoder path to obtain the feature map of the corresponding level, and pass the first frequency component to the next level for decomposition.

[0029] S272. Input the noisy target image into the encoder main path of the core cross-modal image segmentation engine, and perform step-by-step feature extraction and downsampling;

[0030] S273. The residual convolution terminal is used to process each layer of the encoder path sequentially, and the temporal embedding vector is received for modulation, while the wavelet features injected at the corresponding level are received.

[0031] S274. The decoder path restores spatial resolution through upsampling. Each layer concatenates the upsampled features with the features of the corresponding layer of the encoder, and processes them using residual convolution terminals and self-attention mechanisms.

[0032] S275. The feature map is processed sequentially by normalization, nonlinear activation function and convolutional layer to obtain the predicted noise. Using the core cross-modal image segmentation engine, the noise map after prediction and upsampling restoration at each scale is fused and returned to the multi-scale estimation framework for fusion to obtain the final noise prediction.

[0033] Furthermore, the formula for the self-attention mechanism is:

[0034] ;

[0035] In the formula, This indicates the overall bullish attention operation. This indicates a splicing operation. Represents a linear transformation matrix. These represent the query, key, and value, respectively, and h represents the total number of attention heads. This represents the output of the i-th attention head;

[0036] The calculation method for each attention point is as follows:

[0037] ;

[0038] In the formula, This represents the output of the i-th attention head. This indicates scaling dot product attention. The dimension of the key vector. , , Each represents a learnable weight matrix specific to the i-th attention head. These represent query, key, and value, respectively. Let T represent the activation function, and T represent the transpose.

[0039] Furthermore, the initial image transformation model is updated and optimized using an adaptive moment estimation optimization algorithm, resulting in the target image transformation model, which also includes:

[0040] S2011. The parameters of the initial image conversion model are optimized and updated using the adaptive moment estimation optimization algorithm;

[0041] S2012. Utilize the first-order moment estimate and second-order moment estimate with bias correction in the gradient of the loss function to adaptively calculate and update the step size for the parameters of the initial image conversion model.

[0042] S2013. Based on the parameter update rule of the adaptive moment estimation optimization algorithm, the weights of the initial image conversion model are iteratively updated until the initial image conversion model converges, thus obtaining the target image conversion model.

[0043] Furthermore, the formula for optimizing and updating the parameters of the initial image transformation model using the adaptive moment estimation optimization algorithm is as follows:

[0044] ;

[0045] In the formula, This represents the model parameters at time t. Indicates the learning rate. This represents the first-order moment estimate after bias correction. This represents the second-order moment estimate after bias correction. This represents a small constant to prevent division by zero.

[0046] Furthermore, the preprocessed multi-source image sample data is input into the target image conversion model for denoising conversion, resulting in multi-source image sample data images including:

[0047] S31. Input the pre-processed multi-source image sample data into the trained target image conversion model;

[0048] S32. Using the denoising diffusion framework in the target image conversion model, the preprocessed multi-source image sample data is denoised and iterated step by step to generate a clear target image based on random noise.

[0049] S33. Using peak signal-to-noise ratio as an evaluation index, the quality of the generated image is quantitatively evaluated by calculating the pixel-level difference between the clear target image and the real image.

[0050] S34. Using the structural similarity index as an evaluation index, the similarity between the target clear image and the real image in terms of brightness, contrast and structure is calculated, and the perceptual quality of the target clear image is quantitatively evaluated. Based on the evaluation results, a quantitative evaluation of the performance of the target image conversion model is obtained.

[0051] Secondly, the present invention also provides an ultrasound-CT image conversion system, the system comprising:

[0052] The sample data processing module is used to acquire paired multi-source image sample data and perform preprocessing to obtain preprocessed multi-source image sample data.

[0053] The model building and optimization module is used to build an initial image conversion model based on a multi-scale estimation framework and a conditional wavelet-guided terminal, and to update and optimize the initial image conversion model using an adaptive moment estimation optimization algorithm to obtain the target image conversion model.

[0054] The model data processing module is used to input the preprocessed multi-source image sample data into the target image conversion model for denoising conversion, and obtain the multi-source image sample data image.

[0055] The beneficial effects of this invention are as follows:

[0056] (1) This invention uses the denoising diffusion framework in the image conversion model as the basic architecture for image generation, which solves the problem of unstable training of traditional GAN ​​networks. The denoising diffusion framework is trained through a stable and well-defined noise prediction target, without adversarial game, with good convergence, fundamentally avoiding the mode collapse problem, and ensuring that the model can learn the accurate mapping from US to CT in a robust and stable manner.

[0057] (2) This invention uses precisely paired US-CT images for supervised training, which solves the problem of structural inaccuracy caused by non-pairing methods. Through pixel-level direct supervision, the model can learn the high-fidelity mapping relationship between the two modalities in anatomical structures, improving the accuracy of the generated CT images in terms of morphology, position and boundaries.

[0058] (3) This invention integrates a conditional wavelet guidance module and a multi-scale estimation framework. The conditional wavelet guidance module provides fine structural guidance for the model by injecting high-frequency features such as edges and textures from the US image, thereby improving the detail clarity of the generated image. The multi-scale estimation framework suppresses artifacts and enhances the overall consistency and quality of the generated image by fusing prediction results from multiple resolutions. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of an ultrasound-CT image conversion method according to an embodiment of the present invention;

[0061] Figure 2 This is a schematic block diagram of an ultrasound-CT image conversion system according to an embodiment of the present invention;

[0062] Figure 3 This is a network framework diagram of a model in an ultrasound-CT image conversion method according to an embodiment of the present invention;

[0063] Figure 4This is a schematic diagram of the multi-scale estimation framework in an ultrasound-CT image conversion method according to an embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram of the conditional wavelet-guided module in an ultrasound-CT image conversion method according to an embodiment of the present invention;

[0065] Figure 6 This is a schematic diagram of the U-Net core engine in an ultrasound-CT image conversion method according to an embodiment of the present invention;

[0066] Figure 7 This is a conversion example diagram in an ultrasound-CT image conversion method according to an embodiment of the present invention. Detailed Implementation

[0067] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0068] According to an embodiment of the present invention, an ultrasound-CT image conversion method and system are proposed.

[0069] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, according to an embodiment of the present invention, an ultrasound-CT image conversion method includes:

[0070] Step S1: Obtain the paired multi-source image sample data and perform preprocessing to obtain preprocessed multi-source image sample data;

[0071] Step S2: Based on the multi-scale estimation framework and conditional wavelet-guided terminal, an initial image conversion model is constructed, and the initial image conversion model is updated and optimized using the adaptive moment estimation optimization algorithm to obtain the target image conversion model;

[0072] Step S3: Input the preprocessed multi-source image sample data into the target image conversion model for denoising conversion to obtain the multi-source image sample data image.

[0073] In this optional embodiment, the paired multi-source image sample data is acquired and preprocessed to obtain preprocessed multi-source image sample data, including:

[0074] S11. Based on the fixed box containing the sample, and using a preset parallel ultrasound scanning terminal and a standard computed tomography scanning device, the sample is scanned along the same axis and path to obtain two-dimensional ultrasound slice images and two-dimensional computed tomography slice images respectively.

[0075] S12. For the acquired original ultrasound and computed tomography (CT) image sequences, and with the CT images as fixed images and the ultrasound images as floating images, spatial registration of the ultrasound and CT images is performed using affine transformation to ensure that the ultrasound and CT images represent the same anatomical structure in spatial location.

[0076] S13. After spatial registration, the ultrasound and computed tomography data are preprocessed by resampling and center cropping to obtain preprocessed paired ultrasound and computed tomography images.

[0077] In this optional embodiment, the preprocessing of resampling and center cropping is performed on the spatially registered ultrasound and computed tomography data to obtain preprocessed paired ultrasound and computed tomography images further includes:

[0078] S131. Randomly shuffle the preprocessed paired ultrasound and computed tomography image samples as a whole.

[0079] S132. Based on the shuffled samples, a corresponding number of samples are taken out as the training set by rounding down according to a preset first ratio; based on the remaining samples, a corresponding number of samples are taken out as the validation set according to a preset second ratio; based on the remaining samples, the test set is used to achieve the division of the training set, validation set and test set.

[0080] Specifically, the input sample data is constructed, including the construction of the experimental setup, image acquisition, and image preprocessing. Experimental setup construction: A device for fixing the samples is constructed. Image acquisition: The sample-containing container is fixed, and a customized parallel ultrasound scanning device (parallel ultrasound scanning terminal) and a standard CT scanner are used to perform rapid and continuous scanning of the samples along the same axis and path, acquiring a series of two-dimensional ultrasound slice images and two-dimensional CT slice images respectively.

[0081] Image preprocessing: In 3dslicer, the acquired raw US and CT image sequences are registered using affine transformations, with CT as a fixed image and US as a floating image, to ensure that the images of both modalities represent the same anatomical structure in spatial location. Subsequently, to provide the deep learning model with input data of consistent size and uniform specifications, the US and CT data are resampled and center-cropped to obtain paired US and CT images.

[0082] Dataset construction: First, count the total number of samples N, for example, 10 samples, each with a paired image. Randomly shuffle the samples, take out N = 0.8 samples and round down as the training set, take out N = 0.1 samples and round down as the validation set, and take the remaining N = 0.1 samples and round down as the test set.

[0083] Specifically, this invention uses pork as the experimental subject, placing pork tissue tightly inside a box, and pre-positioning tubular structures inside the tissue to simulate internal structures such as biological cysts. For example... Figure 7 The image shown is an example of the model's converted image. From left to right, the image shows the original US image, the real CT image, and the CT image converted by the model. It can be seen from the image that the model has a good conversion effect on this embodiment, effectively capturing the features of the image and ensuring the quality and consistency of the generated image.

[0084] In this optional embodiment, constructing the initial image transformation model based on a multi-scale estimation framework and a conditional wavelet-guided terminal includes:

[0085] S21. Construct an initial image conversion model based on a multi-scale estimation framework, a conditional wavelet-guided terminal, and a cross-modal image segmentation engine;

[0086] S22. The multi-scale estimation framework includes a cross-modal image segmentation engine, and utilizes distribution, parallel processing and fusion strategies to invoke the core engine;

[0087] S23. Using a multi-scale estimation framework, receive the tensors of the stitched multimodal input time steps and record the original spatial dimension of the input tensors. According to the preset list of scale factors, the image is downsampled cyclically.

[0088] S24. Input the downsampled tensor and time step into the cross-modal image segmentation engine simultaneously for forward propagation to obtain the noise prediction map corresponding to the current scale; and upsample the noise prediction maps at each scale to restore them to the original spatial dimension of the record.

[0089] S25. Based on the noise prediction tensor that restores the original size, stack the tensors using the scale dimension and calculate the average value to obtain the final noise prediction tensor. Then, split the input tensor of the multi-scale estimation framework into the conditional image and the noisy target image.

[0090] S26. Input the conditional image into the conditional wavelet guiding terminal, and decompose the conditional image using discrete wavelet transform to obtain the first frequency component and the second frequency component.

[0091] S27. The second frequency component obtained from each level of wavelet decomposition is stitched together along the channel dimension, and the feature map is matched with the number of channels of the core cross-modal image segmentation encoder based on the projection of the convolutional layer.

[0092] In this optional embodiment, the second frequency component obtained from each level of wavelet decomposition is stitched together along the channel dimension, and the feature map, based on the projection of the convolutional layer and matching the number of channels of the core cross-modal image segmentation encoder, further includes:

[0093] S271. Input the projected second frequency feature map into the core cross-modal image segmentation encoder path to obtain the feature map of the corresponding level, and pass the first frequency component to the next level for decomposition.

[0094] S272. Input the noisy target image into the encoder main path of the core cross-modal image segmentation engine, and perform step-by-step feature extraction and downsampling;

[0095] S273. The residual convolution terminal is used to process each layer of the encoder path sequentially, and the temporal embedding vector is received for modulation, while the wavelet features injected at the corresponding level are received.

[0096] S274. The decoder path restores spatial resolution through upsampling. Each layer concatenates the upsampled features with the features of the corresponding layer of the encoder, and processes them using residual convolution terminals and self-attention mechanisms.

[0097] S275. The feature map is processed sequentially by normalization, nonlinear activation function and convolutional layer to obtain the predicted noise. Using the core cross-modal image segmentation engine, the noise map after prediction and upsampling restoration at each scale is fused and returned to the multi-scale estimation framework for fusion to obtain the final noise prediction.

[0098] Specifically, the deep learning-based image conversion model constructed in this invention is a composite architecture, with the overall framework as follows: Figure 3 As shown, the model consists of a multi-scale estimation framework, a conditional wavelet guidance module for enhancing conditional guidance, and a core U-Net engine (i.e., a cross-modal image segmentation engine). These will be elaborated upon below:

[0099] The model includes an external multi-scale estimation framework, the detailed structure of which is as follows: Figure 4 As shown, this framework acts as a wrapper, enclosing the core U-Net engine. It does not contain learnable weights itself, but instead invokes the core engine through a distribution, parallel processing, and fusion strategy to improve model stability and accuracy. The steps are as follows:

[0100] The module mainly accepts two inputs: x_t: a concatenated four-dimensional tensor with dimensions [B,2,H,W] (size [B,2,256,256]), where B is the batch size, and the channel dimension 2 represents the US image and the noisy CT image used as conditions, respectively; t: a one-dimensional tensor representing the time step corresponding to each sample in the current batch.

[0101] Before processing begins, the module records the original spatial dimensions (width and height) of the input tensor x_t as a reference for subsequent size recovery. The program obtains scale factors (e.g., 0.75, 0.75, 1.0) from a preset list of resolution scales, and then iteratively downsamples the image based on these scale factors. To ensure image quality during downsampling, the interpolation mode is set to area.

[0102] The downsampled tensor x_scaled, along with the time step t, is fed into the core U-Net engine (i.e., the engine wrapped by this framework) Figure 6 As shown in the figure, the core engine performs a complete forward propagation at the current scale (including internal wavelet guidance, encoding, decoding, etc.) and outputs a noise prediction map corresponding to the current scale.

[0103] To unify the prediction results across all scales for fusion, an interpolation function is used to upsample the noise prediction map obtained in the previous step, restoring it to the original spatial dimensions in the record. For example, a noise prediction of size [B,1,128,128] is amplified back to [B,1,256,256]. To ensure the smoothness of the upsampling results, bilinear interpolation is used.

[0104] Multi-scale prediction results are fused using a stacking and averaging approach. Multiple tensors are stacked along a new dimension (scale dimension) to form a higher-dimensional tensor. The average value is calculated along the newly added scale dimension, thus fusing the prediction results from all scales pixel-by-pixel into a single tensor. The fusion process can be represented as follows:

[0105] ;

[0106] In the formula, This represents the final noise prediction after multi-scale fusion. Indicates the quantity of scale, and Represents the noisy CT image and the conditional US image at the i-th scale. This represents the noise prediction of the core U-Net engine at the i-th scale. This represents an upsampling function that enlarges the spatial size of the input feature map, restoring it to its original size. This indicates the original spatial dimensions.

[0107] This module ultimately outputs a fused noise prediction tensor with dimensions [B, 1, 256, 256]. During training, this output is fed into a loss function and compared with the real noise; during image generation, this output is used to guide the next iteration of denoising.

[0108] For noise prediction, U-Net is needed to train and predict the noise. The multi-scale data is first split into two independent conditional US images y with tensor size [B, 1, 256, 256] and a noisy CT image x_main with size [B, 1, 256, 256]. The y image is fed into a conditional wavelet-guided module for multi-level high-frequency feature extraction (i.e., the second frequency component). Simultaneously, t is encoded as a temporal embedding vector temb.

[0109] The model includes a conditional wavelet-guided module, the detailed structure of which is as follows: Figure 5 As shown, this module acts as a parallel feature extractor. It does not contain learnable weights (except for the projection layer), but instead utilizes fixed mathematical transformations to extract high-frequency details from the conditioned US image, providing precise structural guidance for the core U-Net engine. The input is the original conditioned US image y with dimensions [B, 1, 256, 256]. The input image (or low-frequency components) is fed into the DWT_2D layer. This layer decomposes the input into four parts, each half the size of the input, by applying low-pass and high-pass filters to the rows and columns, respectively. The single-level decomposition expression is:

[0110] ;

[0111] In the formula, LL represents the low-frequency component (first frequency component), which, after being subjected to row and column low-pass filtering, results in a blurred, halved-size thumbnail of the original image. This component will serve as the input for the next level of wavelet decomposition. LH represents the image after row low-pass and column high-pass filtering, primarily extracting horizontal edge features; HL represents the image after row high-pass and column low-pass filtering, primarily extracting vertical edge features; and HH represents the image after row and column high-pass filtering, primarily extracting diagonal edges and corner features.

[0112] The three high-frequency components LH, HL, and HH obtained from the above decomposition together constitute all the detailed information of the image at this scale, and are concatenated along the channel dimension. For a 128x128 input, three tensors of size [B, 1, 64, 64] are concatenated to obtain a high-frequency feature map of size [B, 3, 64, 64]. The concatenated high-frequency feature map is then fed into a 1x1 convolutional layer. The function of this convolutional layer is to project or translate the 3-channel edge map into a feature map that matches the number of feature channels in the core U-Net at the corresponding layer, for example, 256. Figure 6 As shown.

[0113] The high-frequency feature map after projection is injected into the feature map of the corresponding resolution level in the core U-Net encoder path. The low-frequency component LL generated at each level of decomposition continues to be passed down as the input for the next level of wavelet decomposition, and the above process is repeated until the deepest layer of the U-Net encoder is reached.

[0114] U-Net uses a classic U-shaped network to predict noise, consisting of an encoder path, a bottleneck layer, a decoder path, and skip connections. The noisy CT image x_main enters the encoder's main path, where it undergoes progressive feature extraction and downsampling. At the entry point, it passes through a 3x3 convolution, changing the size from [B,1,256,256] to [B,128,256,256], and a copy is saved for the decoder. For the first layer, the data is sequentially passed through two residual convolutional modules (ResNetBlock) and modulated using temporal embeddings (TEMB), changing the size from [B,128,256,256] to [B,128,128,128]. The second layer also sequentially passes through two residual convolutional modules (ResNetBlock) and modulates using temporal embeddings (TEMB), changing the size from [B,128,128,128] to [B,128,64,64], similarly passing through layers 3, 4, 5, and 6.

[0115] The input feature map for the sixth layer is first downsampled, processed through two ResNet blocks, and then injected with wavelet features of the corresponding scale. In the fifth layer with a resolution of 16x16, an additional self-attention module (AttnBlock) is added after each ResNet block to extract global features.

[0116] For the decoder, the bottleneck layer output concatenates the two input parts to obtain a feature map of [B, 1024, 8, 8], which is then processed by 3 ResNetBlocks. The next layer upsamples the features to size [B, 512, 16, 16], concatenates them to obtain a feature map of [B, 1024, 16, 16], and then passes it through 3 ResNetBlocks and 3 AttnBlocks in sequence. There are also six upsampling layers in total.

[0117] The final output of the top layer of the decoder, with dimensions [B, 128, 256, 256], passes through GroupNorm normalization, SiLU activation function, and a final 3x3 convolutional layer. The noise predicted by the model, with dimensions [B, 1, 256, 256], is then returned to the output. Figure 4 The aforementioned multi-scale estimation framework is then used for final fusion.

[0118] In this optional embodiment, the attention mechanism used for the bottleneck layer, specifically the self-attention mechanism processing formula, is as follows:

[0119] ;

[0120] In the formula, This indicates the overall bullish attention operation. This indicates a concatenate operation, which merges the outputs of all attention heads along the channel dimension; Represents a linear transformation matrix. These represent the query, key, and value, respectively, and h represents the total number of attention heads. This represents the output of the i-th attention head;

[0121] The calculation method for each attention point is as follows:

[0122] ;

[0123] In the formula, This represents the output of the i-th attention head. This indicates scaling dot product attention. Indicates the dimension of the key vector, used for scaling; , , Each represents a learnable weight matrix specific to the i-th attention head. These represent query, key, and value, respectively. Let T represent the activation function, and T represent the transpose.

[0124] Specifically, the input feature map x is first normalized and then fed into three independent 1x1 convolutional layers. These three new feature maps, identical in size to the input, are denoted as Q (Query), K (Key), and V (Value). Simultaneously, attention scores are calculated using matrix multiplication to obtain the attention matrix. This matrix is ​​then scaled and activated by the Softmax activation function (a non-linear activation function), and converted into normalized weights, summing to 1 to represent the attention allocation ratio. The resulting attention weight matrix is ​​then multiplied with the V feature map to obtain a feature map incorporating global information. This is then processed by a 1x1 convolution (proj_out) for final feature integration. Finally, this result is added to the original input x (residual connection) to obtain a feature map [B, 512, 8, 8] with the exact same size and number of channels as the input. After multi-scale fusion, the predicted noise is obtained. The loss function for the predicted noise and the true noise can be calculated using the mean squared error loss, expressed as:

[0125] ;

[0126] In the formula, Indicates loss, Represents real noise. This represents the noise in the prediction. US image representing conditions, This indicates the time step, telling the model how high the current noise level is. and This indicates that the ratio of signal to noise is determined based on time step t, which is used to construct a noisy image; Denotes the square of the L2 norm. This refers to the original, clear image, i.e., a CT image; It expresses expectation.

[0127] In this optional embodiment, updating and optimizing the initial image transformation model using an adaptive moment estimation optimization algorithm to obtain the target image transformation model further includes:

[0128] S2011. The parameters of the initial image conversion model are optimized and updated using the adaptive moment estimation optimization algorithm;

[0129] S2012. Utilize the first-order moment estimate and second-order moment estimate with bias correction in the gradient of the loss function to adaptively calculate and update the step size for the parameters of the initial image conversion model.

[0130] S2013. Based on the parameter update rule of the adaptive moment estimation optimization algorithm, the weights of the initial image conversion model are iteratively updated until the initial image conversion model converges, thus obtaining the target image conversion model.

[0131] In this optional embodiment, model training is accomplished by optimizing the loss function. The core of this algorithm is to adaptively calculate the update step size for each parameter using biased first-moment estimates (momentum) and second-moment estimates (rate of change) of the gradient. The formula for optimizing and updating the parameters of the initial image transformation model using the adaptive moment estimation optimization algorithm is as follows:

[0132] ;

[0133] In the formula, This represents the model parameters at time t. Indicates the learning rate. This represents the first-order moment estimate after bias correction. This represents the second-order moment estimate after bias correction. This represents a small constant to prevent division by zero.

[0134] In this optional embodiment, the preprocessed multi-source image sample data is input into the target image conversion model for denoising conversion, resulting in a multi-source image sample data image including:

[0135] S31. Input the pre-processed multi-source image sample data into the trained target image conversion model;

[0136] S32. Using the denoising diffusion framework in the target image conversion model, the preprocessed multi-source image sample data is denoised and iterated step by step to generate a clear target image based on random noise.

[0137] S33. Using peak signal-to-noise ratio as an evaluation index, the quality of the generated image is quantitatively evaluated by calculating the pixel-level difference between the clear target image and the real image.

[0138] S34. Using the structural similarity index as an evaluation index, the similarity between the target clear image and the real image in terms of brightness, contrast and structure is calculated, and the perceptual quality of the target clear image is quantitatively evaluated. Based on the evaluation results, a quantitative evaluation of the performance of the target image conversion model is obtained.

[0139] Specifically, the input sample data is fed into the deep learning model, trained within the denoising diffusion framework, and finally used to generate images. This invention provides a framework for model training and application of denoising diffusion probabilistic models. This framework includes a forward denoising process and a reverse denoising process. The formula for the forward denoising process is:

[0140] ;

[0141] In the formula, Represents the original, fully clear image (i.e., a CT image); This represents the noisy image generated at time step t; Let represent a conditional probability distribution, which defines the probability distribution given a sharp image. Noisy images The generation rules; Represent a Gaussian normal distribution; This represents the mean of a Gaussian distribution; This represents a coefficient that gradually decreases from 1 to 0. Therefore, as t increases, this mean gets closer and closer to 0, indicating that less and less information is left in the original image. and Both represent constants that vary with t; The variance represents the Gaussian distribution, and the total intensity of the noise added cumulatively at time step t. This represents a coefficient that gradually increases from 0 to 1, so the noise intensity increases as t increases. I represents the identity matrix. This formula represents the noisy image at step t. It is from the original image The signal is partially attenuated and then mixed with Gaussian noise of corresponding intensity.

[0142] For the noise reduction process:

[0143] ;

[0144] In the formula, This represents the noisy image generated at time step t; This indicates the noise that needs to be predicted. This represents a clearer image calculated by the model from the previous time step; This represents a conditional probability distribution learned by the model, defining how to guess the previous image based on the current image and the predicted noise. This indicates that it follows a Gaussian normal distribution; This represents the mean of a Gaussian distribution. This represents the variance of the Gaussian distribution; and This represents the same noise-adding process; prediction is achieved through U-Net network training, and noise is gradually removed to finally obtain the required CT image.

[0145] Specifically, the performance of the final model is evaluated: to quantitatively assess the quality of the generated CT images, industry-standard Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are used as evaluation metrics. Peak Signal-to-Noise Ratio measures the pixel-level difference between the generated image and the real image, and its calculation formula is as follows:

[0146] ;

[0147] In the formula, PSNR represents the maximum possible value of image pixels, and MSE represents the mean square error between the generated image and the real image. The higher the PSNR value, the less image distortion.

[0148] The Structural Similarity Index (SSIM) measures the similarity between images based on three aspects: brightness, contrast, and structure. This method is more consistent with human visual perception, and its calculation formula is as follows:

[0149] ;

[0150] In the formula, and These represent the real image and the generated image, respectively. and These represent the mean and variance, respectively. Represents covariance; and This represents a minimal constant used to maintain stability, in order to avoid a denominator of 0; and Representing images respectively and Pixel variance measures the contrast of an image; and Representing images respectively and The average pixel value; the closer the SSIM value is to 1, the more similar the two images are.

[0151] like Figure 2 As shown, according to another embodiment of the present invention, an ultrasound-CT image conversion system is also provided, the system comprising:

[0152] Sample data processing module 1 is used to acquire paired multi-source image sample data and perform preprocessing to obtain preprocessed multi-source image sample data;

[0153] Model building and optimization module 2 is used to build an initial image conversion model based on a multi-scale estimation framework and a conditional wavelet-guided terminal, and to update and optimize the initial image conversion model using an adaptive moment estimation optimization algorithm to obtain the target image conversion model.

[0154] Model data processing module 3 is used to input the preprocessed multi-source image sample data into the target image conversion model for denoising conversion to obtain multi-source image sample data images.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for converting ultrasound images to CT images, characterized in that, include: S1. Obtain the paired multi-source image sample data and perform preprocessing to obtain preprocessed multi-source image sample data; S2. Based on the multi-scale estimation framework and conditional wavelet-guided terminal, an initial image conversion model is constructed, and the initial image conversion model is updated and optimized using the adaptive moment estimation optimization algorithm to obtain the target image conversion model; S3. Input the preprocessed multi-source image sample data into the target image conversion model for denoising conversion to obtain the multi-source image sample data image.

2. The ultrasound-CT image conversion method according to claim 1, characterized in that, The process of acquiring paired multi-source image sample data and performing preprocessing to obtain preprocessed multi-source image sample data includes: S11. Based on the fixed box containing the sample, and using a preset parallel ultrasound scanning terminal and a standard computed tomography scanning device, the sample is scanned along the same axis and path to obtain two-dimensional ultrasound slice images and two-dimensional computed tomography slice images respectively. S12. For the acquired original ultrasound and computed tomography (CT) image sequences, and with the CT images as fixed images and the ultrasound images as floating images, spatial registration of the ultrasound and CT images is performed using affine transformation to ensure that the ultrasound and CT images represent the same anatomical structure in spatial location. S13. After spatial registration, the ultrasound and computed tomography data are preprocessed by resampling and center cropping to obtain preprocessed paired ultrasound and computed tomography images.

3. The ultrasound-CT image conversion method according to claim 2, characterized in that, The preprocessing of the spatially registered ultrasound and computed tomography data, which involves resampling and center cropping to obtain preprocessed paired ultrasound and computed tomography images, further includes: S131. Randomly shuffle the preprocessed paired ultrasound and computed tomography image samples as a whole. S132. Based on the shuffled samples, a corresponding number of samples are taken out as the training set by rounding down according to a preset first ratio; based on the remaining samples, a corresponding number of samples are taken out as the validation set according to a preset second ratio; based on the remaining samples, the test set is used to achieve the division of the training set, validation set and test set.

4. The ultrasound-CT image conversion method according to claim 1, characterized in that, The method of constructing the initial image conversion model based on the multi-scale estimation framework and conditional wavelet-guided terminal includes: S21. Construct an initial image conversion model based on a multi-scale estimation framework, a conditional wavelet-guided terminal, and a cross-modal image segmentation engine; S22. The multi-scale estimation framework includes a cross-modal image segmentation engine, and utilizes distribution, parallel processing and fusion strategies to invoke the core engine; S23. Using a multi-scale estimation framework, receive the tensors of the stitched multimodal input time steps and record the original spatial dimension of the input tensors. According to the preset scale factor list, the image is downsampled cyclically. S24. Input the downsampled tensor and time step into the cross-modal image segmentation engine simultaneously for forward propagation to obtain the noise prediction map corresponding to the current scale; and upsample the noise prediction maps at each scale to restore them to the original spatial dimension of the record. S25. Based on the noise prediction tensor that restores the original size, stack the tensors using the scale dimension and calculate the average value to obtain the final noise prediction tensor. Then, split the input tensor of the multi-scale estimation framework into the conditional image and the noisy target image. S26. Input the conditional image into the conditional wavelet guiding terminal, and decompose the conditional image using discrete wavelet transform to obtain the first frequency component and the second frequency component. S27. The second frequency component obtained from each level of wavelet decomposition is stitched together along the channel dimension, and the feature map is matched with the number of channels of the core cross-modal image segmentation encoder based on the projection of the convolutional layer.

5. The ultrasound-CT image conversion method according to claim 4, characterized in that, The feature map, which concatenates the second frequency components obtained from each wavelet decomposition along the channel dimension and matches the number of channels of the core cross-modal image segmentation encoder based on the convolutional layer projection, also includes: S271. Input the projected second frequency feature map into the core cross-modal image segmentation encoder path to obtain the feature map of the corresponding level, and pass the first frequency component to the next level for decomposition. S272. Input the noisy target image into the encoder main path of the core cross-modal image segmentation engine, and perform step-by-step feature extraction and downsampling; S273. The residual convolution terminal is used to process each layer of the encoder path sequentially, and the temporal embedding vector is received for modulation, while the wavelet features injected at the corresponding level are received. S274. The decoder path restores spatial resolution through upsampling. Each layer concatenates the upsampled features with the features of the corresponding layer of the encoder, and processes them using residual convolution terminals and self-attention mechanisms. S275. The feature map is processed sequentially by normalization, nonlinear activation function and convolutional layer to obtain the predicted noise. Using the core cross-modal image segmentation engine, the noise map after prediction and upsampling restoration at each scale is fused and returned to the multi-scale estimation framework for fusion to obtain the final noise prediction.

6. The ultrasound-CT image conversion method according to claim 5, characterized in that, The self-attention mechanism processing formula is as follows: ； In the formula, This indicates the overall bullish attention operation. This indicates a splicing operation. Represents a linear transformation matrix. These represent the query, key, and value, respectively, and h represents the total number of attention heads. This represents the output of the i-th attention head; The calculation method for each attention head is as follows: ； In the formula, This represents the output of the i-th attention head. This indicates scaling dot product attention. The dimension of the key vector. , , Each represents a learnable weight matrix specific to the i-th attention head. These represent query, key, and value, respectively. Let T represent the activation function, and T represent the transpose.

7. The ultrasound-CT image conversion method according to claim 6, characterized in that, The step of updating and optimizing the initial image conversion model using an adaptive moment estimation optimization algorithm to obtain the target image conversion model further includes: S2011. The parameters of the initial image conversion model are optimized and updated using the adaptive moment estimation optimization algorithm; S2012. Utilize the first-order moment estimate and second-order moment estimate with bias correction in the gradient of the loss function to adaptively calculate and update the step size for the parameters of the initial image conversion model. S2013. Based on the parameter update rule of the adaptive moment estimation optimization algorithm, the weights of the initial image conversion model are iteratively updated until the initial image conversion model converges, thus obtaining the target image conversion model.

8. The ultrasound-CT image conversion method according to claim 7, characterized in that, The formula for optimizing and updating the parameters of the initial image transformation model using the adaptive moment estimation optimization algorithm is as follows: ； In the formula, This represents the model parameters at time t. Indicates the learning rate. This represents the first-order moment estimate after bias correction. This represents the second-order moment estimate after bias correction. This represents a small constant to prevent division by zero.

9. The ultrasound-CT image conversion method according to claim 1, characterized in that, The step of inputting the preprocessed multi-source image sample data into the target image conversion model for denoising conversion to obtain the multi-source image sample data image includes: S31. Input the pre-processed multi-source image sample data into the trained target image conversion model; S32. Using the denoising diffusion framework in the target image conversion model, the preprocessed multi-source image sample data is denoised and iterated step by step to generate a clear target image based on random noise. S33. Using peak signal-to-noise ratio as an evaluation index, the quality of the generated image is quantitatively evaluated by calculating the pixel-level difference between the clear target image and the real image. S34. Using the structural similarity index as an evaluation index, the similarity between the target clear image and the real image in terms of brightness, contrast and structure is calculated, and the perceptual quality of the target clear image is quantitatively evaluated. Based on the evaluation results, a quantitative evaluation of the performance of the target image conversion model is obtained.

10. An ultrasound-CT image conversion system for implementing the ultrasound-CT image conversion method according to any one of claims 1-9, characterized in that, include: The sample data processing module is used to acquire paired multi-source image sample data and perform preprocessing to obtain preprocessed multi-source image sample data. The model building and optimization module is used to build an initial image conversion model based on a multi-scale estimation framework and a conditional wavelet-guided terminal, and to update and optimize the initial image conversion model using an adaptive moment estimation optimization algorithm to obtain the target image conversion model. The model data processing module is used to input the preprocessed multi-source image sample data into the target image conversion model for denoising conversion, and obtain the multi-source image sample data image.

Images