Method and system for generating CT image from MR image under multi-condition constraint

Abstract

The invention belongs to the technical field of image processing, and provides a method and a system for generating a CT image from an MR image under multi-condition constraint in order to solve the problem of poor quality synthesis effect of an existing CT image. The method for generating the CT image from the MR image under the multi-condition constraint comprises the following steps: acquiring the MR image of a current patient and a 3D CT image of a reference similar case; on the basis of constraint conditions of the MR image of the current patient and the 3D CT image of the reference similar case, utilizing the trained generator to obtain a CT image corresponding to the MR image of the current patient; wherein the generator comprises a feature encoder, a cascade cross-domain transformer module and a feature decoder. The method considers and refers to similar case 3D CT images, does not completely depend on the quality and the number of training samples, and further improves the quality of the synthesized CT images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and system for generating CT images from MR images under multiple condition constraints. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] CT (Computed Tomography) is a medical imaging technology. It uses X-ray beams to scan the human body and uses computer processing to produce detailed images of the body's internal structures. CT scanning is widely used in the diagnosis of various diseases due to its fast and clear imaging capabilities.

[0004] The prior art provides a method for synthesizing CT images from MR images based on a residual Transformer generative adversarial network. By setting a deep feature extraction module in the generator and a feature enhancement module in the deep feature extraction module, a new generator is obtained. The deep feature extraction module in the generator can extract deeper features and make up for the shortcomings of the traditional generative adversarial network in capturing global information. However, the quality of the MR image-synthesized CT image in the prior art depends on the quality and quantity of the training samples. This may lead to low training accuracy when the samples are insufficient, thereby affecting the quality of the synthesized CT image. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for generating CT images from MR images under multi-condition constraints, which considers reference to 3D CT images of similar cases and does not completely rely on the quality and quantity of training samples, thereby improving the quality of synthesized CT images.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: A first aspect of the present invention provides a method for generating a CT image from an MR image under multiple condition constraints.

[0007] In one or more embodiments, a method for generating a CT image from an MR image under multiple condition constraints is provided, comprising: Obtain the current patient's MR image and reference similar case 3D CT images; Based on the constraints of the current patient's MR image and the 3D CT image of a reference similar case, the trained generator is used to obtain the CT image corresponding to the current patient's MR image. The generator includes a feature encoder, a cascaded cross-domain transformer module and a feature decoder; the feature encoder includes two branches, which respectively sample and encode the current patient's MR image and the reference similar case 3D CT image to obtain 2D MRI features and 3D CT features; The cascaded cross-domain transformer module is used to expand the 2D MRI feature by one dimension, combine it with the batch dimension conversion, and then splice it with the 3D CT feature along the depth dimension, merge the spliced ​​features with the batch dimension in turn along the depth dimension and merge them with the width dimension and the height dimension, and then splice a set of trainable parameters with the merged features in the channel dimension to obtain the spliced ​​features; sequentially perform position encoding on the width and height dimensions and position encoding on the depth dimension on the spliced ​​features, pass the features encoded in the depth dimension through the Transformer module of the depth dimension to obtain the first fused feature, pass the first fused feature through the height-width Transformer module to obtain the second fused feature, and use the multi-head attention mechanism to screen the first fused features and the second fused features to obtain the screened fused features; The feature decoder is used to decode the screened fusion features to obtain the corresponding CT image.

[0008] As an implementation manner, each branch of the feature encoder includes four sampling modules, and the last two sampling modules include a degradation model, and the degradation model includes two independent processes: a blurring process and a noise adding process.

[0009] As an implementation method, the degradation model is expressed as: ; in, and represents the Lth layer features and the L+1th layer features, represents the downsampling process, is the blur kernel; is the added noise; Represents the features obtained through the fuzzy process.

[0010] As an implementation method, in the cascaded cross-domain transformer module, the formula for screening the initial fusion features using the multi-head attention mechanism is: ; in, is the fusion feature after screening; is the height-width-depth global attention feature; MLP is the multi-layer perceptron function.

[0011] As an embodiment, the feature decoder includes a layer-gated fusion upsampling layer of the first two layers and a self-attention upsampling layer of the last two layers.

[0012] As an implementation method, the expression of the gated fusion upsampling layer is: ; in , , is the training parameter matrix, Represents the upsampling process The input of the layer, Represents the downsampling process The output of the layer degradation model, Represents the upsampling process The output of the layer, is the sigmoid function, which produces values ​​between 0 and 1; tanh is the tangent function.

[0013] A second aspect of the present invention provides a system for generating CT images from MR images under multiple condition constraints.

[0014] In one or more embodiments, a system for generating a CT image from an MR image under multiple condition constraints includes: An image acquisition unit, which is used to acquire the current patient's MR image and reference similar case 3D CT images; A CT image generation unit, which is used to obtain a CT image corresponding to the current patient's MR image by using a trained generator under the constraints of the current patient's MR image and a reference 3D CT image of a similar case; Among them, in the CT image generation unit, the generator includes a feature encoder, a cascaded cross-domain transformer module and a feature decoder; the feature encoder includes two branches, which respectively sample and encode the current patient MR image and the reference similar case 3D CT image to obtain 2D MRI features and 3D CT features; The cascaded cross-domain transformer module is used to expand the 2D MRI feature by one dimension, combine it with the batch dimension conversion, and then splice it with the 3D CT feature along the depth dimension, merge the spliced ​​features with the batch dimension in turn along the depth dimension and merge them with the width dimension and the height dimension, and then splice a set of trainable parameters with the merged features in the channel dimension to obtain the spliced ​​features; sequentially perform position encoding on the width and height dimensions and position encoding on the depth dimension on the spliced ​​features, pass the features encoded in the depth dimension through the Transformer module of the depth dimension to obtain the first fused feature, pass the first fused feature through the height-width Transformer module to obtain the second fused feature, and use the multi-head attention mechanism to screen the first fused features and the second fused features to obtain the screened fused features; The feature decoder is used to decode the screened fusion features to obtain the corresponding CT image.

[0015] As an implementation manner, in the CT image generation unit, each branch of the feature encoder includes four sampling modules, and the last two sampling modules include a degradation model, and the degradation model includes two independent processes: a blurring process and a noise adding process.

[0016] A third aspect of the present invention provides a computer-readable storage medium.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the method for generating a CT image from an MR image under multiple condition constraints as described above.

[0018] A fourth aspect of the present invention provides an electronic device.

[0019] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for generating a CT image from an MR image under multiple condition constraints as described above are implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention designs a cascaded cross-domain transformer module, which integrates the attention mechanism of 2D features and 3D features in the depth dimension and the attention mechanism of 2D features and 3D features in the height and width dimensions, realizes feature screening of 3D features and 2D features and heterogeneous features based on the attention mechanism. It takes into account the reference 3D CT images of similar cases, does not completely rely on the quality and quantity of training samples, and improves the quality of synthesized CT images.

[0021] (2) The generator of the present invention adopts an encoder-decoder structure, introduces a degradation model in the decoder stage, and introduces a gated fusion mechanism and a self-attention mechanism in the decoder stage, which improves the generator effect and ultimately improves the quality of the synthesized CT images. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0023] Figure 1 is a flow chart of a method for generating a CT image from an MR image under multiple condition constraints according to an embodiment of the present invention; Figure 2 is a schematic diagram of the generator structure of an embodiment of the present invention; Figure 3 is a structural diagram of a cascaded cross-domain transformer according to an embodiment of the present invention; Figure 4 This is a diagram of the gated fusion structure of an embodiment of the present invention. Figure 5 It is a comparison diagram of the input MR image, the true value of the CT image corresponding to the MR image, and the synthesized CT image; Figure 6 is a schematic diagram of the system structure of generating CT images from MR images under multiple condition constraints in an embodiment of the present invention; Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0025] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0026] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0027] Figure 1 FIG. 1 is a flow chart of a method for generating a CT image from an MR image under multiple condition constraints in an embodiment of the present invention. Figure 1 The method for generating a CT image from an MR image under multiple condition constraints in the embodiment shown may include: S101, obtaining the MR image of the current patient and the 3D CT image of a reference similar case; S102, based on the constraints of the current patient's MR image and the reference similar case 3D CT image, the trained generator is used to obtain the CT image corresponding to the current patient's MR image.

[0028] like Figure 2 As shown, the generator includes a feature encoder, a cascaded cross-domain transformer module and a feature decoder; the feature encoder includes two branches, which respectively sample and encode the current patient's MR image and the reference similar case 3D CT image to obtain 2D MRI features and 3D CT features; The cascaded cross-domain transformer module is used to expand the 2D MRI feature by one dimension, combine it with the batch dimension conversion, and then splice it with the 3D CT feature along the depth dimension, merge the spliced ​​features with the batch dimension in turn along the depth dimension and merge them with the width dimension and the height dimension, and then splice a set of trainable parameters with the merged features in the channel dimension to obtain the spliced ​​features; sequentially perform position encoding on the width and height dimensions and position encoding on the depth dimension on the spliced ​​features, pass the features encoded in the depth dimension through the Transformer module of the depth dimension to obtain the first fused feature, pass the first fused feature through the height-width Transformer module to obtain the second fused feature, and use the multi-head attention mechanism to screen the first fused features and the second fused features to obtain the screened fused features; The feature decoder is used to decode the screened fusion features to obtain the corresponding CT image.

[0029] In the embodiment of the present invention, each branch of the feature encoder includes four down-sampling modules, and the last two down-sampling modules include a degradation model, and the degradation model includes two independent processes: a blurring process and a noise adding process.

[0030] It should be noted here that, in other embodiments, the number of sampling modules and the number of degradation models included in each branch of the feature encoder can be specifically set according to actual conditions.

[0031] Specifically, the first branch takes the reference similar case 3DCT as input and passes through four sampling blocks. Its structure is shown in Table 1. Each sampling block in the table shows the kernel size, kernel number, and step size. The 3DCT feature representation is obtained as .

[0032] Table 1 3DCT feature sampling structure table

[0033] The second branch takes the current patient MR as input and also uses four sampling blocks. The structures of the first two sampling blocks are shown in Table 2, where each sampling block shows the kernel size, kernel number, and step size.

[0034] Table 2 2D MRI feature sampling structure

[0035] The last two sampling blocks use a degradation model, which includes two processes, one is a blurring process and the other is a noise adding process, which can be described by the following formula: (1); in, and represents the Lth layer features and the L+1th layer features, represents the downsampling process, is the blur kernel; is the added noise; Represents the features obtained through the fuzzy process.

[0036] Define a priori random variable that follows a multidimensional normal distribution , learned through a set of convolutional networks: (2); in , k =3, indicating the size of the convolution weight, which indicates that the learned blur kernel is spatially correlated. h = w =1 means It can be approximated by a space-invariant kernel. is the dimension of the normal distribution, the third sampling block is 256, and the fourth sampling block is 512. The last layer adds Softmax to ensure that the blur kernel The sum of each row of is 1. The added noise, the fuzzy process and the noise process are independent of each other, and the added noise is different from the previous one in that the noise is feature-related. The feature of the fuzzy kernel output is a priori random variable that obeys a multidimensional normal distribution. Feed it into a set of convolutional networks for learning: (3); in, ; The third sampling block h = w =64, c =256, is the dimension of the normal distribution, set to 64; the fourth sampling block h = w =32, c =512, is the dimension of the normal distribution, set to 64.

[0037] In summary, the degradation model formula is: (4); in, and represents the Lth layer features and the L+1th layer features, represents the downsampling process, is the blur kernel; is a priori random variable that follows a multidimensional normal distribution; is a priori random variable from a multidimensional normal distribution.

[0038] The 2DMRI features output after the two-layer degradation model are: .

[0039] like Figure 3 As shown in the figure, the cascaded cross-domain transformer is composed of two cascaded transformers, which realizes the functions of feature screening and heterogeneous feature fusion of 3DCT features and 2DMRI features.

[0040] Specifically, 2DMRI features After expanding one dimension, combined with the batch dimension, it becomes , and 3DCT features Splicing along the depth dimension gives .right Merge the depth dimension with the batch dimension, and merge the width dimension with the width dimension to get . Set a set of trainable parameters as cls_token, with After concatenation in the channel dimension, we get .

[0041] The role of cls_token is manifested in three aspects: This set of parameters is randomly initialized and continuously updated as the network is trained, and it can encode the statistical characteristics of the entire data set; Can and The global feature aggregation effect is achieved by cls_token and Transformer, and since it is not based on image content, it can avoid and Produce bias; cls_token is placed and The initial position of the depth dimension uses a fixed position encoding to avoid the output being disturbed by the position encoding.

[0042] First, position encoding is performed on the width and height dimensions. This position encoding is still a set of trainable parameters. , this parameter is related to Perform tensor addition: (5); Perform position encoding in the depth dimension, which is still a set of trainable parameters , this parameter is related to Perform tensor addition: (6); The encoded features are sent to the Transformer in the depth dimension, and the encoded That corresponds to Transformer , the multi-head attention mechanism is expressed as follows: (7); Multi-head attention mechanism After normalization and Performing addition yields , and then normalized and added again after passing through the feedforward neural network. The formula is as follows: (8); (9); In order to distinguish it from the output of the subsequent height-width Transformer, the depth Transformer output is represented as The feature first passes through a fully connected layer and then is sent to the height-width Transformer, whose structure is the same as the depth Transformer. Formulas (7), (8), and (9) are executed again to obtain . and Sent into the MLP, it is expressed as follows: (10); in, is the fusion feature after screening; is the height-width-depth global attention feature; MLP is the multi-layer perceptron function. Features with higher attention scores are filtered out through MLP to achieve the effect of removing redundant features.

[0043] In an embodiment of the present invention, the feature decoder includes the first two layers of layer gated fusion upsampling layers and the last two layers of self-attention upsampling layers. The final decoded CT slice image .

[0044] like Figure 4 As shown, the expression of the gated fusion upsampling layer is: ; in , , is the training parameter matrix, Represents the upsampling process The input of the layer, Represents the downsampling process The output of the layer degradation model, Represents the upsampling process The output of the layer, is the sigmoid function, which produces values ​​between 0 and 1; tanh is the tangent function.

[0045] The last two upsampling layers of the feature decoder both include a self-attention mechanism. After upsampling, the layer features are Figure 3 The attention mechanism of medium width and height is different from the above Transformer in that no position encoding is added here, that is, The upsampled output features are used between. According to formulas (7), (8), (9), we get The output of the layer.

[0046] In this embodiment of the present invention, the total loss function of the generator includes perceptual loss, contextual loss and structural loss, and its formula is as follows: ; is the perceptual loss, which is estimated by and real It is sent to a pre-trained VGG19 for feature extraction, and then the high-level features of the two are compared. The formula is as follows: ; in Represents operations for extracting high-level features.

[0047] It is the context loss, which is used to ensure the consistency of local texture features. Its formula is as follows: ; in is a coefficient used to balance the continuity effect of multi-scale hierarchical features; Indicates selecting multiple hierarchical features.

[0048] is the structural loss, calculated using the learning-perceptual image patch similarity (LPIPS) method, and the results are as follows: .

[0049] The present invention designs a cascaded cross-domain transformer module, which is composed of two cascaded transformers. The first transformer is an attention mechanism for 2D and 3D features in the depth dimension, and the second transformer is an attention mechanism for 2D and 3D features in the height and width dimensions. The cascaded cross-domain transformer is the first attention mechanism module that realizes feature screening and heterogeneous feature fusion for 3D and 2D features.

[0050] The generator of the embodiment of the present invention adopts an encoder-decoder structure. In order to make the generator more effective, a degradation model is introduced in the decoder stage, and a gated fusion mechanism and a self-attention mechanism are introduced in the decoder stage.

[0051] Figure 5 is a comparison diagram of the input MR image, the true value of the CT image corresponding to the MR image, and the synthesized CT image of this embodiment; wherein the first column is the input MR image, the second column is the true value of the CT image corresponding to the MR image, and the third column is the synthesized CT image. Figure 5 It can be seen that the CT image obtained by the method of generating CT images from MR images under multiple condition constraints in the embodiment of the present invention has a consistency of 85-90% compared with the true value, which can be used as a clinical reference.

[0052] Figure 6 is a schematic diagram of a system structure for generating CT images from MR images under multiple condition constraints in an embodiment of the present invention. Figure 1 The method of generating CT images from MR images under multiple condition constraints corresponds to Figure 6 As shown, the system for generating CT images from MR images under multiple condition constraints in this embodiment may include: An image acquisition unit 601 is used to acquire the current patient's MR image and a reference similar case's 3D CT image; A CT image generating unit 602 is used to obtain a CT image corresponding to the current patient's MR image by using a trained generator under the constraints of the current patient's MR image and a reference 3D CT image of a similar case; Among them, in the CT image generation unit, the generator includes a feature encoder, a cascaded cross-domain transformer module and a feature decoder; the feature encoder includes two branches, which respectively sample and encode the current patient MR image and the reference similar case 3D CT image to obtain 2D MRI features and 3D CT features; The cascaded cross-domain transformer module is used to expand the 2D MRI feature by one dimension, combine it with the batch dimension conversion, and then splice it with the 3D CT feature along the depth dimension, merge the spliced ​​features with the batch dimension in turn along the depth dimension and merge them with the width dimension and the height dimension, and then splice a set of trainable parameters with the merged features in the channel dimension to obtain the spliced ​​features; sequentially perform position encoding on the width and height dimensions and position encoding on the depth dimension on the spliced ​​features, pass the features encoded in the depth dimension through the Transformer module of the depth dimension to obtain the first fused feature, pass the first fused feature through the height-width Transformer module to obtain the second fused feature, and use the multi-head attention mechanism to screen the first fused features and the second fused features to obtain the screened fused features; The feature decoder is used to decode the screened fusion features to obtain the corresponding CT image.

[0053] Wherein, in the CT image generation unit, each branch of the feature encoder includes four sampling modules, and the last two sampling modules include a degradation model, and the degradation model includes two independent processes: a blurring process and a noise adding process.

[0054] It should be noted here that Figure 6 The various modules in the system of generating CT images from MR images under multiple conditional constraints in Figure 1 The various steps in the method for generating CT images from MR images under multiple condition constraints in correspond to each other one by one, and the specific implementation process is the same, which will not be repeated here.

[0055] Reference Figure 7 , a schematic diagram of an electronic device is given. It should be noted that, Figure 7 The electronic device 700 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0056] like Figure 7As shown, the electronic device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703. In the RAM 703, various programs and data required for system operation are also stored. The central processing unit 701, the ROM 702 and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0057] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a local area network (LAN) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0058] When the central processing unit 701 in the electronic device of this embodiment executes the program, the following is achieved: Figure 1 The steps in the method of generating a CT image from an MR image under multiple condition constraints are shown.

[0059] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, the computer program including a computer program for executing Figure 1 In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit 701, various functions defined in the apparatus of the present application are executed.

[0060] in, Figure 1 The computer program instructions corresponding to the method shown may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0061] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating CT images from MR images under multiple condition constraints, characterized in that: include: Obtain the current patient's MR image and reference similar case 3D CT images; Based on the constraints of the current patient's MR image and the reference 3D CT image of similar cases, the trained generator is used to obtain the CT image corresponding to the current patient's MR image. The generator includes a feature encoder, a cascaded cross-domain transformer module and a feature decoder; the feature encoder includes two branches, which respectively sample and encode the current patient's MR image and the reference similar case 3D CT image to obtain 2D MRI features and 3D CT features; The cascaded cross-domain transformer module is used to expand the 2D MRI feature by one dimension, combine it with the batch dimension conversion, and then splice it with the 3D CT feature along the depth dimension, merge the spliced ​​features with the batch dimension in turn along the depth dimension and merge them with the width dimension and the height dimension, and then splice a set of trainable parameters with the merged features in the channel dimension to obtain the spliced ​​features; sequentially perform position encoding on the width and height dimensions and position encoding on the depth dimension on the spliced ​​features, pass the features encoded in the depth dimension through the Transformer module of the depth dimension to obtain the first fused feature, pass the first fused feature through the height-width Transformer module to obtain the second fused feature, and use the multi-head attention mechanism to screen the first fused features and the second fused features to obtain the screened fused features; The feature decoder is used to decode the screened fusion features to obtain the corresponding CT image.

2. The method for generating CT images from MR images under multiple condition constraints as claimed in claim 1, characterized in that: Each branch of the feature encoder includes four sampling modules, and the last two sampling modules include a degradation model, which includes two independent processes: a blurring process and a noise adding process.

3. The method for generating CT images from MR images under multiple condition constraints as claimed in claim 2, characterized in that: The expression of the degradation model is: ； in, and represents the Lth layer features and the L+1th layer features, represents the downsampling process, is the blur kernel; is the added noise; Represents the features obtained through the fuzzy process.

4. The method for generating CT images from MR images under multiple condition constraints as claimed in claim 1, characterized in that: In the cascaded cross-domain transformer module, the formula for screening the initial fusion features using the multi-head attention mechanism is: ； in, is the fusion feature after screening; is the height-width-depth global attention feature; MLP is the multi-layer perceptron function.

5. The method for generating CT images from MR images under multiple condition constraints as claimed in claim 1, characterized in that: The feature decoder includes a layer-gated fusion upsampling layer for the first two layers and a self-attention upsampling layer for the last two layers.

6. The method for generating CT images from MR images under multiple condition constraints as claimed in claim 5, characterized in that: The expression of the gated fusion upsampling layer is: ； in , , is the training parameter matrix, Represents the upsampling process The input of the layer, Represents the downsampling process The output of the layer degradation model, Represents the upsampling process The output of the layer, It is the sigmoid function, which produces values ​​between 0 and 1; tanh is the tangent function.

7. A system for generating CT images from MR images under multiple condition constraints, characterized in that: include: An image acquisition unit, which is used to acquire the current patient's MR image and reference similar case 3D CT images; A CT image generation unit, which is used to obtain a CT image corresponding to the current patient's MR image by using a trained generator under the constraints of the current patient's MR image and a reference 3D CT image of a similar case; Among them, in the CT image generation unit, the generator includes a feature encoder, a cascaded cross-domain transformer module and a feature decoder; the feature encoder includes two branches, which respectively sample and encode the current patient MR image and the reference similar case 3D CT image to obtain 2D MRI features and 3D CT features; The cascaded cross-domain transformer module is used to expand the 2D MRI feature by one dimension, combine it with the batch dimension conversion, and then splice it with the 3D CT feature along the depth dimension, merge the spliced ​​features with the batch dimension in turn along the depth dimension and merge them with the width dimension and the height dimension, and then splice a set of trainable parameters with the merged features in the channel dimension to obtain the spliced ​​features; sequentially perform position encoding on the width and height dimensions and position encoding on the depth dimension on the spliced ​​features, pass the features encoded in the depth dimension through the Transformer module of the depth dimension to obtain the first fused feature, pass the first fused feature through the height-width Transformer module to obtain the second fused feature, and use the multi-head attention mechanism to screen the first fused features and the second fused features to obtain the screened fused features; The feature decoder is used to decode the screened fusion features to obtain the corresponding CT image.

8. The system for generating CT images from MR images under multiple condition constraints as claimed in claim 7, characterized in that: In the CT image generation unit, each branch of the feature encoder includes four sampling modules, and the last two sampling modules include a degradation model, and the degradation model includes two independent processes: a blurring process and a noise adding process.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for generating a CT image from an MR image under multiple condition constraints as described in any one of claims 1 to 6 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the method for generating a CT image from an MR image under multiple condition constraints as described in any one of claims 1 to 6 are implemented.

Images