Method and system for generating virtual DWI images of stroke based on multi-scale feature conversion

Abstract

The present invention relates to a method and system for generating a virtual DWI image of stroke based on multi-scale feature conversion, the method comprising acquiring and pairing a CT image and a DWI image of the brain, preprocessing the paired CT image and DWI image, and generating a sample set; constructing a conversion model, and inputting the preprocessed CT image and DWI image in the sample set into the conversion model for training; and inputting the preprocessed target CT image into the trained conversion model to obtain a target virtual DWI image. The present invention preprocesses the paired CT image and the DWI image, and trains the constructed conversion model, extracts different scale feature maps of the target CT image through the encoder of the conversion model, and performs feature conversion processing. In the decoding process, the converted feature map is spliced ​​layer by layer with the feature map of the decoder, and the target virtual DWI image is obtained through iteration, which can accurately capture the lesion information in the CT image and generate a more accurate DWI image.

Description

Technical Field

[0001] The present invention relates to the field of imaging medical technology, and in particular to a method and system for generating a virtual DWI image of a stroke based on multi-scale feature conversion. Background Art

[0002] CT is the preferred imaging tool recommended by international guidelines for acute ischemic stroke, but the early ischemic lesions are very different from normal brain tissue and are often difficult to distinguish with the naked eye within 8 hours of onset. The optimal time window for thrombolytic therapy is only 4.5 hours, and missing the time window can lead to irreversible brain damage. Magnetic resonance imaging, as the gold standard, can show obvious lesions within minutes, but it is not suitable for emergency use due to its defects such as long examination time, expensive examination costs, and metal contraindications. Existing methods have strategies to solve this problem:

[0003] (1) Using MRI as the gold standard, CT is used to segment and annotate ischemic lesions. An AI model is built through CNN and transformer networks to achieve automatic segmentation of ischemic lesions based on CT. However, since early lesions are invisible, it is difficult for doctors to decide whether to accept the results after the AI ​​model gives a conclusion;

[0004] (2) Generate virtual MRI using CT, but other studies can only generate MRI sections for CT sections with lesions (head CT of patients suspected of acute ischemic stroke contains about 30 sections, and only a few sections have ischemic lesions in the acute phase). Using this AI model, doctors need to first determine whether the section contains ischemic lesions, which is difficult for actual clinical application. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for generating a virtual DWI image of stroke based on multi-scale feature conversion in view of the deficiencies of the above-mentioned prior art.

[0006] The technical solution of the present invention to solve the above technical problem is as follows: A method for generating a virtual DWI image of stroke based on multi-scale feature conversion, comprising the following steps:

[0007] Acquire and pair CT images and DWI images of the brain, preprocess the paired CT images and DWI images, and generate a sample set;

[0008] Constructing a conversion model for generating virtual DWI images according to CT images, and inputting the preprocessed CT images and DWI images in the sample set into the conversion model for training;

[0009] The preprocessed target CT image is input into the trained conversion model to obtain the target virtual DWI image.

[0010] The beneficial effects of the present invention are as follows: the method for generating a virtual DWI image of stroke based on multi-scale feature conversion of the present invention preprocesses paired CT images and DWI images, and trains a conversion model based on multi-scale feature conversion, extracts different scale feature maps of the target CT image through an encoder of the conversion model, and performs feature conversion processing on the target CT image. During the decoding process, the converted feature map is spliced ​​layer by layer with the feature map of the decoder, and after iteration, the target virtual DWI image is obtained, which can accurately capture the lesion information in the CT image, thereby generating a more accurate DWI image, and is insensitive to noise.

[0011] On the basis of the above technical solution, the present invention can also be improved as follows:

[0012] Further: the preprocessing of the paired CT image and DWI image specifically includes the following steps:

[0013] Slice the paired CT three-dimensional image and DWI three-dimensional image to obtain a CT two-dimensional image and a DWI two-dimensional image;

[0014] The pixel values ​​of the CT two-dimensional image are cropped according to a preset pixel threshold range, and the cropped CT two-dimensional image and the DWI two-dimensional image are normalized;

[0015] Calculating a mask image of the normalized DWI two-dimensional image using the Otsu algorithm, removing discrete areas in the mask image that are smaller than a preset pixel threshold using a maximum connected domain, and multiplying the mask image with the normalized CT two-dimensional image and the DWI two-dimensional image, respectively, to obtain a preprocessed CT two-dimensional image and a DWI two-dimensional image;

[0016] A sample set is generated based on the preprocessed CT two-dimensional images and DWI two-dimensional images.

[0017] The beneficial effect of the above further scheme is: by slicing the paired CT three-dimensional image and DWI three-dimensional image, the corresponding CT two-dimensional image and DWI two-dimensional image can be obtained, and then the pixel values ​​are cropped, and the CT two-dimensional image and the DWI two-dimensional image are normalized so as to obtain a consistent gray value range in different scanned images, which is convenient for image comparison and analysis. Through denoising, the influence of the skull and other noises in the CT two-dimensional image on the model training can be avoided.

[0018] Further: the step of inputting the preprocessed CT images and DWI images in the sample set into the conversion model for training specifically includes the following steps:

[0019] The conversion model uses the encoder to extract feature maps of different scales from the preprocessed CT two-dimensional image in the input sample set to obtain feature maps of different scales. The feature map extraction formula is as follows:

[0020] E1=LReLU(IN(Con 4×4 (x)))

[0021] E2=LReLU(IN(Con 4×4 (E1)))

[0022] E3=LReLU(IN(Con 4×4 (E2)))

[0023] E4=LReLU(IN(Con 4×4 (E3)))

[0024] Among them, x is the input CT two-dimensional image, Con 4×4 represents a 4×4 convolution operation, IN represents the InstanceNorm2d layer of the encoder, LReLU represents the Leaky ReLU activation function, E1 represents the output feature map of the first layer of the encoder, E2 represents the output feature map of the second layer of the encoder; E3 represents the output feature map of the third layer of the encoder, and E4 represents the output feature map of the fourth layer of the encoder;

[0025] Performing feature conversion processing on the feature maps of different scales respectively to obtain corresponding feature conversion maps;

[0026] The feature conversion map is decoded. During the decoding process, the feature map after feature conversion is concatenated with the decoded features layer by layer, and a virtual DWI image is obtained through iteration.

[0027] A loss function is constructed, and the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image is calculated using the loss function, and the model parameters with the smallest loss value are used as the parameters of the conversion model to complete the training.

[0028] The beneficial effect of the above further scheme is: by extracting feature maps of different scales from the preprocessed CT two-dimensional image, feature maps of different scales are obtained, and then feature conversion processing can be performed on the feature maps of different scales to obtain feature conversion maps, and then decoding and iterative splicing processing is performed to finally obtain a virtual DWI image, and then the final parameters are selected as the recognition parameters of the conversion model according to the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image, so that the conversion model has a higher precision recognition function.

[0029] Further: performing feature conversion processing on the feature maps of different scales to obtain corresponding feature conversion maps specifically includes the following steps:

[0030] The feature graph E1 output by the first layer of the encoder and the feature graph E2 output by the second layer of the conversion model are respectively subjected to a first feature conversion process to obtain conversion feature graphs E1′ and E2′. The first feature conversion formula is as follows:

[0031] E1′=(ReLU(Con 3×3 (E1)))5

[0032] E2′=(ReLU(Con 3×3 (E2)))5

[0033] Among them, Con 3×3 represents a 3×3 convolution operation, ReLU represents the ReLU activation function, (·)5 represents five 3×3 convolution operations and ReLU activations, E1′ and E2′ represent the converted feature maps of the first and second layer output feature maps of the encoder after the first feature transformation, respectively;

[0034] The feature graph E3 output by the third layer of the encoder and the feature graph E4 output by the fourth layer of the conversion model are respectively subjected to the second feature conversion processing to obtain conversion feature graphs E3″′ and E4″′. The second feature conversion formula is as follows:

[0035] E3′=Flatten(E3)

[0036] E3″=(ReLU(Linear(E3′))))5

[0037] E3″′=Reshape(E3″,(b1,c1,h1,w1))

[0038] E4′=Flatten(E4)

[0039] E4″=(ReLU(Linear(E4′))))5

[0040] E4″′=Reshape(E4″,(b2,c2,h2,w2))

[0041] Among them, Flatten means flattening the input feature map E3 into a one-dimensional vector, Linear means a fully connected layer, ReLU means a ReLU activation function, (·)5 means five fully connected operations and ReLU activations, Reshape means reshaping a one-dimensional vector into a specified multidimensional tensor shape, E3′ and E4′ conveniently represent the one-dimensional vectors of the third-layer output feature map and the fourth-layer output feature map of the encoder in the conversion model after flattening, E3″ and E4″ respectively represent the one-dimensional vectors corresponding to the third-layer output feature map and the fourth-layer output feature map after the second feature conversion; (b1, c1, h1, w1) and (b2, c2, h2, w2) respectively represent the batch size, number of channels, height, and width of the feature maps output by the third and fourth layers of the encoder, and E3″′ and E4″′ represent the converted feature maps after shaping.

[0042] The beneficial effect of the above further scheme is: by performing different feature conversion processing on the feature map E1 output by the first layer and the feature map E2 output by the second layer of the encoder of the conversion model and the feature map E3 output by the third layer and the feature map E4 output by the fourth layer, the corresponding feature conversion map is obtained as the basis for the subsequent splicing and restoration of the virtual DWI.

[0043] Further: the step of splicing the feature conversion map with the feature map decoded by the decoder of the conversion model to obtain a virtual DWI image specifically comprises the following steps:

[0044] The decoder of the conversion model decodes the conversion feature maps E3″′, E4″′, E1′ and E2′, and iteratively splices the decoded conversion feature maps to obtain a virtual DWI image. The splicing formula is as follows:

[0045]

[0046] D4=Tanh(Up(Con 4×4 (D3)))

[0047] Among them, ConT 4×4 Represents a 4×4 transposed convolution operation, Con 4×4 represents a 4×4 convolution operation, IN represents an InstanceNorm2d layer, ReLU represents a ReLU activation function, and Up represents an upsampling operation. It indicates that two feature maps are concatenated in the channel dimension, Tanh indicates the Tanh activation function, D1, D2 and D3 respectively represent the output feature maps of the first layer, the second layer and the third layer of the decoder of the conversion model, and D4 is the final generated virtual DWI image.

[0048] The beneficial effect of the above further scheme is that by decoding the conversion feature maps E3″′, E4″′, E1′ and E2′ by the decoder and then performing iterative splicing processing, a virtual DWI image can be restored very accurately as a reference for clinical medical staff, so as to quickly obtain a virtual DWI image based on the CT image in the absence of an MRI image, thereby making a quick and accurate diagnosis and saving the patient to the greatest extent possible.

[0049] Further: the method for generating a virtual DWI image of stroke based on multi-scale feature conversion also includes the following steps:

[0050] The structural similarity value between the virtual DWI image and the paired real DWI image in the sample set is calculated, and the virtual DWI image is evaluated according to the structural similarity value.

[0051] Further: The calculation formula of the structural similarity value is:

[0052]

[0053] Among them, X is the virtual DWI image, Y is the real DWI image, μ X and μ Y represents the mean of the virtual DWI image and the real DWI image, δ X and δ Y represents the variance of the virtual DWI image and the real DWI image, and c1 and c2 are constants.

[0054] The beneficial effect of the above further solution is that by calculating the structural similarity value between the virtual DWI image and the paired real DWI image in the sample set, the similarity between the two images can be evaluated, thereby better simulating the perception of the image.

[0055] The present invention also provides a system for generating virtual DWI images of stroke based on multi-scale feature conversion, including a preprocessing module, a building module and a training conversion module.

[0056] The preprocessing module is used to obtain and pair CT images and DWI images of the brain, preprocess the paired CT images and DWI images, and generate a sample set;

[0057] The construction module constructs a conversion model for generating a virtual DWI image according to a CT image, and inputs the preprocessed CT image and DWI image in the sample set into the conversion model for training;

[0058] The training conversion module inputs the preprocessed target CT image into the trained conversion model to obtain a target virtual DWI image.

[0059] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method for generating a virtual DWI image of a stroke based on multi-scale feature conversion is implemented.

[0060] The present invention also provides a device for generating a virtual DWI image of a stroke based on multi-scale feature conversion, comprising a communication interface, a memory, a communication bus and a processor, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0061] The memory is used to store computer programs;

[0062] The processor is used to implement the steps of the method for generating a virtual DWI image of stroke based on multi-scale feature conversion when executing the program stored in the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of a flow chart of a method for generating a virtual DWI image of a stroke based on multi-scale feature conversion according to an embodiment of the present invention;

[0064] Figure 2 Schematic diagram of comparison between a virtual DWI image obtained by a method for generating a virtual DWI image of stroke based on multi-scale feature conversion according to an embodiment of the present invention and an original DWI image obtained by a conventional Pix2Pix method;

[0065] Figure 3 The present invention is a schematic diagram of the structure of a system for generating virtual DWI images of stroke based on multi-scale feature conversion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0067] like Figure 1 As shown, a method for generating a virtual DWI image of stroke based on multi-scale feature conversion includes the following steps:

[0068] S1: Obtain and pair CT images and DWI images of the brain, preprocess the paired CT images and DWI images, and generate a sample set;

[0069] S2: constructing a conversion model for generating virtual DWI images according to CT images, and inputting the preprocessed CT images and DWI images in the sample set into the conversion model for training;

[0070] S3: Input the preprocessed target CT image into the trained conversion model to obtain the target virtual DWI image.

[0071] The method for generating a virtual DWI image of stroke based on multi-scale feature conversion of the present invention pre-processes paired CT images and DWI images, trains a conversion model based on multi-scale feature conversion, extracts feature maps of different scales of a target CT image through an encoder of the conversion model, and performs feature conversion processing on the target CT image. During the decoding process, the converted feature map is spliced ​​layer by layer with the feature map of the decoder, and after iteration, a target virtual DWI image is obtained, which can accurately capture the lesion information in the CT image, thereby generating a more accurate DWI image, and is insensitive to noise.

[0072] In one or more embodiments of the present invention, the preprocessing of the paired CT image and DWI image specifically includes the following steps:

[0073] S11: Slice the paired CT three-dimensional image and DWI three-dimensional image to obtain a CT two-dimensional image and a DWI two-dimensional image;

[0074] In practice, in order to improve the quality of the training image pairs, in the present invention, the CT two-dimensional images whose brain regions are smaller than a set proportion (eg, 10%) of the entire image area are deleted.

[0075] S12: cropping the pixel values ​​of the CT two-dimensional image according to a preset pixel threshold range, and normalizing the cropped CT two-dimensional image and the DWI two-dimensional image;

[0076] Since CT scans of different patients or CT scans of the same patient at different times may have contrast differences, a consistent grayscale value range can be obtained in different scanned CT images through unified threshold standardization, which facilitates image comparison and analysis. We first crop the pixel values ​​of the CT two-dimensional image to be within the threshold range of [0,80]:

[0077]

[0078] Where I(x, y) is the value of the CT two-dimensional image at the pixel position (x, y), L is the lower threshold, and U is the upper threshold. In the present invention, they are set to 0 and 80 respectively. clip (x,y) is the pixel value after cropping.

[0079] S13: using the Otsu algorithm to calculate the mask image of the normalized DWI two-dimensional image, using the maximum connected domain to remove discrete areas in the mask image that are smaller than a preset pixel threshold, and multiplying the mask image with the normalized CT two-dimensional image and the DWI two-dimensional image, respectively, to obtain a preprocessed CT two-dimensional image and a DWI two-dimensional image;

[0080] Here, the normalized formula is:

[0081]

[0082] in, is the normalized CT two-dimensional image, is the normalized DWI two-dimensional image, x is the preprocessed DWI two-dimensional image, x max and x min is the maximum pixel value and the minimum pixel value of the preprocessed DWI image.

[0083] The calculation formula for multiplying the mask image with the normalized CT two-dimensional image and DWI two-dimensional image is:

[0084]

[0085] in, and They are the final CT image and DWI image after the above preprocessing steps, respectively.

[0086] S14: Generate a sample set according to the preprocessed CT two-dimensional image and DWI two-dimensional image.

[0087] By slicing the paired CT three-dimensional image and DWI three-dimensional image, the corresponding CT two-dimensional image and DWI two-dimensional image can be obtained, and then the pixel values ​​are cropped, and the CT two-dimensional image and the DWI two-dimensional image are normalized so as to obtain a consistent gray value range in different scanned images, which is convenient for image comparison and analysis. Through denoising, the influence of the skull and other noises in the CT two-dimensional image on the model training can be avoided.

[0088] In one or more embodiments of the present invention, the step of inputting the preprocessed CT images and DWI images in the sample set into the conversion model for training specifically includes the following steps:

[0089] S21: The conversion model uses the encoder to extract feature maps of different scales from the preprocessed CT two-dimensional image in the input sample set to obtain feature maps of different scales. The feature map extraction formula is as follows:

[0090] E1=LReLU(IN(Con4×4 (x)))

[0091] E2=LReLU(IN(Con 4×4 (E1)))

[0092] E3=LReLU(IN(Con 4×4 (E2)))

[0093] E4=LReLU(IN(Con 4×4 (E3)))

[0094] Among them, x is the input CT two-dimensional image, Con 4×4 represents a 4×4 convolution operation, IN represents the InstanceNorm2d layer of the encoder, LReLU represents the Leaky ReLU activation function, E1 represents the output feature map of the first layer of the encoder, which is obtained by performing convolution, normalization and activation operations on the input CT two-dimensional image, E2 represents the output feature map of the second layer of the encoder, which is obtained by performing the same operation (convolution, normalization, activation) on E1; E3 represents the output feature map of the third layer of the encoder, which is obtained by performing the same operation on E2, and E4 represents the output feature map of the fourth layer of the encoder, which is obtained by performing the same operation on E3;

[0095] S22: performing feature conversion processing on the feature maps of different scales respectively to obtain corresponding feature conversion maps;

[0096] S23: decoding the feature conversion map. During the decoding process, the feature map after feature conversion is concatenated with the decoded features layer by layer, and a virtual DWI image is obtained through iteration.

[0097] S24: construct a loss function, use the loss function to calculate the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image, and use the model parameters with the smallest loss value as the parameters of the conversion model to complete the training.

[0098] By extracting feature maps of different scales from the preprocessed CT two-dimensional image, feature maps of different scales are obtained, and then feature conversion processing can be performed on the feature maps of different scales to obtain feature conversion maps, which are then decoded and iterated and spliced ​​to finally obtain a virtual DWI image. Then, the final parameters are selected as the recognition parameters of the conversion model based on the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image, so that the conversion model has a higher-precision recognition function.

[0099] In one or more embodiments of the present invention, performing feature conversion processing on the feature maps of different scales to obtain corresponding feature conversion maps specifically includes the following steps:

[0100] S221: Performing a first feature conversion process on the feature graph E1 output by the first layer and the feature graph E2 output by the second layer of the encoder of the conversion model respectively to obtain conversion feature graphs E1′ and E2′. The first feature conversion formula is as follows:

[0101] E1′=(ReLU(Con 3×3 (E1)))5

[0102] E2′=(ReLU(Con 3×3 (E2)))5

[0103] Among them, Con 3×3 represents a 3×3 convolution operation, ReLU represents a ReLU activation function, (·)5 represents five 3×3 convolution operations and ReLU activations, E1′ and E2′ represent the converted feature maps of the first and second layer output feature maps of the encoder respectively after the first feature conversion; here, the first feature conversion process adopts a CNN convolutional neural network.

[0104] S222: Perform a second feature conversion process on the feature map E3 output by the third layer of the encoder and the feature map E4 output by the fourth layer of the encoder of the conversion model to obtain conversion feature maps E3″′ and E4″′. The second feature conversion formula is as follows:

[0105] E3′=Flatten(E3)

[0106] E3″=(ReLU(Linear(E3′))))5

[0107] E3″′=Reshape(E3″,(b1,c1,h1,w1))

[0108] E4′=Flatten(E4)

[0109] E4″=(ReLU(Linear(E4′))))5

[0110] E4″′=Reshape(E4″,(b2,c2,h2,w2))

[0111] Among them, Flatten means flattening the input feature map E3 into a one-dimensional vector, Linear means a fully connected layer, ReLU means a ReLU activation function, (·)5 means five fully connected operations and ReLU activations, Reshape means reshaping a one-dimensional vector into a specified multi-dimensional tensor shape, E3′ and E4′ conveniently represent the one-dimensional vectors of the third-layer output feature map and the fourth-layer output feature map of the encoder in the conversion model after flattening, E3″ and E4″ respectively represent the one-dimensional vectors corresponding to the third-layer output feature map and the fourth-layer output feature map after the second feature conversion; (b1, c1, h1, w1) and (b2, c2, h2, w2) respectively represent the batch size, number of channels, height, and width of the feature maps output by the third and fourth layers of the encoder, and E3″′ and E4″′ represent the converted feature maps after reshaping. Here, the second feature conversion process uses an MLP convolutional neural network.

[0112] By performing different feature conversion processing on the feature map E1 output by the first layer and the feature map E2 output by the second layer of the encoder of the conversion model, the feature map E3 output by the third layer and the feature map E4 output by the fourth layer, the corresponding feature conversion map is obtained as the basis for the subsequent splicing and restoration of the virtual DWI.

[0113] In one or more embodiments of the present invention, the step of splicing the feature conversion map with the feature map decoded by the decoder of the conversion model to obtain a virtual DWI image specifically comprises the following steps:

[0114] S231: The decoder of the conversion model decodes the conversion feature maps E3″′, E4″′, E1′ and E2′, and iteratively splices the decoded conversion feature maps to obtain a virtual DWI image. The splicing formula is as follows:

[0115]

[0116] D4=Tanh(Up(Con 4×4 (D3)))

[0117] Among them, ConT 4×4 Represents a 4×4 transposed convolution operation, Con 4×4 represents a 4×4 convolution operation, IN represents an InstanceNorm2d layer, ReLU represents a ReLU activation function, and Up represents an upsampling operation. It indicates that two feature maps are concatenated in the channel dimension, Tanh indicates the Tanh activation function, D1, D2 and D3 respectively represent the output feature maps of the first layer, the second layer and the third layer of the decoder of the conversion model, and D4 is the final generated virtual DWI image.

[0118] By using the decoder to decode the conversion feature maps E3″′, E4″′, E1′ and E2′ and then performing iterative splicing processing, a virtual DWI image can be restored very accurately as a reference for clinical medical staff, so that a virtual DWI image can be quickly obtained based on a CT image in the absence of an MRI image, thereby making a quick and accurate diagnosis and saving time for saving patients to the maximum extent.

[0119] In one or more embodiments of the present invention, the loss function construction and the calculation of the loss value between the two images in S24 are prior arts and will not be described in detail in the present invention.

[0120] In one or more embodiments of the present invention, the method for generating a virtual DWI image of stroke based on multi-scale feature conversion further includes the following steps:

[0121] S4: Calculate a structural similarity value between the virtual DWI image and a paired real DWI image in the sample set, and evaluate the virtual DWI image according to the structural similarity value.

[0122] Specifically, in one or more embodiments of the present invention, the calculation formula of the structural similarity value is:

[0123]

[0124] Among them, X is the virtual DWI image, Y is the real DWI image, μ X and μ Y represents the mean of the virtual DWI image and the real DWI image, δ X and δ Y represents the variance of the virtual DWI image and the real DWI image, and c1 and c2 are constants.

[0125] By calculating the structural similarity value between the virtual DWI image and the paired real DWI image in the sample set, the similarity between the two images can be evaluated, thereby better simulating the perception of the image.

[0126] In the present invention, the method of the present invention and the Pix2Pix method in the prior art are used to perform calculations, and the results are shown in Table 1 below:

[0127] Table 1

[0128] method SSIM Pix2Pix 0.861 The present invention 0.886

[0129] According to the results in Table 1, it can be seen that the structural similarity value SSIM of the results generated by the method of the present invention has a significant improvement over Pix2Pix. Figure 2It can also be seen that the visual effect of the method of the present invention is better than that of Pix2Pix, and the generation of lesions is more accurate.

[0130] like Figure 3 As shown, the present invention also provides a system for generating virtual DWI images of stroke based on multi-scale feature conversion, including a preprocessing module, a building module and a training conversion module.

[0131] The preprocessing module is used to obtain and pair CT images and DWI images of the brain, preprocess the paired CT images and DWI images, and generate a sample set;

[0132] The construction module constructs a conversion model for generating a virtual DWI image according to a CT image, and inputs the preprocessed CT image and DWI image in the sample set into the conversion model for training;

[0133] The training conversion module inputs the preprocessed target CT image into the trained conversion model to obtain a target virtual DWI image.

[0134] In one or more embodiments of the present invention, the preprocessing module preprocesses the paired CT image and DWI image in the following specific implementations:

[0135] Slice the paired CT three-dimensional image and DWI three-dimensional image to obtain a CT two-dimensional image and a DWI two-dimensional image;

[0136] The pixel values ​​of the CT two-dimensional image are cropped according to a preset pixel threshold range, and the cropped CT two-dimensional image and the DWI two-dimensional image are normalized;

[0137] Calculating a mask image of the normalized DWI two-dimensional image using the Otsu algorithm, removing discrete areas in the mask image that are smaller than a preset pixel threshold using a maximum connected domain, and multiplying the mask image with the normalized CT two-dimensional image and the DWI two-dimensional image, respectively, to obtain a preprocessed CT two-dimensional image and a DWI two-dimensional image;

[0138] A sample set is generated based on the preprocessed CT two-dimensional images and DWI two-dimensional images.

[0139] By slicing the paired CT three-dimensional image and DWI three-dimensional image, the corresponding CT two-dimensional image and DWI two-dimensional image can be obtained, and then the pixel values ​​are cropped, and the CT two-dimensional image and the DWI two-dimensional image are normalized so as to obtain a consistent gray value range in different scanned images, which is convenient for image comparison and analysis. Through denoising, the influence of the skull and other noises in the CT two-dimensional image on the model training can be avoided.

[0140] In one or more embodiments of the present invention, the construction module inputs the preprocessed CT images and DWI images in the sample set into the conversion model for training in a specific manner as follows:

[0141] The conversion model uses the encoder to extract feature maps of different scales from the preprocessed CT two-dimensional image in the input sample set to obtain feature maps of different scales. The feature map extraction formula is as follows:

[0142] E1=LReLU(IN(Con 4×4 (x)))

[0143] E2=LReLU(IN(Con 4×4 (E1)))

[0144] E3=LReLU(IN(Con 4×4 (E2)))

[0145] E4=LReLU(IN(Con 4×4 (E3)))

[0146] Among them, x is the input CT two-dimensional image, Con 4×4 represents a 4×4 convolution operation, IN represents the InstanceNorm2d layer of the encoder, LReLU represents the Leaky ReLU activation function, E1 represents the output feature map of the first layer of the encoder, E2 represents the output feature map of the second layer of the encoder; E3 represents the output feature map of the third layer of the encoder, and E4 represents the output feature map of the fourth layer of the encoder;

[0147] Performing feature conversion processing on the feature maps of different scales respectively to obtain corresponding feature conversion maps;

[0148] The feature conversion map is decoded. During the decoding process, the feature map after feature conversion is concatenated with the decoded features layer by layer, and a virtual DWI image is obtained through iteration.

[0149] A loss function is constructed, and the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image is calculated using the loss function, and the model parameters with the smallest loss value are used as the parameters of the conversion model to complete the training.

[0150] By extracting feature maps of different scales from the preprocessed CT two-dimensional image, feature maps of different scales are obtained, and then feature conversion processing can be performed on the feature maps of different scales to obtain feature conversion maps, which are then decoded and iterated and spliced ​​to finally obtain a virtual DWI image. Then, the final parameters are selected as the recognition parameters of the conversion model based on the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image, so that the conversion model has a higher-precision recognition function.

[0151] In one or more embodiments of the present invention, the construction module performs feature conversion processing on the feature maps of different scales respectively to obtain the corresponding feature conversion maps as follows:

[0152] The feature graph E1 output by the first layer of the encoder and the feature graph E2 output by the second layer of the conversion model are respectively subjected to a first feature conversion process to obtain conversion feature graphs E1′ and E2′. The first feature conversion formula is as follows:

[0153] E1′=(ReLU(Con 3×3 (E1)))5

[0154] E2′=(ReLU(Con 3×3 (E2)))5

[0155] Among them, Con 3×3 represents a 3×3 convolution operation, ReLU represents the ReLU activation function, (·)5 represents five 3×3 convolution operations and ReLU activations, E1′ and E2′ represent the converted feature maps of the first and second layer output feature maps of the encoder after the first feature transformation, respectively;

[0156] The feature graph E3 output by the third layer of the encoder and the feature graph E4 output by the fourth layer of the conversion model are respectively subjected to the second feature conversion processing to obtain conversion feature graphs E3″′ and E4″′. The second feature conversion formula is as follows:

[0157] E3′=Flatten(E3)

[0158] E3″=(ReLU(Linear(E3′))))5

[0159] E3″′=Reshape(E3″,(b1,c1,h1,w1))

[0160] E4′=Flatten(E4)

[0161] E4″=(ReLU(Linear(E4′))))5

[0162] E4″′=Reshape(E4″,(b2,c2,h2,w2))

[0163] Among them, Flatten means flattening the input feature map E3 into a one-dimensional vector, Linear means a fully connected layer, ReLU means a ReLU activation function, (·)5 means five fully connected operations and ReLU activations, Reshape means reshaping a one-dimensional vector into a specified multidimensional tensor shape, E3′ and E4′ conveniently represent the one-dimensional vectors of the third-layer output feature map and the fourth-layer output feature map of the encoder in the conversion model after flattening, E3″ and E4″ respectively represent the one-dimensional vectors corresponding to the third-layer output feature map and the fourth-layer output feature map after the second feature conversion; (b1, c1, h1, w1) and (b2, c2, h2, w2) respectively represent the batch size, number of channels, height, and width of the feature maps output by the third and fourth layers of the encoder, and E3″′ and E4″′ represent the converted feature maps after shaping.

[0164] By performing different feature conversion processing on the feature map E1 output by the first layer and the feature map E2 output by the second layer of the encoder of the conversion model, the feature map E3 output by the third layer and the feature map E4 output by the fourth layer, the corresponding feature conversion map is obtained as the basis for the subsequent splicing and restoration of the virtual DWI.

[0165] In one or more embodiments of the present invention, the construction module splices the feature conversion map with the feature map decoded by the decoder of the conversion model to obtain a virtual DWI image, which specifically includes the following steps:

[0166] The decoder of the conversion model decodes the conversion feature maps E3″′, E4″′, E1′ and E2′, and iteratively splices the decoded conversion feature maps to obtain a virtual DWI image. The splicing formula is as follows:

[0167]

[0168] D4=Tanh(Up(Con 4×4 (D3)))

[0169] Among them, ConT 4×4 Represents a 4×4 transposed convolution operation, Con 4×4 represents a 4×4 convolution operation, IN represents an InstanceNorm2d layer, ReLU represents a ReLU activation function, and Up represents an upsampling operation. It indicates that two feature maps are concatenated in the channel dimension, Tanh indicates the Tanh activation function, D1, D2 and D3 respectively represent the output feature maps of the first layer, the second layer and the third layer of the decoder of the conversion model, and D4 is the final generated virtual DWI image.

[0170] By using the decoder to decode the conversion feature maps E3″′, E4″′, E1′ and E2′ and then performing iterative splicing processing, a virtual DWI image can be restored very accurately as a reference for clinical medical staff, so that a virtual DWI image can be quickly obtained based on a CT image in the absence of an MRI image, thereby making a quick and accurate diagnosis and saving patients to the greatest extent possible.

[0171] The present invention also provides a system for generating a virtual DWI image of stroke based on multi-scale feature conversion, which also includes an evaluation module for calculating a structural similarity value between the virtual DWI image and a paired real DWI image in the sample set, and evaluating the virtual DWI image according to the structural similarity value; the calculation formula of the structural similarity value is:

[0172]

[0173] Among them, X is the virtual DWI image, Y is the real DWI image, μ X and μ Y represents the mean of the virtual DWI image and the real DWI image, δ X and δ Y represents the variance of the virtual DWI image and the real DWI image, and c1 and c2 are constants.

[0174] The present invention also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method for generating a virtual DWI image of a stroke based on multi-scale feature conversion is implemented.

[0175] The present invention also provides a device for generating a virtual DWI image of a stroke based on multi-scale feature conversion, comprising a communication interface, a memory, a communication bus and a processor, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0176] The memory is used to store computer programs;

[0177] The processor is used to implement the steps of the method for generating a virtual DWI image of stroke based on multi-scale feature conversion when executing the program stored in the memory.

[0178] The method and system for generating a virtual DWI image of stroke based on multi-scale feature conversion of the present invention constructs a conversion module, extracts different scale feature maps of a target CT image through an encoder, performs feature conversion processing, decodes the obtained feature conversion map, and performs iterative splicing to obtain a target virtual DWI image. In this way, lesion information can be captured more accurately, thereby generating a more accurate DWI image.

[0179] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating a virtual DWI image of stroke based on multi-scale feature conversion, characterized in that: The steps include: Acquire and pair CT images and DWI images of the brain, preprocess the paired CT images and DWI images, and generate a sample set; Constructing a conversion model for generating virtual DWI images according to CT images, and inputting the preprocessed CT images and DWI images in the sample set into the conversion model for training; The preprocessed target CT image is input into the trained conversion model to obtain the target virtual DWI image; The step of inputting the preprocessed CT images and DWI images in the sample set into the conversion model for training specifically includes the following steps: The conversion model uses the encoder to extract feature maps of different scales from the preprocessed CT two-dimensional image in the input sample set to obtain feature maps of different scales. The feature map extraction formula is as follows: E1=LReLU(IN(Con 4×4 (x))) <h2 style=";text-align:left;direction:ltr">E2 = LReLU(IN(Con<h2 style=";text-align:left;direction:ltr"> 4×4 <h2 style=";text-align:left;direction:ltr"> (E1))) <h2 style=";text-align:left;direction:ltr">E3 = LReLU(IN(Con<h2 style=";text-align:left;direction:ltr"> 4×4 <h2 style=";text-align:left;direction:ltr"> (E2))) E4=LReLU(IN(Con 4×4 (E3))) Among them, x is the input CT two-dimensional image, Con 4×4 represents a 4×4 convolution operation, IN represents the InstanceNorm2d layer of the encoder, LReLU represents the Leaky ReLU activation function, E1 represents the output feature map of the first layer of the encoder, E2 represents the output feature map of the second layer of the encoder; E3 represents the output feature map of the third layer of the encoder, and E4 represents the output feature map of the fourth layer of the encoder; Performing feature conversion processing on the feature maps of different scales respectively to obtain corresponding feature conversion maps; The feature conversion map is decoded. During the decoding process, the feature map after feature conversion is concatenated with the decoded features layer by layer, and a virtual DWI image is obtained through iteration. A loss function is constructed, and the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image is calculated using the loss function, and the model parameters with the smallest loss value are used as the parameters of the conversion model to complete the training.

2. The method for generating a virtual DWI image of stroke based on multi-scale feature conversion according to claim 1, characterized in that: The preprocessing of the paired CT image and DWI image specifically includes the following steps: Slice the paired CT three-dimensional image and DWI three-dimensional image to obtain a CT two-dimensional image and a DWI two-dimensional image; The pixel values ​​of the CT two-dimensional image are cropped according to a preset pixel threshold range, and the cropped CT two-dimensional image and the DWI two-dimensional image are normalized; Calculating a mask image of the normalized DWI two-dimensional image using the Otsu algorithm, removing discrete areas in the mask image that are smaller than a preset pixel threshold using a maximum connected domain, and multiplying the mask image with the normalized CT two-dimensional image and the DWI two-dimensional image, respectively, to obtain a preprocessed CT two-dimensional image and a DWI two-dimensional image; A sample set is generated based on the preprocessed CT two-dimensional images and DWI two-dimensional images.

3. The method for generating a virtual DWI image of stroke based on multi-scale feature conversion according to claim 1, characterized in that: The step of performing feature conversion processing on the feature maps of different scales to obtain corresponding feature conversion maps specifically includes the following steps: The feature graph E1 output by the first layer of the encoder and the feature graph E2 output by the second layer of the conversion model are respectively subjected to a first feature conversion process to obtain conversion feature graphs E1′ and E2′. The first feature conversion formula is as follows: E1′=(ReLU(Con 3×3 (E1)))5 <h2 style=";text-align:left;direction:ltr">E2′ = (ReLU(Con<h2 style=";text-align:left;direction:ltr"> 3×3 <h2 style=";text-align:left;direction:ltr"> (E2)))5 Among them, Con 3×3 represents a 3×3 convolution operation, ReLU represents the ReLU activation function, (·)5 represents five 3×3 convolution operations and ReLU activations, E1′ and E2′ represent the converted feature maps of the first and second layer output feature maps of the encoder after the first feature transformation, respectively; The feature graph E3 output by the third layer of the encoder and the feature graph E4 output by the fourth layer of the conversion model are respectively subjected to the second feature conversion processing to obtain conversion feature graphs E3″′ and E4″′. The second feature conversion formula is as follows: E3′=Flatten(E3) E3″=(ReLU(Linear(E3′))))5 E3″′=Reshape(E3″,(b1,c1,h1,w1)) E4′=Flatten(E4) E4″=(ReLU(Linear(E4′))))5 E4″′=Reshape(E4″,(b2,c2,h2,w2)) Among them, Flatten means flattening the input feature map E3 into a one-dimensional vector, Linear means a fully connected layer, ReLU means a ReLU activation function, (·)5 means five fully connected operations and ReLU activations, Reshape means reshaping a one-dimensional vector into a specified multidimensional tensor shape, E3′ and E4′ conveniently represent the one-dimensional vectors of the third-layer output feature map and the fourth-layer output feature map of the encoder in the conversion model after flattening, E3″ and E4″ respectively represent the one-dimensional vectors corresponding to the third-layer output feature map and the fourth-layer output feature map after the second feature conversion; (b1, c1, h1, w1) and (b2, c2, h2, w2) respectively represent the batch size, number of channels, height, and width of the feature maps output by the third and fourth layers of the encoder, and E3″′ and E4″′ represent the converted feature maps after shaping.

4. The method for generating a virtual DWI image of stroke based on multi-scale feature conversion according to claim 3, characterized in that: The step of splicing the feature conversion map with the feature map decoded by the decoder of the conversion model to obtain a virtual DWI image specifically comprises the following steps: The decoder of the conversion model decodes the conversion feature maps E3″′, E4″′, E1′ and E2′, and iteratively splices the decoded conversion feature maps to obtain a virtual DWI image. The splicing formula is as follows: D4=Tanh(Up(Con 4×4 (D3))) Among them, ConT 4×4 Represents a 4×4 transposed convolution operation, Con 4×4 represents a 4×4 convolution operation, IN represents an InstanceNorm2d layer, ReLU represents a ReLU activation function, and Up represents an upsampling operation. It indicates that two feature maps are concatenated in the channel dimension, Tanh indicates the Tanh activation function, D1, D2 and D3 respectively represent the output feature maps of the first layer, the second layer and the third layer of the decoder of the conversion model, and D4 is the final generated virtual DWI image.

5. The method for generating a virtual DWI image of stroke based on multi-scale feature conversion according to claim 1, characterized in that: The following steps are also included: The structural similarity value between the virtual DWI image and the paired real DWI image in the sample set is calculated, and the virtual DWI image is evaluated according to the structural similarity value.

6. The method for generating a virtual DWI image of stroke based on multi-scale feature conversion according to claim 5, characterized in that: The calculation formula of the structural similarity value is: Among them, X is the virtual DWI image, Y is the real DWI image, μ X and μ Y represents the mean of the virtual DWI image and the real DWI image, δ X and δ Y represents the variance of the virtual DWI image and the real DWI image, and c1 and c2 are constants.

7. A system for generating virtual DWI images of stroke based on multi-scale feature conversion, characterized in that: Including preprocessing module, construction module and training conversion module, The preprocessing module is used to obtain and pair CT images and DWI images of the brain, preprocess the paired CT images and DWI images, and generate a sample set; The construction module constructs a conversion model for generating a virtual DWI image according to a CT image, and inputs the preprocessed CT image and DWI image in the sample set into the conversion model for training; The training conversion module inputs the preprocessed target CT image into the trained conversion model to obtain a target virtual DWI image; The specific implementation of the construction module inputting the preprocessed CT images and DWI images in the sample set into the conversion model for training is as follows: The conversion model uses the encoder to extract feature maps of different scales from the preprocessed CT two-dimensional image in the input sample set to obtain feature maps of different scales. The feature map extraction formula is as follows: E1=LReLU(IN(Con 4×4 (x))) <h2 style=";text-align:left;direction:ltr">E2 = LReLU(IN(Con<h2 style=";text-align:left;direction:ltr"> 4×4 <h2 style=";text-align:left;direction:ltr"> (E1))) <h2 style=";text-align:left;direction:ltr">E3 = LReLU(IN(Con<h2 style=";text-align:left;direction:ltr"> 4×4 <h2 style=";text-align:left;direction:ltr"> (E2))) E4=LReLU(IN(Con 4×4 (E3))) Among them, x is the input CT two-dimensional image, Con 4×4 represents a 4×4 convolution operation, IN represents the InstanceNorm2d layer of the encoder, LReLU represents the Leaky ReLU activation function, E1 represents the output feature map of the first layer of the encoder, E2 represents the output feature map of the second layer of the encoder; E3 represents the output feature map of the third layer of the encoder, and E4 represents the output feature map of the fourth layer of the encoder; Performing feature conversion processing on the feature maps of different scales respectively to obtain corresponding feature conversion maps; The feature conversion map is decoded. During the decoding process, the feature map after feature conversion is concatenated with the decoded features layer by layer, and a virtual DWI image is obtained through iteration. A loss function is constructed, and the loss value between the virtual DWI image and the preprocessed DWI two-dimensional image is calculated using the loss function, and the model parameters with the smallest loss value are used as the parameters of the conversion model to complete the training.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for generating a virtual DWI image of stroke based on multi-scale feature conversion according to any one of claims 1 to 6 is implemented.

9. A device for generating virtual DWI images of stroke based on multi-scale feature conversion, characterized in that: It includes a communication interface, a memory, a communication bus and a processor, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the steps of the method for generating a virtual DWI image of stroke based on multi-scale feature conversion as described in any one of claims 1 to 6 when executing the program stored in the memory.

Images