Contrast agent-free myocardial infarction image segmentation method, system and device

Abstract

The application discloses a contrast agent-free myocardial infarction image segmentation method, system and device, which comprises collecting myocardial infarction magnetic resonance image data, extracting features from the image, and deeply fusing the obtained features through a capsule network module; edge feature extraction modules are utilized to sufficiently learn the edge information of the myocardial infarction region; a decoder is constructed through a convolution method to recover the features of the encoder layer by layer, and the results of edge detection are spliced with the results of the recovered image features in the decoder to obtain a segmentation result; and the real-time collected contrast agent-free contrast-enhanced myocardial infarction data is sent into the saved model to obtain a segmentation result image. The application pays attention to the local and spatial information of the image through the convolution structure and the Transformer, and provides more comprehensive and specific image features; the edge information of the myocardial infarction region is learned through the multi-task edge detection module, and the edge features are displayed and corrected, so that the segmentation accuracy of the overall region is improved.

Description

Technical Field

[0001] This invention relates to the field of digital image processing and pattern recognition technology, specifically to a method, system, and apparatus for image segmentation of myocardial infarction without contrast agents. Background Technology

[0002] Myocardial infarction is one of the leading causes of death. It occurs when coronary artery disease leads to a sharp reduction or interruption of coronary blood supply, resulting in severe and persistent acute ischemic necrosis of the corresponding myocardium. Segmentation and early detection of the infarcted area are crucial; lesion segmentation is a key pretreatment step for detection, staging, and postoperative treatment. Segmentation allows for the division of the infarcted area to predict the recovery of dysfunctional segments in chronic ischemic heart disease or to select appropriate treatment options. It also allows for the estimation of the presence, location, and transmural extent of acute and chronic myocardial infarction.

[0003] Current clinical methods still face high risks and are time-consuming. The standard clinical approach primarily involves two steps: 1) delayed-contrast imaging using magnetic resonance imaging (MRI) and gadolinium contrast agents; and 2) manual segmentation and quantification of relevant myocardial infarction markers from the delayed-contrast imaging data. The use of gadolinium contrast agents in these steps can cause side effects such as headache, nausea, and vomiting in patients. Furthermore, due to individual differences in metabolic rate, the time it takes for the contrast agent to be eliminated from the body varies, and it may even harm the kidneys. Therefore, an increasing number of studies are using contrast-free, non-contrast-enhanced MRI as a segmentation and detection tool.

[0004] However, segmentation of myocardial infarction areas on contrast-free images still faces significant challenges: 1) the size, shape, location, and appearance of myocardial infarction areas vary greatly, leading to inaccurate segmentation; 2) low contrast or blurred boundaries between tissues at the boundaries can affect the segmentation results. Therefore, there is an urgent need to develop accurate, reliable, and automated techniques for accurate segmentation of myocardial infarction areas using contrast-free MRI scans, which would aid clinicians in diagnosis and treatment planning. Summary of the Invention

[0005] The present invention provides a contrast-free myocardial infarction image segmentation method, device and storage medium, which can at least solve one of the technical problems in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A contrast-free myocardial infarction image segmentation method includes the following steps:

[0008] Step 1: Collect magnetic resonance imaging data of myocardial infarction with and without gadolinium contrast enhancement, and perform corresponding annotations. Obtain the enhanced training set and validation set through data preprocessing.

[0009] Step 2: Input the processed data into the feature extraction network, extract features through the dense connection module and the Transformer model, and then use the capsule network module to deeply fuse the obtained features;

[0010] Step 3: Using the intermediate features from Step 1 as the source of edge information, the edge feature extraction module is used to learn them fully, thereby obtaining the edge region of the myocardial infarction area;

[0011] Step 4: Construct a decoder using a conventional convolution method to recover the features of the encoder layer by layer, and concatenate the final result with the output obtained in Step 3 to obtain the segmentation result, and save the model with the best performance in the validation set.

[0012] Step 5: Input the real-time acquired gadolinium-free contrast-enhanced myocardial infarction data into the saved model to obtain the segmentation result map.

[0013] This invention utilizes a Generative Adversarial Network (GAN) to process initial myocardial infarction data with and without gadolinium-enhanced imaging, achieving data augmentation. Then, in the decoder section of the network, the superior feature learning capabilities of dense connections and the long-range modeling capabilities of the Transformer effectively learn the local and spatial information of the image data. Finally, the obtained local and spatial information is deeply fused using a capsule network.

[0014] Furthermore, step one includes the following steps:

[0015] S11: Collect image data of myocardial infarction with and without gadolinium contrast enhancement, as well as corresponding pixel-level annotation data;

[0016] S12: All data augmentation is performed to obtain the augmented data. This invention employs a comprehensive data augmentation method, combining basic data augmentation methods with GAN methods. By generating high-quality data with diversity and robustness, it improves the segmentation accuracy and robustness of the segmentation network. This method not only increases the amount of training data but also improves the model's generalization ability and robustness, thus achieving better results in practical applications.

[0017] Specifically, the basic data augmentation methods used in this invention include the following three: 1) Random cropping: randomly cropping a portion of the original image and then scaling it to a specified size. 2) Random rotation: randomly rotating the original image... Figure 13) Random scaling: Randomly scale the original image to a specified size. This invention uses the GAN-based data augmentation method DAGAN, which generates more data by learning fine-grained features of the image. Specifically, regularization terms are used to constrain the training process of the generator and discriminator, enabling them to better learn the fine-grained features of the image.

[0018] S13: Normalize the enhanced image data so that the mean of all images is 0 and the standard deviation is 1, thus ensuring that the image grayscale distribution follows a normal distribution. The normalization formula is shown below:

[0019]

[0020] Where x is the gray value of the pixel to be processed, μ is the average gray value of all pixels in the entire image, and σ is the standard deviation of the gray values ​​of all pixels in the entire image. It is the grayscale value of the pixel after processing.

[0021] Furthermore, step two specifically includes the following steps:

[0022] S21: Local feature extraction from MRI images of myocardial infarction is performed using densely connected coding blocks. The input image size is H×W. Initial convolutional blocks are used before the first encoder layer. Each small convolutional block includes a convolutional layer (Conv), batch normalization (BN), and a linear ReLU layer, resulting in an initial feature map of size H×W×64 with 64 channels. Densely connected modules are then used to enhance feature learning. Finally, 2×2 convolutions with a stride of 2 are used for downsampling, compressing the feature map to a size of H×W×64. By combining dense connection modules and downsampling four times, a feature map containing more image information is obtained, with a size of [size missing].

[0023] Output=ReLU(BN(Conv(Input)))

[0024] S22: The feature maps obtained from the image data are serialized into input, and features are learned from this sequence based on the SwinTransformer encoding blocks. The input image is divided into N image blocks through convolution operations and flattened into a sequence, resulting in a sequence of size [size missing]. The initial sequence is then transformed linearly through convolution in the linear embedding layer, mapping it to a space of dimension C, with a sequence size of [value missing]. Next, after downsampling and the SwinT encoding module, the result is obtained. The feature sequence; the final feature sequence obtained through repeated operations is of size . The formula for the SwingTransformer coded block is shown below:

[0025] z l =MLP(LN(W_MSA(LN(z) l-1 ))+z l-1 ))+W_MSA(LN(z l-1 ))+z l-1 ,

[0026] z l+1 =MLP(LN(SW_MSA(LN(z) l ))+z l ))+SW_MSA(LN(z l ))+z l

[0027] In the above formula, z l-1 These are features of the input layer, z l z is the output feature of the intermediate layer. l+1 These are the final output features. MLP(*) represents a multilayer perceptron layer, LN(*) represents a normalization layer, and W_MSA(*) and SW_MSA(*) represent window-based multi-head self-attention mechanism and window displacement-based multi-head self-attention mechanism, respectively.

[0028] S23: The capsule network module aggregates the feature maps from channels S21 and S22 to capture long-range dependencies between parts and the whole, enhancing feature learning. The capsule network uses vectors to represent a set of neurons, representing not only the spatial relationships between features but also the probability of feature existence. Simultaneously, a dynamic routing algorithm iteratively calculates the weights of each forward pass to adjust the contribution of the part to the whole, thus better modeling the relationship between the local and the global. Before entering the capsule network module, the output features from the dense connection module and the Transformer module are fused to obtain the input feature map for this module, with a size of [size missing]. Then reshape as Where N = 8 is the number of capsules, and D = 64 is the dimension of each capsule; after 3 routing iterations, the resulting size is... The output features are then reshaped before being passed to the decoding module. The feature map is shown. Throughout the capsule module, the kernel size is 3×3 with a stride of 1. The formula for the capsule network is shown below:

[0029]

[0030] In the above formula, take It is the set of capsule blocks in the l-th layer. It is the set of capsule blocks in the (l+1)th layer, W ij To obtain the learned weight matrix, the capsule block of layer l is... Linear mapping to capsule blocks in layer l+1 r ij It is the coupling coefficient between capsules i and j, assigned according to the dynamic routing allocation algorithm.

[0031] Furthermore, step three specifically includes the following steps:

[0032] S31: The four-layer features in the encoding stage are used as the source of edge information. The low-level features provide local edge information, while the high-level features have a larger receptive field. The edge information of the region to be segmented is obtained through feature fusion operation. This process uses a multi-scale edge feature extraction module, a multi-head self-attention mechanism, and a cross-attention mechanism to extract, analyze, and transmit edge information.

[0033] S32: The multi-scale edge feature module aims to explore the multi-scale information of each layer of feature map, thereby obtaining features with more edge discrimination; the multi-head self-attention mechanism aims to explore the spatial information of the feature map; and the cross-attention mechanism can connect features of different scales.

[0034] Furthermore, step four specifically includes the following steps:

[0035] S41: The decoder is constructed through convolution and deconvolution in upsampling. During the encoding stage, the feature map is enlarged, the number of channels is continuously halved, and the size is doubled. Skip connections are used to concatenate the feature map with the corresponding low-level feature map in the channel dimension until the feature map is restored to the same size as the input feature map.

[0036] S42: The outputs of the decoder and the edge detector are concatenated to form the final segmentation result. The entire network is trained using a joint loss function, and the model that performs best on the validation set is saved. The formula for the relevant loss function is as follows:

[0037] L Total =L Seg +L Edge

[0038] L seg =L T

[0039] L Edge =λ1L Dice +λ2L BCE

[0040]

[0041]

[0042] L BCE =-w*(target*ln(pt)+(1-target)*ln(1-pt))

[0043] In the above formula, l T The loss function addresses the segmentation problem and adaptively learns the optimal tolerance level for label noise during backpropagation. The loss function for edge segmentation is defined by L... Dice and L BCE The loss function L is composed of two hyperparameters, λ1 and λ2. Dice middle, y true and y pred These represent the true value and the predicted value, respectively; θ represents a very small value to prevent the denominator from being zero; the loss function L... BCE In this context, target and pt represent the actual value and the predicted value, respectively, and w represents the weight.

[0044] On the other hand, the present invention also discloses a contrast-free myocardial infarction image segmentation system. The image segmentation system uses the contrast-free myocardial infarction image segmentation method described above to perform semantic segmentation on the acquired image, thereby obtaining the segmentation prediction result of the target region.

[0045] The myocardial infarction image segmentation system includes: a data augmentation module, a feature extraction module, an edge feature extraction module, and a feature decoding module. The feature extraction module includes a convolutional feature extraction module, a Swin Transformer feature extraction module, and a capsule feature fusion module.

[0046] The data augmentation module is used to augment the medical images to be segmented using a GAN network, random cropping, random rotation, and random scaling, and to normalize and adjust the pixel distribution of the images.

[0047] The feature extraction module consists of three parts: a convolutional feature extraction module, a Swin Transformer feature extraction module, and a capsule feature fusion module.

[0048] The convolutional feature extraction module uses a densely connected network to perform local feature extraction on the input image, and then uses a 2×2 convolution with a stride of 2 to perform downsampling to finally obtain the convolutional features.

[0049] The Swin Transformer feature extraction module extracts features layer by layer from the serialized feature map of the input image, performs downsampling using convolution with a stride of 2, and finally obtains the Swin Transformer features.

[0050] The capsule feature fusion module performs deep fusion of the output convolutional features and Swing Transformer features to obtain the final output of the feature extraction module;

[0051] The edge feature extraction module learns edge features from the output features of the intermediate layers of the convolutional feature extraction module. Low-level features aim to provide local edge information, while high-level features have a larger receptive field and are more spatially invariant. The edge information is learned by using a multi-scale boundary feature extraction module, multi-head self-attention, and cross-attention.

[0052] The feature decoding module receives the fused features output by the feature extraction module, and restores the size of the feature map through layer-by-layer convolution and upsampling. The skip connections transmit semantic features with greater layer-level discriminative power to the upsampling network. The output of the decoding module and the output of the edge feature extraction module are fused to obtain the final segmentation result.

[0053] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0054] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0055] As can be seen from the above technical solutions, the purpose of this invention is to provide a method, system, and apparatus for contrast-free myocardial infarction image segmentation based on CNN and Transformer. By using convolutional structures and Transformer to focus on the local and spatial information of the image, more comprehensive and specific image features are provided; a multi-task edge detection module is used to learn the edge information of the myocardial infarction region, explicitly correcting edge features, thereby improving the overall segmentation accuracy of the region.

[0056] The contrast-free myocardial infarction image segmentation method and system of the present invention includes the following steps: S1: Data preparation stage: collect MRI image data of myocardial infarction without gadolinium contrast, add corresponding annotations to the collected data, and perform data processing to obtain data-enhanced training and validation sets; S2: Feature encoding stage: input the processed data into the feature extraction network, perform feature extraction through a dense connection module and a Transformer model, and then use a capsule network module to deeply fuse the obtained features; S3: Edge detection stage: use the intermediate features in the feature extraction stage as the source of edge information, and use the edge feature extraction module to fully learn it, thereby obtaining the edge region of the myocardial infarction area; S4: Feature decoding stage: construct a decoder through a common convolution method, combine skip connections to recover the features of the encoder layer by layer, and concatenate the results of the edge detection stage with the results of the image features recovered in the decoder to obtain the segmentation result, and save the model with the best performance in the validation set; S5: Real-time testing stage: send the real-time acquired gadolinium-free contrast-enhanced myocardial infarction data into the saved model to obtain the segmentation result image.

[0057] The contrast-free myocardial infarction image segmentation method and system of this invention combine basic data augmentation methods with GAN-based data augmentation methods to generate a high-quality dataset with diversity and robustness, thereby improving the segmentation effect. A densely connected encoding network can fully extract local information from the image data and effectively alleviate the problems of gradient vanishing and model degradation. Then, a Transformer encoding path is used to model the spatial information of the image, effectively solving the problem of long-distance dependencies. Finally, a capsule network module is used to deeply integrate the output features of the previous two stages, enhancing feature learning. In addition, the edge feature module of this invention can effectively learn the edge information of the myocardial infarction region, thereby explicitly correcting the segmentation results and improving the overall network segmentation performance. Attached Figure Description

[0058] Figure 1 This is a flowchart of an embodiment of the present invention;

[0059] Figure 2 This is an overall architecture diagram of an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram of the multi-scale boundary feature extraction module structure according to an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the multi-head self-attention mechanism structure according to an embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of the cross-attention mechanism structure according to an embodiment of the present invention;

[0063] Figure 6 This is a schematic diagram of the feature aggregation module structure according to an embodiment of the present invention;

[0064] Figure 7 This is a schematic diagram of the feature selection module structure according to an embodiment of the present invention;

[0065] Figure 8 This is a schematic diagram of the structure of an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0067] Example 1

[0068] This embodiment provides an angiography-free myocardial infarction image segmentation method based on CNN and Transformer. The entire process and overall structure are as follows: Figure 1 and Figure 2 As shown, the method includes the following steps:

[0069] S1. Data preparation stage: Collect magnetic resonance imaging data of myocardial infarction without gadolinium contrast enhancement, and perform corresponding annotations. Obtain the data-enhanced training set and validation set through data processing.

[0070] S2, Feature Extraction Stage: The processed data is input into the feature extraction network, and features are extracted through the dense connection module and the Transformer model. Then, the capsule network module is used to deeply fuse the obtained features.

[0071] S3, Edge Detection Stage: The intermediate features in the feature extraction stage are used as the source of edge information. The edge feature extraction module is used to learn the edge information fully, thereby obtaining the edge region of the myocardial infarction area.

[0072] S4. Feature recovery stage: A decoder is constructed using convolutional methods to recover the features of the encoder layer by layer. The results of the edge detection stage are then concatenated with the results of the image features recovered in the decoder to obtain the segmentation result, which is then saved as the model with the best performance in the validation set.

[0073] S5. Method Validation Stage: The real-time acquired gadolinium-free contrast-enhanced myocardial infarction data is fed into the saved model to obtain the segmentation result map.

[0074] Furthermore, the step S1 includes the following steps:

[0075] S11: Collect image data of myocardial infarction with and without gadolinium contrast enhancement, as well as corresponding pixel-level annotation data;

[0076] S12: Perform data augmentation on all image data to obtain augmented data. This invention employs a comprehensive data augmentation method that combines basic data augmentation methods with GAN methods. By generating high-quality data with diversity and robustness, it improves the segmentation accuracy and robustness of the segmentation network. This method not only increases the amount of training data but also improves the model's generalization ability and robustness, thus achieving better results in practical applications.

[0077] Specifically, the basic data augmentation methods used in this invention include the following three: 1) Random cropping: randomly cropping a portion of the original image and then scaling it to a specified size. 2) Random rotation: randomly rotating the original image... Figure 1 3) Random scaling: Randomly scale the original image to a specified size. This invention uses the GAN-based data augmentation method DAGAN, which generates more data by learning fine-grained features of the image. Specifically, regularization terms are used to constrain the training process of the generator and discriminator, enabling them to better learn the fine-grained features of the image.

[0078] S13: Normalize the enhanced image data so that the mean of all images is 0 and the standard deviation is 1, thus ensuring that the image grayscale distribution follows a normal distribution. The normalization formula is shown below:

[0079]

[0080] Where x is the gray value of the pixel to be processed, μ is the average gray value of all pixels in the entire image, and σ is the standard deviation of the gray values ​​of all pixels in the entire image. It is the grayscale value of the pixel after processing.

[0081] Furthermore, step S2 includes the following steps:

[0082] S21: Local feature extraction from MRI images of myocardial infarction is performed using densely connected coding blocks. The input image size is H×W. Initial convolutional blocks are used before the first encoder layer. Each small convolutional block includes a convolutional layer (Conv), batch normalization (BN), and a linear ReLU layer, resulting in an initial feature map of size H×W×64 with 64 channels. Densely connected modules are then used to enhance feature learning. Finally, 2×2 convolutions with a stride of 2 are used for downsampling, compressing the feature map to a size of H×W×64. By combining dense connection modules and downsampling four times, a feature map containing more image information is obtained, with a size of [size missing].

[0083] Output=ReLU(BN(Conv(Input)))

[0084] S22: The feature maps obtained from the image data are serialized into input, and features are learned from this sequence based on the SwinTransformer encoding blocks. The input image is divided into N image blocks through convolution operations and flattened into a sequence, resulting in a sequence of size [size missing]. The initial sequence is then transformed linearly through convolution in the linear embedding layer, mapping it to a space of dimension C, with a sequence size of [value missing]. Next, after downsampling and the SwinT encoding module, the result is obtained. The feature sequence; the final feature sequence obtained through repeated operations is of size . The formula for the SwingTransformer coded block is shown below:

[0085] z l =MLP(LN(W_MSA(LN(z) l-1 ))+z l-1 ))+W_MSA(LN(z l-1 ))+z l-1 ,

[0086] z l+1 =MLP(LN(SW_MSA(LN(z) l ))+z l ))+SW_MSA(LN(z l ))+z l

[0087] In the above formula, z l-1 These are features of the input layer, z l z is the output feature of the intermediate layer. l+1 These are the final output features. MLP(*) represents a multilayer perceptron layer, LN(*) represents a normalization layer, and W_MSA(*) and SW_MSA(*) represent window-based multi-head self-attention mechanism and window displacement-based multi-head self-attention mechanism, respectively.

[0088] S23: The capsule network module aggregates the feature maps from channels S21 and S22 to capture long-range dependencies between parts and the whole, enhancing feature learning. The capsule network uses vectors to represent a set of neurons, representing not only the spatial relationships between features but also the probability of feature existence. Simultaneously, a dynamic routing algorithm iteratively calculates the weights of each forward pass to adjust the contribution of the part to the whole, thus better modeling the relationship between the local and the global. Before entering the capsule network module, the output features from the dense connection module and the Transformer module are fused to obtain the input feature map for this module, with a size of [size missing]. Then reshape as Where N = 8 is the number of capsules, and D = 64 is the dimension of each capsule; after 3 routing iterations, the resulting size is... The output features are then reshaped before being passed to the decoding module. The feature map is shown. Throughout the capsule module, the kernel size is 3×3 with a stride of 1. The formula for the capsule network is shown below:

[0089]

[0090] In the above formula, take It is the set of capsule blocks in the l-th layer. It is the set of capsule blocks in the (l+1)th layer, W ij To obtain the learned weight matrix, the capsule block of layer l is... Linear mapping to capsule blocks in layer l+1 r ij It is the coupling coefficient between capsules i and j, assigned according to the dynamic routing allocation algorithm.

[0091] Furthermore, step S3 includes the following steps:

[0092] S31: The four-layer feature set from the encoding stage is used as the source of edge information. Low-level features provide local edge information, while high-level features have a larger receptive field. Edge information for the region to be segmented is obtained through feature fusion. This process uses... Figure 3 The multi-scale edge feature extraction module shown Figure 4 The multi-head self-attention mechanism shown and Figure 5 The cross-attention mechanism shown is used for edge information extraction, analysis, and transmission.

[0093] S32: The multi-scale edge feature module aims to explore the multi-scale information of each layer of feature map, thereby obtaining features with more edge discrimination; the multi-head self-attention mechanism aims to explore the spatial information of the feature map; and the cross-attention mechanism can connect features of different scales.

[0094] Furthermore, step S4 includes the following steps:

[0095] S41: The decoder is constructed through convolution and deconvolution in upsampling. The feature map in the encoding stage is enlarged by continuously halving the number of channels and doubling the size. Skip connections are used to concatenate the feature map with the corresponding lower-level feature map along the channel dimension until a feature map of the same size as the input feature map is restored. Simultaneously, [the following is used:] Figure 6 The feature aggregation module shown and Figure 7 The feature selection module shown processes the features of the encoding layer, thereby passing more semantic information to the decoding layer.

[0096] S42: The outputs of the decoder and the edge detector are concatenated to form the final segmentation result. The entire network is trained using a joint loss function, and the model that performs best on the validation set is saved. The formula for the relevant loss function is as follows:

[0097] L Total =L Seg +L Edge

[0098] L Seg =L T

[0099] L Edge =λ1L Dice +λ2L BCE

[0100]

[0101]

[0102] L BCE =-w*(target*ln(pt)+(1-target)*ln(1-pt))

[0103] In the above formula, L T The loss function addresses the segmentation problem and adaptively learns the optimal tolerance level for label noise during backpropagation. The loss function for edge segmentation is defined by L... Dice and L BCE The loss function L is composed of two hyperparameters, λ1 and λ2. Dice middle, y true and y predThese represent the true value and the predicted value, respectively; θ represents a very small value to prevent the denominator from being zero; the loss function L... BCE In this context, target and pt represent the actual value and the predicted value, respectively, and w represents the weight.

[0104] Example 2

[0105] This embodiment provides an angiography-free myocardial infarction image segmentation system based on CNN and Transformer. The image segmentation system adopts the angiography-free myocardial infarction image segmentation method based on CNN and Transformer described in Example 1, performs semantic segmentation on the acquired image, and then obtains the segmentation prediction result of the target region.

[0106] like Figure 7 As shown, the myocardial infarction image segmentation system includes: a data augmentation module, a feature extraction module (convolutional feature extraction module, Swin Transformer feature extraction module and capsule feature fusion module), an edge feature extraction module and a feature decoding module.

[0107] The data augmentation module mainly uses GAN networks, random cropping, random rotation, and random scaling to augment the medical images to be segmented, and normalizes the pixel distribution of the images.

[0108] The feature extraction module comprises three parts: a convolutional feature extraction module, a Swing Transformer feature extraction module, and a capsule feature fusion module. The convolutional feature extraction module uses a densely connected network to perform local feature extraction on the input image and downsamples it using a 2×2 convolution with a stride of 2 to obtain the convolutional features. The Swing Transformer feature extraction module extracts features layer by layer from the serialized feature map of the input image, downsamples it using a convolution with a stride of 2, and finally obtains the Swing Transformer features. The capsule feature fusion module deeply fuses the output convolutional features and Swing Transformer features to obtain the final output of the feature extraction module.

[0109] The edge feature extraction module primarily utilizes the output features of the intermediate layers of the convolutional feature extraction module to learn edge features from the image data. Low-level features aim to provide local edge information, while high-level features have a larger receptive field and greater spatial invariance. The learning of edge information is accomplished using methods such as a multi-scale boundary feature extraction module, multi-head self-attention, and cross-attention.

[0110] The feature decoding module receives the fused features output by the feature extraction module and restores the size of the feature map through layer-by-layer convolution and upsampling. Skip connections allow semantic features with greater layer-level discriminative power to be passed to the upsampling network. The outputs of the decoding module and the edge feature extraction module are then fused to obtain the final segmentation result.

[0111] Example 3

[0112] This embodiment provides an angiography-free myocardial infarction image segmentation device based on CNN and Transformer. The device includes a memory, a processor, a computer program stored in the memory and executable on the processor, and related image data. When the processor executes the program, it implements the steps of the angiography-free myocardial infarction image segmentation method based on CNN and Transformer as described in Embodiment 1.

[0113] In this embodiment, the memory is used to store computer programs and related data, and may include various types of storage media, including random access memory (RAM) such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and dual data rate random access memory (DDR RAM); read-only memory (ROM) such as programmable read-only memory (PROM) and electrically erasable programmable read-only memory (EEPROM); magnetic memory such as hard disk drives (HDD) and solid-state drives (SSD); optical disk storage and mobile storage, etc.

[0114] In this example, the processor can be a central processing unit (CPU), a microprocessor, or other data processing chip, mainly used to execute the computer program in the memory or process data to realize the processing process of the angiography-free myocardial infarction image segmentation method based on CNN and Transformer in the aforementioned embodiment, thereby obtaining the segmentation result of myocardial infarction and other feature information in the image based on the given medical image.

[0115] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0116] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0117] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0118] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A contrast-free image segmentation method for myocardial infarction, characterized in that, Includes the following steps, S1. Data preparation stage: Collect magnetic resonance imaging data of myocardial infarction without gadolinium contrast enhancement, and perform corresponding annotations. Obtain the data-enhanced training set and validation set through data processing. S2, Feature Extraction Stage: The processed data is input into the feature extraction network, and features are extracted through the dense connection module and the Transformer model. Then, the capsule network module is used to deeply fuse the obtained features. S3, Edge Detection Stage: The intermediate features in the feature extraction stage are used as the source of edge information. The edge feature extraction module is used to learn the edge information fully, thereby obtaining the edge region of the myocardial infarction area. S4. Feature recovery stage: A decoder is constructed using convolutional methods to recover the features of the encoder layer by layer. The results of the edge detection stage are then concatenated with the results of the image features recovered in the decoder to obtain the segmentation result, which is then saved as the model with the best performance in the validation set. S5. Method validation stage: The real-time collected gadolinium-free contrast-enhanced myocardial infarction data is fed into the saved model to obtain the segmentation result map; Step S2 includes the following steps: S21: Local feature extraction from MRI images of myocardial infarction using densely connected coding blocks; the size of the input image is [size missing]. Before the first encoder layer, initial convolutional blocks are used for initial convolution. Each small convolutional block includes: convolutional Conv, batch normalization (BN), and a linear ReLU layer, resulting in a size of... The initial feature map is generated with 64 channels; then a densely connected module is used to enhance feature learning; next, a 2×2 convolution with a stride of 2 is used for downsampling, compressing the feature map size to [size missing]. By combining dense connection modules and downsampling four times, a feature map containing more image information is obtained, with a size of [size missing]. ; S22: The feature maps obtained from the image data are serialized into input, and features are learned from the sequence based on the SwinTransformer encoding blocks; the input image is divided into N image blocks through convolution operations and flattened into a sequence, resulting in a sequence of size [missing information]. The initial sequence is then transformed linearly through convolution in the linear embedding layer, mapping it to a space of dimension C, with a sequence size of [value missing]. Next, after downsampling and the SwinT encoding module, the result is obtained. The feature sequence; the final feature sequence obtained through repeated operations is of size . The formula for the SwingTransformer coded block is shown below: In the above formula, These are features of the input layer. The output features of the intermediate layer, It is the final output feature. For a multi-layer perceptron, Represents the normalization layer. and These represent window-based multi-head self-attention mechanisms and window displacement-based multi-head self-attention mechanisms, respectively. S23: The capsule network module is used to aggregate the feature maps of the two channels S21 and S22 to capture the long-distance dependency between the part and the whole, and enhance feature learning. The capsule network uses vectors to represent a set of neurons, which can not only represent the spatial relationship between features, but also the probability of feature existence. At the same time, the dynamic routing algorithm iteratively calculates the weight of each forward pass to adjust the contribution of the part to the whole, thereby better modeling the relationship between the local and the whole. Before entering the capsule network module, the output features from the dense connection module and the Transformer module are fused to obtain the input feature map for this module, with a size of [size missing]. Then reshape as ,in The number of capsules in the set. The dimension of each capsule group; after 3 routing iterations, the size is obtained as follows: The output features are then reshaped before being passed to the decoding module. Feature maps; throughout the capsule module, the kernel size is The stride is 1; the kernel size is [value missing] throughout the capsule module. The step size is 1; the formula for capsule networks is as follows: In the above formula, take It is the first A collection of capsule blocks in layers. It is the first A collection of capsule blocks in layers. For the learned weight matrix, the first... Layered capsule blocks Linear mapping to the first capsule blocks in the layer , The capsules are allocated based on the dynamic routing allocation algorithm. and The coupling coefficient between them.

2. The image segmentation method for myocardial infarction without contrast agent according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: Collect gadolinium-free contrast-enhanced myocardial infarction image data and corresponding pixel-level annotation data; S12: Perform random rotation and cropping enhancement operations on all collected data to obtain enhanced data; S13: Normalize the enhanced image data so that the mean of all images is 0 and the standard deviation is 1, so that the image grayscale distribution follows a normal distribution.

3. The image segmentation method for myocardial infarction without contrast agent according to claim 1, characterized in that: The specific steps of step S3 are as follows: S31: The four-layer features in the encoding stage are used as the source of edge information. The low-level features provide local edge information, while the high-level features have a larger receptive field. The edge information of the region to be segmented is obtained through feature fusion operation. This process uses a multi-scale edge feature extraction module, a multi-head self-attention mechanism, and a cross-attention mechanism to extract, analyze, and transmit edge information. S32: The multi-scale edge feature module aims to explore the multi-scale information of each layer of feature map, thereby obtaining features with greater edge discrimination. Multi-head self-attention mechanisms aim to explore the spatial information of feature maps; cross-attention mechanisms, on the other hand, connect features at different scales.

4. The contrast-free myocardial infarction image segmentation method according to claim 1, characterized in that: Step S4 specifically includes the following steps: S41: Construct the decoder through convolution and deconvolution in upsampling; during the encoding stage, the feature map is enlarged, the number of channels is continuously halved and the size is doubled, and skip connections are used to concatenate it with the corresponding low-level feature map in the channel dimension until the feature map is restored to the same size as the input feature map. S42: The outputs of the decoder and the edge detector are concatenated to form the final segmentation result; the entire network is trained using a joint loss function, and the model that performs best on the validation set is saved; the formula for the relevant loss function is as follows: In the above formula, The loss function is used to solve the segmentation problem and can adaptively learn the optimal tolerance level for label noise during backpropagation; the loss function for edge segmentation is... and Composition, in which and There are two hyperparameters; the loss function. middle, and These represent the actual value and the predicted value, respectively. Represents a very small value, to prevent the denominator from being 0; loss function middle, and These represent the actual value and the predicted value, respectively. Represents weight.

5. A contrast-free myocardial infarction image segmentation system, wherein the image segmentation system employs the contrast-free myocardial infarction image segmentation method as described in any one of claims 1-4 to perform semantic segmentation on the acquired image, thereby obtaining the segmentation prediction result of the target region; This myocardial infarction image segmentation system includes: The module includes a data augmentation module, a feature extraction module, an edge feature extraction module, and a feature decoding module. The feature extraction module includes a convolutional feature extraction module, a Swin Transformer feature extraction module, and a capsule feature fusion module. The data augmentation module is used to augment the medical images to be segmented using a GAN network, random cropping, random rotation, and random scaling, and to normalize and adjust the pixel distribution of the images. The feature extraction module consists of three parts: a convolutional feature extraction module, a Swin Transformer feature extraction module, and a capsule feature fusion module. The convolutional feature extraction module uses a densely connected network to perform local feature extraction on the input image, and then uses a 2×2 convolution with a stride of 2 to perform downsampling to finally obtain the convolutional features. The Swin Transformer feature extraction module extracts features layer by layer from the serialized feature map of the input image, performs downsampling using convolution with a stride of 2, and finally obtains the Swin Transformer features. The capsule feature fusion module performs deep fusion of the output convolutional features and Swing Transformer features to obtain the final output of the feature extraction module; The edge feature extraction module learns edge features from the output features of the intermediate layers of the convolutional feature extraction module. Low-level features aim to provide local edge information, while high-level features have a larger receptive field and are more spatially invariant. The learning of edge information is accomplished by using a multi-scale boundary feature extraction module, multi-head self-attention, and cross-attention methods. The feature decoding module receives the fused features output by the feature extraction module, and restores the size of the feature map through layer-by-layer convolution and upsampling. The skip connections transmit semantic features with greater layer-level discriminative power to the upsampling network. The output of the decoding module and the output of the edge feature extraction module are fused to obtain the final segmentation result.

6. A contrast-free myocardial infarction image segmentation apparatus, comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 4.

Images