DSA coronary artery image synthesis method and segmentation method

Abstract

The invention relates to a synthesis method and a segmentation method of a DSA coronary artery image, and the method comprises the steps: firstly carrying out the multi-round MAE processing of an original DSA image, so as to remove an original coronary artery structure in the image, and only keeping a part as a background; next, a series of tree structures similar to natural coronary arteries are randomly generated based on the morphological characteristics and structural modes of the real coronary arteries, the artificial but highly-vivid structures are used as artificial labels, and finally, the randomly generated artificial labels are combined with a previously prepared background image to obtain a background image; and final synthesis is completed under the condition of controllable parameters, so that coronary DSA image data with different parameter settings can be generated. A segmentation network framework is composed of two independent backbone networks, image features extracted by a jump connection part between an encoder and a decoder in the two backbone networks are subjected to cross guidance, the coronary artery structure is identified and segmented more accurately, and the adaptability of the model to diversified data is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of medical image segmentation, and in particular to a synthesis method and a segmentation method of a DSA coronary artery image. The segmentation method is based on controllable synthetic artificial data and data perception. Background Art

[0002] Digital subtraction angiography (DSA) is a medical imaging technology used to observe the internal structure of blood vessels, and plays an important role in the diagnosis and treatment of coronary artery disease. However, in practical applications, DSA images often contain a lot of background noise and irrelevant information, which brings certain difficulties to doctors' diagnosis. Traditional coronary segmentation methods include threshold-based segmentation methods, edge detection-based segmentation methods, etc. However, these operations are very sensitive to noise in the image, and in actual operations, it is often necessary to manually adjust parameters to adapt to different image features and segmentation requirements, which increases the complexity and subjectivity of the processing.

[0003] The rise of convolutional neural networks (CNNs) has driven the development of fully supervised learning algorithms that use pre-labeled tags to achieve end-to-end semantic segmentation of medical images. By using deep learning models such as U-Net, DSA images can be automatically and efficiently segmented and the overall performance can be improved. The introduction of attention mechanisms further enhances segmentation accuracy. However, fully supervised learning relies on a large amount of high-quality labeled data, which is time-consuming and expensive. In addition, such models may perform poorly on unseen data due to overfitting.

[0004] In order to reduce the reliance on labeled data and reduce costs, semi-supervised learning methods have been introduced for medical image segmentation. Consistency regularization generates multiple views through data augmentation, requiring the model predictions to be consistent to improve generalization. However, this increases computational costs, and the choice of data augmentation strategy is critical. Pseudo-labeling methods use preliminarily trained models to predict unlabeled data and generate pseudo-labels for retraining, but poor initial model performance may lead to pseudo-label noise, affecting the final performance.

[0005] When training a coronary DSA segmentation model, we often have to deal with multiple data sets from multiple centers. Images from different data sets may have different contrast and brightness and different clarity and equipment parameters, which will affect the training effect of the model. To solve this problem, previous solutions mostly used methods such as data preprocessing and enhancement, multi-scale feature extraction and transfer learning. However, these methods are based on known data sets, have poor generalization capabilities and unstable results. When applied to new data sets, the results may decline significantly.

[0006] In summary, among the current deep learning-based DSA coronary segmentation methods, the fully supervised segmentation method relies on a large amount of labeled data to ensure its performance, and the semi-supervised method requires a small amount of manually labeled data and a large amount of unlabeled data, and the effect is unstable. The above methods are less effective when dealing with multi-center and multi-datasets, especially when testing on datasets that have never been seen before. Summary of the invention

[0007] In view of the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide a synthesis method and a segmentation method for DSA coronary artery images. The segmentation method does not require any manual annotation and can be generalized to any unknown data set.

[0008] The technical solution of the present invention to solve the technical problem is:

[0009] In a first aspect, the present invention provides a method for synthesizing a DSA coronary artery image, the method comprising the following steps:

[0010] Training masked autoencoder MAE: Obtain DSA coronary images to train masked autoencoder, divide each original image into several small blocks, then set the occlusion ratio s, randomly select small blocks from these small blocks according to the occlusion ratio for occlusion processing, and set all pixel values ​​in the selected area to black; MAE learns the feature representation of the unoccluded part and uses this information to reconstruct the complete picture as much as possible, that is, when the image after random masking is input into MAE, predict and fill the covered parts. During the training process, the mean square error MSE loss is used to measure the difference between the reconstructed image and the real image. By minimizing the mean square error MSE loss, the MAE parameters are continuously adjusted until the best performance is achieved;

[0011] Reconstruct the background image to obtain the artificial background: Use the trained mask autoencoder MAE to reconstruct the image, repeatedly apply mask processing to the same image, and use MAE to reconstruct it. After several rounds of iterations, a background image that only retains the background elements in the original image is obtained to obtain the artificial background I bg ;

[0012] Generate artificial labels based on the morphological structure of coronary arteries in real coronary artery images: an image with height H, width W, and all zero pixel values ​​I = 0 H×W As a canvas, first determine a starting point position (x0, y0) and an initial angle θ0. This starting point is regarded as the starting point of the coronary artery, and the initial angle represents the initial extension direction of the blood vessel. Set the moving step length. In each step, move the current point forward by one step length according to the current angle to obtain the new coordinates, and set the pixel value of the moving path on the image to 255;

[0013] After each movement is completed, the direction of the next point is randomly adjusted to simulate the serpentine structure of the coronary artery;

[0014] Introduce randomness to decide whether to create a new branch after a certain step length, that is, set a small probability P for each step, and use this probability to decide whether to form a new branch. If no new branch is currently formed, increase the possibility of generating a branch in the next step according to the small probability, and form a more diverse and realistic tree structure to simulate the branch structure of the coronary artery.

[0015] After moving a few steps, the current path is subjected to a region growth operation to simulate the change of the coronary artery from the aorta to the capillaries from thick to thin;

[0016] Finally, the generated results are checked to obtain artificial labels that can reflect the real coronary artery characteristics observed on the DSA images; if the generated results have obvious deviations, the small probability P, random angle, and step size parameters are fine-tuned to regenerate the artificial labels;

[0017] Using artificial background and artificial labels to synthesize simulated coronary DSA images: First, the artificial background I bg and manual labeling I l Add random Gaussian noise to the pixel. If the pixel value after adding Gaussian noise is greater than 255, the pixel value will be updated to 255. Otherwise, the current pixel value after adding Gaussian noise will be retained to simulate the difference in noise levels between various devices.

[0018] Next, a point is randomly selected on the artificial label as the center of the light source, and the brightness around the point is kept constant and gradually weakened as it moves away, reproducing the phenomenon of uneven distribution of contrast agents, and obtaining a processed artificial label;

[0019] Afterwards, the artificial background processed by Gaussian noise and the processed artificial label are mixed and superimposed according to a random pixel ratio a:b to obtain a synthetic image; the value range of the random pixel ratio is a random number between 7:3 and 1:9;

[0020] After that, the radius r of the circular area is set, and an additional operation is performed on the synthetic image with a probability of 50%, that is, except for the content in the circular area, the pixel values ​​of the remaining parts are set to 0, and the artificial original image I is obtained, which meets the theoretical expectations and can be used for subsequent model training. f and its corresponding label data.

[0021] Furthermore, the process of randomly adjusting the direction of the next point to simulate the serpentine structure of the coronary artery is: changing the direction of advance by randomly increasing or decreasing a preset angle value, so as to imitate the natural curvature of the coronary artery; the preset angle value is randomly generated in the range of 0-30°.

[0022] Furthermore, the specific operation of the region growing is formula (1):

[0023]

[0024] Where I = 0 H×W As a canvas, I i,j Represents the pixel value of point (i, j) in the canvas.

[0025] Furthermore, the noise adding process is shown in formula (1) and formula (2),

[0026] I' i,j =I i,j +|α|,α~N(0,σ),I∈{I bg ,I l},(1)

[0027]

[0028] Where N(0,σ) represents a Gaussian distribution with a mean of 0 and a variance of σ; σ is a random number between 16 and 64; I i,j I is the artificial background before adding noise bg and manual labeling I l The pixel value of point (i, j); I' i,j is the pixel value after adding Gaussian noise; I” i,j is the pixel value after noise correction.

[0029] Furthermore, the process of mixed superposition is as follows:

[0030] I f0 i,j =β×I bg i,j +(1-β)×I l i,j ,(1 <i<H-1,1<j<W-1),(4)

[0031] Where β is a random number between [0.1, 0.7], and the range can be adjusted according to the actual effect; I f0 i,j is the pixel value after mixed superposition, I bg i,j is the pixel value of point (i, j) in the artificial background; I l i,j is the pixel value of point (i, j) in the processed artificial label.

[0032] In a second aspect, the present invention provides a DSA coronary artery image segmentation method, wherein the segmentation method uses the artificial original image obtained by the synthesis method and its corresponding label data for segmentation training.

[0033] Furthermore, the segmentation method includes the following steps: constructing a segmentation network consisting of two identical encoder-decoders, the segmentation network including a background segmentation model F bg and coronary segmentation model F l , a background segmentation model focuses on background segmentation, and a coronary segmentation model is dedicated to the accurate segmentation of coronary arteries; in the segmentation network training process, the synthetic artificial original image I is first used f As input data, for the background segmentation model F bg Its output includes not only the segmented background image P bg , and also includes a weight matrix P used to measure the possibility that each pixel belongs to the background bg_w ; The coronary segmentation model F l The specific segmentation results of the coronary artery and the corresponding weight matrix P will be generated. l_w , the trained segmentation network is used for DSA coronary artery segmentation.

[0034] Furthermore, the background segmentation model and the coronary segmentation model in the segmentation network are skipped and connected, and the extracted image features are cross-attentioned. The extracted background features are used to help the coronary segmentation model understand the style features of the image, so as to perform accurate coronary segmentation. At the same time, the extracted coronary features are used to guide the background segmentation model to extract coronary information from the image.

[0035] During the segmentation network training process, the segmentation loss L is calculated for the coronary segmentation results. seg , the predicted coronary artery area P l The corresponding manual label I l For comparison, the Dice coefficient is used to measure the overlap between the two, and the binary cross entropy BCE loss function is used together on this branch to improve the model's ability to identify the coronary artery boundaries in small or complex morphologies, as shown in formula (10);

[0036] L seg =(1-Dice(P l ,I l ))+BCE(P l ,I l )(10)

[0037] Then calculate the reconstruction loss L recon , using the segmented background P bg and the background weight matrix P bg_w Calculate the dot product Tbg , using the coronary segmentation result P l and the coronary weight matrix P l_w Calculate the dot product T l , and add the two calculation results to get the reconstructed new artificial original image P recon ; Finally, by minimizing the reconstructed new artificial original image P recon Compared with the original input artificial image I f The MSE loss between the two is used to adjust the model parameters in order to achieve the goal of accurately distinguishing the foreground while maintaining good visual effects.

[0038] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the synthesis method or the steps of the segmentation method.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The present invention proposes a synthesis method for DSA coronary artery images, which can generate coronary DSA images in an artificial and controllable manner. The synthesis method first performs multiple rounds of MAE processing on the original DSA image, aiming to remove the original coronary structure in the image and retain only the part serving as the background. Next, based on the morphological characteristics and structural patterns of the real coronary arteries, a series of tree-like structures similar to natural coronary arteries are randomly generated. These artificial but highly realistic structures will be used as artificial labels. The last step is to combine these randomly generated artificial labels with the previously prepared background image, and complete the final synthesis under controllable parameter conditions, so that coronary DSA image data with different parameter settings can be generated. This method provides more flexible and diverse data support for subsequent research.

[0041] 2. Since there is a one-to-one correspondence between the synthesized DSA coronary images and the manual labels, no additional manual annotation is required in the entire process, which solves the problem of high manual annotation costs, greatly simplifies the data processing process, improves work efficiency, and ensures data consistency and accuracy. The dataset generated in this way can be directly used in the subsequent deep learning segmentation network training, providing strong support for DSA coronary image segmentation research.

[0042] 3. The segmentation network framework of the present invention is composed of two independent backbone networks. The two models have the same structure, but the initial parameters and subsequent parameter optimization do not affect each other. Both adopt an encoder-decoder structure. In order to further enhance the generalization ability and adaptability of the model, the image features extracted by the jump connection part between the encoder and the decoder in the two backbone networks are cross-guided. Specifically, by introducing a cross-attention mechanism, the data set features captured by the background extraction backbone are used to guide the coronary segmentation process. This method enables the coronary segmentation model to more accurately identify and segment the coronary structure when facing data sets of different styles or sources, thereby significantly improving the model's adaptability to diversified data.

[0043] 4. The present invention has demonstrated excellent performance in the coronary DSA segmentation task and can achieve high-precision segmentation results without relying on any manual annotation. For some data sets, its performance is particularly outstanding and can be used directly as the final segmentation results; for some other data sets, although it may not meet the standards for direct application, it can still provide preliminary and valuable segmentation references. Manual correction or supplementation based on such a rough segmentation will greatly reduce the time and cost required for comprehensive manual annotation on new data sets, thereby effectively improving work efficiency and economic benefits. This method not only improves the speed and accuracy of data processing, but also lays a solid foundation for subsequent more refined image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the overall process of an embodiment of a method for segmenting a DSA coronary artery image of the present invention;

[0045] Figure 2 A schematic diagram of the structure of a backbone network according to an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of network training of a masked autoencoder MAE according to an embodiment of the present invention;

[0047] Figure 4 A sample schematic diagram of an artificial background, an artificial label and an artificial original image in the synthesis process of the present invention;

[0048] Figure 5 A training architecture diagram of a segmentation network according to an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the structure of an attention module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The specific embodiments of the present invention are given below. The specific embodiments are only used to further illustrate the present invention in detail and do not limit the protection scope of the claims of this application.

[0051] The synthesis method of the DSA coronary artery image of the present invention comprises the following steps:

[0052] Step 1: Training the Masked Autoencoder MAE: Using 20,000 DSA coronary images as the training dataset, train a Masked Autoencoder (MAE). The specific process is as follows: Figure 3 As shown in the figure, the number of channels C is set to 64, the image width and height H and W are set to 512, and the number of output categories NC is set to 1. For each original image, it is divided into multiple small blocks of 16×16 pixels. Then, 50% of the small blocks are randomly selected for occlusion processing, that is, all pixel values ​​in the selected area are set to black, which helps to enhance the model's ability to understand and restore missing information.

[0053] The MAE includes an encoder and a decoder. In this embodiment, the same decoder and encoder as the backbone model of the segmentation network can be used to perform this task. The core idea of ​​MAE is to learn the feature representation of the unoccluded part and use this information to reconstruct the complete picture as much as possible, that is, when the image after random masking is input into MAE, it will try to predict and fill in the covered parts. During the MAE training process, the mean square error (MSE) loss is used to measure the difference between the reconstructed image and the real image. By minimizing the mean square error (MSE) loss, the model parameters can be continuously adjusted until the optimal performance state is achieved. Before calculating the MSE, the two images are usually standardized to ensure a fair comparison.

[0054] The second step is to reconstruct the background image to obtain an artificial background: When the trained masked autoencoder (MAE) is used to reconstruct the image, the background information can be restored well, but there are certain omissions in the restoration of the coronary artery area. Although this limitation may be due to certain factors in the MAE training process, it is precisely because of the missing characteristics of the coronary artery that it provides a unique application idea. Specifically, by repeatedly applying mask processing to the same image and reconstructing it using MAE, after multiple rounds of iterations, the final image almost only retains the background elements in the original image. By cleverly transforming the limitations of existing technologies into valuable tools, these pure background images obtained are very suitable as basic data for subsequent research work. The specific effects are as follows: Figure 4 shown.

[0055] Step 3: Generate artificial labels based on the morphological structure of coronary arteries in real coronary artery images: In order to imitate the morphological structure of coronary arteries in real coronary artery images, the following algorithm is proposed. H×W As a canvas, first determine a starting point position (x0, y0) and an initial angle θ0. This starting point is regarded as the starting point of the coronary artery, and the initial angle represents the initial extension direction of the blood vessel. In each step, the current point coordinates are (x, y), and the current angle is θ. According to the current angle, the current point is moved forward 3 pixels, that is, a step length, to obtain the new coordinates (x+3cosθ, y+3sinθ), and the pixel value of the moving path on the image is set to 255.

[0056] After each move, the direction of the next point is randomly adjusted by randomly increasing or decreasing the random angle value in the range of 0° to 30°. To achieve this change, the new angle is This setting can simulate the serpentine structure of the coronary artery.

[0057] Introduce randomness to decide whether to create a new branch after a certain step length, that is, set a small probability P = 6 × 10 for each step -5 , and use this probability to decide whether to form a new branch. If no new branch is formed at present, the probability of generating a branch in the next step increases by 6×10 -5 This can help form a more diverse and realistic tree structure, simulating the branching structure of the coronary arteries.

[0058] After moving a few steps, the current path is subjected to a regional growth operation, making the part close to the starting point thicker and the part far from the starting point relatively thinner, which can simulate the change of the coronary artery from thick to thin from the aorta to the capillaries.

[0059] In this embodiment, the region growth operation is performed once every 50 steps. The specific formula of the region growth operation is shown in formula (1):

[0060]

[0061] Among them I i,j Represents the pixel value of point (i, j) in the image.

[0062] Finally, by visually inspecting the generated results, we can obtain artificial labels that can reflect the real coronary characteristics observed on the DSA images. If the generated results have obvious deviations (such as when the capillary branches are too large or the aorta is too thick or too thin, which obviously does not conform to the real coronary characteristics), we can fine-tune the small probability P, random angle, and step size parameters to regenerate the artificial labels to ensure that the final artificial labels can well reflect the real coronary characteristics observed on the DSA images. The effect is as follows: Figure 4 shown.

[0063] The fourth step is to use artificial background and artificial labels to synthesize simulated coronary DSA images:

[0064] First, to the artificial background I bg and manual labeling I l Random Gaussian noise is added to simulate the difference in noise levels between various devices. If the pixel value after adding Gaussian noise is greater than 255, the pixel value is updated to 255. Otherwise, the current pixel value after adding Gaussian noise is retained, which can more realistically reflect the characteristics of different medical devices. The noise addition process is shown in formulas (2) and (3).

[0065] I' i,j =I i,j +|α|,α~N(0,σ),I∈{I bg ,I l},(2)

[0066]

[0067] Where N(0,σ) represents a Gaussian distribution with a mean of 0 and a variance of σ. σ is a random number between 16 and 64, and the range can be adjusted according to the actual effect. Formula (3) means that the pixel values ​​above 255 are changed to 255. I i,j I is the artificial background before adding noise bg and manual labeling I l The pixel value of point (i, j); I' i,j is the pixel value after adding Gaussian noise; I” i,j is the pixel value after noise correction.

[0068] Next, a point light source effect was introduced on the artificial label, that is, a point was randomly selected on the artificial label as the center of the light source, and the brightness around the point was kept unchanged and gradually weakened as it moved away, so as to reproduce the uneven distribution of contrast agents and obtain the processed artificial label.

[0069] The artificial background I processed by Gaussian noise bg Compared with the processed manual label I lThe mixed superposition is performed at a random ratio between 7:3 and 1:9 to obtain a composite image I f0 , the composite image fully takes into account the differences in contrast processing methods of different imaging systems. The specific mixed superposition formula is shown in formula (4):

[0070] I f0 i,j =β×I bg i,j +(1-β)×I l i,j ,(1 <i<H-1,1<j<W-1),(4)

[0071] Where β is a random number between (0.1, 0.7], and the range can be adjusted according to the actual effect; I f0 i,j is the pixel value after mixed superposition, I bg i,j is the pixel value of point (i, j) in the artificial background; I l i,j is the pixel value of point (i, j) in the processed artificial label.

[0072] Since the image range captured by some devices may be circular or other non-standard shapes, there will be unlit areas around the actual image, that is, black borders. Therefore, after completing the above steps, an additional operation will be performed on the synthetic image with a probability of 50%, that is, in addition to retaining the content in a circular area, the remaining pixel values ​​will be set to 0, so as to further approach various scenes that may appear in actual situations. The specific operation is shown in formula (5).

[0073]

[0074] Where x is a random number between [0.4H, 0.6H], y is a random number between [0.4W, 0.6W], and r is a random number between [0.6H, 0.75H]. The specific range can be adjusted according to the actual effect; f i,j is the pixel value in the final artificial original image.

[0075] Through this series of processes, we can finally synthesize an artificial original image that meets theoretical expectations and can be used for subsequent model training. f and its corresponding label data (in this case, artificial labels are generated by imitating the morphological structure of the coronary artery in the real coronary artery image as label data), the effect is as follows Figure 4 shown.

[0076] The backbone network of the encoder-decoder structure used in the present invention is as follows Figure 2As shown. The encoder part includes a combination structure consisting of a series of alternating convolution blocks ConvBlock and Mamba modules, namely the ConvBlock and Mamba modules in the figure. The number of combined structures in this embodiment is 5. Among them, downsampling operation is performed by ConvBlock to reduce the spatial dimension of the feature map and extract preliminary feature information. Then the Mamba module is introduced to further enhance the feature extraction capability. Mamba is an autoregressive model based on the attention mechanism, which can effectively capture long-distance dependencies in the sequence, thereby improving the quality of feature representation. In order to enable information between different levels to flow better, jump connections are added between each layer of encoder and decoder. These connections allow low-level details to be directly passed to high-level layers, which helps to restore more refined structural information and alleviate the problem of gradient disappearance. In the decoder part, upsampling is first achieved through the deconvolution layer to restore the image size, and then the additional convolution layer is used to parse and refine the features.

[0077] like Figure 2 As shown, (1) represents the input image. The image is first downsampled by the convolution block in (8), reducing the height H and width W to half of the original, and expanding the number of channels to C, where C is a preset hyperparameter that can be adjusted according to needs.

[0078] The specific structure of the convolution block is shown in (13). The parameters accepted by the convolution block are the number of input channels (in_ch), the number of output channels (out_ch), the convolution kernel size (k), the convolution step size (s), and the convolution padding size (p). First, a convolution layer is passed to implement downsampling and channel expansion operations. The parameters of the convolution layer are set to the parameters accepted by the current convolution block. Then, the batch normalization layer is passed to normalize the mean and variance of each batch of data to make the input distribution of each layer more stable. Finally, the ReLU activation function layer is passed to map the linear input to a nonlinear output. Then, the feature extraction is performed through the Mamba block in (8), where the specific structure of the Mamba block is shown in (14). The parameters accepted by the Mamba block are the number of dimensions (dim) and the number of layers (num_layer). First, a Flatten operation is performed to convert the H×W two-dimensional image into a one-dimensional sequence of length L to meet the dimensional requirements of subsequent operations. Then, this one-dimensional sequence is used as the input of the Mamba layer to extract the global feature. The dimension of the Mamba layer is the parameter dim accepted by the current Mamba block. The feature data is then normalized through the LayerNorm layer to improve the stability of the model. Finally, a residual connection operation is performed, as shown in (15), to add the input of the Mamba layer to the output of the LayerNorm layer. The Mamba layer, LayerNorm layer and residual connection operations are repeated num_layer times, where num_layer is a preset hyperparameter. Finally, two layers of MLP multi-layer perceptrons are performed to improve the expression ability of the model, and a restoration operation is performed to restore the feature dimension to the original H×W.

[0079] After passing through the convolution block and Mamba block in (8), the feature map obtained is shown in (2). A copy of (2) is saved as the input of the subsequent jump connection, and then (2) is used as the input of (9) to obtain feature map (3). Similarly, feature maps (4), (5), and (6) are obtained after passing through modules (10), (11), and (12). Subsequently, (6) is upsampled (17) to enlarge the height H and width W to twice the original. The specific structure of upsampling is shown in (16). The parameters accepted by upsampling are the number of input channels (in_ch), the number of output channels (out_ch), the convolution kernel size (k), the convolution step size (s), and the convolution padding size (p). First, a deconvolution layer is used for upsampling. The in_ch and out_ch of the deconvolution layer are the parameters accepted by the current upsampling. k is fixed to 1, s is fixed to 2, and p is fixed to 2. Then, a batch normalization layer and a ReLU activation function are used to obtain the output.

[0080] After upsampling, the feature map (6) has the same dimension as the feature map (5). The Concat operation can be used to splice them together by channel as shown in (18). After feature extraction through the convolution block (19) and upsampling through (20), they can be spliced ​​together with the feature map (4) by channel through (21). Similarly, after (23)-(27) and splicing with the feature maps (3) and (2), the width and height of the feature map are restored to the same as the input (1) through (28) and (29). The number of channels is scaled to the number of channels of the desired output NC through (30). The in_ch of the convolution layer (30) is the set hyperparameter C, out_ch is the number of output categories (num_class), k is fixed to 1, s is fixed to 1, and p is fixed to 0. Finally, the final output value is mapped to [0,1] through the Sigmoid activation function to obtain the final output (7).

[0081] The parameter settings for the convolution block are shown in Table 1, the parameter settings for the Mamba block are shown in Table 2, and the parameter settings for upsampling are shown in Table 3.

[0082] Table 1 Convolution block parameter settings

[0083] in_ch out_ch k s p ConvBlock(8) 1 C 3 2 1 ConvBlock(9) C 2C 3 2 1 ConvBlock(10) 2C 4C 3 2 1 ConvBlock(11) 4C 8C 3 2 1 ConvBlock(12) 8C 8C 3 2 1 ConvBlock(19) 16C 4C 3 1 1 ConvBlock(22) 8C 2C 3 1 1 ConvBlock(25) 4C C 3 1 1 ConvBlock(28) 2C C 3 1 1

[0084] Table 2 Mamba block parameter settings

[0085] dim num_layer MambaBlock(8) C 3 MambaBlock(9) 2C 4 MambaBlock(10) 4C 6 MambaBlock(11) 8C 3 MambaBlock(12) 8C 3

[0086] Table 3 Upsampling parameter settings

[0087] in_ch out_ch UpSample(17) 8C 8C UpSample(20) 4C 4C UpSample(23) 2C 2C UpSample(26) C C UpSample(29) C C

[0088] The DSA coronary artery image segmentation method of the present invention uses the artificial original image synthesized by the above synthesis method and its corresponding label data for segmentation training.

[0089] The segmentation model used in the segmentation method includes the background segmentation model F bg and coronary segmentation model F l , both models use Figure 2 The backbone network is shown in Figure 2. The segmentation model is as follows: Figure 5 As shown, a background segmentation model F bg Focusing on background segmentation, that is, feature extraction of the data set, another coronary segmentation model F l It is specifically used for accurate segmentation of coronary arteries, where the number of channels C is set to 64, the image width and height H and W are set to 512, and the number of output categories NC is set to 2. In order to further enhance the generalization ability and adaptability of the model, the background segmentation model F bg and coronary segmentation model F lThe jump connection between the encoder and the decoder is performed five times, and the image feature S extracted by the encoder before each jump connection is bg and S l , input to the attention module F attn In the process, the background features S are extracted and processed by the attention module and then sent to the corresponding decoder. bg To help the coronary segmentation model F l Understand the style features of the image to perform accurate coronary segmentation; at the same time, use the extracted coronary features to guide the background segmentation model F bg Extract coronary artery information from images.

[0090] Attention Module F attn The specific structure of Figure 6 As shown, during the training process, we first use the synthetic artificial original image I f As input data. For the background segmentation model F bg The output includes not only the segmented background image with height H and width W It also includes a weight matrix that measures the likelihood that each pixel belongs to the background The coronary segmentation model F l The specific segmentation results of the coronary arteries will be generated And the corresponding weight matrix As shown in formulas (6)-(9).

[0091]

[0092] The attention module in the present invention is implemented by cross attention. The number of cross attentions can be the same as the number of jump connections between the encoder and the decoder. Each time the output of the encoder before the jump connection is processed by a cross attention to process the features of the two models, and then sent to the corresponding decoders respectively. Alternatively, only one cross attention can be set, and the output features of the encoders of the two models at different levels are spliced ​​in sequence, and then processed by cross attention to obtain the input features of each layer of the decoders of the two models.

[0093] Next, when calculating the loss, we first calculate the segmentation loss L for the coronary artery segmentation result. seg , the predicted coronary artery area P l The corresponding manual label I l For comparison, the Dice coefficient is used to measure the degree of overlap between the two, and the binary cross entropy (BCE) loss function is used together on this branch to improve the model's ability to identify the boundaries of coronary arteries with small or complex morphologies, as shown in formula (10).

[0094] L seg=(1-Dice(P l ,I l ))+BCE(P l ,I l ) (10)

[0095] Then calculate the reconstruction loss L recon , using the segmented background P bg and the background weight matrix P bg_w Calculate the dot product as shown in formula (11), where T bg represents the calculation result, (i, j) represents the pixel value at the position with height i and width j, and the coronary segmentation result P is used. l and the coronary weight matrix P l_w Calculate the dot product, as shown in formula (12), and add the two calculation results to obtain the reconstructed new artificial original image P recon , as shown in formula (13). Finally, by minimizing the reconstructed new artificial original image P recon Compared with the original input artificial image I f The MSE loss between is used to adjust the network parameters in order to achieve the goal of accurately distinguishing the foreground while maintaining good visual effects, as shown in formula (14).

[0096]

[0097] P recon =T bg +T l (13)

[0098] L recon =MSE(P recon ,I f ) (14)

[0099] Example

[0100] Through the first to fourth steps above, we obtain synthetic artificial original images. We use a large number of synthetic artificial original images as data sets to train the segmentation network. For the specific process, see Figure 1 After training, the method is tested on the real dataset.

[0101] Taking the public datasets XCAD (126 images), DCA1 (134 images) and the private dataset Private (300 images) as examples, the segmentation network of the present invention is used to perform DSA coronary artery image segmentation tests on the above three datasets, with Dice similarity coefficient (DSC), sensitivity (SN), specificity (SP) and accuracy (ACC) as evaluation indicators, among which DSC is considered to be the most authoritative indicator. The experimental results are shown in Table 4.

[0102] Table 4 Comparison of experimental results of various methods

[0103]

[0104] As can be seen from the above table, it can be seen from the above that the Dice similarity coefficient of the segmentation network of this embodiment is about 70%. At this stage, the Dice similarity coefficients of some fully supervised models on the same data set are generally around 80%. Although there is still a certain gap, compared with the fully supervised model, the segmentation network training of this application does not need to rely on any manual annotation of professional doctors, and it can be applied to most data sets after one training is completed. Moreover, when a new data set without any annotations appears, the segmentation network of this application can be used to obtain a rough segmentation result, and the doctor can subsequently refine this rough segmentation result, thereby greatly reducing the consumption of manual annotation.

[0105] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A method for synthesizing a DSA coronary artery image, characterized in that: The synthesis method comprises the following steps: Training masked autoencoder MAE: Obtain DSA coronary images to train masked autoencoder, divide each original image into several small blocks, then set the occlusion ratio s, randomly select small blocks from these small blocks according to the occlusion ratio for occlusion processing, and set all pixel values ​​in the selected area to black; MAE learns the feature representation of the unoccluded part and uses this information to reconstruct the complete picture as much as possible, that is, when the image after random masking is input into MAE, predict and fill the covered parts. During the training process, the mean square error MSE loss is used to measure the difference between the reconstructed image and the real image. By minimizing the mean square error MSE loss, the MAE parameters are continuously adjusted until the best performance is achieved; Reconstruct the background image to obtain the artificial background: Use the trained mask autoencoder MAE to reconstruct the image, repeatedly apply mask processing to the same image, and reconstruct it using MAE. After several rounds of iterations, a background image that only retains the background elements in the original image is obtained to obtain the artificial background I. bg ; Generate artificial labels based on the morphological structure of coronary arteries in real coronary artery images: an image with height H, width W, and all zero pixel values ​​I = 0 H×W As a canvas, first determine a starting point position (x0, y0) and an initial angle θ0. This starting point is regarded as the starting point of the coronary artery, and the initial angle represents the initial extension direction of the blood vessel. Set the moving step length. In each step, move the current point forward by one step length according to the current angle to obtain the new coordinates, and set the pixel value of the moving path on the image to 255; After each movement is completed, the direction of the next point is randomly adjusted to simulate the serpentine structure of the coronary artery; Introduce randomness to decide whether to create a new branch after a certain step length, that is, set a small probability P for each step, and use this probability to decide whether to form a new branch. If no new branch is currently formed, increase the possibility of generating a branch in the next step according to the small probability, and form a more diverse and realistic tree structure to simulate the branch structure of the coronary artery. After moving a few steps, the current path is subjected to a region growth operation to simulate the change of the coronary artery from the aorta to the capillaries from thick to thin; Finally, the generated results were checked to obtain artificial labels that reflect the real coronary artery features observed on DSA images; Using artificial background and artificial labels to synthesize simulated coronary DSA images: First, the artificial background I bg and manual labeling I l Add random Gaussian noise to the pixel. If the pixel value after adding Gaussian noise is greater than 255, the pixel value will be updated to 255. Otherwise, the current pixel value after adding Gaussian noise will be retained to simulate the difference in noise levels between various devices. Next, a point is randomly selected on the artificial label as the center of the light source, and the brightness around the point is kept constant and gradually weakened as it moves away, reproducing the phenomenon of uneven distribution of contrast agents, and obtaining a processed artificial label; Afterwards, the artificial background processed by Gaussian noise and the processed artificial label are mixed and superimposed according to a random pixel ratio a:b to obtain a synthetic image; the value range of the random pixel ratio is a random number between 7:3 and 1:9; After that, the radius r of the circular area is set, and an additional operation is performed on the synthetic image with a probability of 50%, that is, except for the content in the circular area, the pixel values ​​of the remaining parts are set to 0, and the artificial original image I is obtained, which meets the theoretical expectations and can be used for subsequent model training. f and its corresponding label data.

2. The synthesis method according to claim 1, characterized in that The process of randomly adjusting the direction of the next point to simulate the serpentine structure of the coronary artery is: the change of the direction of advance is achieved by randomly increasing or decreasing a preset angle value, so as to imitate the natural curvature of the coronary artery; the preset angle value is randomly generated in the range of 0-30°.

3. The synthesis method according to claim 1, characterized in that The specific operation of the region growth is formula (1): Where I = 0 H×W As a canvas, I i,j Represents the pixel value of point (i, j) in the canvas.

4. The synthesis method according to claim 1, characterized in that The noise adding process is shown in formula (1) and formula (2), I' i,j =I i,j +|α|,α~N(0,σ),I∈{I bg ,I l }, (1) Where N(0,σ) represents a Gaussian distribution with a mean of 0 and a variance of σ; σ is a random number between 16 and 64; I i,j I is the artificial background before adding noise bg and manual labeling I l The pixel value of point (i, j); I' i,j is the pixel value after adding Gaussian noise; I” i,j is the pixel value after noise correction.

5. The synthesis method according to claim 1, characterized in that The process of mixed superposition is formula (5), Where β is a random number between [0.1, 0.7], and the range can be adjusted according to the actual effect; is the pixel value after mixed superposition, I bg i,j is the pixel value of point (i, j) in the artificial background; I l i,j is the pixel value of point (i, j) in the processed artificial label.

6. A method for segmenting a DSA coronary artery image, characterized in that: The segmentation method uses the artificial original image obtained by any synthesis method described in claims 1-4 and its corresponding label data for segmentation training.

7. The segmentation method according to claim 6, characterized in that: The segmentation method Includes the following: Construct a segmentation network consisting of two identical encoder-decoders, the segmentation network includes a background segmentation model F bg and coronary segmentation model F l , a background segmentation model focuses on background segmentation, and a coronary segmentation model is dedicated to the accurate segmentation of coronary arteries; in the segmentation network training process, the synthetic artificial original image I is first used f As input data, for the background segmentation model F bg Its output includes not only the segmented background image P bg , and also includes a weight matrix P used to measure the possibility that each pixel belongs to the background bg_w ; The coronary segmentation model F l The specific segmentation results of the coronary artery and the corresponding weight matrix P will be generated. l_w , the trained segmentation network is used for DSA coronary artery segmentation.

8. The segmentation method according to claim 7, characterized in that: In the segmentation network, the background segmentation model and the coronary segmentation model are skipped and connected. The extracted image features are cross-attentioned. The extracted background features are used to help the coronary segmentation model understand the style features of the image, so as to perform accurate coronary segmentation. At the same time, the extracted coronary features are used to guide the background segmentation model to extract coronary information from the image. During the segmentation network training process, the segmentation loss L is calculated for the coronary segmentation results. seg , the predicted coronary artery area P l The corresponding manual label I l For comparison, the Dice coefficient is used to measure the overlap between the two, and the binary cross entropy BCE loss function is used together on this branch to improve the model's ability to identify the coronary artery boundaries in small or complex morphologies, as shown in formula (10); L seg =(1-Dice(P l ,I l ))+BCE(P l ,I l ) (10) Then calculate the reconstruction loss L recon , using the segmented background P bg and the background weight matrix P bg_w Calculate the dot product T bg , using the coronary segmentation result P l and the coronary weight matrix P l_w Calculate the dot product T l , and add the two calculation results to get the reconstructed new artificial original image P recon ; Finally, by minimizing the reconstructed new artificial original image P recon Compared with the original input artificial image I f The MSE loss between the two is used to adjust the model parameters in order to achieve the goal of accurately distinguishing the foreground while maintaining good visual effects.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the synthesis method described in any one of claims 1 to 5 or the steps of the segmentation method described in any one of claims 6 to 8 can be implemented.

Images