Image segmentation model training method and device, equipment and storage medium

Abstract

The application discloses an image segmentation model training method and device, equipment and a storage medium, which utilizes a small amount of first sample image data with labels and a large amount of second sample image data without labels to input into a first segmentation network and a second segmentation network to respectively perform prediction, obtain respective segmentation results of the first segmentation network and the second segmentation network, and respectively perform supervised training on the first segmentation network and the second segmentation network based on a loss function constructed based on the segmentation results and the labels. Meanwhile, a pseudo-label semi-supervised loss constructed based on the segmentation result of the first segmentation network and the segmentation result of the second segmentation network is used to guide mutual learning of the first segmentation network and the second segmentation network. The application can reduce label sample data required for training a model, complete training of the model by using a small amount of sample data with labels, and improve robustness of the model by using mutual guidance and learning of the two segmentation networks.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image segmentation model training method, apparatus, device, and storage medium. Background Technology

[0002] Digital subtraction angiography (DSA) is a widely used imaging technique for visualizing and examining blood vessels within the human body. Due to its time efficiency and cost-effectiveness, it remains the "gold standard" for assessing coronary artery disease. Extracting coronary artery segments or entire trees from X-ray DSA images is a crucial process that facilitates quantitative coronary artery analysis. DSA imaging collects a series of two-dimensional digital X-ray projection images as contrast agent is injected into the vessel of interest. The image acquired at the beginning of the sequence is called the mask image, and the vessel is not visible because the contrast agent has not yet entered. Following the mask image, DSA imaging collects contrast images, where the vessel is visible because the contrast agent has flowed through it during imaging. In these X-ray projection images, the vessel is difficult to distinguish from background content such as ribs, spine, ducts, and organs. Typically, the mask image and the corresponding contrast image are registered, having equal grayscale distributions. Subtracting the mask image from the corresponding contrast image removes the background and reveals the visible vessel. However, due to human motion and fluctuations in X-ray power and noise in the images, the resulting images are not always perfectly clear. Moreover, the terminal parts of the coronary arteries are quite small and numerous, making them difficult to distinguish using conventional extraction methods.

[0003] With the rapid development of machine learning algorithms, a deep learning-based image segmentation method has been proposed to address the aforementioned blood vessel image segmentation problem. Its basic framework is a Unet model, which employs convolution operations, pooling downsampling, skip connections, and upsampling to train on paired data consisting of the original DSA image and manually labeled data. Specifically, it first extracts features through convolution operations of different dimensions, restores the original image size through skip connections and upsampling, and then classifies each pixel using an activation function to obtain segmentation labels. Next, it calculates a loss function between the network-calculated labels and the manually labeled data, backpropagates the loss, and updates the network parameters using gradient descent to reduce the loss.

[0004] However, while the Unet model has good segmentation performance for blocky organ tissue lesions, it is less adaptable to tubular structures. Furthermore, it requires a large amount of image and labeled data for training, which is a significant limitation for medical images. In the absence of sufficient labeled image data, it is difficult to train a model with ideal accuracy. Summary of the Invention

[0005] In view of this, this application provides an image segmentation model training method, apparatus, device and storage medium to solve the problems that existing blood vessel image segmentation models have poor accuracy and require a large amount of labeled data to complete training.

[0006] To address the aforementioned technical problems, this application provides a technical solution: an image segmentation model training method, characterized in that the method includes: acquiring sample image data, which includes labeled first sample image data and unlabeled second sample image data; inputting the sample image data into a first segmentation network and a second segmentation network for segmentation to obtain a first probability map and a second probability map; then processing the first probability map and the second probability map using activation functions to obtain a first binary image and a second binary image; calculating a loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and a preset loss function; and iteratively training the first segmentation network and the second segmentation network, wherein the preset loss function includes a supervised loss reflecting the segmentation results of the first segmentation network and the second segmentation network themselves, and a pseudo-label semi-supervised loss reflecting the mutual guidance and learning between the first segmentation network and the second segmentation network.

[0007] As a further improvement of this application, before inputting the sample image data into the first segmentation network and the second segmentation network for segmentation, the method further includes: performing a contrast enhancement operation on the sample image data based on a contrast-limited adaptive histogram equalization.

[0008] As a further improvement of this application, the first segmentation network includes a Unet segmentation network, and the second segmentation network includes an ISUnet segmentation network. The ISUnet segmentation network includes an embedding layer, an encoder, a decoder, and a linear mapping layer.

[0009] As a further improvement of this application, the encoder includes an alternating cyclic feature extraction module and a downsampling module, and the decoder includes an alternating cyclic feature extraction module and an upsampling module. The feature extraction module includes a feature extraction network and a feedforward network. The feature extraction network includes a segmentation layer, a first high-frequency image processing module, a second high-frequency image processing module, a low-frequency image processing module, and a fusion layer. The first high-frequency image processing module includes a max pooling layer, a first linear mapping layer, and a first upsampling operation layer. The second high-frequency image processing module includes a second linear mapping layer and a convolution operation layer. The low-frequency image processing module includes an average pooling layer, an attention layer constructed based on a window multi-head self-attention mechanism and a sliding window multi-head self-attention mechanism, and a second upsampling operation layer.

[0010] As a further improvement of this application, the loss is calculated based on a first probability map, a second probability map, a first binary image, a second binary image, a label, and a preset loss function, including: calculating a first loss function value based on the first probability map, the first binary image, and the label corresponding to each first sample image; calculating a second loss function value based on the second probability map, the second binary image, and the label corresponding to each first sample image; calculating a third loss function value based on the first probability map and the second binary image; and calculating a fourth loss function value based on the second probability map and the first binary image.

[0011] As a further improvement to this application, the preset loss function is expressed as:

[0012] loss=SupervisedLoss1+SupervisedLoss2+0.1*exp(-5(1-ttmax2*CrossTeachingLoss3+CrossTeachingLoss4;

[0013] Where loss represents the preset loss function, SupervisedLoss1 represents the first loss function value, SupervisedLoss2 represents the second loss function value, CrossTeachingLoss3 represents the third loss function value, CrossTeachingLoss4 represents the fourth loss function value, t represents the number of training iterations in the current iteration, and tmax represents the preset total number of training iterations.

[0014] As a further improvement to this application, the first loss function value is expressed as:

[0015]

[0016] The second loss function value is expressed as:

[0017]

[0018] The third loss function value is expressed as:

[0019]

[0020] The fourth loss function value is expressed as:

[0021]

[0022] Where SupervisedLoss1 represents the first loss function value, SupervisedLoss2 represents the second loss function value, CrossTeachingLoss3 represents the third loss function value, CrossTeachingLoss4 represents the fourth loss function value, N represents the number of pixels in an image, and y i This represents the label corresponding to the i-th pixel in the first sample image data. This represents the i-th pixel in the first probability map. This represents the i-th pixel in the second probability map. This represents the i-th pixel in the first binary image. This represents the i-th pixel in the second binary image.

[0023] To address the aforementioned technical problems, another technical solution adopted in this application is: providing an image segmentation model training device, characterized in that the device comprises: an acquisition module for acquiring sample image data, the sample image data including labeled first sample image data and unlabeled second sample image data; a segmentation module for inputting the sample image data into a first segmentation network and a second segmentation network respectively for segmentation to obtain a first probability map and a second probability map, and then processing the first probability map and the second probability map respectively using activation functions to obtain a first binary image and a second binary image; and a training module for calculating the loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and a preset loss function, and iteratively training the first segmentation network and the second segmentation network, the preset loss function including a supervised loss reflecting the segmentation results of the first segmentation network and the second segmentation network themselves, and a pseudo-label semi-supervised loss reflecting the mutual guidance and learning between the first segmentation network and the second segmentation network.

[0024] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a computer device, the computer device including a processor and a memory coupled to the processor, the memory storing program instructions, and when the program instructions are executed by the processor, causing the processor to perform the steps of the image segmentation model training method as described above.

[0025] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a storage medium storing program instructions capable of implementing the image segmentation model training method described above.

[0026] The beneficial effects of this application are as follows: The image segmentation model training method of this application uses a small amount of labeled first sample image data and a large amount of unlabeled second sample image data as input to the first segmentation network and the second segmentation network respectively for prediction, to obtain the segmentation results of the first segmentation network and the second segmentation network respectively. Then, the first segmentation network and the second segmentation network are trained under supervision based on the loss function constructed based on the segmentation results and labels. At the same time, the pseudo-label semi-supervised loss constructed based on the segmentation results of the first segmentation network and the second segmentation network is used to guide the first segmentation network and the second segmentation network to learn from each other. This can reduce the amount of labeled sample data required to train the model, and the model can be trained using a small amount of labeled sample data. At the same time, the robustness of the model is improved by using the mutual guidance learning between the two segmentation networks. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the image segmentation model training method according to the first embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the structure of the second segmentation network according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the feature extraction module of the second segmentation network in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the feature extraction network structure of the feature extraction module in an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of the attention layer structure of the feature extraction network in an embodiment of the present invention;

[0032] Figure 6 This is a flowchart illustrating the dual-segmentation network semi-supervised learning framework according to an embodiment of the present invention.

[0033] Figure 7 This is a flowchart illustrating the image segmentation model training method according to the second embodiment of the present invention;

[0034] Figure 8 This is a simplified diagram illustrating the principle of contrast-limited adaptive histogram equalization in an embodiment of the present invention.

[0035] Figure 9 These are the images and their grayscale distribution histograms before and after contrast-limited adaptive histogram equalization processing according to embodiments of the present invention.

[0036] Figure 10 This is a schematic diagram of the functional modules of the image segmentation model training device according to an embodiment of the present invention;

[0037] Figure 11This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0038] Figure 12 This is a schematic diagram of the structure of the storage medium according to an embodiment of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0040] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0041] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0042] Figure 1 This is a flowchart illustrating the image segmentation model training method according to the first embodiment of the present invention. It should be noted that if substantially the same result is obtained, the method of the present invention is not necessarily identical. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, the image segmentation model training method includes the following steps:

[0043] Step S101: Obtain sample image data, which includes labeled first sample image data and unlabeled second sample image data.

[0044] It should be noted that in this embodiment, the image segmentation model is applied to blood vessel image segmentation. The sample image data includes a small amount of labeled first sample image data and a large amount of unlabeled second sample image data. For example, the first sample image data and the second sample image data can be configured in a 2:8 ratio.

[0045] Step S102: Input the sample image data into the first segmentation network and the second segmentation network respectively for segmentation to obtain the first probability map and the second probability map. Then, use the activation function to process the first probability map and the second probability map respectively to obtain the first binary image and the second binary image.

[0046] Specifically, the sample image data is input into the first segmentation network, which processes the sample image data to obtain a first probability map output by the first segmentation network and a second probability map output by the second segmentation network. Then, the first probability map is input into the Softmax activation function for processing to obtain a first binary image, and the second probability map is input into the Softmax activation function for processing to obtain a second binary image.

[0047] The first segmentation network includes the Unet segmentation network, and the second segmentation network includes the ISUnet (InceptionSwinUnet) segmentation network. The ISUnet segmentation network includes an embedding layer, an encoder, a decoder, and a linear mapping layer.

[0048] It should be noted that after training the Unet segmentation network and the ISUnet segmentation network, the Unet segmentation network was used for angiography image segmentation.

[0049] like Figure 2 As shown, the second segmentation network includes an embedding layer, an encoder, a decoder, and a linear mapping layer. The embedding layer performs a linear transformation on the dimensions of the sample image data to segment the sample image into a series of image patches. The encoder includes alternating cyclic feature extraction and downsampling modules, and the decoder includes alternating cyclic feature extraction and upsampling modules. Figure 3 As shown, the feature extraction module includes a feature extraction network and a feedforward network, with a normalization layer (LayerNorm) between them. Figure 4As shown, the feature extraction network includes a segmentation layer, a first high-frequency image processing module, a second high-frequency image processing module, a low-frequency image processing module, and a fusion layer. The first high-frequency image processing module includes a max pooling layer, a first linear mapping layer, and a first upsampling operation layer. The second high-frequency image processing module includes a second linear mapping layer and a convolution operation layer. The low-frequency image processing module includes an average pooling layer, an attention layer constructed based on window multi-head self-attention mechanism and sliding window multi-head self-attention mechanism, and a second upsampling operation layer. Figure 5 As shown, the attention layer specifically includes windowed multi-head self-attention (W-MSA) and sliding windowed multi-head self-attention (SW-MSA). The linear mapping layer is used to convert the image data obtained after encoder and decoder operations into the desired classification information.

[0050] Specifically, the encoder of the second segmentation network processes the sample images as follows:

[0051] 1. Input the sample image data into the embedding layer and perform a linear transformation to obtain the linearly transformed data;

[0052] 2. The sample image data after linear transformation by the embedding layer is input into the first feature extraction module of the encoder for feature extraction, specifically as follows:

[0053] Y = X + ITM(LayerNorm(X));

[0054] Z = Y + FFN(LayerNorm(Y));

[0055] Where X represents the sample image data after linear transformation, LayerNorm(X) represents the first normalization process of the sample image data after linear transformation, ITM represents the processing of the data after the first normalization process using a feature extraction network, Y represents the data after processing using ITM, LayerNorm(Y) represents the second normalization process of the data after processing using ITM, FFN represents the processing of the data after the second normalization process using a feedforward network, and Z represents the data after processing using FFN.

[0056] The process by which the feature extraction network processes the data after the first normalization is represented as follows:

[0057] X h1 ,X h2 ,X l =Split(X) (1);

[0058] The above formula represents the segmentation layer used to divide the data input to the feature extraction network into high-frequency data and low-frequency data, resulting in the first high-frequency image data X. h1 Second high-frequency image data Xh2 and low-frequency image data X l It should be noted that in the shallow stages of the network, the number of channels is small, and the proportion of high-frequency data is large while the proportion of low-frequency data is small. As the network depth increases, the number of channels increases, and the proportion of high-frequency data decreases while the proportion of low-frequency data increases after network processing. Therefore, comparing the alternating feature extraction modules in the encoder, the ratio between high-frequency and low-frequency data decreases sequentially in each feature extraction module. For example, in the first feature extraction module, the ratio between high-frequency and low-frequency data can be set to 2:1; in the second feature extraction module, it can be set to 1:1; in the third feature extraction module, it can be set to 1:2; and in the fourth feature extraction module, it can be set to 1:3. Then, considering the sensitivity of the max pooling filter and the detail perception of the convolution operation, after dividing the data into high-frequency data and low-frequency data, the high-frequency data is further divided proportionally to obtain the first high-frequency data and the second high-frequency data. Then, the first high-frequency data, the second high-frequency data, and the low-frequency data are processed in parallel using the first high-frequency image processing module, the second high-frequency image processing module, and the low-frequency image processing module in the feature extraction network.

[0059] Y h1 =Upsample1(Linear1(MaxPool(X h1 ))) (2);

[0060] Y h2 =DWConv1(Linear2(X h2 )) (3);

[0061] Y l =Upsample2(Attention(AvePool(X l ))) (4);

[0062] Among them, Y h1 Y represents the data obtained by the first high-frequency image processing module after processing the first high-frequency data. h2 Y represents the data obtained by the second high-frequency image processing module after processing the second high-frequency data. lThis represents the data obtained after processing low-frequency data by the low-frequency image processing module. MaxPool represents the max pooling layer, Linear1 represents the first linear mapping layer, Upsample1 represents the first upsampling operation layer, Linear2 represents the second linear mapping layer, DWConv represents the convolution operation layer, AvePool represents the average pooling layer, Attention represents the attention layer, and Upsample2 represents the second upsampling operation layer. It should be noted that in this embodiment, the max pooling layer reduces the image size during image processing. Therefore, an upsampling operation is performed by the first upsampling operation layer to restore the image. This pooling method reduces image loss during pooling and also reduces the overall computational cost of the model.

[0063] Attention(X)={SWMSABlock(WMSABlock(X))}×depth (5);

[0064]

[0065]

[0066]

[0067] Equations (5) to (8) above illustrate the data processing process of the attention layer, where WMSABlock represents the multi-head self-attention module of the window. z represents an intermediate variable. l Let represent the l-th feature block among multiple small blocks in an image; WMSA represents a windowed multi-head self-attention layer; LN represents a normalization layer; MLP represents a multi-layer linear mapping layer; SWMSABlock represents a sliding windowed multi-head self-attention module; SWMSA represents a sliding windowed multi-head self-attention layer; WMSA(x) represents the processing of input x by the windowed multi-head self-attention layer; softmax represents the activation function; Q, K, and V represent query, key, and value, respectively, and these three parameters are calculated based on the input x; d k K represents the dimension of the parameter K, and T represents the transpose of the matrix.

[0068] Y c =Concatenate((Y h1 ,Y h2 ,Y l ),dim=C) (9);

[0069] Equation (9) represents the relationship between Y and Y. h1 Y h2 Y l To merge, dim=C means to merge along the dimension with channel C.

[0070] Y = ITM(X) = Linear(Y) c +DWConv(Y c )) (10);

[0071] Equation (10) represents the expression for Y... h1 Y h2 Y l The merged data is then subjected to convolution, residual connection, and linear mapping operations.

[0072] 3. Input the feature data obtained by the first feature extraction module into the first downsampling module for downsampling to obtain feature extraction data. Then, use the subsequent feature extraction modules and downsampling modules to process the feature extraction data in sequence until the encoding is completed.

[0073] It should be noted that the feature extraction module in the decoder has the same structure as the feature extraction module in the encoder. Therefore, based on the encoder's processing of the sample image described above, we can know the decoder's processing of the encoder's output data, which will not be repeated here.

[0074] Step S103: Calculate the loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and the preset loss function, and iteratively train the first segmentation network and the second segmentation network. The preset loss function includes a supervised loss that reflects the segmentation results of the first segmentation network and the second segmentation network themselves, and a pseudo-label semi-supervised loss that reflects the mutual guidance and learning between the first segmentation network and the second segmentation network.

[0075] Specifically, such as Figure 6 As shown, Figure 6 A semi-supervised learning framework for dual segmentation networks is demonstrated. The input includes a small amount of labeled first sample image data and a large amount of unlabeled second sample image data. These are fed into a first segmentation network and a second segmentation network for segmentation processing, respectively. The first segmentation network outputs a first probability map and a first binary image after processing the first probability map with a Softmax function, while the second segmentation network outputs a second probability map and a second binary image after processing the second probability map with a Softmax function. The first and second segmentation networks are then trained using a first loss function, a second loss function, a third loss function, and a fourth loss function.

[0076] The preset loss function is expressed as:

[0077] loss=SupervisedLoss1+SupervisedLoss2+0.1*exp(-5(1-ttmax2*CrossTeachingLoss3+CrossTeachingLoss4;

[0078] Where loss represents the preset loss function, SupervisedLoss1 represents the first loss function value, SupervisedLoss2 represents the second loss function value, CrossTeachingLoss3 represents the third loss function value, CrossTeachingLoss4 represents the fourth loss function value, t represents the number of training iterations in the current iteration, and tmax represents the preset total number of training iterations.

[0079] Furthermore, the loss is calculated based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and the preset loss function, including:

[0080] 1. Calculate the first loss function value based on the first probability map, the first binary image, and the label corresponding to each first sample image.

[0081] 2. Calculate the second loss function value based on the second probability map, second binary image, and label corresponding to each first sample image.

[0082] 3. Calculate the value of the third loss function based on the first probability map and the second binary image.

[0083] 4. Calculate the fourth loss function value based on the second probability map and the first binary image.

[0084] It should be noted that the calculation processes for the first, second, third, and fourth loss function values ​​can be performed simultaneously.

[0085] The first loss function value is expressed as:

[0086]

[0087] The second loss function value is expressed as:

[0088]

[0089] The third loss function value is expressed as:

[0090]

[0091] The fourth loss function value is expressed as:

[0092]

[0093] Where SupervisedLoss1 represents the first loss function value, SupervisedLoss2 represents the second loss function value, CrossTeachingLoss3 represents the third loss function value, CrossTeachingLoss4 represents the fourth loss function value, N represents the number of pixels in the image, and y i This represents the label corresponding to the i-th pixel in the first sample image data. This represents the i-th pixel in the first probability map. This represents the i-th pixel in the second probability map. This represents the i-th pixel in the first binary image. This represents the i-th pixel in the second binary image.

[0094] The image segmentation model training method of the first embodiment of the present invention uses a small amount of labeled first sample image data and a large amount of unlabeled second sample image data as inputs to a first segmentation network and a second segmentation network for prediction, respectively, to obtain the segmentation results of the first segmentation network and the second segmentation network. Then, based on the loss function constructed by the segmentation results and labels, the first segmentation network and the second segmentation network are trained under supervision. At the same time, a pseudo-label semi-supervised loss constructed based on the segmentation results of the first segmentation network and the second segmentation network is used to guide the first segmentation network and the second segmentation network to learn from each other. This reduces the amount of labeled sample data required to train the model, and the model can be trained with a small amount of labeled sample data. At the same time, the robustness of the model is improved by using the mutual guidance learning between the two segmentation networks.

[0095] Figure 7 This is a flowchart illustrating the image segmentation model training method according to the second embodiment of the present invention. It should be noted that if substantially the same result is obtained, the method of the present invention does not necessarily require further clarification. Figure 7 The illustrated process sequence is limited. For example... Figure 7 As shown, the image segmentation model training method includes the following steps:

[0096] Step S201: Obtain sample image data, which includes labeled first sample image data and unlabeled second sample image data.

[0097] In this embodiment, Figure 7 Step S201 and Figure 1 The steps in step S101 are similar and will not be repeated here for the sake of simplicity.

[0098] Step S202: Perform contrast enhancement on the sample image data based on the method of limited contrast adaptive histogram equalization.

[0099] Specifically, Contrast-Limited Adaptive Histogram Equalization (CLAHE) evolved from Adaptive Histogram Equalization (AHE). Adaptive Histogram Equalization uses the gray values ​​of local windows to construct a mapping function, ensuring that the gray values ​​are distributed relatively evenly between 0 and 255. To avoid image discontinuities and over-enhancement caused by Adaptive Histogram Equalization, a method to limit the histogram distribution is introduced, namely, Contrast-Limited Adaptive Histogram Equalization. Compared to Adaptive Histogram Equalization, Contrast-Limited Adaptive Histogram Equalization proposes two improvements: First, it proposes a method to limit the histogram distribution by setting a threshold. If a gray level in the histogram exceeds the threshold, it is cropped, and then the portion exceeding the threshold is evenly distributed among the gray levels. The threshold can typically be set by directly specifying the frequency of the gray level or by setting it as a percentage of the total pixels. The latter method is easier to implement, and the cumulative distribution function graph corresponding to the changed image will not change drastically, thus avoiding over-enhancement of noise points. Second, an interpolation method is proposed to accelerate histogram equalization. First, the image is divided into blocks, and a histogram cumulative distribution function is calculated for each block. Each block is referred to as a window. Second, for each pixel in the image, its four neighboring windows are found, and the mapping values ​​of the histogram cumulative distribution functions of the top-left, top-right, bottom-left, and bottom-right windows to a given pixel are calculated. Then, bilinear interpolation is performed to obtain the final mapping value for that pixel. A simplified diagram of the contrast-limited adaptive histogram equalization principle is shown below. Figure 8 As shown, the histograms of the images and their grayscale distribution values ​​before and after contrast-limited adaptive histogram equalization processing for angiography images are as follows. Figure 9 As shown.

[0100] Step S203: Input the sample image data into the first segmentation network and the second segmentation network respectively for segmentation to obtain the first probability map and the second probability map. Then, use the activation function to process the first probability map and the second probability map respectively to obtain the first binary image and the second binary image.

[0101] In this embodiment, Figure 7 Step S203 and Figure 1 Step S102 is similar and will not be repeated here for the sake of simplicity.

[0102] Step S204: Calculate the loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and the preset loss function, and iteratively train the first segmentation network and the second segmentation network. The preset loss function includes a supervised loss that reflects the segmentation results of the first segmentation network and the second segmentation network themselves, and a pseudo-label semi-supervised loss that reflects the mutual guidance and learning between the first segmentation network and the second segmentation network.

[0103] In this embodiment, Figure 7 Step S204 and Figure 1 Step S103 is similar and will not be repeated here for the sake of simplicity.

[0104] The image segmentation model training method of the second embodiment of the present invention enhances the contrast of the sample image data by using contrast adaptive histogram equalization after acquiring the sample image data, thereby improving the signal-to-noise ratio of the sample image data and making it more conducive to accurate coronary artery extraction.

[0105] Figure 10 This is a schematic diagram of the functional modules of the image segmentation model training device according to an embodiment of the present invention. Figure 10 As shown, the image segmentation model training device 20 includes an acquisition module 21, a segmentation module 22, and a training module 23.

[0106] The acquisition module 21 is used to acquire sample image data, which includes labeled first sample image data and unlabeled second sample image data.

[0107] The segmentation module 22 is used to input sample image data into the first segmentation network and the second segmentation network respectively for segmentation to obtain a first probability map and a second probability map. Then, the activation function is used to process the first probability map and the second probability map respectively to obtain a first binary image and a second binary image.

[0108] Training module 23 is used to calculate the loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and the preset loss function, and to iteratively train the first segmentation network and the second segmentation network. The preset loss function includes a supervised loss that reflects the segmentation results of the first segmentation network and the second segmentation network themselves, and a pseudo-label semi-supervised loss that reflects the mutual guidance learning between the first segmentation network and the second segmentation network.

[0109] Optionally, before the segmentation module 22 performs the operation of inputting the sample image data into the first segmentation network and the second segmentation network for segmentation, it is further used to: perform a contrast enhancement operation on the sample image data based on the method of limiting contrast adaptive histogram equalization.

[0110] Optionally, the first segmentation network includes a Unet segmentation network, and the second segmentation network includes an ISUnet segmentation network. The ISUnet segmentation network includes an embedding layer, an encoder, a decoder, and a linear mapping layer.

[0111] Optionally, the encoder includes an alternating cyclic feature extraction module and a downsampling module, and the decoder includes an alternating cyclic feature extraction module and an upsampling module. The feature extraction module includes a feature extraction network and a feedforward network. The feature extraction network includes a segmentation layer, a first high-frequency image processing module, a second high-frequency image processing module, a low-frequency image processing module, and a fusion layer. The first high-frequency image processing module includes a max pooling layer, a first linear mapping layer, and a first upsampling operation layer. The second high-frequency image processing module includes a second linear mapping layer and a convolution operation layer. The low-frequency image processing module includes an average pooling layer, an attention layer constructed based on a window multi-head self-attention mechanism and a sliding window multi-head self-attention mechanism, and a second upsampling operation layer.

[0112] Optionally, the training module 23 performs the operation of calculating the loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and the preset loss function. Specifically, this includes: calculating the first loss function value based on the first probability map, the first binary image, and the label corresponding to each first sample image; calculating the second loss function value based on the second probability map, the second binary image, and the label corresponding to each first sample image; calculating the third loss function value based on the first probability map and the second binary image; and calculating the fourth loss function value based on the second probability map and the first binary image.

[0113] Optionally, the preset loss function is expressed as:

[0114] loss=SupervisedLoss1+SupervisedLoss2+0.1*exp(-5(1-ttmax2*CrossTeachingLoss3+CrossTeachingLoss4;

[0115] Where loss represents the preset loss function, SupervisedLoss1 represents the first loss function value, SupervisedLoss2 represents the second loss function value, CrossTeachingLoss3 represents the third loss function value, CrossTeachingLoss4 represents the fourth loss function value, t represents the number of training iterations in the current iteration, and tmax represents the preset total number of training iterations.

[0116] Optionally, the first loss function value is expressed as:

[0117]

[0118] The second loss function value is expressed as:

[0119]

[0120] The third loss function value is expressed as:

[0121]

[0122] The fourth loss function value is expressed as:

[0123]

[0124] Where SupervisedLoss1 represents the first loss function value, SupervisedLoss2 represents the second loss function value, CrossTeachingLoss3 represents the third loss function value, CrossTeachingLoss4 represents the fourth loss function value, N represents the number of pixels in the image, and y i This represents the label corresponding to the i-th pixel in the first sample image data. This represents the i-th pixel in the first probability map. This represents the i-th pixel in the second probability map. This represents the i-th pixel in the first binary image. This represents the i-th pixel in the second binary image.

[0125] For other details regarding the implementation techniques of each module in the image segmentation model training device of the above embodiments, please refer to the description in the image segmentation model training method of the above embodiments, which will not be repeated here.

[0126] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0127] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Figure 11 As shown, the computer device 30 includes a processor 31 and a memory 32 coupled to the processor 31. The memory 32 stores program instructions. When the program instructions are executed by the processor 31, the processor 31 performs the image segmentation model training method steps described in any of the above embodiments.

[0128] The processor 31 can also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip with signal processing capabilities. The processor 31 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0129] See Figure 12 , Figure 12 This is a schematic diagram of the structure of the storage medium according to an embodiment of the present invention. The storage medium of this embodiment stores program instructions 41 capable of implementing the above-described image segmentation model training method. These program instructions 41 can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or computer devices such as computers, servers, mobile phones, and tablets.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed computer devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0131] Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for training an image segmentation model, characterized in that, The method includes: Acquire sample image data, which includes labeled first sample image data and unlabeled second sample image data; The sample image data is input into the first segmentation network and the second segmentation network respectively for segmentation to obtain the first probability map and the second probability map. Then, the activation function is used to process the first probability map and the second probability map respectively to obtain the first binary image and the second binary image. The loss is calculated based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and a preset loss function. The first segmentation network and the second segmentation network are then iteratively trained. The preset loss function includes a supervised loss reflecting the segmentation results of the first and second segmentation networks themselves, and a pseudo-label semi-supervised loss reflecting the mutual guidance and learning between the first and second segmentation networks. The first segmentation network includes a Unet segmentation network, and the second segmentation network includes an ISUnet segmentation network. The ISUnet segmentation network includes an embedding layer, an encoder, a decoder, and a linear mapping layer. The encoder includes an alternating cyclic feature extraction module and a downsampling module, and the decoder includes an alternating cyclic feature extraction module and an upsampling module. The feature extraction module includes a feature extraction network and a feedforward network. The feature extraction network includes a segmentation layer, a first high-frequency image processing module, a second high-frequency image processing module, a low-frequency image processing module, and a fusion layer. The first high-frequency image processing module includes a max pooling layer, a first linear mapping layer, and a first upsampling operation layer. The second high-frequency image processing module includes a second linear mapping layer and a convolution operation layer. The low-frequency image processing module includes an average pooling layer, an attention layer constructed based on window multi-head self-attention mechanism and sliding window multi-head self-attention mechanism, and a second upsampling operation layer.

2. The image segmentation model training method according to claim 1, characterized in that, Before inputting the sample image data into the first segmentation network and the second segmentation network for segmentation, the method further includes: The sample image data is subjected to contrast enhancement based on a contrast-restricted adaptive histogram equalization method.

3. The image segmentation model training method according to claim 1, characterized in that, The step of calculating the loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and the preset loss function includes: Calculate the first loss function value based on the first probability map, the first binary image, and the label corresponding to each first sample image; The second loss function value is calculated based on the second probability map, the second binary image, and the label corresponding to each of the first sample images. Calculate the third loss function value based on the first probability map and the second binary image; The fourth loss function value is calculated based on the second probability map and the first binary image.

4. An image segmentation model training apparatus using the image segmentation model training method of claim 1, characterized in that, The device includes: The acquisition module is used to acquire sample image data, which includes labeled first sample image data and unlabeled second sample image data; The segmentation module is used to input the sample image data into the first segmentation network and the second segmentation network respectively for segmentation to obtain the first probability map and the second probability map. Then, the activation function is used to process the first probability map and the second probability map respectively to obtain the first binary image and the second binary image. The training module is used to calculate the loss based on the first probability map, the second probability map, the first binary image, the second binary image, the label, and a preset loss function, and to iteratively train the first segmentation network and the second segmentation network. The preset loss function includes a supervised loss that reflects the segmentation results of the first segmentation network and the second segmentation network themselves, and a pseudo-label semi-supervised loss that reflects the mutual guidance learning between the first segmentation network and the second segmentation network.

5. A computer device, characterized in that, The computer device includes a processor and a memory coupled to the processor, the memory storing program instructions that, when executed by the processor, cause the processor to perform the steps of the image segmentation model training method as described in any one of claims 1-3.

6. A storage medium, characterized in that, The system stores program instructions capable of implementing the image segmentation model training method as described in any one of claims 1-3.

Images