A method and system for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation

Abstract

The present invention provides a method and system for constructing a rib segmentation model based on local annotation in CT images using active contrast learning. The method first uses contrast learning to extract common features from a large number of unlabeled samples, reducing the model's reliance on labeled samples. Active learning then selects representative and high-value samples from the dataset as unlabeled samples. Pixel entropy is then used to identify high-uncertainty regions within these unlabeled samples for local annotation, further reducing the amount of annotation required. High-confidence predictions are then added to the training pool as pseudo-labels to expand the training sample set. By selecting more representative samples for learning, the method reduces the likelihood of sample overfitting and reduces the cost of model training.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a method for rib segmentation in CT images, and more particularly to a method and system for rib segmentation in CT images based on active contrast learning using local annotation. Background Art

[0002] Rib segmentation holds significant clinical value in various clinical applications, such as fracture detection, assessing the severity of chest trauma, and providing a reference for lung volume estimation, bone abnormality quantification, and pediatric spinal deformity assessment. However, CT images often contain plain scans, which greatly complicates manual interpretation and increases the workload for imaging experts and medical professionals. Therefore, to meet the needs of automated image interpretation, traditional image processing methods and deep learning methods have been proposed for rib segmentation on CT. However, traditional image processing methods rely heavily on threshold settings and are unable to achieve good segmentation results on difficult CT scans due to individual differences (such as those in osteoporosis patients). Therefore, they cannot solve the rib segmentation problem for all CT scans. In recent years, deep learning has made significant progress in the field of imaging, and its powerful performance in rib segmentation has also gained favor among many scholars and clinical practitioners.

[0003] However, deep learning methods rely on large amounts of labeled data, and rib CT images, unlike natural images, are difficult to collect and label. Collection is difficult because it concerns patient privacy, and medical images cannot be publicly collected in large quantities like natural images. Labeling is also difficult because CT images are large, and the rib pixels in each slice are relatively sparse. This requires experts to spend a considerable amount of time and effort to label a single CT image, shifting the pressure of manual image reading to the pressure of data labeling. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for constructing a rib segmentation model based on local annotation in CT images. The present invention can segment the ribs in CT images with a small amount of annotation and ensure that the segmentation quality is comparable to that when the data is fully annotated.

[0005] To achieve the above objectives, this technical solution provides a method for constructing a rib segmentation model based on local annotation in CT images using active contrast learning. The main principle is to first use contrast learning to extract common features from a large number of unlabeled samples to reduce the model's dependence on labeled samples. At the same time, through active learning, representative and high-value samples in the dataset are selected as samples to be labeled, and / or pixel entropy is used to find high-uncertainty areas in these samples to be labeled for local annotation, further reducing the amount of annotation. Some high-confidence prediction results are added to the training pool as pseudo-labels to expand the training samples. Specifically, the following steps are included:

[0006] Step 1: Obtain a data set; the data set includes unlabeled chest CT image samples and labeled chest CT image samples;

[0007] Step 2: Based on some unlabeled chest CT image samples, bone pixel blocks and bone pixel block pairs consisting of their adjacent pixel blocks are obtained. Based on the bone pixel block pairs, pixel contrast learning is used to pre-train the contrast learning model including the encoder;

[0008] Step 3: After assigning the encoder weights from the pre-trained contrastive learning model to the encoder of the segmentation model, the model is fine-tuned using labeled chest CT image samples. The fine-tuned segmentation model is then used to perform multi-view predictions on the remaining unlabeled chest CT image samples in the dataset. The prediction results are used to distinguish between high-uncertainty samples and high-confidence samples. The more views with the same prediction results, the more consistent the multi-view opinions are, and the higher the confidence level of the sample.

[0009] Step 4: Use the prediction results of high-confidence samples as pseudo-labels, directly cut them out of the sliding window and add them to the training pool and / or obtain the pixel entropy distribution of high-uncertainty samples, send the high-entropy areas to experts for labeling, and cut the labeled areas into the training pool through the sliding window;

[0010] Step 5: Train the segmentation model based on the data in the training pool to obtain the constructed segmentation model.

[0011] Furthermore, there is a partial overlapping area between two pixel blocks in the bone pixel block pair.

[0012] Furthermore, the method for obtaining bone pixel blocks is as follows:

[0013] Non-bone pixels on unlabeled chest CT image samples are screened out based on the first pixel threshold. The approximate location of the ribs is determined based on the connected regions and the remaining regions are removed. The rib regions are then cropped with a sliding window to obtain bone pixel blocks. Bone pixel blocks with larger bone pixels are further obtained based on the second pixel threshold.

[0014] According to the obtained bone pixel block position, another bone pixel block having a partially overlapping area with the bone pixel block is obtained by offsetting and cropping the original unlabeled chest CT image sample to form a bone pixel block pair.

[0015] Furthermore, the pixel contrast learning based on the bone pixel block is used to pre-train the contrast learning model including the encoder, specifically as follows:

[0016] The contrastive learning model includes an encoder, a momentum encoder, and two projection heads connected to the output terminals of the encoder and the momentum encoder, respectively. Each bone pixel block pair is fed into the encoder and the momentum encoder, and the encoder feature representation and the momentum encoder feature representation are output from the two corresponding projection heads, respectively.

[0017] The encoder feature representation and momentum encoder feature representation are upsampled and mapped back to the original 3D space. Based on the encoder feature representation and momentum encoder feature representation mapped back to the original 3D space, the Euclidean distance between the two pixels in the corresponding pixel blocks in the bone pixel block pair is calculated. When the distance is greater than the threshold λ, the pixels are considered to be positive pixels of each other, otherwise they are negative pixels of each other.

[0018] Construct the total loss of the overlapping area of ​​each bone pixel block and back-propagate it to the encoder. The momentum encoder is updated by the exponential moving average until the requirement is met and pre-training is completed.

[0019] The total loss of the overlapping area of ​​each bone pixel block is expressed as follows:

[0020]

[0021]

[0022] Where S and S' represent the pixel sets corresponding to the overlapping areas of each bone pixel block in a pair of bone pixel blocks, S = S'; L pix (i) is the pixel contrast loss for pixel i in each overlapping region, X + 、X - are the sets of pixels that are positive and negative with respect to pixel i, respectively. x and x′ correspond to the feature representations output by the two projection heads, and τ is a hyperparameter.

[0023] Furthermore, the segmentation model adopts AttUNet, UNet, FracNet or nnUNet.

[0024] Furthermore, the multi-view is obtained by the following method:

[0025] Add blank pixels with a width of n / k around each remaining unlabeled chest CT image sample and use a sliding window with a sliding window size of n×n×n and a sliding window step size of n / k to slide the CT image sample, and regard each sliding window as a view; then each pixel will be k 3 View prediction.

[0026] Furthermore, the method of distinguishing high uncertainty samples from high confidence samples according to the prediction results is as follows:

[0027] Calculate the prediction score of each pixel in each sample, where the prediction score of each pixel is the sum of the values ​​assigned to the predicted category of the pixel in each view; the prediction score range is [ak 3 ,bk 3 ], where a and b are the values ​​corresponding to the smaller and larger values ​​in the two categories of predicted categories: rib pixels and non-rib pixels;

[0028] Calculate the uncertainty score of each sample, where the uncertainty score of each sample is the ratio of the product of the number of pixels of each predicted score within the score range of each sample and the weight of the corresponding predicted score to the sum of the number of pixels of each predicted score within the score range of each sample, where the closer the predicted score is to the middle value of the score range, the greater the weight of the predicted score;

[0029] High uncertainty samples and high confidence samples are distinguished based on the uncertainty score of each sample. The higher the uncertainty score, the greater the probability of being a high uncertainty sample.

[0030] Furthermore, in the pixel entropy distribution of the high uncertainty sample, the probability entropy of each pixel is expressed as:

[0031]

[0032] Where C is the total number of categories, p i represents the predicted probability of the i-th category.

[0033] Furthermore, the method further includes repeating steps 1 to 4 until the segmentation model reaches convergence.

[0034] A rib segmentation method based on active contrast learning in CT images based on local annotation is proposed, specifically:

[0035] A chest CT image sample to be segmented is obtained, the chest CT image sample to be segmented is input into the segmentation model constructed by the method, and the segmentation result is obtained by output.

[0036] A system for constructing a rib segmentation model based on active contrast learning in CT images using local annotations includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for constructing a rib segmentation model based on active contrast learning in CT images using local annotations is implemented.

[0037] Compared with the existing technology, this technical solution has the following characteristics and beneficial effects:

[0038] 1. The present invention combines contrastive learning, active learning and local labeling methods. It first extracts more robust features through contrastive learning, and then uses contrastive learning to find high-uncertainty samples and high-confidence samples in the data set, eliminating the labeling cost of other redundant samples. It can also use local labeling technology to further find high-entropy areas in high-uncertainty samples and label them, greatly reducing the amount of labeling in the data set.

[0039] 2. The present invention selects more representative samples for learning, which not only reduces the possibility of sample overfitting, but also reduces the training cost of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of a method for constructing a rib segmentation model using active contrast learning based on local annotation, provided in a specific embodiment of the present invention.

[0041] Figure 2 This is a structural diagram of a system for constructing a rib segmentation model using active contrast learning based on local annotation, provided by a specific embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of multi-view prediction provided by a specific embodiment of the present invention.

[0043] Figure 4 It is a three-dimensional (A, B) and two-dimensional (C, D) schematic diagram locally labeled according to pixel entropy provided by a specific embodiment of the present invention.

[0044] Figure 5 FIG. 4 is a graph showing how the IOU changes with the number of samples in an ablation experiment according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0046] The present invention provides a method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation. Figure 1 As shown, the following steps are included:

[0047] Step 1: Acquire a data set; the data set includes unlabeled chest CT image samples and labeled chest CT image samples;

[0048] Step 2: Obtain bone pixel blocks and bone pixel block pairs consisting of adjacent pixel blocks based on some unlabeled chest CT image samples, and pre-train a contrastive learning model including an encoder using pixel contrastive learning based on the bone pixel block pairs;

[0049] In the present invention, a large number of unlabeled samples are first preprocessed to obtain bone pixel blocks and bone pixel block pairs consisting of their adjacent pixel blocks. Pixel contrast learning is then used to compare semantically similar positive pixel pairs with semantically dissimilar negative pixel pairs in the bone pixel block pairs. Through contrast loss, the contrastive learning model makes the semantically similar positive pixel pairs closer in the representation space, while the semantically dissimilar negative pixel pairs are farther apart in the representation space, thereby preliminarily obtaining an encoder capable of segmenting ribs.

[0050] In a specific implementation case, there is a partial overlap between the two pixel blocks in the bone pixel block pair, wherein the overlapped area is a set of semantically similar positive pixel pairs. In a more specific implementation case, the bone pixel blocks and bone pixel block pairs are obtained by the following method:

[0051] The window width and window position of the unlabeled chest CT image were adjusted, and most non-bone pixels were screened out according to the self-set first pixel threshold (90). The approximate position of the ribs was determined based on the connected area and the remaining area was removed. The rib area was cropped with a sliding window to obtain bone pixel blocks. The second pixel threshold (1000) was then set to further screen out bone pixel blocks with fewer bone pixels and obtain bone pixel blocks with larger bone pixels.

[0052] During the sliding window cropping process, the position of each bone pixel block in the original image is saved. By slightly offsetting the position, another bone pixel block can be cropped to form a pixel block pair. Specifically, assuming the three-dimensional coordinates of the upper right corner of the original pixel block are (x0, y0, z0), then the coordinates of the upper right corner of the offset pixel block are (x0+s·l, y0+s·l, z0+s·l), where s is the offset (0<s<1) and l is the length of the pixel block (l=64).

[0053] Pixel contrastive learning uses a contrastive learning framework to extract common features from a large amount of unlabeled data. In a specific implementation case, the contrastive learning model framework includes an encoder, a momentum encoder, and two projection heads connected to the output of the encoder and momentum encoder respectively. The encoder uses the same structure as the segmentation model encoder to ensure that the weights can be assigned to subsequent fine-tuning tasks. In contrastive learning, the bone pixel block pairs are sent to the encoder and momentum encoder respectively, and the encoder features and momentum encoder features are obtained respectively. In order to prevent the contrast collapse caused by the two being too similar, the projection head projects a more differentiated feature representation, that is, the encoder feature representation and momentum encoder feature representation are obtained respectively. Among them, the projection head can adopt a structure consisting of a convolutional layer, a Batch Norm layer, a ReLU layer and a convolutional layer connected in sequence.

[0054] The pixel contrast loss is then calculated between the two feature representations and back-propagated to the encoder, while the momentum encoder is updated by EMA (Exponential Moving Average) until the requirements are met (such as the set number of iterations or the loss function converges), completing the pre-training.

[0055] Among them, the contrast loss construction process is as follows:

[0056] First, the encoder feature representation and momentum encoder feature representation are upsampled and mapped back to the original 3D space, and the Euclidean distance between each pixel is calculated. When the distance is less than the threshold λ (0.7), the pixel pair is considered positive, otherwise it is negative. Then the contrast loss L of each pixel in the overlapping area is pix (i) and EMA are as follows:

[0057]

[0058] θ t+1 =m·θ t +(1-m)·φ t (2) Where i is the pixel in the overlapping area of ​​the bone pixel block pair, S and S' respectively represent the pixel set corresponding to the overlapping area of ​​each bone pixel block in a pair of bone pixel blocks, S = S', X + 、X - They are respectively the set of the other pixels in the positive pixel pair and the negative pixel pair with pixel i, that is, X + represents the set of positive pixels, X - represents the negative pixel set, x, x′ correspond to the feature representations output by the two projection heads, namely the encoder feature representation and the momentum encoder feature representation, respectively, and τ is a hyperparameter, which is equal to 0.3 in this embodiment.

[0059] Theta in EMA t+1is the parameter of the next round of momentum encoder, θ t is the parameter of this round t momentum encoder, φ t is the parameter of this round of encoder, m is the momentum coefficient, which is between 0 and 1. m close to 1 means that the new information has little influence, and m close to 0 means that the new information has a greater influence. Then calculate the total loss of the overlapping area:

[0060]

[0061] Among them L pix (i) are the pixel contrast losses obtained by the two feature representations, and |S| represents the size of the set S.

[0062] Step 3: After assigning the encoder weights in the pre-trained contrastive learning model to the encoder of the segmentation model, it is trained and fine-tuned using labeled chest CT image samples. The fine-tuned segmentation model is used to perform multi-view prediction on the remaining unlabeled chest CT image samples in the dataset. High-uncertainty samples and high-confidence samples are distinguished based on the prediction results. The more views with the same prediction results, the more consistent the opinions of the multiple views are, and the higher the confidence of the samples.

[0063] Among them, the segmentation model adopts any segmentation model containing an encoder structure, such as AttUNet, UNet, FracNet and nnUNet. In a specific embodiment case, the segmentation model adopts AttUNet to assign the encoder weights pre-trained in contrastive learning to the encoder of AttUNet, and then fine-tune it with a small amount of labeled samples to further improve the segmentation accuracy of the segmentation model. The fine-tuned model is used to predict the remaining unlabeled samples in the data set to obtain new training data.

[0064] The present invention uses multi-view prediction results to assist in screening, and calculates the prediction score of each pixel in each sample, where the prediction score of each pixel is the sum of the corresponding values ​​of the predicted category of the pixel in each view; the score range of the prediction score is [a, b], where a and b are the smaller and larger values ​​corresponding to the two categories of predicted categories, rib pixels and non-rib pixels, respectively; and then calculates the uncertainty score of each sample, where the uncertainty score of each sample is the ratio of the product of the sum of the number of pixels of each predicted score within the score range of each sample and the weight of the corresponding predicted score to the sum of the number of pixels of each predicted score within the score range of each sample, where the closer the prediction score is to the middle value of the score range, the greater the weight of the prediction score. In a specific embodiment case, during the prediction process, a sliding window is used to slide each CT to obtain multiple views. Assuming that the sliding window size is n×n×n (for example, n=64), the step size of the sliding window is n / k (for example, k=2), and each sliding window is regarded as a view, then each pixel point will be k 3In order to predict the CT boundary, k 3 Secondly, this embodiment intentionally adds blank pixels with a width of n / k around the CT. Figure 3 The figure shows a schematic diagram of multi-view prediction when n = 64 and k = 2. For example, if the prediction categories are rib pixels and non-rib pixels, and the values ​​are assigned to 1 and 0 respectively, if a pixel in a view is considered a rib pixel, the prediction score of the pixel will be increased by 1. As the sliding window predicts the entire CT, each pixel will receive a prediction score in the range of [0, k 3 ]. The pixel score is close to the middle of the interval, indicating that k 3 The prediction results of the two views for this pixel point have a greater difference, and the closer to the two sides, the greater the difference in k 3 The opinions of the views tend to be consistent. Based on this, the uncertainty score of each sample is as follows:

[0065]

[0066] Among them, score(i) represents the number of pixels with score i, and weight(i) represents the weight assigned to the pixel with score i. Because the pixels with scores at the two ends of the interval (scores 0 and 8) are the most confident of the model, and the pixels in the middle of the interval (score 4) are the least confident of the model, it is necessary to assign higher weights to the pixels with scores close to the middle of the interval, and assign lower weights to the pixels with scores close to the two ends of the interval. In this embodiment, the weights from score 4 to the two ends are set to 0.4, 0.3, 0.2, 0.1 and 0 respectively.

[0067] The higher the uncertainty score, the greater the probability of a high-uncertainty sample. Finally, based on each sample's uncertainty score, we distinguish between high-uncertainty samples and high-confidence samples. This distinction can be made using a threshold, or we can simply select several samples with the highest uncertainty scores as high-uncertainty samples and several samples with the lowest uncertainty scores as high-confidence samples. In this example, the top 20 samples with the highest scores in each round are selected as high-uncertainty samples, and those with scores less than 0.007 are selected as high-confidence samples.

[0068] Step 4: The prediction results of high-confidence samples are directly added to the training pool as pseudo-labels and / or the pixel entropy distribution of high-uncertainty samples is obtained, the high-entropy areas are sent to experts for annotation, the annotated areas are cropped with sliding windows, and added to the training pool; further, the pixel entropy distribution of high-uncertainty samples is obtained, the high-entropy areas are sent to experts for annotation, the cropped areas are cropped with sliding windows, and added to the training pool;

[0069] In the above technical solution, the prediction results of high-confidence samples are used as pseudo-labels and added to the training pool as new training data. This method is self-training. The prediction results of high-confidence samples are post-processed to remove some noise by removing small connected domains. In this embodiment, all connected domains with a size of less than 5000 are discarded. The processed prediction results are regarded as pseudo-labels and used to train the model.

[0070] Furthermore, in the above technical solution, we can further find areas in the sample that are more valuable for annotation. We first use pixel entropy to visualize the uncertainty distribution of the sample. The probability entropy of each pixel is expressed as:

[0071]

[0072] Where C is the total number of categories, p i represents the predicted probability of the i-th category.

[0073] The entropy of all pixels within the sample is normalized, and a threshold is set to demarcate high and low uncertainty regions. In this example, regions with entropy greater than a threshold of 0.6 are considered high uncertainty regions, while those below 0.6 are considered low uncertainty regions. Bounding boxes are drawn around high entropy regions and sent to experts for annotation. Finally, the annotated regions are cropped using a sliding window and added to the training pool as new training data, further expanding the training sample size. Figure 4 It is a three-dimensional (A, B) and two-dimensional (C, D) schematic diagram locally labeled according to pixel entropy provided by a specific embodiment of the present invention.

[0074] Step 5: Train the segmentation model based on the data in the training pool to obtain the constructed segmentation model.

[0075] In a specific embodiment, in order to further improve the accuracy of the segmentation model, the above steps 1 to 4 may be repeated until the model reaches convergence.

[0076] Corresponding to the aforementioned embodiment of a method for constructing a rib segmentation model by active contrast learning based on local annotation in CT images, the present invention also provides an embodiment of a system for constructing a rib segmentation model by active contrast learning based on local annotation in CT images.

[0077] An embodiment of the present invention provides a system for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation, including one or more processors for implementing a method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation in the above embodiment.

[0078] An embodiment of a system for constructing a rib segmentation model based on active contrast learning in CT images using local annotations in the present invention can be applied to any device with data processing capabilities, such as a computer or other device.

[0079] The device embodiments may be implemented through software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logically defined device, a processor of any device with data processing capabilities reads the corresponding computer program instructions from a non-volatile memory into the memory and executes them. From a hardware perspective, this includes a processor, memory, a network interface, and a non-volatile memory. In addition, any device with data processing capabilities in which the device in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0080] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0081] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0082] In another specific embodiment, the system framework of the present invention is as follows Figure 2 As shown. It contains four modules: contrast learning module, active learning module, local labeling module and training module. 1) The contrast learning module is used to obtain bone pixel blocks and bone pixel block pairs composed of their adjacent pixel blocks based on some unlabeled chest CT image samples, and pre-train the contrast learning model containing the encoder using pixel contrast learning based on the bone pixel block pairs; a large number of pixel block pairs are intercepted from the unlabeled CT images, and a large number of pixel points are compared to achieve the purpose of distinguishing rib pixels from non-rib pixels, so that the encoder of the segmentation model has the ability to extract more detailed features. 2) The active learning module relies on uncertainty to select samples, such as Figure 3The multi-view prediction method shown in the figure predicts the unlabeled samples in the dataset to obtain the uncertainty score of each sample. The prediction results of the samples with lower scores are used as pseudo labels, and the samples with higher scores are sent to the local annotation module. 3) The local annotation module uses pixel entropy to explore the uncertainty distribution of the sample. After normalizing the pixel entropy in CT and setting a threshold, the distribution of the uncertainty area is as follows: Figure 4 As shown, red areas represent high uncertainty regions, which are more valuable to label than other areas. Therefore, only these areas are sent to experts for labeling. 4) The training module adds the pseudo-labels output by the sample selection module and the locally labeled samples output by the local labeling module to the training pool to train the model, thereby achieving the goal of training the segmentation model with fewer annotations.

[0083] The following is a detailed description of the technical effects of the present invention in combination with performance testing and experimental analysis. In order to verify the effect of the present invention in reducing the labeling cost, this embodiment selects the segmentation model AttUNet to use the local labeled samples selected by the present invention in the data set RibSegV2 to compare with the full labeled samples. The RibSegV2 data set contains a total of 660 samples (training: 420, verification: 80, test: 160). We randomly select 200 samples in the training set as unlabeled samples for comparative learning, and randomly select 20 samples as the initial training set for active learning. The remaining 200 samples in the training set are used as unlabeled samples for the model to select high-confidence samples and high-uncertainty samples. The results are shown in the following table:

[0084]

[0085] As can be seen from the above table, the present invention can achieve results comparable to those of full-scale labeled data using only a small amount of labeled data, and greatly shortens the training time.

[0086] In order to further prove the effectiveness of the present invention, an ablation experiment was also conducted. The present invention combines contrastive learning (step 2), active learning (relying on uncertainty to select samples, such as Figure 3 The multi-view prediction method shown in the figure predicts the unlabeled samples in the dataset to obtain the uncertainty score of each sample, uses the prediction results of samples with lower scores as pseudo labels, and distinguishes high uncertainty samples from high confidence samples based on the prediction results), local annotation (obtains the pixel entropy distribution of high uncertainty samples, sends the high entropy area to experts for annotation, and slides and crops the labeled area to add it to the training pool), and self-training (the prediction results of high confidence samples are used as pseudo labels as new training data and added to the training pool). The four parts are added in sequence, and the results are compared with the full amount of labeled data. The results are shown in the figure. Figure 5 As shown, Figure 5The IOU results of the model test corresponding to adding 20 samples in each round are shown. The effect of traditional deep learning using the full amount of labeled data (420 samples) is comparable to the effect of the present invention using contrastive learning. Contrastive learning + active learning further reduces the demand for labeled samples, and a good effect is achieved using only 120 labeled data. Contrastive learning + active learning + local labeling reduces the area that needs to be labeled for a single sample, while also using only 120 labeled data, and maintains the model performance at the same level. Contrastive learning + active learning + local labeling + self-training improves the level of model training, allowing the model to maintain a high level in any round.

[0087] In summary, the above experiments demonstrate the effectiveness of the present invention, which significantly reduces the number of CT image annotations while still enabling the model to maintain the same segmentation performance as that achieved with full annotated data.

[0088] Corresponding to the aforementioned embodiment of a method for constructing a rib segmentation model by active contrast learning based on local annotation in CT images, the present invention also provides an embodiment of a method for active contrast learning based on local annotation in CT images.

[0089] A rib segmentation method based on active contrast learning in CT images based on local annotation is proposed, specifically:

[0090] Obtain a chest CT image sample to be segmented, input the chest CT image sample to be segmented into the segmentation model constructed by the above method, and output the segmentation result.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation, characterized in that: The following steps are involved: Step 1: Obtain a data set; the data set includes unlabeled chest CT image samples and labeled chest CT image samples; Step 2: Based on some unlabeled chest CT image samples, bone pixel blocks and bone pixel block pairs consisting of their adjacent pixel blocks are obtained. Based on the bone pixel block pairs, pixel contrast learning is used to pre-train the contrast learning model including the encoder; Step 3: After assigning the encoder weights from the pre-trained contrastive learning model to the encoder of the segmentation model, the model is fine-tuned using labeled chest CT image samples. The fine-tuned segmentation model is then used to perform multi-view predictions on the remaining unlabeled chest CT image samples in the dataset. The prediction results are used to distinguish between high-uncertainty samples and high-confidence samples. The more views with the same prediction results, the more consistent the multi-view opinions are, and the higher the confidence level of the sample. Step 4: Use the prediction results of high-confidence samples as pseudo-labels, directly cut them out of the sliding window and add them to the training pool and / or obtain the pixel entropy distribution of high-uncertainty samples, send the high-entropy areas to experts for labeling, and cut the labeled areas into the training pool through the sliding window; Step 5: Train the segmentation model based on the data in the training pool to obtain the constructed segmentation model.

2. The method according to claim 1, characterized in that There is a partial overlapping area between two pixel blocks in the bone pixel block pair.

3. The method according to claim 2, characterized in that The method to obtain bone pixel blocks is as follows: Non-bone pixels on unlabeled chest CT image samples are screened out based on the first pixel threshold. The approximate location of the ribs is determined based on the connected regions and the remaining regions are removed. The rib regions are then cropped with a sliding window to obtain bone pixel blocks. Bone pixel blocks with larger bone pixels are further obtained based on the second pixel threshold. According to the obtained bone pixel block position, another bone pixel block having a partially overlapping area with the bone pixel block is obtained by offsetting and cropping the original unlabeled chest CT image sample to form a bone pixel block pair.

4. The method according to claim 2, characterized in that The pixel contrast learning based on the bone pixel block is used to pre-train the contrast learning model including the encoder, specifically as follows: The contrastive learning model includes an encoder, a momentum encoder, and two projection heads connected to the output terminals of the encoder and the momentum encoder, respectively. Each bone pixel block pair is fed into the encoder and the momentum encoder, and the encoder feature representation and the momentum encoder feature representation are output from the two corresponding projection heads, respectively. The encoder feature representation and momentum encoder feature representation are upsampled and mapped back to the original 3D space. Based on the encoder feature representation and momentum encoder feature representation mapped back to the original 3D space, the Euclidean distance between the two pixels in the corresponding pixel blocks in the bone pixel block pair is calculated. When the distance is greater than the threshold λ, the pixels are considered to be positive pixels of each other, otherwise they are negative pixels of each other. Construct the total loss of the overlapping area of ​​each bone pixel block and back-propagate it to the encoder. The momentum encoder is updated by the exponential moving average until the requirement is met and pre-training is completed. The total loss of the overlapping area of ​​each bone pixel block is expressed as follows: Where S and S' represent the pixel sets corresponding to the overlapping areas of each bone pixel block in a pair of bone pixel blocks, S = S'; L pix (i) is the pixel contrast loss for pixel i in each overlapping region, X + 、X - are the sets of pixels that are positive and negative with respect to pixel i, respectively. x and x′ correspond to the feature representations output by the two projection heads, and τ is a hyperparameter.

5. The method according to claim 1, wherein The segmentation model adopts AttUNet, UNet, FracNet or nnUNet.

6. The method according to claim 1, characterized in that The multi-view is specifically obtained by the following method: Add blank pixels with a width of n / k around each remaining unlabeled chest CT image sample and use a sliding window with a sliding window size of n×n×n and a sliding window step size of n / k to slide the CT image sample, and regard each sliding window as a view; then each pixel will be k 3 View prediction.

7. The method according to claim 6, characterized in that The method of distinguishing high uncertainty samples from high confidence samples based on the prediction results is as follows: Calculate the prediction score of each pixel in each sample, where the prediction score of each pixel is the sum of the values ​​assigned to the predicted category of the pixel in each view; the prediction score range is [ak 3 ,bk 3 ], where a and b are the values ​​corresponding to the smaller and larger values ​​in the two categories of predicted categories: rib pixels and non-rib pixels; Calculate the uncertainty score of each sample, where the uncertainty score of each sample is the ratio of the product of the number of pixels of each predicted score within the score range of each sample and the weight of the corresponding predicted score to the sum of the number of pixels of each predicted score within the score range of each sample, where the closer the predicted score is to the middle value of the score range, the greater the weight of the predicted score; High uncertainty samples and high confidence samples are distinguished based on the uncertainty score of each sample. The higher the uncertainty score, the greater the probability of being a high uncertainty sample.

8. The method according to claim 1, characterized in that In the pixel entropy distribution of the high uncertainty sample, the probability entropy of each pixel is expressed as: Where C is the total number of categories, p i represents the predicted probability of the i-th category.

9. A rib segmentation method based on active contrast learning in CT images based on local annotation, characterized in that: Specifically: Obtain a chest CT image sample to be segmented, input the chest CT image sample to be segmented into the segmentation model constructed by the method according to any one of claims 1 to 8, and output the obtained segmentation result.

10. A system for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation as described in any one of claims 1 to 8.

Images