Method and system for constructing rib active contrast learning segmentation model based on local annotation in CT (Computed Tomography) image

Abstract

The invention provides a method and system for constructing a rib active comparative learning segmentation model based on local labeling in a CT image, and the method comprises the steps: extracting common features from a large number of unlabeled samples through comparative learning, and reducing the dependence of a model on labeled samples; meanwhile, representative and high-value samples in the data set are selected through active learning to serve as to-be-labeled samples, high-uncertainty areas in the to-be-labeled samples are found out through pixel entropy to conduct local labeling, the labeling amount is further reduced, some high-confidence prediction results serve as pseudo labels to be added into a training pool, and the prediction accuracy is improved. And expanding the training samples. According to the method, more representative samples are screened out for learning, so that the possibility of sample overfitting is reduced, and the training cost of the model is reduced.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a rib segmentation method in CT images, and more particularly to a rib active contrast learning segmentation method and system based on local annotation in CT images. Background Art

[0002] Rib segmentation has important clinical value in various clinical applications, such as fracture detection, assessment of chest trauma severity, and reference for lung volume estimation, bone abnormality quantification, and pediatric spinal deformity. However, CT images have a large number of plain scans, which makes manual reading very difficult and increases the workload of imaging experts and medical workers. Therefore, in order to meet the needs of automatic reading, traditional image processing methods and deep learning methods are proposed to perform rib segmentation on CT. However, traditional image processing methods rely too much on threshold setting, and cannot achieve good segmentation effects on individual difficult CTs (such as osteoporosis patients) due to individual differences. Therefore, they cannot solve the rib segmentation problem of all CTs. In recent years, deep learning has made great achievements in the field of images. In the task of rib segmentation, it has also won the favor of many scholars and clinical workers with its powerful performance.

[0003] However, deep learning methods rely on a large amount of labeled data, and rib CT images are different from natural images in that they are difficult to collect and difficult to label. Collection is difficult because it involves patient privacy, and medical images cannot collect a large amount of data publicly like natural images; labeling is difficult because CT images are too large, and the rib pixels of each slice are relatively sparse, which means that experts need to spend a lot of time and energy to label a CT image, causing medical staff to shift the pressure of manual film reading to the pressure of labeling data. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for constructing a rib active contrast learning segmentation model based on local annotation in CT images. The present invention can segment the ribs in CT images with a small amount of annotation and ensure that the segmentation quality is comparable to that when the data is fully annotated.

[0005] To achieve the above objectives, this technical solution provides a method for constructing a rib segmentation model based on local annotation in CT images by active contrast learning. The main principle is to first use contrast learning to extract common features from a large number of unlabeled samples to reduce the model's dependence on labeled samples, and then select representative and high-value samples in the data set as samples to be labeled through active learning, and / or use pixel entropy to find high-uncertainty areas in these samples to be labeled for local annotation, further reducing the amount of annotation, and adding some high-confidence prediction results as pseudo-labels to the training pool to expand the training samples. Specifically, the following steps are included:

[0006] Step 1: Obtain a data set; the data set includes unlabeled chest CT image samples and labeled chest CT image samples;

[0007] Step 2: Based on some unlabeled chest CT image samples, bone pixel blocks and bone pixel block pairs consisting of their adjacent pixel blocks are obtained, and pixel contrast learning is used to pre-train a contrast learning model including an encoder based on the bone pixel block pairs;

[0008] Step 3: After assigning the encoder weights in the pre-trained contrastive learning model to the encoder of the segmentation model, the labeled chest CT image samples are used for training and fine-tuning. The fine-tuned segmentation model is used to perform multi-view prediction on the remaining unlabeled chest CT image samples in the dataset, and high-uncertainty samples and high-confidence samples are distinguished according to the prediction results. The more views with the same prediction results, the more consistent the opinions of the multiple views are, and the higher the confidence of the samples is.

[0009] Step 4: Use the prediction results of high-confidence samples as pseudo labels, directly slide and crop them into the training pool, and / or obtain the pixel entropy distribution of high-uncertainty samples, send the high-entropy areas to experts for labeling, and slide and crop the labeled areas into the training pool;

[0010] Step 5: Train the segmentation model based on the data in the training pool to obtain the constructed segmentation model.

[0011] Furthermore, there is a partial overlapping area between two pixel blocks in the bone pixel block pair.

[0012] Furthermore, the method for obtaining the bone pixel block is as follows:

[0013] According to the first pixel threshold, non-bone pixels on the unlabeled chest CT image samples are screened out, the approximate position of the ribs is determined according to the connected area and the remaining area is removed, and the rib area is cropped with a sliding window to obtain bone pixel blocks; according to the second pixel threshold, bone pixel blocks with larger bone pixels are further obtained;

[0014] According to the obtained bone pixel block position, another bone pixel block having a partially overlapping area with the bone pixel block is obtained by offsetting and cropping the original unlabeled chest CT image sample to form a bone pixel block pair.

[0015] Furthermore, the bone pixel block-based pixel contrast learning is used to pre-train the contrast learning model including the encoder, as follows:

[0016] The contrastive learning model includes an encoder, a momentum encoder and two projection heads respectively connected to the output ends of the encoder and the momentum encoder; each bone pixel block pair is respectively sent to the encoder and the momentum encoder, and the encoder feature representation and the momentum encoder feature representation are respectively obtained from the outputs of the two corresponding projection heads;

[0017] The encoder feature representation and the momentum encoder feature representation are upsampled and mapped back to the original 3D space, and the Euclidean distance between the two pixels in the corresponding pixel blocks in the bone pixel block pair is calculated based on the encoder feature representation and the momentum encoder feature representation mapped back to the original 3D space. When the distance is greater than a threshold λ, the pixels are considered to be positive pixels of each other, otherwise they are negative pixels of each other;

[0018] Construct the total loss of the overlapping area of ​​each bone pixel block and back-propagate it to the encoder. The momentum encoder is updated by the exponential moving average until the requirement is met and pre-training is completed.

[0019] The total loss of the overlapping area of ​​each bone pixel block is expressed as follows:

[0020]

[0021]

[0022] Where S and S' represent the pixel sets of the overlapping areas corresponding to each bone pixel block in a pair of bone pixel blocks, S = S'; L pix (i) is the pixel contrast loss for pixel i in each overlapping region, X + , X - are the sets of pixels that are positive and negative pixels with respect to pixel i, respectively. x and x′ correspond to the feature representations output by the two projection heads, respectively. τ is a hyperparameter.

[0023] Furthermore, the segmentation model adopts AttUNet, UNet, FracNet or nnUNet.

[0024] Furthermore, the multiple views are obtained by the following method:

[0025] Add blank pixels with a width of n / k around each remaining unlabeled chest CT image sample and use a sliding window with a sliding window size of n×n×n and a sliding window step size of n / k to slide the CT image sample, and regard each sliding window as a view; then each pixel point will be k 3 View prediction.

[0026] Furthermore, the method of distinguishing high uncertainty samples from high confidence samples according to the prediction results is specifically as follows:

[0027] Calculate the prediction score of each pixel in each sample, where the prediction score of each pixel is the sum of the values ​​assigned to the predicted category of the pixel in each view; the prediction score range is [ak 3 ,bk 3 ], where a and b are the values ​​corresponding to the smaller and larger values ​​in the two categories of predicted categories, rib pixels and non-rib pixels, respectively;

[0028] Calculate the uncertainty score of each sample, where the uncertainty score of each sample is the ratio of the product of the number of pixels of each predicted score within the score range of each sample and the weight of the corresponding predicted score to the sum of the number of pixels of each predicted score within the score range of each sample, wherein the closer the predicted score is to the middle value of the score range, the greater the weight of the predicted score;

[0029] High uncertainty samples and high confidence samples are distinguished based on the uncertainty score of each sample. The higher the uncertainty score, the greater the probability of being a high uncertainty sample.

[0030] Furthermore, in the pixel entropy distribution of the high uncertainty sample, the probability entropy of each pixel is expressed as:

[0031]

[0032] Where C is the total number of categories, p i represents the predicted probability of the i-th category.

[0033] Furthermore, the method further includes repeating steps one to four until the segmentation model reaches convergence.

[0034] A rib segmentation method based on active contrast learning of local annotation in CT images, specifically:

[0035] A chest CT image sample to be segmented is obtained, the chest CT image sample to be segmented is input into the segmentation model constructed by the method, and the segmentation result is obtained by output.

[0036] A system for constructing a rib segmentation model based on active contrast learning in CT images based on local annotations includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotations is implemented.

[0037] Compared with the prior art, this technical solution has the following characteristics and beneficial effects:

[0038] 1. The present invention combines contrastive learning, active learning and local labeling methods. It first extracts more robust features through contrastive learning, and then uses contrastive learning methods to find high-uncertainty samples and high-confidence samples in the data set, thereby eliminating the labeling cost of other redundant samples. It can also use local labeling technology to further find high-entropy areas in high-uncertainty samples for labeling, thereby greatly reducing the amount of labeling in the data set.

[0039] 2. The present invention selects more representative samples for learning, which not only reduces the possibility of sample overfitting, but also reduces the training cost of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of a method for constructing a rib segmentation model based on active contrast learning based on local annotation provided by a specific embodiment of the present invention.

[0041] Figure 2 It is a structural diagram of a system for constructing a rib segmentation model based on active contrast learning based on local annotation provided by a specific embodiment of the present invention.

[0042] Figure 3 It is a schematic diagram of multi-view prediction provided by a specific embodiment of the present invention.

[0043] Figure 4 It is a three-dimensional (A, B) and two-dimensional (C, D) schematic diagram locally labeled according to pixel entropy provided by a specific embodiment of the present invention.

[0044] Figure 5 It is a curve diagram showing the change of IOU with the number of samples in an ablation experiment provided by a specific embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] The present invention provides a method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation, such as Figure 1 As shown, the following steps are included:

[0047] Step 1: Obtain a data set; the data set includes unlabeled chest CT image samples and labeled chest CT image samples;

[0048] Step 2: based on some unlabeled chest CT image samples, bone pixel blocks and bone pixel block pairs consisting of adjacent pixel blocks are obtained, and a contrast learning model including an encoder is pre-trained using pixel contrast learning based on the bone pixel block pairs;

[0049] In the present invention, a large number of unlabeled samples are first preprocessed to obtain bone pixel blocks and bone pixel block pairs composed of their adjacent pixel blocks, and pixel contrast learning is used to compare semantically similar positive pixel pairs and semantically dissimilar negative pixel pairs in the bone pixel block pairs. Through contrast loss, the contrast learning model makes the semantically similar positive pixel pairs closer in the representation space, while the semantically dissimilar negative pixel pairs are farther away in the representation space, and preliminarily obtains an encoder with the ability to segment ribs.

[0050] In a specific implementation case, there is a partial overlap between two pixel blocks in the bone pixel block pair, wherein the overlapped area is a set of positive pixel pairs with similar semantics. In a more specific implementation case, the bone pixel blocks and bone pixel block pairs are obtained by the following method:

[0051] The window width and window position of the unlabeled chest CT image were adjusted, and most of the non-bone pixels were screened out according to the self-set first pixel threshold (90). The approximate position of the ribs was determined according to the connected area and the remaining area was removed. The rib area was cropped by sliding the window to obtain the bone pixel block, and then the second pixel threshold (1000) was set to further screen out the bone pixel blocks with fewer bone pixels to obtain the bone pixel blocks with larger bone pixels.

[0052] In the process of sliding window cropping, the position of each bone pixel block in the original image is saved, and another bone pixel block can be cropped by slightly offsetting the position to form a pixel block pair. Specifically, assuming that the three-dimensional coordinates of the upper right corner of the initial pixel block are (x0, y0, z0), then the coordinates of the upper right corner of the offset pixel block are (x0+s·l, y0+s·l, z0+s·l), where s is the offset (0<s<1) and l is the length of the pixel block (l=64).

[0053] Pixel contrast learning uses a contrast learning framework to extract common features from a large amount of unlabeled data. In a specific implementation case, the contrast learning model framework includes an encoder, a momentum encoder, and two projection heads connected to the output of the encoder and momentum encoder respectively. The encoder uses the same structure as the segmentation model encoder to ensure that weights can be assigned to subsequent fine-tuning tasks. In contrastive learning, the bone pixel block pairs are sent to the encoder and momentum encoder in turn to obtain encoder features and momentum encoder features respectively. In order to prevent the contrast collapse caused by the two being too similar, the projection head projects a more differentiated feature representation, that is, the encoder feature representation and momentum encoder feature representation are obtained respectively. Among them, the projection head can adopt a structure consisting of a convolutional layer, a Batch Norm layer, a ReLU layer and a convolutional layer connected in sequence.

[0054] The pixel contrast loss is then calculated between the two feature representations and back-propagated to the encoder, while the momentum encoder is updated by EMA (Exponential Moving Average) until the requirements are met (such as reaching the set number of iterations or the loss function converges), completing the pre-training.

[0055] Among them, the contrast loss construction process is as follows:

[0056] First, the encoder feature representation and momentum encoder feature representation are upsampled and mapped back to the original 3D space, and the Euclidean distance between each pixel is calculated. When the distance is less than the threshold λ(0.7), the pixel pair is considered positive, otherwise it is negative. Then the contrast loss L of each pixel in the overlapping area is pix (i) and EMA are as follows:

[0057]

[0058] θ t+1 =m·θ t +(1-m)·φ t (2) Where i is the pixel in the overlapping area of ​​the bone pixel block pair, S and S' respectively represent the pixel set corresponding to the overlapping area of ​​each bone pixel block in a pair of bone pixel blocks, S = S', X + , X - are the sets of the other pixels in the positive pixel pair and the negative pixel pair that form a pair with the i pixel, that is, X + represents the positive pixel set, X - represents a set of negative pixels, x, x′ respectively correspond to the feature representations output by the two projection heads, namely, the encoder feature representation and the momentum encoder feature representation, and τ is a hyperparameter, which is equal to 0.3 in this embodiment.

[0059] Theta in EMAt+1 is the parameter of the next round of momentum encoder, θ t is the parameter of this round of momentum encoder, φ t is the parameter of this round of encoder, m is the momentum coefficient, which is between 0 and 1. m close to 1 means that the new information has little impact, and m close to 0 means that the new information has a greater impact. Then calculate the total loss of the overlapping area:

[0060]

[0061] Where L pix (i) are the pixel contrast losses obtained by the two feature representations, and |S| represents the size of the set S.

[0062] Step 3: After assigning the encoder weights in the pre-trained contrastive learning model to the encoder of the segmentation model, the model is fine-tuned using labeled chest CT image samples. The fine-tuned segmentation model is used to perform multi-view prediction on the remaining unlabeled chest CT image samples in the dataset, and high-uncertainty samples and high-confidence samples are distinguished based on the prediction results. The more views with the same prediction results, the more consistent the opinions of the multiple views are, and the higher the confidence of the samples.

[0063] Among them, the segmentation model adopts any segmentation model containing an encoder structure, such as AttUNet, UNet, FracNet and nnUNet. In a specific implementation case, the segmentation model adopts AttUNet to assign the encoder weights pre-trained in contrastive learning to the encoder of AttUNet, and then fine-tune it with a small amount of labeled samples to further improve the segmentation accuracy of the segmentation model, and use the fine-tuned model to predict the remaining unlabeled samples in the data set to obtain new training data.

[0064] The present invention uses multi-view prediction results to assist in screening, and calculates the prediction score of each pixel in each sample, wherein the prediction score of each pixel is the sum of the corresponding values ​​of the prediction category of the pixel in each view; the prediction score range is [a, b], wherein a and b are the corresponding values ​​of the smaller and larger values ​​in the two categories of prediction categories, rib pixels and non-rib pixels, respectively; and then calculates the uncertainty score of each sample, wherein the uncertainty score of each sample is the ratio of the product of the sum of the number of pixels of each prediction score within the score range of each sample and the weight of the corresponding prediction score to the sum of the number of pixels of each prediction score within the score range of each sample, wherein the closer the prediction score is to the middle value of the score range, the greater the weight of the prediction score. In a specific embodiment case, during the prediction process, a sliding window is used to slide each CT to obtain multiple views, assuming that the sliding window size is n×n×n (for example, n=64), the step size of the sliding window is n / k (for example, k=2), and each sliding window is regarded as a view, then each pixel point will be k3 In order to predict the CT boundary, k 3 Secondly, this embodiment intentionally adds blank pixels with a width of n / k around CT. Figure 3 The figure shows a schematic diagram of multi-view prediction when n = 64 and k = 2. For example, if a pixel in a view is regarded as a rib pixel, then the prediction score of the pixel is increased by 1. As the sliding window predicts the entire CT, each pixel will get a prediction score in the range of [0, k 3 ]. The pixel score is close to the middle of the interval, indicating that k 3 The prediction results of the two views for this pixel point have a large difference, and the closer to the two sides, the more k 3 The opinions of the views tend to be consistent. Based on this, the uncertainty score of each sample is as follows:

[0065]

[0066] Among them, score(i) represents the number of pixels with score i, and weight(i) represents the weight assigned to the pixel with score i. Because the pixels with scores at the two ends of the interval (scores 0 and 8) are the most confident of the model, and the pixels in the middle of the interval (score 4) are the least confident of the model, it is necessary to assign higher weights to the pixels with scores close to the middle of the interval, and assign lower weights to the pixels with scores close to the two ends of the interval. In this embodiment, the weights from score 4 to the two ends are set to 0.4, 0.3, 0.2, 0.1 and 0 respectively.

[0067] The higher the uncertainty score, the greater the probability of being a high uncertainty sample. Finally, the high uncertainty samples and high confidence samples are distinguished according to the uncertainty score of each sample. A threshold can be used for distinction, or several samples with the largest uncertainty scores can be directly selected as high uncertainty samples and several samples with the smallest uncertainty scores can be selected as high confidence samples according to the uncertainty score. In this embodiment, the top 20 samples with the highest scores are selected in each round as high uncertainty samples, and those with scores less than 0.007 are high confidence samples.

[0068] Step 4: directly add the prediction results of high-confidence samples as pseudo labels to the training pool and / or obtain the pixel entropy distribution of high-uncertainty samples, send the high-entropy area to experts for annotation, and slide the window to crop the annotated area and add it to the training pool; further obtain the pixel entropy distribution of high-uncertainty samples, send the high-entropy area to experts for annotation, slide the window to crop it and add it to the training pool;

[0069] In the above technical solution, the prediction results of high-confidence samples are added to the training pool as pseudo labels and new training data. This method belongs to self-training. The prediction results of high-confidence samples are post-processed to remove some noise points by removing smaller connected domains. In this embodiment, all connected domains with a size less than 5000 are discarded. The processed prediction results are regarded as pseudo labels for training the model.

[0070] Furthermore, in the above technical solution, we can further find out the areas in the sample that are more valuable for annotation. We first use pixel entropy to visualize the uncertainty distribution of the sample. The probability entropy of each pixel is expressed as:

[0071]

[0072] Where C is the total number of categories, p i represents the predicted probability of the i-th category.

[0073] The entropy of all pixels in the sample is standardized, and a threshold is set to divide high and low uncertainty areas. In this embodiment, the area with entropy greater than the threshold of 0.6 is regarded as a high uncertainty area, and the area with entropy less than 0.6 is regarded as a low uncertainty area. The high entropy area is demarcated as a bounding box and sent to the expert for annotation. Finally, the annotated area is clipped by sliding window and added to the training pool as new training data to further expand the training sample size. Figure 4 It is a three-dimensional (A, B) and two-dimensional (C, D) schematic diagram locally labeled according to pixel entropy provided by a specific embodiment of the present invention.

[0074] Step 5: Train the segmentation model based on the data in the training pool to obtain the constructed segmentation model.

[0075] In a specific implementation manner, in order to further improve the accuracy of the segmentation model, the above steps 1 to 4 may be repeated until the model converges.

[0076] Corresponding to the aforementioned embodiment of a method for constructing a rib segmentation model by active contrast learning based on local annotation in a CT image, the present invention also provides an embodiment of a system for constructing a rib segmentation model by active contrast learning based on local annotation in a CT image.

[0077] An embodiment of the present invention provides a system for constructing a rib segmentation model based on active contrast learning in a CT image based on local annotation, comprising one or more processors for implementing a method for constructing a rib segmentation model based on active contrast learning in a CT image based on local annotation in the above embodiment.

[0078] An embodiment of a system for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation in the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer.

[0079] The device embodiment can be implemented by software, hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capability in which it is located to read the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, it includes a processor, memory, network interface, and non-volatile memory. In addition, any device with data processing capability in which the device in the embodiment is located can also include other hardware according to the actual function of the device with data processing capability, which will not be described in detail.

[0080] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0081] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0082] In another specific embodiment, the system framework of the present invention is as follows Figure 2 As shown. It contains four modules: contrast learning module, active learning module, local labeling module and training module. 1) The contrast learning module is used to obtain bone pixel blocks and bone pixel block pairs composed of their adjacent pixel blocks based on some unlabeled chest CT image samples, and pre-train the contrast learning model containing the encoder using pixel contrast learning based on the bone pixel block pairs; a large number of pixel block pairs are intercepted from the unlabeled CT images, and a large number of pixel points are compared to achieve the purpose of distinguishing rib pixels from non-rib pixels, so that the encoder of the segmentation model has the ability to extract more detailed features. 2) The active learning module relies on uncertainty to select samples, such as Figure 3The multi-view prediction method shown in Figure 1 predicts the unlabeled samples in the dataset to obtain the uncertainty score of each sample. The prediction results of the samples with lower scores are used as pseudo labels, and the samples with higher scores are sent to the local annotation module. 3) The local annotation module uses pixel entropy to explore the uncertainty distribution of the sample. After normalizing the pixel entropy in CT and setting the threshold, the distribution of the uncertainty area is as follows: Figure 4 As shown in the figure, the red area represents a high uncertainty area, which is more valuable to be labeled than other areas, so only this area is sent to the expert for labeling. 4) The training module adds the pseudo labels output by the sample selection module and the local labeled samples output by the local labeling module to the training pool to train the model, thereby achieving the purpose of training the segmentation model with less labeling.

[0083] The following is a detailed description of the technical effects of the present invention in combination with performance testing and experimental analysis. In order to verify the effect of the present invention in reducing the labeling cost, this embodiment selects the segmentation model AttUNet to use the local labeled samples selected by the present invention in the data set RibSegV2 to compare with the full labeled samples. The RibSegV2 data set contains a total of 660 samples (training: 420, verification: 80, test: 160). We randomly select 200 samples in the training set as unlabeled samples for comparative learning, and randomly select 20 samples as the initial training set for active learning. The remaining 200 samples in the training set are used as unlabeled samples for the model to select high confidence samples and high uncertainty samples. The results are shown in the following table:

[0084]

[0085] As can be seen from the above table, the present invention can achieve an effect comparable to that of full-amount labeled data using only a small amount of labeled data, and greatly shortens the training time.

[0086] In order to further prove the effectiveness of the present invention, an ablation experiment was also conducted. The present invention combines contrastive learning (step 2), active learning (relying on uncertainty to select samples, such as Figure 3 The multi-view prediction method shown in FIG. predicts the unlabeled samples in the dataset to obtain the uncertainty score of each sample, and uses the prediction results of samples with lower scores as pseudo labels to distinguish high uncertainty samples from high confidence samples according to the prediction results), local annotation (obtaining the pixel entropy distribution of high uncertainty samples, sending the high entropy area to experts for annotation, and sliding window cropping the labeled area to add to the training pool), and self-training (the prediction results of high confidence samples are used as pseudo labels as new training data and added to the training pool). The four parts are added in sequence, and the obtained results are compared with the full amount of labeled data. The results are shown in FIG. Figure 5 As shown, Figure 5The IOU results of the model test corresponding to adding 20 samples in each round are shown. The effect of traditional deep learning using the full amount of labeled data (420 samples) is equivalent to the effect of contrastive learning used in the present invention. Contrastive learning + active learning further reduces the demand for labeled samples, and a good effect is achieved using only 120 labeled data. Contrastive learning + active learning + local labeling reduces the area that needs to be labeled for a single sample, while also using only 120 labeled data, and maintains the model performance at the same level. Contrastive learning + active learning + local labeling + self-training improves the level of model training, allowing the model to maintain a high level in any round.

[0087] In summary, the above experiments prove the effectiveness of the present invention. The present invention can significantly reduce the number of CT image annotations and still enable the model to maintain the same segmentation performance as the full amount of annotated data.

[0088] Corresponding to the aforementioned embodiment of a method for constructing a rib segmentation model by active contrast learning based on local annotation in CT images, the present invention also provides an embodiment of a method for active contrast learning based on local annotation in CT images.

[0089] A rib segmentation method based on active contrast learning of local annotation in CT images, specifically:

[0090] Obtain a chest CT image sample to be segmented, input the chest CT image sample to be segmented into the segmentation model constructed by the above method, and output the segmentation result.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation, characterized in that: The following steps are involved: Step 1: Obtain a data set; the data set includes unlabeled chest CT image samples and labeled chest CT image samples; Step 2: Based on some unlabeled chest CT image samples, bone pixel blocks and bone pixel block pairs consisting of their adjacent pixel blocks are obtained, and pixel contrast learning is used to pre-train a contrast learning model including an encoder based on the bone pixel block pairs; Step 3: After assigning the encoder weights in the pre-trained contrastive learning model to the encoder of the segmentation model, the labeled chest CT image samples are used for training and fine-tuning. The fine-tuned segmentation model is used to perform multi-view prediction on the remaining unlabeled chest CT image samples in the dataset, and high-uncertainty samples and high-confidence samples are distinguished according to the prediction results. The more views with the same prediction results, the more consistent the opinions of the multiple views are, and the higher the confidence of the samples is. Step 4: Use the prediction results of high-confidence samples as pseudo-labels, directly slide and crop them into the training pool, and / or obtain the pixel entropy distribution of high-uncertainty samples, send the high-entropy areas to experts for annotation, and slide and crop the labeled areas into the training pool; Step 5: Train the segmentation model based on the data in the training pool to obtain the constructed segmentation model.

2. The method according to claim 1, characterized in that There is a partial overlapping area between two pixel blocks in the bone pixel block pair.

3. The method according to claim 2, characterized in that The method to obtain the bone pixel block is as follows: According to the first pixel threshold, non-bone pixels on the unlabeled chest CT image samples are screened out, the approximate position of the ribs is determined according to the connected area and the remaining area is removed, and the rib area is cropped with a sliding window to obtain bone pixel blocks; according to the second pixel threshold, bone pixel blocks with larger bone pixels are further obtained; According to the obtained bone pixel block position, another bone pixel block having a partially overlapping area with the bone pixel block is obtained by offsetting and cropping the original unlabeled chest CT image sample to form a bone pixel block pair.

4. The method according to claim 2, characterized in that: The pixel contrast learning based on the bone pixel block is used to pre-train the contrast learning model including the encoder, as follows: The contrastive learning model includes an encoder, a momentum encoder and two projection heads respectively connected to the output ends of the encoder and the momentum encoder; each bone pixel block pair is respectively sent to the encoder and the momentum encoder, and the encoder feature representation and the momentum encoder feature representation are respectively obtained from the outputs of the two corresponding projection heads; The encoder feature representation and the momentum encoder feature representation are upsampled and mapped back to the original 3D space, and the Euclidean distance between the two pixels in the corresponding pixel blocks in the bone pixel block pair is calculated based on the encoder feature representation and the momentum encoder feature representation mapped back to the original 3D space. When the distance is greater than a threshold λ, the pixels are considered to be positive pixels of each other, otherwise they are negative pixels of each other; Construct the total loss of the overlapping area of ​​each bone pixel block and back-propagate it to the encoder. The momentum encoder is updated by the exponential moving average until the requirement is met and pre-training is completed. The total loss of the overlapping area of ​​each bone pixel block is expressed as follows: Where S and S' represent the pixel sets of the overlapping areas corresponding to each bone pixel block in a pair of bone pixel blocks, S = S'; L pix (i) is the pixel contrast loss for pixel i in each overlapping region, X + , X - are the sets of pixels that are positive and negative pixels with respect to pixel i, respectively. x and x′ correspond to the feature representations output by the two projection heads, respectively. τ is a hyperparameter.

5. The method according to claim 1, characterized in that The segmentation model adopts AttUNet, UNet, FracNet or nnUNet.

6. The method according to claim 1, characterized in that The multi-view is specifically obtained by the following method: Add blank pixels with a width of n / k around each remaining unlabeled chest CT image sample and use a sliding window with a sliding window size of n×n×n and a sliding window step size of n / k to slide the CT image sample, and regard each sliding window as a view; then each pixel point will be k 3 View prediction.

7. The method according to claim 6, characterized in that The method of distinguishing high uncertainty samples from high confidence samples according to the prediction results is as follows: Calculate the prediction score of each pixel in each sample, where the prediction score of each pixel is the sum of the values ​​assigned to the predicted category of the pixel in each view; the prediction score range is [ak 3 ,bk 3 ], where a and b are the values ​​corresponding to the smaller and larger values ​​in the two categories of predicted categories, rib pixels and non-rib pixels, respectively; Calculate the uncertainty score of each sample, where the uncertainty score of each sample is the ratio of the product of the number of pixels of each predicted score within the score range of each sample and the weight of the corresponding predicted score to the sum of the number of pixels of each predicted score within the score range of each sample, wherein the closer the predicted score is to the middle value of the score range, the greater the weight of the predicted score; High uncertainty samples and high confidence samples are distinguished based on the uncertainty score of each sample. The higher the uncertainty score, the greater the probability of being a high uncertainty sample.

8. The method according to claim 1, characterized in that In the pixel entropy distribution of the high uncertainty sample, the probability entropy of each pixel is expressed as: Where C is the total number of categories, p i represents the predicted probability of the i-th category.

9. A rib segmentation method based on active contrast learning of local annotation in CT images, characterized in that: Specifically: Obtain a chest CT image sample to be segmented, input the chest CT image sample to be segmented into a segmentation model constructed by the method according to any one of claims 1 to 8, and output the segmentation result.

10. A system for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for constructing a rib segmentation model based on active contrast learning in CT images based on local annotation is implemented as described in any one of claims 1-8.

Images