CT image region segmentation method based on sequence position optimization and conditional U-Net

Abstract

The invention belongs to the field of medical image detection and image processing, and discloses a CT image region segmentation method based on sequence position optimization and conditional U-Net, which adopts a lung CT (computed tomography) image as input, analyzes the relative position of each tomographic image in a human body on the basis of a relative position prediction model and a CRF model, and obtains a CT image region. On the basis, a U-Net network is constructed to realize accurate position detection; the method specifically comprises the steps of designing a cross-sectional image preprocessing and enhancing method; designing a relative position prediction model based on a double-branch feature extraction network; designing a sequence position optimization algorithm based on the CRF model; and designing a pixel-level density anomaly region detection algorithm based on U-Net.

Description

Technical Field

[0001] This invention relates to the field of medical image detection and image processing, and to a CT image region segmentation method based on sequence position optimization and conditional U-Net. Background Technology

[0002] Artificial intelligence technology has made great strides in the field of image processing, making it possible to use machines to replace doctors in detecting areas of abnormal density and thus assist in determining location. However, current lung diagnosis still mainly relies on manual methods. The training cost of professional radiologists is extremely high, and there are significant differences in individual skill levels. Hospitals in remote areas often experience more misdiagnosis.

[0003] For certain issues, researchers have developed diagnostic algorithms based on machine learning technology. However, these algorithms have several shortcomings: First, they are not universally applicable to different parts of the human body and require specialized algorithms for the lungs. Second, current detection algorithms often only target a single tomographic image and lack comprehensive analysis of before and after images in a complete computed tomographic scan, resulting in a high false alarm rate.

[0004] Therefore, there is an urgent need for a highly accurate detection method for areas of abnormal lung density, which can perform preliminary screening on CT images obtained from hospitals and provide prediction results of areas of abnormal density to assist doctors in diagnosis and treatment. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for segmenting abnormal density regions in lung CT images based on relative position prediction and conditional U-Net, which addresses the shortcomings of the existing technology.

[0006] To address the aforementioned technical problems, this invention discloses a CT image region segmentation method based on sequence position optimization and conditional U-Net, comprising the following steps:

[0007] Step 1: Obtain the set of lung CT tomographic image sequences to be processed, and perform data preprocessing and enhancement on each tomographic image contained in each set of tomographic image sequences. The tomographic images include lung window tomographic images and mediastinal window tomographic images.

[0008] Step 2: For each preprocessed tomographic image, construct a 2.5D image patch consisting of it and the adjacent tomographic images in the same sequence, and input them into the relative position prediction model based on a dual-branch feature extraction network to extract lung window features and mediastinal window features. After feature fusion, the initial probability distribution of the relative position of the tomographic image is obtained through a fully connected layer and a Softmax function.

[0009] Step 3: Using the entire set of lung CT tomographic image sequences as the processing object, construct a one-dimensional chained conditional random field (CRF) model, optimize the relative position of each tomographic image, obtain the globally optimal position label of each tomographic image, and encode the position label into a One-Hot vector;

[0010] Step 4: Input the One-Hot vector into the lightweight modulation parameter generation network to generate multiple sets of scaling and offset parameters for feature modulation; input the preprocessed tomographic image into the conditional U-Net segmentation network, and in the decoder part of the U-Net network, use the scaling and offset parameters to perform affine transformation on the feature map to achieve network parameter conditionalization based on relative position, and finally output a pixel-level density anomaly region probability map.

[0011] Step 5: Threshold the probability map of the density anomaly region and optimize the spatiotemporal consistency by combining the processing results of adjacent tomographic images to generate the final binary label mask of the density anomaly region.

[0012] In step 1, the data preprocessing and enhancement specifically include:

[0013] During the model training phase, enhancement operations are performed on the input image blocks. These enhancement operations include random translation, scaling, rotation, and brightness / contrast adjustment. Specifically, for all tomographic images belonging to the same set of tomographic image sequences (a single lung CT scan corresponds to a set of tomographic image sequences), all enhancement parameters are kept consistent to maintain spatial consistency within the sequence.

[0014] Step 2 specifically includes:

[0015] Step 2-1: 2.5D image patch construction: For each tomographic image in the same group, it is stitched together with the previous and next tomographic images in the channel dimension to form a three-channel input image patch; for the boundary images at the beginning and end, the missing adjacent images are processed by copying and filling.

[0016] Step 2-2: Dual-branch feature extraction: Lung window branch: ResNet-34 is used as the backbone network to capture local texture details of density abnormal areas in lung window tomographic images; Mediastinal window branch: An improved ResNet-34 network is used as the backbone network, and at least one standard convolutional layer in its rear part is replaced with a hollow convolutional layer with a mixed dilation rate to expand the receptive field of the network and capture global context and anatomical structure information related to density abnormal areas in mediastinal window tomographic images;

[0017] Steps 2-3: Feature Fusion and Initial Classification: The feature maps output from the lung window branch and the mediastinal window branch are concatenated along the channel dimension. A 1×1 convolutional layer is used for feature integration and dimensionality reduction. Then, a global average pooling layer and a fully connected layer are passed through the softmax activation function to obtain the initial probability distribution of the tomographic image belonging to K predefined relative position categories, where K is an integer greater than 1.

[0018] The true labels for the relative position categories mentioned in steps 2-3 are obtained in the following way:

[0019] For each complete lung CT image sequence, the position of the lung apex near the neck in the tomographic image is used as the reference, and the normalization is made to 0; the position of the lung base apex near the abdomen in the tomographic image is used as the reference, and the normalization is made to 1.

[0020] The continuous interval from 0 to 1 is divided into K continuous sub-intervals. Based on the specific coordinate values ​​of each tomographic image on the normalized longitudinal coordinate axis, the sub-intervals are classified and labeled into the corresponding discrete location categories.

[0021] The purpose of the above classification is to divide CT images according to their location on the body. Lung CT images are transverse cross-sections along the body. Cross-sections near the head and those near the abdomen are quite different. Therefore, the cross-sections are first classified according to their location, and a training model is used to learn the rules of this classification. Then, for new CT images, the pre-trained classification model is used to classify them into a certain category before subsequent image segmentation is performed under that category, thus obtaining better segmentation results.

[0022] In step 3, the optimization and encoding of the relative position of the tomographic image specifically includes:

[0023] Step 3-1: Conditional Random Field Modeling: Model the CT tomographic image sequence containing N tomographic images as a one-dimensional chain graph structure. Each node For a tomographic image, each edge Connect adjacent tomographic images; the energy function of the CRF model is defined as:

[0024]

[0025] in, Tomographic images The predicted location label, Y is the label sequence of the entire CT tomographic image sequence, and the univariate potential function. The tomographic image obtained in step 2 The initial probability of belonging to category k;

[0026] Binary potential function ,in For tag compatibility functions, As the smoothing coefficient, when A larger penalty value is applied when the situation is anatomically unreasonable, and a smaller value or zero is applied otherwise. This is to penalize class jumps at non-adjacent positions that are anatomically unreasonable, and to encourage smooth transitions in sequence position labels.

[0027] Step 3-2: Relative position label solution: The Viterbi algorithm is used to minimize the energy function of the above CRF model to obtain the globally optimal relative position label sequence for the entire CT tomographic image sequence.

[0028] ;

[0029] Step 3-3: Position Encoding: The optimal discrete position label from the obtained relative position label sequence... Transform into a K-dimensional One-Hot vector .

[0030] The lightweight modulation parameter generation network described in step 4 is a lightweight multilayer perceptron (MLP). The MLP contains at least one hidden layer and uses the ReLU activation function. Its output layer dimension is... Where C is the number of channels in the feature map of the corresponding stage of the U-Net decoder; the output vector is split into two parts, which are used as scaling parameters respectively. and offset parameters ;

[0031] The conditional U-Net segmentation network structure includes:

[0032] The encoder consists of four downsampling modules. Each module contains two 3×3 convolutional layers, a batch normalization (BN) layer, and a ReLU activation function, and is finally connected to a 2×2 max pooling layer for downsampling.

[0033] Decoder: Consists of a four-level upsampling module, a BN layer, and a ReLU activation function.

[0034] Step 4 specifically includes:

[0035] Step 4-1: Modulation Parameter Generation: The lightweight modulation parameter generation network generates the input K-dimensional One-Hot vector. The mapping is to modulation parameters, namely scaling and offset parameters;

[0036] Step 4-2: Input the preprocessed tomographic image into the encoder. The encoder performs feature extraction in four downsampling stages in sequence. Each downsampling stage includes two 3×3 convolutional layers, a batch normalization (BN) layer, a ReLU activation function, and a 2×2 max pooling downsampling operation, and finally outputs a feature map.

[0037] Step 4-3: Each upsampling module of the decoder first performs 2×2 deconvolution or bilinear interpolation upsampling on the received feature map, then concatenates the upsampling result with the feature map passed from the corresponding layer of the encoder through skip connections, and finally outputs the feature map after a 3×3 convolution layer. ;

[0038] Step 4-4: Feature Modulation: In each upsampling stage of the decoder, the output feature map of the 3×3 convolutional layer is modulated based on the modulation parameters. Apply a position-based affine transformation, with the following formula:

[0039]

[0040] in, This indicates channel-by-channel multiplication. and These correspond to the current tomographic image location and the modulation parameters of the k-th level of the decoder; feature map Will replace The calculation of the next upsampling module of the decoder is involved, and steps 4-3 and 4-4 will be repeated four times;

[0041] Steps 4-5: Network Output: The output of the last layer of the decoder is passed through a 1×1 convolutional layer to map the number of channels to 1, and then through the Sigmoid activation function to generate a pixel-level density anomaly region probability map with the same size as the input image.

[0042] During the training phase, a combined loss function is used for end-to-end joint optimization. For Dice's loss With weighted cross-entropy loss The weighted sum, i.e.:

[0043]

[0044] in, and is the weighting coefficient, with a value range of [0,1], used to balance the importance of the two loss functions;

[0045] The formulas for calculating the two loss functions are as follows:

[0046]

[0047] Where M is the probability map of density anomaly regions obtained in steps 4-5, and G is the binary mask of the actual annotation (i.e., as shown in the image). Figure 6The third column shows the ground truth, where x and y represent the x and y coordinates of the pixels in the probability map of density anomalies, respectively.

[0048]

[0049] Among them, weight Based on the dynamic calculation of the ratio of positive to negative pixels in the dataset, this method addresses the class imbalance problem where pixels in areas of abnormal density are far fewer than those in normal tissue, and assigns higher weights to the pixels.

[0050] The spatiotemporal consistency optimization performed in step 5, which combines the processing results of adjacent tomographic images, specifically involves:

[0051] For each tomographic image in the same set of tomographic images, iterate through each pixel position; if more than Q tomographic images in the current tomographic image and its P preceding and following tomographic images are initially identified as density anomalous regions at that position, then the pixel at that position in the current tomographic image is finally determined to be a density anomalous pixel; otherwise, it is determined to be a non-density anomalous pixel; where P and Q are preset positive integers, and Q ≤ P.

[0052] Beneficial effects:

[0053] 1. By connecting the contextual relationships between the preceding and following tomographic images in each tomographic image sequence, a more complete feature representation can be obtained, thereby achieving higher detection accuracy based on this.

[0054] 2. A relative position estimation algorithm for tomographic images based on the Conditional Random Field (CRF) model was adopted, which estimated relatively accurate relative position information for each tomographic image, thereby assisting the algorithm to obtain more accurate detection results.

[0055] 3. In the post-processing of segmentation results, an innovative decision fusion strategy based on sequence consistency was introduced, which effectively suppressed the noise and false positives commonly found in the image segmentation results of single tomographic images, and improved the smoothness and clinical reliability of the three-dimensional segmentation results. Attached Figure Description

[0056] Figure 1 This is the flowchart for this method.

[0057] Figure 2 Flowchart for labeling the relative positions of CT images.

[0058] Figure 3 This is a flowchart of an image relative position prediction algorithm based on a dual-branch ResNet.

[0059] Figure 4 This is a flowchart of a relative position optimization algorithm based on CRF.

[0060] Figure 5 A schematic diagram of a conditional U-Net segmentation network.

[0061] Figure 6 Examples of database annotations and prediction results. Detailed Implementation

[0062] This method uses image annotation to mark the relative position information of a large number of images in the database, and uses image classification combined with a conditional random field model to estimate the relative position of each tomographic image in the CT images to be detected.

[0063] The image segmentation model structure is designed to combine the relative position estimation results with the information of the tomographic image and the information of the tomographic images before and after it, and extract image features for the segmentation of the part image. A large number of CT images with density anomalous regions are annotated at the pixel level to train the image segmentation model, thereby obtaining accurate image segmentation results, namely pixel-level density anomalous region segmentation results.

[0064] During the inference phase, prior knowledge that density anomaly regions should have spatial continuity in CT images is utilized to optimize the initial segmentation results of a single tomographic image at the sequence context level, further improving the robustness and accuracy of the results.

[0065] like Figure 1 As shown, the implementation of this invention can be divided into two stages: model training (right side) and clinical reasoning (left side).

[0066] ① Training phase

[0067] During the training process, lung CT images from hundreds of lung cancer patients were first collected as a set of lung CT tomographic image sequences. Each patient's CT scan corresponded to a set of CT tomographic image sequences (i.e., commonly referred to as CT images), each containing hundreds of tomographic images. Each tomographic image included lung window and mediastinal window images, numbered sequentially from the upper to the lower part of the body. Based on... Figure 2 The method shown labels each tomographic image with its relative location category (see S2). Subsequently, data enhancement is performed on each set of CT images, including random translation within ±10%, random scaling within 0.8 to 1.2 times, random rotation within ±5°, and brightness / contrast adjustment within ±10%. Crucially, to maintain spatial consistency within the CT images, the same enhancement parameters are applied to all tomographic images within the same set of CT images. Using this enhancement strategy, up to 20 sets of enhanced data can be generated for each set of original CT image images, with their location category labels remaining unchanged.

[0068] At the same time, professional physicians annotate the areas of density abnormalities at the pixel level in each tomographic image (such as... Figure 6 When enhancing an image, the corresponding region labels also undergo the exact same geometric transformation to ensure data consistency.

[0069] Using the augmented data, two core models are trained in parallel:

[0070] A. Relative position prediction model based on dual-branch feature extraction network: Using the current tomographic image and two tomographic images before and after it (a total of 6 images) as input, predict its relative position category (see S3 for details).

[0071] B. Conditional U-Net segmentation network: Taking the dual-window image of a single tomographic image and its (after subsequent CRF optimization) location and category information as input, it outputs a pixel-level density anomaly region probability map of the tomographic image (see S5 for details).

[0072] ② Reasoning stage

[0073] After the model training is complete, new CT images are processed according to the following procedure:

[0074] First, using the trained relative position prediction model, predict the relative position category for each tomographic image in the CT image (see flowchart). Figure 3 ).

[0075] Next, a one-dimensional chain conditional random field (CRF) model is constructed to globally optimize the above relative position prediction results, resulting in smooth and coherent relative position labels (see process). Figure 4 ).

[0076] Then, the dual-window image of each tomographic image and its optimized location label (encoded as a One-Hot vector) are input into the trained conditional U-Net segmentation network to obtain a preliminary pixel-level density anomaly region probability map (see process). Figure 5 ).

[0077] Finally, spatiotemporal consistency post-processing is performed: for each tomographic image, each pixel is traversed. If a pixel is identified as a density anomalous pixel in three or more of the prediction results of the current tomographic image and the two tomographic images before and after it (a total of five consecutive tomographic images), then the pixel is finally confirmed as a density anomalous pixel. This step can effectively filter out isolated false positives caused by noise in a single tomographic image and enhance the continuity of real lesions in the sequence, ultimately generating a more robust binary density anomalous region annotation mask.

[0078] S2. Relative Position Labeling Method

[0079] like Figure 2 As shown, we find the image corresponding to the upper edge of the lung lobe in each group of CT images, which is numbered N1, and then find the image corresponding to the lower edge of the lung lobe, which is numbered N2. We calculate N2-N1+1 as the number of valid images in the sequence. Then we divide this number by five to get the number of images in each relative position category in this embodiment, and then classify the corresponding images into these five categories. For example, when N1=10 and N2=359, the number of valid images is 350, and the number of images in each category is 70. Therefore, numbers 10 to 79 are category a, 80 to 149 are category b, and so on, with 280 to 359 being category e.

[0080] S3. Implementation of the Relative Position Prediction Model

[0081] like Figure 3 As shown, the relative position prediction model of this invention adopts a dual-branch feature extraction and fusion architecture. For each tomographic image in the CT image sequence... Please follow these steps:

[0082] 2.5D Input Construction: Constructing Tomographic Images Compared with the previous tomographic image The next tomographic image The images are stitched together along the channel dimension to form three-channel input image blocks for the lung window and mediastinal window, respectively. For edge-mounted tomographic images, a copy-filling strategy is used to fill in missing adjacent tomographic images.

[0083] Dual-branch feature extraction:

[0084] Lung window branch: Using a ResNet-34 pre-trained on ImageNet as the backbone network, its classification head is removed. Input lung window image patch, network output size is... High-level semantic feature map .

[0085] The septate window branch uses an improved ResNet-34 network, replacing the standard convolutions in its back-end layers 3 and 4 with dilated convolutions (with dilation rates set to 2 and 4). The input is a septate window image patch, and the network output size is the same. Feature map .

[0086] ③ Feature fusion and dimensionality reduction: This involves fusing feature maps... and By splicing along the channel dimension, a size of [size missing] is obtained. The fused feature map is then processed by a 1×1 convolutional layer to integrate and reduce its dimensionality, resulting in an output size of [size missing]. Feature map after fusion .

[0087] ④ Global feature aggregation and classification: This involves analyzing the feature maps. Global average pooling is applied to obtain a 512-dimensional feature vector. This vector is then input into a fully connected layer with the number of neurons equal to the preset number of relative position categories K (K=5 in this embodiment). Finally, the tomographic image is output through the Softmax activation function. Probability distribution of belonging to K position categories .

[0088] By performing the above process independently on each tomographic image in the sequence, the initial relative position category probability distribution of all tomographic images in the entire sequence can be obtained. This distribution will serve as the univariate potential function input to the subsequent Conditional Random Field (CRF) model.

[0089] S4. Implementation process of relative position optimization algorithm

[0090] like Figure 4 As shown, the relative position optimization algorithm based on Conditional Random Fields (CRF) mainly includes three parts: calculation of the univariate potential function based on the probability distribution of relative position categories, calculation of the binary potential function based on sequence smoothing constraints, and optimization of the relative position label sequence based on the Viterbi algorithm. The details are as follows:

[0091] ① Univariate potential function The tomographic image obtained in step S200 The initial probability of belonging to class k. Assuming a tomographic image has probabilities of belonging to five classes in relative position prediction as [0.01, 0.25, 0.65, 0.08, 0.01], if class 3 is ultimately chosen as the optimized result, its univariate potential function in that tomographic image is -log(0.65) = 0.43, while if class 2 is chosen, it is -log(0.25) = 1.39. The sum of the univariate potential functions of all tomographic images in the entire CT image sequence constitutes the univariate potential function of the entire CT image sequence, which is one part that the Viterbi algorithm needs to optimize subsequently.

[0092] ② Bivariate potential function ,in For tag compatibility functions, in this embodiment, when The value is set to 10 if the label difference is not anatomically significant and otherwise to 1. This penalizes anatomically unreasonable class jumps between non-adjacent locations and encourages smooth transitions in sequence position labels. σ is a smoothing coefficient that controls the flexibility of label changes; in this embodiment, σ = 0.5. This value results in a lower penalty for smaller label differences. The sum of the binary potential functions between consecutive tomographic images is the binary potential function of the entire sequence.

[0093] ③ Global optimal solution: The Viterbi algorithm is used to minimize the energy function.

[0094]

[0095] Solving for the globally optimal relative position label sequence Y of the entire CT tomographic image sequence. ∗ The algorithm dynamically calculates the minimum cumulative energy for each tomographic image in each state, and finally backtracks to obtain the optimal path.

[0096] For example, the initial prediction (highest probability class) of a short sequence [1,2,3,4,5] is [2, 2, 4, 2,3]. It can be seen that the prediction "4" in the tomographic image 3 is an anomalous jump.

[0097] When optimizing the Viterbi algorithm:

[0098] If tomographic image 3 is kept in category 4, then the absolute difference between its label and those of its neighboring tomographic images 2 (category 2) and 4 (category 2) is 2. In this case, μ(2,4) = μ(4,2) = 10.0, and the exponential term is very small, so ψp contributes a large penalty energy. If tomographic image 3 is changed to category 2 or 3, although its univariate potential energy ψu may increase due to the low initial probability, the absolute difference between its label and those of its neighboring labels is ≤1, μ = 1.0, and the exponential penalty is moderate, resulting in a total energy far lower than the scheme of keeping it in category 4.

[0099] Therefore, the algorithm will definitely correct the label of tomographic image 3 from "4" to "2" or "3", thereby eliminating unreasonable jumps in the sequence and obtaining a smooth and coherent relative position label with a high probability of [2, 2, 2, 2, 3] and a low probability of [2, 2, 3, 2, 3]. For a complete CT image sequence containing 300 tomographic images, this process can systematically correct all similar local anomaly predictions.

[0100] Finally, the optimal relative position label Y output by the Viterbi algorithm is... ∗ Each discrete label y in i ∗ Convert it into a 5-dimensional One-Hot vector for use by the subsequent segmentation network. For example, if the label is 0, it is converted to [1, 0, 0, 0, 0], and if the label is 2, it is converted to [0, 0, 1, 0, 0].

[0101] S5. Image Segmentation Network Implementation Process

[0102] like Figure 5 As shown, the conditional U-Net segmentation network of the present invention employs a position condition modulation mechanism, namely relative position optimization, and the specific implementation is as follows:

[0103] ① Input preparation.

[0104] For each tomographic image in the CT sequence, its lung window and mediastinal window tomographic images are stitched together along the channel dimension to form a dual-channel input image. Simultaneously, the K-dimensional One-Hot vector corresponding to the optimized location label of the tomographic image is obtained from the previous step. .

[0105] ② Modulation parameter generation.

[0106] Design a lightweight multilayer perceptron (MLP) as a modulation parameter generator. The MLP consists of an input layer, a hidden layer, and an output layer.

[0107] Input layer: Receives K-dimensional One-Hot vectors In this embodiment, K=5;

[0108] Hidden layer: Contains 128 neurons, using the ReLU activation function;

[0109] Output layer: Contains 2×C neurons, where C is the number of feature map channels in the corresponding stage of the U-Net decoder, and no activation function is set (linear output);

[0110] The 2×C-dimensional vector output by the MLP is split into two parts: the first C dimensions are used as scaling parameters. The C dimension is then used as the offset parameter. These two parameter vectors will be used to modulate the feature map of the U-Net decoder channel by channel.

[0111] ③Conditional U-Net forward propagation.

[0112] Dual-channel input image Input the improved U-Net network:

[0113] The encoder path consists of four downsampling stages. Each stage contains two 3×3 convolutional layers, followed by batch normalization (BN) and ReLU activation, and finally downsampling via 2×2 max pooling. The number of channels in the feature maps output by the encoder are 64, 128, 256, and 512, respectively.

[0114] The decoder path consists of four upsampling stages, symmetrical to the encoder. Each stage first performs upsampling through a 2×2 deconvolution, then concatenates the upsampling result with the feature map passed from the corresponding layer of the encoder through skip connections, followed by a 3×3 convolutional layer, BN, and ReLU activation.

[0115] Feature modulation: After each 3×3 convolutional layer in the upsampling stage, the output feature map is modulated. Apply a position-based affine transformation:

[0116]

[0117] Where ⊙ represents channel-by-channel multiplication. and These correspond to the current tomographic image location and the modulation parameters at the k-th level of the decoder. Modulated feature map. It will participate in subsequent calculations in the decoder.

[0118] Network output: The feature map output from the last layer of the decoder is passed through a 1×1 convolutional layer to map the number of channels to 1, and then through a sigmoid activation function to generate a pixel-level probability map with the same size as the input image. .

[0119] ④ Model training and loss function.

[0120] During the training phase, an end-to-end joint optimization strategy is employed, simultaneously training the modulation parameters to generate the parameters of both the MLP and U-Net segmentation networks. This embodiment uses a combined loss function. Loss due to Dice and weighted cross-entropy loss The weighted summation consists of:

[0121] Dice loss: used to directly optimize the overlap of segmented regions, and has good robustness to class imbalance problems.

[0122]

[0123] Where M is the predicted probability map, G is the true labeled binary mask, and x and y represent the horizontal and vertical coordinates, respectively.

[0124] Weighted cross-entropy loss: To address the class imbalance problem where pixels in density anomalous regions are far fewer than those in normal tissue, higher weights are applied to pixels with density anomalous regions.

[0125]

[0126] Among them, weight The ratio of positive to negative pixels in the dataset is dynamically calculated and set to 10.0 in this embodiment.

[0127] Combination loss function:

[0128]

[0129] In this embodiment, the settings are as follows: This is to balance the contributions of the two loss functions.

[0130] Training Process: The Adam optimizer was used, with an initial learning rate of 1e-4 and a batch size of 8. The training data consisted of enhanced CT images and corresponding pixel-level density anomaly region annotations. During training, the parameters of the modulation parameter generation MLP and the U-Net segmentation network were updated synchronously, and the combined loss function was minimized using the backpropagation algorithm. .

[0131] ⑤ Inference application and post-processing optimization.

[0132] After the model training is completed, for a new CT image, the optimized location label (One-Hot encoding) of each tomographic image is first obtained through S3 and S4. Then, the dual-window image of each tomographic image and the corresponding location encoding are input into the conditional U-Net to obtain the preliminary pixel-level density anomaly region probability map M.

[0133] To improve the spatiotemporal consistency and robustness of the results, a decision-level fusion based on sequence context is performed on the preliminary probability map. Specifically, for each spatial location (x, y) of the t-th tomographic image in the CT image sequence, its prediction within a window [t-2, t+2] centered on that tomographic image is examined. Only if the prediction probability of this pixel being a density anomalous pixel in at least 3 out of 5 consecutive tomographic images at that location exceeds a set threshold (e.g., 0.7) is the pixel at position (x, y) in the t-th tomographic image ultimately determined to be a density anomalous pixel. This process can be formalized as follows:

[0134]

[0135] Where I[·] is the indicator function. This post-processing step effectively utilizes the strong prior knowledge that density anomaly regions should have continuity in three-dimensional space, significantly reducing false positives and filling the gaps caused by low contrast in a single tomographic image, ultimately outputting high-quality, anatomically consistent three-dimensional density anomaly region annotations.

[0136] The conditional U-Net segmentation network in this embodiment can adaptively adjust the feature response according to the relative position of the tomographic image in the lung, effectively improving the accuracy and robustness of detecting abnormal density regions in different anatomical areas.

[0137] In summary, this invention addresses the issue of significant image differences at different locations in lung CT tomographic images. It constructs a conditional U-Net segmentation network based on relative position, trains the network end-to-end, and implements a positional image segmentation method adaptable to tomographic images with different relative positions. A relative position prediction network is designed for this network, and accurate relative position prediction results are obtained through data annotation and training. Furthermore, the CRF method ensures that the relative position prediction results in the complete CT image sequence are more reasonable. This invention, through the combined use of multiple methods, achieves the detection of density anomaly regions, with pixel-level prediction results shown in [see figure]. Figure 6 By comparing with the ground truth, it can be found that the model has better prediction results. For example... Figure 6 As shown, the fourth column is the prediction result image. The white area in the image is the density abnormal area. Based on the size and shape of the abnormal area, doctors can help determine what kind of disease the patient might have corresponding to the CT image.

[0138] This invention provides a CT image region segmentation method based on sequence position optimization and conditional U-Net. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A CT image region segmentation method based on sequence position optimization and conditional U-Net, characterized in that, Includes the following steps: Step 1: Obtain the set of lung CT tomographic image sequences to be processed, and perform data preprocessing and enhancement on each tomographic image contained in each set of tomographic image sequences. The tomographic images include lung window tomographic images and mediastinal window tomographic images. Step 2: For each preprocessed tomographic image, construct a 2.5D image patch consisting of it and the adjacent tomographic images in the same sequence, and input them into the relative position prediction model based on a dual-branch feature extraction network to obtain the initial probability distribution of the relative position of the tomographic image. Step 3: Construct a one-dimensional chained conditional random field (CRF) model, perform relative position optimization on each tomographic image to obtain the globally optimal position label for each tomographic image, and encode the position label into a One-Hot vector; Step 4: Input the One-Hot vector into the lightweight modulation parameter generation network to generate scaling and offset parameters for feature modulation; input the preprocessed tomographic image into the conditional U-Net segmentation network, and in the decoder part of the U-Net segmentation network, implement network parameter conditionalization based on relative position, and finally output a pixel-level density anomaly region probability map. Step 5: Threshold the probability map of the density anomaly region and optimize the spatiotemporal consistency by combining the processing results of adjacent tomographic images to generate the final binary label mask of the density anomaly region.

2. The CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 1, characterized in that, In step 1, the data preprocessing and enhancement specifically include: During the model training phase, enhancement operations are performed on the input image patches, including random translation, scaling, rotation, and brightness / contrast adjustment.

3. The CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 1, characterized in that, Step 2 specifically includes: Step 2-1: For each tomographic image in the same group, stitch it together with the previous and next tomographic images in the channel dimension to form a three-channel input image block; for the boundary images at the beginning and end, use the copy-fill method to process the missing adjacent images. Step 2-2: Dual-branch feature extraction: Lung window branch: captures local texture details of areas with abnormal density in lung window tomographic images; Mediastinal window branch: captures global context and anatomical information related to areas with abnormal density in mediastinal window tomographic images; Steps 2-3: Feature Fusion and Initial Classification: The feature maps output by the lung window branch and the mediastinal window branch are concatenated along the channel dimension for feature integration and dimensionality reduction. Then, after passing through a global average pooling layer and a fully connected layer, the initial probability distribution of the tomographic image belonging to K predefined relative position categories is obtained through the Softmax activation function.

4. The CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 3, characterized in that, The true labels for the relative position categories mentioned in steps 2-3 are obtained in the following way: For each complete lung CT image sequence, the position of the lung apex near the neck in the tomographic image is normalized to 0. Based on the position of the lung base apex near the abdomen in the tomographic image, normalize to 1; The continuous interval from 0 to 1 is divided into K continuous sub-intervals. Based on the specific coordinate values ​​of each tomographic image on the normalized longitudinal coordinate axis, the sub-intervals are classified and labeled into the corresponding discrete location categories.

5. The CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 1, characterized in that, In step 3, the optimization and encoding of the relative position of the tomographic image specifically includes: Step 3-1: CRF Model Construction: Model the CT tomographic image sequence containing N tomographic images as a one-dimensional chain graph structure. Each node For a tomographic image, each edge Connect adjacent tomographic images; Step 3-2: Solving for relative position labels: The Viterbi algorithm is used to minimize the energy function of the above CRF model to obtain the globally optimal relative position label sequence for the entire CT tomographic image sequence; Step 3-3: Position Encoding: Convert the optimal discrete position label in the obtained relative position label sequence into a K-dimensional One-Hot vector.

6. The CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 1, characterized in that, In step 4, the lightweight modulation parameter generation network is a lightweight multilayer perceptron (MLP). The MLP contains at least one hidden layer and uses the ReLU activation function. Its output layer dimension is... , where C is the number of channels in the feature map of the corresponding stage of the U-Net decoder; the output vector is split into two parts, which are used as scaling parameters and offset parameters respectively.

7. The CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 1, characterized in that, The conditional U-Net segmentation network structure described in step 4 includes: The encoder consists of four downsampling modules. Each module contains two 3×3 convolutional layers, a batch normalization (BN) layer, and a ReLU activation function, and is finally connected to a 2×2 max pooling layer for downsampling. Decoder: Consists of a four-level upsampling module, a BN layer, and a ReLU activation function.

8. The CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 7, characterized in that, Step 4 specifically includes: Step 4-1: Modulation parameter generation: The lightweight modulation parameter generation network maps the input K-dimensional One-Hot vector to modulation parameters, namely scaling parameters and offset parameters; Step 4-2: Input the preprocessed tomographic image into the encoder. The encoder performs feature extraction in four downsampling stages in sequence and finally outputs a feature map. Step 4-3: Each upsampling module of the decoder first performs deconvolution or bilinear interpolation upsampling on the received feature map, then concatenates the upsampling result with the feature map passed from the corresponding layer of the encoder through skip connections, and then outputs the feature map through a 3×3 convolution layer. ; Step 4-4: Feature Modulation: In each upsampling stage of the decoder, the output feature map is modulated based on the modulation parameters. The feature map is obtained by applying a position-based affine transformation. This will lead to the replacement It participates in the calculation of the next upsampling module of the decoder; Steps 4-5: Network output: The last layer of the decoder generates a pixel-level density anomaly region probability map with the same size as the input image.

9. A CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 7, characterized in that, During the training phase, a combined loss function is used for end-to-end joint optimization.

10. A CT image region segmentation method based on sequence position optimization and conditional U-Net according to claim 1, characterized in that, The spatiotemporal consistency optimization performed in step 5, which combines the processing results of adjacent tomographic images, specifically involves: For each tomographic image in the same set of tomographic images, iterate through each pixel position; if more than Q tomographic images in the current tomographic image and its P preceding and following tomographic images are initially identified as density anomalous regions at that position, then the pixel at that position in the current tomographic image is finally determined to be a density anomalous pixel; otherwise, it is determined to be a non-density anomalous pixel; where P and Q are preset positive integers, and Q ≤ P.

Images