Lesion region recognition model training method and device based on CT image, and product

Abstract

The application provides a lesion region recognition model training method and device based on a CT image, and a product, comprising: obtaining a first weakly-labeled CT image dataset with labeled three-dimensional boundary boxes of lesions and a first strongly-labeled CT image dataset with simultaneously labeled accurate pixel positions and three-dimensional boundary boxes of lesions; performing adjustment processing on the first strongly-labeled CT image dataset to obtain a second strongly-labeled CT image dataset with different sizes of lesions occupying the same proportion in images; training a preset teacher model by using the second strongly-labeled CT image dataset; inputting the first weakly-labeled CT image dataset into the trained teacher model to perform multiple disturbance inferences to obtain a second weakly-labeled CT image dataset with pseudo labels of accurate pixel positions of lesions and corresponding uncertainty maps; and training a preset student model by using the first strongly-labeled CT image dataset and the second weakly-labeled CT image dataset to obtain a trained student model.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method, apparatus, electronic device, and product for determining lesion regions based on CT images. Background Technology

[0002] Early diagnosis and precise treatment of cancer are crucial for improving patient survival rates. Computed tomography (CT), due to its high imaging speed and spatial resolution, has become a widely used imaging technique in cancer screening, diagnosis, and treatment planning. Accurate identification and segmentation of cancerous lesions in CT images is a fundamental requirement for developing artificial intelligence-based cancer lesion identification, quantitative analysis, clinical decision support, and treatment efficacy evaluation. For example, during radiotherapy, accurately delineating the tumor target area is essential to ensure sufficient radiation dose to the tumor and minimize damage to surrounding healthy tissues. Currently, precise pixel segmentation and annotation of tumor lesions in CT images still relies primarily on manual or semi-automatic methods. This process is highly dependent on the operator's experience, time-consuming, labor-intensive, costly, and time-consuming, resulting in a very limited amount of highly annotated sample data. Another method, which uses low-cost annotation methods such as bounding boxes to annotate tumor lesion areas in CT images, reduces annotation costs and makes it easy to obtain a large amount of weakly annotated sample data. However, the bounding boxes often contain a large number of non-lesion areas. Therefore, how to simultaneously apply strongly annotated sample data and weakly annotated sample data to train the tumor lesion area recognition model, and improve the training efficiency and training quality of the model, has become an urgent technical problem to be solved. Summary of the Invention

[0003] In view of this, the present invention provides a method, device and product for training a lesion region recognition model based on CT images.

[0004] In a first aspect, the present invention provides a method for training a lesion region recognition model based on CT images, the method comprising: Obtain the first weakly annotated CT image dataset with 3D bounding boxes of lesions and the first strongly annotated CT image dataset with precise pixel positions of lesions and 3D bounding boxes. The first strongly annotated CT image dataset was adjusted to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the images. Using the second strongly annotated CT image dataset, the pre-set teacher model is trained to obtain the trained teacher model. The trained teacher model can output a location probability map that indicates the lesion area based on the input original CT image. The location probability map includes the lesion pixel location probability map and the lesion bounding box probability map. The first weakly labeled CT image dataset is input into the trained teacher model and subjected to multiple perturbation inferences to obtain a second weakly labeled CT image dataset with pseudo-labels containing the precise pixel locations of lesions and corresponding uncertainty maps. The pre-defined student model was trained using a first strongly annotated CT image dataset and a second weakly annotated CT image dataset to obtain a trained student model. The trained student model is able to determine the precise pixel region of the annotated lesion based on the input original CT image.

[0005] According to an embodiment of the present invention, a first strongly annotated CT image dataset is adjusted to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image, including: Based on the lesion bounding box mask of the first strongly annotated CT image dataset, determine the voxel coordinates and voxel spacing; Determine the ideal resolution based on voxel coordinates, voxel spacing, target cropping size, and preset coverage. An interval constraint is applied to the ideal resolution to obtain an adaptive resampling vector; The images in the first strongly labeled CT image dataset are resampled using an adaptive resampling vector, and the geometric center of the lesion mask in the resampled image is determined. Based on the resampled image, the image is cropped to a preset size according to the geometric center. The area outside the image boundary is filled with a preset background value. The affine matrix of the original image is updated according to the offset vector of the cropping starting point, resulting in a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image.

[0006] According to an embodiment of the present invention, the teacher model adopts a 3D U-Net architecture, consisting of a 4-layer encoder, a 1-layer bridging layer, and a 4-layer decoder, which is used to extract multi-scale semantic features of CT images and output lesion segmentation results.

[0007] According to an embodiment of the present invention, the feature extraction process in teacher model training is as follows: Based on the input 3D binarized bounding box mask, a preliminary mask corresponding to the layer resolution of the encoder is generated using a trilinear interpolation operator. Lightweight convolutional layers are used to perform feature mapping on the initial mask to obtain a guided map with spatial weights; Using the guide graph, the original image features extracted from each layer of the encoder are reconstructed to obtain the reconstructed features; Global average pooling and multilayer perceptron operations are performed on the reconstructed features to obtain key channel features; Convolutional processing is performed on the key channel features to obtain refined features; Enhanced features are obtained based on the original image features and refined features; The original image features and the enhanced features are concatenated and input into the decoder. The decoder outputs a probability map that indicates the location of the lesion area.

[0008] According to an embodiment of the present invention, the teacher model is a single-tensor dual-channel output architecture, with the final layer outputting a single tensor; wherein, in the dual channels, the first channel outputs a probability map of lesion pixel locations, and the second channel outputs a probability map of lesion bounding boxes; the training process of the teacher model is as follows: The images in the second strongly annotated CT image dataset are randomly flipped and then input into the teacher model; The pixel-level lesion segmentation annotations and bounding box annotations of CT images are concatenated along the channel dimension to form a joint supervised objective in a single channel dimension. This objective is then matched with the single-tensor dual-channel recognition results output by the teacher model, achieving joint supervised learning for both the accurate lesion segmentation task and the bounding box localization task. The loss function used in the joint supervised learning is a weighted fusion of Dice loss and binary cross-entropy loss. Dice loss represents the degree of overlap between the model's predicted results and the ground truth labeled regions, while binary cross-entropy loss represents the accuracy of the probability distribution of the model's predicted results.

[0009] According to an embodiment of the present invention, a first weakly labeled CT image dataset is input into a trained teacher model, and multiple perturbation inferences are performed to obtain a second weakly labeled CT image dataset with pseudo-labels containing precise pixel locations of lesions and corresponding uncertainty maps, including: The teacher model is set to training mode, and Monte Carlo sampling inference is used to perform a preset number of iterative inferences on the images in the first weakly labeled CT image dataset to obtain a sequence of lesion probability maps generated by multiple inferences. Based on the lesion probability map sequence, the mean map and standard deviation map are calculated; The mean plot is used as the pseudo-label reference probability, and the standard deviation plot is used as the uncertainty measure plot. Based on the first weakly labeled CT image dataset and the corresponding pseudo-label reference probability and uncertainty metric map of the images, a second weakly labeled CT image dataset with the precise pixel location of the lesion and the corresponding uncertainty map is obtained.

[0010] According to an embodiment of the present invention, the training of the student model includes a first warm-up training phase and a second fine distillation phase: The first warm-up training phase includes: training the student model using the Dice loss function based on the first strongly labeled CT image dataset and the second weakly labeled CT image dataset; The second refinement distillation stage includes: when a batch of training samples contains only image samples from the first strongly labeled CT image dataset, the student model is trained using the Dice loss function; when a batch of training samples contains image samples from the second weakly labeled CT image dataset, the student model is trained using the weighted Dice loss function; wherein, the weighted Dice loss function is a loss function obtained by dynamically modulating the gradient contribution of each pixel in the space based on the uncertainty map and reconstructing the Dice loss function.

[0011] Secondly, the present invention provides a lesion region recognition model training device based on CT images, comprising: The acquisition module is used to acquire the first weakly annotated CT image dataset with the three-dimensional bounding boxes of the lesions, as well as the first strongly annotated CT image dataset with the precise pixel positions of the lesions and the three-dimensional bounding boxes. The adjustment module is used to adjust the first strongly annotated CT image dataset to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image. The first training module is used to train the preset teacher model using the second strongly labeled CT image dataset to obtain the trained teacher model. The trained teacher model can output a location probability map that indicates the lesion area based on the input original CT image. The location probability map includes the lesion pixel location probability map and the lesion bounding box probability map. The generation module is used to input the first weakly labeled CT image dataset into the trained teacher model, perform multiple perturbation inferences, and obtain the second weakly labeled CT image dataset with pseudo-labels containing the precise pixel locations of lesions and the corresponding uncertainty maps. The second training module is used to train the preset student model using the first strongly annotated CT image dataset and the second weakly annotated CT image dataset to obtain the trained student model. The trained student model can determine the precise pixel region of the annotated lesion based on the input original CT image.

[0012] Thirdly, the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the above-described method.

[0014] Fifthly, the present invention provides a computer program product, including computer-executable instructions, which, when executed, are used to implement the above-described method.

[0015] According to embodiments of the present invention, the present invention provides a training method, apparatus, and product for a lesion region recognition model based on CT images. The method involves acquiring a first weakly labeled CT image dataset with annotated 3D bounding boxes of lesions, and a first strongly labeled CT image dataset with annotated precise pixel locations of lesions and their 3D bounding boxes. The first strongly labeled CT image dataset is adjusted to obtain a second strongly labeled CT image dataset where lesions of different sizes occupy the same proportion in the image. A preset teacher model is trained using the second strongly labeled CT image dataset to obtain a trained teacher model. The first weakly labeled CT image dataset is input into the trained teacher model, and multiple perturbation inferences are performed to obtain a second weakly labeled CT image dataset with pseudo-labels containing precise pixel locations of lesions and corresponding uncertainty maps. A preset student model is trained using the first strongly labeled CT image dataset and the second weakly labeled CT image dataset to obtain a trained student model. This invention provides a training method, apparatus, and product for a lesion region recognition model based on CT images. It can simultaneously train the model using a first weakly annotated CT image dataset with labeled 3D bounding boxes of lesions and a first strongly annotated CT image dataset with precisely labeled pixel locations and 3D bounding boxes of lesions. This ensures training quality while fully utilizing a large, readily available dataset of weakly annotated CT images with only labeled 3D bounding boxes of lesions. This effectively reduces training costs, improves training efficiency, and shortens the training cycle. Simultaneously, by utilizing the spatial prior information provided by the bounding box annotations in the training samples, the teacher model can focus on potential lesion regions and suppress background interference when segmenting lesion regions, thereby improving the reliability of pseudo-labels in the generated weakly annotated data and ultimately improving the accuracy of the trained student model in labeling the precise pixel regions of lesions. Attached Figure Description

[0016] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 A flowchart illustrating a method for training a lesion region recognition model based on CT images according to an embodiment of the present invention is shown. Figure 2This schematic diagram illustrates the structural block diagram of a lesion region recognition model training device based on CT images according to an embodiment of the present invention; Figure 3 A block diagram schematically illustrates an electronic device suitable for implementing a lesion region recognition model training method based on CT images, according to an embodiment of the present invention. Figure 4 This is a comparison image of identifying lesion areas in the original CT image to be identified using the method provided by the present invention; Figure 5 This is a comparison image of lesion region identification in the original CT image to be identified using the method provided by the present invention. Detailed Implementation

[0017] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0018] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0019] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0020] In the embodiments of this invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0021] In the embodiments of the present invention, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0022] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0023] According to embodiments of the present invention, such as Figure 1 As shown, a method for training a lesion region recognition model based on CT images is described, the method comprising: Step S101: Obtain the first weakly annotated CT image dataset with the three-dimensional bounding box of the lesion annotated, and the first strongly annotated CT image dataset with the precise pixel position of the lesion and the three-dimensional bounding box annotated simultaneously. For example, the first weakly annotated CT image dataset for labeling the 3D bounding boxes of lesions could be the Lung-PET-CT-Dx (PET-Lung) lung cancer CT image dataset; while the first strongly annotated CT image dataset for labeling the precise pixel locations and 3D bounding boxes of lesions could be the Medical Segmentation Decathlon Task06_Lung (MSD) lung cancer CT image dataset and the NSCLC-Radiomics (NSCLC) lung cancer CT image dataset. The 3D bounding boxes of the lesions are determined based on the lesion segmentation mask. All images are aligned with their corresponding annotation files (pixel-level masks, bounding box annotations) in terms of spatial location, size, and coordinate system to ensure a strict match between the annotation information and the image data.

[0024] The MSD lung cancer CT image dataset and the NSCLC lung cancer CT image dataset are each divided into 70% training, 15% validation, and 15% testing datasets.

[0025] Step S102: Adjust the first strongly annotated CT image dataset to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image. Step S103: Using the second strongly annotated CT image dataset, the preset teacher model is trained to obtain the trained teacher model; wherein, the trained teacher model can output a location probability map that indicates the lesion area based on the input original CT image, the location probability map includes the lesion pixel location probability map and the lesion bounding box probability map. Step S104: Input the first weakly labeled CT image dataset into the trained teacher model and perform multiple perturbation inferences to obtain the second weakly labeled CT image dataset with the precise pixel location of the lesion and the corresponding uncertainty map. Step S105: The preset student model is trained using the first strongly annotated CT image dataset and the second weakly annotated CT image dataset to obtain the trained student model; wherein, the trained student model can determine the precise pixel region of the annotated lesion based on the input original CT image.

[0026] According to embodiments of the present invention, a training method, apparatus, and product for a lesion region recognition model based on CT images are provided. This method can simultaneously utilize a first weakly annotated CT image dataset with labeled 3D bounding boxes of lesions and a first strongly annotated CT image dataset with precisely labeled pixel positions and 3D bounding boxes of lesions. This ensures training quality while fully utilizing a large, readily available dataset of first weakly annotated CT images with only labeled 3D bounding boxes of lesions to train the lesion region recognition model. This effectively reduces training costs, improves training efficiency, and shortens the training cycle. Furthermore, by utilizing the spatial prior information provided by the bounding box annotations, the teacher model can focus on potential lesion regions and suppress interference from background regions when segmenting lesion regions, thereby improving the reliability of pseudo-labels generated from weakly annotated data and the accuracy of the final segmentation results from the student model.

[0027] According to an embodiment of the present invention, a first strongly annotated CT image dataset is adjusted to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image, including: Based on the lesion bounding box mask of the first strongly annotated CT image dataset, determine the voxel coordinates and voxel spacing; Determine the ideal resolution based on voxel coordinates, voxel spacing, target cropping size, and preset coverage. An interval constraint is applied to the ideal resolution to obtain an adaptive resampling vector; For example, the lesion is in axis( The physical length under () is expressed as Physical length The formula is expressed as follows:

[0028] in, Indicates the lesion is in Maximum voxel coordinates under the axis, Indicates the lesion is in The smallest voxel coordinate under the axis, The original voxel spacing.

[0029] Furthermore, the ideal resolution is expressed as Ideal resolution The formula is expressed as follows:

[0030] in, C This indicates the preset coverage rate, with a value of 0.75. S p This indicates the target cutting size, with a value of 128. Indicates the lesion is in x The physical length of the shaft Indicates the lesion is in y The physical length of the shaft.

[0031] Furthermore, the constrained target voxel spacing is expressed as... Constrained target voxel spacing The formula is expressed as follows:

[0032] Furthermore, the adaptive resampling vector is represented as D Adaptive resampling vector D The formula is expressed as follows: D = ( d f , d f , 1.5) In one implementation, the CT image is further truncated in grayscale values, for example, values ​​lower than [a certain value] are truncated. The grayscale value of 1024HU is set to 1024. Gray values ​​higher than 1000HU are set to 1000 to remove irrelevant gray-level interference in the image and retain the effective gray-level range of the lesion.

[0033] The images in the first strongly labeled CT image dataset are resampled using an adaptive resampling vector, and the geometric center of the lesion mask in the resampled image is determined. Based on the resampled image, the image is cropped to a preset size according to the geometric center. The area outside the image boundary is filled with a preset background value. The affine matrix of the original image is updated according to the offset vector of the cropping starting point, resulting in a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image.

[0034] For example, the geometric center is represented as C Geometric center C The formula is expressed as follows:

[0035] , in, Indicates the lesion mask in Minimum coordinates on the axis Indicates the lesion mask in The maximum coordinate on the axis.

[0036] For example, the preset size for cutting can be 128×128×128.

[0037] The updated affine matrix of the original image is represented as follows The updated affine matrix of the original image The formula is expressed as follows:

[0038] in, Represents the original affine matrix. S This is the offset vector of the clipping starting point.

[0039] According to an embodiment of the present invention, the teacher model adopts a 3D U-Net architecture, consisting of a 4-layer encoder, a 1-layer bridging layer, and a 4-layer decoder, which is used to extract multi-scale semantic features of CT images and output lesion segmentation results.

[0040] According to an embodiment of the present invention, the feature extraction process in teacher model training is as follows: Based on the input 3D binarized bounding box mask, a preliminary mask corresponding to the layer resolution of the encoder is generated using a trilinear interpolation operator. For example, the initial mask is represented as S i Preliminary mask S i The formula is expressed as follows:

[0041] in, M b Represented as a 3D binary bounding box mask; Indicates the encoder's first... layer; f ( ) represents the trilinear interpolation operator.

[0042] Lightweight convolutional layers are used to perform feature mapping on the initial mask to obtain a guided map with spatial weights; For example, the guide graph is represented as M i Guide map M i The formula is expressed as follows:

[0043] in, This represents the Sigmoid activation function, used to map the bootstrap weights to the [0, 1] interval; This indicates a batch normalization operation; Indicates the convolution operation; Using the guide graph, the original image features extracted from each layer of the encoder are reconstructed to obtain the reconstructed features; For example, the reshaping feature is represented as Reshaping features The formula is expressed as follows:

[0044] in, Represents the features of the original image. This indicates element-wise multiplication.

[0045] Global average pooling and multilayer perceptron operations are performed on the reconstructed features to obtain key channel features; For example, key channel features are represented as follows Key channel characteristics The formula is expressed as follows:

[0046] in, , The weights are the coefficients; AvgPool represents global average pooling; It is an activation function; This indicates the Sigmod activation function.

[0047] Convolutional processing is performed on the key channel features to obtain refined features; For example, the refining feature is represented as Refined characteristics Key channel features It was calculated using a 3×3 convolutional layer.

[0048] Enhanced features are obtained based on the original image features and refined features; For example, the enhanced feature is represented as Enhanced features The formula is expressed as follows:

[0049] The original image features and the enhanced features are concatenated and input into the decoder. The decoder outputs a probability map that indicates the location of the lesion area.

[0050] For example, after concatenating the original image features and the enhanced features, the resulting feature representation is as follows: Features after splicing The formula is expressed as follows:

[0051] According to an embodiment of the present invention, the teacher model is a single-tensor dual-channel output architecture, with the final layer outputting a single tensor; wherein, in the dual channels, the first channel outputs a probability map of lesion pixel locations, and the second channel outputs a probability map of lesion bounding boxes; the training process of the teacher model is as follows: The images in the second strongly annotated CT image dataset are randomly flipped and then input into the teacher model; For example, images in the second strongly annotated CT image dataset are randomly flipped to improve the model's generalization ability to lesions of different scales, specifically including along... x axis 、y shaft and z Random axis flipping operations and three-dimensional random rotation operations.

[0052] The pixel-level lesion segmentation and bounding box annotations of CT images are concatenated along the channel dimension to form a joint supervised objective in a single channel dimension. This objective is then matched with the single-tensor dual-channel recognition results output by the teacher model, achieving joint supervised learning for both lesion segmentation and bounding box localization tasks. The loss function used in the joint supervised learning is a weighted fusion of Dice loss and binary cross-entropy loss. Dice loss represents the degree of overlap between the model's prediction results and the ground truth regions, while binary cross-entropy loss represents the accuracy of the probability distribution of the model's prediction results.

[0053] For example, the loss function used in joint supervised learning is expressed as: loss function The formula is expressed as follows:

[0054] The single tensor output by the final layer of the teacher model is represented as follows: P The pixel-level lesion segmentation annotation of CT images is represented as P m The bounding box annotation is represented as T m The joint supervision objective in a single-channel dimension is represented as T Joint supervision objectives in a single-channel dimension T It involves segmenting and labeling lesions at the pixel level in CT images. P m With bounding box annotation T m Obtained by splicing along the channel dimension; λ 1. λ 2 represents the loss weighting coefficient; L Dice ( ) represents Dice loss. This represents the binary cross-entropy loss.

[0055] For example, during training, the model is trained using the Adam optimizer, with an initial learning rate set to 1×10⁻⁶. 4 The basic number of channels is 32, the channel expansion coefficient is 2, and the network feature channel sequence is [64, 128, 256, 512, 1024]. The training batch size is set to 1, and the gradient accumulation step is 8 to adapt to the high computational power requirements of 3D medical imaging. The binarization threshold of the model output probability map is set to 0.5. The maximum number of training iterations is 350. After each training iteration, a validation set evaluation is performed, with the Dice coefficient of the lesion channel as the core evaluation indicator. The model weights are saved every 50 iterations, and the model weights with the optimal validation set loss are also saved as the teacher model for subsequent weakly labeled data inference.

[0056] According to an embodiment of the present invention, a first weakly labeled CT image dataset is input into a trained teacher model, and multiple perturbation inferences are performed to obtain a second weakly labeled CT image dataset with pseudo-labels containing precise pixel locations of lesions and corresponding uncertainty maps, including: The teacher model is set to training mode, and Monte Carlo sampling inference is used to perform a preset number of iterative inferences on the images in the first weakly labeled CT image dataset to obtain a sequence of lesion probability maps generated by multiple inferences. For example, 10 iterations of inference are performed for each input sample. During each inference iteration, a small Gaussian noise perturbation is applied to the bounding box prior mask, generating a perturbation mask with subtle differences, denoted as... Disturbance mask The formula is expressed as follows:

[0057] Among them, Clamp( ) indicates a truncation operation; T b Indicates the prior mask of the bounding box; represents Gaussian noise with a standard deviation of 0.05; ~ indicates that the generated random numbers must conform to a normal distribution.

[0058] Based on the lesion probability map sequence, the mean map and standard deviation map are calculated; The mean plot is used as the pseudo-label reference probability, and the standard deviation plot is used as the uncertainty measure plot. For example, the pseudo-label reference probability is represented as P p The uncertainty measure graph is represented as U The lesion probability map sequence is represented as follows: ,in, K This represents the total number of probability graphs.

[0059] Pseudo-label reference probability P p and uncertainty measurement graph U The formula is expressed as follows:

[0060] In another implementation, in order to enhance the uncertainty representation of low-brightness areas and make them more recognizable in subsequent loss calculations, a nonlinear transformation with gamma correction is performed on the uncertainty metric map to obtain an enhanced uncertainty map, which strengthens the model's response capability at lesion boundaries and suspected areas. Enhanced uncertainty diagram representation is as follows Enhanced uncertainty graph The formula is expressed as follows:

[0061] in, δ It is a small constant to avoid the denominator being zero.

[0062] Based on the first weakly labeled CT image dataset and the corresponding pseudo-label reference probability and uncertainty metric map of the images, a second weakly labeled CT image dataset with the precise pixel location of the lesion and the corresponding uncertainty map is obtained.

[0063] According to an embodiment of the present invention, the training of the student model includes a first warm-up training phase and a second fine distillation phase: The first warm-up training phase includes: training the student model using the Dice loss function based on the first strongly labeled CT image dataset and the second weakly labeled CT image dataset; In the first 20 iterations, the model is in the stage of building basic feature extraction capabilities. At this time, no uncertainty information is introduced. All training samples (including strongly labeled data and pseudo-labeled data) are trained using the standard Dice loss function. The core purpose of this stage is to enable the model to quickly learn the basic anatomical features and segmentation rules of lung lesions, lay a solid training foundation, and avoid model instability due to the disturbance of uncertainty map in the early stage of training.

[0064] The second refinement distillation stage includes: when a batch of training samples contains only image samples from the first strongly labeled CT image dataset, the student model is trained using the Dice loss function; when a batch of training samples contains image samples from the second weakly labeled CT image dataset, the student model is trained using the weighted Dice loss function; wherein, the weighted Dice loss function is a loss function obtained by dynamically modulating the gradient contribution of each pixel in the space based on the uncertainty map and reconstructing the Dice loss function.

[0065] Starting from the 21st iteration, the system automatically identifies the source of the supervision signal, realizes implicit differential processing of mixed datasets, and adopts differentiated supervision strategies for different types of data.

[0066] For example, the weighted Dice loss function is expressed as follows: Weighted Dice loss function The formula is as follows:

[0067] in, P s The lesion probability map output for the student model. Pseudo-labels generated for the teacher model; To smooth out the terms, avoid having a denominator of 0. It is a normalized uncertainty measure graph U， This loss modulation mechanism can achieve gradient suppression in high uncertainty regions (usually ambiguous regions such as lesion edges) and supervision reinforcement in low uncertainty regions (regions with high confidence in teacher model predictions), enabling student models to focus on learning reliable knowledge from teacher models while avoiding interference from pseudo-label noise.

[0068] For example, the student model is trained using the Adam optimizer with an initial learning rate of 2.5 × 10⁻⁶. The batch size is 2, and the maximum number of training epochs is 350. To ensure the authenticity and objectivity of the evaluation results, the validation set uses only strongly labeled data, and the model output probability is set to a probability threshold of 0.5 during the validation phase. Figure 2 Values ​​were calculated, and the Dice coefficient of the lesion region was used as the core evaluation indicator. During training, the model with the best performance was saved based on the optimal loss on the validation set, and a stage model was saved every 50 rounds to ensure that the final model has stable clinical performance.

[0069] For example, the student model adopts the classic 3D U-Net architecture, which is a general benchmark network in the field of medical image segmentation. The model takes a single-channel lung CT image as input and directly outputs a single-channel segmentation probability map of the lesion region. It consists of an encoding module, a bridging module, and a decoding module. The encoding module extracts features at multiple scales through convolution, batch normalization, and the LeakyReLU activation function, and performs downsampling through max pooling. The bridging module captures the global semantic information of the highest layer. The decoding module gradually restores the feature map resolution through deconvolution upsampling and skip connections, and finally outputs the lesion probability map through the Sigmoid activation function. The model has a base number of 32 channels, a channel expansion coefficient of 2, and a feature channel sequence of [32, 64, 128, 256]. The overall structure is simple and efficient, suitable for lightweight segmentation tasks of lung lesions.

[0070] During the training phase of the student model, to adapt to the heterogeneous characteristics of strongly labeled and pseudo-labeled data and achieve unified loading and training of both types of data, a robust preprocessing pipeline was designed. The core of this pipeline lies in automatically compatibility with data lacking uncertainty information: by automatically completing zero-uncertainty tensors and mirroring image or label metadata, key-value anomalies during data loading can be avoided without manually splitting data or modifying training code, thereby improving the versatility and robustness of the training process. By mixing and indexing the strongly labeled dataset and the weakly labeled dataset with uncertainty, a random shuffling mechanism is used during the training batch generation phase. This allows the model to alternately encounter deterministic gold standards and confidence-modulated pseudo-labels in the macro-training stream, achieving collaborative optimization of strong and weak supervision signals.

[0071] For example, when training a lesion region recognition model for lung tumors, a lesion-aware balanced cropping strategy is adopted during the training phase to alleviate the sample imbalance problem caused by the extremely small lesion region and excessive background ratio in 3D lung images. Using the lesion region in the pseudo-label as the anchor point, a fixed cropping block size of 192×192×96 is set, and the positive center region containing the lesion and the negative background region are randomly sampled at a 1:1 ratio to ensure that the model fully learns the lesion features from a microscopic perspective. This strategy allows the model to fully access the subtle features of the lesion during training, significantly improving the robustness of the lightweight network in distinguishing small nodules, weak boundaries, and blurred lesions.

[0072] The training process employs random data augmentation to improve the model's generalization ability, including along... x, y The axis flips randomly. xy Operations such as random 90° rotations within the plane are performed, with the enhancement probability set to 0.1. All training images have undergone lung region extraction, HU value truncation, voxel spacing resampling, and size normalization before entering the network, achieving spatial standardization across datasets.

[0073] The process of inputting the original CT image into the trained student model and determining the precise pixel region for annotating lesions is as follows: For example, the preprocessed complete lung CT image is directly input into the student model, the mode is set to evaluation mode and gradient calculation is turned off, and the lesion segmentation probability map is output through forward inference; the probability map is binarized according to the preset probability threshold, the continuous probability result is converted into a binary segmentation mask, the lesion area and the background area are initially divided, and the original lesion segmentation result is obtained, which is the precise pixel area of ​​the lesion.

[0074] Furthermore, a three-dimensional maximum connected component filtering algorithm is used to post-process the preliminary segmentation results, removing isolated noise and small artifact regions that have no anatomical significance, and retaining only the connected components of the core lesions with complete spatial structure. This optimizes the regularity of the segmentation results, making them more consistent with the actual morphological characteristics of lung lesions and improving their practicality.

[0075] The processed lesion segmentation results are saved in NIfTI format, strictly inheriting the spatial affine matrix of the original CT images to ensure accurate spatial registration between the segmentation results and the original images. The segmentation results and original CT samples are categorized and stored using corresponding naming rules for easy subsequent visualization verification and retrieval. Based on the ground truth labels, the loss value and Dice similarity coefficient of the model segmentation results are calculated to quantitatively evaluate the segmentation accuracy. Simultaneously, an inference log recording mechanism is established to record the filename, evaluation indicators, and result saving status for each sample, and the overall average performance indicators are statistically analyzed to generate a complete model inference performance verification report.

[0076] like Figure 4 and Figure 5 As shown, according to the lesion region recognition model training method based on CT images provided by the present invention, after training the recognition model, the trained model is used to recognize the original CT image to be recognized. The green area in the figure is the original lesion segmentation result, that is, the precise pixel area of ​​the lesion. Therefore, it can be seen that, according to the lesion region recognition model training method based on CT images provided by the present invention, after training the recognition model, the recognition model can accurately and with high quality annotate the precise pixel area of ​​the lesion.

[0077] Based on the above-mentioned training method for lesion region recognition model based on CT images, this invention also provides a training device for lesion region recognition model based on CT images. The following will combine... Figure 2 The device is described in detail.

[0078] Figure 2 The diagram illustrates the structure of a training device for a lesion region recognition model based on CT images according to an embodiment of the present invention.

[0079] like Figure 2As shown, the lesion region recognition model training device 200 based on CT images in this embodiment includes an acquisition module 210, an adjustment module 220, a first training module 230, a generation module 240, and a second training module 250.

[0080] The acquisition module 210 is used to acquire the first weakly labeled CT image dataset with the three-dimensional bounding box of the lesion and the first strongly labeled CT image dataset with the precise pixel position of the lesion and the three-dimensional bounding box. The adjustment module 220 is used to adjust the first strongly annotated CT image dataset to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image. The first training module 230 is used to train the preset teacher model using the second strongly labeled CT image dataset to obtain the trained teacher model. The trained teacher model can output a location probability map that indicates the lesion area based on the input original CT image. The location probability map includes the lesion pixel location probability map and the lesion bounding box probability map. The generation module 240 is used to input the first weakly labeled CT image dataset into the trained teacher model, perform multiple perturbation inferences, and obtain a second weakly labeled CT image dataset with pseudo-labels containing the precise pixel locations of lesions and the corresponding uncertainty maps. The second training module 250 is used to train the preset student model using the first strongly annotated CT image dataset and the second weakly annotated CT image dataset to obtain the trained student model; wherein, the trained student model can determine the precise pixel region of the annotated lesion based on the input original CT image.

[0081] According to embodiments of the present invention, any plurality of modules among the acquisition module 210, adjustment module 220, first training module 230, generation module 240, and second training module 250 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the acquisition module 210, adjustment module 220, first training module 230, generation module 240, and second training module 250 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these.

[0082] Figure 3 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of the present invention, is shown schematically. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0083] like Figure 3 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage portion 308 into a random access memory (RAM) 303. The processor 301 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 301 may also include onboard memory for caching purposes. The processor 301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0084] RAM 303 stores various programs and data required for the operation of electronic device 300. Processor 301, ROM 302, and RAM 303 are interconnected via bus 304. Processor 301 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 302 and / or RAM 303. It should be noted that programs may also be stored in one or more memories other than ROM 302 and RAM 303. Processor 301 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0085] According to an embodiment of the present invention, the electronic device 300 may further include an input / output (I / O) interface 305, which is also connected to a bus 304. The electronic device 300 may also include one or more of the following components connected to the input / output (I / O) interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the input / output (I / O) interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 310 as needed so that computer programs read from it can be installed into the storage section 308 as needed.

[0086] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by processor 301, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0087] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0088] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0089] For example, according to embodiments of the present invention, a computer-readable storage medium may include the ROM 302 and / or RAM 303 described above and / or one or more memories other than ROM 302 and RAM 303.

[0090] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the computer system to implement the methods provided in the embodiments of the present invention.

[0091] When the computer program is executed by the processor 301, it performs the functions defined in the system / apparatus of this embodiment of the invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0092] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via communication section 309, and / or installed from removable medium 311. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0093] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the processor 301, it performs the functions defined in the system of this embodiment of the invention. According to embodiments of the invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0094] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0097] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A method for training a lesion region recognition model based on CT images, characterized in that, The method includes: Obtain the first weakly annotated CT image dataset with 3D bounding boxes of lesions and the first strongly annotated CT image dataset with precise pixel positions of lesions and 3D bounding boxes. The first strongly annotated CT image dataset is adjusted to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image. Using the second strongly labeled CT image dataset, a preset teacher model is trained to obtain a trained teacher model; wherein, the trained teacher model can output a location probability map that indicates the lesion area based on the input original CT image, the location probability map including the lesion pixel location probability map and the lesion bounding box probability map. The first weakly labeled CT image dataset is input into the trained teacher model and subjected to multiple perturbation inferences to obtain a second weakly labeled CT image dataset with pseudo-labels containing the precise pixel locations of lesions and corresponding uncertainty maps. The preset student model is trained using the first strongly annotated CT image dataset and the second weakly annotated CT image dataset to obtain a trained student model; wherein, the trained student model can determine the precise pixel region of the annotated lesion based on the input original CT image.

2. The method for training a lesion region recognition model based on CT images according to claim 1, characterized in that, The first strongly annotated CT image dataset is adjusted to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the images, including: Based on the lesion bounding box mask of the first strongly annotated CT image dataset, determine the voxel coordinates and voxel spacing. The ideal resolution is determined based on the voxel coordinates, the voxel spacing, the target cropping size, and the preset coverage. An interval constraint is applied to the ideal resolution to obtain an adaptive resampling vector; The adaptive resampling vector is used to resample the images in the first strongly labeled CT image dataset, and the geometric center of the lesion mask in the resampled image is determined. Based on the resampled image, the image is cropped to a preset size according to the geometric center. Areas exceeding the image boundary are filled with a preset background value. The affine matrix of the original image is updated according to the offset vector of the cropping starting point, resulting in a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image.

3. The method for training a lesion region recognition model based on CT images according to claim 1, characterized in that, The teacher model adopts a 3D U-Net architecture, consisting of a 4-layer encoder, a 1-layer bridging layer, and a 4-layer decoder, and is used to extract multi-scale semantic features of CT images and output lesion segmentation results.

4. The method for training a lesion region recognition model based on CT images according to claim 3, characterized in that, The feature extraction process during the training of the teacher model is as follows: Based on the input 3D binarized bounding box mask, a preliminary mask corresponding to the layer resolution of the encoder is generated using a trilinear interpolation operator. The initial mask is feature-mapped using a lightweight convolutional layer to obtain a guided map with spatial weights; Using the aforementioned guiding graph, the original image features extracted from each layer of the encoder are reconstructed to obtain reconstructed features; Global average pooling and multilayer perceptron operations are performed on the reconstructed features to obtain key channel features; The key channel features are convolutionally processed to obtain refined features; Enhanced features are obtained based on the original image features and the refined features; The original image features and the enhanced features are concatenated and input into a decoder. The decoder outputs a probability map indicating the location of the lesion area.

5. The method for training a lesion region recognition model based on CT images according to claim 3, characterized in that, The teacher model is a single-tensor dual-channel output architecture, with the final layer outputting a single tensor. Specifically, the first channel outputs the probability map of the lesion pixel location, and the second channel outputs the probability map of the lesion bounding box. The training process of the teacher model is as follows: The images in the second strongly labeled CT image dataset are randomly flipped and then input into the teacher model; The pixel-level lesion segmentation annotations and bounding box annotations of CT images are concatenated along the channel dimension to form a joint supervised objective in a single channel dimension. This objective is then matched with the single-tensor dual-channel recognition results output by the teacher model to achieve joint supervised learning for the tasks of accurate lesion segmentation and bounding box localization. The loss function used in the joint supervised learning is a weighted fusion of Dice loss and binary cross-entropy loss. The Dice loss represents the degree of overlap between the model's predicted results and the ground truth labeled regions, while the binary cross-entropy loss represents the accuracy of the probability distribution of the model's predicted results.

6. The method for training a lesion region recognition model based on CT images according to claim 3, characterized in that, The process of inputting the first weakly labeled CT image dataset into the trained teacher model and performing multiple perturbation inferences to obtain a second weakly labeled CT image dataset with pseudo-labels containing precise pixel locations of lesions and corresponding uncertainty maps includes: The teacher model is set to training mode, and Monte Carlo sampling inference is used to perform a preset number of iterative inferences on the images in the first weakly labeled CT image dataset to obtain a sequence of lesion probability maps generated by multiple inferences. Based on the lesion probability map sequence, the mean map and standard deviation map are calculated; The mean plot is determined as the pseudo-label reference probability; the standard deviation plot is determined as the uncertainty measure plot. Based on the first weakly labeled CT image dataset, the pseudo-label reference probability corresponding to the image, and the uncertainty metric map, a second weakly labeled CT image dataset with pseudo-labels containing precise pixel locations of lesions and corresponding uncertainty maps is obtained.

7. The method for training a lesion region recognition model based on CT images according to claim 3, characterized in that, The training of the student model includes a first warm-up training phase and a second fine distillation phase: The first warm-up training phase includes: training the student model using the Dice loss function based on the first strongly labeled CT image dataset and the second weakly labeled CT image dataset; The second refinement distillation stage includes: when a batch of training samples contains only image samples from the first strongly labeled CT image dataset, training the student model using the Dice loss function; when a batch of training samples contains image samples from the second weakly labeled CT image dataset, training the student model using the weighted Dice loss function; wherein the weighted Dice loss function is a loss function obtained by dynamically modulating the gradient contribution of each pixel in the space based on the uncertainty map and reconstructing the Dice loss function.

8. A training device for a lesion region recognition model based on CT images, comprising: The acquisition module is used to acquire the first weakly annotated CT image dataset with the three-dimensional bounding boxes of the lesions, as well as the first strongly annotated CT image dataset with the precise pixel positions of the lesions and the three-dimensional bounding boxes. The adjustment module is used to adjust the first strongly annotated CT image dataset to obtain a second strongly annotated CT image dataset in which lesions of different sizes occupy the same proportion in the image. The first training module is used to train a preset teacher model using the second strongly labeled CT image dataset to obtain a trained teacher model; wherein, the trained teacher model can output a location probability map that indicates the lesion area based on the input original CT image, and the location probability map includes a lesion pixel location probability map and a lesion bounding box probability map. The generation module is used to input the first weakly labeled CT image dataset into the trained teacher model, perform multiple perturbation inferences, and obtain a second weakly labeled CT image dataset with pseudo-labels containing the precise pixel locations of lesions and corresponding uncertainty maps. The second training module is used to train a preset student model using the first strongly annotated CT image dataset and the second weakly annotated CT image dataset to obtain a trained student model; wherein the trained student model can determine the precise pixel region of the annotated lesion based on the input original CT image.

9. An electronic device, comprising: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.

Images