Reinforcement learning based method for lung cancer recognition from CT images

Abstract

The application discloses a CT image lung cancer recognition method based on reinforcement learning, generates a lung structure mask, outputs an initial lesion candidate region, an initial lesion mask, a morphological confidence map and a boundary response map, obtains a focused feature map, obtains an initial lesion fine mask, a boundary probability map and an uncertainty heat map, encodes a reinforcement learning state vector, establishes a DSAC-T refining intelligent agent, obtains an updated lesion fine mask, calculates a reward value, trains the DSAC-T refining intelligent agent by using the reward value, repeats the steps until a preset convergence condition is met or a maximum iteration number is reached, and outputs a final lesion fine mask; consistent lesion masks are generated, and comprehensive evaluation results for clinical diagnosis and follow-up are formed. The application improves the fine segmentation stability under a fuzzy boundary, high noise and extreme samples, and improves the detection rate of micro-lesions and clinical applicability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of CT image lung cancer, and particularly relates to a CT image lung cancer recognition method based on reinforcement learning. BACKGROUND

[0002] With the popularization of lung cancer screening and early diagnosis, the recognition of lung nodules and lesion regions based on chest CT images has become an important research direction in the field of intelligent medical images. The existing technology mainly relies on two-dimensional or three-dimensional convolutional neural networks for automatic segmentation and recognition of lung cancer CT images. Generally, an end-to-end feedforward segmentation structure is used to directly output a lesion mask. However, in the face of complex scenes such as small lesions, ground glass nodules, and adhesion to blood vessels, the following shortcomings are generally present:

[0003] Due to the blurred boundaries, low contrast, and easy interference of imaging noise and artifacts between lung nodules and surrounding blood vessels, pleura and other tissues, the traditional feedforward segmentation method often appears over-segmentation or under-segmentation. The change of segmentation threshold and post-processing parameters on ground glass nodules, blood vessel adhesion or sub-centimeter lesions will cause the segmentation result to be extremely unstable, and it is difficult to guarantee the accuracy and consistency of clinical use.

[0004] Different devices, scanning parameters and reconstruction kernels result in large differences in the performance of the same lesion in multi-slice CT slices. The traditional method is difficult to effectively establish consistent constraints across slices and domains. The voxel-level mask is prone to jumping or breaking in three-dimensional volume and spatial structure, affecting the reliability of lesion volume evaluation and follow-up comparison. In addition, most current methods lack effective dynamic correction mechanisms for boundary fuzzy regions and high uncertainty positions, and are highly sensitive to reward scales and threshold changes. The stability and generalization ability of training and reasoning are limited. SUMMARY

[0005] One object of the present application is to provide a CT image lung cancer recognition method based on reinforcement learning. The present application improves the stability of fine segmentation in the presence of blurred boundaries, high noise and extreme samples, and improves the detection rate of small lesions and clinical applicability.

[0006] According to the CT image lung cancer recognition method based on reinforcement learning of the present application, the method comprises the following steps:

[0007] Collecting chest CT image original data and performing standardization processing, performing lung structure analysis on the standardized chest CT image data set, and generating a lung structure mask;

[0008] Constructing an initial detection-segmentation network, inputting the standardized chest CT image data set and the lung structure mask into the initial detection-segmentation network, and outputting an initial lesion candidate region, an initial lesion mask, a morphological confidence map and a boundary response map;

[0009] The lesion region attention map is generated by fusing the initial lesion candidate region, lung structure mask and morphological confidence map. The lesion region attention map and the boundary response map are used together in the multi-scale feature extraction process to obtain the focused feature map.

[0010] A segmentation backbone network is constructed. The focused feature map is input into the segmentation backbone network to obtain the initial fine mask of the lesion, the boundary probability map and the uncertainty heatmap, and the reinforcement learning state vector is encoded to establish the DSAC-T refined agent.

[0011] The action space is defined, including threshold adjustment operation, morphological filtering operation, region growth and shrinkage operation and boundary smoothing operation. The DSAC-T refinement agent selects and executes refinement actions from the action space according to the current reinforcement learning state vector to obtain the updated disease fine mask.

[0012] Calculate the reward value, use the reward value to train the DSAC-T refined agent, repeat the steps until the preset convergence condition is met or the maximum number of iterations is reached, and output the final disease fine mask.

[0013] Cross-slice three-dimensional connectivity analysis was performed on the final fine lesion mask and lung structure mask to generate a consistent lesion mask and form a comprehensive evaluation result for clinical diagnosis and follow-up.

[0014] Optionally, generating the lung structure mask includes:

[0015] The raw data of chest CT images are collected and scanned according to the specified voxel spacing standard. All collected voxels together constitute the raw chest CT image dataset.

[0016] Normalize the position of each voxel in the original chest CT image dataset to obtain the normalized chest CT image dataset.

[0017] The normalized chest CT image dataset was resampled by voxel spacing to obtain a standardized chest CT image dataset.

[0018] Lung field extraction processing was performed on the standardized chest CT image dataset to obtain the final refined lung field mask;

[0019] Lung structure analysis was performed based on a standardized chest CT image dataset, and the results, together with the final refined lung field mask, constituted a lung structure mask set.

[0020] Optionally, the initial detection-segmentation network includes:

[0021] The standardized chest CT image dataset and the lung structure mask set are fused in a channel-cascaded manner to obtain the fused input tensor;

[0022] constructing an initial detection-segmentation network, inputting the fused input tensor, and outputting an initial lesion candidate region, an initial lesion mask, a morphological confidence map, and a boundary response map;

[0023] The initial lesion candidate region is a spatial position distribution of suspected lesions in the chest CT image data.

[0024] The initial lesion mask is a voxel-level segmentation map generated in the initial lesion candidate region, used to represent whether each spatial position belongs to the lesion region.

[0025] The morphological confidence map assigns a morphological stability score to each lesion candidate region.

[0026] The boundary response map is an edge discrimination heat map output by the network, representing the discrimination response intensity of each spatial position at the lesion mask boundary.

[0027] Optionally, the lesion region attention map and the boundary response map jointly act on the multi-scale feature extraction process, including:

[0028] The initial lesion candidate region, the lung structure mask set, and the morphological confidence Figure One With the input attention guide module, the lesion region attention weight coefficient is calculated at each position in the three-dimensional spatial position.

[0029] For each three-dimensional spatial position, the lesion region attention weight coefficient is multiplied by the voxel gray value of the corresponding position in the standardized chest CT image data set to obtain the voxel value fused with the attention weight in the spatial position.

[0030] In the process of generating the lesion region attention map, the lung lobe mask, the blood vessel mask, and the morphological confidence map are used as context guide inputs to extract channel weighting coefficients and spatial enhancement factors, respectively.

[0031] The lesion region attention map and the boundary response Figure One With the input multi-scale feature extraction module, for each three-dimensional spatial position, the voxel value fused with the attention weight is channel-level spliced with the boundary discrimination response intensity of the spatial position, and the channel-level spliced result is used as the input feature into the feature extraction network.

[0032] The input feature is processed using a multi-scale feature pyramid structure, and a downsampling operation and a horizontal connection fusion operation are performed at each pyramid level to output a focused feature map of each layer. The focused feature maps of all levels are integrated into a focused feature with a unified spatial dimension through upsampling and fusion operations.

[0033] Optionally, the DSAC-T refining agent is established, including:

[0034] The focused feature map is input into a segmentation backbone network, and through multi-layer convolutional encoding and decoding operations of the segmentation backbone network, an initial lesion fine mask, a boundary probability map and an uncertainty heat map are respectively output;

[0035] For each voxel position in the uncertainty heat map, the prediction probabilities of all classification categories by the segmentation backbone network are respectively counted, the prediction probability value of each category is multiplied by its logarithmic probability value, the sum of the products of all categories is taken and the inverse is taken to obtain the uncertainty value of the voxel position;

[0036] The initial lesion fine mask, the lesion region attention map, the boundary probability map, the uncertainty heat map and the historical action trajectory information are encoded in a channel-level connection manner to form a current reinforcement learning state vector;

[0037] A DSAC-T refining agent including a policy network and a value distribution network is constructed, the current reinforcement learning state vector is input into the DSAC-T refining agent, the policy network outputs a sampling probability distribution of each selectable refining operation under the current state according to the current reinforcement learning state vector, and the value distribution network outputs a reward value distribution corresponding to each selectable refining operation under the current state.

[0038] Optionally, the process of calculating the reward value and training the DSAC-T refining agent includes the following steps:

[0039] The Hausdorff distance between the initial lesion fine mask and the manually labeled gold standard mask is subtracted by the Hausdorff distance between the updated lesion fine mask and the manually labeled gold standard mask to obtain a Hausdorff distance reduction amount;

[0040] The information entropy values of all voxel positions in the initial uncertainty heat map and the updated uncertainty heat map are respectively counted, and the average of the difference between the information entropy values of all voxel positions is obtained to obtain an average information entropy reduction amount;

[0041] The proportion of voxels in the non-lesion region of the updated lesion fine mask that are mislabeled as lesion regions is taken as a false positive penalty ratio;

[0042] The Hausdorff distance reduction amount, the average information entropy reduction amount and the false positive penalty ratio are combined by weighting to construct an immediate reward function to obtain a single-step reward value;

[0043] The current state, the single-step reward value and the updated lesion fine mask are combined into a triple, which is input into an experience replay pool of the DSAC-T refining agent, and the state updating, the action sampling, the single-step reward value calculation and the model training are repeatedly performed until a preset convergence condition is met or a maximum iteration step number is reached, and finally, the lesion fine mask is output.

[0044] Optionally, the comprehensive evaluation result for clinical diagnosis and follow-up comprises:

[0045] The final lesion fine mask and the lung structure mask set are in a three-dimensional space, and it is determined whether each voxel position has a 26-adjacent relationship in the three-dimensional space with any voxel in the current connected region, and all voxels meeting the connectivity requirement are defined as a three-dimensional voxel connected component;

[0046] The number of voxels contained in each three-dimensional voxel connected component is counted, the three-dimensional voxel connected component with a voxel number less than a specified connected volume threshold is removed, only the three-dimensional voxel connected component with a voxel number greater than or equal to the connected volume threshold and spatial distribution in the fine lung field mask range is retained, the voxel positions of all retained three-dimensional voxel connected components are marked as lesion regions in the unified lesion mask, and the remaining voxel positions are marked as non-lesion regions;

[0047] Boundary curvature extraction and boundary clarity quantification are performed on the unified lesion mask to obtain a boundary clarity index;

[0048] The information entropy values of all voxel positions corresponding to the uncertainty heat map in the unified lesion mask region are counted, and the average of all information entropy values is obtained to obtain a regional average uncertainty index;

[0049] According to the boundary clarity index and the regional average uncertainty index, the preset normalization weighting coefficients are respectively assigned and summed to obtain a lesion recognition confidence index;

[0050] The comprehensive evaluation result including the boundary clarity index, the regional average uncertainty index, the lesion recognition confidence index and the unified lesion mask is output.

[0051] The beneficial effects of the present application are:

[0052] The application deeply fuses the distributed reinforcement learning DSAC-T algorithm and the lesion area attention mechanism in the lung cancer CT image fine recognition process, encodes multi-dimensional features into reinforcement learning states, uses the DSAC-T agent to step by step fine-tune the post-processing actions, and uses the Hausdorff distance reduction, information entropy reduction and false positive penalty medical related indicators as reward driving, realizes controllable and closed loop fine repair of the segmentation boundary, significantly reduces the over-segmentation and missing segmentation phenomenon of blood vessel adhesion, ground glass nodule difficult lesion, improves the fine segmentation stability under the boundary blur, high noise and extreme sample, and improves the detection rate and clinical applicability of the micro-lesion.

[0053] In the feature extraction stage, the application fuses the lesion candidate area, the lung structure mask, the morphological confidence map and the boundary response map, dynamically weights the structural features in the three-dimensional space based on the channel and spatial attention mechanism, and suppresses the artifact interference and interlayer breakage through the multi-scale feature pyramid and three-dimensional connectivity analysis, realizes the consistent mask generation of the lesion area under the cross-slice, cross-device and different reconstruction parameters, and greatly improves the coherence and repeatability of the micro, wall-adhered and complex morphological lesions in the three-dimensional volume evaluation and follow-up comparison process by introducing the screening and spatial structure constraint of the three-dimensional voxel connectivity component.

[0054] The application introduces the uncertainty heat map information entropy analysis and the boundary sharpness index to the final recognition result, realizes the multi-dimensional comprehensive evaluation output of the regional average uncertainty, the boundary curvature and the lesion recognition reliability through the structure analysis of the consistent lesion mask, can provide the fine segmentation boundary visualization, the model prediction uncertainty differentiation and the high reliability index quantification for the doctor in the result output layer, enables the clinical user to intuitively judge the reliable interval of the automatic recognition result, and assists individualized diagnosis and treatment and image follow-up. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, illustrate the application together with the embodiments of the application, and do not constitute a limitation of the application. In the drawings:

[0056] Figure 1 A flowchart of a CT image lung cancer recognition method based on reinforcement learning is provided. DETAILED DESCRIPTION

[0057] Embodiment 1: Reference Figure 1 A CT image lung cancer recognition method based on reinforcement learning, comprising:

[0058] Collecting chest CT image original data and performing standardization processing, performing lung structure analysis on the standardized chest CT image data set, and generating a lung structure mask;

[0059] In this embodiment, a lung structure mask is generated, comprising:

[0060] Collecting chest CT image original data, and scanning the chest CT image original data according to a specified voxel spacing standard, all collected voxels collectively constituting an original chest CT image data set;

[0061] The voxel spacing standard includes a transverse resolution, a longitudinal resolution and an axial resolution, respectively corresponding to the transverse, longitudinal and axial resampling resolutions of the chest CT image, and each chest CT image voxel collected includes a gray value in a three-dimensional coordinate system, and the number of voxels in the original chest CT image data set is equal to the total number of collected voxels.

[0062] Normalizing each voxel position in the original chest CT image data set to obtain a normalized chest CT image data set;

[0063] The normalization process clips the gray value of each voxel position between the normalization lower limit value and the normalization upper limit value, subtracts the normalization lower limit value from the clipped gray value, and divides the normalization upper limit value by the difference between the normalization upper limit value and the normalization lower limit value to obtain the normalized voxel gray value. The normalized chest CT image data set is used to suppress the influence of extreme high density and extreme low density on the lung field extraction process.

[0064] Resampling the voxel spacing of the normalized chest CT image data set to obtain a standardized chest CT image data set;

[0065] The voxel spacing resampling process takes the original voxel spacing and the target voxel spacing as parameters, and reconstructs the voxel distribution of the normalized chest CT image data set into a standardized chest CT image data set with the target voxel spacing by a trilinear interpolation method. The spatial resolution of each voxel position of the standardized chest CT image data set conforms to the voxel spacing standard.

[0066] Performing lung field extraction processing on the standardized chest CT image data set to obtain a final refined lung field mask;

[0067] The lung field extraction processing is based on a gray threshold value, and the lung field candidate region is screened out by threshold segmentation, combined with connected domain screening and morphological closing operation to form a preliminary lung field mask, and the lung door area is excluded according to the anatomical structure rule to obtain the final refined lung field mask. The final refined lung field mask is used to distinguish voxels belonging to lung structure from voxels not belonging to lung structure.

[0068] Based on the standardized chest CT image data set, the lung structure is analyzed, and the final refined lung field mask is used to form a lung structure mask set together with the final refined lung field mask;

[0069] In Example 1, the lung structure parsing is performed based on the standardized chest CT image dataset, the lung structure parsing is performed by inputting the standardized chest CT image dataset and the final refined lung field mask into the multi-channel anatomical structure segmentation network, the multi-channel anatomical structure segmentation network outputs a structure mask of a lung lobe region, a structure mask of a bronchial tree region and a structure mask of a blood vessel region according to the input standardized chest CT image dataset and the final refined lung field mask, the obtained structure mask of the lung lobe region, the structure mask of the bronchial tree region and the structure mask of the blood vessel region correspond to different anatomical structures in the chest CT image, and finally the structure mask of the lung lobe region, the structure mask of the bronchial tree region and the structure mask of the blood vessel region are combined with the final refined lung field mask to form a lung structure mask set.

[0070] The initial detection-segmentation network is constructed, the standardized chest CT image dataset and the lung structure mask are input into the initial detection-segmentation network, and an initial lesion candidate region, an initial lesion mask, a morphological confidence map and a boundary response map are output.

[0071] In the embodiment, the initial detection-segmentation network comprises:

[0072] The standardized chest CT image dataset and the lung structure mask set are fused in a channel cascading manner to obtain a fused input tensor.

[0073] The fused input tensor includes all voxel information of the standardized chest CT image dataset and spatial structure information of each type of spatial structure in the lung structure mask set, the fused input tensor is completely consistent with the standardized chest CT image dataset in the spatial dimension, the number of channels is equal to the sum of the number of all structure masks in the standardized chest CT image dataset and the lung structure mask set, and each channel corresponds to a type of spatial structure information in the standardized chest CT image dataset or the lung structure mask set.

[0074] The initial detection-segmentation network is constructed, and the input is the fused input tensor, and the output includes an initial lesion candidate region, an initial lesion mask, a morphological confidence map and a boundary response map.

[0075] In Example 1, the initial detection-segmentation network is a convolutional neural network model based on deep learning, the input of the initial detection-segmentation network is the fused input tensor, the fused input tensor includes voxel information of the standardized chest CT image dataset and spatial structure information of each type of spatial structure in the lung structure mask set, all channels are cascaded in a fixed order, the initial detection-segmentation network adopts a 2.5D convolutional structure, and the 2.5D convolutional structure utilizes multiple continuous slice information at each time of forward calculation to improve the ability to capture spatial context.

[0076] The network structure of the initial detection-segmentation network includes an input layer, multiple convolutional layers, multiple pooling layers, a feature fusion module, a spatial pyramid pooling module, and a multi-branch output layer. The network is trained in an end-to-end manner. The training data is a standardized chest CT image dataset and a lung structure mask set after normalization, resampling, and structure mask fusion processing. The training label is a chest CT lesion segmentation mask and a lesion region position annotation manually drawn by a doctor.

[0077] During the training process, the weighted sum of cross-entropy loss, Dice loss, and boundary loss is used as the objective function. The network parameters are optimized through the back propagation algorithm, so that the network can simultaneously output the initial lesion candidate region, the initial lesion mask, the morphological confidence map, and the boundary response map.

[0078] After the training is completed, the obtained initial detection-segmentation network model parameters and network structure are used as the basis for the inference stage. In the inference stage, the input is a new standardized chest CT image dataset and a lung structure mask set. The output includes the initial lesion candidate region, the initial lesion mask, the morphological confidence map, and the boundary response map. The spatial resolution of all outputs is consistent with the input data.

[0079] The initial lesion candidate region is the spatial position distribution of suspected lesions in the chest CT image data.

[0080] The spatial position distribution is formed by threshold screening, connected component sorting, and non-maximum suppression of the region proposal map predicted by the initial detection-segmentation network. The spatial position distribution includes all positions with a score value greater than the lesion region screening threshold. The score value is the score of the corresponding voxel in the lesion region score map predicted by the network.

[0081] The initial lesion mask is a voxel-level segmentation map generated within the initial lesion candidate region, which is used to represent whether each spatial position belongs to a lesion region.

[0082] For each spatial position, if the voxel-level lesion probability value output by the initial detection-segmentation network is greater than the mask generation threshold, the spatial position is marked as a lesion region in the initial lesion mask. Otherwise, it is marked as a non-lesion region.

[0083] The morphological confidence map assigns a morphological stability score to each lesion candidate region.

[0084] The morphological stability score is determined by the morphological encoding submodule in the initial detection-segmentation network based on the continuity of the lesion edge, the consistency within the region, and the compactness of the region shape.

[0085] The boundary response map is an edge discrimination heat map output by the network, which represents the discrimination response intensity of each spatial position on the lesion mask boundary.

[0086] The lesion region attention map is generated based on fusion of the initial lesion candidate region, the lung structure mask and the morphological confidence map, and the lesion region attention map and the boundary response map are jointly used in a multi-scale feature extraction process to obtain a focused feature map;

[0087] In the embodiment, the lesion region attention map and the boundary response map are jointly used in a multi-scale feature extraction process, which includes:

[0088] The initial lesion candidate region, the lung structure mask set and the morphological confidence map are fused to generate the lesion region attention map. Figure One The input attention guide module is used to calculate the lesion region attention weight coefficient at each position in the three-dimensional space.

[0089] In the embodiment, the lung structure mask set, the initial lesion candidate region and the morphological confidence map in each three-dimensional space position are respectively input as feature channels, and the feature response value of the channel at the current space position is extracted for each feature channel. A unique channel weighting coefficient is assigned to each feature channel to measure the contribution degree of the channel in the attention fusion at the space position. The feature response value of each feature channel at the current space position is multiplied by the corresponding channel weighting coefficient to obtain a weighted feature response. The weighted feature responses of all channels are summed at the space position to obtain a channel fusion response value. A unique spatial enhancement factor is assigned to each three-dimensional space position, and the spatial enhancement factor reflects the local context significance of the space position. The channel fusion response value and the spatial enhancement factor are added to obtain a fusion response value, and the fusion response value is input to an activation function. The output of the activation function is used as the lesion region attention weight coefficient of the current three-dimensional space position.

[0090] For each three-dimensional space position, the lesion region attention weight coefficient is weighted and multiplied by the voxel gray value of the corresponding position in the standardized chest CT image data set to obtain a voxel value that has fused the attention weight in the space position.

[0091] In the process of generating the lesion region attention map, the lung lobe mask, the blood vessel mask and the morphological confidence map are used as context guide inputs, and the channel weighting coefficient and the spatial enhancement factor are extracted.

[0092] The channel weighting coefficient is used to measure the weight of each feature channel in the attention fusion process, the spatial enhancement factor is used to measure the local response of each three-dimensional space position, the lesion region attention weight coefficient is obtained by multiplying all channel weighting coefficients by the feature response of the corresponding feature channel, adding the spatial enhancement factor, limiting the numerical range through an activation function, and using the output of the activation function as the lesion region attention weight coefficient.

[0093] The lesion region attention map and the boundary response Figure OneThe input multi-scale feature extraction module, for each three-dimensional spatial position, splices the voxel value fused with the attention weight and the boundary discrimination response intensity of the spatial position at a channel level, and the result after the channel-level splicing is taken as an input feature entering a feature extraction network;

[0094] The input feature is processed using a multi-scale feature pyramid structure, a down-sampling operation and a horizontal connection fusion operation are respectively performed at each pyramid level, a focused feature map of each layer is output, and the focused feature maps of all levels are integrated into a focused feature of a unified spatial dimension through an up-sampling and fusion operation.

[0095] The focused feature map is input into the segmentation backbone network to obtain an initial lesion fine mask, a boundary probability map and an uncertainty heat map, and an encoded reinforcement learning state vector is obtained to establish a DSAC-T refining agent.

[0096] In the embodiment, the DSAC-T refining agent is established, including:

[0097] The focused feature map is input into the segmentation backbone network, and the segmentation backbone network outputs an initial lesion fine mask, a boundary probability map and an uncertainty heat map through multi-layer convolutional encoding and decoding operations.

[0098] In the embodiment 1, the focused feature map is input into the segmentation backbone network, the segmentation backbone network performs multi-layer convolutional encoding operations on the focused feature map to extract deep features at different spatial scales, and the deep features are gradually restored to the spatial size of the input focused feature map through multi-layer convolutional decoding operations. In the restoration process, the segmentation backbone network outputs an initial lesion fine mask, a boundary probability map and an uncertainty heat map for each three-dimensional spatial position. Figure One

[0099] Each voxel position of the initial lesion fine mask is thresholded by the segmentation backbone network to determine the probability that the voxel belongs to the lesion region. The voxels with a probability value greater than a specified threshold are marked as lesion regions, otherwise as non-lesion regions. Each voxel position of the boundary probability map is output by the segmentation backbone network to determine the probability that the voxel belongs to the lesion boundary. The higher the probability value, the stronger the boundary discrimination response. Each voxel position of the uncertainty heat map is calculated by the segmentation backbone network to determine the information entropy of the prediction probability distribution of the voxel in all classification categories. The greater the information entropy value, the higher the uncertainty of the model at the position.

[0100] For each voxel position in the uncertainty heat map, the prediction probability of each classification category is counted, the prediction probability value of each category is multiplied by its logarithmic probability value, the sum of the products of all categories is taken and the reciprocal is taken to obtain the uncertainty value of the voxel position.

[0101] ​The uncertainty value is used to measure the reliability of the model in the voxel position on the classification result, and the number of categories is equal to the number of all classification categories divided by the method, including the lesion region and the non-lesion region.

[0102] The initial lesion fine mask, the lesion region attention map, the boundary probability map, the uncertainty heat map and the historical motion trajectory information are encoded in a channel-level connection manner to form a current reinforcement learning state vector;

[0103] In embodiment 1, for each voxel position in a three-dimensional space, the values of the voxel position in the initial lesion fine mask, the lesion region attention map, the boundary probability map, the uncertainty heat map and the historical motion trajectory information are sequentially spliced into a feature vector. After the feature vectors of all voxel positions are arranged according to the spatial distribution, a state feature tensor of the entire three-dimensional space is formed. The feature tensor after all channels are spliced is completely consistent with the standardized chest CT image dataset in terms of spatial size. The number of channels is equal to the sum of the number of channels of the initial lesion fine mask, the lesion region attention map, the boundary probability map, the uncertainty heat map and the historical motion trajectory information. The obtained state feature tensor is used as the current reinforcement learning state vector for the DSAC-T refining agent to make action decisions and refining processing.

[0104] A DSAC-T refining agent including a policy network and a value distribution network is constructed. The current reinforcement learning state vector is input into the DSAC-T refining agent. The policy network outputs a sampling probability distribution of each selectable refining operation under the current state according to the current reinforcement learning state vector. The value distribution network outputs a reward value distribution corresponding to each selectable refining operation under the current state.

[0105] Each selectable refining operation is evaluated by the policy network and the value distribution network. By jointly referring to the sampling probability distribution and the reward value distribution, the sampling probability distribution is used to measure the possibility of sampling and selecting each refining operation under the current state, and the reward value distribution is used to measure the reward that each refining operation can obtain under the current state.

[0106] An action space including a threshold adjustment operation, a morphological filtering operation, a region growing contraction operation and a boundary smoothing operation is defined. The DSAC-T refining agent selects a refining action from the action space according to the current reinforcement learning state vector and performs the refining action to obtain an updated lesion fine mask.

[0107] The reward value is calculated, the DSAC-T refining agent is trained using the reward value, and the steps are repeated until a preset convergence condition is met or a maximum iteration number is reached, and a final lesion fine mask is output.

[0108] In this embodiment, the process of calculating the reward value and training the DSAC-T refined agent includes the following steps:

[0109] The Hausdorff distance reduction amount is obtained by subtracting the Hausdorff distance between the updated lesion fine mask and the manually annotated gold standard mask from the Hausdorff distance between the initial lesion fine mask and the manually annotated gold standard mask.

[0110] The Hausdorff distance reduction amount represents the improvement degree of boundary deviation between the lesion fine mask boundary and the manually annotated gold standard mask, and the greater the Hausdorff distance reduction amount represents the more significant the boundary precision improvement.

[0111] The average information entropy reduction amount is obtained by respectively calculating the information entropy values of all voxel positions in the initial uncertainty heat map and the updated uncertainty heat map, and then averaging the difference between the information entropy values of all voxel positions.

[0112] The average information entropy reduction amount is used to measure the improvement degree of the overall prediction uncertainty of the model, and the greater the average information entropy reduction amount represents the greater the overall uncertainty reduction of the model.

[0113] The false positive penalty ratio is the proportion of voxels in the non-lesion region of the updated lesion fine mask that are mistakenly labeled as lesion regions.

[0114] The voxel proportion is obtained by counting the number of voxels in the updated lesion fine mask that are labeled as lesion regions but are actually non-lesion regions in the manually annotated gold standard mask, and dividing by the total number of voxels labeled as lesion regions in the updated lesion fine mask. The voxel proportion is used to punish mask overexpansion or false detection of artifacts, and the greater the voxel proportion represents the more serious the false positive.

[0115]

[0116] wherein G(i) = 0 represents that position i belongs to the gold standard non-lesion region, represents that position i is labeled as a lesion region by the current model, and R FP ∈ [0, 1], represents the refined false positive penalty ratio, which is used to punish mask overexpansion or false detection of artifacts.

[0117] The Hausdorff distance reduction amount, the average information entropy reduction amount, and the false positive penalty ratio are combined by weighting to construct an instant reward function, and a single-step reward value is obtained.

[0118] The instant reward function is obtained by assigning normalized weight coefficients to the boundary improvement value, the average information entropy reduction and the false positive penalty term respectively, and weighting and summing the boundary improvement value and the average information entropy reduction as positive rewards and the false positive penalty term as a negative penalty to obtain a single-step reward value, which is used to measure the actual contribution of the current refinement action in the current state.

[0119] The current state, the single-step reward value and the updated lesion fine mask are combined into a triple, which is input into the experience replay pool of the DSAC-T refinement agent. The state updating, action sampling, single-step reward value calculation and model training are repeatedly performed until the preset convergence condition is met or the maximum iteration step is reached, and the final lesion fine mask is output.

[0120] The cross-slice three-dimensional connectivity analysis is performed on the final lesion fine mask and the lung structure mask to generate a consistent lesion mask, and a comprehensive evaluation result for clinical diagnosis and follow-up is formed.

[0121] In the embodiment, the comprehensive evaluation result for clinical diagnosis and follow-up includes:

[0122] In the three-dimensional space, the final lesion fine mask and the lung structure mask are used to determine whether each voxel position has a 26-adjacent relationship with any voxel in the current connected region, and all voxels that meet the connectivity requirement are defined as a three-dimensional voxel connected component.

[0123] The 26-adjacent relationship refers to the connectivity between a voxel and its 26 adjacent voxels in a three-dimensional cube.

[0124] The number of voxels contained in each three-dimensional voxel connected component is counted, and the three-dimensional voxel connected components whose voxel number is less than a specified connected volume threshold are removed, and only the three-dimensional voxel connected components whose voxel number is greater than or equal to the connected volume threshold and whose spatial distribution is within the fine lung mask range are retained. The voxel positions of all retained three-dimensional voxel connected components are marked as lesion regions in the consistent lesion mask, and the remaining voxel positions are marked as non-lesion regions.

[0125]

[0126] wherein, C j is the jth three-dimensional voxel connected component, |C j | is the number of voxels contained in the jth connected region, which is used to measure the spatial scale of the connected region, V min is the connected volume threshold, which is set to the minimum number of voxels required to retain significant connected regions, and is used to filter out isolated or small noise artifact regions, For the fine lung field mask, a three-dimensional lung structure space range after structure analysis and rule exclusion is represented as an anatomical constraint for limiting the lesion region space, and M con For the consistent lesion mask, a final retained lesion region voxel set satisfying the structure constraint is obtained through three-dimensional connectivity analysis and volume and spatial position screening.

[0127] Boundary curvature extraction and boundary sharpness quantification are performed on the consistent lesion mask to obtain a boundary sharpness index.

[0128] The boundary sharpness index Q edge The boundary sharpness index Q is a ratio of a sum of absolute values of curvatures of all boundary voxel positions to a total number of boundary voxels, the boundary voxels refer to voxels adjacent to non-lesion regions in the consistent lesion mask, the absolute value of the curvature is used to reflect a structure mutation degree of each boundary voxel position, and a larger boundary sharpness index value indicates a sharper boundary transition and a clearer structure boundary.

[0129]

[0130] wherein, the consistent lesion mask is represented by a boundary voxel set, a two-dimensional or three-dimensional Laplacian operator is represented by, the total number of boundary voxels is represented by.

[0131] An uncertainty heat map information entropy value corresponding to each voxel position in the consistent lesion mask region is counted, and a region average uncertainty index is obtained by averaging all information entropy values.

[0132] The region average uncertainty index is used to reflect the overall stability and reliability of the model prediction result in the consistent lesion mask region, and a smaller region average uncertainty index value indicates a more determined classification result of the model for the region.

[0133] According to the boundary sharpness index and the region average uncertainty index, a lesion recognition confidence index is obtained by weighting and summing according to a preset normalization weighting coefficient.

[0134] The lesion recognition confidence index is used to reflect the clinical reliability of the final lesion recognition result, and a larger lesion recognition confidence index value indicates a higher confidence degree of the recognition result.

[0135] The comprehensive evaluation result including the boundary sharpness index, the region average uncertainty index, the lesion recognition confidence index, and the consistent lesion mask is output.

[0136] Embodiment 2: In the lung tumor early screening center of the respiratory department of a provincial tumor hospital, from July 2024 to October 2024, real lung cancer screening image data from different hospital imaging equipment and CT scanning protocols of multiple manufacturers were collected. In view of the clinical pain points of complex and micro-lesion boundary difficult to stably and accurately segment, cross-slice consistency and weak explainability, the application was applied to the complete clinical process.

[0137] Patient Zhang, male, 57 years old, underwent low-dose spiral CT (LDCT) examination in August 2024 due to chest discomfort. The device model is GE Revolution CT, the scanning layer thickness is 1.0 mm, the reconstruction kernel is lung, the spatial resolution is 512x512x420, and the doctor found a 6mm diameter ground glass nodule in the right upper lobe near the pleura during preliminary reading. This type of nodule often has the following problems in traditional automatic segmentation systems: on the one hand, due to the close contact between the nodule and the blood vessels, the conventional segmentation algorithm often mistakenly includes the blood vessels or pleura in the segmentation mask, resulting in serious boundary false positives; on the other hand, due to the small size of the lesion and low gray scale contrast, the convolutional neural network is very sensitive to segmentation threshold and post-processing parameters, often resulting in voxel mask jumping between different slices, making follow-up volume assessment unreliable. Therefore, 1200 cases of lung nodule image data collected from the hospital from 2022 to 2024 were selected to construct training and verification datasets, including 500 cases of ground glass nodules, 300 cases of wall-adherent nodules, 200 cases of blood vessel-adherent nodules, and 200 cases of ordinary solid nodules. 100 cases of ground glass and blood vessel-adherent nodule samples were randomly selected as an independent test set.

[0138] In the specific application process of the method of the application, the chest CT original image of patient Zhang was subjected to CT value normalization and voxel spacing resampling processing to ensure the consistency of all sample images in spatial resolution and gray scale. Subsequently, a lung structure mask automatic extraction algorithm based on a deep segmentation network was used to accurately separate the lung field, lung lobes, bronchial tree and blood vessel trunk structure, and generate a mask based on the anatomical region, significantly reducing the interference of non-lung artifacts.

[0139] The standardized chest CT image and lung structure mask were input into the initial detection-segmentation network as multi-channel input. The initial detection-segmentation network scanned the volume image layer by layer in a 2.5D sliding window manner to automatically generate an initial lesion candidate region, a voxel-level lesion mask, a morphological confidence map and a boundary response map for each layer. In the Zhang sample, the system successfully generated a high-confidence candidate mask in the right upper lobe of the lung, and accurately distinguished the boundary response at the blood vessel-adherent site. For the other 100 cases of ground glass and blood vessel-adherent nodule samples, the initial screening algorithm detected lesions in an average of 97.5% of the cases, with a false detection rate of 2.1%, which was significantly better than the false detection rate of 4.7% of the traditional method at the same period.

[0140] The lesion region attention fusion module inputs the lesion candidate region, the lung structure mask and the morphological confidence map channel, automatically calculates the attention weight of each spatial position, and takes the Zhang sample as an example. The interface region between the ground glass nodule trunk and the blood vessels is given a high weight, the irrelevant region adjacent to the pleura and away from the nodule core area is automatically suppressed, the boundary response of the fused feature pyramid is closely matched with the actual shape of the attention distribution, and the discrimination ability for irregular and wall-adherent boundaries is significantly improved.

[0141] The segmentation main network performs multi-layer convolutional encoding and decoding on the focused feature map, outputs an initial lesion fine mask, a boundary probability map and an uncertainty heat map, and compared with the traditional end-to-end U-Net, the Dice coefficient of the focused feature mask of the present application in the nodule region of the Zhang sample reaches 0.912, the Hausdorff distance is reduced to 2.7 mm, and the information entropy is reduced by 34.6%, which is obviously improved compared with the traditional U-Net (Dice: 0.872, Hausdorff: 4.3 mm). In the 100 test sets, the average Dice coefficient of the present application is 0.905±0.018, the Hausdorff distance is 3.2±0.7 mm, and the false positive rate is 0.48%, while the average Dice coefficient of the traditional method is 0.864±0.027, the Hausdorff distance is 5.1±1.3 mm, and the false positive rate is 1.85%. In addition, for the 80 nodule samples with obvious interlayer structure jump, the three-dimensional voxel connectivity rate of the present application is improved to 98.3%, and the connectivity rate of the traditional method is 92.4%.

[0142] In the DSAC-T refining intelligent agent module, the initial mask, the attention map, the boundary probability, the uncertainty heat map and the historical action trajectory jointly construct a reinforcement learning state vector, and the DSAC-T strategy network samples multiple refining actions according to the current state, including adaptive adjustment of the segmentation threshold, morphological filtering, region growing and boundary smoothing. Each action is based on the real-time calculation of rewards of Hausdorff distance reduction, average information entropy reduction and false positive penalty ratio, and the action decision highly depends on the medical actual quality target. Taking the Zhang sample as an example, the Hausdorff distance between the lesion mask boundary and the gold standard is reduced from 3.0 mm to 2.2 mm, the average information entropy is reduced from 0.179 to 0.109, and the false positive pixel ratio is reduced from 1.1% to 0.34% in 7-step refining iterations. The average time consumption of the whole DSAC-T refining process is 0.84 seconds per case, which is better than that of the traditional RL (average time consumption of 2.13 seconds per case).

[0143] In the cross-slice three-dimensional connectivity analysis stage, the method of the present application removes isolated noise connected domains by analyzing the voxel 26 adjacency relationship and spatial connectivity components, ensures the integrity and spatial consistency of the nodule region in the three-dimensional volume, and for Zhang's ground glass nodule, the mask volume three-dimensional connectivity rate is 100%, the boundary curvature clarity index is improved by 28%, the regional average uncertainty is reduced from 0.148 to 0.093, and the output result comprehensively considers the three indexes of boundary clarity, regional average uncertainty and recognition reliability, and the final recognition reliability score of Zhang is 0.93, which is located in the Top 10 of the hospital's historical ground glass nodule samples.

[0144] The comparative analysis shows that when the traditional U-Net or DeepLabV3+ scheme is used, the average segmentation Dice coefficient of 100 test samples is 0.864 and 0.877 respectively, the three-dimensional voxel connectivity rate is less than 94%, the follow-up volume jump rate is 9.7%, and the false positive error rate is about 2.1%; the average Dice coefficient of the present application on the same data set is improved to 0.905, the three-dimensional voxel connectivity rate is 98.3%, the volume jump rate is reduced to 3.1%, and the false positive error rate is controlled at 0.48%, and the segmentation boundary stability, spatial coherence and reliability quantification ability of the present application are obviously better than those of the traditional technical route in complex adherent and vascular adhesion nodule samples.

[0145] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1.A CT image lung cancer recognition method based on reinforcement learning, characterized in that, The method comprises the following steps: Collecting chest CT image original data and performing standardization processing, and performing lung structure analysis on the standardized chest CT image data set to generate a lung structure mask; An initial detection-segmentation network is constructed, and the standardized chest CT image data set and the lung structure mask are input into the initial detection-segmentation network to output an initial lesion candidate region, an initial lesion mask, a morphological confidence map and a boundary response map; A lesion region attention map is generated based on the initial lesion candidate region, the lung structure mask and the morphological confidence map fusion, and the lesion region attention map and the boundary response map are used in the multi-scale feature extraction process to obtain a focused feature map; A segmentation backbone network is constructed, and the focused feature map is input into the segmentation backbone network to obtain an initial lesion fine mask, a boundary probability map and an uncertainty heat map, and an encoded reinforcement learning state vector is obtained, and a DSAC-T refining agent is established; An action space including threshold adjustment operations, morphological filtering operations, region growing contraction operations and boundary smoothing operations is defined, and the DSAC-T refining agent selects a refining action from the action space according to the current reinforcement learning state vector and executes the refining action to obtain an updated lesion fine mask; A reward value is calculated, the DSAC-T refining agent is trained using the reward value, and the steps are repeated until a preset convergence condition is met or a maximum iteration number is reached, and a final lesion fine mask is output; A cross-slice three-dimensional connectivity analysis is performed on the final lesion fine mask and the lung structure mask to generate a consistent lesion mask, and a comprehensive evaluation result for clinical diagnosis and follow-up is formed; The DSAC-T refining agent is established, comprising: The focused feature map is input into the segmentation backbone network, and the initial lesion fine mask, the boundary probability map and the uncertainty heat map are output through the multi-layer convolutional encoding and decoding operations of the segmentation backbone network; For each voxel position in the uncertainty heat map, the prediction probabilities of all classification categories by the segmentation backbone network are counted respectively, the prediction probability value of each category is multiplied by its logarithmic probability value, the sum of the products of all categories is calculated and the reciprocal is taken to obtain the uncertainty value of the voxel position; The initial lesion fine mask, the lesion region attention map, the boundary probability map, the uncertainty heat map and the historical action trajectory information are encoded in a channel-level connection manner to form a current reinforcement learning state vector; A DSAC-T refining agent including a policy network and a value distribution network is constructed, and the current reinforcement learning state vector is input into the DSAC-T refining agent, the policy network outputs a sampling probability distribution of each selectable refining operation under the current state according to the current reinforcement learning state vector, and the value distribution network outputs a reward value distribution corresponding to each selectable refining operation under the current state; The process of calculating the reward value and training the DSAC-T refining agent comprises the following steps: The Hausdorff distance between the initial lesion fine mask and the artificially labeled gold standard mask is subtracted from the Hausdorff distance between the updated lesion fine mask and the artificially labeled gold standard mask to obtain a Hausdorff distance reduction amount; Statistically count the information entropy values of all voxel positions in the initial uncertainty heat map and the updated uncertainty heat map respectively, and obtain the average information entropy reduction by taking the difference between the information entropy values of all voxel positions and then averaging them; Take the proportion of voxels in the non-lesion region of the updated fine lesion mask that are mistakenly labeled as lesion regions as the false positive penalty ratio; Combine the Hausdorff distance reduction, the average information entropy reduction, and the false positive penalty ratio by weighting, construct an instant reward function, and obtain a single-step reward value; Combine the current state, the single-step reward value, and the updated fine lesion mask into a triple, input the triple into the experience replay pool of the DSAC-T refining agent, and repeatedly execute state updating, action sampling, single-step reward value calculation, and model training until the preset convergence condition is met or the maximum iteration step is reached, and output the final fine lesion mask. 2.The CT image lung cancer recognition method based on reinforcement learning according to claim 1, wherein, The generation of the lung structure mask comprises: Collecting chest CT image original data, scanning the chest CT image original data according to a specified voxel spacing standard, and all collected voxels collectively constituting an original chest CT image data set; Performing normalization processing on each voxel position in the original chest CT image data set to obtain a normalized chest CT image data set; Performing voxel spacing resampling processing on the normalized chest CT image data set to obtain a standardized chest CT image data set; Performing lung field extraction processing on the standardized chest CT image data set to obtain a final fine lung field mask; Performing lung structure analysis based on the standardized chest CT image data set, and combining the final fine lung field mask to form a lung structure mask set. 3.The CT image lung cancer recognition method based on reinforcement learning according to claim 2, characterized in that, The initial detection-segmentation network comprises: Fusing the standardized chest CT image data set and the lung structure mask set in a channel cascading manner to obtain a fused input tensor; Constructing an initial detection-segmentation network, inputting the fused input tensor, and outputting an initial lesion candidate region, an initial lesion mask, a morphology confidence map, and a boundary response map; The initial lesion candidate region is the spatial position distribution of suspected lesions in the chest CT image data; The initial lesion mask is a voxel-level segmentation map generated in the initial lesion candidate region, used to represent whether each spatial position belongs to a lesion region; The morphology confidence map assigns a morphology stability score to each lesion candidate region; The boundary response map is an edge discrimination heat map output by the network, representing the discrimination response intensity of each spatial position at the lesion mask boundary. 4.The CT image lung cancer recognition method based on reinforcement learning according to claim 3, characterized in that, The lesion region attention map and the boundary response map jointly act on the multi-scale feature extraction process, comprising: Inputting the initial lesion candidate region, the lung structure mask set, and the morphology confidence map into an attention guiding module to calculate a lesion region attention weight coefficient at each position in the three-dimensional space; For each three-dimensional spatial position, multiply the lesion region attention weight coefficient and the voxel gray value at the corresponding position in the standardized chest CT image data set to obtain a voxel value fused with attention weight in the spatial position; and In the process of generating the lesion area attention map, the lung lobe mask, the blood vessel mask and the morphological confidence map are taken as context guide inputs to extract channel weighting coefficients and spatial enhancement factors, respectively; The lesion area attention map and the boundary response map are input into a multi-scale feature extraction module. For each three-dimensional spatial position, the voxel value fused with the attention weight and the boundary discrimination response intensity of the spatial position are channel-level spliced, and the result of the channel-level splicing is taken as the input feature of the feature extraction network; The input feature is processed by a multi-scale feature pyramid structure. Down-sampling operation and horizontal connection fusion operation are performed at each pyramid level, respectively, to output the focusing feature map of each layer. The focusing feature maps of all levels are integrated into a unified spatial dimension focusing feature through up-sampling and fusion operation. 5.The CT image lung cancer recognition method based on reinforcement learning according to claim 1, characterized in that, The comprehensive evaluation result for clinical diagnosis and follow-up is formed, including: In the three-dimensional space, the final lesion fine mask and the lung structure mask set are used to determine whether each voxel position has a 26-adjacent relationship with any voxel in the current connected region. All voxels that meet the connectivity requirement are defined as a three-dimensional voxel connected component; The number of voxels included in each three-dimensional voxel connected component is counted. Three-dimensional voxel connected components with a voxel number less than a specified connected volume threshold are removed. Only three-dimensional voxel connected components with a voxel number greater than or equal to the connected volume threshold and spatial distribution within the fine lung mask range are retained. The voxel positions of all retained three-dimensional voxel connected components are marked as lesion regions in the unified lesion mask. The remaining voxel positions are marked as non-lesion regions. Boundary curvature extraction and boundary sharpness quantification are performed on the unified lesion mask to obtain a boundary sharpness index. The information entropy values of the uncertainty heat map corresponding to all voxel positions in the unified lesion mask region are counted. The average of all information entropy values is taken to obtain a regional average uncertainty index. According to the boundary sharpness index and the regional average uncertainty index, the preset normalization weighting coefficients are respectively assigned and summed to obtain a lesion recognition confidence index. The comprehensive evaluation result including the boundary sharpness index, the regional average uncertainty index, the lesion recognition confidence index and the unified lesion mask is output.

Images