Lung tumor CT image automatic segmentation method based on deep learning

Abstract

The invention relates to the technical field of image segmentation, in particular to a lung tumor CT image automatic segmentation method based on deep learning. The method comprises the following steps: firstly, acquiring three-dimensional CT images of different breathing time phases and a tumor area in the first image; then acquiring a rigid transformation model of lung movement; marking points are determined in a preset range of the tumor area, marking points in each non-first image and deformation vectors of the marking points are determined, stable marking points are screened out, deformation vectors of the unstable marking points are corrected, and then each three-dimensional CT image and correction positions of the marking points in the three-dimensional CT image are input into a trained segmentation model to obtain a segmentation model; and obtaining a lung tumor segmentation result. According to the method, the respiratory movement of the lung tissue is decomposed and analyzed, the movement changes of the mark points in the tumor area in different respiratory time phases are accurately evaluated, then automatic segmentation is carried out based on the segmentation model of deep learning, and the lung tumor CT image segmentation effect based on deep learning is improved.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and specifically to an automatic segmentation method for lung tumor CT images based on deep learning. Background Technology

[0002] Respiratory movements can cause changes in the shape and location of lung tumors. The same lung tumor may exhibit different morphological features at different respiratory phases. This morphological change makes it impossible for the tumor segmentation results of a two-dimensional or three-dimensional CT image in a static phase to accurately represent all the information of the tumor, which may lead to blurred tumor boundaries, missed segmentation, or missegmentation. However, by integrating tumor segmentation information from two-dimensional or three-dimensional CT images in multiple phases, it is possible to more comprehensively and accurately identify and analyze the true tumor morphology.

[0003] Currently, deep learning-based segmentation models are typically used to segment tumors sequentially in two-dimensional or three-dimensional CT images at different respiratory phases. While these models perform well on lung CT images at each fixed phase, they lack the ability to model the dynamic evolution of tumors during respiratory motion. Overall or local deformation of lung tumors at different respiratory phases can affect the stability and consistency of segmentation, and image noise can also affect segmentation accuracy, potentially leading to inconsistent segmentation results, blurred boundaries, or missed segments at different respiratory phases. Summary of the Invention

[0004] To address the technical problem of poor segmentation results in lung tumor CT images based on deep learning, the present invention aims to provide an automatic segmentation method for lung tumor CT images based on deep learning. The specific technical solution adopted is as follows: A deep learning-based automatic segmentation method for lung tumor CT images, the method comprising: Acquire three-dimensional CT images of the patient's lungs at different respiratory phases, as well as the tumor region in the first three-dimensional CT image; Obtain all motion trajectory point sets, each containing all matching feature points between different 3D CT images; based on the positional changes of the matching feature points in each motion trajectory point set and the differences between the corresponding voxel information, obtain a rigid transformation model of lung motion. Marker points are randomly determined within a preset range of the tumor region; for any marker point, the tracking position of the marker point in each non-first 3D CT image is determined based on the tracking matching algorithm, and the matching marker point of the marker point in each non-first 3D CT image is determined based on the rigid transformation model. In each non-first 3D CT image, the deformation vector of the corresponding marker point is obtained based on the deviation of the marker point's position from the tracking position. Stable marker points are selected from all marker points based on the spatial distance between different marker points and the difference between the corresponding deformation vectors, and the deformation vectors of unstable marker points are corrected. The positions of the markers are corrected based on the deformation vectors of the markers in the non-first 3D CT images; each 3D CT image and the corrected positions of the markers within it are input into the trained segmentation model to obtain the lung tumor segmentation results.

[0005] Furthermore, the method for obtaining the rigid transformation model includes: Based on the differences between voxel information within a preset spatial range corresponding to the feature points matched in each motion trajectory point set, motion stability parameters are obtained. Motion trajectory point sets whose motion stability parameters are greater than a preset stability threshold are taken as stable motion trajectory point sets, and feature points within stable motion trajectory point sets are taken as stable points. Between temporally adjacent 3D CT images, using the motion stability parameters as weights, a rigid transformation sub-model is fitted based on weighted least squares and the position coordinate differences between all matched stable points. The rigid transformation sub-model includes a rotation matrix and a translation vector. By combining the rigid transformation sub-models between all temporally adjacent 3D CT images, a rigid transformation model for lung motion is determined.

[0006] Furthermore, the method for obtaining the motion stability parameters includes: For each set of motion trajectory points, a voxel block of a preset size is determined in the 3D CT image to which each feature point belongs, with the voxel corresponding to each feature point as the center; motion stability parameters are obtained based on the distribution characteristics of the mutual information values ​​between the voxel blocks corresponding to the feature points that match each set of motion trajectory points.

[0007] Furthermore, marker points are randomly determined within a predetermined range of the tumor region, including: The preset region includes the interior of the tumor region, the region boundary, and the preset periphery; within the preset region, marker points are randomly generated at a preset density.

[0008] Furthermore, the method for obtaining the deformation vector includes: Using any marker point within a preset range of the tumor region as the target marker point, the matching marker point determined based on the rigid transformation model in each non-first 3D CT image is used as the predicted marker point; In each non-first 3D CT image, the vector pointing from the coordinates of the predicted marker point to the coordinates of the tracking position corresponding to the target marker point is used as the deformation vector of the reference marker point.

[0009] Furthermore, the method for obtaining the stable marker points includes: In each three-dimensional lung CT image, marker points are used as grid points to construct a marker point network; and between each connected marker point in each grid, the deformation consistency weight between connected marker points is obtained based on the negative correlation mapping result of spatial distance and the negative correlation mapping result of the difference between the corresponding deformation vectors. The normalized cosine values ​​of the angles between the deformation vectors corresponding to the connected marker points are weighted using the deformation consistency weight. The weighted average of all connected marker points in the grid is used as the deformation consistency parameter. Grids with deformation consistency parameters greater than a preset threshold are designated as stable grids, and the marker points corresponding to the grid points in the stable grids are designated as stable marker points.

[0010] Furthermore, the deformation vector of the unstable marker point is corrected by: For each unstable marker point, a preset number of its nearest stable marker points are selected as reference marker points, and the negative correlation mapping result of the spatial distance between the unstable marker point and each reference marker point is used as the reference weight. The deformation vector of each reference marker point is weighted and averaged using the reference weights, and the weighted average result is used as the deformation vector after correction for the unstable marker point.

[0011] Furthermore, the method for obtaining the position of the correction marker includes: In each non-first 3D CT image, the position coordinates of each stable marker point are added to the deformation vector to obtain the corrected position of the stable marker point; the position coordinates of each unstable marker point are added to the corrected deformation vector to obtain the corrected position of the unstable marker point.

[0012] Furthermore, the method for obtaining the set of motion trajectory points includes: Based on the 3D scale-invariant feature transformation algorithm, all feature points in each 3D CT image are obtained, and feature points between different 3D CT images are tracked and matched. The set of matching feature points between different 3D CT images is used as the motion trajectory point set.

[0013] Furthermore, the method for obtaining the marker network includes: A marker point network is constructed based on the 3D Delaunay triangulation algorithm.

[0014] The present invention has the following beneficial effects: This invention acquires three-dimensional CT images of a patient's lungs at different respiratory phases, providing a data foundation for subsequent analysis of respiratory motion. It also acquires the tumor region from the first three-dimensional CT image, preparing for subsequent assessment of tumor motion changes in conjunction with respiratory motion. Furthermore, it acquires a set of all motion trajectory points and, based on the positional changes of matching feature points in each motion trajectory point set and the differences between corresponding voxel information, obtains a rigid transformation model characterizing the overall motion or macroscopic displacement of the lungs. Marker points are randomly determined within a preset range of the tumor region. Then, in each subsequent three-dimensional CT image, the deviation between the position of the marker points determined based on the rigid motion model and the tracking position of the marker points determined based on the matching pursuit algorithm is analyzed. This invention obtains deformation vectors representing the motion of corresponding marker points, excluding rigid motion. Based on the spatial distance between different marker points and the differences between their corresponding deformation vectors, stable marker points with similar deformations within the local tumor tissue are selected from all marker points. The deformation vectors of unstable marker points are further corrected to filter out motion deviations caused by suspected noise interference, in addition to rigid motion and local deformation. The positions of the marker points are corrected based on their deformation vectors in non-first 3D CT images to accurately assess their positions after motion in different respiratory phases. Finally, each 3D CT image and the corrected positions of its marker points are input into a trained segmentation model to obtain lung tumor segmentation results. This invention analyzes the respiratory motion characteristics of lung tissue in different respiratory phases, decomposes respiratory motion into rigid motion and local elastic deformation, and eliminates respiratory motion deviations caused by noise. This allows for the assessment of motion changes of marker points within the tumor region in different respiratory phases, and then automatic segmentation based on a deep learning-based segmentation model. This effectively overcomes the problem of tumor position and morphological changes caused by respiration, improving the segmentation effect of deep learning-based lung tumor CT images. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating an automatic segmentation method for lung tumor CT images based on deep learning, provided as an embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for obtaining a rigid transformation model according to an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a deep learning-based automatic segmentation method for lung tumor CT images proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details a specific scheme for an automatic segmentation method for lung tumor CT images based on deep learning, provided by this invention.

[0020] Please see Figure 1 The document illustrates a flowchart of an automatic segmentation method for lung tumor CT images based on deep learning, according to an embodiment of the present invention, which specifically includes: Step S1: Obtain three-dimensional CT images of the patient's lungs at different respiratory phases, as well as the tumor region in the first three-dimensional CT image.

[0021] In one embodiment of the present invention, the patient's respiratory cycle is first monitored in real time using a respiratory pressure sensor or breathing belt, and the respiratory signal is synchronously transmitted to a multi-slice spiral CT with thin-slice scanning capability, such as a 256-slice MSCT, with the slice thickness set to 1 mm, which can also be adjusted by the practitioner. The scan is triggered under a preset respiratory phase, and cross-sectional images of the lungs under different respiratory phases are quickly acquired, so that the cross-sectional images of the lungs correspond precisely to the respiratory phases. The cross-sectional images of the lungs are two-dimensional sequence images. Three-dimensional reconstruction is performed on the two-dimensional sequence images under each respiratory phase to obtain a three-dimensional CT image. The three-dimensional CT image refers to a three-dimensional model of the lungs, which is composed of voxels.

[0022] The respiratory phases are divided as follows: a preset number of time points are determined in a complete respiratory cycle (e.g., the beginning of inhalation to the end of exhalation), and each time point corresponds to a respiratory phase; in this embodiment, the preset number is 10, and 10 respiratory phases are obtained; in other embodiments, the implementer can adjust the preset number according to the actual collection situation, and can also customize the preset respiratory phases such as the beginning of inhalation, the end of inhalation, the beginning of exhalation, and the end of exhalation.

[0023] It should be noted that the above-mentioned three-dimensional CT images are required to cover the entire lung region and the complete respiratory cycle; the three-dimensional CT images only contain lung tissue and need to be preprocessed to remove other tissues to avoid interfering with the subsequent analysis of lung motion before three-dimensional reconstruction is performed; the above-mentioned operating methods are all well-known technologies and will not be described in detail.

[0024] It should be noted that the above acquisition is prospective respiratory gating (triggering medical image acquisition at a specific respiratory phase); in other embodiments, the implementer may also use retrospective respiratory gating, that is, continuous scanning throughout the entire respiratory cycle, synchronously recording respiratory signals, and later classifying and acquiring images according to respiratory phase labels. Both of these and prospective respiratory gating are well-known technical means, and the specific acquisition process will not be described in detail.

[0025] Considering that lung tumors can shift or deform with respiration, in order to solve the problem of poor segmentation results caused by respiratory motion, this embodiment of the invention further obtains the tumor region in the first three-dimensional CT image, providing prior location information for subsequent automatic segmentation. This allows for the subsequent analysis of the movement changes of lung tissue in different respiratory phases throughout the complete respiratory cycle, thereby calculating the true location and shape of the tumor region in the three-dimensional CT images of other respiratory phases, and thus accurately segmenting lung tumors in three-dimensional CT images of different respiratory phases.

[0026] In one embodiment of the present invention, the first three-dimensional CT image is input into a trained tumor segmentation model to mark the tumor region.

[0027] It should be noted that the current tumor segmentation model is only applied to tumor segmentation of the first 3D CT image. Its training and application are well-known techniques and will not be described in detail here. In other embodiments, the tumor region can be determined by manual labeling.

[0028] Step S2: Obtain all motion trajectory point sets. Each motion trajectory point set contains all matching feature points between different 3D CT images. Based on the positional changes of the matching feature points in each motion trajectory point set and the differences between the corresponding voxel information, obtain the rigid transformation model of lung motion.

[0029] Considering that the lungs move and rotate as a whole during respiration, which is a rigid motion, meaning that the relative positions of all points on the lung tissue remain unchanged; secondly, the lung tissue may also undergo additional local deformations such as expansion or contraction during the overall rigid motion, which is a non-rigid motion; in addition, due to image acquisition noise, partial volume effects, and the blurring of soft tissue boundaries, the positions of points on the lung tissue in 3D CT images at different respiratory phases may deviate to some extent, which is offset motion or error caused by noise interference; therefore, the motion of lung tumors can be decomposed into rigid motion, non-rigid motion, and noise offset motion.

[0030] To accurately analyze the motion patterns of lung tumors, this embodiment of the invention first analyzes the rigid motion of the lungs, which characterizes the overall motion of all lung tissues. Therefore, this embodiment of the invention first obtains all motion trajectory point sets, each containing all matching feature points between different three-dimensional CT images. The motion trajectory point sets initially provide information on the motion changes of local points on the lung tissue at different respiratory phases, providing a reference for local tissue motion in subsequent analysis of rigid lung motion.

[0031] Preferably, in one embodiment of the present invention, considering that feature points refer to unique and identifiable key points in an image, these points are highly robust to noise, occlusion, or changes in image rotation and scaling, enabling them to be accurately tracked between different frames; the 3D Scale-Invariant Feature Transform (3D-SIFT) algorithm can extract and describe feature points in three-dimensional space and effectively identify the same object in different scenes or from different perspectives; based on this, the method for obtaining the motion trajectory point set includes: Based on the 3D scale-invariant feature transformation algorithm, all feature points in each 3D CT image are obtained, and feature points between different 3D CT images are tracked and matched. The set of matching feature points between different 3D CT images is used as the motion trajectory point set.

[0032] It should be noted that extracting feature points and performing inter-frame matching based on the three-dimensional scale-invariant feature transformation algorithm is a well-known technology and will not be elaborated further; in other embodiments, implementers may also use other three-dimensional tracking and matching algorithms.

[0033] Considering that the feature points matched in each motion trajectory point set can be regarded as the corresponding voxels of the same local lung tissue in different 3D CT images, after obtaining all motion trajectory point sets, the relative motion changes and morphological changes of the corresponding voxels of the same local lung tissue can be evaluated based on the positional changes of the feature points matched in the motion trajectory point sets and the differences between the corresponding voxel information. Furthermore, considering that some feature points may move abnormally during respiration due to local deformation, image artifacts, or matching errors, directly analyzing all motion trajectory point sets inevitably introduces errors. However, when local lung tissue undergoes rigid motion, its relative position and shape within the lung tissue remain unchanged. For example, when lung tissue moves with respiration, a local tissue located in the upper left corner of the lung remains relatively in the upper left corner of the lung at different respiratory phases, and its local texture features change relatively little. Therefore, based on the changes in the motion morphology of local lung tissue corresponding to all motion trajectory points, the overall motion of lung tissue can be comprehensively evaluated to obtain a rigid transformation model of lung motion. The rigid transformation model reflects the macroscopic displacement pattern or overall motion pattern of lung tissue between adjacent respiratory phases, preparing for subsequent prediction of the location of lung tumors at different respiratory phases after the overall movement of lung tissue.

[0034] Preferably, in one embodiment of the present invention, the method for obtaining the rigid transformation model includes: Please see Figure 2 The flowchart illustrates a method for obtaining a rigid transformation model according to an embodiment of the present invention, specifically including: Step S201: Based on the difference between the voxel information corresponding to the feature points in each motion trajectory point set within a preset spatial range, obtain motion stability parameters, take the motion trajectory point set with motion stability parameters greater than a preset stability threshold as the stable motion trajectory point set, and take the feature points within the stable motion trajectory point set as stable points.

[0035] Considering that the voxel information of the feature points in the corresponding three-dimensional CT image within the preset spatial range can characterize the texture and morphology of the corresponding local lung tissue in each motion trajectory point set, analyzing the local texture and morphology information corresponding to the matching feature points can help measure whether the local lung tissue undergoes morphological changes during respiration. This can then be used to evaluate the motion stability parameters of the motion trajectory point set. The motion stability parameters reflect the probability that the motion trajectory point set is a rigid motion trajectory point. The larger the value, the smaller the deformation that occurs during respiration, and the greater the probability that it moves with the overall lung tissue, i.e., rigid motion. This provides greater motion reference value for subsequent analysis and construction of a rigid motion model.

[0036] Preferably, in one embodiment of the present invention, to facilitate the analysis of local morphological differences between matching feature points in each motion trajectory feature point set, each feature point can first be used as the center, and a voxel block can be constructed in the three-dimensional CT image to which each feature point belongs. The voxel block contains several voxels, representing the three-dimensional morphology of local lung tissue. Considering that mutual information can effectively measure the consistency of gray-level distribution between two voxel blocks and is not affected by linear changes in brightness, and that feature points with high motion stability should always have highly similar gray-level textures in different respiratory phases, i.e., high mean mutual information value and small fluctuation, motion stability parameters can be determined based on the distribution of mutual information values. The method for obtaining motion stability parameters includes: For each set of motion trajectory points, a voxel block of a preset size is determined in the 3D CT image corresponding to each feature point, with the voxel corresponding to each feature point as the center; motion stability parameters are obtained based on the distribution characteristics of the mutual information values ​​between the voxel blocks corresponding to the feature points that match each set of motion trajectory points.

[0037] Specifically, taking any set of motion trajectory points as an example, the preset size is set to 7×7×7, in mm; the implementer can also adjust it themselves; taking each feature point in the set of motion trajectory points as the center, the corresponding voxel block is determined in the 3D CT image to which each feature point belongs, and the voxel block contains 7×7×7 voxels; then the mean value of the mutual information between different voxel blocks corresponding to different feature points in the set of motion trajectory points is calculated, and the variance of the mutual information between different voxel blocks is calculated; The larger the mean, the more similar the morphological features of the voxel blocks corresponding to different feature points; the smaller the variance, the smaller the variation of the mutual information value and the more balanced the distribution; the variance is mapped to the exponential function exp(-x) with the natural constant e as the base, the smaller the variance, the larger the value of the exponential function; then the exponential function value is multiplied by the mean, and the product is normalized to obtain the motion stability parameters.

[0038] It should be noted that the normalization method used in this embodiment is linear normalization. Linear normalization is performed in the dimension of the product corresponding to all motion trajectory point sets. Its calculation with mutual information value is a well-known technical means and will not be described in detail here. In other embodiments, the implementer may also use other normalization methods, or use texture information such as the grayscale mean of voxel blocks to replace mutual information value.

[0039] The preset stability threshold is set to 0.7, which can also be adjusted by the implementer. The set of motion trajectory points with motion stability parameters greater than 0.7 is taken as the stable motion trajectory point set. The stable motion trajectory point set represents a series of feature points that undergo stable rigid motion with the lung tissue as a whole. For ease of subsequent analysis and description, the feature points in the stable motion trajectory point set are taken as stable points.

[0040] Step S202: Between temporally adjacent 3D CT images, using motion stability parameters as weights, and based on weighted least squares and the position coordinate differences between all matched stable points, a rigid transformation sub-model is fitted. The rigid transformation sub-model includes a rotation matrix and a translation vector. By combining the rigid transformation sub-models between all temporally adjacent 3D CT images, a rigid transformation model for lung motion is determined.

[0041] Considering that a set of stable motion trajectory points with larger motion stability parameters provides greater motion reference value for subsequent analysis and construction of rigid motion models; and that estimating rigid transformation based on the least squares method can help find the transformation rules between matching stable points between temporally adjacent 3D CT images; Based on this, a rigid transformation sub-model can be fitted between temporally adjacent 3D CT images, using motion stability parameters as weights, and based on weighted least squares and the position coordinate differences between all matched stable points. The rigid transformation sub-model includes a rotation matrix and a translation vector, which are used to characterize the transformation rules between matched stable points between temporally adjacent 3D CT images.

[0042] It should be noted that the fitting method of the rigid transformation sub-model between temporally adjacent three-dimensional CT images is the same. In one embodiment of the present invention, the rigid transformation sub-model between the first three-dimensional CT image and the second three-dimensional CT image in the time sequence is used as an example for analysis and description, and will not be repeated one by one.

[0043] Fitting a rigid transformation sub-model based on weighted least squares and the positional coordinate differences between all matched stable points between the first and second 3D CT images is a well-known technique. The general process is briefly described here: (1) Take all stable motion trajectory points as the source point set P (the set of points to be transformed) that belong to the stable points in the first 3D CT image: { }; Set all stable motion trajectory points into the stable points belonging to the second 3D CT image as the target point set Q (transformation target): { }; where D is the total number of points in the stable motion trajectory set; This represents a stable point in the first 3D CT image that belongs to the D-th stable motion trajectory point set; This represents a stable point in the second 3D CT image that belongs to the D-th stable motion trajectory point set; (2) The goal of fitting the rigid transformation sub-model based on the weighted least squares method is to find a rotation matrix R and a translation vector T such that the overall weighted difference (i.e., the weighted mean square error) between the transformed source point set P and the target point set Q is minimized; then the initial rigid transformation sub-model can be determined. ;in, This is the initial rigid transformation sub-model between the first and second 3D CT images; This is the rotation matrix in the initial rigid transformation sub-model. This refers to the translation vector in the initial rigid transformation sub-model; (3) Obtain the initial rigid transformation sub-model Then, further model optimization was performed: based on the initial rigid transformation sub-model. Calculate the weighted mean square error between each pair of matched stable points between the first and second 3D CT images. Matched stable points with a weighted mean square error greater than or equal to a preset error threshold are identified as outliers and removed. The preset error threshold is the median of the weighted mean square errors between all matched stable points. Using the remaining matched stable points with a weighted mean square error less than the preset error threshold, repeat step (2) to refit the rigid transformation sub-model. Iterate continuously until no outliers can be removed, at which point the fitting is considered converged, and the final rigid transformation sub-model is obtained. .

[0044] Sequentially obtain rigid transformation sub-models between all temporally adjacent 3D CT images. The temporal sorting sequence of all rigid transformation sub-models is taken as the rigid transformation model.

[0045] Step S3: Randomly determine marker points within a preset range of the tumor region; for any marker point, determine the tracking position of the marker point in each non-first 3D CT image based on the tracking matching algorithm, and determine the matching marker point of the marker point in each non-first 3D CT image based on the rigid transformation model.

[0046] Lung tissue, especially the tumor itself, undergoes complex local elastic deformation due to respiration, which cannot be described by rigid transformation models. Considering that by randomly determining marker points within a local area of ​​the tumor region that has already been marked in the first 3D CT image, it is possible to analyze the additional motion of the marker points within the tumor region in addition to rigid motion based on the rigid transformation model, thereby helping to assess the complex motion characteristics of the tumor region.

[0047] Preferably, in one embodiment of the present invention, the preset region includes the interior of the tumor region, the region boundary, and the preset periphery to ensure coverage of the tumor and its surrounding key deformation areas; within the preset region, marker points are randomly generated at a preset density.

[0048] In this embodiment, the preset region is determined as follows: the Euclidean distance between the centroid of the tumor region and the farthest voxel on the tumor region is used as the radius, and a sphere is constructed with the centroid as the center of the sphere. The sphere is used as the preset region of the tumor region, and the region outside the tumor region in the preset region is the preset periphery.

[0049] In other embodiments, the implementer may also expand the tumor region by a preset number of layers, such as 3 times, with the thickness of each layer being the width of one voxel, and use the expanded region as the preset periphery; and use the expanded tumor region as the preset region.

[0050] After determining the preset area, marker points are randomly generated at a preset density within the preset area using a uniform random distribution method. This is an existing technical method and will not be described in detail here. The preset density is 10-50 marker points per cubic centimeter. The preset density can be adjusted according to the voxel size and tumor size. In this embodiment, 30 marker points per cubic centimeter are used.

[0051] Furthermore, considering that the tracking matching algorithm can help determine the tracking position of the marker in each non-first 3D CT image, the tracking position can reflect the true position of the marker after motion (including rigid motion and other complex motions); while the rigid transformation model can help determine the matching marker in each non-first 3D CT image, the position of the matching marker reflects the position of the marker after rigid motion; the deviation between the above two positions can help assess the additional motion of the marker in the tumor area in addition to rigid motion, thereby helping to assess the complex motion characteristics of the tumor area.

[0052] Specifically, the tracking and matching algorithm uses the 3D-SIFT algorithm. Implementers may also choose other three-dimensional tracking algorithms such as the three-dimensional optical flow method, which are all well-known technical means and will not be elaborated here. The tracking point of each marker point in the first three-dimensional CT image is determined in each non-first three-dimensional CT image, and the position of the tracking point is taken as its tracking position in the non-first three-dimensional CT image.

[0053] Further, based on the rotation matrix R and translation vector T in the rigid transformation model between adjacent 3D CT images obtained in step S2, a rigid transformation is performed on each marker point in the first 3D CT image (a well-known technique, which will not be elaborated here), and its position point after rigid transformation in each non-first 3D CT image is obtained. This position point is then used as the matching marker point.

[0054] Step S4: In each non-first 3D CT image, based on the deviation of the marker point's position from the tracking position, obtain the deformation vector of the corresponding marker point. Based on the spatial distance between different marker points and the difference between the corresponding deformation vectors, select stable marker points from all marker points and correct the deformation vectors of unstable marker points.

[0055] For any marker point in the first 3D CT image, after obtaining its tracking position and matching marker point in each non-first 3D CT image, the difference between the position of the matching marker point and the tracking position can be further analyzed to obtain the deformation vector of the (matching) marker point in each non-first 3D CT image. The deformation vector reflects the additional motion of the marker point in addition to rigid motion, including the local complex deformation of tumor tissue, and may also include error motion caused by noise.

[0056] Preferably, in one embodiment of the present invention, for ease of analysis and description, firstly, any marker point within a preset range of the tumor region is taken as the target marker point, and the matching marker point determined based on the rigid transformation model in each non-first three-dimensional CT image is taken as the predicted marker point; then, in each non-first three-dimensional CT image, the vector of the coordinates of the predicted marker point pointing to the coordinates of the tracking position corresponding to the target marker point is taken as the deformation vector of the reference marker point.

[0057] Considering that the deformation vector may also include motion errors caused by noise, and considering that real physiological activities act on a continuous biomechanical structure, which will cause adjacent tissues to produce coordinated and similar displacements, the motion patterns of adjacent tissues in the lung tumor region are similar, and the corresponding deformation vector differences should also be small; however, noise is random, and the direction and magnitude of deformation caused by noise at different locations are random and unrelated, which may lead to large differences between deformation vectors. Based on this, in order to accurately capture the local deformation of the tumor area and eliminate noise interference, the embodiments of the present invention further select stable markers from all markers in each non-first three-dimensional CT image according to the spatial distance between different markers and the difference between the corresponding deformation vectors. Stable markers are those that are less affected by noise interference and can reflect the true local deformation of the lung tumor.

[0058] Preferably, in one embodiment of the present invention, considering that the marker points in different three-dimensional CT images are spatially discrete, transforming the discrete marker points into a continuous mechanical network can help simulate the physically interconnected and mutually constraining characteristics of lung or tumor tissue, so that the movement between adjacent marker points is no longer isolated, in order to evaluate the differences between deformation vectors and thus screen out stable marker points with consistent deformation; then the method for obtaining stable marker points includes: In each 3D lung CT image, marker points are used as grid points to construct a marker point network; and between each connected marker point in each grid, deformation consistency weights between connected marker points are obtained based on the negative correlation mapping results of spatial distance and the negative correlation mapping results of the differences between corresponding deformation vectors. The deformation consistency weight is used to weight the cosine of the angle between the deformation vectors corresponding to the connected marker points, and the weighted average result of all connected marker points in the grid is used as the deformation consistency parameter; the grid with the deformation consistency parameter greater than the preset threshold is used as the stable grid, and the marker points corresponding to the grid points in the stable grid are used as stable marker points.

[0059] Specifically, in each three-dimensional lung CT image, the marker points are first used as grid points to construct a marker point network. In a preferred embodiment of the present invention, the marker point network is constructed based on the three-dimensional Delaunay triangulation algorithm. Implementers may also choose other network construction methods, which are well-known technical means and will not be described in detail here. The grid in the marker point network is a triangle. Then, within each triangular grid, the spatial distance between connected marker points (adjacent triangle vertices) is negatively correlated and mapped as its reciprocal. Since the spatial distance cannot be zero, the reciprocal is meaningful. The difference between the deformation vectors corresponding to the connected marker points, such as the Euclidean norm, is mapped to an exponential function exp(-x) with the natural constant e as the base. The reciprocal is multiplied by the exponential function value to obtain the deformation consistency weight between the connected marker points. The smaller the spatial distance, the closer the marker points are, and the more consistent the deformation should be. The smaller the difference in deformation vectors, the more consistent the change pattern, and the greater the deformation consistency weight of the connected marker points. Furthermore, within each triangular mesh, the consistency of deformation is measured by the cosine of the angle between the deformation vectors corresponding to connected marker points, and this cosine value is mapped to... The value range is adjusted to 0-1 for normalization, and the normalized value of the cosine of the angle between the deformation vectors corresponding to the connected marker points is weighted using deformation consistency weight. The weighted average result of all connected marker points is used as the deformation consistency parameter. The deformation consistency parameter reflects the deformation consistency characteristics of all marker points in the grid. Since the deformation consistency parameter ranges from 0 to 1, and the preset threshold is set to 0.65, the implementer can also adjust it themselves. Grids with deformation consistency parameters greater than the preset threshold are considered stable grids, and the grid points in the stable grids are considered as the corresponding marker points.

[0060] It should be noted that the same grid point may participate in multiple deformation consistency analyses in different grids. If the corresponding grid is identified as an unstable grid in any deformation consistency analysis, it will be marked as an unstable marker point.

[0061] Once all stable markers are identified, the deformation vectors of unstable markers can be further modified to eliminate noise interference, thus preparing for subsequent comprehensive assessment of local deformation motion of markers in lung tumors, excluding rigid motion, based on all markers.

[0062] Preferably, in one embodiment of the present invention, considering that for each unstable marker point, the deformation vector of its surrounding stable marker points can provide a certain proximity reference for analyzing its deformation, and the local deformation pattern of stable marker points that are closer in spatial distance is more similar to that of stable marker points, the reference value can be evaluated based on spatial distance, and then the deformation vector can be comprehensively corrected; therefore, correcting the deformation vector of the unstable marker point includes: For each unstable marker point, a preset number of its nearest stable marker points are selected as reference marker points, and the negative correlation mapping result of the spatial distance between the unstable marker point and each reference marker point is used as the reference weight. The deformation vector of each reference marker point is weighted and averaged using the reference weight, and the weighted average result is used as the corrected deformation vector of the unstable marker point.

[0063] Specifically, the preset number is set to 8, which can be adjusted by the implementer. For each unstable marker point, the 8 nearest stable marker points are selected as reference marker points. The spatial distance between the unstable marker point and each reference marker point is negatively correlated and mapped as an inverse, and the inverse is used as the reference weight for the reference marker point. Then, the deformation vector of each reference marker point is weighted and averaged using the reference weight to obtain the deformation vector of the unstable marker point after correction.

[0064] Thus, by removing noise interference from the deformation vectors of all markers in non-first 3D CT images, the true motion of the tumor tissue, excluding rigid motion, is reflected, providing a pattern reference for the complex motion of the tumor tissue and preparing for subsequent accurate assessment of the marker positions.

[0065] Step S5: Correct the position of the marker points based on the deformation vector of the marker points in the non-first 3D CT images; input each 3D CT image and the corrected position of the marker points within it into the trained segmentation model to obtain the lung tumor segmentation result.

[0066] Considering that the position of the marker point in the non-first 3D CT image is determined based on the rigid transformation model, local deformation can be superimposed on the rigid motion to correct and adjust the position of the marker point determined based on the rigid transformation model in each non-first 3D CT image. After obtaining the deformation vector (deformation vector of stable marker point and correction deformation vector of unstable marker point) of each marker point in the non-first 3D CT image, the position of the marker point can be directly corrected based on the deformation vector.

[0067] Preferably, in one embodiment of the present invention, considering that stable and unstable marker points need to be discussed separately, the method for obtaining the position of the corrected marker point includes: In each non-first 3D CT image, the position coordinates of each stable marker point are added to the deformation vector to obtain the corrected position of the stable marker point; the position coordinates of each unstable marker point are added to the corrected deformation vector to obtain the corrected position of the unstable marker point.

[0068] It should be noted that the above-mentioned calculation and acquisition of the corrected position is an existing technical method, and will not be elaborated further.

[0069] In another embodiment of the present invention, the motion trajectory of each marker point during the respiratory cycle can be evaluated based on the position of all marker points in the first three-dimensional CT image and the corrected position in the remaining non-first three-dimensional CT images (containing only rigid motion and local deformation). Then, the thin plate spline transformation function is solved to construct a spatially smooth deformation model based on physical constraints. Thus, based on the known motion of all marker points, the motion trajectory of any voxel in the tumor region during the respiratory cycle can be automatically interpolated, thereby roughly estimating the tumor region in each three-dimensional CT image.

[0070] After obtaining the corrected position of the marker point in each 3D CT image, each 3D CT image and the corrected position of its marker point can be further input into the trained segmentation model, and the model will automatically output the lung tumor segmentation result.

[0071] It should be noted that the training and application of segmentation models are well-known techniques. Here is a brief description of the general application process: 3D CT images with marked points at different respiratory phases are used as input to the segmentation model. The marked points can initially provide a rough mask of the tumor region in the 3D CT images at different respiratory phases, enabling the segmentation model to focus on the tumor region, effectively suppressing the interference caused by respiratory motion, and improving the accuracy and robustness of segmentation. This segmentation model employs an encoder-decoder structure, such as a variant of U-Net. The encoder extracts multi-scale features, and the decoder gradually restores spatial details and outputs fine-grained segmentation results. During training, a loss function combining Dice loss and cross-entropy loss is used to optimize boundary fit and class determination.

[0072] In summary, this invention acquires three-dimensional CT images of a patient's lungs at different respiratory phases, as well as the tumor region in the first three-dimensional CT image; acquires all motion trajectory point sets; obtains a rigid transformation model of lung motion based on the positional changes of matching feature points in each motion trajectory point set and the differences between corresponding voxel information; randomly determines marker points within a preset range of the tumor region; for any marker point, determines the tracking position of the marker point in each non-first three-dimensional CT image based on a tracking matching algorithm, and determines the matching marker point in each non-first three-dimensional CT image based on the rigid transformation model; in each non-first three-dimensional CT image, obtains the deformation vector of the corresponding marker point based on the deviation of the marker point's position from the tracking position; selects stable marker points from all marker points based on the spatial distance between different marker points and the differences between corresponding deformation vectors, and corrects the deformation vectors of unstable marker points; corrects the position of the marker points based on the deformation vectors of the marker points in the non-first three-dimensional CT images; and inputs each three-dimensional CT image and the corrected position of the marker points within it into a trained segmentation model to obtain lung tumor segmentation results. This invention analyzes the respiratory motion characteristics of lung tissue at different respiratory phases, decomposes respiratory motion into rigid motion and local elastic deformation, and eliminates respiratory motion deviations caused by noise. This allows for the assessment of the motion changes of marker points within the tumor region at different respiratory phases, and then performs automatic segmentation based on a deep learning segmentation model. This effectively overcomes the problem of changes in tumor location and morphology caused by respiration, and improves the segmentation effect of lung tumor CT images based on deep learning.

[0073] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for automatic segmentation of lung tumor CT images based on deep learning, characterized in that, The method includes: Acquire three-dimensional CT images of the patient's lungs at different respiratory phases, as well as the tumor region in the first three-dimensional CT image; Obtain all motion trajectory point sets, each containing all matching feature points between different 3D CT images; based on the positional changes of the matching feature points in each motion trajectory point set and the differences between the corresponding voxel information, obtain a rigid transformation model of lung motion. Marker points are randomly determined within a preset range of the tumor region; for any marker point, the tracking position of the marker point in each non-first 3D CT image is determined based on the tracking matching algorithm, and the matching marker point of the marker point in each non-first 3D CT image is determined based on the rigid transformation model. In each non-first 3D CT image, the deformation vector of the corresponding marker point is obtained based on the deviation of the marker point's position from the tracking position. Stable marker points are selected from all marker points based on the spatial distance between different marker points and the difference between the corresponding deformation vectors, and the deformation vectors of unstable marker points are corrected. The positions of the markers are corrected based on the deformation vectors of the markers in the non-first 3D CT images; each 3D CT image and the corrected positions of the markers within it are input into the trained segmentation model to obtain the lung tumor segmentation results.

2. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 1, characterized in that, The method for obtaining the rigid transformation model includes: Based on the differences between voxel information within a preset spatial range corresponding to the feature points matched in each motion trajectory point set, motion stability parameters are obtained. Motion trajectory point sets whose motion stability parameters are greater than a preset stability threshold are taken as stable motion trajectory point sets, and feature points within stable motion trajectory point sets are taken as stable points. Between temporally adjacent 3D CT images, using the motion stability parameters as weights, a rigid transformation sub-model is fitted based on weighted least squares and the position coordinate differences between all matched stable points. The rigid transformation sub-model includes a rotation matrix and a translation vector. By combining the rigid transformation sub-models between all temporally adjacent 3D CT images, a rigid transformation model for lung motion is determined.

3. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 2, characterized in that, The method for obtaining the motion stability parameters includes: For each set of motion trajectory points, a voxel block of a preset size is determined in the 3D CT image to which each feature point belongs, with the voxel corresponding to each feature point as the center; motion stability parameters are obtained based on the distribution characteristics of the mutual information values ​​between the voxel blocks corresponding to the feature points that match each set of motion trajectory points.

4. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 1, characterized in that, Randomly select marker points within a preset range of the tumor region, including: The preset region includes the interior of the tumor region, the region boundary, and the preset periphery; within the preset region, marker points are randomly generated at a preset density.

5. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 1, characterized in that, The method for obtaining the deformation vector includes: Using any marker point within a preset range of the tumor region as the target marker point, the matching marker point determined based on the rigid transformation model in each non-first 3D CT image is used as the predicted marker point; In each non-first 3D CT image, the vector pointing from the coordinates of the predicted marker point to the coordinates of the tracking position corresponding to the target marker point is used as the deformation vector of the reference marker point.

6. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 1, characterized in that, The method for obtaining the stable marker points includes: In each three-dimensional lung CT image, marker points are used as grid points to construct a marker point network; and between each connected marker point in each grid, the deformation consistency weight between connected marker points is obtained based on the negative correlation mapping result of spatial distance and the negative correlation mapping result of the difference between the corresponding deformation vectors. The normalized cosine values ​​of the angles between the deformation vectors corresponding to the connected marker points are weighted using the deformation consistency weight. The weighted average of all connected marker points in the grid is used as the deformation consistency parameter. Grids with deformation consistency parameters greater than a preset threshold are designated as stable grids, and the marker points corresponding to the grid points in the stable grids are designated as stable marker points.

7. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 1, characterized in that, Correcting the deformation vector of unstable marker points includes: For each unstable marker point, a preset number of its nearest stable marker points are selected as reference marker points, and the negative correlation mapping result of the spatial distance between the unstable marker point and each reference marker point is used as the reference weight. The deformation vector of each reference marker point is weighted and averaged using the reference weights, and the weighted average result is used as the deformation vector after correction for the unstable marker point.

8. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 1, characterized in that, The method for obtaining the position of the correction marker includes: In each non-first 3D CT image, the position coordinates of each stable marker point are added to the deformation vector to obtain the corrected position of the stable marker point; the position coordinates of each unstable marker point are added to the corrected deformation vector to obtain the corrected position of the unstable marker point.

9. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 1, characterized in that, The method for obtaining the set of motion trajectory points includes: Based on the 3D scale-invariant feature transformation algorithm, all feature points in each 3D CT image are obtained, and feature points between different 3D CT images are tracked and matched. The set of matching feature points between different 3D CT images is used as the motion trajectory point set.

10. The method for automatic segmentation of lung tumor CT images based on deep learning according to claim 6, characterized in that, The method for obtaining the marker network includes: A marker point network is constructed based on the 3D Delaunay triangulation algorithm.

Images