Patient condition tracking method and system

Abstract

The present invention discloses a patient condition tracking method and system, and relates to the technical field of medical information processing, including collecting patient lesion image data, rigidly registering the lesion image data through affine transformation; predicting lesion probability distribution based on deep convolutional neural network, and performing lesion segmentation and marking based on conditional random field CRF. The method of the present invention improves the accuracy of lesion area segmentation through the combination of deep convolutional neural network and conditional random field, reduces human intervention, and can quickly achieve global alignment of images through the combination of rigid registration and model recognition, so that the changes in the lesion area can be tracked in detail, and the displacement of pixels can be smoothly interpolated between control points through the non-rigid registration of B-spline basis function, and the generated image can accurately align the morphology and position of the lesion area, which provides reliable basic data for subsequent lesion change analysis, morphological change quantification and other work.

Description

Technical Field

[0001] The present invention relates to the technical field of medical information processing, and in particular to a method and system for tracking a patient's condition. Background Art

[0002] With the rapid development of medical imaging technology, the acquisition and analysis of medical images play a vital role in clinical diagnosis, treatment plan formulation and efficacy evaluation. Traditional medical imaging equipment, such as CT, MRI and PET-CT, can provide detailed anatomical structure and functional information, allowing doctors to observe the morphology, location and size of lesions. Existing lesion tracking methods mostly rely entirely on manual analysis of images, which is not only time-consuming and labor-intensive, but also easily affected by human factors. In order to improve the degree of automation of image analysis, a variety of automated image analysis methods based on machine learning and deep learning have emerged in recent years. In particular, lesion recognition and segmentation technologies based on algorithms such as convolutional neural networks (CNNs) have made significant progress;

[0003] However, the existing technology still has shortcomings in several key aspects. First, traditional image registration methods usually rely on global rigid transformations. Although they work well in global image alignment, they cannot cope with local morphological changes in the lesion area, especially in tracking special lesions and lesions that change greatly over time. Secondly, although deep convolutional neural networks have been widely used in lesion segmentation, relying solely on CNN for segmentation may still face problems such as blurred boundaries and high misclassification rates. In particular, under different imaging conditions or noise interference, the segmentation accuracy of the lesion area is difficult to guarantee, and it is difficult to accurately reflect the dynamic change trend of the lesion. Especially in long-term tracking, it is difficult to effectively predict the volume growth rate and change trend of the lesion. Summary of the invention

[0004] In view of the problems existing in the above-mentioned existing patient condition tracking methods and systems, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that traditional image registration methods usually rely on global rigid transformation. Although they work well in global image alignment, they cannot cope with local morphological changes in the lesion area, especially in tracking lesions that change greatly over time, such as special lesions. Secondly, although deep convolutional neural networks have been widely used in lesion segmentation, relying solely on CNN for segmentation may still face problems such as blurred boundaries and high misclassification rates. In particular, under different imaging conditions or noise interference, the segmentation accuracy of the lesion area is difficult to guarantee, and it is difficult to accurately reflect the dynamic change trend of the lesion, especially in long-term tracking, it is difficult to effectively predict the volume growth rate and change trend of the lesion.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for tracking a patient's condition, comprising:

[0007] Collect the patient's lesion image data, and perform rigid registration on the lesion image data through affine transformation;

[0008] Predict lesion probability distribution based on deep convolutional neural network, and segment and mark lesions based on conditional random field (CRF);

[0009] The B-spline basis function is used to describe the local displacement to perform non-rigid registration on the image data of the lesion area to generate the final image;

[0010] The lesion area was selected for volume calculation, and the lesion volume growth rate and change rate were determined using nonlinear differential equations.

[0011] As a preferred solution of the patient condition tracking method of the present invention, the collecting of patient lesion image data includes:

[0012] Collect image data according to the patient's specific condition needs;

[0013] Determine the frequency of image data collection based on physician experience;

[0014] Preprocessing is performed based on the image data, including image data denoising, grayscale standardization, and image cropping.

[0015] As a preferred solution of the patient condition tracking method of the present invention, the rigid registration of the lesion image data by affine transformation includes:

[0016] Based on the preprocessed image data, the image data of multiple time points are preliminarily aligned through affine transformation, and the affine transformation matrix is ​​defined. Through matrix multiplication and vector addition operations, the new position of each pixel is calculated for rigid registration, which is expressed as: in Indicates the changed coordinates of the pixel coordinate point. Represents the pixel coordinates of the original image, and b are the affine transformation matrix and displacement vector respectively;

[0017] Determine the reference image based on the training set, select the mean square error MSE as the similarity measurement function, the goal is to minimize the difference between the target image and the reference image, use the Adam optimizer and the gradient descent method to perform iterative optimization, and stop the iteration when the loss calculated in the continuous iteration process no longer decreases significantly, determine the optimal A and b, and output the optimal affine transformation matrix;

[0018] Through rigid registration, the coordinates of each pixel are transformed to the new position coordinates , generate aligned images.

[0019] As a preferred solution of the patient condition tracking method of the present invention, the prediction of lesion probability distribution based on deep convolutional neural network and the segmentation and marking of lesions based on conditional random field CRF include:

[0020] A lesion area model is constructed based on a deep convolutional neural network, including an input layer, a convolutional layer, a maximum pooling layer, a fully connected layer, and an output layer;

[0021] Use the training set to train the model, and select the cross entropy loss function to calculate the difference between the model prediction result and the actual label of the lesion area. Use the Adam optimizer to perform gradient descent optimization and update the model parameters. If the model loss no longer decreases significantly during the continuous iteration process, stop iterating and output the model.

[0022] A lesion probability distribution map based on the aligned image is obtained based on the lesion region segmentation prediction result output by the lesion region model;

[0023] Based on the conditional random field CRF, the probability of each pixel belonging to the lesion area is taken as a single-point term, the similarity between pixels is defined as a double-point term, and the energy function is defined based on the single-point term and the double-point term, which is expressed as: ; ; ;in Indicates the cost of the i-th pixel belonging to the lesion, represents the probability that the i-th pixel belongs to the lesion area, represents the similarity cost between the adjacent i-th pixel and j-th pixel, and Represent the spatial coordinates of the i-th pixel and the j-th pixel respectively, represents the weight coefficient, and Represent the grayscale values ​​of the i-th pixel and the j-th pixel respectively, and denote the spatial scale parameter and the color scale parameter respectively, Represents the total energy function value, N represents the total number of pixels, Represents the neighboring pixel area of ​​the i-th pixel;

[0024] Use the training set and select the cross entropy loss function to calculate the loss between the lesion prediction results of the conditional random field CRF and the actual label, use the Adam optimizer for gradient descent optimization, and update the conditional random field CRF parameters, including , and , if the calculated loss no longer decreases significantly during the continuous iteration process, the iteration is stopped, the model parameters are output, and the model is updated;

[0025] The mean-field approximation method is used to optimize the energy function and determine the optimization formula, which is expressed as: ;in represents the normalization factor, represents the classification probability, Represents the classification probability of adjacent pixels;

[0026] The predicted probability map of the lesion area model is used as the initial ,Sure ;

[0027] based on The probability distribution of each pixel is updated in the continuous iteration process. If the change in the value of is no longer significantly reduced, the iteration is stopped and the final output of the model is ;

[0028] Based on the final Generate the corresponding lesion area probability map, segment the pixel values ​​of the probability map based on the historical average value as the segmentation threshold, and mark the lesion area.

[0029] As a preferred solution of the patient condition tracking method of the present invention, the method of using the B-spline basis function to describe the local displacement to perform non-rigid registration on the image data of the lesion area to generate the final image includes:

[0030] Generate a grid based on the lesion area of ​​the image data, define the grid nodes as control points, and perform non-rigid registration;

[0031] Based on each control point, the B-spline basis function is used to describe the local displacement, which is expressed as: ; ;in represents the displacement of pixel point x, n represents the number of control points adjacent to the pixel point, represents the displacement vector of control point i, represents the B-spline basis function, represents the coordinates of the i-th control point, represents the third-order B-spline basis function;

[0032] The reference image is determined based on the training set, and the mean square error (MSE) is selected as the similarity measurement function. The goal is to minimize the difference between the target image and the reference image, which is expressed as: ;in represents the non-rigid registration calculation error, Represents the total number of pixels. represents the coordinates of the i-th pixel, is the corresponding pixel coordinate in the reference image, M represents the total number of control points, represents the smoothness regularization parameter, which is determined based on historical data, represents the displacement vector gradient of control point j, Represents pixel The displacement of

[0033] Use the Levenberg-Marquardt optimization algorithm to calculate the displacement of the current control point , calculate the value of the non-rigid registration calculation error, and calculate the displacement of each control point by the objective function The gradient of the control point is then updated. When the calculated error no longer decreases significantly during the continuous optimization process, the iteration is stopped and the displacement of the control point is optimized. ;

[0034] Calculate pixel displacement based on control point displacement through non-rigid registration , and further adjust the pixel positions based on the aligned images to generate the final image.

[0035] As a preferred solution of the patient condition tracking method of the present invention, wherein: the step of selecting the lesion area for volume calculation includes:

[0036] Based on the image scanning equipment, three-dimensional image data is formed by stacking multiple layers of image data. Based on the final image generated after non-rigid registration, the lesion change area is determined and the voxels in the lesion change area are selected for volume calculation.

[0037] As a preferred solution of the patient condition tracking method of the present invention, the method of using a nonlinear differential equation to determine the lesion volume growth rate and change rate includes:

[0038] The volume change data of the lesion is fitted using a nonlinear differential equation, expressed as: ;in represents the lesion volume at time t, represents the growth rate of the lesion volume, Indicates the maximum limit value of the lesion volume;

[0039] Using historical data and using the squared error as the loss function, the difference between the observed volume and the predicted volume is minimized;

[0040] The loss is calculated iteratively using the nonlinear least squares method. If the model loss no longer decreases significantly during the continuous iteration, the iteration is stopped and the final K and ;

[0041] The rate of change of the growth rate was determined based on the change in the lesion volume growth rate at different times, and the rate of change was evaluated based on the physician's experience.

[0042] Another object of the present invention is to provide a system for a patient's condition tracking method, which comprises:

[0043] An image data processing module is used to collect the patient's lesion image data and perform image preprocessing;

[0044] Rigid registration module, used for rigid registration through affine transformation;

[0045] Lesion recognition and segmentation module, which is used to predict the probability distribution of lesion areas based on deep convolutional neural networks and use conditional random fields to segment and mark lesions;

[0046] The non-rigid registration module is used to describe the local displacement through the B-spline basis function and perform non-rigid registration;

[0047] A lesion volume calculation module is used to select the lesion area for volume calculation based on the image data after non-rigid registration;

[0048] The volume change analysis module is used to determine the volume growth rate and change rate of the lesion using nonlinear differential equations.

[0049] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned patient condition tracking method when executing the computer program.

[0050] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned patient condition tracking method when executed by a processor.

[0051] The beneficial effects of the present invention are as follows: through the combination of deep convolutional neural networks and conditional random fields, the accuracy of lesion area segmentation is improved, and human intervention is reduced. Through the combination of rigid registration and model recognition, global alignment of images can be quickly achieved, so that changes in the lesion area can be tracked in detail. Through the non-rigid registration of the B-spline basis function, the displacement of pixels can be smoothly interpolated between control points, and the generated image can accurately align the morphology and position of the lesion area, providing reliable basic data for subsequent lesion change analysis, morphological change quantification and other work. The volume calculation by voxel can provide precise and reliable lesion volume measurement, especially for complex lesion morphology or fuzzy lesion edges. By using historical lesion volume data and fitting the model parameters by nonlinear least squares method, the volume change of the lesion in the future can be accurately predicted, which can help doctors predict the development trend of the lesion. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0053] Figure 1 Flowchart of the patient condition tracking method.

[0054] Figure 2 This is a schematic diagram of the patient condition tracking system. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0058] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and provides a method for tracking a patient's condition, the method comprising:

[0059] S1, collecting the patient's lesion image data, and performing rigid registration on the lesion image data through affine transformation;

[0060] Preferably, collecting patient lesion image data includes:

[0061] Image data collection is performed according to the patient's specific condition needs. Image data collection can be performed using imaging examinations such as CT, MRI, PET-CT or ultrasound. CT is used for hard tissue imaging, MRI is used for soft tissue imaging, PET-CT is suitable for metabolic imaging, and ultrasound is suitable for real-time dynamic imaging;

[0062] Determine the frequency of image data collection based on physician experience;

[0063] Preprocessing is performed based on the image data, including image data denoising, grayscale standardization, and image cropping.

[0064] By determining a reasonable frequency of image data acquisition based on the patient's specific condition and historical data on disease progression, we can avoid excessive repeated examinations, reduce the burden and radiation exposure of patients, and reduce acquisition costs. The denoising process can significantly improve image clarity and remove random errors caused by noise in imaging equipment (such as artifacts in MRI, radiation noise in CT, etc.). The denoised image can more clearly show the boundaries and details of the lesion. Grayscale standardization can standardize the grayscale values ​​of images from different devices or at different time points to a unified range, making the images consistent. Image cropping can remove irrelevant areas in the image and reduce data redundancy.

[0065] Furthermore, the lesion image data is rigidly registered through affine transformation, including:

[0066] Based on the preprocessed image data, the image data of multiple time points are preliminarily aligned through affine transformation, and the affine transformation matrix is ​​defined. Through matrix multiplication and vector addition operations, the new position of each pixel is calculated for rigid registration, which is expressed as: in Indicates the changed coordinates of the pixel coordinate point. Represents the pixel coordinates of the original image, and b are the affine transformation matrix and displacement vector, respectively, expressed as: ; ; Where A is a 3*3 affine transformation matrix used to describe rotation and scaling, and each matrix element Controls that are clearly defined as rotation, scaling, or shearing operations, using image metadata (such as image pose information) or image analysis tools to obtain the image's rotation angle on each axis, and obtaining the scaling factor by comparing the image's pixel resolution. , and The displacement values ​​along the x, y and z axes in the displacement vector are determined by image analysis tools (such as the automatic feature point detection algorithm SURF) to determine the position difference of the same feature point at different time points, thereby calculating the translation vector;

[0067] The reference image is determined based on the training set, and the mean square error (MSE) is selected as the similarity measurement function. The goal is to minimize the difference between the target image and the reference image, which is expressed as: ;in represents the calculation error, Represents the total number of pixels. represents the coordinates of the i-th pixel, is the corresponding pixel coordinate in the reference image;

[0068] Iterate and optimize using the Adam optimizer and gradient descent method. When the calculated loss no longer decreases significantly during the continuous iteration, the iteration is stopped, the optimal A and b are determined, and the optimal affine transformation matrix is ​​output;

[0069] Through rigid registration, the coordinates of each pixel are transformed to the new position coordinates , generate aligned images.

[0070] Through rigid registration based on affine transformation, slight differences in patient posture during medical imaging can be compensated, so that image data at different time points can be spatially aligned, eliminating errors caused by position and angle changes. Subsequent analysis can focus on changes in the lesion itself without being affected by deviations during image acquisition, providing a reliable data basis for subsequent predictions of lesion volume changes and morphological change trends.

[0071] S2, predicts the probability distribution of lesions based on deep convolutional neural networks, and performs lesion segmentation and labeling based on conditional random fields (CRF);

[0072] Preferably, predicting the probability distribution of lesions based on a deep convolutional neural network and performing lesion segmentation and labeling based on a conditional random field (CRF) include:

[0073] A lesion area model is constructed based on a deep convolutional neural network, including an input layer, a convolutional layer, a maximum pooling layer, a fully connected layer, and an output layer;

[0074] The input layer inputs the aligned image after rigid adaptation, the convolution layer extracts local features by scanning the local area of ​​the image, the maximum pooling layer reduces the size of the feature map and highlights the significance of the features, and the fully connected layer maps the features to the output layer, which outputs the predicted results of the lesion area segmentation;

[0075] Use the training set to train the model, and select the cross entropy loss function to calculate the difference between the model prediction result and the actual label of the lesion area. Use the Adam optimizer to perform gradient descent optimization and update the model parameters. If the model loss no longer decreases significantly during the continuous iteration process, stop iterating and output the model.

[0076] A lesion probability distribution map based on the aligned image is obtained based on the lesion region segmentation prediction result output by the lesion region model;

[0077] Based on the conditional random field CRF, the probability of each pixel belonging to the lesion area is taken as a single-point term, the similarity between pixels is defined as a double-point term, and the energy function is defined based on the single-point term and the double-point term, which is expressed as: ; ; ;in Indicates the cost of the i-th pixel belonging to the lesion, represents the probability that the i-th pixel belongs to the lesion area, represents the similarity cost between the adjacent i-th pixel and j-th pixel, and Represent the spatial coordinates of the i-th pixel and the j-th pixel respectively, represents the weight coefficient, and Represent the grayscale values ​​of the i-th pixel and the j-th pixel respectively, and denote the spatial scale parameter and the color scale parameter respectively, Represents the total energy function value, N represents the total number of pixels, represents the neighboring pixel area of ​​the i-th pixel, represents the Euclidean distance between the i-th pixel and the j-th pixel;

[0078] Use the training set and select the cross entropy loss function to calculate the loss between the lesion prediction results of the conditional random field CRF and the actual label, use the Adam optimizer for gradient descent optimization, and update the conditional random field CRF parameters, including , and , if the calculated loss no longer decreases significantly during the continuous iteration process, the iteration is stopped, the model parameters are output, and the model is updated;

[0079] The mean-field approximation method is used to optimize the energy function and determine the optimization formula, which is expressed as: ;in represents the normalization factor, represents the classification probability, Represents the classification probability of adjacent pixels;

[0080] The predicted probability map of the lesion area model is used as the initial ,Sure ;

[0081] based on The probability distribution of each pixel is updated in the continuous iteration process. If the change in the value of is no longer significantly reduced, the iteration is stopped and the final output of the model is ;

[0082] Based on the final Generate the corresponding lesion area probability map, segment the pixel values ​​of the probability map based on the historical average value as the segmentation threshold, and mark the lesion area.

[0083] By using a deep convolutional neural network, local features can be extracted based on scanned images, and important information such as the edge, shape, and texture of the lesion area can be learned. The size of the feature map can be reduced through the maximum pooling layer, which reduces the computational burden while retaining the important information of the features. Through the fully connected layer and the output layer, the model can map the image features to the probability space and generate the predicted probability distribution of the lesion area, thus realizing basic recognition based on the scanned image.

[0084] Through the use of conditional random fields (CRF), the probability of lesions of each pixel and the similarity cost of adjacent pixels are modeled based on the definition of single-point items and double-point items. With the optimization of the mean field approximation method, the classification probability of each pixel is gradually updated, so that each pixel not only considers its own prediction results, but also considers the influence of adjacent pixels, making the classification of pixels more robust and effectively solving the classification error problem caused by local noise in the image. Through multiple iterative updates, the probability distribution map of the lesion area can be gradually optimized, making the segmentation result more refined and accurate;

[0085] Through the combination of deep convolutional neural networks and conditional random fields (CRF), the accuracy of lesion area segmentation is improved, human intervention is reduced, and it can adapt to complex lesion morphology. Lesions can be marked efficiently and accurately, providing a reliable basis for subsequent diagnosis and treatment planning.

[0086] S3, using B-spline basis functions to describe local displacements for non-rigid registration to generate the final image;

[0087] Preferably, a B-spline basis function is used to describe the local displacement for non-rigid registration to generate a final image, including:

[0088] Generate a grid based on the lesion area of ​​the image data, define the grid nodes as control points, and perform non-rigid registration;

[0089] Based on each control point, the B-spline basis function is used to describe the local displacement, which is expressed as: ; ;in represents the displacement of pixel point x, n represents the number of control points adjacent to the pixel point, represents the displacement vector of control point i, represents the B-spline basis function, represents the coordinates of the i-th control point, represents the third-order B-spline basis function, expressed as: ; Determine the reference image based on the training set, select the mean square error MSE as the similarity measurement function, the goal is to minimize the difference between the target image and the reference image, expressed as: ;in represents the non-rigid registration calculation error, Represents the total number of pixels. represents the coordinates of the i-th pixel, is the corresponding pixel coordinate in the reference image, M represents the total number of control points, represents the smoothness regularization parameter, which is determined based on historical data, represents the displacement vector gradient of control point j, Represents pixel The displacement of

[0090] Use the Levenberg-Marquardt optimization algorithm to calculate the displacement of the current control point , calculate the value of the non-rigid registration calculation error, and calculate the displacement of each control point by the objective function The gradient of the control point is then updated. When the calculated error no longer decreases significantly during the continuous optimization process, the iteration is stopped and the displacement of the control point is optimized. ;

[0091] Calculate pixel displacement based on control point displacement through non-rigid registration , and further adjust the pixel positions based on the aligned images to generate the final image.

[0092] By generating a grid in the lesion area and defining the grid nodes as control points, a basis is provided for non-rigid registration to accurately control the local deformation of the lesion area. The B-spline basis function can smoothly interpolate the displacement of pixels between the control points, ensuring that the change of local displacement is smooth and does not produce mutations, avoiding artificial traces or unnatural edges that may appear in the registration results. The third-order B-spline basis function allows the registration process to capture smaller morphological changes in the lesion area, such as subtle deformation of the lesion edge, so that non-rigid registration can align the boundaries of the lesion area in local details, providing higher segmentation accuracy. The Levenberg-Marquardt optimization algorithm combines the advantages of gradient descent and Newton's method to quickly and effectively find the optimal displacement vector of the control point.

[0093] The displacement of pixels is calculated by the displacement of control points to ensure that each pixel can be accurately moved to a new position, thus achieving higher-precision alignment of the lesion area. In particular, when the morphology of the lesion area changes significantly at different time points, non-rigid registration can accurately capture these changes. After global and local adjustments are made to the image based on the displacement of control points, the generated image can accurately align the morphology and position of the lesion area, providing reliable basic data for subsequent lesion change analysis, morphology change quantification, and other work, which can significantly improve the accuracy and consistency of image analysis.

[0094] The probability map of the lesion area identified by the model provides an accurate initial position of the lesion for registration, and the non-rigid registration further improves the recognition accuracy through local adjustment. This collaborative optimization cannot be achieved by a single technology, especially in the complex lesion morphological changes. The combined effect can significantly reduce the error. At the same time, the combination of rigid registration and non-rigid registration greatly improves the stability of the registration, avoiding the global instability problem that may be caused by relying solely on non-rigid registration. Through the combination of rigid registration and model recognition, the global alignment of the image can be quickly achieved, so that the changes in the lesion area can be tracked meticulously, especially for the subtle changes in the expansion or contraction of the lesion during the progression of the disease, which can be captured by non-rigid registration and provide a reliable initial position for subsequent local lesion changes, ensuring the accurate alignment and analysis of the lesion area during subtle morphological changes, assisting doctors in tracking the patient's condition, and helping doctors to dynamically track the changes in the lesion area through image data. Especially for patients with long-term treatment, doctors can analyze the effect of treatment through these image changes and make treatment adjustments accordingly. Automated image analysis greatly reduces the workload of doctors' manual annotation and analysis, and realizes more sophisticated lesion area analysis through algorithms, improving the intelligence level of image analysis.

[0095] S4, select the lesion area for volume calculation, and use nonlinear differential equations to determine the lesion volume growth rate and change rate;

[0096] Preferably, selecting the lesion area for volume calculation includes:

[0097] Based on the image scanning device, three-dimensional image data is formed by stacking multiple layers of image data. Based on the final image generated by non-rigid registration, the lesion change area is determined and the voxels in the lesion change area are selected for volume calculation, which is expressed as: ; Where V represents the volume of the lesion area, , and They represent the length, width and height of the voxel in three-dimensional space respectively, i, j and k are the indexes of the voxel respectively, representing the voxels in all lesion areas on the three-dimensional grid.

[0098] Calculating volume through voxels can provide precise and reliable lesion volume measurement, especially for complex lesion morphology or lesion with blurred edges, and can capture the true spatial distribution of the lesion. Through non-rigid registration and voxel volume calculation, its three-dimensional volume can be analyzed more accurately, and the patient's disease progression can be dynamically tracked. Especially during the treatment process, the changes in lesion volume can be regularly evaluated to help doctors make more accurate judgments.

[0099] Furthermore, nonlinear differential equations were used to determine the lesion volume growth rate and change rate, including,

[0100] The volume change data of the lesion is fitted using a nonlinear differential equation, expressed as: ;in represents the lesion volume at time t, represents the growth rate of the lesion volume, Indicates the maximum limit value of the lesion volume;

[0101] Using historical data and using the mean square error (MSE) as the loss function, the difference between the observed volume and the predicted volume is minimized;

[0102] The nonlinear least squares method is used to iteratively calculate the loss. If the model loss no longer decreases significantly during the continuous iteration, the iteration is stopped and the final K and ;

[0103] The rate of change of the growth rate was determined based on the change in the lesion volume growth rate at different times, and the rate of change was evaluated based on the physician's experience.

[0104] By utilizing historical lesion volume data and fitting the model parameters using the nonlinear least squares method, the volume changes of lesions in the future can be accurately predicted. This can help doctors predict the development trend of lesions, facilitate the formulation of intervention measures in advance, and evaluate the expansion or reduction of lesions in the future, thereby achieving early warning of the disease, reducing patients' risks, and improving prognosis.

[0105] Example 2, reference Figure 2 , which is the second embodiment of the present invention, is different from the previous embodiment and provides a system for tracking a patient's condition, including:

[0106] An image data processing module is used to collect the patient's lesion image data and perform image preprocessing;

[0107] Rigid registration module, used for rigid registration through affine transformation;

[0108] Lesion recognition and segmentation module, which is used to predict the probability distribution of lesion areas based on deep convolutional neural networks and use conditional random fields to segment and mark lesions;

[0109] The non-rigid registration module is used to describe the local displacement through the B-spline basis function and perform non-rigid registration;

[0110] A lesion volume calculation module is used to select the lesion area for volume calculation based on the image data after non-rigid registration;

[0111] The volume change analysis module is used to determine the volume growth rate and change rate of the lesion using nonlinear differential equations.

[0112] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0113] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0114] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0115] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit with a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit with a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for tracking a patient's condition, characterized in that: include: Collect the patient's lesion image data, and perform rigid registration on the lesion image data through affine transformation; Predict lesion probability distribution based on deep convolutional neural network, and segment and mark lesions based on conditional random field (CRF); The B-spline basis function is used to describe the local displacement to perform non-rigid registration on the image data of the lesion area to generate the final image; The lesion area is selected for volume calculation, and the lesion volume growth rate and change rate are determined using nonlinear differential equations. The collecting of patient lesion image data comprises: Collect image data according to the patient's specific condition needs; Determine the frequency of image data collection based on physician experience; Preprocessing is performed based on image data, including denoising, grayscale standardization, and image cropping. The rigid registration of the lesion image data by affine transformation comprises: Based on the preprocessed image data, the image data of multiple time points are preliminarily aligned through affine transformation, and the affine transformation matrix is ​​defined. Through matrix multiplication and vector addition operations, the new position of each pixel is calculated for rigid registration, which is expressed as: ； in Indicates the changed coordinates of the pixel coordinate point. Represents the pixel coordinates of the original image, and are the affine transformation matrix and displacement vector respectively; Determine the reference image based on the training set, select the mean square error MSE as the similarity measurement function, the goal is to minimize the difference between the target image and the reference image, use the Adam optimizer and the gradient descent method to perform iterative optimization, and stop the iteration when the loss calculated in the continuous iteration process no longer decreases significantly, determine the optimal A and b, and output the optimal affine transformation matrix; Through rigid registration, the coordinates of each pixel are transformed to the new position coordinates , generate aligned images, The method predicts the probability distribution of lesions based on a deep convolutional neural network and performs lesion segmentation and labeling based on a conditional random field (CRF), including: A lesion area model is constructed based on a deep convolutional neural network, including an input layer, a convolutional layer, a maximum pooling layer, a fully connected layer, and an output layer; Use the training set to train the model, and select the cross entropy loss function to calculate the difference between the model prediction result and the actual label of the lesion area. Use the Adam optimizer to perform gradient descent optimization and update the model parameters. If the model loss no longer decreases significantly during the continuous iteration process, stop iterating and output the model. A lesion probability distribution map based on the aligned image is obtained based on the lesion region segmentation prediction result output by the lesion region model; Based on the conditional random field CRF, the probability of each pixel belonging to the lesion area is taken as a single-point term, the similarity between pixels is defined as a double-point term, and the energy function is defined based on the single-point term and the double-point term, which is expressed as: ； ； ； in Indicates the cost of the i-th pixel belonging to the lesion, represents the probability that the i-th pixel belongs to the lesion area, represents the similarity cost between the adjacent i-th pixel and j-th pixel, and Represent the spatial coordinates of the i-th pixel and the j-th pixel respectively, represents the weight coefficient, and Represent the grayscale values ​​of the i-th pixel and the j-th pixel respectively, and denote the spatial scale parameter and the color scale parameter respectively, Represents the total energy function value, N represents the total number of pixels, Represents the neighboring pixel area of ​​the i-th pixel; Use the training set and select the cross entropy loss function to calculate the loss between the lesion prediction results of the conditional random field CRF and the actual label, use the Adam optimizer for gradient descent optimization, and update the conditional random field CRF parameters, including , and , if the calculated loss no longer decreases significantly during the continuous iteration process, the iteration is stopped, the model parameters are output, and the model is updated; The mean-field approximation method is used to optimize the energy function and determine the optimization formula, which is expressed as: ； in represents the normalization factor, represents the classification probability, Represents the classification probability of adjacent pixels; The predicted probability map of the lesion area model is used as the initial ,Sure ; based on The probability distribution of each pixel is updated in the continuous iteration process. If the change in the value of is no longer significantly reduced, the iteration is stopped and the final output of the model is ; Based on the final Generate the corresponding lesion area probability map, segment the pixel values ​​of the probability map based on the historical average value as the segmentation threshold, and mark the lesion area. The method uses the B-spline basis function to describe the local displacement and non-rigidly registers the image data of the lesion area to generate the final image, including: Generate a grid based on the lesion area of ​​the image data, define the grid nodes as control points, and perform non-rigid registration; Based on each control point, the B-spline basis function is used to describe the local displacement, which is expressed as: ； ； in represents the displacement of pixel point x, n represents the number of control points adjacent to the pixel point, represents the displacement vector of control point i, represents the B-spline basis function, represents the coordinates of the i-th control point, represents the third-order B-spline basis function; The reference image is determined based on the training set, and the mean square error (MSE) is selected as the similarity measurement function. The goal is to minimize the difference between the target image and the reference image, which is expressed as: ； in represents the non-rigid registration calculation error, Represents the total number of pixels. represents the coordinates of the i-th pixel, is the corresponding pixel coordinate in the reference image, M represents the total number of control points, represents the smoothness regularization parameter, which is determined based on historical data, represents the displacement vector gradient of control point j, Represents pixel The displacement of Use the Levenberg-Marquardt optimization algorithm to calculate the displacement of the current control point , calculate the value of the non-rigid registration calculation error, and calculate the displacement of each control point by the objective function The gradient of the control point is then updated. When the calculated error no longer decreases significantly during the continuous optimization process, the iteration is stopped and the displacement of the control point is optimized. ; Calculate pixel displacement based on control point displacement through non-rigid registration , and further adjust the pixel positions based on the aligned image to generate the final image. The step of selecting a lesion area for volume calculation includes: Based on the image scanning equipment, three-dimensional image data is formed by stacking multiple layers of image data. Based on the final image generated by non-rigid registration, the lesion change area is determined and the voxels in the lesion change area are selected for volume calculation. The method of using a nonlinear differential equation to determine the growth rate and change rate of the lesion volume comprises: The volume change data of the lesion is fitted using a nonlinear differential equation, expressed as: ; in represents the lesion volume at time t, represents the growth rate of the lesion volume, Indicates the maximum limit value of the lesion volume; Using historical data and using the squared error as the loss function, the difference between the observed volume and the predicted volume is minimized; The nonlinear least squares method is used to iteratively calculate the loss. If the model loss no longer decreases significantly during the continuous iteration, the iteration is stopped and the final K and ; The rate of change of the growth rate was determined based on the change in the lesion volume growth rate at different times, and the rate of change was evaluated based on the physician's experience.

2. A system based on the patient condition tracking method according to claim 1, characterized in that: include: An image data processing module is used to collect the patient's lesion image data and perform image preprocessing; Rigid registration module, used for rigid registration through affine transformation; Lesion recognition and segmentation module, which is used to predict the probability distribution of lesion areas based on deep convolutional neural networks and use conditional random fields to segment and mark lesions; The non-rigid registration module is used to describe the local displacement through the B-spline basis function and perform non-rigid registration; A lesion volume calculation module is used to select the lesion area for volume calculation based on the image data after non-rigid registration; The volume change analysis module is used to determine the volume growth rate and change rate of the lesion using nonlinear differential equations.

3. A computer device comprising: Memory and processor; The memory stores a computer program, wherein the processor implements the steps of the patient condition tracking method according to claim 1 when executing the computer program.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the patient condition tracking method according to claim 1 are implemented.

Images