Target region overextension distance determination method

Abstract

The present application provides a medical image data processing method for determining an outer expansion distance of a target volume, comprising the steps of: receiving a first image describing an anatomical structure of a patient, the first image comprising at least a target tumor of the patient and an organ at risk of the target tumor; receiving a prediction model trained for predicting an outer expansion distance of a general target volume corresponding to the target tumor in at least one direction based on at least the first image; and determining, by at least one processor, the outer expansion distance of the general target volume in the at least one direction based on the first image by using the prediction model, to determine a planning target volume of the target tumor based on the outer expansion distance of the target tumor in each direction. The present application realizes a non-uniform specific expansion scheme to adapt to the non-uniform outer expansion of the target tumor, so that the determined planning target volume can reduce the damage to the normal cells around the target volume as much as possible on the basis of ensuring the thoroughness of treatment.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for determining the outer distance of a target area. Background Technology

[0002] The radiotherapy target volume characterizes the location and extent of the tumor and is an important benchmark for determining the radiotherapy plan. Generally speaking, the radiotherapy target volume usually involves the gross target volume and the planning target volume. The gross target volume (GTV) represents the actual area of ​​tumor cells, while the planning target volume (PTV) represents the area where radiotherapy will be performed.

[0003] Specifically, considering the normal movement of organs and body movement caused by respiration during treatment, and in order to reduce the possibility of recurrence caused by the spread of tumor cells, the planned target area is usually the area obtained by expanding the gross target area.

[0004] Currently, the planned target area is usually determined by uniformly expanding outward by a distance of 5 millimeters, based on the experience of the staff, on the basis of the gross target area. In order to reduce the possibility of recurrence, the planned target area is usually determined based on the maximum possible expansion distance, and then the radiotherapy plan is formulated based on the planned target area. This method makes the normal area around the tumor more susceptible to radiation damage, thus having a negative impact on the patient's health. Summary of the Invention

[0005] This invention provides a method for determining the target area extension distance, which solves the defect of the existing method of determining the planned target area by extension, which easily causes the normal area around the tumor to be subjected to excessive radiation damage. This invention achieves a more accurate method for determining the target area extension distance, so as to determine a more precise planned target area.

[0006] This invention provides a medical image data processing method for determining the outer dimensions of a target area for medical treatment, comprising performing the following steps on at least one processor of at least one computer:

[0007] Receive a first image describing the patient's anatomy, the first image including at least the patient's target tumor and the organs at risk of the target tumor;

[0008] Receive a prediction model trained to predict the outward extension distance of the gross target region corresponding to the target tumor in at least one direction, based at least on the first image; and

[0009] At least one processor determines the outward expansion distance of the gross target region in at least one direction based on the first image using the prediction model, in order to determine the planned target region of the target tumor based on the outward expansion distance of the target tumor in each direction.

[0010] According to a medical image data processing method provided by the present invention, the step of determining the outward distance of the gross target region in at least one direction by at least one processor using the prediction model based on the first image specifically includes:

[0011] The first image is input into a prediction model by at least one processor, and the prediction model delineates the gross target region and the organ at risk region of the first image, wherein the gross target region and the organ at risk region of the first image are used to describe the location and size of the target tumor and the organ at risk.

[0012] The prediction model determines and outputs the outward expansion distance of the gross target area in at least one direction based on the gross target area and the region of organs at risk in the first image.

[0013] According to a medical image data processing method provided by the present invention, the first image includes a first type of medical image and a second type of medical image;

[0014] The first type of medical image represents the location and size of the target tumor before the patient undergoes the current medical treatment; the second type of medical image represents the predicted location and size of the target tumor after the patient undergoes the current medical treatment.

[0015] According to a medical image data processing method provided by the present invention, when the first image includes a second type of medical image, before the step of receiving the first image describing the anatomical structure of the patient, the method further includes:

[0016] Receive the patient's first type of medical image before the current medical treatment and the number of treatment sessions corresponding to the current medical treatment;

[0017] Receive a generative model trained to predict a patient's target tumor in a second type of medical image after the current medical treatment, based at least on the input medical image; and

[0018] The second type of medical image is determined by at least one processor using the generation model, based at least on the first type of medical image.

[0019] According to a medical image data processing method provided by the present invention, the input to the generating model further includes one or more of the patient's historical medical treatment records, the patient's current medical treatment plan, the management plan for organs at risk, and the expected treatment time for the current medical treatment.

[0020] According to a medical image data processing method provided by the present invention, the step of determining a second type of medical image by at least one processor using the generation model based on at least the first type of medical image further includes:

[0021] Receive raw training data, which includes at least raw training images of the patient acquired before multiple different rounds of medical treatment, and the raw training images include at least the patient's target tumor and the organs at risk of the target tumor;

[0022] Receive the target training data, each target training data corresponding to the original training data;

[0023] Determine the architecture of the generative model; and

[0024] The generative model is trained using the original training data and the target training data.

[0025] According to a medical image data processing method provided by the present invention, the step of determining a second type of medical image by at least one processor using the generation model based at least on a first type of medical image includes:

[0026] Image features are extracted from the first type of medical images;

[0027] The patient's historical medical treatment records, the patient's current medical treatment plan, the management plan for organs at risk, and the estimated treatment time for the current medical treatment are compiled into text information, and text features are extracted.

[0028] A time code is constructed based on the number of treatment sessions corresponding to the current medical treatment.

[0029] The image features, text features, and time encoding are input into the generation model to obtain the second type of medical image output by the generation model.

[0030] The present invention also provides a medical image processing system, comprising:

[0031] Input interface, the input interface being configured to:

[0032] Receive a first image describing the patient's anatomy, the first image including at least the patient's target tumor and organs at risk of the target tumor; and

[0033] Receive a neural network model trained to predict the outward distance of the gross target area corresponding to the target tumor in at least one direction based on the first image;

[0034] At least one storage device configured to store the first image and the neural network model; and

[0035] An image processing module is configured to determine the outward extension distance of the gross target region in at least one direction based on the first image using the neural network model, so as to determine the planned target region of the target tumor based on the outward extension distance of the target tumor in each direction.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the medical image data processing method described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the medical image data processing method as described above.

[0038] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the medical image data processing method as described above.

[0039] The medical image data processing method provided by this invention is used to determine the outer expansion size of the target area for medical treatment. By constructing and training a neural network model, and taking into account the impact of organs at risk on the tumor, a non-uniform specific expansion scheme is realized to adapt to the non-uniform expansion of the target tumor. This allows the determined planned target area to minimize damage to normal cells around the target area while ensuring the thoroughness of the treatment. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is one of the flowcharts illustrating the medical image data processing method provided by the present invention;

[0042] Figure 2 This is a schematic diagram of the uniform outward expansion process in the medical image data processing method provided by the present invention;

[0043] Figure 3 This is a flowchart illustrating the non-uniform expansion process in the medical image data processing method provided by the present invention:

[0044] Figure 4 This is a schematic diagram of the generation model in the medical image data processing method provided by the present invention;

[0045] Figure 5 This is the second flowchart of the medical image data processing method provided by the present invention;

[0046] Figure 6 This is a schematic diagram of the structure of the medical image data processing system provided by the present invention;

[0047] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] The following is combined with Figures 1 to 5 This invention introduces a method for determining the target area extension distance, specifically including a medical image data processing method for determining the target area extension size for medical treatment, comprising performing the following steps on at least one processor of at least one computer:

[0050] Step 101: Receive a first image describing the patient's anatomical structure, the first image including at least the patient's target tumor and the organs at risk of the target tumor;

[0051] Optionally, the first image is an image acquired directly or indirectly by a medical imaging device. For example, it may be a CT image of a patient acquired using a computed tomography (CT) scanner, an MRI image acquired using an magnetic resonance imaging (MRI) scanner, or a combined image obtained by registering a CT image with an MRI image.

[0052] The first image contains at least the patient's target tumor to be treated and the organ at risk (OAR) affected by the target tumor, so that the physiological behavior of the OAR is taken into account when assessing the direction and distance of movement of the target tumor.

[0053] Optionally, the organ at risk is determined based on the type and location of the target tumor.

[0054] For example, when the target tumor is pancreatic cancer, rectal cancer, or cervical cancer, the bladder and rectum are identified as organs at risk of the target tumor, and it is believed that the degree of filling and / or normal physiological activity of the bladder and rectum themselves affect the spread of tumor cells of the target tumor.

[0055] Step 102: Receive a prediction model trained to predict the outward distance of the gross target area corresponding to the target tumor in at least one direction, based at least on the first image;

[0056] Understandably, the impact of organs on the target tumor usually leads to uneven spread of the tumor cells. Taking cervical cancer as an example, if the patient's bladder is full, the target tumor is more likely to spread away from the bladder and the spread range is greater. The target tumor is less likely to spread closer to the bladder and the spread range is smaller, making the spread of the target tumor present a non-uniform spread pattern.

[0057] Based on this, in order to ensure the therapeutic effect of radiotherapy and reduce the risk of recurrence caused by the spread of tumor cells, if a conventional and conservative treatment strategy is adopted, considering the farthest possible spread distance as the outward spread distance and determining the planned target area based on the gross target area of ​​the target tumor, normal cells in the direction with a low probability of spread will suffer excessive radiation damage during radiotherapy, which will affect the patient's health.

[0058] Therefore, in this embodiment, a prediction model is pre-built and trained to determine the outward expansion distance of the gross target region in at least one direction based on the input image. The gross target region is then expanded outward by the outward expansion distance determined in multiple directions, corresponding to the uneven outward expansion of tumor cells of the target tumor. This constructs an uneven outward expansion scheme for the gross target region, resulting in the planned target region.

[0059] Alternatively, the prediction model can be a neural network model based on an encoder-decoder structure, such as a convolutional neural network model (e.g., CNN, U-net).

[0060] Furthermore, at least one direction can be determined based on any coordinate system, as long as it can be used to describe the non-uniform expansion of the general target area. For example, if the first image is a CT image, the coordinate system can be the image coordinate system corresponding to the CT image.

[0061] In this embodiment, for ease of description, three basic axes of the anatomical direction are selected to define the direction of expansion: the vertical axis represents the up-down direction, the sagittal axis represents the anterior-posterior direction, and the coronal axis represents the lateral direction. By calculating the expansion distance of the gross target area in each of these six directions, a non-uniform expansion scheme for the gross target area is constructed to obtain the planned target area.

[0062] Based on this, in one feasible implementation, the input of the convolutional neural network model is defined as the first image, and the output is defined as the outward expansion distance of the gross target area of ​​the target tumor in six directions, thereby constructing and training the convolutional neural network model.

[0063] In one feasible implementation, anatomical images of the patient before and after each radiotherapy session can be acquired. The acquired images include at least the patient's target tumor and organs at risk. The images acquired before radiotherapy are used as input, and the images before and after radiotherapy are delineated to determine the target area before and after radiotherapy. The distance difference between the target area before and after radiotherapy in each of the six directions is used as a label to train the constructed neural network model. After multiple iterations, the model whose difference between the outward expansion distance in the six directions and the corresponding label is less than a preset threshold is used as the trained prediction model.

[0064] In this embodiment, anatomical images of patients before and after radiotherapy are collected to construct a dataset. The location and size data of the target area and critical organs are determined by automatic delineation. The coverage of the target area corresponding to the predicted expansion distance is used to evaluate the mobility of the target area in different directions. Finally, the model with a coverage greater than a preset threshold is used as the training prediction model. Based on the input first image, the expansion distance in six directions is output to construct a specific expansion scheme.

[0065] Specifically, anatomical images (FBCT1) before each treatment session and (FBCT3) after each treatment session were pre-collected from patients who had completed treatment. The acquired images were DICOM RTSTRUCT files containing delineations of the anatomical structures, including the gross target area and regions of organs at risk. Based on this, the registered contours were converted into 3D bipolar masks using MATLAB for processing. Volumetric convolution between the original contours and the anisotropic center points was used to explore contour edge expansion in different directions to determine the anisotropic expansion distance.

[0066] In a specific contour edge expansion exploration method, the result of uniform expansion in all directions is used as the starting point for the search of anisotropic expansion.

[0067] The prediction model determines the minimum boundary and expansion distance for isotropic uniform expansion based on the inputs FBCT1 and FBCT2. For example... Figure 2 As shown, CTV1 is the delineation result of FBCT1, and CTV3 is the delineation result of FBCT3. The prediction model performs isotropic expansion on the CTVs in FBCT1, that is, the expansion direction is the same in each direction, in order to find all possible edge values ​​that can cover 99% of the CTVs in CTV3. The minimum value in each direction is recorded as the minimum boundary value of isotropic expansion.

[0068] ;

[0069] In the formula, This represents the minimum recorded edge measurement value, which represents the outward expansion distance values ​​in the six directions of left, right, front, back, up, and down. At this time, the outward expansion distance values ​​in the six directions are the same.

[0070] Based on this, the prediction model uses isotropic edges as the starting point for anisotropic edge searches, seeking the minimum margin in each direction. Taking cervical cancer as an example, considering the significant variations in the uterine body as an OAR (Occupational Angiogenesis Assessment), the target area is divided into the uterine body... and remaining clinical target areas The process involves iterating in six directions to reduce the residual value, as follows: Figure 3 As shown, this continues until the obtained PTV1 can cover more than 99% of CTV3 or the margin value in a certain direction reaches 1mm:

[0071] ;

[0072] ;

[0073] In the formula, and , , , , and These represent the minimum edge measurement value and the outward distance values ​​in the six directions: left, right, front, back, top, and bottom.

[0074] This is used as the output of the prediction model, which is the predicted outward distance in six directions.

[0075] Understandably, the input to the prediction model may also include patient history and treatment information, current radiotherapy fractions, and other data related to the patient and their current treatment. Optionally, relevant data can be acquired and a corresponding feature extraction network can be constructed in the input layer of the prediction model, thereby using the aforementioned relevant data as input to the prediction model to achieve the prediction of the outward propagation distance.

[0076] Step 103: At least one processor determines the outward extension distance of the gross target region in at least one direction based on the first image using the prediction model, so as to determine the planned target region of the target tumor based on the outward extension distance of the target tumor in each direction.

[0077] After the processor acquires the first image of the current patient and the pre-trained prediction model, it inputs the first image into the prediction model to obtain the approximate target area outward distance in at least one direction output by the prediction model.

[0078] Based on the gross target area, the planned target area of ​​the target tumor can be obtained by expanding outward in each direction according to the outward distance in that direction, which is used to formulate the radiotherapy plan for this treatment.

[0079] This invention, by constructing and training a prediction model, takes into account the impact of organs at risk on tumors, and realizes a non-uniform, specific expansion scheme to adapt to the non-uniform expansion of the target tumor. This allows the determined planned target area to minimize damage to normal cells around the target area while ensuring the thoroughness of treatment.

[0080] The medical image data processing method provided by this invention includes, in part, the step of determining the outward distance of the gross target region in at least one direction by at least one processor using the prediction model based on the first image, specifically comprising:

[0081] The first image is input into a prediction model by at least one processor, and the prediction model delineates the gross target region and the organ at risk region of the first image, wherein the gross target region and the organ at risk region of the first image are used to describe the location and size of the target tumor and the organ at risk.

[0082] The prediction model determines and outputs the outward expansion distance of the gross target area in at least one direction based on the gross target area and the region of organs at risk in the first image.

[0083] The prediction model provided in this embodiment is built based on a convolutional neural network model. The first image is a medical image obtained directly from a medical imaging device before the current patient undergoes the current treatment session.

[0084] The predictive model first receives a first image of the current patient as input by the processor, and then delineates the first image to identify the gross target area of ​​the patient's target tumor and the organ at risk region (OAR region), thereby determining the size and location of the target tumor and the organ at risk.

[0085] For example, if the first image is a CT image, the target area and OAR region of the target tumor can be obtained by manually, automatically or semi-automatically delineating the first image.

[0086] Understandably, manual delineation means the model receives data generated by manual delineation by staff to obtain the general target area and organ at risk regions in the first image; automatic delineation means the model automatically delineates the first image to obtain the general target area and organ at risk regions in the first image; semi-automatic delineation means that based on automatic delineation, the model receives adjustment data from staff on the automatic delineation results, and obtains the general target area and organ at risk regions in the first image based on manual correction.

[0087] Based on this, the delineation results of the gross target area and the organ at risk region of the first image can be an image representation directly delineated on the first image, and / or, based on the location and size data of the gross target area and the organ at risk determined by the delineation, it can be used to characterize the location and size of the target tumor and the organ at risk.

[0088] The predictive model determines the gross target area's outward expansion distance in at least one direction based on the location and size of the target tumor and organs at risk.

[0089] In the medical image data processing method provided by the present invention, the first image includes a first type of medical image and a second type of medical image;

[0090] The first type of medical image represents the location and size of the target tumor before the patient undergoes the current medical treatment; the second type of medical image represents the predicted location and size of the target tumor after the patient undergoes the current medical treatment.

[0091] Alternatively, the medical treatment may be any form of tumor treatment, such as online adaptive radiotherapy.

[0092] Understandably, in order to predict the outward spread distance, medical imaging equipment can be used to directly or indirectly acquire medical images characterizing the location and size of the target tumor and organs at risk before each stage of medical treatment as the first image.

[0093] For example, the CT image of the current patient can be obtained before the current treatment as the first image; or, for example, the CT image and MRI image of the current patient can be obtained before the current treatment, and the image obtained after registration of the two can be used as the first image to obtain a more accurate gross target area and OAR region.

[0094] Furthermore, considering that each OART procedure takes time from acquiring medical images of the patient using medical imaging equipment to actually treating the target tumor, and that the location of the target tumor may change during this time compared to when the medical images were acquired, the first type of medical image could also be a predicted image based on directly acquired medical images. This predicted image characterizes the location and size of the gross target area and OART region before actual treatment.

[0095] Furthermore, in order to more accurately predict the specific outward spread distance and further reduce the damage of radiotherapy to normal cells, in this embodiment, images before and after radiotherapy, namely the first type of medical images and the second type of medical images, are used as inputs to the prediction model.

[0096] Specifically, the first image first includes a first type of medical image characterizing the gross target area and OAR region before radiotherapy. The first type of medical image can be acquired directly or indirectly by medical imaging equipment before the patient undergoes the current fraction of radiotherapy.

[0097] The first image also includes a second type of medical image that characterizes the gross target area and OAR region after radiotherapy, which can be predicted based on the first type of medical image.

[0098] Compared to predicting the external propagation distance using only pre-radiotherapy images, or predicting the external propagation distance using both pre- and post-radiotherapy images, this method, based on the gross target area directly delineated from second-type medical images, clarifies the gross target area after radiotherapy during the prediction of the external propagation distance. By directly incorporating the location and volume of the gross target area after radiotherapy, a more accurate prediction of the external propagation distance is achieved.

[0099] In one feasible implementation, a dual-branch network is used in the input layer to process the first type of medical images and the second type of medical images respectively, and to extract and fuse features of the first type of medical images and the second type of medical images. In the output layer, a six-dimensional vector is defined as the output, which corresponds to the outward expansion distance in six directions respectively. A neural network model is constructed and trained based on coverage loss and regularization constraints as a prediction model to achieve prediction of outward expansion distance based on images before and after radiotherapy.

[0100] In another feasible implementation, the predicted gross target area can be defined based on the gross target area of ​​the first type of medical image and the predicted outward spread distance. The gross target area after radiotherapy can be determined based on the second type of medical image. The ratio of the gross target area after radiotherapy to the predicted gross target area can be used as the coverage rate. An objective function can be constructed with the goal of maximizing the coverage rate as the objective, and constraints can be constructed based on the location and size of the organs at risk. The objective function can be solved by an optimization algorithm such as particle swarm optimization to obtain the optimal outward spread distance and output it.

[0101] This invention predicts a second type of medical image representing the gross target area of ​​the target tumor after the current fractionation treatment based on a first type of medical image during the current fractionation treatment. The first and second types of medical images are used together as input to the prediction model to introduce more realistic information on the location and size of the gross target area of ​​the target tumor after treatment in the process of predicting the outward expansion distance, thereby achieving a more accurate prediction of the outward expansion distance in at least one direction.

[0102] In the medical image data processing method provided by the present invention, when the first image includes the second type of medical image, before the step of receiving the first image describing the anatomical structure of the patient, the method further includes:

[0103] Receive the patient's first type of medical image before the current medical treatment and the number of treatment sessions corresponding to the current medical treatment;

[0104] Receive a generative model trained to predict a patient's target tumor in a second type of medical image after the current medical treatment, based at least on the input medical image; and

[0105] The second type of medical image is determined by at least one processor using the generation model, based at least on the first type of medical image.

[0106] In order to acquire a second type of medical image that can characterize the location and size of the target tumor after treatment before the current treatment session, in this embodiment, a generative model is pre-built and trained to predict the second type of medical image.

[0107] Specifically, the input to the generative model includes at least the first type of medical image acquired before the current medical treatment and the number of treatment fractions corresponding to the current medical treatment.

[0108] The first type of medical image can be a CT image acquired before the current treatment fraction, serving as the basis for predicting the second type of medical image. Simultaneously, since the treatment effect on the target tumor varies with each fraction, the number of fractions corresponding to the current treatment fraction must also be used as input to generate a more accurate second type of medical image.

[0109] Optionally, the generative model can be built and trained based on neural networks such as Generative Adversarial Networks (GANs), diffusion models, or U-Net networks that can perform image-to-image conversion tasks.

[0110] The processor receives a pre-trained generative model and a first-class medical image obtained from the current medical treatment session. The first-class medical image and the number of sessions corresponding to the current session are input into the generative model to obtain a second-class medical image output by the generative model, which can be used as input to the prediction model.

[0111] In the medical image data processing method provided by the present invention, the input of the generation model also includes one or more of the following: the patient's historical medical treatment records, the patient's current medical treatment plan, the management plan for organs at risk, and the expected treatment time for the current medical treatment.

[0112] Understandably, the images of the target tumor after each fraction of medical treatment are not only affected by the first type of medical image before medical treatment and the current treatment fraction, but also by the patient's individual physical condition, historical treatment records, and current treatment plan.

[0113] Therefore, in this embodiment, in order to take into account the above factors when generating the second type of medical image, one or more of the patient's historical medical treatment records, the patient's current medical treatment plan, the management plan for organs at risk, and the expected treatment time of the current medical treatment are also used as inputs to the generation model.

[0114] Optionally, the patient's medical records can be used as a historical record of medical treatment. It is understood that the patient's medical records contain information about the patient's physical condition, medication and / or treatment plan at different stages of the disease.

[0115] Optionally, the patient's current medical treatment plan is a pre-designed plan based on the patient's disease course, including the irradiation location and dose of radiotherapy, thereby taking into account the impact of the current radiotherapy plan on the location and size of the target tumor in the generation of the second type of medical pattern. It should be noted that the patient's current medical treatment plan is a treatment plan estimated before the current radiotherapy operation, not the actual medical treatment plan executed. The actual medical treatment plan is determined based on the planned target area determined by the predicted spread distance.

[0116] Optionally, the patient's organ-at-risk management plan refers to the pre-determined management measures for the organs at risk during the current treatment session. Taking the bladder as an example, the patient's pre-radiotherapy fluid intake, bladder fluid content, and bladder fluid output constitute the organ-at-risk management plan. Similarly, taking the lungs as an example, the patient's respiratory time, respiratory rate, and lung air content constitute the lung management plan.

[0117] Optionally, the expected treatment time for the current medical treatment is the expected duration of the current radiotherapy. It is understood that the duration of the current radiotherapy has a direct impact on the size and location of the target tumor. Therefore, the expected treatment time for the current medical treatment is also used as input to the generative model to predict the second type of medical image.

[0118] This invention improves the prediction accuracy of second-type medical images by incorporating information such as the patient's first-type medical image prior to the current medical treatment, the bladder management strategy during the current medical treatment, the expected treatment time for the current medical treatment, and the patient's historical medical records and physical condition records. Specifically, by introducing the expected treatment time as temporal information, the prediction accuracy of second-type medical images can be effectively improved.

[0119] The medical image data processing method provided by this invention, the step of determining the second type of medical image by at least one processor using the generation model based at least on the first type of medical image, specifically further includes:

[0120] Receive raw training data, which includes at least raw training images of the patient acquired before multiple different rounds of medical treatment, and the raw training images include at least the patient's target tumor and the organs at risk of the target tumor;

[0121] Receive the target training data, each target training data corresponding to the original training data;

[0122] Determine the architecture of the generative model; and

[0123] The generative model is trained using the original training data and the target training data.

[0124] In this embodiment, the processor receives the original training data and the target training data in advance, and completes the training of the generative model based on the determined generative model architecture.

[0125] Optionally, the data types contained in the original training data can be determined first, and then a suitable generative model type can be selected based on the data types, and the corresponding generative model architecture can be constructed; alternatively, the type and architecture of the generative model can be defined first, and then suitable original training data can be collected accordingly.

[0126] Understandably, the original training data is obtained by collecting treatment information from multiple patients across multiple sessions throughout their history, and should at least include medical images of the patient before the current session, such as CT images, as well as the number of sessions corresponding to the current session.

[0127] Simultaneously, corresponding to the original training data, target training data is collected, where the target training data is the medical image after medical treatment corresponding to the original training data.

[0128] In one specific implementation, a corresponding training dataset is established for each type of tumor in order to train a generative model for each type of tumor.

[0129] In the training dataset corresponding to each type of tumor, the original training data and target training data of different patients at each fraction are collected, and the original training data and fractions are organized into the input dataset. The target training data corresponding to each piece of original training data is used as the label.

[0130] Based on this, the training dataset is divided into a training set, a validation set, and a test set for training to obtain a generative model.

[0131] Taking the generative model constructed by generative adversarial network as an example, the predicted image generated based on the original training data and the number of times is compared with the target training data. If the similarity is greater than a preset threshold, the generative model is considered to have completed training.

[0132] In the medical image data processing method provided by the present invention, the step of determining the second type of medical image by at least one processor using the generation model based at least on the first type of medical image includes:

[0133] Image features are extracted from the first type of medical images;

[0134] The patient's historical medical treatment records, the patient's current medical treatment plan, the management plan for organs at risk, and the estimated treatment time for the current medical treatment are compiled into text information, and text features are extracted.

[0135] A time code is constructed based on the number of treatment sessions corresponding to the current medical treatment.

[0136] The image features, text features, and time encoding are input into the generation model to obtain the second type of medical image output by the generation model.

[0137] In this embodiment, in order to make full use of the information of each modality to achieve prediction of the second type of medical images, a generative adversarial network (GAN) is used as the basis, and a generator and a discriminator are used to perform image transformation to construct a generative model.

[0138] like Figure 4 As shown, the generation model in this embodiment first includes an image feature extraction model for extracting features from the input first type of medical image, and extracts the image features.

[0139] In addition, to incorporate the information of treatment frequency into the prediction process, the generative model also includes an encoder to convert the treatment frequency corresponding to the current medical treatment into time codes.

[0140] Furthermore, to utilize the patient's historical medical treatment records, current medical treatment plan, management plan for organs at risk, and estimated treatment time for the current medical treatment, the generative model also includes a text encoder. First, the patient's historical medical treatment records, current medical treatment plan, management plan for organs at risk, and estimated treatment time for the current medical treatment are formatted as text. Then, the text encoder is used to extract features from these textual features.

[0141] The obtained image features, text features, and time codes are fused and then input into the generator of the trained GAN to obtain the second type of medical image output by the generator.

[0142] It should be noted that in this embodiment, in order to achieve prediction based on multimodal information, a generator with Visiontransformer as the backbone network is adopted to improve the generation quality of image details. During the training process of the generation model, image features, time encoding and text features are used as inputs to the generator. Similarly, the discriminator also uses time encoding and text features as inputs based on image features to determine the authenticity of the generated image.

[0143] The generator generates an image Fa that is as close as possible to the real image based on the input features, while the discriminator is used to distinguish the similarity between the generated Fa and the real post-radiotherapy image Ra, thus constraining the generator. Its training process is described by an adversarial loss function. Minimum-Maximum Optimization Task:

[0144] ;

[0145] In the formula, Discriminator D B For the real domain B image I B The ability to recognize, Represents generator G A Deceive D B The training process maximizes the discriminator's recognition ability and minimizes the generator's deception ability.

[0146] Similarly, another set of generative adversarial losses This can be expressed as:

[0147] ;

[0148] In the formula, Discriminator D A For the real domain A image I A The ability to recognize, This represents the ability of the generator GB to deceive the discriminator DA. Training maximizes the discriminator's recognition ability and minimizes the generator's deception ability.

[0149] In the formula, the cycle-consistent loss is used. This ensures that a domain image can be restored as much as possible after being transformed by two generators. While constraining the generation directions of the two generators, it avoids direct interaction between the two domain images, thus achieving unsupervised network training.

[0150] ;

[0151] Furthermore, an identity loss is introduced:

[0152] ;

[0153] In summary, the total loss function is a weighted sum of the generative adversarial loss and the cycle-consistent loss, as shown in the following equation:

[0154] ;

[0155] In the formula, and They are and The weights are used to control the importance of the corresponding losses.

[0156] This invention constructs a generation module that integrates time-GAN and cycle-GAN. By generating images in a cyclic manner, it achieves higher accuracy and obtains more realistic second-type medical images. By introducing time encoding through time-GAN, it enables the generation of second-type medical images for each round in online adaptive radiotherapy. In online adaptive radiotherapy, the generated second-type medical images, together with the first-type medical images, can be used as input to the prediction model to obtain a more accurate outward expansion distance in at least one direction, resulting in a planned target area with higher coverage. Based on the planned target area, the radiotherapy plan is specified to minimize damage to normal cells in each round of radiotherapy.

[0157] A specific implementation method based on the above content is as follows: Figure 5 As shown, for a treatment fraction, a first type of medical image is first acquired before treatment as the pre-treatment image. The first type of medical image is automatically delineated to determine the gross target area and organ at risk region. The delineated first type of medical image, the expected treatment time of the current fraction, and the organ at risk management strategy (bladder management strategy in the figure) are input into a pre-trained generative model (CycleGAN in the figure) to obtain a second type of medical image output by the generative model, which represents the patient's image after this radiotherapy. The second type of medical image is automatically delineated to determine the gross target area and organ at risk region. The first type of medical image and the second type of medical image are then used as the first image and input into a pre-trained prediction model to evaluate the degree of movement of the gross target area before radiotherapy, and obtain the outward expansion distance in multiple directions (six directions in this embodiment) for the current fraction, thereby forming a specific margin scheme for the current fraction.

[0158] The planned target area is determined based on the obtained outward expansion distance, and then the radiotherapy plan for this radiotherapy is determined based on the planned target area, so as to reduce the damage to normal cells during radiotherapy by using a planned target area with higher coverage.

[0159] The medical image data processing system provided by the present invention is described below. The medical image data processing system described below can be referred to in correspondence with the medical image data processing method described above.

[0160] like Figure 6 As shown, the medical image data processing system includes:

[0161] Input interface 601, the input interface being configured to:

[0162] Receive a first image describing the patient's anatomy, the first image including at least the patient's target tumor and organs at risk of the target tumor; and

[0163] Optionally, the first image is an image acquired directly or indirectly by a medical imaging device. For example, it may be a CT image of a patient acquired using a computed tomography (CT) scanner, an MRI image acquired using an magnetic resonance imaging (MRI) scanner, or a combined image obtained by registering a CT image with an MRI image.

[0164] The first image contains at least the patient to be treated, the target tumor to be treated, and the organ at risk (OAR) affected by the target tumor, so as to take into account the physiological behavior of the OAR as a factor in assessing the direction and distance of movement of the target tumor.

[0165] Optionally, the organ at risk is determined based on the type and location of the target tumor.

[0166] For example, when the target tumor is pancreatic cancer, rectal cancer, or cervical cancer, the bladder and rectum are identified as organs at risk of the target tumor, and it is believed that the degree of filling and / or normal physiological activity of the bladder and rectum themselves affect the spread of tumor cells of the target tumor.

[0167] Receive a neural network model trained to predict the outward distance of the gross target area corresponding to the target tumor in at least one direction based on the first image;

[0168] Understandably, the impact of organs on the target tumor usually leads to uneven spread of the tumor cells. Taking cervical cancer as an example, if the patient's bladder is full, the target tumor is more likely to spread away from the bladder and the spread range is greater. The target tumor is less likely to spread closer to the bladder and the spread range is smaller, making the spread of the target tumor present a non-uniform spread pattern.

[0169] Based on this, in order to ensure the therapeutic effect of radiotherapy and reduce the risk of recurrence caused by the spread of tumor cells, if a conventional and conservative treatment strategy is adopted, considering the farthest possible spread distance as the outward spread distance and determining the planned target area based on the gross target area of ​​the target tumor, normal cells in the direction with a low probability of spread will suffer excessive radiation damage during radiotherapy, which will affect the patient's health.

[0170] Therefore, in this embodiment, a prediction model is pre-built and trained to determine the outward expansion distance of the gross target region in at least one direction based on the input image. The gross target region is then expanded outward by the outward expansion distance determined in multiple directions, corresponding to the uneven outward expansion of tumor cells of the target tumor. This constructs an uneven outward expansion scheme for the gross target region, resulting in the planned target region.

[0171] Alternatively, the prediction model can be a neural network model based on an encoder-decoder structure, such as a convolutional neural network model (e.g., CNN, U-net).

[0172] Furthermore, at least one direction can be determined based on any coordinate system, as long as it can be used to describe the non-uniform expansion of the general target area. For example, if the first image is a CT image, the coordinate system can be the image coordinate system corresponding to the CT image.

[0173] In this embodiment, for ease of description, three basic axes of the anatomical direction are selected to define the direction of expansion: the vertical axis represents the up-down direction, the sagittal axis represents the anterior-posterior direction, and the coronal axis represents the lateral direction. By calculating the expansion distance of the gross target area in each of these six directions, a non-uniform expansion scheme for the gross target area is constructed to obtain the planned target area.

[0174] Based on this, in one feasible implementation, the input of the convolutional neural network model is defined as the first image, and the output is defined as the outward expansion distance of the gross target area of ​​the target tumor in six directions, thus constructing the convolutional neural network model.

[0175] In one feasible implementation, anatomical images of the patient before and after each radiotherapy session can be acquired. The acquired images include at least the patient's target tumor and organs at risk. The images acquired before radiotherapy are used as input, and the images before and after radiotherapy are delineated to determine the target area before and after radiotherapy. The distance difference between the target area before and after radiotherapy in each of the six directions is used as a label to train the constructed neural network model. After multiple iterations, the model whose difference between the outward expansion distance in the six directions and the corresponding label is less than a preset threshold is used as the trained prediction model.

[0176] In this embodiment, anatomical images of patients before and after radiotherapy are collected to construct a dataset. The location and size data of the target area and critical organs are determined by automatic delineation. The coverage of the target area corresponding to the predicted expansion distance is used to evaluate the mobility of the target area in different directions. Finally, the model with a coverage greater than a preset threshold is used as the training prediction model. Based on the input first image, the expansion distance in six directions is output to construct a specific expansion scheme.

[0177] Understandably, the input to the prediction model may also include patient history and treatment information, current radiotherapy fractions, and other data related to the patient and their current treatment. Optionally, relevant data can be acquired and a corresponding feature extraction network can be constructed in the input layer of the prediction model, thereby using the aforementioned relevant data as input to the prediction model to achieve the prediction of the outward propagation distance.

[0178] At least one storage device 602, the storage device being configured to store the first image and the neural network model; and

[0179] Image processing module 603 is configured to determine the outward extension distance of the gross target region in at least one direction based on the first image using the neural network model, so as to determine the planned target region of the target tumor based on the outward extension distance of the target tumor in each direction.

[0180] After the processor acquires the first image of the current patient and the pre-trained prediction model, it inputs the first image into the prediction model to obtain the approximate target area outward distance in at least one direction output by the prediction model.

[0181] Based on the gross target area, the planned target area of ​​the target tumor can be obtained by expanding outward in each direction according to the outward distance in that direction, which is used to formulate the radiotherapy plan for this treatment.

[0182] This invention, by constructing and training a neural network model, takes into account the impact of organs at risk on tumors, and realizes a non-uniform, specific expansion scheme to adapt to the non-uniform expansion of the target tumor. This allows the determined planned target area to minimize damage to normal cells around the target area while ensuring the thoroughness of treatment.

[0183] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may invoke logical instructions in the memory 730 to execute a medical image data processing method, the method comprising: performing the steps on at least one processor of at least one computer: receiving a first image describing the anatomical structure of a patient, the first image including at least a target tumor of the patient and organs at risk of the target tumor; receiving a prediction model trained to predict the outward extension distance of a gross target region corresponding to the target tumor in at least one direction, at least based on the first image; and having at least one processor determine the outward extension distance of the gross target region in at least one direction based on the first image using the prediction model, to determine a planned target region of the target tumor based on the outward extension distance of the target tumor in each direction.

[0184] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0185] On the other hand, the present invention also provides a computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is capable of performing the medical image data processing method provided by the methods described above. The method includes: performing the steps of: receiving a first image describing the anatomical structure of a patient, the first image including at least a target tumor of the patient and organs at risk of the target tumor; receiving a prediction model trained to predict the outward extension distance of a gross target area corresponding to the target tumor in at least one direction based at least on the first image; and having the at least one processor determine the outward extension distance of the gross target area in at least one direction based on the first image using the prediction model, so as to determine the planned target area of ​​the target tumor based on the outward extension distance of the target tumor in each direction.

[0186] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the medical image data processing method provided by the methods described above, the method comprising: performing the steps of: receiving a first image describing the anatomical structure of a patient, the first image including at least a target tumor of the patient and organs at risk of the target tumor; receiving a prediction model trained to predict the outward extension distance of a gross target region corresponding to the target tumor in at least one direction based at least on the first image; and having the at least one processor determine the outward extension distance of the gross target region in at least one direction based on the first image using the prediction model, so as to determine the planned target region of the target tumor based on the outward extension distance of the target tumor in each direction.

[0187] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0188] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical image data processing method for determining the outer dimensions of a target area for medical treatment, characterized in that, This includes performing the steps on at least one processor of at least one computer: Receive a first image describing the patient's anatomy, the first image including at least the patient's target tumor and the organs at risk of the target tumor; Receive a prediction model trained to predict the outward distance of the gross target region corresponding to the target tumor in at least one direction, based at least on the first image; as well as At least one processor determines the outward expansion distance of the gross target region in at least one direction based on the first image using the prediction model, so as to achieve non-uniform outward expansion of the gross target region based on the outward expansion distance of the target tumor in each direction, and determine the planned target region of the target tumor; The prediction model is trained in the following manner: Collect images of the patient's anatomical structures before and after each radiotherapy session, and delineate the gross target area before and after radiotherapy. The distance difference between the gross target area before and after radiotherapy in each predicted direction is used as a label to train the neural network model, thus obtaining the prediction model. The first image includes a first type of medical image and a second type of medical image; The first type of medical image represents the location and size of the target tumor before the patient undergoes the current medical treatment; the second type of medical image represents the predicted location and size of the target tumor after the patient undergoes the current medical treatment.

2. The medical image data processing method according to claim 1, characterized in that, The step of determining the outward extension distance of the general target region in at least one direction by at least one processor using the prediction model based at least on the first image specifically includes: The first image is input into a prediction model by at least one processor, and the prediction model delineates the gross target region and the organ at risk region of the first image, wherein the gross target region and the organ at risk region of the first image are used to describe the location and size of the target tumor and the organ at risk. The prediction model determines and outputs the outward expansion distance of the gross target area in at least one direction based on the gross target area and the region of organs at risk in the first image.

3. The medical image data processing method according to claim 1, characterized in that, If the first image includes the second type of medical image, the step of receiving the first image describing the patient's anatomical structure further includes: Receive the patient's first type of medical image before the current medical treatment and the number of treatment sessions corresponding to the current medical treatment; Receive a generative model trained to predict a patient's target tumor in a second type of medical image after the current medical treatment, based at least on a first type of medical image; and The second type of medical image is determined by at least one processor using the generation model, based at least on the first type of medical image.

4. The medical image data processing method according to claim 3, characterized in that, The inputs to the generative model also include one or more of the patient's historical medical treatment records, the patient's current medical treatment plan, the management plan for organs at risk, and the expected treatment time for the current medical treatment.

5. The medical image data processing method according to claim 3, characterized in that, The step of determining the second type of medical image by at least one processor using the generative model based at least on the first type of medical image specifically further includes: Receive raw training data, which includes at least raw training images of the patient acquired before multiple different rounds of medical treatment, and the raw training images include at least the patient's target tumor and the organs at risk of the target tumor; Receive the target training data, each target training data corresponding to the original training data; Determine the architecture of the generative model; and The generative model is trained using the original training data and the target training data.

6. The medical image data processing method according to claim 4, characterized in that, The step of determining the second type of medical image by at least one processor using the generative model based at least on the first type of medical image includes at least: Image features are extracted from the first type of medical images; The patient's historical medical treatment records, the patient's current medical treatment plan, the management plan for organs at risk, and the estimated treatment time for the current medical treatment are compiled into text information, and text features are extracted. A time code is constructed based on the number of treatment sessions corresponding to the current medical treatment. The image features, text features, and time encoding are input into the generation model to obtain the second type of medical image output by the generation model.

7. A medical image data processing system, characterized in that, include: Input interface, the input interface being configured to: Receive a first image describing the patient's anatomy, the first image including at least the patient's target tumor and the organs at risk of the target tumor; as well as Receive a prediction model trained to predict the outward distance of the gross target region corresponding to the target tumor in at least one direction based on the first image; At least one storage device configured to store the first image and the prediction model; as well as An image processing module is configured to determine the outward expansion distance of the gross target region in at least one direction based on the first image using the prediction model, so as to achieve non-uniform outward expansion of the gross target region based on the outward expansion distance of the target tumor in each direction, and determine the planned target region of the target tumor. The prediction model is trained in the following manner: Collect images of the patient's anatomical structures before and after each radiotherapy session, and delineate the gross target area before and after radiotherapy. The distance difference between the gross target area before and after radiotherapy in each predicted direction is used as a label to train the neural network model, thus obtaining the prediction model. The first image includes a first type of medical image and a second type of medical image; The first type of medical image represents the location and size of the target tumor before the patient undergoes the current medical treatment; the second type of medical image represents the predicted location and size of the target tumor after the patient undergoes the current medical treatment.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the medical image data processing method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the medical image data processing method as described in any one of claims 1 to 6.

Images