Method for predicting lesion recurrence by image analysis - Patent Application 20070122997

Abstract

The present invention relates to a method for post-treatment evaluation of ablation of a portion (160) of an anatomical structure of interest (130) of an individual (110), the anatomical structure of interest comprising at least one lesion (165). The post-treatment evaluation method comprises, in particular, using an automatic learning method of neural network type, automatically assessing the risk of recurrence of a lesion of the anatomical structure of interest of an individual based on an analysis of pairs of pre- and post-operative medical images (170) of the anatomical structure of interest of the individual, said method being pre-loaded during a so-called training phase onto a database comprising a plurality of pairs of medical images of the same anatomical structure of interest for a plurality of individuals, each medical image pair of the database being associated with a recurrence status of a lesion of the anatomical structure of interest of said patient. The present invention also relates to an electronic device (150) comprising a processor and a computer memory for storing instructions for such an evaluation method.

Description

[Technical Field]

[0001] Technical field of the invention The field of the invention is that of evaluation of medical interventions.

[0002] The present invention more particularly relates to a method for post-treatment assessment of the risk of recurrence of an ablated lesion.

[0003] The present invention is particularly applicable to assessing minimally invasive medical interventions by predicting the risk of recurrence or metastasis of lesions such as tumors. Such minimally invasive medical interventions correspond, for example, to percutaneous ablation of lesions, such as tumors in the liver, lungs, kidneys, or any other organ. Percutaneous ablation generally involves using imaging to guide the insertion of one or more needles through the skin to reach and destroy the lesion. [Background technology]

[0004] Description of the Prior Art The prior art discloses techniques for assessing the effectiveness of interventions and predicting the risk of lesion recurrence.

[0005] Such techniques include, for example, determining the extent to which the ablation region encompasses the lesion following ablation of the lesion. By comparing the volume of the ablation region to the volume of the lesion, it is possible to determine the ablation margin. Indeed, it is generally recommended to have an ablation margin of at least 5 millimeters.

[0006] To identify these margins, the volume of the lesion is typically compared to the volume of the ablation region, which is identified when planning the intervention and segmented by the surgeon on at least one postoperative image.

[0007] The main drawback is that the volume of the ablation region is typically inaccurately determined, often depending on the surgeon performing the segmentation. Furthermore, the quality of postoperative images is often poor, thereby introducing uncertainty into the segmentation. Therefore, it is difficult to establish a correlation between the ablation margin and the risk of lesion recurrence.

[0008] To improve upon techniques from the prior art, it is known to use methods for automatically segmenting the ablation region.

[0009] Such a technique is described, for example, in the scientific publication entitled "Detection and Monitoring of Thermal Lesions Induced by Microwave Ablation Using Ultrasound Imaging and Convolutional Neural Networks" by Zhang et al., published in September 2019. The segmentation method described in this publication allows for calculating the margins of the ablation area by segmenting pre-operative and post-operative ultrasound images.

[0010] However, the segmentation method described in this publication is unable to predict the risk of recurrence due to the low accuracy of the automatic segmentation of the ablation region. First, the segmentation method is limited to a subsampled matrix of the ablation region image, the size of which is fixed, typically 4 mm. 2This limits the use of this method to small sized regions. Furthermore, it is necessary to know the location of the ablation region in order to set the position of the subsampling matrix, making this method difficult to use in the absence of a fixed reference point in the subsampling matrix of the pre- and post-operative images. Finally, due to the nature and quality of two-dimensional ultrasound images, it may be difficult to locate the anatomical structure of interest, leading to inaccurate segmentation of the region, and in particular to the inaccurate observation that the ablation region, when segmented, does not contain the lesion to be ablated.

[0011] Furthermore, automatic segmentation methods provide consistent results for homogeneous regions, i.e., for example, bone, blood vessels, or lesions, or when the image contains a known number of regions. In the case of ablation regions, the segmentation results obtained are inconsistent because ablation regions are typically very complex regions consisting of various materials, such as gas, necrotic cells, healthy cells, contrast agent residues, calcifications, etc. Furthermore, segmentation is typically performed on medical images, which are typically blurry and have poor contrast, making it difficult to automatically segment images.

[0012] None of the current systems makes it possible to simultaneously meet all of the required needs, in particular to provide a technique that allows for a detailed assessment of the risk of recurrence of previously ablated lesions, especially based on unclear and / or poorly contrasted medical images, without the need for the intervention of an experienced operator. Summary of the Invention [Problem to be solved by the invention]

[0013] Summary of the Invention The present invention aims to remedy some or all of the above-mentioned drawbacks of the prior art. [Means for solving the problem]

[0014] To this end, the present invention is directed to a method for post-treatment assessment of ablation of a portion of an anatomical structure of interest in an individual, the anatomical structure of interest comprising at least one lesion, the ablated portion of the anatomical structure of interest being referred to as the ablation region.

[0015] Ablation is percutaneous or minimally invasive, which generally involves inserting at least one needle through the skin to reach and destroy the lesion. Several ablation techniques are possible: radiofrequency, microwave, electroporation, laser, cryo, ultrasound, etc.

[0016] The anatomical structure of interest may be the liver, lungs, kidneys or any other organ prone to pathology.

[0017] According to the present invention, the post-treatment evaluation method comprises: - obtaining a post-operative medical image of the individual's anatomical structure of interest; - realigning the postoperative image with a medical image of the individual's anatomical structure of interest that is acquired before the surgical treatment, called a preoperative medical image, wherein the realigned preoperative medical image and the realigned postoperative medical image form a pair of medical images of the individual's anatomical structure of interest; - analyzing pairs of medical images of an anatomical structure of interest of an individual using a neural network-based machine learning method to assess the individual's risk of recurrence of a lesion of the anatomical structure of interest, the machine learning method being pre-trained in what is called a training phase on a database containing multiple pairs of medical images of the same anatomical structure of interest of a set of patients, each pair of medical images in the database being associated with a recurrence status of a lesion of the anatomical structure of interest of said patient. Includes.

[0018] Therefore, the risk of recurrence of a lesion is predicted based on previous clinical data by performing a very large-scale analysis of pairs of medical images of a given anatomical structure, for example, the liver if the anatomical structure of interest to be treated is the liver. This analysis makes it possible to determine a more accurate estimate of the risk of recurrence of the lesion to be ablated, without the need for an experienced operator. Therefore, the post-treatment evaluation method according to the present invention allows medical staff to better consider the treatment to be administered to an individual by enabling them to evaluate the need for additional treatment if the risk of recurrence is found to be significant.

[0019] The recurrence status generally takes either a so-called positive value if recurrence is observed on the postoperative examination date, or a so-called negative value if recurrence is not observed.

[0020] The risk of recurrence, for its part, generally takes the form of a probability between 0 and 1.

[0021] Advantageously, the risk of recurrence is assessed at a predetermined date after treatment, and each pair of medical images in the database is also associated with a date of recurrence if the recurrence status is positive.

[0022] This date may be, for example, 1 month, 3 months, 6 months, 1 year, 2 years, 5 years or even 10 years after the ablation treatment.

[0023] The term pre-operative medical images is understood to mean medical images acquired before an ablation treatment, and the term post-operative medical images is understood to mean medical images acquired after an ablation treatment.

[0024] In some specific implementations of the present invention, all or some of the training image pairs in the database are cropped around an ablation region containing at least one lesion after realignment, the images are cropped within a common frame of a predetermined size, and the set of centers of the ablation regions in the cropped image pairs form a set of distinct locations within the common frame.

[0025] Therefore, distributing the locations of the ablation regions within the common frame can reduce the prediction error of the machine learning method: if all of the ablation regions were in the same location within the common frame, the machine learning method would primarily consider image pairs with ablation regions at this particular location, which would result in prediction errors if the ablation regions were in different locations.

[0026] In some specific implementations of the present invention, for all pairs of previously cropped medical images in the database, the portion of the individual's body contained in the images is divided into multiple basic units of a single size, and the number of basic units is divided into two approximately equal parts between the portion of the human body defined by the ablation region and the remainder of the portion of the individual's body contained in the images.

[0027] In other words, the number of basic units is divided into two approximately equal parts between the portion of the human body that is delimited by the ablation region and the remainder of the individual's body part that is included in the cropped image pair in the database.

[0028] It should be emphasized that the even distribution between elementary units corresponding to ablated and non-ablated regions can be analyzed at the level of an image or at the level of all images as a whole.

[0029] The basic unit is commonly called a pixel in the context of a two-dimensional image or a voxel in the context of a three-dimensional image.

[0030] Approximately equal portions are understood to mean when the two sets of base units consist of the same number of base units or when the difference in the number of base units in each of the two sets is, for example, less than 5% of the number of base units in the two sets.

[0031] In some particular implementations of the present invention, the database of medical image pairs includes at least one pair of images without an ablation region.

[0032] Therefore, machine learning methods are best trained by having at least one pair of images where no ablation has been performed.

[0033] In some specific implementations of the present invention, the post-treatment evaluation method also includes determining an additional ablation mask if the risk of recurrence exceeds a predetermined threshold.

[0034] Therefore, treatment suggestions are estimated to improve the risk of recurrence using post-treatment evaluation methods. It should be emphasized that these additional treatment suggestions are non-binding and may or may not be followed by medical staff.

[0035] It should be emphasized that the additional ablation mask is a representation of the area to be ablated during additional treatment. The ablation mask is generally used to determine whether the risk of recurrence is - After detecting and segmenting the lesion and ablation areas in the pre- and post-operative images, respectively; - After determining the location of at least one recurrence is generated when the threshold value, e.g., 0.5, is exceeded.

[0036] The detection and segmentation may further be performed automatically or manually by the operator.

[0037] In some particular implementations of the present invention, determining the additional ablation mask comprises the substep of segmenting the ablation region in a post-operative image of the individual's anatomical structure of interest.

[0038] The segmentation can be performed automatically or manually by the operator. It should be emphasized that the ablation region is generally pre-detected in the image, in which case at least one operator has indicated its location in the image.

[0039] In some particular implementations of the present invention, the step of determining the additional ablation mask includes the substep of detecting an ablation region in a post-operative image of the individual's anatomical structure of interest.

[0040] In some particular implementations of the present invention, the step of determining the additional ablation mask includes the substep of detecting a lesion in a pre-operative image of the individual's anatomical structure of interest.

[0041] Thus, segmentation can be performed around lesions detected in medical images.

[0042] In some specific implementations of the present invention, the step of determining the additional ablation mask comprises: - the ablation margin between the ablation area and the lesion, - the distance between the center of mass of the lesion and the center of mass of the ablation area, - regularity and sharpness of the edges of the ablation area relative to the surrounding healthy tissue; - the ratio between the volume of the lesion and the volume of the ablation area, - Location of the lesion relative to the center of the anatomical structure of interest determining the location of recurrence in response to at least one risk predictor selected from a list comprising:

[0043] In some particular implementations of the present invention, the medical images are three-dimensional images.

[0044] It should be emphasized that a three-dimensional image may correspond, for example, to a collection of two-dimensional images taken generally at regular intervals along a predefined axis.

[0045] In some specific implementations of the present invention, each preoperative image is acquired using a first image acquisition technique, and each postoperative image is acquired using a second image acquisition technique, where the first technique and the second technique are the same or different.

[0046] In some specific implementations of the present invention, the post-treatment evaluation method also includes a step of proposing a trajectory of the medical tool to a target point in the ablation region defined by the additional ablation mask.

[0047] It should be emphasized that this step of proposing a trajectory, which may also be called the step of planning a trajectory, is carried out before any further treatment, in particular any medical intervention on the individual. It should also be emphasized that this trajectory is non-binding and may or may not be followed by medical staff.

[0048] In some specific implementations of the present invention, the post-treatment evaluation method also includes a step of assisting the operator to follow the planned trajectory of the medical instrument.

[0049] In some particular implementations of the present invention, the planned trajectory and / or guidance instructions are displayed in real time on the screen of an augmented reality device.

[0050] The invention also relates to an electronic device including a processor and a computer memory storing instructions for the method according to any one of the above-mentioned implementations.

[0051] Such electronic devices may for example be a control device, a navigation system, a robotic device or an augmented reality device. The control device may be a computer present in particular in the operating room or a remote server.

[0052] In other words, the present invention provides an electronic device including a processor and computer memory storing instructions for a method for post-treatment evaluation of ablation of a portion of an anatomical structure of interest in an individual, the anatomical structure of interest including at least one lesion, the ablated portion of the anatomical structure of interest being referred to as an ablation region, the processor, upon execution of the instructions, performing: - Obtaining post-operative medical images of the individual's anatomical structure of interest, including all or part of the ablation area; - realigning the postoperative image with a medical image of the individual's anatomical structure of interest that is obtained before the surgical treatment, called a preoperative medical image, where the realigned preoperative medical image and the realigned postoperative medical image form a pair of medical images of the individual's anatomical structure of interest; - using a neural network-based machine learning method to analyze pairs of medical images of an anatomical structure of interest of an individual to assess the individual's risk of recurrence of a lesion of the anatomical structure of interest, wherein the machine learning method is pre-trained in what is referred to as a training phase on a database containing multiple pairs of medical images of the same anatomical structure of interest of a set of patients, each pair of medical images in the database being associated with a recurrence status of a lesion of the anatomical structure of interest of the patient; The present invention relates to an electronic device configured to perform the following:

[0053] In some specific embodiments of the invention, the processor is configured to assess the risk of recurrence at a predetermined date after treatment, and each pair of medical images in the database is also associated with a date of recurrence if the recurrence status is positive.

[0054] In some specific embodiments of the present invention, the processor is further configured to, after realignment, crop all or some of the image pairs in the database around the ablation regions included in all or some of the postoperative images of the training image pairs, the images being cropped within a common frame of a predetermined size, and the set of centers of the ablation regions in the cropped image pairs forming a set of distinct locations within the common frame.

[0055] In some particular embodiments of the present invention, for all pairs of previously cropped medical images in the database, the processor is further configured to divide the portion of the individual's body included in the images into a plurality of basic units of a single size, the number of basic units being divided into two approximately equal parts between the portion of the human body delimited by the ablation region and the remainder of the portion of the individual's body included in the images.

[0056] In some particular embodiments of the present invention, the database of medical image pairs includes at least one pair of images without an ablation region.

[0057] In some particular embodiments of the invention, the processor is further configured to determine an additional ablation mask if the risk of recurrence is above a predetermined threshold.

[0058] In some particular embodiments of the invention, the processor is further configured to detect an ablation region in a post-operative image of the individual's anatomical structure of interest when determining the additional ablation mask.

[0059] In some particular embodiments of the invention, the processor is further configured to segment the ablation region in a post-operative image of the individual's anatomical structure of interest when determining the additional ablation mask.

[0060] In some particular embodiments of the invention, the processor is further configured to detect lesions in pre-operative images of the individual's anatomical structure of interest when determining the additional ablation mask.

[0061] In some particular embodiments of the invention, the processor is further configured to segment the lesion in the pre-operative image of the individual's anatomical structure of interest when determining the additional ablation mask.

[0062] In some specific embodiments of the present invention, when determining the additional ablation mask, the processor: - the ablation margin between the ablation area and the lesion, - the distance between the center of mass of the lesion and the center of mass of the ablation area, - regularity and sharpness of the edges of the ablation area relative to the surrounding healthy tissue; - the ratio between the volume of the lesion and the volume of the ablation area, - Location of the lesion relative to the center of the anatomical structure of interest and determining a location of recurrence in response to at least one risk predictor selected from the list comprising:

[0063] In some particular embodiments of the present invention, the medical image is a three-dimensional image.

[0064] In some specific embodiments of the present invention, each preoperative image is acquired using a first image acquisition technique, and each postoperative image is acquired using a second image acquisition technique, wherein the first technique and the second technique are the same or different.

[0065] Brief description of the diagram Other advantages, objects and particular features of the present invention will become apparent from the following non-limiting description of at least one particular embodiment of the device and method forming the subject of the invention, which description is made with reference to the accompanying drawings. [Brief explanation of the drawings]

[0066] [Figure 1] Schematic diagram of medical intervention. [Figure 2] Figure 1 shows an overview of the post-treatment evaluation method for medical interventions. [Figure 3] FIG. 3 is an example of a three-dimensional medical image in which the ablation region is highlighted, and this image is used in the method shown in FIG. 2. [Figure 4] 3A-3C are examples of four medical images each containing an ablation mask manually delimited by the surgeon and predicted by the neural network of the method shown in FIG. 2. [Figure 5] 3 is an example of a medical image in which the lesion and ablation regions are automatically segmented in the method steps shown in FIG. 2. [Figure 6] 10 shows an example of additional ablation as possibly suggested by the method shown in FIG. 2. DETAILED DESCRIPTION OF THE INVENTION

[0067] Detailed Description of the Invention The description is given without limitation, and each feature of one embodiment can be combined in an advantageous manner with any other feature of any other embodiment.

[0068] Please note below that the drawings are not to scale.

[0069] Example of one specific implementation 1 shows a schematic diagram of a medical intervention in which an individual 110 lying on a table 115 is treated with a medical instrument 120. In this non-limiting example of the invention, the medical intervention corresponds to the ablation of a lesion 165 in an anatomical structure of interest 130, in this case the liver of the individual 110, by the medical instrument 120, in this case a semi-rigid needle. In this case, the medical intervention is a percutaneous procedure, during which the body of the individual 110 is not opened.

[0070] The handling of the medical instrument 120 by the surgeon 140 may be advantageously guided by a guidance device, which in this non-limiting example of the invention is an augmented reality device, for example, a headset 150 worn by the surgeon 140. The medical instrument 120 may also be associated with a robotic medical device 125.

[0071] The headset 150 includes a semi-transparent screen 155, allowing the operator to see normally. Images are displayed on the screen 155 in an overlay format, displaying markers that allow the operator 140 to guide the operation of the medical instrument 120 in order to treat, by ablation, an area 160, called the ablation area, around a lesion 165 identified in the anatomical structure of interest 130. The markers may include, in particular, an ablation mask, which was previously estimated on a medical image 170 of the anatomical structure of interest 130 acquired before the procedure. The medical image 170 is hereinafter referred to as the pre-operative medical image 170.

[0072] At the end of the operation, the surgical treatment is evaluated using a method 200 for post-treatment evaluation of ablations as shown in the overview of Figure 2, the instructions of which are stored in a computer memory 180 of an electronic control unit 181 connected to the headset 155 by cable or wireless technology. The post-treatment evaluation method 200, the instructions of which are processed by a computer processor 182 of the electronic control unit 181, makes it possible to determine, in particular, the risk of recurrence associated with the lesion, verifying whether the surgical treatment carried out during the operation was sufficient or whether it is advisable to continue the treatment, for example by carrying out additional ablations.

[0073] It should be emphasized that the electronic device 181 may advantageously be integrated into the headset 150 .

[0074] The post-treatment evaluation method 200 includes a first step 210 of acquiring post-operative medical images of the anatomical structure of interest 130 .

[0075] It should be emphasized that the pre- and post-operative medical images are preferably acquired using computed tomography, although alternatively they may be acquired using magnetic resonance imaging.

[0076] The techniques used to acquire the pre-operative and post-operative medical images are preferably similar or even identical.

[0077] In other words, the techniques used to acquire the pre-operative and post-operative medical images are advantageously the same as those used to acquire the medical images in the training database of the machine learning method.

[0078] However, the techniques used to acquire post-operative medical images may differ from the techniques used to acquire pre-operative medical images.

[0079] In this case, the techniques used to acquire the pre-operative and post-operative medical images are advantageously the same as those used to acquire the medical images in the training database of the machine learning method.

[0080] The pre- and post-operative medical images are advantageously, in this non-limiting example of the invention, images acquired in three dimensions. In fact, each medical image acquired in three dimensions generally corresponds to a collection of two-dimensional medical images, each corresponding to a cross-section of the anatomical structure of interest 130 taken at regular intervals along a predetermined axis. A three-dimensional representation of the anatomical structure of interest can be reconstructed from this collection of two-dimensional medical images. The term three-dimensional image is therefore understood to mean both the collection of medical images and the three-dimensional representation. The term voxel is understood to mean the basic unit of resolution of a three-dimensional image.

[0081] Alternatively, pre-operative and post-operative medical images are each acquired in two dimensions, and therefore the basic unit of resolution for two-dimensional images is commonly referred to as a pixel.

[0082] The pre- and post-operative medical images are images that either include the entire anatomical structure of interest or are cropped around the ablation region within a predefined frame. In three-dimensional images, the frame surrounding the ablation region corresponds to a cube, while in two-dimensional images, the frame corresponds to a square.

[0083] A frame surrounding the ablation region, also known as a "bounding box," can be automatically generated around the ablation region following an action by the surgeon. Such an action can correspond, for example, to the fact that the surgeon indicates a location in a postoperative medical image that belongs to the ablation region, and a frame is generated around this location. As an example, in connection with minimally invasive ablation treatment of small lesions, i.e., lesions with a maximum diameter of, for example, about 5 cm ± 10%, or more preferably, about 3 cm ± 10%, each side of the cube or each side of the square is 5 to 10 cm.

[0084] 3 is an illustration of a three-dimensional medical image 300 in which an ablation region 310 is surrounded by a frame 320. The frame 320 is a cube and corresponds to squares 330 in cross-sectional views 340, 350, and 360 along the sagittal, axial, and coronal views, respectively.

[0085] The post-operative medical image is realigned with the pre-operative medical image 170 in a second step 220 of the method 200 shown in Figure 2. The realignment to find matches of anatomical structure locations in the two medical images is performed using methods known to those skilled in the art. The realignment can be performed rigidly, i.e., all locations in the image are changed in the same way, or loosely, i.e., each location in the image can have a specific change.

[0086] The two realigned pre- and post-operative images form a medical image pair of the anatomy of interest 130 of the individual 110 .

[0087] Therefore, in a third step 230, the medical image pairs of the anatomical structure of interest 130 are analyzed by a machine learning method, a neural network, to automatically assess the risk of recurrence associated with the ablation treatment.

[0088] To this end, the neural network is pre-trained on a database of pairs of medical images of the same anatomical structure of interest, i.e., the liver in this case, from a set of patients in a pre-training stage 290. Each pair of medical images includes a pre-operative image and a post-operative image of the anatomical structure of interest having the same features as the anatomical structure of interest 130.

[0089] Advantageously, the pairs of medical images of the anatomical structure of interest 130 of the individual 110 are acquired using the same technique as the medical images in the pairs in the training database of the neural network. In other words, the pre-operative medical images are acquired using a first medical imaging technique, and the post-operative medical images are acquired using a second medical imaging technique, which may be the same as or different from the first medical imaging technique. Using the same technique for the same type of medical images, i.e., the pre-operative and / or post-operative medical images, possibly with different parameters, makes it possible to improve the results obtained by the neural network by reducing biases associated with different medical imaging techniques.

[0090] When medical images of the anatomical structure of interest 130 of the individual 110 are cropped, the cube or square dimensions of these medical images are advantageously the same as the cube or square dimensions used to train the neural network.

[0091] To train the neural network, each pair of medical images in the database is associated with a recurrence status of the person's liver lesion, which status indicates whether recurrence has occurred, optionally associated with the date of recurrence. It should be emphasized that each of these pairs of medical images typically comes from a different person.

[0092] The neural network training stage 290 generally comprises several steps: - Training step 291; - Verification step 292; - Test Step 293 It will be carried out in.

[0093] Thus, the database of medical image pairs is divided into three databases containing separate pairs of medical images. The three databases are referred to as the training base, validation base, and test base, respectively. In this non-limiting example of the present invention, 60-98% of the medical image pairs in the database of medical images are classified into the training base, 1-20% into the validation base, and 1-20% into the test base. The percentages, which generally depend on the number of images in the database of medical images, are given herein as an indication.

[0094] The first two steps 291 and 292 of the neural network training stage 290 are the main steps that can be repeated several times. The third test step is optional for its part.

[0095] In the first step 291 of the training phase 290, the weights W and biases b for each neuron of the neural network are determined from pairs of medical images in the training base.

[0096] It should be emphasized that the training base may advantageously include image pairs without ablation regions.

[0097] Furthermore, it may be preferable that the set of images in the database contain as many voxels belonging to ablated regions as to non-ablated regions, with this percentage being calculated based on the classification of the voxels as manually determined by the surgeon.

[0098] In other words, for every pair of cropped medical images in the database, the portion of the individual's body contained in the images is divided into a number of basic units of a single size, and the number of basic units is divided into two approximately equal parts between the portion of the human body bounded by the ablation region and the remainder of the portion of the individual's body contained in the images.

[0099] Approximately equal portions are understood to mean when the two sets of base units consist of the same number of base units or when the difference in the number of base units in each of the two sets is, for example, less than 5% of the number of base units in the two sets.

[0100] Furthermore, the location of the ablation region in some of the realigned medical image pairs may be advantageously randomly shifted relative to the center of the image to avoid introducing a bias into the neural network that would learn that the ablation region is primarily the region at the center of the frame. Specifically, some medical images may be inaccurately framed by the surgeon around the ablation region. Therefore, a bounded random variable may be advantageously added to the frame location to limit this bias in locating the lesion at the center of the frame.

[0101] A second step 292 of the training phase 290 makes it possible to validate the results of the neural network, in particular the prediction error, by activating the weights W and biases b pre-determined for each neuron of the neural network from the pairs of medical images in the validation database, i.e. by comparing each pair of medical images in the validation database, i.e. the risk of recurrence obtained using the neural network, with the recurrence status associated with the pair of medical images.

[0102] If the prediction error at the end of this second step is too large, the two steps of training 291 and validation 292 are performed again to retrain the neural network by reusing the same pairs of medical images to improve the values ​​of the weights W and biases b of each neuron.

[0103] Alternatively, while retraining the neural network, a first step 291 uses resampling of medical image pairs by considering for training the medical image pairs in the training database and some of the medical image pairs in the validation base, and then validates the weights W and biases b obtained at the end of the first retraining step using the rest of the medical image pairs in the validation base.

[0104] It should be emphasized that the neural network can be retrained as many times as necessary until the prediction error is acceptable, i.e., below a predetermined value.

[0105] Once the two steps 291 and 292 of the training phase 290 have been performed at least once, the final performance of the neural network can be tested using pairs of medical images in the test base in a planned third test step 293. These pairs of medical images, which are separate from the pairs of medical images in the training and validation bases, make it possible to verify whether the neural network, as constituted by the parameters W and b for each neuron, is capable of accurately predicting the risk of recurrence in all situations that the neural network is likely to encounter. However, unlike the validation step 292, this planned third test step 293 does not result in a new training cycle for the neural network.

[0106] It should be emphasized that the images used in the testing step 293 are typically carefully selected to cover a variety of locations and sizes of ablation regions in the anatomical structure of interest in order to optimally test the predictive capabilities of the neural network.

[0107] Prediction of the risk of recurrence associated with surgical treatment is obtained directly by a neural network pre-trained on a database of pairs of medical images each associated with a recurrence status.

[0108] The associated recurrence status is, for example, equal to 0 (negative status) if there is no recurrence of the lesion within a given period after ablation, or 1 (positive status) if there is recurrence of the lesion within a given period after ablation. The period associated with the estimation of the recurrence status is preferably 6 months. However, if the recurrence status is positive, it is possible to estimate the recurrence status at various days after ablation by associating the recurrence date. For example, the neural network can also be trained to predict the recurrence status associated with a pair of medical images every 6 months after ablation, or at 1 month, 3 months, 6 months, 1 year, 2 years, 5 years, or 10 years after ablation.

[0109] The risk value for lesion recurrence generally takes the form of a probability between 0 and 1.

[0110] If the value of the risk of recurrence exceeds a predetermined threshold, i.e. possibly the probability of recurrence occurring within a given time period, an additional treatment can be suggested by the post-treatment evaluation method 200 by estimating an additional ablation mask estimated in a fourth step 240 of the post-treatment evaluation method 200. This additional non-mandatory treatment corresponds, for example, to an additional ablation in areas where the ablation margin is below a given value, for example 5 millimeters.

[0111] The step 240 of generating additional ablation masks generally includes five sub-steps, labeled 241-245 in FIG.

[0112] In sub-step 241, lesions 165 are detected in pre-operative medical images of the anatomical structure of interest 130 of the individual 110. This detection can be performed automatically or manually by the surgeon.

[0113] In sub-step 242, a second neural network generates an ablation mask around at least one lesion in the anatomical structure of interest based on learning medical images from a second database, where a majority of the medical images include ablation regions previously detected and segmented in patients exhibiting liver lesions.

[0114] Figure 4 shows four medical images 400 in this second database. Each medical image 400 was manually annotated by a surgeon who delimited an ablation region 410. The neural network generated an ablation mask 420.

[0115] It should be emphasized that for each medical image in the second database, the ablation region was advantageously manually determined by at least two surgeons to enhance the training and thus the relevance of the analysis results obtained by the second neural network. Specifically, it may be difficult for a surgeon to define the extent of the ablation region, especially when contrast in the image is poor, such as in image 400 of FIG. 4 . Therefore, using several surgeons to annotate the medical image allows for improved identification of the ablation region. Thus, in this non-limiting example of the present invention, the ablation region associated with a pair of realigned pre- and post-operative medical images corresponds to the union of ablation regions proposed by multiple surgeons. Alternatively, the ablation region associated with a medical image may correspond to the intersection, agreement, or adjudication of ablation regions proposed by multiple surgeons. The neural network is further trained to classify voxels of the medical image into ablation or non-ablative regions.

[0116] Alternatively, training can be performed using a single expert annotator who defines ablation regions in medical images, so operator experience is important to enable the second neural network to arrive at well-defined ablation regions.

[0117] It should also be emphasized that the second database may have the same advantages as the first database for limiting the training bias of the second neural network, specifically, it may include images without ablation regions, the location of the ablation regions may not be symmetrical about the center of the image, and the set of images in the database may include the same number of voxels in the ablation regions as in the non-ablation regions of the anatomical portion visible in the image.

[0118] From post-operative medical images of the individual's anatomical structure of interest 130, the second neural network classifies each voxel into an ablation region or a non-ablation region. This prediction may take the form of an ablation mask that is superimposed on the pair of medical images of the anatomical structure of interest 130. The ablation mask is typically realigned to voxels that belong to the ablation region predicted by the second neural network. It should be emphasized that the ablation mask is typically defined by a region or contour in the context of a two-dimensional image.

[0119] In sub-step 243, the lesion 165 is automatically segmented on a pre-operative medical image of the anatomical structure of interest 130 of the individual 110. Alternatively, the segmentation is performed manually by the surgeon.

[0120] The lesions are automatically segmented using methods known to those skilled in the art, for example, segmentation is performed using methods based on the histogram of the image, such as Otsu's method or deep learning methods.

[0121] This segmentation substep 243 is illustrated in Figure 5, which shows a pre-operative medical image 500 on which automatic segmentation based on deep learning methods is performed to determine the three-dimensional locations of the lesion 510 and ablation region 520. Equivalent results can be obtained using a third neural network, separate from the two previously used.

[0122] Then, in substep 244, an ablation margin is determined between the lesion segmentation and the previously established ablation mask. The ablation margin corresponds to the minimum margin, i.e., the shortest distance, allowed between the lesion segmentation and the ablation mask. In other words, the ablation margin corresponds to the shortest distance calculated between the location of the lesion and the location of the ablation region, and is calculated for all locations of the lesion.

[0123] Determining the ablation margin makes it possible to ensure that the ablation area actually covers the lesion.

[0124] 6 shows an example of additional ablation after ablation treatment at a lesion 600 at risk of recurrence. The post-treatment evaluation method 200 identified an area where the ablation margin 605 between the lesion 600 and the ablation area 610 was insufficient, resulting in the generation of an additional ablation mask 620. It should be emphasized that the additional ablation mask 620 is generated while limiting the ablation area as far as possible. Therefore, a target point 630 to be reached by the ablation needle can be further defined within the mask 620.

[0125] The predicted location of recurrence may be further evaluated in substep 245 by comparing the calculated ablation margin with reference ablation margin values ​​associated with recurrence status and stored in a database. For example, if the ablation margin is 5 mm or greater, the risk of recurrence is considered low or even zero.

[0126] Additionally or alternatively, predictors of risk of recurrence other than ablation margins may be used. The location of recurrence may be estimated by weighting all or some of these various predictors. By way of example, predictors of risk of recurrence include: - the distance between the surface of the lesion or part of the surface of the lesion and the ablation area; - the distance between the center of mass of the lesion and the center of mass of the ablation area; - the distance between the surface of the lesion or a part of the surface of the lesion and the ablation zone and, in particular in the case of subcapsular lesions, the distance between the center of mass of the lesion and the center of mass of the ablation zone, taking into account the proximity of the capsule of the anatomical structure of interest; - regularity of the edges of the ablation area relative to the surrounding healthy tissue; - Sharpness of the margins of the ablation area relative to the surrounding healthy tissue; - the ratio between the volume of the lesion and the volume of the ablation area; - Location of the lesion in the anatomical structure of interest It could be.

[0127] The reference value for the center of mass depends on the ablation margin: if the ablation margin is equal to 10 mm and the reference value for the ablation margin is 5 mm, the distance between the center of mass of the lesion and the center of mass of the ablation region must be 5 mm or less.

[0128] Method 200 also includes step 250 of planning a trajectory to be followed by medical instrument 120, associated with either the ablation mask or an additional ablation mask, and step 260 of guiding medical instrument 120 along the planned trajectory, which may guide the surgeon when handling medical instrument 120.

[0129] An example of a planning method is described in French patent application No. 1 914 780 entitled "Methode de planification automatique d'une trajectoire pour une intervention medicale" [Method for automatically planning a trajectory for a medical intervention].

[0130] It should be emphasized that guidance, in this non-limiting example of the present invention, is visual and includes displaying the planned trajectory and / or guidance instructions on the screen 155 of the headset 150.

[0131] Alternatively, the medical instrument 120 may be guided by a navigation system that provides position and orientation information for the medical instrument 120. This may include mechanical guidance using a robotic device coupled to such a navigation system.

[0132] It should be emphasized that steps 230-260 may be repeated until the risk of recurrence is zero or nearly zero or until the ablation margin is sufficient.

Claims

1. 1. A method (200) for post-treatment assessment of an ablation of a portion of an anatomical structure of interest (130) of an individual (110), said anatomical structure of interest comprising at least one lesion (165, 510, 600), said ablated portion of said anatomical structure of interest being referred to as an ablation region (160), comprising: acquiring (210) a post-operative medical image of the anatomical structure of interest of the individual, including all or a portion of the ablation region; applying image registration between a postoperative image and a medical image (170, 500) of the anatomical structure of interest of the individual, which is acquired before the surgical treatment, referred to as a preoperative medical image, wherein the preoperative medical image to which the image registration has been applied and the postoperative medical image to which the image registration has been applied form a pair of medical images of the anatomical structure of interest of the individual; inputting the pair of medical images of the anatomical structure of interest of the individual into a neural network and outputting a risk of recurrence of a lesion of the anatomical structure of interest of the individual, wherein the neural network has been pre-trained in what is called a training phase (290) on a database containing a plurality of pairs of medical images (300, 400) of the same anatomical structure of interest of a set of patients, each pair of medical images in the database being associated with a recurrence status of a lesion of the anatomical structure of interest of the patient, and the neural network is trained through comparison of each pair of medical images in the database with the recurrence status associated with each pair of medical images until the prediction error of the neural network is acceptable; A method (200) comprising:

2. 2. The post-treatment evaluation method of claim 1, wherein the risk of recurrence is assessed at a predetermined date after the surgical treatment, and each pair of medical images in the database is also associated with a date of recurrence if the recurrence status is positive.

3. 3. The post-treatment evaluation method according to claim 1 or 2, wherein after image registration, all or some of the pairs of medical images in the database are cropped around the ablation regions included in the post-operative images of all or some of the pairs of medical images, the medical images are cropped within a common frame of a predetermined size, and a set of centers of the ablation regions in the cropped pairs of medical images form a set of distinct locations within the common frame.

4. 4. The post-treatment evaluation method of claim 3, wherein for all pairs of previously cropped medical images in the database, the portion of the individual's body included in the medical images is divided into a plurality of basic units of a single size, and the number of basic units is divided into two approximately equal parts between the portion of the human body defined by the ablation region and the remaining portion of the portion of the individual's body included in the medical images.

5. The post-treatment evaluation method according to any one of claims 1 to 4, wherein said database of pairs of medical images includes at least one pair of images without an ablation region.

6. The post-treatment evaluation method according to any one of claims 1 to 5, further comprising the step of determining an additional ablation mask if the risk of recurrence exceeds a predetermined threshold.

7. 7. The post-treatment evaluation method of claim 6, wherein the step of determining an additional ablation mask comprises the substep of detecting the ablation region in the post-operative image of the anatomical structure of interest of the individual.

8. 8. The post-treatment evaluation method according to claim 6 or 7, wherein the step of determining an additional ablation mask comprises the sub-step of segmenting the ablation region in the post-operative image of the anatomical structure of interest of the individual.

9. 9. The post-treatment evaluation method according to claim 6, wherein the step of determining an additional ablation mask comprises the sub-step of detecting the lesion in a pre-operative image of the anatomical structure of interest of the individual.

10. 10. The post-treatment evaluation method according to claim 6, wherein the step of determining an additional ablation mask comprises a sub-step of segmenting the lesion in a pre-operative image of the anatomical structure of interest of the individual.

11. The step of determining an additional ablation mask comprises: an ablation margin between the ablation region and the lesion; the distance between the center of mass of the lesion and the center of mass of the ablation region; the regularity and sharpness of the edges of the ablation area relative to the surrounding healthy tissue; the ratio between the volume of the lesion and the volume of the ablation zone; The location of the lesion relative to the center of the anatomical structure of interest 10. The post-treatment evaluation method according to any one of claims 6 to 9, comprising a sub-step of determining the location of the recurrence depending on at least one risk predictor selected from the list comprising:

12. The post-treatment evaluation method according to any one of claims 1 to 11, wherein the medical image is a three-dimensional image.

13. 13. The post-treatment evaluation method according to any one of claims 1 to 12, wherein each pre-operative image is acquired using a first image acquisition technique and each post-operative image is acquired using a second image acquisition technique, and the first image acquisition technique and the second image acquisition technique are the same or different.

14. An electronic device (150) comprising a processor and a computer memory storing instructions for a post-treatment assessment method according to any one of claims 1 to 13.

15. 15. The electronic device of claim 14, which may be a control device, a navigation system, a robotic device or an augmented reality device.

Images