METHOD FOR AUTOMATED DETECTION OF A BIOLOGICAL ELEMENT IN A TISSUE SAMPLE

Abstract

To automatically detect a predetermined biological element in a tissue sample of a human or animal being: at least one artificial neural network is fed (12) with a plurality of machine learning images of the biological element; an image of the tissue sample is processed so as to extract (22) therefrom regions of interest, where the biological element is to be detected; each region of interest is subdivided (24) into a plurality of thumbnail images; in each thumbnail image, using the at least one artificial neural network, a prediction of detection of the biological element is automatically obtained (26); and the image of the tissue sample of the individual is reconstructed (28) comprising the predictions of detection of the biological element. Figure for the abstract: Fig. 1

Description

Title of the invention: METHOD FOR THE AUTOMATED DETECTION OF A BIOLOGICAL ELEMENT IN A TISSUE SAMPLE

[0001] The present invention relates to a method for the automated detection of a predetermined biological element in a tissue sample taken from a human or animal being.

[0002] The invention belongs to the medical field. It finds application in particular, but not exclusively, in the field of determining the density of intraepidermal nerve fibers.

[0003] Small fiber neuropathy (SFN) is characterized by sensory symptoms in the lower extremities, pain, and a quantitative deficit of small nerve fibers.

[0004] Skin biopsy with assessment of intraepidermal nerve fiber density (IENFD) is currently considered the best technique to enable the practitioner to diagnose SFN.

[0005] However, this technique involves manually counting intraepidermal nerve fibers, which is very time-consuming even for an experienced operator and also results in high variability of results, as well as errors.

[0006] More generally, when it comes to detecting a predetermined biological element in a tissue sample, there is a need for speed, reproducibility, robustness and reliability of the detection technique.

[0007] The present invention aims to remedy the aforementioned drawbacks of the prior art.

[0008] For this purpose, the present invention proposes a method for the automated detection of a predetermined biological element in a tissue sample of a human or animal being, remarkable in that it comprises steps consisting of: obtaining a plurality of machine learning images of the biological element, from a plurality of tissue samples of at least one subject; feeding at least one artificial neural network with the plurality of machine learning images; obtaining a stack of images, along a plurality of parallel planes, of the tissue sample of the human or animal being; concatenate the stack images into a single image; perform automatic edge detection in the single image, apply the detected edges to each image in the stack; extract from each image of the stack a region of interest, where the biological element is to be detected; subdivide each region of interest into a plurality of thumbnails; in each image, using at least one artificial neural network, automatically obtain a prediction of detection of the biological element; combining each plurality of thumbnails comprising the detection predictions into an image of each region of interest, then stacking the images of each region of interest, so as to obtain a reconstructed image of the tissue sample of the human or animal being comprising the detection predictions of the biological element.

[0009] The main advantage of the method according to the invention is that it is automatic. Automatic detection proves to be highly accurate and offers a significant time saving compared to current manual techniques. In addition, the successive steps of decomposition and analysis of images implemented by the method according to the invention have the originality of allowing the detection of biological elements of a size much smaller than those likely to be detected by conventional image processing techniques.

[0010] In a particular embodiment, the step of obtaining a stack of images, along a plurality of parallel planes, of the tissue sample of the human or animal being comprises the use of an immunofluorescence scanner.

[0011] This type of device makes it possible to obtain stacks of images in several dozen planes in a simple and direct manner.

[0012] In a particular embodiment, the automatic contour detection step comprises image blurring, dilation and erosion steps.

[0013] Conventional signal processing in computer vision involving blurring, dilation and erosion makes it possible to attenuate local artifacts typically present in medical imaging.

[0014] In a particular embodiment, the step of extracting the region of interest implements a semantic segmentation algorithm, i.e. the pixel-wise detection of the region of interest.

[0015] In a particular embodiment, each detection prediction is materialized by a rectangular box which is the smallest rectangle containing the image of the biological element whose detection is predicted.

[0016] In a particular embodiment, the step of automatically obtaining the detection prediction using the at least one artificial neural network comprises a step of calculating a posteriori probability for images.

[0017] In this embodiment, according to a possible particular characteristic, the step The calculation of a posteriori probability for images uses a Kalman filter to compensate for noisy measurements, which is often the case for medical imaging.

[0018] This helps to smooth and improve the results of semantic segmentation.

[0019] In a particular embodiment, the tissue is the skin and the biological element is an intraepidermal nerve fiber.

[0020] The invention is in fact particularly advantageously applicable to the automatic detection of intraepidermal nerve fibers in the context of the search for small nerve fibers. Brief description of the drawings

[0021] Other aspects and advantages of the invention will appear on reading the detailed description below of particular embodiments, given as non-limiting examples, with reference to the appended drawings, in which:

[0022] [Fig-1] is a flowchart illustrating the main steps of a process in accordance with the present invention, in a particular embodiment.

[0023] [Fig.2] is a flowchart illustrating the use of a Kalman filter applied to images, included in a particular embodiment of the method according to the present invention. Description of embodiment(s)

[0024] The method according to the invention implements an artificial neural network. It is therefore necessary to provide machine learning data to this artificial neural network so that it can operate.

[0025] Thus, as shown in the flowchart of [Fig. 1], in a particular embodiment, a method, in accordance with the present invention, for automated detection of a predetermined biological element in a tissue sample of a human or animal being comprises a first step 10 consisting of obtaining a plurality of machine learning images of the biological element, from a plurality of biological tissue samples of one or more subjects.

[0026] By way of non-limiting example, the invention will be described in its application to the automated detection of intraepidermal nerve fibers.

[0027] In this example, the tissue considered is the skin and the biological element considered is a nerve fiber.

[0028] The plurality of tissue samples are typically obtained by taking samples from one or more subjects. In the non-limiting example described herein, the tissue samples may be obtained by skin biopsy from one or more locations, for example, the ankle, thigh, and / or wrist, if the subject is a human.

[0029] Machine learning images are obtained, for example, using a medical imaging device such as an immunofluorescence scanner. They are label so as to identify the biological element and possibly a number of other constituent elements, or biomarkers, of the tissue samples. In the non-limiting example described herein, elements such as the basement membrane, small nerve fibers, the dermis and nerve fibers crossing the dermo-epidermal intersection are identified in each image.

[0030] In a subsequent step 12 of the method according to the invention, these machine learning images are provided as input to one or more artificial neural networks. Different neural networks may be used and multiple versions of each machine learning method are advantageously used. Different labels may be applied to the training data depending on the chosen neural network model and for each model, different inputs, different representations of the data and / or different training parameters may be used.

[0031] Then, during a step 14, a tissue sample from a specific human or animal is considered, where the predetermined biological element is to be detected. Using the same medical imaging device, for example an immunofluorescence scanner, a stack of images of the tissue sample from this human or animal is obtained, along a plurality of parallel planes, for example 20 planes. In a particular embodiment, grayscale images can be obtained. Alternatively, color images can be obtained. In this case, it is possible to obtain several stacks of images, including one per channel of different color, each of these channels allowing the detection of a constituent element, or biomarker, different from the tissue sample. By way of non-limiting example, the different channels can be, for example, a green channel and a red channel.

[0032] In the non-limiting example described herein, the green and red channels improve the detection of biomarkers. The green channel facilitates the detection of the epidermis-dermis region, while the red channel improves the detection of nerve fibers. Generally, the use of multiple colors improves the robustness of detection algorithms in medical imaging, but remains optional.

[0033] At this stage of the method, the so-called whole slide images in immunofluorescence (in English "fluorescence whole slide imaging") are very large and not usable in view of the small size of the biological element to be detected. This is why, then, during a step 16, the images of the stack are concatenated into a single image. In the case where there are several channels, the images of each stack are concatenated, then all the images thus concatenated into a single image. This single image is obtained in order to automatically identify sections in the whole slide images, the objective being to create a coarse-mesh model to quickly detect the sections made in the tissue sample, knowing that the same slide is likely to contain several cups.

[0034] The next step 20 consists of automatically detecting the contours in this single image. For this purpose, the image can optionally be resized, for example by a factor of 0.1, which allows computer vision techniques to be applied. This step 20 of automatic contour detection can include steps of blurring to attenuate local artifacts, image dilation and image erosion involving thresholding in order to eliminate noise. The automatic contour detection can for example implement an algorithm of the type available in the OpenCV graphics library.

[0035] Once the contours have been detected in the resized image, the detected contours can be enlarged and applied to each concatenated image (for each of the aforementioned channels if there are several channels), then to each image in the stack (for each of the channels if there are several channels).

[0036] Since the images are still too large to allow the desired biological element to be detected, the next step 22 consists of extracting from each image of the stack (for each of the channels if there are several channels) a region of interest, in which the biological element is to be detected. In the non-limiting example described here, the region of interest represents the intraepidermal region, where the analysis of the nerve fibers must be carried out. To accelerate the process of detecting the region of interest, it is possible, for example, to use images of size 256 x 256 pixels.

[0037] Step 22 of extracting the region of interest can for example implement a semantic segmentation algorithm and the result of this segmentation can be enlarged before proceeding with the analysis of the region of interest.

[0038] For this analysis, during the following step 24, each region of interest is subdivided into a plurality of imagettes (in English “patches”).

[0039] In the following step 26, in each image, using the artificial neural network(s), a prediction of detection of the predetermined biological element is automatically obtained, i.e. a prediction of detection of the nerve fibers in the non-limiting example described here.

[0040] In the non-limiting example described here, to accelerate this process, three semantic segmentation algorithms known per se are applied in parallel, to respectively perform a segmentation of the nerve fibers, a segmentation of the basal membrane and a segmentation of the dermis zone. In order to improve the robustness and reproducibility of the prediction, it is possible to use techniques, known per se, of data augmentation which enrich the data set, by randomly creating images which have undergone, for example, a partial cutting and / or a rotation and / or a modification of the colors.

[0041] In a particular embodiment, one can take advantage of the partial properties particular elements of a constituent element, or biomarker, of the tissue studied to apply to it, during step 26 of obtaining the detection predictions, an a posteriori probability calculation. In the non-limiting example described here, the basement membrane has the property of generally being a continuous line in each plane among the parallel planes of the stack. It is advantageous to carry out an a posteriori probability calculation step on the results of the segmentation of the basement membrane, making it possible to greatly reduce the noise in an image. The use of the Kalman filter for the images, which can be implemented in a particular embodiment of the method according to the present invention, is innovative in this respect. As an alternative to the Kalman filter, the method can include any other process making it possible to denoise the data.

[0042] As shown in [Fig.2], this optional step of calculating a posteriori probability can for example implement a Kalman filter. For this, all the prediction results for the basal membrane are used, on the two-dimensional images according to all the parallel planes of the stack (it is assumed in the drawing as a non-limiting example that there are Z planes designated by z = -10, z = -9, ..., z = -1, z = 1, ..., z= 9, z = 10). These prediction results (the “predicted 2D images” 200 in the drawing) are concatenated so as to obtain a three-dimensional image, for example of size 512x512x20 pixels, three-dimensional thumbnails having a specific kernel size are created in this image by normalizing the images during a step 202 (using for example a logistic function for the transformation of the images) and a Kalman filter is applied to each three-dimensional thumbnail.The size of each three-dimensional image is chosen to be adapted to the computational cost of the Kalman filter and knowing that only the pixels of the nearest neighbors will be necessary to reconstruct the image of the basement membrane.

[0043] In step 204, the initial state of the Kalman filter is chosen not randomly, but by using the average of the predictions of each three-dimensional image along the axis of the stack. In other words, we work on two-dimensional images which correspond to the different layers of the stack. After applying the Kalman filter to each image during a step 206, we have new prediction results (the “new predicted 2D images” 208 in the drawing) and we reconstruct the three-dimensional image with an updated and therefore improved prediction for each three-dimensional image according to its closest plane in the stack, that is to say we use the prediction state of a layer of the stack to predict the prediction state of the closest layer.

[0044] In the non-limiting example described here, from the prediction of detection of the basement membrane, it is possible to obtain a detection of the dermis, by combining the prediction of detection of the basement membrane with techniques known per se for vision by computer.

[0045] At the end of step 26 of automatically obtaining the detection predictions, a step 28 of the method according to the invention is carried out, in which each plurality of thumbnails comprising the detection predictions is combined into an image of each region of interest, then the images of each region of interest are stacked, so as to obtain a reconstructed image of the tissue sample of the human or animal being examined, comprising the detection predictions of the predetermined biological element, i.e. the nerve fibers in the non-limiting example described here.

[0046] In the non-limiting example described herein, the detection predictions from the semantic segmentations of the nerve fibers, the basement membrane, and the dermis are combined into a single image, where a different color channel represents the prediction of each of these constituent elements, or biomarkers, of the sample: for example, a red channel represents the prediction of the nerve fibers, a green channel represents the prediction of the basement membrane, and a blue channel represents the prediction of the dermis.

[0047] In a particular embodiment, each prediction of detection of the predetermined biological element is materialized by a rectangular box (in English "bounding box"), which is the smallest rectangle containing the image of the biological element whose detection is predicted.

[0048] At the start of step 28 of obtaining a reconstructed image of the sample comprising the detection predictions, the rectangular boxes are present in the two-dimensional thumbnails of the stacks along the plurality of parallel planes. Then all the thumbnails are combined, that is to say all the predictions expressed by the rectangular boxes are stacked, in order to recreate a section of the tissue sample. The pairwise distance is then calculated for each prediction, that is to say for each rectangular box, that is to say the distance between the rectangular boxes taken two by two. A matrix of the pairwise distances is thus obtained. By considering the stack of the rectangular boxes, the three-dimensional intersection corresponding to the crossing of the rectangular boxes over the entire stack is thus obtained.

[0049] For example, one can decide that the closer two rectangular boxes are to each other, the closer their pairwise distance is to 1 and that conversely, a value close to 0 means that the two rectangular boxes of the pair considered are not at all close to each other. Using unsupervised learning techniques and clustering techniques, one can then cluster the rectangular boxes that have been calculated to be close to each other and thus arrive at a map of the tissue sample locating each occurrence of the predetermined biological element, in accordance with the detection predictions obtained.

Claims

Claims

1. A method for automated detection of a predetermined biological element in a tissue sample of a human or animal, said method comprising steps consisting of: obtaining (10) a plurality of machine learning images of said biological element, from a plurality of tissue samples of at least one subject; feeding (12) at least one artificial neural network with said plurality of machine learning images; obtaining (14) a stack of images, along a plurality of parallel planes, of said tissue sample of said human or animal; concatenating (16) the images of said stack into a single image; performing (20) automatic contour detection in said single image, applying the detected contours to each image of said stack; extracting (22) from each image of said stack a region of interest, where said biological element is to be detected;subdividing (24) each region of interest into a plurality of thumbnails; in each thumbnail, using said at least one artificial neural network, automatically obtaining (26) a detection prediction of said biological element, the step (26) of automatically obtaining said detection prediction using said at least one artificial neural network comprising a step of calculating a posteriori probability for images; combining each plurality of thumbnails comprising said detection predictions into an image of each region of interest, then stacking the images of each region of interest, so as to obtain (28) a reconstructed image of said tissue sample of said human being or animal comprising said detection predictions of said biological element, said method being characterized in that said step of calculating a posteriori probability for images implements a process of denoising the image data.;

2. Method according to claim 1, characterized in that the step (14) of obtaining a stack of images, along a plurality of parallel planes, of said tissue sample of said human or animal being comprises the use of an immunofluorescence scanner.

3. Method according to claim 1 or 2, characterized in that step (20) Automatic edge detection includes steps of image routing, dilation and erosion.

4. Method according to claim 1, 2 or 3, characterized in that the step (22) of extracting said region of interest implements a semantic segmentation algorithm.

5. Method according to any one of the preceding claims, characterized in that each detection prediction is materialized by a rectangular box which is the smallest rectangle containing the image of said biological element whose detection is predicted.

6. Method according to claim 6, characterized in that said step of calculating a posteriori probability for images implements a Kalman filter.

7. Method according to any one of the preceding claims, characterized in that said tissue is skin and said biological element is an intraepidermal nerve fiber.

Images