Method for automatically detecting a biological element in a tissue sample

EP4469971A1Pending Publication Date: 2024-12-04CAPGEMINI CONSULTING +3
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
EP2023703635
Authority / Receiving Office
EP · EP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-01-24
Filing Date
2023-01-17
Publication Date
2024-12-04

AI Technical Summary

Technical Problem

Current methods for detecting biological elements in tissue samples, such as intraepidermal nerve fibers, are time-consuming, prone to high variability, and error-prone due to manual counting, lacking speed, reproducibility, and reliability.

Method used

An automated method using machine learning images, artificial neural networks, and image processing techniques like edge detection, semantic segmentation, and a posteriori probability calculation with a Kalman filter to accurately detect biological elements, particularly intraepidermal nerve fibers, by obtaining and processing stacks of images from tissue samples.

Benefits of technology

The method provides highly accurate, efficient, and reproducible detection of biological elements, including smaller sizes not detectable by conventional techniques, significantly reducing time and improving reliability compared to manual methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 1.1
    Figure 1.1
Patent Text Reader

Abstract

The invention relates to a method for automatically detecting a predetermined biological element in a human or animal tissue sample, which method comprises: feeding (12) at least one artificial neural network with a plurality of machine-learning images of the biological sample; processing an image of the tissue sample such as to extract (22) therefrom regions of interest in which the biological element is to be detected; subdividing (24) each region of interest into a plurality of patches; in each patch, using the at least one artificial neural network to automatically obtain (26) a prediction for the detection of the biological element; and reconstructing (28) the image of the tissue sample from the individual, comprising the detection predictions of the biological element.
Need to check novelty before this filing date? Find Prior Art

Description

Description Title of the invention: METHOD FOR AUTOMATED DETECTION OF A BIOLOGICAL ELEMENT IN A TISSUE SAMPLE

[0001] The present invention relates to a method for the automated detection of a predetermined biological element in a tissue sample taken from a human or animal being.

[0002] The invention belongs to the medical field. It is particularly applicable, but not exclusively so, in the field of determining the density of intraepidermal nerve fibers.

[0003] Small fiber neuropathy (SFN) is characterized by sensory symptoms in the lower extremities, pain, and a quantitative deficit of small nerve fibers.

[0004] Skin biopsy with assessment of intraepidermal nerve fiber density (IENFD) is currently considered the best technique to allow the practitioner to diagnose SFN.

[0005] However, this technique involves manually counting intraepidermal nerve fibers, which is very time-consuming even for an experienced operator and also leads to high variability in results, as well as errors.

[0006] More generally, when it comes to detecting a predetermined biological element in a tissue sample, there is a need for speed, reproducibility, robustness and reliability of the detection technique.

[0007] The present invention aims to remedy the aforementioned drawbacks of the prior art.

[0008] To this end, the present invention proposes a method for the automated detection of a predetermined biological element in a tissue sample from a human or animal, notable in that it comprises steps consisting of: to obtain a plurality of machine-learning images of the biological element from a plurality of tissue samples from at least one subject; to feed at least one artificial neural network with the plurality of machine-learning images; to obtain a stack of images, along a plurality of parallel planes, of the human or animal tissue sample; to concatenate the images of the stack into a single image; to perform automatic edge detection in the single image, to apply the detected edges to each image of the stack; to extract from each image of the stack a region of interest, where the biological element is to be detected; to subdivide each region of interest into a plurality of imagelets; in each imagelet, using the at least one artificial neural network, to automatically obtain a prediction of the detection of the biological element;combine each plurality of imagettes containing the detection predictions into an image of each region of interest, then stack the images of each region of interest, so as to obtain a reconstructed image of the human or animal tissue sample containing the detection predictions of the biological element.

[0009] The main advantage of the method according to the invention is its automation. Automatic detection proves to be highly accurate and offers significant time savings compared to current manual techniques. Furthermore, the successive image decomposition and analysis steps implemented by the method according to the invention are unique in that they allow the detection of biological elements much smaller than those detectable by conventional image processing techniques.

[0010] In a particular embodiment, the step of obtaining a stack of images, following a plurality of parallel planes, of the human or animal tissue sample involves the use of an immunofluorescence scanner.

[0011] This type of device allows for the simple and direct creation of image stacks following several dozen planes.

[0012] In a particular embodiment, the automatic edge detection step includes image blurring, dilation and erosion steps.

[0013] Classical signal processing in computer vision involving blurring, dilation and erosion helps to mitigate local artifacts typically present in medical imaging.

[0014] In a particular embodiment, the region of interest extraction step implements a semantic segmentation algorithm, i.e., pixel-by-pixel detection of the region of interest.

[0015] In a particular embodiment, each detection prediction is materialized by a rectangular box which is the smallest rectangle containing the image of the biological element whose detection is predicted.

[0016] In a particular embodiment, the step of automatically obtaining the detection prediction using at least one artificial neural network includes a step of calculating a posteriori probability for images.

[0017] In this embodiment, according to a particular possible characteristic, the post-hoc probability calculation step for images implements a Kalman filter to compensate for noisy measurements, which is often the case for medical imaging.

[0018] This helps to smooth and improve the results of semantic segmentation.

[0019] In one particular embodiment, the tissue is the skin and the biological element is an intraepidermal nerve fiber.

[0020] The invention is particularly advantageously applicable to the automatic detection of intraepidermal nerve fibers in the context of small nerve fiber research. Brief description of the drawings

[0021] Other aspects and advantages of the invention will become apparent from the following detailed description of particular embodiments, given by way of non-limiting examples, with reference to the accompanying drawings, in which:

[0022] [Fig. 1] is a flowchart illustrating the main steps of a process according to the present invention, in a particular embodiment.

[0023] [Fig. 2] is a flowchart illustrating the use of a Kalman filter applied to images, included in a particular embodiment of the method according to the present invention. Description of method(s) of implementation

[0024] The method according to the invention implements an artificial neural network. It is therefore necessary to provide machine learning data to this artificial neural network in order for it to function.

[0025] Thus, as shown in the flowchart of Figure 1, in a particular embodiment, a method, according to the present invention, for the automated detection of a predetermined biological element in a tissue sample from a human or animal includes a first step 10 consisting of obtaining a plurality of machine learning images of the biological element, from a plurality of biological tissue samples from one or more subjects.

[0026] By way of non-limiting example, the invention will be described in its application to the automated detection of intraepidermal nerve fibers.

[0027] In this example, the tissue considered is the skin and the biological element considered is a nerve fiber.

[0028] Multiple tissue samples are typically obtained by taking samples from one or more subjects. In the non-limiting example described here, tissue samples may be obtained by skin biopsy from one or more locations, for example, the ankle, thigh, and / or wrist, if the subject is human.

[0029] Machine learning images are obtained, for example, using a medical imaging device such as an immunofluorescence scanner. They are labeled to identify the biological element and possibly a number of other constituent elements, or biomarkers, of the tissue samples. In the non-limiting example described here, elements such as the basement membrane, the small nerve fibers, the dermis and nerve fibers crossing the dermo-epidermal intersection.

[0030] In a subsequent step 12 of the method according to the invention, these machine learning images are provided as input to one or more artificial neural networks. Different neural networks can be used, and advantageously, multiple versions of each machine learning method are employed. Different labels can be applied to the training data depending on the chosen neural network model, and for each model, different inputs, different data representations, and / or different learning parameters can be used.

[0031] Then, in step 14, a tissue sample from a specific human or animal is considered, in which the predetermined biological element is to be detected. Using the same medical imaging device, for example, an immunofluorescence scanner, a stack of images of the tissue sample from this human or animal is obtained along a plurality of parallel planes, for example, 20 planes. In a particular embodiment, grayscale images can be obtained. Alternatively, color images can be obtained. In this case, several image stacks can be obtained, one for each different color channel, each of these channels enabling the detection of a different constituent, or biomarker, of the tissue sample. By way of non-limiting example, the different channels could be, for example, a green channel and a red channel.

[0032] In the non-limiting example described here, the green and red channels improve biomarker detection. The green channel facilitates detection of the epidermis-dermis region, while the red channel enhances the detection of nerve fibers. Generally, the use of multiple colors improves the robustness of detection algorithms in medical imaging, but remains optional.

[0033] At this stage of the process, the so-called whole-slide immunofluorescence images are very large and unusable given the small size of the biological element to be detected. Therefore, in step 16, the images are concatenated. The stacking is combined into a single image. If there are multiple channels, the images from each stack are concatenated, and then all the concatenated images are combined into a single image. This single image is used to automatically identify sections within the whole-slide images, the goal being to create a high-resolution model to quickly detect sections within the tissue sample, given that a single slide may contain multiple sections.

[0034] The next step, step 20, involves automatically detecting edges in this single image. To this end, the image can optionally be resized, for example by a factor of 0.1, which allows for the application of computer vision techniques. This automatic edge detection step 20 may include blurring steps to mitigate local artifacts, image dilation, and image erosion involving thresholding to eliminate noise. Automatic edge detection can, for example, implement an algorithm of the type available in the OpenCV graphics library.

[0035] Once the contours are detected in the resized image, we can enlarge the detected contours and apply them to each concatenated image (for each of the aforementioned channels if there are multiple channels), and then to each image in the stack (for each of the channels if there are multiple channels).

[0036] Since the images are still too large to detect the desired biological element, the next step (22) consists of extracting a region of interest from each image in the stack (for each channel if there are multiple channels), where the biological element is to be detected. In the non-limiting example described here, the region of interest represents the intraepidermal region, where the nerve fiber analysis is to be performed. To accelerate the process of detecting the region of interest, images of size 256 x 256 pixels can be used, for example.

[0037] Step 22 of extracting the region of interest can, for example, implement a semantic segmentation algorithm, and the result of this segmentation can be enlarged before proceeding with the analysis of the region of interest.

[0038] For this analysis, in the next step 24, each region of interest is subdivided into a plurality of image patches.

[0039] In the next step 26, in each image, using the artificial neural network(s), we automatically obtain a prediction of detection of the predetermined biological element, that is to say a prediction of detection of nerve fibers in the non-limiting example described here.

[0040] In the non-limiting example described here, to accelerate this process, three well-known semantic segmentation algorithms are applied in parallel to perform nerve fiber segmentation, basement membrane segmentation, and dermal segmentation, respectively. To improve the robustness and reproducibility of the prediction, well-known data augmentation techniques can be used to enrich the dataset by randomly creating images that have undergone, for example, partial cropping, rotation, and / or color modification.

[0041] In a particular embodiment, the specific properties of a constituent element, or biomarker, of the tissue under study can be exploited to apply a posteriori probability calculation to it during step 26 of obtaining detection predictions. In the non-limiting example described here, the basement membrane is generally a continuous line in each plane among the parallel planes of the stack. It is advantageous to perform a posteriori probability calculation on the results of the basement membrane segmentation, as this greatly reduces noise in an image. The use of the Kalman filter for images, which can be implemented in a particular embodiment of the method according to the present invention, is therefore innovative. As an alternative to the Kalman filter, the method can include any other process for denoising the data.

[0042] As Figure 2 shows, this optional posterior probability calculation step can, for example, implement a Kalman filter. To do this, all the prediction results for the basement membrane are used on the two-dimensional images along all parallel planes of the stacking (in the diagram, as a non-limiting example, we assume there are Z planes designated by z = -10, z = -9, ..., z = -1, z = 1, ..., z = 9, z = 10). These prediction results (the "predicted 2D images" in the diagram) are concatenated to obtain a three-dimensional image, for example, of size 512 x 512 x 20 pixels. This image consists of three-dimensional images with a specific kernel size, obtained by normalizing the images during a step 202 (using, for example, a logistic function for image transformation), and a Kalman filter is applied to each three-dimensional image. The size of each three-dimensional image is chosen to be suitable for the computational cost of the Kalman filter, knowing that only the nearest-neighbor pixels will be needed to reconstruct the image of the basement membrane.

[0043] In step 204, the initial state of the Kalman filter is chosen not randomly, but by using the average of the predictions of each three-dimensional image along the stacking axis. In other words, we work with two-dimensional images that correspond to the different layers of the stack. After applying the Kalman filter to each image in step 206, we obtain new prediction results (the "new predicted 2D images" 208 in the diagram) and we reconstruct the three-dimensional image with an updated and therefore improved prediction for each three-dimensional image according to its nearest plane in the stack; that is, we use the prediction state of one layer of the stack to predict the prediction state of the nearest layer.

[0044] In the non-limiting example described here, from the prediction of basement membrane detection, one can obtain dermal detection, by combining the prediction of basement membrane detection with techniques known in themselves from computer vision.

[0045] At the end of step 26 of automatic obtaining of detection predictions, a step 28 of the method according to the invention is carried out, in which each plurality of imagettes containing the detection predictions is combined into an image of each region of interest, then the images of each region of interest are stacked, so as to obtain a reconstructed image of the tissue sample of the human or animal being examined, containing the detection predictions of the predetermined biological element, i.e. nerve fibers in the non-limiting example described here.

[0046] In the non-limiting example described here, detection predictions derived from semantic segmentations of nerve fibers, the basal membrane, and the dermis are combined into a single image, where a different colored channel represents the prediction of each of these constituent elements, or biomarkers, of the sample: for example, a red channel represents the prediction of nerve fibers, a green channel represents the prediction of the basement membrane and a blue channel represents the prediction of the dermis.

[0047] In a particular embodiment, each prediction of detection of the predetermined biological element is materialized by a rectangular box (in English "bounding box"), which is the smallest rectangle containing the image of the biological element whose detection is predicted.

[0048] At the beginning of step 28, which involves obtaining a reconstructed image of the sample containing the detection predictions, rectangular boxes are present in the two-dimensional imagelets of the stacks along the plurality of parallel planes. Then, all the imagelets are combined; that is, all the predictions expressed by the rectangular boxes are stacked to recreate a cross-section of the tissue sample. For each prediction, i.e., for each rectangular box, the pairwise distance is then calculated—that is, the distance between any two rectangular boxes. This yields a matrix of pairwise distances. By considering the stacking of the rectangular boxes, we thus obtain the three-dimensional intersection corresponding to the crossing of the rectangular boxes across the entire stack.

[0049] For example, we can decide that the closer two rectangular boxes are to each other, the closer their pairwise distance will be to 1, and conversely, a value close to 0 means that the two rectangular boxes in the pair considered are not close to each other at all. Using unsupervised learning and clustering techniques, we can then group the rectangular boxes that have been calculated to be close to each other, thus resulting in a tissue sample map locating each occurrence of the predetermined biological element, in accordance with the detection predictions obtained.

Claims

Demands

1. A method for the automated detection of a predetermined biological element in a tissue sample from a human or animal, characterized in that it comprises the steps of: obtaining (10) a plurality of machine-learning images of said biological element from a plurality of tissue samples from at least one subject; feeding (12) at least one artificial neural network with said plurality of machine-learning images; obtaining (14) a stack of images, along a plurality of parallel planes, of said tissue sample from said human or animal; concatenating (16) the images of said stack into a single image; performing (20) automatic edge detection in said single image, applying the detected edges to each image of said stack for each of said channels; extracting (22) from each image of said stack a region of interest, where said biological element is to be detected;subdivide (24) each region of interest into a plurality of imagelets; in each imagelet, using said at least one artificial neural network, automatically obtain (26) a prediction of detection of said biological element; combine each plurality of imagelets containing said predictions of detection into an image of each region of interest, and then stack the images of each region of interest, so as to obtain (28) a reconstructed image of said tissue sample of said human or animal containing said predictions of detection of said biological element.

2. A method according to claim 1, characterized in that step (14) of obtaining a stack of images, along a plurality of parallel planes, of said tissue sample of said human or animal involves the use of an immunofluorescence scanner.

3. Method according to claim 1 or 2, characterized in that the automatic edge detection step (20) includes image blurring, dilation and erosion steps.

4. Method according to claim 1, 2 or 3, characterized in that the step (22) of extraction of said region of interest implements a semantic segmentation algorithm.

5. A method according to any one of the preceding claims, characterized in that each detection prediction is materialized by a rectangular box which is the smallest rectangle containing the image of said biological element whose detection is predicted.

6. A method according to any one of the preceding claims, characterized in that the step (26) of automatically obtaining said detection prediction using said at least one artificial neural network comprises a step of calculating a posteriori probability for images.

7. Method according to claim 6, characterized in that said posterior probability calculation step for images implements a Kalman filter.

8. A method according to any one of the preceding claims, characterized in that said tissue is skin and said biological element is an intraepidermal nerve fiber.