Method for analysing a multispectral video previously acquired by endoscopy performed on a region of biological tissue, and corresponding endoscopic imaging system
The method enhances endoscopic imaging by acquiring and analyzing multispectral videos to improve lesion detection and tissue analysis, addressing the limitations of current systems in texture recognition and interpretation assistance.
Patent Information
- Application Number
- PCT/EP2024/085553
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-12-11
- Filing Date
- 2024-12-10
- Publication Date
- 2025-06-19
AI Technical Summary
Current endoscopic imaging systems provide insufficiently defined and recognizable textures of mucous membranes, increasing the risk of missing lesions, and lack assistance in interpreting images.
A method using a light-generating device with n successive spectral bands to acquire a multispectral video, followed by image extraction, non-rigid inter-band registration, and spectral reconstruction of a reflectance cube to enhance image quality and provide data for analysis.
The method achieves improved spectral resolution and image quality, allowing for better detection of lesions and providing data that can be exploited by artificial intelligence for enhanced tissue analysis.
Smart Images

Figure EP2024085553_19062025_PF_FP_ABST
Abstract
Description
[0001] TITLE: METHOD FOR ANALYZING A MULTISPECTRAL VIDEO PREVIOUSLY ACQUIRED BY ENDOSCOPY ON AN AREA OF BIOLOGICAL TISSUE, AND CORRESPONDING ENDOSCOPIC IMAGING SYSTEM.
[0002] Technical field of the invention
[0003] The field of the invention is that of medical imaging.
[0004] More specifically, the invention relates to the acquisition of data from imaging resulting from endoscopic exploration of biological tissues, and more specifically, of hollow organs or cavities of humans or animals.
[0005] Technical background
[0006] In the medical field, endoscopes are of crucial importance in enabling visual observation of hollow organs and cavities in the human body, and remain essential for patient monitoring, diagnosis and interventional procedures.
[0007] Endoscopes are particularly useful for detecting lesions or signs of inflammation present on the surface of tissues. Indeed, various hollow organ cancers are the consequence of a carcinological process resulting from chronic inflammation. The pre-cancerous lesions at the origin of these cancers are difficult to detect due to their appearance, which is not very different from healthy mucosa and their flat nature. Thus, these lesions are often not detected and the diagnosis is made at the cancer stage, which leads to heavy treatment and a decrease in patient survival rates.
[0008] In recent years, medical imaging techniques have been developed to attempt to detect tissue changes early, and in particular pre-cancerous or cancerous lesions.
[0009] There are methods of illuminating the surface of the organ to be studied using a light source, while moving the endoscope in the human cavity, and acquiring a video by a color camera. Such a method makes it possible to obtain a video reflecting the reflectance of light by the illuminated tissue. The color camera records the light signal in the form of three bands (red, green, and blue) covering the visible spectrum, while the light source is either white light or light composed of two or four wavelength bands.
[0010] The resulting visuals vary depending on the light source used. Video acquisition under white light produces images similar to those perceived by the human visual system. In contrast, video acquisition under light composed of two or four wavelength bands produces false-color images by combining images by spectral band.
[0011] However, current systems provide true or false color images whose textures, particularly of the mucous membranes, are insufficiently defined and recognizable, which increases the risk for the operator of not detecting lesions. In addition, such systems do not provide any assistance in interpreting the images.
[0012] There is therefore an unmet need to provide a method and device for acquiring endoscopic imaging making it possible to obtain images of improved quality and to provide data which can subsequently be exploited, in particular by artificial intelligence.
[0013] Summary of the invention
[0014] The present invention makes it possible to solve the problems raised by the prior art, in particular by developing a method for acquiring and analyzing data by endoscopy on biological tissues using a light-generating device associated with an endoscope with a camera, said method comprising:
[0015] - the generation of light on an area of biological tissue by a light device capable of generating n successive spectral bands of different wavelengths so as to cover the visible spectrum and the near infrared and generate a multispectral video in which each image corresponds to one of the spectral bands;
[0016] - cutting said endoscopic video thus obtained so as to extract images from spectral bands of different wavelengths;
[0017] - the registration of said spectral band images to obtain at least one registered multispectral image comprising an image for each spectral band of wavelengths; the spectral reconstruction of a reflectance cube, from said registered multispectral image, in which each pixel corresponds to a vector reflecting the reflectance of said area of biological tissue.
[0018] More particularly, a first subject of the present invention is a method for analyzing a multispectral video previously acquired by endoscopy on an area of biological tissue, said multispectral video having been previously acquired using an endoscopic imaging system associating a light generating device from an LED light source with an endoscope with a camera, said endoscope being configured to capture a video stream, said method comprising a step i) called acquisition comprising the following sub-steps: the generation by said light generating device of a light with n spectral bands of successive different wavelengths so as to cover the visible spectrum and the near infrared, the acquisition of a multispectral video by the camera of the endoscope, and successive illumination of the area to be observed by the different spectral bands,the successive lighting being carried out in a synchronized manner with the camera rate in which each image corresponds to one of said spectral bands; said method characterized in that it further comprises, at the end of the acquisition step i), a step ii) called generation of multispectral images comprising in particular: a step of generation of multispectral images by cutting said multispectral video thus obtained, so as to extract one or more multispectral images, each comprising at least one image representative of each of the spectral bands; a step of non-rigid inter-band registration of said spectral band images to obtain at least one registered multispectral image in which each pixel corresponds to a vector with n components and corresponding to the number of spectral bands used; and in that said method further comprises a step iii) of spectral reconstruction of a reflectance cube,from one or more recalibrated multispectral images, in which each pixel corresponds to a vector which reflects the reflectance of said area of biological tissue. The images obtained in the spectral bands covering the near infrared are advantageously used to characterize the tissues in a more absolute manner by complementarity with the spectral bands covering the visible spectrum.,
[0019] This multiplicity of spectral bands covering the visible spectrum and the near infrared allows, firstly, the obtaining of multispectral images having an improved and extended spectral resolution compared to color images, and secondly, to generate reflectance cubes in which each pixel corresponds to a vector reflecting only the reflectance of the corresponding tissue area. In comparison with the images acquired by the n spectral bands, such reflectance cubes also include a better spectral resolution over the visible and near infrared range in each pixel specific to the reflectance of the tissue after having freed itself from the irradiance of the illuminant and the sensitivity of the camera.
[0020] Advantageously, the wavelength bands may be between 380 and 1000 nm, preferably between 400 and 850 nm.
[0021] Advantageously, n can be equal to six or a multiple of six.
[0022] Advantageously, at least four of the spectral bands may have wavelengths spanning the visible spectrum and at least one of said spectral bands has wavelengths spanning the near infrared.
[0023] According to a particular characteristic, at least four of said spectral bands may have wavelengths covering the visible spectrum and at least two of the spectral bands may have wavelengths covering the near infrared.
[0024] The use of a second spectral band covering the infrared makes it possible to increase the quality of the multispectral image obtained.
[0025] Advantageously, the camera can capture 20 to 30 frames per second during the acquisition of the multispectral video, and only every other frame can be alternatively selected for the production of a multispectral image when cutting the obtained multispectral video.
[0026] According to a first advantageous embodiment of the method according to the invention, the spectral reconstruction of a reflectance cube from the multispectral image may comprise processing of the multispectral image by a deep neural network. According to a second advantageous embodiment of the method according to the invention, the spectral reconstruction of a reflectance cube from the recalibrated multispectral images may comprise:
[0027] - a second registration (or “mosaicing”) of the registered multispectral images in order to obtain a mosaic of multispectral images;
[0028] - processing of the multispectral image mosaic by a deep neural network or by a light-tissue interaction model.
[0029] Mosaicking a set of images consists of positioning the different images in a common reference frame, thus covering a field of view larger than that of the images taken individually.
[0030] The implementation of the method according to the second embodiment makes it possible to obtain several multispectral images representing a common area of the observed biological tissue. Thus, the generation of a reflectance spectrum can be carried out from several reflectance data vectors of the same area but acquired with a different positioning of the endoscope, which makes it possible to improve the estimation of the reflectance of the biological tissue.
[0031] Preferably, whether for the first or second embodiment, the deep neural network can be pre-trained from reflectance spectra data characteristic of healthy and pathological tissues.
[0032] The method may further comprise a histological analysis associated with the area of the tissue, in particular by microscopic analysis. This histological analysis provides ground truth potentially necessary for training for the purposes of automatic analysis or characterization of tissues.
[0033] Advantageously, the method may further comprise analyzing the reflectance cube by artificial intelligence.
[0034] It should be noted that the method according to the present description can be implemented indifferently in real time, that is to say during the endoscopic examination, or in a deferred time, that is to say after the endoscopic examination.
[0035] The present invention also relates to an endoscopic imaging system which comprises: an endoscope configured to capture a video stream, at least one box generating light from an LED source, and - a display screen configured to display images acquired by the endoscope and / or all or part of the information contained in the reflectance cubes obtained with a data acquisition method as described previously.
[0036] Brief description of the figures
[0037] Other characteristics and advantages of the invention will appear more clearly on reading the following description of preferred embodiments, given as illustrative and non-limiting examples, and the appended figures among which:
[0038] - [Fig. 1] illustrates the main steps of a method for acquiring data by endoscopy from biological tissues according to the invention;
[0039] - [Fig. 2] illustrates an example of non-rigid interband registration according to the invention;
[0040] - [Fig. 3] illustrates the main steps of the method of [Fig. 1], according to a first embodiment;
[0041] - [Fig. 4] schematically illustrates the processing step by a deep neural network or by a light-tissue interaction model;
[0042] - [Fig. 5] illustrates the main steps of the method of [Fig. 1], according to a second embodiment;
[0043] - [Fig. 6] schematically illustrates the multispectral image registration step;
[0044] - [Fig. 7] illustrates the main steps of the method of [Fig. 4], according to an alternative embodiment; and
[0045] - [Fig. 8] illustrates the absorption spectra of hemoglobin.
[0046] Detailed description of the invention
[0047] The general principle of the method of acquiring data by endoscopy from biological tissues is described in more detail in relation to Figure 1, and the two embodiments of Figure 3 on the one hand and Figures 5 and 7 on the other hand.
[0048] During a step i) called acquisition 11, a light device associated with an endoscope generates light with n spectral bands of different and successive wavelengths so as to generate a multispectral video in which each image corresponds to one of the spectral bands.
[0049] More precisely, this step 11 comprises the following sub-steps: the generation 111 by the light generating device of a light with n spectral bands of successive different wavelengths so as to cover the visible spectrum and the near infrared, the acquisition 112 of a multispectral video by the camera of the endoscope, and successive illumination 113 of the area to be observed by the different spectral bands, the successive illumination being carried out in a manner synchronized with the rate of the camera in which each image corresponds to one of said spectral bands;
[0050] With respect to the generation 111 of light having wavelength bands covering the visible spectrum and the near infrared, this makes it possible to cover wavelength ranges between 380-780 nanometers (nm) and 780-1000 nm respectively. The light device may illuminate the tissue to be observed successively with the different spectral bands, or alternatively, may illuminate the tissue to be observed with a plurality of spectral bands. Each of the spectral bands may cover a wavelength range from a few nanometers to several tens of nanometers, for example a wavelength range of 50 nanometers, preferably a wavelength range of 40 nanometers.
[0051] The spectral bands of the generated light can preferably be six in number. Thus, it is possible to select at least four bands covering the visible spectrum and at least one spectral band covering the infrared.
[0052] The four spectral bands covering the visible spectrum make it possible to obtain images, each comprising different absorption peaks.
[0053] The accuracy of these results is refined by images obtained with the infrared spectral band(s). Indeed, the spectral bands covering the infrared make it possible to obtain additional absorption peaks, and thus offer the possibility of obtaining information that cannot be perceived with spectral bands covering the visible spectrum.
[0054] For example, the following six wavelength bands may be used: a first band centered on 415 nm, a second band centered on 450 nm, a third band centered on 540 nm, a fourth band centered on 605 nm, a fifth band between 650 and 700 nm, a sixth band between 700 and 750 nm. According to a first alternative, the light emitted by the light device replaces the light emitted by the endoscope, the latter being consequently switched off during the implementation of the present method. In other words, in this alternative, the white light may be switched on and used to position the camera of the endoscope in the organ and the area to be observed, then switched off during the recording of the video.
[0055] In a second alternative, the light emitted by the light device complements the light emitted by the endoscope. In other words, the white light of the endoscope remains on even during video acquisition.
[0056] This second alternative can be implemented in particular when the power of the LED source is sufficiently high to illuminate the observed tissue area and acquire characteristic images of the spectral bands. Indeed, the LED source must ensure that the relevant details of the spectral band of a given wavelength range are captured despite the presence of the white light of the endoscope.
[0057] In addition to the aforementioned generation sub-step 111, step 11 further comprises the acquisition of a multispectral video 112 by the endoscope camera, as well as successive illumination 113 of the observed area by the different spectral bands. This results in an endoscopic video in which each image corresponds to one of the spectral bands with which the tissue area was illuminated, as indicated above. This successive illumination 113 is carried out in a manner synchronized with the frame rate of the camera. To do this, frame rate information of the endoscopic camera coming from the video stream is recovered so as to generate a timer based on an electronic device which allows the triggering of the spectral bands emitted by the light source.
[0058] At the end of the acquisition step i), the method according to the invention comprises a step ii) called generation of multispectral images (illustrated in particular in the diagrams of figures 3, 5 and 7) comprising in particular: a step 12) of generation of multispectral images by cutting said multispectral video thus obtained, so as to extract one or more multispectral images, each comprising at least one image representative of each of the spectral bands; and a step 13 of non-rigid interband registration of said spectral band images to obtain at least one registered multispectral image in which each pixel corresponds to a vector with n components and corresponding to the number of spectral bands used.
[0059] During step 12, a division 121 of the endoscopic video obtained is carried out so as to extract images from spectral bands of different wavelengths. The video needs to be divided so as to eliminate so-called “transition” images. “Transition” images refer to images obtained between two successive illuminations by different spectral bands, and which generally present unusable data resulting from the change of spectral bands. The camera can, for example, capture 20 to 30 images per second during the acquisition of the video. When the camera has this latter rate, a division step can be carried out so as to select every other image from the endoscopic video. This division results in one or more multispectral images. Each multispectral image comprises at least one image representative of each of the spectral bands.The size of the images can be chosen so that the images can encompass one or more lesions present on the surface of the tissue. The dimensions of the images can vary depending on various parameters such as the acquisition conditions, the nature of the organ studied and the pathological areas likely to develop there.
[0060] Step ii) comprises, at the end of step 12 of cutting the multispectral video, a step 13 of non-rigid inter-band registration of said spectral band images to obtain at least one registered multispectral image in which each pixel corresponds to a vector with n components and corresponding to the number of spectral bands used. The images of the spectral bands of the same multispectral image are superimposed on each other so that the characteristic elements of the analyzed tissues, present in each of them, are aligned. The characteristic elements are all the variations of tissues which may be on the surface of the observed organs, and may be for example lesions, signs of inflammation, blood vessels or variations in tissue granulosity.
[0061] In particular, a so-called “non-rigid” interband registration is carried out. In this description, “non-rigid interband registration” means a non-linear registration for all pixels to respond to the elastic deformations of the observed tissues.
[0062] Thus, unlike rigid interband registration which consists of aligning all the characteristic elements of an image by a single geometric transformation composed of a rotation and a translation, non-rigid interband registration consists of aligning each characteristic element obtained in the different spectral bands independently in one or more of the images of the multispectral image. Indeed, the movements of the observed organ coupled with those of the endoscope, in particular when it is a stomach, are likely to cause a displacement in space of the observed characteristic element, and thus, the characteristic element has a different position and shape in each image of the multispectral image.
[0063] For illustration purposes, Figure 2 represents an example of non-rigid interband registration of a point A in a multispectral image. Figure 2 shows that before registration, one of the points A (corresponding to a characteristic element) is not aligned with the other corresponding points A, despite the alignment of the different images constituting the multispectral image. This type of shift in the positioning of point A can in particular be the result of a movement of the internal wall of the observed organ. In order to take this movement into account, the unaligned point A is recentered so as to be aligned with the corresponding points A.
[0064] This type of registration can be achieved by a geometric approach. The latter is based on the extraction from each of the images of geometric primitives which are paired in order to determine the transformation which governs the deformations, movements and displacements between the two-by-two images. It can also be carried out by a calculation providing an optical flow between the two-by-two images in order to determine a displacement vector field at each pixel.
[0065] This step results in a realigned multispectral image in which each pixel corresponds to a vector with n components and preferably corresponding to the number of spectral bands used. Each of the components then corresponds to one of the spectral bands. Thus, each pixel of a multispectral image has n reflectance values for the same point of the observed tissue. However, although these values partly integrate the reflectance of the observed tissue, they also depend on the characteristics of the lighting device used and the sensitivity of the endoscope camera.
[0066] Finally, at the end of step ii, the method according to the invention comprises a step 14: a spectral reconstruction iii) of reflectance cubes from realigned multispectral images is implemented. More precisely, a reflectance cube is obtained in which each pixel corresponds to a vector reflecting only the reflectance of an area of the tissue or organ observed.
[0067] In this description, reflectance is defined by the quantity of energy re-emitted relative to the quantity of energy received, and this for each of the wavelengths covered by the spectral bands.
[0068] This vector corresponds to the reflectance spectrum of the tissue for the pixel in question over a given wide wavelength range, i.e. between 380-780 nm and 780-1000 nm. Each of its values corresponds to the reflectance for a narrow band of the wavelength range which can be from a few nanometers to several tens of nanometers.
[0069] This spectral reconstruction can be carried out from one or more multispectral images as described previously, according to one of the embodiments presented below.
[0070] According to a first embodiment of the method for acquiring data by endoscopy from biological tissues represented by figure 3, the spectral reconstruction comprises processing of the multispectral image by a deep neural network.
[0071] The deep neural network is trained on a large amount of data so that the neural network generates reflectance spectra from multispectral images. The amount of data is preferably at least a few hundred. This step allows going from a multispectral image to a reflectance cube, as illustrated in Figure 4. While the information contained in the multispectral image represents the reflectance of the biological tissue combined with the irradiance of the light source and the sensitivity of the camera, the information contained in the reflectance cube represents only the reflectance of the biological tissue.
[0072] Training the deep neural network involves providing it with a set of data pairs. Each pair is composed of a vector similar to those present in the multispectral images (input) and a vector similar to those present in the reflectance cubes (output) and representing the theoretical reflectance spectrum of the biological tissue alone corresponding to the input data.
[0073] Alternatively, or in addition, the deep neural network can be pre-trained using pairs of data from healthy tissues and pathological or abnormal tissues, whose reflectance spectrum comes from an additional device.
[0074] According to a second embodiment of the method for acquiring data by endoscopy from biological tissues represented by figure 5, the spectral reconstruction comprises a prior non-rigid registration of the multispectral images so as to obtain a corresponding mosaic of multispectral images (illustrated in figure 6) and a processing of the mosaic by a deep neural network (similar to that illustrated in figure 4).
[0075] The spectral reconstruction in this second embodiment requires the use of a plurality of multispectral images, which can be obtained successively over time. Preferably, the images forming the different multispectral images are obtained by the same spectral bands. Figure 6 shows in particular the same physical point A of the biological tissue, which is present in several multispectral images (left part of Figure 6). After registration, this point A is superimposed in the mosaic of multispectral images (right part of Figure 6).
[0076] The multispectral image mosaic is processed by a new neural network, similar to the one shown in Figure 4. The neural network has been previously trained to provide a reflectance cube (output) from a multispectral image mosaic (input).
[0077] For this embodiment also, this neural network processing step makes it possible to move from a mosaic of multispectral images to a reflectance cube. While the information contained in the multispectral images represents the reflectance of the biological tissue combined with the irradiance of the light source and the sensitivity of the camera, the information contained in the reflectance cube represents only the reflectance of the biological tissue.
[0078] As described previously, the deep neural network can be pre-trained using pairs of data from healthy tissues and pathological or abnormal tissues, whose reflectance spectrum comes from an additional device.
[0079] Figure 7 represents a variant of the second embodiment of the method previously presented. According to this variant, the spectral reconstruction comprises the registration of the multispectral images so as to obtain a corresponding mosaic of multispectral images, and a processing of the mosaic by a light-tissue interaction model.
[0080] The light-tissue interaction model is an analytical model described by equations. It is obtained after characterizing each of the tissues from their optical properties such as light absorption and scattering.
[0081] In particular, the equations of the light-tissue interaction model include characteristic values for each tissue.
[0082] For example, it is possible to determine with certain spectral bands of specific wavelengths the quantity of hemoglobin present in an area of the observed tissue. This information concerning the quantity of hemoglobin can give an indication of the presence of an anomaly, a cancerous or precancerous lesion.
[0083] The propagation of light in a layer of tissue can be modeled via two optical processes: diffusion (defined by its reduced diffusion coefficient p' s ) and absorption (defined by its coefficient p a ).
[0084] With this model, the reflectance R can be described according to the following formula:
[0085] [Math. 1]
[0086] R = RO (pa, p's)
[0087] RO is based on light transport theory (scattering theory or Monte Carlo simulations). The absorption spectra of hemoglobin are shown in Figure 8.
[0088] Preferably, the spectral reconstruction according to the first or second embodiment results in a reflectance cube of spatial size identical to the acquired images, but each pixel of which corresponds to a spectral vector of dimension greater than the number of initial bands.
[0089] It is also possible to obtain a reflectance cube of a spatial size smaller than the acquired images. Indeed, the captured images may represent the same area with a more or less significant shift, and consequently one or more parts of an image may be absent from the other images intended to form the multispectral image. The registration of the images between them may thus require reducing the sizes of the images so that they have identical sizes between them and represent the same and unique area of biological tissue.
[0090] On the contrary, when several multispectral images are acquired, the spatial size of the reflectance cube can be larger than the size of the acquired images. The multispectral images constituting the mosaic comprise a plurality of images covering the observed tissue area, which makes it possible to construct a reflectance cube whose size results from the combination of the different multispectral images.
[0091] According to another aspect, an endoscopic imaging system for implementing the method according to the present description comprises an endoscope, one or more boxes generating light from an LED source, and a viewing screen.
[0092] In particular, endoscopic video may be acquired using an endoscopic imaging system, comprising an endoscope and a viewing screen.
[0093] The endoscope typically comprises an operating channel equipped with a light beam (which can be conducted by optical fibers), and an optical or video vision system (such as a camera) positioned at the end present in the organ to be inspected.
[0094] The endoscope is intended to analyze human or animal biological tissues, and to be inserted into cavities or hollow organs such as a stomach, a bladder, or even a colon.
[0095] The LED source can be introduced into the working channel of the endoscope via an optical fiber. Thus, the LED source can be added to a large number of endoscopes on the market. Preferably, the LED source used in the endoscopic imaging system generates n spectral bands having different wavelengths and preferably located between 380 and 1000 nanometers.
[0096] Each LED source preferably generates six spectral bands, with at least four spectral bands covering the visible spectrum and at least one spectral band covering the near-infrared spectrum. Thus, for example, it is possible to assemble two housings to the endoscope so as to have twelve spectral bands covering the visible spectrum and the near-infrared.
[0097] The box is positioned next to the endoscopic column of the endoscope, or alternatively on the endoscopic column, and allows the programming and emission of light with one or more spectral bands covering defined wavelength ranges. In addition, the box offers the possibility of generating one or more spectral bands with a band change frequency that can be high.
[0098] During or at the end of the implementation of the data acquisition method according to the present description, the display screen can display the images acquired by the endoscope, the reflectance cubes obtained, and all the information obtained from the method described above. The display screen can also display results of an artificial intelligence processing of the information from the reflectance cubes obtained.
[0099] Such a system then makes it possible to alert the endoscope user to the possible presence of one or more anomalies, particularly early ones, in the biological tissues observed.
Claims
CLAIMS 1. Method for analyzing a multispectral video previously acquired by endoscopy on an area of biological tissue, said multispectral video having been previously acquired using an endoscopic imaging system associating a light generating device from an LED light source with an endoscope with a camera, said endoscope being configured to capture a video stream, said method comprising a step 11) comprising the following sub-steps: 111) the generation by said light generating device of light with n spectral bands of successive different wavelengths so as to cover the visible spectrum and the near infrared, 112) acquisition of multispectral video by the endoscope camera, and 113) successive illumination of the area to be observed by the different spectral bands, the successive illumination being carried out in a synchronized manner with the frame rate of the camera in which each image corresponds to one of said spectral bands; said method characterized in that it further comprises, at the end of step 11), a step ii) comprising in particular: a step 12) of generating multispectral images by cutting said multispectral video thus obtained, so as to extract one or more multispectral images, each comprising at least one image representative of each of the spectral bands; a step 13 of non-rigid inter-band registration of said spectral band images to obtain at least one registered multispectral image in which each pixel corresponds to a vector with n components and corresponding to the number of spectral bands used;and in that said method further comprises a step iii) of spectral reconstruction 14) of a reflectance cube, from one or more recalibrated multispectral images, in which each pixel corresponds to a vector which reflects the reflectance of said area of biological tissue.; 2. Method according to claim 1, wherein the wavelength bands are located between 380 and 1000 nm, preferably between 400 and 850 nm.
3. A method according to any preceding claim, wherein n is six or a multiple of six.
4. The method of claim 3, wherein at least four of said spectral bands have wavelengths spanning the visible spectrum and at least one of said spectral bands has wavelengths spanning the near infrared.
5. The method of claim 4, wherein at least four of said spectral bands have wavelengths spanning the visible spectrum and at least two of said spectral bands have wavelengths spanning the near infrared.
6. Method according to any one of the preceding claims, in which the camera captures 20 to 30 images per second during the acquisition of the multispectral video, and in which only one image out of two is alternately selected for the production of a multispectral image during the cutting of said multispectral video obtained.
7. A method according to any one of claims 1 to 6, wherein the spectral reconstruction of a reflectance cube from said multispectral image comprises processing said multispectral image by a deep neural network.
8. Method according to any one of claims 1 to 6, wherein the spectral reconstruction of a reflectance cube from said registered multispectral images comprises: a second registration of said registered multispectral images so as to obtain a mosaic of multispectral images; a processing of said mosaic of multispectral images by a deep neural network or by a light-tissue interaction model.
9. Method according to any one of claims 7 and 8, in which the deep neural network is pre-trained from reflectance spectra data characteristic of healthy and pathological tissues.
10. A method according to any one of claims 1 to 9, wherein it further comprises analyzing the reflectance cube by artificial intelligence.
11. Endoscopic imaging system intended to implement the method as defined according to any one of claims 1 to 10, characterized in that it comprises: an endoscope with camera, said endoscope being configured to capture an endoscopic video using said camera, at least one box generating light from an LED source, characterized in that it further comprises a display screen configured to display images acquired by the endoscope and / or all or part of the information contained in reflectance cubes obtained with the method as defined according to any one of claims 1 to 10.
Citation Information
Patent Citations
System and method for high precision multi-aperture spectral imaging
US20220156903A1
Portable hyperspectral system
WO2022211820A1