Method for determining a divergence score between two images and associated synchronisation method, device and system

Abstract

The invention relates to a method for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterised in that it comprises a step (204) of detecting in each image a plurality of specular spots, a step (210) of pairing specular spots of the two images, and a step (214) of calculating a divergence score between the two images as a function of the distances between the descriptors of the paired specular spots and / or the number of paired spots. The invention also relates to a synchronisation method comprising in particular a step (112) of determining the time offset of one channel with respect to another channel as a function of the divergence score and a step (114) of synchronising the channels by applying the determined time offset to one of the channels.

Description

[0001] DESCRIPTION

[0002] TITLE OF THE INVENTION: METHOD FOR DETERMINING A DIVERGENCE SCORE BETWEEN TWO IMAGES AND ASSOCIATED SYNCHRONIZATION METHOD, DEVICE AND SYSTEM

[0003] Technical field of the invention

[0004] The invention relates to a method and a device for determining a divergence score between two images of two channels of a stereoscopic video stream, in particular for augmented reality applications, in particular for endoscopic procedures, in particular laparoscopic procedures, for example for the purposes of surgery or robotic interventions. The invention relates more particularly to a method and an imaging system allowing the synchronization of the two channels according to this divergence score.

[0005] Technological background

[0006] Laparoscopy, also known as coelioscopy, is a medical technique for visually exploring the inside of a patient's body using an endoscope, or more specifically a coelioscope or laparoscope when used to observe the intra-abdominal and / or pelvic cavity of a patient. An endoscope generally includes a light source and a means of capturing the light, for example, optical fibers and / or a video sensor.

[0007] During laparoscopic surgery, the laparoscope provides direct or remote vision of the intra-abdominal cavity and allows observation of the surgical site and direct intervention using surgical tools. This surgical technique has the advantage of not requiring a large opening of the abdominal wall (unlike laparotomy or laparotomy), making it a minimally invasive technique.

[0008] Similarly, minimally invasive surgical procedures using an endoscope can be performed in the thoracic cavity (thoracoscopy) or in the pelvic cavity. This is generally referred to as an endoscopic operation or surgical endoscopy.

[0009] Recent technological advances have led to the evolution of laparoscopy from a simple vision by medical personnel of the image of the area to be operated on to an augmented vision, which allows additional information to be displayed on the screen about the image being viewed in order to assist medical personnel during the operation.

[0010] In particular, computer vision techniques are used on the image obtained by the laparoscope in real time to provide additional information through augmented reality. For example, a hidden structure in the organ, such as a tumor, can be displayed on the image. In particular, one may wish to display a site of intervention (incision for example) on the image of the organ. More generally, we are talking here about computer-guided procedures or surgery.

[0011] The endoscope used in such operations may be a stereoscopic endoscope, stereo endoscope or 3D endoscope, forming a binocular sensor that can capture two video streams of stereoscopic images comprising two channels, a left channel and a right channel, each captured by a dedicated RGB or multispectral type sensor, or by a single sensor capturing two concatenated channels. These channels can thus each be displayed to a left or right eye of a user of the endoscope to allow three-dimensional (3D) vision.

[0012] When processing the video stream from a video endoscope, for example for displaying augmented data, the channels are considered to be synchronous, i.e. the left channel images and the right channel images processed at a given time t were acquired concurrently. Synchronization of the streams is important for the correct visualization and computer processing of image pairs.

[0013] Some stereoscopic video sensors can mix the two channels into a single stream by interlacing or concatenating the images from the channels into a single stream.

[0014] 3D endoscopes used in the surgical context generally use two independent channels, which are processed by internal acquisition chains perfectly synchronized by trigger mechanisms and the channels are considered synchronous. However, the exploitation of the channels by software or a component not provided by the internal acquisition chain can be a sensitive step because the internal synchronization mechanisms are generally not accessible. Synchrony can then no longer be guaranteed because it depends on the processing applied to each channel, the characteristics of each optical sensor, but also the physical length of the signal transmission lines.

[0015] In particular, the presence of a recording device regularly causes channel desynchronization because the recording device is unable to open the channels synchronously and maintain a constant recording frequency. An initial time lag appears as well as a variable lag related to the refresh rate.

[0016] Poor synchrony degrades or eliminates the perception of depth in the stereoscopic video stream, and therefore of the three-dimensional information associated with this stream. Such degraded operation does not allow taking advantage of the additional information provided by stereoscopic vision and there is therefore no longer any advantage compared to a monocular system with a single sensor and a single channel, or even a disadvantage if the result distorts the user's 3D perception, particularly if the offset is significant.

[0017] Solutions have been proposed to enable this channel synchronization.

[0018] A first group of solutions are hardware solutions, requiring the addition of external elements to enable synchronization, for example by a binary state light source (LED type), GPS positioning, oscillating objects, etc. These solutions have the major drawback that they are difficult to use in a surgical context (or operating context), on the one hand because the operating setting is an environment with controlled sanitary conditions, and on the other hand because the devices present in the operating setting are medical devices subject to standards that cannot be modified for the implementation of these hardware solutions.

[0019] A second group of solutions are software solutions, which are based solely on the parameters of the received signals, in particular metadata related to encoding, timestamp data or the content of the signals themselves, in particular images and image pixels.

[0020] Among these software solutions, these fall into several categories: solutions using only the image, in particular its diffuse component: o brightness analysis: in an operating context, this analysis is not very informative because the brightness is controlled and quasi-constant; o motion analysis: this analysis is not relevant in an operating context because the phases of interest can be quasi-static. In addition, the visual field can also be disturbed by movements of high-frequency instruments; o image similarity analysis, using robust descriptors on the image, such as temporal descriptors (e.g. vectors representing activity), or image-based descriptors (e.g. self-similarity matrices (SSM)).Like motion analysis, the interest of this analysis is reduced because the phases of interest can be quasi-static; o image correlation implementing global descriptors such as frequency analysis or phase correlation, fingerprint calculation, histogram correlation, etc. These techniques generally do not allow to achieve the precision requirements necessary in the surgical setting; o descriptor matching (in particular keypoint matching) such as SIFT which uses local detectors and descriptors: this method also does not allow to achieve the precision requirements necessary in the surgical setting.solutions using timestamps: the use of a network protocol and a timestamp type timestamp in English requires hardware and / or software modification of the operating room equipment, which is strongly discouraged for the health and standards reasons explained above; solutions using encoding metadata: the analysis of the video bit rate profile requires the implementation of a video compression algorithm (for example to save video files in compressed format), which is contrary to the desired real-time use in the operating room. In addition, a single value to represent an image is not sufficiently discriminating.

[0021] The inventors thus sought to provide a method allowing the synchronization of non-synchronized channels or the resynchronization of desynchronized channels, in particular a method based solely on the processing of images captured by the stereoscopic sensor.

[0022] Objectives of the invention

[0023] The invention aims to provide on the one hand a method allowing the determination of a divergence score between two stereoscopic images, in particular coming from different channels.

[0024] The invention also aims to provide a method for synchronizing or resynchronizing images of a stream comprising two channels,

[0025] The invention aims to provide a method for determining a divergence score or a synchronization method particularly suitable for images representing organ tissues.

[0026] The invention also aims to provide a computer program product, a device for determining a divergence score and an associated endoscopic imaging system.

[0027] The invention also aims to provide, in at least one embodiment, such methods, computer program, device and endoscopic imaging system.

[0028] Statement of the invention

[0029] To do this, the invention relates to a method for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises the following steps: a step of detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, a step of defining a descriptor for each specular spot detected in each image, a step of matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched,a step of calculating a divergence score between the two images based on the distances between the descriptors of the paired specular spots and / or the number of paired spots.,

[0030] A determination method according to the invention therefore makes it possible, solely from images, to determine a divergence score using specular reflections generally present in large quantities on organ tissues. The determination method is thus simple to calculate and implement, while providing better precision and robustness performance in scenes where specular reflections are omnipresent. The divergence score, which may also be a divergence metric, makes it possible to determine the divergence or convergence (depending on the value of the score) of the two images to envisage a comparison between stereoscopic image streams to enable synchronization, when the method for determining the divergence score is executed as part of a stereoscopic image synchronization method.

[0031] Specular reflections are direct reflections associated with mirror-like behavior of tissues that result in localized and direct reflection of incident light in a restricted set of directions on which the light energy is concentrated. Specular reflections generally appear on the image as a bright spot because they induce saturation of the sensor. They most frequently occur on materials with low roughness, on organ tissues (also called organic tissues or biological tissues), liquids and metals.

[0032] Images depicting scenes containing organ tissue, particularly scenes captured by an endoscope, notably during laparoscopy, generally include a significant number of specular reflections on the tissues, which allows the implementation of this method. These specular reflections are all the more present when the elements of the scene are wet.

[0033] The reflection of light in the scene also includes a diffuse component that is sometimes used in existing synchronization solutions. Specular reflections are generally considered an image disturbance when analyzing the diffuse component.

[0034] The use of specular reflections for determining a divergence score between images is not incompatible with post-processing allowing the removal of specular spots on the image displayed on a viewing device, if these specular spots are considered to be disturbing for the viewing of the stereoscopic video stream by a human operator.

[0035] Machine learning model, more commonly known as a machine learning model, and in particular a machine learning model adapted to image processing and / or a deep learning model, for example a neural network, for example a convolutional neural network and in particular a fully convolutional neural network (or ECN for Fully Convolutional Network in English). An example of a fully convolutional neural network, in particular for semantic segmentation, is described for example in the publication "LONG, Jonathan, SHELHAMER, Evan, and DARRELL, Trevor. Lully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. p. 3431-3440.".Advantageously and according to the invention, in the step of detecting a plurality of specular spots, the method comprises a sub-step of generating a segmentation mask comprising the descriptors of each specular spot.

[0036] According to this aspect of the invention, the use of a segmentation mask makes it possible to simplify the processing of specular spots and thus allow faster execution of the determination method.

[0037] Advantageously and according to the invention, the determination method comprises a step of isolating each specular spot detected in the step of detecting the specular spots.

[0038] According to this aspect of the invention, the isolation of specular spots makes it possible to facilitate the calculation of the descriptor of each spot, in particular when the descriptor is the centroid of the spot. In particular, the isolation of specular spots makes it possible to simplify the definition of a local descriptor for each spot, compared to a global descriptor which would concern the entire image.

[0039] Advantageously and according to the invention, the descriptor defined in the definition step is a geometric descriptor and / or a radiometric descriptor and / or a location descriptor.

[0040] The invention also relates to a method for synchronizing or resynchronizing a video stream of stereoscopic images comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises the following steps: a step of receiving the video stream of stereoscopic images, a step of saving, in at least one memory device, a plurality of images of each channel, a step of determining a divergence score between a most recent image of one channel and the plurality of saved images of the other channel according to a method for determining a divergence score according to the invention,a step of determining the time shift of one channel relative to the other channel based on the divergence score calculated between the most recent image of one channel and the images of the other channel, a step of synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.,

[0041] A synchronization method therefore makes it possible, from the determination of a divergence score of the determination method, to enable the synchronization of the images coming from the two channels of the video stream to allow its use as a stereoscopic video stream and thus guarantee the 3D vision of the images of the stream. Synchronization is simple and fast and therefore usable in real time during a live capture in an operating context, but the method can also be used a posteriori on a recording. In particular, the divergence score makes it possible to characterize the proximity or not of two images from different channels and thus to determine the images considered as synchronous by comparing the scores of several images from each channel of the stream.

[0042] Throughout the application, the term "synchronization" will be used to refer to both synchronization and resynchronization, the difference between the two being based solely on the type of stream received and processed, depending on whether the channels of this stream have not been synchronized, or have already been synchronized in the processing chain but have undergone desynchronization.

[0043] The time shift determination step allows the determination of the time shift: of the left channel relative to the right channel as a function of the divergence score calculated between the most recent image of the left channel and the images of the right channel, of the right channel relative to the left channel as a function of the divergence score calculated between the most recent image of the right channel and the images of the left channel, of a combination of these last two possibilities, the determination being carried out successively or simultaneously for the two channels and the time shift between the channels is determined as a function of these two calculations.

[0044] The method may also include a temporal smoothing step for processing the video image stream to reduce noise. The temporal smoothing is based on a regression and / or motion model, for example a Kalman filter, a smoothing and mapping (or SAM) method such as GTSAM, a sliding window averaging filter, and / or a constant acceleration model.

[0045] If the calculated divergence scores do not identify a pair of close images, a time shift cannot be determined. The method can then retain the last calculated time shift and continue the process with a next image in the processed channel.

[0046] The divergence score can be combined with another existing score, such as an interest point-based score, to refine the determination of which image of one channel is closest to the other channel.

[0047] Advantageously and according to the invention, the step of determining the time shift relative to the other channel comprises: a sub-step of comparing the divergence scores between the most recent image of a channel and the plurality of saved images of the other channel, a sub-step of determining the saved image of a channel closest to the saved image of the other channel as a function of said comparison of the divergence scores, a sub-step of calculating the time shift between the left channel and the right channel of the video stream of stereoscopic images by calculating the time shift between the most recent image of a channel and the image determined as closest to the other channel.

[0048] The image of a channel closest to the other channel is a function of the divergence score value. This divergence score can be expressed in different ways, for example, we can consider that the closest images are those with the lowest divergence score value if a high divergence score value is representative of a greater divergence between the two images, or conversely we can consider that the closest images are those with the highest divergence score value if a high divergence score value is representative of a lower divergence between the two images.

[0049] Advantageously and according to the invention, the time shift is expressed as a number of shift images in the video stream between the most recent image of one channel and the image determined as the closest to the other channel.

[0050] According to this aspect of the invention, this definition of the time value makes it possible to simplify synchronization by applying an offset to one channel depending on the number of frames ahead or behind the other channel.

[0051] The invention also relates to a computer program product for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, the two images being a left image and a right image, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computer, steps of a method comprising: a step of detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, a step of defining a descriptor for each specular spot detected in each image, a step of matching each specular spot of at least one of the two images with at least one specular spot of the other image,by searching for the specular spot closest to each specular spot to be matched, a step of calculating a divergence score based on the distances between the descriptors of the matched specular spots and / or the number of matched spots.,

[0052] Advantageously, the computer program product according to the invention is configured to implement the method for determining a divergence score according to the invention.

[0053] Advantageously, the method for determining a divergence score according to the invention is implemented by a computer program product according to the invention.

[0054] The invention also relates to a computer program product for synchronizing or resynchronizing a video stream of stereoscopic images comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computer, the steps of a synchronization or resynchronization method according to the invention.

[0055] The computer program product for synchronization or resynchronization advantageously comprises the program code instructions of the computer program product for determining a divergence score according to the invention.

[0056] The invention also relates to a device for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises: a module for detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, a module for defining a descriptor for each specular spot detected in each image, a module for matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, a module for calculating a divergence score as a function of the distances between the descriptors of the matched specular spots and / or the number of matched spots.

[0057] A module may for example consist of a computing device such as a computer, a set of computing devices, an electronic component or a set of electronic components, or for example a computer program, a set of computer programs, a library of a computer program or a function of a computer program executed by a computing device such as a computer, a set of computing devices, an electronic component or a set of electronic components.

[0058] Advantageously, the device for determining a divergence score according to the invention is configured to implement the method for determining a divergence score according to the invention.

[0059] Advantageously, the method for determining a divergence score according to the invention is implemented by a device for determining a divergence score according to the invention.

[0060] The invention also relates to a stereoscopic endoscopic imaging system comprising a stereoscopic endoscope, a light source, and a memory device, characterized in that it comprises a system for synchronizing or resynchronizing a video stream of stereoscopic images transmitted by the stereoscopic endoscope and comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by the light source, said synchronization or resynchronization system comprising: a module for receiving the video stream of stereoscopic images, a module for saving, in said at least one memory device, a plurality of images of each channel, a device for determining a divergence score according to the invention,configured to determine a divergence score between a most recent image of one channel with the plurality of saved images of the other channel, a module for determining the time offset of a channel relative to the other channel based on the divergence score calculated between the most recent image of one channel and the images of the other channel, a module for synchronizing or resynchronizing the channels by applying to one of the channels the determined time offset so that the channels are synchronized.,

[0061] Preferably, the system is used for laparoscopic, thoracoscopic or pelvioscopic imaging.

[0062] The light source is preferably attached to the stereoscopic endoscope and / or arranged at a small relative distance from the stereoscopic endoscope.

[0063] The invention also relates to a method for determining a divergence score, a synchronization or resynchronization method, a computer program product, a device for determining a divergence score and an endoscopic imaging system characterized in combination by all or some of the characteristics mentioned above or below. List of figures

[0064] Other aims, characteristics and advantages of the invention will appear on reading the following description given solely for non-limiting purposes and which refers to the appended figures in which:

[0065] [Fig. 1] is a schematic view of a synchronization or resynchronization method according to one embodiment of the invention,

[0066] [Fig. 2] is an image representing a scene comprising organ tissues and illuminated by a light source, as taken for example by a stereoscopic endoscope of an endoscopic imaging system according to an embodiment of the invention,

[0067] [Fig. 3] is a schematic view of an endoscopic imaging system according to one embodiment of the invention.

[0068] Detailed description of an embodiment of the invention

[0069] In the figures, scales and proportions are not strictly respected, for the purposes of illustration and clarity.

[0070] Furthermore, identical, similar or analogous elements are designated by the same references in all figures.

[0071] Figure 1 schematically represents a synchronization or resynchronization method 100 according to an embodiment of the invention, called synchronization method 100 in the remainder of the description.

[0072] The synchronization method 100 aims to synchronize or resynchronize a video stream 102 of stereoscopic images comprising a left image channel, called left channel 102a, and a right image channel 102b, called right channel, which are asynchronized or desynchronized.

[0073] The video stream is in particular a stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, as visible in Figure 3 described later.

[0074] The synchronization method 100 thus comprises a step 104 of receiving the stereoscopic image stream, then a step 106 of saving, in at least one memory device, a plurality of images from each channel. In this embodiment, a first series of images 108a from the left channel and a second series of images 108b from the right channel are saved in the same memory device or in a plurality of memory devices.

[0075] The saved images are compared to determine a time difference between the channels. To do this, the synchronization method 100 comprises a step 110 of determining a divergence score between a most recent image of one channel and the plurality of saved images of the other channel. This step 110 comprises, for each pair of images whose divergence score is determined, the execution of a method 200 of determining a divergence score according to one embodiment of the invention.

[0076] The method 200 for determining a divergence score between two stereoscopic images 202a, 202b of a scene comprising organ tissues and illuminated by a light source comprises a step 204 of detecting in each image, by a trained learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue. This step may also comprise a sub-step 204a of generating a segmentation mask comprising the descriptors of each specular spot.

[0077] The determination method 200 may comprise a step 206 of isolating each specular spot detected in the step of detecting the specular spots, making it easier to define the descriptors.

[0078] Figure 2 is an image depicting a scene comprising organ tissues and illuminated by a light source, as captured for example by a stereoscopic endoscope of an endoscopic imaging system according to one embodiment of the invention. The image depicts organ tissues, here perirenal fat and renal parenchyma tissues. Organ tissues, particularly when wet, are smooth and shiny surfaces that generate a large number of specular spots due to illumination by the light source.

[0079] Returning to Figure 1, the determination method 200 comprises a step 208 of defining a descriptor for each specular spot detected in each image. The descriptor is in particular a geometric descriptor, for example the shape of the spot and / or a radiometric (or colorimetric) descriptor and / or a location descriptor, for example the centroid of the spot, or other types of descriptor.

[0080] The determination method 200 then comprises a step 210 of matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched. The matching may for example use a brute-force matching method aimed at finding the specular spot closest to the spot to be matched. This matching method is fast and compatible with real-time processing.

[0081] The determination method 200 may then comprise a step 212 of filtering the matched spots, so that the matched spots meet certain criteria, in particular proximity, for example implementing the Lowe criterion to guarantee good matching.

[0082] The determination method 200 then comprises a step 214 of calculating a divergence score between the two images as a function of the distances between the paired specular spots.

[0083] Returning to the synchronization method 100, all of the scores calculated between the saved images of each channel allow a step 112 of determining the time shift of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel.

[0084] Step 112 of determining the time shift comprises: a sub-step 112a of comparing the divergence scores between the most recent image of one channel and the plurality of saved images of the other channel, a sub-step 112b of determining the saved image of one channel closest to the saved image of the other channel based on said comparison of the divergence scores, a sub-step 112c of calculating the time shift between the left channel and the right channel of the video stream of stereoscopic images by calculating the time shift between the most recent image of one channel and the image determined as closest to the other channel.

[0085] The synchronization method 100 finally comprises a step 114 of synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

[0086] Figure 3 schematically represents an endoscopic imaging system 10, comprising a stereoscopic endoscope 12 configured to capture a scene comprising organ tissues and illuminated by a light source 14, preferably attached to the stereoscopic endoscope 12 and / or arranged at a small relative distance from the stereoscopic endoscope 12.

[0087] For example, in the context of a laparoscopic operation, the endoscope makes it possible to capture images of an abdominal cavity 50 of a patient comprising organs 52. The endoscope 12 makes it possible to capture the scene in and provide two image streams, a left image stream or left channel providing a left image 20a and a right image stream or right channel providing a right image, in which the organ tissues 22a, 22b of the organs 52 visible in the scene appear respectively. The two images 20a, 20b are for example displayed in a stereoscopic display device, for example stereoscopic glasses or any other system used in the operating context to display images from a stereoscopic endoscope.

[0088] The endoscopic imaging system 10 comprises a device 30 for processing the image stream from the endoscope, as may be present in the state of the art. To manage the synchronization or resynchronization constraints, the endoscopic imaging system 10 comprises a system 32 for synchronizing or resynchronizing the video stream of stereoscopic images transmitted by the stereoscopic endoscope and: a module for receiving the video stream of stereoscopic images, a module for saving, in said at least one memory device, a plurality of images of each channel, a device for determining a divergence score, configured to determine a divergence score between a most recent image of one channel with the plurality of saved images of the other channel, in particular using modules configured to execute a method as described with reference to FIG.The device for determining the divergence score creates in particular a segmentation mask 34 allowing the localization of specular spots. a module for determining the time shift of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel, - a module for synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

[0089] The invention is not limited to the embodiment described. In particular, the invention is applicable to any type of endoscopic imaging system in the context of an endoscopy, for example in the thoracic or pelvic cavity, as well as more generally to any system using a stereoscopic endoscope.

Claims

CLAIMS 1. Method for determining a divergence score between two stereoscopic images of a scene comprising organ tissues (22a, 22b) and illuminated by a light source, characterized in that it comprises the following steps: a step (204) of detecting in each image, by a trained machine learning model, a plurality of specular spots (300) corresponding to a plurality of specular reflections of the light source on an organ tissue, a step (208) of defining a descriptor for each specular spot (300) detected in each image, a step (210) of matching each specular spot (300) of at least one of the two images with at least one specular spot (300) of the other image, by searching for the specular spot (300) closest to each specular spot (300) to be matched,a step (214) of calculating a divergence score between the two images as a function of the distances between the descriptors of the paired specular spots (300) and / or the number of paired spots., 2. Method for determining a divergence score according to claim 1, characterized in that in the step (204) of detecting a plurality of specular spots (300), the method comprises a sub-step (204a) of generating a segmentation mask (34) comprising the descriptors of each specular spot.

3. Method for determining a divergence score according to one of claims 1 or 2, characterized in that it comprises a step (206) of isolating each specular spot (300) detected in the step (204) of detecting the specular spots.

4. Method for determining a divergence score according to one of claims 1 to 3, characterized in that the descriptor defined in the definition step is a geometric descriptor and / or a radiometric descriptor and / or a location descriptor.

5. Method for synchronizing or resynchronizing a video stream of stereoscopic images comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues (22a, 22b) and illuminated by a light source, characterized in that it comprises the following steps: a step (104) of receiving the video stream of stereoscopic images, a step (106) of saving, in at least one memory device, a plurality of images of each channel, a step (110) of determining a divergence score between a most recent image of one channel and the plurality of saved images of the other channel according to a method (200) of determining a divergence score according to one of claims 1 to 4,a step (112) of determining the time shift of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel, a step (114) of synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized., 6. Synchronization or resynchronization method according to claim 5, characterized in that the step (112) of determining the time offset relative to the other channel comprises: a sub-step (112a) of comparing the divergence scores between the most recent image of one channel and the plurality of saved images of the other channel, a sub-step (112b) of determining the saved image of one channel closest to the saved image of the other channel based on said comparison of the divergence scores, a sub-step (112c) of calculating the time shift between the left channel and the right channel of the video stream of stereoscopic images by calculating the time shift between the most recent image of one channel and the image determined as closest to the other channel.

7. Synchronization or resynchronization method according to one of claims 5 or 6, characterized in that the time shift is expressed in a number of shift images in the video stream between the most recent image of one channel and the image determined as the closest to the other channel.

8. Computer program product for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, the two images being a left image and a right image, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computer, steps of a method comprising: a step of detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on organ tissue, a step of defining a descriptor for each specular spot detected in each image, a step of matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, a step of calculating a divergence score based on the distances between the descriptors of the matched specular spots and / or the number of matched spots.

9. Device for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises: a module for detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, a module for defining a descriptor for each specular spot detected in each image, a module for matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, a module for calculating a divergence score as a function of the distances between the descriptors of the paired specular spots and / or the number of paired spots.

10. Stereoscopic endoscopic imaging system comprising a stereoscopic endoscope (12), a light source (14), and a memory device, characterized in that it comprises a system (32) for synchronizing or resynchronizing a video stream of stereoscopic images transmitted by the stereoscopic endoscope and comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by the light source, said synchronization or resynchronization system comprising: a module for receiving the video stream of stereoscopic images, a module for saving, in said at least one memory device, a plurality of images of each channel, a device for determining a divergence score according to claim 9, configured to determine a divergence score between a most recent image of a channel with the plurality of saved images of the other channel, a module for determining the time shift of a channel relative to the other channel as a function of the divergence score calculated between the most recent image of a channel and the images of the other channel, a module for synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

Images