METHOD FOR DETERMINING A DIVERGENCE SCORE BETWEEN TWO IMAGES AND ASSOCIATED SYNCHRONIZATION METHOD, DEVICE AND SYSTEM

Abstract

METHOD FOR DETERMINING A DIVERGENCE SCORE BETWEEN TWO IMAGES AND ASSOCIATED SYNCHRONIZATION METHOD, DEVICE AND SYSTEM The invention relates to a method for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises a step (204) of detecting in each image a plurality of specular spots, a step (210) of matching specular spots of the two images, and a step (214) of calculating a divergence score between the two images as a function of the distances between the descriptors of the paired specular spots and / or the number of paired spots.The invention also relates to a synchronization method comprising in particular a step (112) of determining the time offset of a channel relative to another channel as a function of the divergence score and a step (114) of synchronizing the channels by applying the determined time offset to one of the channels. Figure for the abstract: figure 1.

Description

Title of the invention: METHOD FOR DETERMINING A DIVERGENCE SCORE BETWEEN TWO IMAGES AND ASSOCIATED SYNCHRONIZATION METHOD, DEVICE AND SYSTEM Technical field of the invention

[0001] The invention relates to a method and a device for determining a score of di convergence between two images of two channels of a stereoscopic video stream, in particular for augmented reality applications, in particular for endoscopic procedures, in particular laparoscopic procedures, for example for the purposes of surgery or robotic interventions. The invention relates more particularly to a method and an imaging system allowing the synchronization of the two channels according to this divergence score. Technological background

[0002] Coelioscopy, also called laparoscopy, is a medical technique for visually exploring the interior of a patient's body using an endoscope, or more particularly a coelioscope or laparoscope when used to observe the intra-abdominal and / or pelvic cavity of a patient. An endoscope generally comprises a light source and a means of capturing the light, for example optical fibers and / or a video sensor.

[0003] During a surgical intervention by coelioscopy, the coelioscope allows a direct or remote vision of the intra-abdominal cavity and allows observation of the surgical site and direct intervention using surgical tools. This surgical technique has the advantage of not requiring a large opening of the abdominal wall (unlike laparotomy or coeliotomy), making it a minimally invasive technique.

[0004] Similarly, minimally invasive surgical procedures using an endoscope can be performed in the thoracic cavity (thoracoscopy) or in the pelvic cavity. This is generally referred to as an endoscopic operation or surgical endoscopy.

[0005] Recent technological advances have led to the development of coelioscopy from a simple vision by medical personnel of the image of the area to be operated on to an augmented vision, which allows additional information to be displayed on the screen on the image viewed in order to assist medical personnel during the operation.

[0006] In particular, computer vision techniques are used on the image obtained by the laparoscope in real time to provide additional information through augmented reality. For example, a hidden structure in the organ, such as a tumor, can be displayed on the image. In particular, one may wish to display a site of intervention (incision for example) on the image of the organ. More generally, we are talking here about computer-guided procedures or surgery.

[0007] The endoscope used in such operations may be a stereoscopic endoscope, stereo endoscope or 3D endoscope, forming a binocular sensor which makes it possible to capture two video streams of stereoscopic images comprising two channels, a left channel and a right channel, each captured by a dedicated RGB or multi-spectral type sensor, or by a single sensor capturing two concatenated channels. These channels can thus each be displayed to a left or right eye of a user of the endoscope to allow three-dimensional (3D) vision.

[0008] During computer processing carried out on the video stream coming from a video endoscope, for example for the display of augmented data, it is considered that the channels are synchronous, that is to say that the images of the left channel and the images of the right channel processed at a time t were acquired concomitantly. Synchronization of the streams is important for the correct visualization and the correct computer processing of the pairs of images.

[0009] Some stereoscopic video sensors can mix the two channels into a single stream by interlacing or concatenating the images from the channels into a single stream.

[0010] 3D endoscopes used in the surgical context generally use two independent channels, which are processed by internal acquisition chains perfectly synchronized by trigger mechanisms and the channels are considered synchronous.

[0011] However, the exploitation of the channels by software or a component not provided for by the internal acquisition chain can be a sensitive step because the internal synchronization mechanisms are generally not accessible. Synchrony can then no longer be guaranteed because it depends on the processing applied to each channel, the characteristics of each optical sensor, but also the physical length of the signal transmission lines.

[0012] More particularly, the presence of a recording device regularly causes desynchronization of the channels because the recording device is not capable of opening the channels synchronously and maintaining a constant recording frequency. An initial time lag appears as well as a variable lag linked to the refresh rate.

[0013] Poor synchrony degrades or eliminates the perception of depth in the stereoscopic video stream, and therefore of the three-dimensional information associated with it. flow. Such degraded operation does not allow taking advantage of the additional information provided by stereoscopic vision and there is therefore no longer any advantage compared to a monocular system with a single sensor and a single channel, or even a disadvantage if the result distorts the user's 3D perception, particularly if the offset is significant.

[0014] Solutions have been proposed to enable this synchronization of channels.

[0015] A first group of solutions are hardware solutions, requiring the addition of external elements to enable synchronization, for example by a binary state light source (LED type), GPS positioning, oscillating objects, etc. These solutions have the major drawback that they are difficult to use in a surgical context (or operating context), on the one hand because the operating setting is an environment with controlled sanitary conditions, and on the other hand because the devices present in the operating setting are medical devices subject to standards which cannot be modified for the implementation of these hardware solutions.

[0016] A second group of solutions are software solutions, which are based solely on the parameters of the received signals, in particular metadata related to encoding, time stamp data or the content of the signals themselves, in particular images and image pixels.

[0017] Among these software solutions, these fall into several categories: - solutions using only the image, in particular its diffuse component: • brightness analysis: in an operational context, this analysis is not very informative because the brightness is controlled and almost constant; • motion analysis: this analysis is not relevant in an operative context because the phases of interest may be quasi-static. In addition, the visual field may also be disturbed by movements of high-frequency instruments; • image similarity analysis, using robust image descriptors, such as temporal descriptors (e.g., vectors representing activity), or image-based descriptors (e.g., self-similarity matrices (SSMs)). Like motion analysis, the value of this analysis is reduced because the phases of interest can be quasi-static; • image correlation using global descriptors such as frequency analysis or phase correlation, fingerprint calculation, histogram correlation, etc. These techniques generally do not allow the accuracy requirements to be achieved necessary in the operational framework; • descriptor matching (in particular keypoint matching) such as SIFT which uses local detectors and descriptors: this method also does not allow the precision requirements necessary in the operational framework to be achieved. - solutions using timestamping: the use of a network protocol and a timestamp type timestamp in English requires the hardware and / or software modification of the operating environment equipment, which is strongly discouraged for the health and standards reasons set out above; - solutions using encoding metadata: analyzing the video bitrate profile requires the implementation of video compression algorithms (e.g. to save video files in compressed format), which is contrary to the real-time use desired in the operational context. In addition, a single value to represent an image is not sufficiently discriminating.

[0018] The inventors have thus sought to provide a method allowing the synchronization of non-synchronized channels or the resynchronization of desynchronized channels, in particular a method based solely on the processing of images captured by the stereoscopic sensor. Objectives of the invention

[0019] The invention aims to provide on the one hand a method allowing the determination of a divergence score between two stereoscopic images, in particular coming from different channels.

[0020] The invention also aims to provide a method for synchronizing or resynchronizing images of a stream comprising two channels,

[0021] The invention aims to provide a method for determining a divergence score or a synchronization method particularly suited to images representing organ tissues.

[0022] The invention also aims to provide a computer program product, a device for determining a divergence score and an associated endoscopic imaging system.

[0023] The invention also aims to provide, in at least one embodiment, such methods, computer program, device and endoscopic imaging system. Presentation of the invention

[0024] To do this, the invention relates to a method for determining a divergence score between two stereoscopic images of a scene comprising tissues organ and illuminated by a light source, characterized in that it comprises the following steps: - a step of detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, - a step of defining a descriptor for each specular spot detected in each image, - a step of matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, - a step of calculating a divergence score between the two images based on the distances between the descriptors of the paired specular spots and / or the number of paired spots.

[0025] A determination method according to the invention therefore makes it possible, solely from images, to determine a divergence score using specular reflections generally present in large quantities on organ tissues. The determination method is thus simple to calculate and implement, while providing better precision and robustness performance in scenes where specular reflections are omnipresent. The divergence score, which may also be a divergence metric, makes it possible to determine the divergence or convergence (depending on the value of the score) of the two images in order to envisage a comparison between stereoscopic image streams to enable synchronization, when the method for determining the divergence score is executed within the framework of a stereoscopic image synchronization method.

[0026] Specular reflections are direct reflections associated with a mirror behavior of tissues that results in a localized and direct reflection of the incident light in a restricted set of directions on which the light energy is concentrated. Specular reflections generally result in a bright spot on the image because they induce saturation of the sensor. They most frequently occur on materials of low roughness, on organ tissues (also called organic tissues or biological tissues), liquids and metals.

[0027] Images representing scenes comprising organ tissues, in particular scenes captured by an endoscope, in particular during a coelioscopy, generally comprise a significant number of specular reflections on the tissues, which allows the implementation of this method. These specular reflections are all the more present when the elements of the scene are wet.

[0028] The reflection of light in the scene also includes a diffuse component which is sometimes used in existing synchronization solutions. Specular reflections are generally considered as an image disturbance when analyzing the diffuse component.

[0029] The use of specular reflections for determining a divergence score between the images is not incompatible with post-processing allowing the removal of specular spots on the image displayed on a display device, if these specular spots are considered to be disturbing for the viewing of the stereoscopic video stream by a human operator.

[0030] The machine learning model, better known as a machine learning model in English, and in particular a machine learning model adapted to image processing and / or a deep learning model, for example a neural network, for example a convolutional neural network and in particular a fully convolutional neural network (or FCN for Fully Convolutional Network in English). An example of a fully convolutional neural network, in particular for semantic segmentation, is described for example in the publication “LONG, Jonathan, SHELHAMER, Evan, and DARRELL, Trevor. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. p. 3431-3440.”.

[0031] Advantageously and according to the invention, in the step of detecting a plurality of specular spots, the method comprises a sub-step of generating a segmentation mask comprising the descriptors of each specular spot.

[0032] According to this aspect of the invention, the use of a segmentation mask makes it possible to simplify the processing of specular spots and thus allow faster execution of the determination method.

[0033] Advantageously and according to the invention, the determination method comprises a step of isolating each specular spot detected in the step of detecting the specular spots.

[0034] According to this aspect of the invention, the isolation of the specular spots makes it possible to facilitate the calculation of the descriptor of each spot, in particular when the descriptor is the centroid of the spot. In particular, the isolation of the specular spots makes it possible to simplify the definition of a local descriptor for each spot, compared to a global descriptor which would concern the entire image.

[0035] Advantageously and according to the invention, the descriptor defined in the definition step is a geometric descriptor and / or a radiometric descriptor and / or a location descriptor.

[0036] The invention also relates to a method for synchronizing or resynchronizing a video stream of stereoscopic images comprising a left image channel, called the left channel, and a right image channel, called the right channel, asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises the following steps: - a step of receiving the video stream of stereoscopic images, - a step of saving, in at least one memory device, a plurality of images from each channel, - a step of determining a divergence score between a most recent image of a channel with the plurality of saved images of the other channel according to a method of determining a divergence score according to the invention, - a step of determining the time shift of a channel relative to the other channel as a function of the divergence score calculated between the most recent image of a channel and the images of the other channel, - a step of synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

[0037] A synchronization method therefore makes it possible, from the determination of a divergence score of the determination method, to enable the synchronization of the images coming from the two channels of the video stream to enable its use as a stereoscopic video stream and thus guarantee the 3D vision of the images of the stream. Synchronization is simple and rapid and therefore usable in real time during live capture in an operational context, but the method can also be used a posteriori on a recording. In particular, the divergence score makes it possible to characterize the proximity or not of two images of different channels and thus to determine the images considered to be synchronous by comparing the scores of several images of each channel of the stream.

[0038] Throughout the application, the expression "synchronization" will be used to designate both a synchronization and a resynchronization, the difference between the two being solely based on the type of stream received and processed, depending on whether the channels of this stream have not been synchronized, or have already been synchronized in the processing chain but have undergone desynchronization.

[0039] The step of determining the time shift allows the determination of the time shift: - of the left channel compared to the right channel based on the divergence score calculated between the most recent image of the left channel and the images of the right channel, - of the right channel compared to the left channel based on the divergence score calculated between the most recent image of the right channel and the images of the left channel, - a combination of these last two possibilities, the determination being carried out successively or simultaneously for the two channels and the time shift between the channels is determined according to these two calculations.

[0040] The method may also comprise a temporal smoothing step enabling the processing of the video image stream to attenuate the noise. The temporal smoothing is based on a regression and / or motion model, for example a Kalman filter, a smoothing and mapping method (or SAM for smoothing and mapping in English) such as GTSAM, sliding window averaging filter and / or constant acceleration model.

[0041] If the calculated divergence scores do not allow a pair of close images to be identified, a time shift cannot be determined. The method can then keep the last calculated time shift and continue the method with a next image in the processed channel.

[0042] The divergence score may be combined with another existing score, in particular a score based on points of interest, to refine the determination of the image of one channel closest to the other channel.

[0043] Advantageously and according to the invention, the step of determining the time offset relative to the other channel comprises: - a sub-step of comparing the divergence scores between the most recent image of one channel and the plurality of saved images of the other channel, - a sub-step of determining the saved image of one channel closest to the saved image of the other channel based on said comparison of the divergence scores, - a sub-step of calculating the time shift between the left channel and the right channel of the video stream of stereoscopic images by calculating the time shift between the most recent image of one channel and the image determined as the closest to the other channel.

[0044] The image of a channel closest to the other channel is a function of the divergence score value. This divergence score can be expressed in different ways, for example, the closest images can be considered to be those with the lowest divergence score value if a high divergence score value is representative of a greater divergence between the two images, or conversely, the closest images can be considered to be those with the highest divergence score value if a high divergence score value is representative of a lower divergence between the two images.

[0045] Advantageously and according to the invention, the time shift is expressed as a number of shift images in the video stream between the most recent image of one channel and the image determined as the closest to the other channel.

[0046] According to this aspect of the invention, this definition of the time value makes it possible to simplify synchronization by applying an offset to one channel as a function of the number of frames ahead or behind the other channel.

[0047] The invention also relates to a computer program product for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, the two images being a left image and a right image, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computer, steps of a method comprising: - a step of detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, - a step of defining a descriptor for each specular spot detected in each image, - a step of matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, - a step of calculating a divergence score based on the distances between the descriptors of the paired specular spots and / or the number of paired spots.

[0048] Advantageously, the computer program product according to the invention is configured to implement the method for determining a divergence score according to the invention.

[0049] Advantageously, the method for determining a divergence score according to the invention is implemented by a computer program product according to the invention.

[0050] The invention also relates to a computer program product for synchronizing or resynchronizing a video stream of stereoscopic images comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computer, the steps of a synchronization or resynchronization method according to the invention.

[0051] The computer program product for synchronization or resynchronization advantageously comprises the program code instructions of the product computer program for determining a divergence score according to the invention.

[0052] The invention also relates to a device for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises: - a detection module in each image, by a trained machine learning model, of a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, - a module for defining a descriptor for each specular spot detected in each image, - a module for matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, - a module for calculating a divergence score based on the distances between the descriptors of the paired specular spots and / or the number of paired spots.

[0053] A module may for example consist of a computing device such as a computer, a set of computing devices, an electronic component or a set of electronic components, or for example a computer program, a set of computer programs, a library of a computer program or a function of a computer program executed by a computing device such as a computer, a set of computing devices, an electronic component or a set of electronic components.

[0054] Advantageously, the device for determining a divergence score according to the invention is configured to implement the method for determining a divergence score according to the invention.

[0055] Advantageously, the method for determining a divergence score according to the invention is implemented by a device for determining a divergence score according to the invention.

[0056] The invention also relates to a stereoscopic endoscopic imaging system comprising a stereoscopic endoscope, a light source, and a memory device, characterized in that it comprises a system for synchronizing or resynchronizing a video stream of stereoscopic images transmitted by the stereoscopic endoscope and comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by the light source, said synchronization or resynchronization system comprising: - a module for receiving the video stream of stereoscopic images, - a backup module, in said at least one memory device, of a plurality of images from each channel, - a device for determining a divergence score according to the invention, configured to determine a divergence score between a most recent image of a channel with the plurality of saved images of the other channel, - a module for determining the time shift of a channel relative to the other channel as a function of the divergence score calculated between the most recent image of a channel and the images of the other channel, - a module for synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

[0057] Preferably, the system is used for laparoscopic, thoracoscopic or pelvioscopic imaging.

[0058] The light source is preferably attached to the stereoscopic endoscope and / or arranged at a small relative distance from the stereoscopic endoscope.

[0059] The invention also relates to a method for determining a divergence score, a synchronization or resynchronization method, a computer program product, a device for determining a divergence score and an endoscopic imaging system characterized in combination by all or part of the characteristics mentioned above or below. List of figures

[0060] Other aims, characteristics and advantages of the invention will appear on reading the following description given solely for non-limiting purposes and which refers to the appended figures in which:

[0061] [Fig-1] is a schematic view of a synchronization or resynchronization process nization according to an embodiment of the invention,

[0062] [Fig.2] is an image representing a scene comprising organ tissues and illuminated by a light source, as taken for example by a stereoscopic endoscope of an endoscopic imaging system according to an embodiment of the invention,

[0063] [Fig.3] is a schematic view of an endoscopic imaging system according to one embodiment of the invention.

[0064] Detailed description of an embodiment of the invention

[0065] In the figures, the scales and proportions are not strictly respected, for the purposes of illustration and clarity.

[0066] Furthermore, identical, similar or analogous elements are designated by the same references in all the figures.

[0067] [Fig.l] schematically represents a synchronization or resynchronization method 100 according to an embodiment of the invention, called synchronization method 100 in the remainder of the description.

[0068] The synchronization method 100 aims to synchronize or resynchronize a video stream 102 of stereoscopic images comprising a left image channel, called left channel 102a, and a right image channel 102b, called right channel, which are asynchronized or desynchronized.

[0069] The video stream is in particular a stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, as visible in [Fig.3] described below.

[0070] The synchronization method 100 thus comprises a step 104 of receiving the stereoscopic image stream, then a step 106 of saving, in at least one memory device, a plurality of images from each channel. In this embodiment, a first series of images 108a coming from the left channel and a second series of images 108b coming from the right channel are saved in the same memory device or in a plurality of memory devices.

[0071] The saved images are compared to determine a time difference between the channels. To do this, the synchronization method 100 comprises a step 110 of determining a divergence score between a most recent image of one channel and the plurality of saved images of the other channel. This step 110 comprises, for each pair of images whose divergence score is determined, the execution of a method 200 of determining a divergence score according to an embodiment of the invention.

[0072] The method 200 for determining a divergence score between two stereoscopic images 202a, 202b of a scene comprising organ tissues and illuminated by a light source comprises a step 204 of detecting in each image, by a trained learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue. This step may also comprise a sub-step 204a of generating a segmentation mask comprising the descriptors of each specular spot.

[0073] The determination method 200 may comprise a step 206 of isolating each specular spot detected in the step of detecting the specular spots, making it easier to define the descriptors.

[0074] [Fig. 2] is an image representing a scene comprising organ tissues and illuminated by a light source, as taken for example by a stereoscopic endoscope of an endoscopic imaging system according to an embodiment of the invention. The image represents organ tissues, here perirenal fat and renal parenchyma tissues. Organ tissues, in particular when wet, are smooth, shiny surfaces that generate a large number of specular spots due to illumination by the light source.

[0075] Returning to [Fig.l], the determination method 200 comprises a step 208 of defining a descriptor for each specular spot detected in each image. The descriptor is in particular a geometric descriptor, for example the shape of the spot and / or a radiometric (or colorimetric) descriptor and / or a location descriptor, for example the centroid of the spot, or other types of descriptor.

[0076] The determination method 200 then comprises a step 210 of matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched. The matching may for example use a brute-force matching method aimed at finding the specular spot closest to the spot to be matched. This matching method is fast and compatible with real-time processing.

[0077] The determination method 200 may then comprise a step 212 of filtering the matched spots, so that the matched spots meet certain criteria, in particular proximity, for example implementing the Lowe criterion to guarantee good matching.

[0078] The determination method 200 then comprises a step 214 of calculating a divergence score between the two images as a function of the distances between the paired specular spots.

[0079] Returning to the synchronization method 100, all of the scores calculated between the saved images of each channel allow a step 112 of determining the time shift of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel.

[0080] Step 112 of determining the time offset comprises: - a sub-step 112a of comparing the divergence scores between the most recent image of one channel and the plurality of saved images of the other channel, - a sub-step 112b of determining the saved image of one channel closest to the saved image of the other channel as a function of said comparison of the divergence scores, - a sub-step 112c of calculating the time shift between the left channel and the right channel of the video stream of stereoscopic images by calculating the time shift between the most recent image of one channel and the image determined as the closest to the other channel.

[0081] The synchronization method 100 finally comprises a step 114 of synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

[0082] [Fig. 3] schematically represents an endoscopic imaging system 10, comprising a stereoscopic endoscope 12 configured to capture a scene comprising organ tissues and illuminated by a light source 14, preferably attached to the stereoscopic endoscope 12 and / or arranged at a short relative distance from the stereoscopic endoscope 12.

[0083] For example, in the context of a laparoscopic operation, the endoscope makes it possible to capture images of an abdominal cavity 50 of a patient comprising organs 52. The endoscope 12 makes it possible to capture the scene in and provide two image streams, a left image stream or left channel providing a left image 20a and a right image stream or right channel providing a right image, in which the organ tissues 22a, 22b of the organs 52 visible in the scene appear respectively. The two images 20a, 20b are for example displayed in a stereoscopic display device, for example stereoscopic glasses or any other system used in the operating context to display images coming from a stereoscopic endoscope.

[0084] The endoscopic imaging system 10 comprises a device 30 for processing the image stream coming from the endoscope, as may be present in the state of the art. To manage the synchronization or resynchronization constraints, the endoscopic imaging system 10 comprises a system 32 for synchronizing or resynchronizing the video stream of stereoscopic images transmitted by the stereoscopic endoscope and: - a module for receiving the video stream of stereoscopic images, - a backup module, in said at least one memory device, of a plurality of images of each channel, - a device for determining a divergence score, configured to determine a divergence score between a most recent image of one channel with the plurality of saved images of the other channel, in particular using modules configured to execute a method as described with reference to [Fig.l]. The device for determining the divergence score creates in particular a segmentation mask 34 allowing the localization of the specular spots. - a module for determining the time shift of one channel relative to the other channel based on the divergence score calculated between the most recent image of one channel and the images of the other channel, - a module for synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

[0085] The invention is not limited to the embodiment described. In particular, the invention is applicable to any type of endoscopic imaging system in the context of endoscopy, for example in the thoracic or pelvic cavity, as well as more generally to any system using a stereoscopic endoscope.

Claims

Claims

1. Method for determining a divergence score between two stereoscopic images of a scene comprising organ tissues (22a, 22b) and illuminated by a light source, characterized in that it comprises the following steps: • a step (204) of detecting in each image, by a trained machine learning model, a plurality of specular spots (300) corresponding to a plurality of specular reflections of the light source on an organ tissue, • a step (208) of defining a descriptor for each specular spot (300) detected in each image, • a step (210) of matching each specular spot (300) of at least one of the two images with at least one specular spot (300) of the other image, by searching for the specular spot (300) closest to each specular spot (300) to be matched,• a step (214) of calculating a divergence score between the two images as a function of the distances between the descriptors of the paired specular spots (300) and / or the number of paired spots.,

2. Method for determining a divergence score according to claim 1, characterized in that in the step (204) of detecting a plurality of specular spots (300), the method comprises a sub-step (204a) of generating a segmentation mask (34) comprising the descriptors of each specular spot.

3. Method for determining a divergence score according to one of claims 1 or 2, characterized in that it comprises a step (206) of isolating each specular spot (300) detected in the step (204) of detecting the specular spots.

4. Method for determining a divergence score according to one of claims 1 to 3, characterized in that the descriptor defined in the definition step is a geometric descriptor and / or a radiometric descriptor and / or a location descriptor.

5. Method for synchronizing or resynchronizing a video stream of stereoscopic images comprising a left image channel, called the left channel, and a right image channel, called the right channel, asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues (22a, 22b) and illuminated by a light source, characterized in that it comprises the following steps: • a step (104) of receiving the video stream of stereoscopic images, • a step (106) of saving, in at least one memory device, a plurality of images of each channel, • a step (110) of determining a divergence score between a most recent image of one channel with the plurality of saved images of the other channel according to a method (200) of determining a divergence score according to one of claims 1 to 4, • a step (112) of determining the time shift of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel, • a step (114) of synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

6. Synchronization or resynchronization method according to claim 5, characterized in that the step (112) of determining the time offset relative to the other channel comprises: • a sub-step (112a) of comparing the divergence scores between the most recent image of one channel and the plurality of saved images of the other channel, • a sub-step (112b) of determining the saved image of one channel closest to the saved image of the other channel as a function of said comparison of the divergence scores, • a sub-step (112c) of calculating the time shift between the left channel and the right channel of the video stream of stereoscopic images by calculating the time shift between the image most recent from one channel and the image determined to be closest to the other channel.

7. Synchronization or resynchronization method according to one of claims 5 or 6, characterized in that the time shift is expressed in a number of shift images in the video stream between the most recent image of one channel and the image determined as the closest to the other channel.

8. A computer program product for determining a divergence score between two stereoscopic images of a scene comprising organ tissue and illuminated by a light source, the two images being a left image and a right image, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computer, steps of a method comprising: • a step of detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on organ tissue, • a step of defining a descriptor for each specular spot detected in each image, • a step of matching each specular spot of at least one of the two images with at least one specular spot of the other image,by searching for the specular spot closest to each specular spot to be matched, • a step of calculating a divergence score based on the distances between the descriptors of the matched specular spots and / or the number of matched spots.,

9. Device for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises: • a detection module in each image, by a trained machine learning model, of a plurality of spots specular corresponding to a plurality of specular reflections of the light source on an organ tissue, • a module for defining a descriptor for each specular spot detected in each image, • a module for matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, • a module for calculating a divergence score based on the distances between the descriptors of the paired specular spots and / or the number of paired spots.

10. Stereoscopic endoscopic imaging system comprising a stereoscopic endoscope (12), a light source (14), and a memory device, characterized in that it comprises a system (32) for synchronizing or resynchronizing a video stream of stereoscopic images transmitted by the stereoscopic endoscope and comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by the light source, said synchronization or resynchronization system comprising: • a module for receiving the video stream of stereoscopic images, • a module for saving, in said at least one memory device, a plurality of images of each channel, • a device for determining a divergence score according to claim 9, configured to determine a divergence score between a most recent image of one channel with the plurality of saved images of the other channel, • a module for determining the time shift of one channel compared to the other channel based on the divergence score calculated between the most recent image of one channel and the images of the other channel, • a channel synchronization or resynchronization module by applying the time shift to one of the channels determined so that the channels are synchronized.

Images