METHOD FOR DETERMINING A DIVERGENCE SCORE BETWEEN TWO IMAGES AND SYNCHRONIZATION METHOD, ASSOCIATED DEVICE AND SYSTEM

Abstract

METHOD FOR DETERMINING A DIVERGENCE SCORE BETWEEN TWO IMAGES AND SYNCHRONIZATION METHOD, ASSOCIATED DEVICE AND SYSTEM The invention relates to a method for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises a step (204) of detecting in each image a plurality of specular spots, a step (210) of matching specular spots of the two images, and a step (214) of calculating a divergence score between the two images as a function of the distances between the descriptors of the matched specular spots and / or the number of matched spots.The invention also relates to a synchronization method comprising, in particular, a step (112) of determining the time offset of one channel relative to another channel based on the divergence score and a step (114) of synchronizing the channels by applying the determined time offset to one of the channels. Figure for the abstract: Figure 1.

Description

Title of the invention: METHOD FOR DETERMINING A DIVERGENCE SCORE BETWEEN TWO IMAGES AND SYNCHRONIZATION METHOD, ASSOCIATED DEVICE AND SYSTEM Technical field of the invention

[0001] The invention relates to a method and a device for determining a score of di Vergence between two images from two channels of a stereoscopic video stream, particularly for augmented reality applications, notably for endoscopic procedures, especially laparoscopic ones, for example for surgical or robotic interventions. The invention relates more particularly to a method and an imaging system enabling the synchronization of the two channels based on this divergence score. Technological background

[0002] Celioscopy, also called laparoscopy, is a medical technique for visually exploring the inside of a patient's body using an endoscope, or more specifically a celioscope or laparoscope when used to observe the intra-abdominal and / or pelvic cavity of a patient. An endoscope generally includes a light source and a means of capturing the light, for example, optical fibers and / or a video sensor.

[0003] During laparoscopic surgery, the laparoscope allows direct or indirect visualization of the intra-abdominal cavity, enabling observation of the surgical site and direct intervention using surgical instruments. This surgical technique has the advantage of not requiring a large incision in the abdominal wall (unlike laparotomy or celiotomy), making it a minimally invasive technique.

[0004] Similarly, minimally invasive surgical procedures using an endoscope can be performed in the thoracic cavity (thoracoscopy) or in the pelvic cavity. These are generally referred to as endoscopic surgery or endoscopic surgery.

[0005] Recent technological advances have evolved laparoscopy from a simple view by medical personnel of the image of the area to be operated on to an augmented view, which allows additional information to be displayed on the screen on the image viewed in order to assist medical personnel during the operation.

[0006] In particular, computer vision techniques are used on the image The image obtained by the laparoscope in real time provides additional information through augmented reality. For example, a hidden structure within the organ, such as a tumor, can be displayed on the image. In particular, a surgical site (such as an incision) can be shown on the organ image. More generally, this refers to computer-guided procedures or surgeries.

[0007] The endoscope used in such operations may be a stereoscopic endoscope, stereo endoscope, or 3D endoscope, forming a binocular sensor that allows the capture of two stereoscopic video image streams comprising two channels, a left channel and a right channel, each captured by a dedicated RGB or multispectral sensor, or by a single sensor capturing two concatenated channels. These channels can thus each be displayed to the left or right eye of an endoscope user to allow three-dimensional (3D) vision.

[0008] During computer processing of the video stream from a video endoscope, for example for displaying augmented data, the channels are assumed to be synchronous, meaning that the images in the left channel and the images in the right channel processed at a given time t were acquired simultaneously. Synchronization of the streams is important for proper visualization and computer processing of the image pairs.

[0009] Some stereoscopic video sensors can mix the two channels into a single stream by interlacing or concatenating the images from the channels into a single stream.

[0010] 3D endoscopes used in the surgical context generally use two independent channels, which are processed by internal acquisition chains perfectly synchronized by trigger mechanisms and the channels are considered synchronous.

[0011] However, using the channels with software or a component not included in the internal acquisition chain can be a sensitive step because the internal synchronization mechanisms are generally not accessible. Synchronization may then no longer be guaranteed because it depends on the processing applied to each channel, the characteristics of each optical sensor, and also the physical length of the signal transmission lines.

[0012] More specifically, the presence of a recording device regularly causes a desynchronization of the channels because the recording device is not capable of opening the channels synchronously and maintaining a constant recording frequency. An initial time delay appears, as well as a variable delay related to the refresh rate.

[0013] Poor synchronization degrades or eliminates the perception of depth in the stereoscopic video stream, and therefore of the three-dimensional information associated with it. flow. Such degraded operation does not allow taking advantage of the additional information provided by stereoscopic vision and there is therefore no longer any advantage compared to a monocular system with a single sensor and a single channel, or even a disadvantage if the result distorts the user's 3D perception, especially if the shift is significant.

[0014] Solutions have been proposed to enable this synchronization of the channels.

[0015] A first group of solutions are hardware solutions, requiring the addition of external elements to enable synchronization, for example by a binary state light source (such as an LED), GPS positioning, oscillating objects, etc. These solutions have a major drawback that they are difficult to use in a surgical (or operating) context, firstly because the operating environment is one with controlled sanitary conditions, and secondly because the devices present in the operating environment are medical devices subject to standards that cannot be modified for the implementation of these hardware solutions.

[0016] A second group of solutions are software solutions, which are based solely on the parameters of the received signals, in particular metadata related to encoding, timestamp data or the content of the signals themselves, in particular images and image pixels.

[0017] Among these software solutions, they fall into several categories: - solutions using only the image, in particular its diffuse component: • brightness analysis: in an operational context, this analysis is not very informative because the brightness is controlled and almost constant; • Motion analysis: this analysis is not relevant in an operational context because the phases of interest may be quasi-static. In addition, the visual field can also be disturbed by the movements of high-frequency instruments; • Image similarity analysis, using robust image descriptors, such as temporal descriptors (e.g., vectors representing activity), or image-based descriptors (e.g., self-similarity matrices (SSMs)). Like motion analysis, the usefulness of this analysis is limited because the phases of interest may be quasi-static; • Image correlation using global descriptors such as frequency analysis or phase correlation, fingerprinting, histogram correlation, etc. These techniques generally do not meet the required accuracy. necessary within the operational framework; • the matching of descriptors (in particular the matching of key points) such as SIFT which uses local detectors and descriptors: this method also does not allow to meet the accuracy requirements necessary in the operational context. - solutions using timestamps: the use of a network protocol and a timestamp of the English type requires hardware and / or software modification of the operating frame equipment, which is strongly discouraged for health reasons and standards outlined above; - Solutions using encoding metadata: analyzing the video bitrate profile requires implementing video compression algorithms (for example, to save video files in a compressed format), which is incompatible with the desired real-time use within the operational framework. Furthermore, a single value to represent an image is not sufficiently discriminating.

[0018] The inventors have thus sought to provide a method enabling the synchronization of unsynchronized channels or the resynchronization of desynchronized channels, in particular a method based solely on the processing of images captured by the stereoscopic sensor. Objectives of the invention

[0019] The invention aims to provide on the one hand a method enabling the determination of a divergence score between two stereoscopic images, in particular from different channels.

[0020] The invention also aims to provide a method for synchronizing or resynchronizing images of a stream comprising two channels,

[0021] The invention aims to provide a method for determining a divergence score or a synchronization method particularly suited to images representing organ tissues.

[0022] The invention also aims to provide a computer program product, a device for determining a divergence score and an associated endoscopic imaging system.

[0023] The invention also aims to provide, in at least one embodiment, such methods, computer program, device and endoscopic imaging system. Description of the invention

[0024] To this end, the invention relates to a method for determining a divergence score between two stereoscopic images of a scene comprising tissues of an organ and illuminated by a light source, characterized in that it comprises the following steps: - a detection step in each image, by a trained machine learning model, of a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, - a step of defining a descriptor for each specular spot detected in each image, - a step of matching each specular spot from at least one of the two images with at least one specular spot from the other image, by searching for the specular spot closest to each specular spot to be matched, - a step of calculating a divergence score between the two images based on the distances between the descriptors of the paired specular spots and / or the number of paired spots.

[0025] A determination method according to the invention thus makes it possible to determine a divergence score solely from images by using the specular reflections generally present in significant quantities on organ tissues. The determination method is therefore simple to calculate and implement, while providing improved accuracy and robustness in scenes where specular reflections are ubiquitous. The divergence score, which can also be a divergence metric, makes it possible to determine the divergence or convergence (depending on the score value) of the two images for comparison between stereoscopic image streams to enable synchronization, when the divergence score determination method is performed within a stereoscopic image synchronization process.

[0026] Specular reflections are direct reflections associated with the mirror-like behavior of tissues, resulting in a localized and direct reflection of incident light in a limited set of directions where the light energy is concentrated. Specular reflections generally appear as a bright spot in the image because they induce sensor saturation. They occur most frequently on materials with low roughness, organ tissues (also called organic or biological tissues), liquids, and metals.

[0027] Images depicting scenes containing organ tissue, particularly scenes captured by an endoscope, notably during laparoscopy, generally contain a significant number of specular reflections on the tissues, thus enabling the implementation of this method. These specular reflections are even more pronounced when the elements of the scene are moist.

[0028] The reflection of light in the scene also includes a diffuse component which is sometimes used in existing synchronization solutions. Specular reflections are generally considered an image perturbation when analyzing the diffuse component.

[0029] The use of specular reflections for determining a divergence score between images is not incompatible with post-processing allowing the removal of specular spots on the image displayed on a viewing device, if these specular spots are considered to be annoying for the viewing of the stereoscopic video stream by a human operator.

[0030] The machine learning model, and in particular a machine learning model adapted to image processing and / or a deep learning model, for example a neural network, for example a convolutional neural network and in particular a fully convolutional neural network (or FCN). An example of a fully convolutional neural network, in particular for semantic segmentation, is described for example in the publication "LONG, Jonathan, SHELHAMER, Evan, and DARRELL, Trevor. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. p. 3431-3440."

[0031] Advantageously and according to the invention, in the step of detecting a plurality of specular spots, the method includes a substep of generating a segmentation mask comprising the descriptors of each specular spot.

[0032] According to this aspect of the invention, the use of a segmentation mask simplifies the processing of specular spots and thus allows for a faster execution of the determination process.

[0033] Advantageously and according to the invention, the determination method includes a step of isolating each specular spot detected in the specular spot detection step.

[0034] According to this aspect of the invention, isolating specular spots makes it easier to calculate the descriptor of each spot, particularly when the descriptor is the centroid of the spot. In particular, isolating specular spots simplifies the definition of a local descriptor for each spot, compared to a global descriptor that would apply to the entire image.

[0035] Advantageously and according to the invention, the descriptor defined in the definition step is a geometric descriptor and / or a radiometric descriptor and / or a localization descriptor.

[0036] The invention also relates to a method for synchronizing or resynchronizing a stereoscopic video stream comprising a left image channel, referred to as the left channel, and a right image channel, referred to as the right channel, which are asynchronous or asynchronous, the video stream comprising a succession of images representing a scene including organ tissues and illuminated by a light source, characterized in that it comprises the following steps: - a stage for receiving the video stream of stereoscopic images, - a saving step, in at least one memory device, of a plurality images from each channel, - a step of determining a divergence score between the most recent image of one channel and the plurality of saved images of the other channel according to a method for determining a divergence score according to the invention, - a step of determining the time offset of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel, - a step of synchronizing or resynchronizing the channels by applying to one of the channels the determined time offset so that the channels are synchronized.

[0037] A synchronization method thus allows, based on the determination of a divergence score, the synchronization of images from the two channels of the video stream to enable its use as a stereoscopic video stream and thereby guarantee the 3D vision of the stream images. Synchronization is simple and fast and therefore usable in real time during live capture in an operational context, but the method can also be used retrospectively on a recording. In particular, the divergence score makes it possible to characterize the proximity or distance between two images from different channels and thus to determine the images considered synchronous by comparing the scores of several images from each channel of the stream.

[0038] Throughout the application, the term "synchronization" will be used to refer to both synchronization and resynchronization, the difference between the two being based solely on the type of stream received and processed, depending on whether the channels of this stream have not been synchronized, or have already been synchronized in the processing chain but have undergone desynchronization.

[0039] The time shift determination step allows the time shift to be determined: - of the left channel relative to the right channel based on the divergence score calculated between the most recent image of the left channel and the images of the right channel, - of the right channel relative to the left channel based on the divergence score calculated between the most recent image of the right channel and the images of the left channel, - a combination of these last two possibilities, the determination being carried out successively or simultaneously for the two channels and the time offset between the channels is determined according to these two calculations.

[0040] The method may also include a temporal smoothing step enabling the processing of the video image stream to reduce noise. The temporal smoothing relies on a regression and / or motion model, for example a Kalman filter, a smoothing and mapping (SAM) method such as GTSAM, a sliding window averaging filter, and / or a constant acceleration model.

[0041] If the calculated divergence scores do not allow the identification of a pair of close images, a time shift cannot be determined. The process can then retain the last calculated time shift and continue the process with a subsequent image in the processed channel.

[0042] The divergence score can be combined with another existing score, in particular a score based on points of interest, to refine the determination of the image of one channel closest to the other channel.

[0043] Advantageously and according to the invention, the step of determining the time offset relative to the other channel comprises: - a sub-step comparing the divergence scores between the most recent image from one channel and the plurality of saved images from the other channel, - a sub-step of determining the saved image of one channel that is closest to the saved image of the other channel based on said comparison of divergence scores, - a sub-step of calculating the time offset between the left channel and the right channel of the stereoscopic image video stream by calculating the time offset between the most recent image of one channel and the image determined to be the closest of the other channel.

[0044] The image of one channel that is closest to the other channel depends on the value of the divergence score. This divergence score can be expressed in different ways; for example, the closest images can be considered to be those with the lowest divergence score value if a high divergence score value represents a greater divergence between the two images, or conversely, the closest images can be considered to be those with the highest divergence score value if a high divergence score value represents a smaller divergence between the two images.

[0045] Advantageously and according to the invention, the time offset is expressed as a number of offset frames in the video stream between the most recent frame of one channel and the frame determined to be the closest of the other channel.

[0046] According to this aspect of the invention, this definition of the time value makes it possible to simplify synchronization by applying an offset to one channel based on the number of frames ahead or behind the other channel.

[0047] The invention also relates to a computer program product for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, the two images being a left image and a right image, said computer program product comprising program code instructions for executing, when said computer program product is run on a computer, the steps of a method comprising: - a detection step in each image, by a trained machine learning model, of a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, - a step of defining a descriptor for each specular spot detected in each image, - a step of matching each specular spot from at least one of the two images with at least one specular spot from the other image, by searching for the specular spot closest to each specular spot to be matched, - a step of calculating a divergence score based on the distances between the descriptors of the matched specular patches and / or the number of matched patches.

[0048] Advantageously, the computer program product according to the invention is configured to implement the method for determining a divergence score according to the invention.

[0049] Advantageously, the method for determining a divergence score according to the invention is implemented by a computer program product according to the invention.

[0050] The invention also relates to a computer program product for the synchronization or resynchronization of a stereoscopic image video stream comprising a left image channel, referred to as the left channel, and a right image channel, referred to as the right channel, asynchronous or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, said computer program product comprising program code instructions for the execution, when said computer program product is executed on a computer, of steps of a synchronization or resynchronization process according to the invention.

[0051] The computer program product for synchronization or resynchronization advantageously includes the product's program code instructions computer program for determining a divergence score according to the invention.

[0052] The invention also relates to a device for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises: - a detection module in each image, by a trained machine learning model, of a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue, - a module for defining a descriptor for each specular spot detected in each image, - a matching module for each specular spot in at least one of the two images with at least one specular spot in the other image, by searching for the specular spot closest to each specular spot to be matched, - a module for calculating a divergence score based on the distances between the descriptors of the matched specular patches and / or the number of matched patches.

[0053] A module may, for example, consist of a computer device such as a computer, a set of computer devices, an electronic component or a set of electronic components, or, for example, a computer program, a set of computer programs, a computer program library or a computer program function executed by a computer device such as a computer, a set of computer devices, an electronic component or a set of electronic components.

[0054] Advantageously, the device for determining a divergence score according to the invention is configured to implement the method for determining a divergence score according to the invention.

[0055] Advantageously, the method for determining a divergence score according to the invention is implemented by a device for determining a divergence score according to the invention.

[0056] The invention also relates to a stereoscopic endoscopic imaging system comprising a stereoscopic endoscope, a light source, and a memory device, characterized in that it comprises a synchronization or resynchronization system for a video stream of stereoscopic images transmitted by the stereoscopic endoscope and comprising a left image channel, referred to as the left channel, and a right image channel, referred to as the right channel, which are asynchronous or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by the light source, said synchronization or resynchronization system comprising: - a module for receiving the video stream of stereoscopic images, - a backup module, in said at least one memory device, of a plurality of images from each channel, - a device for determining a divergence score according to the invention, configured to determine a divergence score between the most recent image of one channel and the plurality of saved images of the other channel, - a module for determining the time offset of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel, - a channel synchronization or resynchronization module by applying to one of the channels the determined time offset so that the channels are synchronized.

[0057] Preferably, the system is used for laparoscopic, thoracoscopic or pelvioscopic imaging.

[0058] The light source is preferably attached to the stereoscopic endoscope and / or arranged at a relatively short distance from the stereoscopic endoscope.

[0059] The invention also relates to a method for determining a divergence score, a method for synchronizing or resynchronizing, a computer program product, a device for determining a divergence score and an endoscopic imaging system characterized in combination by all or part of the characteristics mentioned above or below. List of figures

[0060] Other objects, features and advantages of the invention will become apparent from the following description, given by way of non-limiting example only, and which refers to the accompanying figures in which:

[0061] [Fig-1] is a schematic view of a synchronization or resynchronization process nisation according to an embodiment of the invention,

[0062] [Fig.2] is an image representing a scene comprising organ tissues and illuminated by a light source, such as taken for example by a stereoscopic endoscope of an endoscopic imaging system according to an embodiment of the invention,

[0063] [Fig.3] is a schematic view of an endoscopic imaging system according to an embodiment of the invention.

[0064] Detailed description of an embodiment of the invention

[0065] In the figures, the scales and proportions are not strictly respected for the purposes of illustration and clarity.

[0066] In addition, identical, similar or analogous elements are designated by the same references in all figures.

[0067] Fig. 1 schematically represents a 100 synchronization or resynchronization method according to an embodiment of the invention, referred to as the 100 synchronization method in the following description.

[0068] The synchronization method 100 aims to synchronize or resynchronize a video stream 102 of stereoscopic images comprising a left image channel, called left channel 102a, and a right image channel 102b, called right channel, which are asynchronous or desynchronized.

[0069] The video stream is in particular a stream comprising a succession of images representing a scene comprising organ tissues and illuminated by a light source, as seen in [Fig.3] described later.

[0070] The synchronization method 100 thus comprises a step 104 of receiving the stereoscopic image stream, followed by a step 106 of saving, in at least one memory device, a plurality of images from each channel. In this embodiment, a first series of images 108a from the left channel and a second series of images 108b from the right channel are saved in the same memory device or in a plurality of memory devices.

[0071] The saved images are compared to determine a time lag between the channels. To do this, the synchronization method 100 includes a step 110 for determining a divergence score between the most recent image of one channel and the plurality of saved images of the other channel. This step 110 includes, for each pair of images whose divergence score is determined, the execution of a method 200 for determining a divergence score according to an embodiment of the invention.

[0072] The method 200 for determining a divergence score between two stereoscopic images 202a, 202b of a scene comprising organ tissues and illuminated by a light source includes a step 204 of detecting, in each image, by a trained learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on an organ tissue. This step may also include a substep 204a of generating a segmentation mask comprising the descriptors of each specular spot.

[0073] The determination method 200 may include a step 206 of isolating each specular spot detected in the specular spot detection step, making it easier to define the descriptors.

[0074] Figure 2 is an image representing a scene comprising organ tissues and illuminated by a light source, such as that taken, for example, by a stereoscopic endoscope of an endoscopic imaging system according to an embodiment of the invention. The image represents organ tissues, here perirenal fat and renal parenchyma tissues. Organ tissues, particularly when moist, are smooth and shiny surfaces which generate a large number of specular spots due to illumination by the light source.

[0075] Returning to [Fig. 1], the determination method 200 includes a step 208 of defining a descriptor for each specular spot detected in each image. The descriptor is in particular a geometric descriptor, for example the shape of the spot and / or a radiometric (or colorimetric) descriptor and / or a location descriptor, for example the centroid of the spot, or other types of descriptor.

[0076] The determination method 200 then includes a step 210 of matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched. The matching can, for example, use a brute-force matching method aimed at finding the specular spot closest to the spot to be matched. This matching method is fast and compatible with real-time processing.

[0077] The determination process 200 can then include a step 212 of filtering the matched spots, so that the matched spots meet certain criteria, in particular proximity, for example implementing the Lowe criterion to ensure a good match.

[0078] The determination method 200 then includes a step 214 of calculating a divergence score between the two images as a function of the distances between the paired specular spots.

[0079] Returning to the synchronization process 100, the set of scores calculated between the saved images of each channel allows a step 112 of determining the time offset of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel.

[0080] Step 112 of determining the time offset includes: - a substep 112a comparing the divergence scores between the most recent image of one channel and the plurality of saved images of the other channel, - a substep 112b determining the saved image of one channel that is closest to the saved image of the other channel based on said comparison of divergence scores, - a sub-step 112c of calculating the time offset between the left channel and the right channel of the stereoscopic image video stream by calculating the time offset between the most recent image of one channel and the image determined to be the closest of the other channel.

[0081] The synchronization method 100 finally includes a step 114 of synchronizing or resynchronizing the channels by applying to one of the channels the determined time offset so that the channels are synchronized

[0082] Fig. 3 schematically represents an endoscopic imaging system 10, comprising a stereoscopic endoscope 12 configured to capture a scene including organ tissues and illuminated by a light source 14, preferably attached to the stereoscopic endoscope 12 and / or arranged at a low relative distance from the stereoscopic endoscope 12.

[0083] For example, during a laparoscopic operation, the endoscope allows images to be captured of a patient's abdominal cavity 50, including organs 52. The endoscope 12 captures the scene and provides two image streams: a left image stream or left channel providing a left image 20a and a right image stream or right channel providing a right image, in which the organ tissues 22a, 22b of the organs 52 visible in the scene appear, respectively. The two images 20a, 20b are, for example, displayed in a stereoscopic display device, such as stereoscopic glasses or any other system used during surgery to display images from a stereoscopic endoscope.

[0084] The endoscopic imaging system 10 includes a device 30 for processing the image stream from the endoscope, such as may be found in the prior art. To manage synchronization or resynchronization requirements, the endoscopic imaging system 10 includes a system 32 for synchronizing or resynchronizing the stereoscopic image video stream transmitted by the stereoscopic endoscope and: - a stereoscopic image video stream receiving module, - a backup module, in said at least one memory device, of a a plurality of images from each channel, - a device for determining a divergence score, configured to determine a divergence score between the most recent image of one channel and the plurality of saved images of the other channel, in particular using modules configured to execute a process as described with reference to [Fig. 1]. The device for determining the divergence score creates in particular a segmentation mask 34 allowing the localization of specular spots. - a module for determining the time lag of one channel relative to the other channel based on the divergence score calculated between the most recent image of one channel and the images of the other channel, - a channel synchronization or resynchronization module by applying to one of the channels the determined time offset so that the channels are synchronized.

[0085] The invention is not limited to the embodiment described. In particular, the invention is applicable to any type of endoscopic imaging system in the context of endoscopy, for example in the thoracic or pelvic cavity, as well as more generally to any system using a stereoscopic endoscope.

Claims

Claims

1. Method for determining a divergence score between two stereoscopic images of a scene comprising organ tissues (22a, 22b) and illuminated by a light source, characterized in that it comprises the following steps: • a step (204) of detecting in each image, by a trained machine learning model, a plurality of specular spots (300) corresponding to a plurality of specular reflections of the light source on an organ tissue, • a step (208) of defining a descriptor for each specular spot (300) detected in each image, • a step (210) of matching each specular spot (300) of at least one of the two images with at least one specular spot (300) of the other image, by searching for the specular spot (300) closest to each specular spot (300) to be matched,• a step (214) of calculating a divergence score between the two images as a function of the distances between the descriptors of the paired specular spots (300) and / or the number of paired spots.,

2. Method for determining a divergence score according to claim 1, characterized in that in the step (204) of detecting a plurality of specular spots (300), the method comprises a sub-step (204a) of generating a segmentation mask (34) comprising the descriptors of each specular spot.

3. Method for determining a divergence score according to one of claims 1 or 2, characterized in that it comprises a step (206) of isolating each specular spot (300) detected in the step (204) of detecting the specular spots.

4. Method for determining a divergence score according to one of claims 1 to 3, characterized in that the descriptor defined in the definition step is a geometric descriptor and / or a radiometric descriptor and / or a location descriptor.

5. Method for synchronizing or resynchronizing a video stream of stereoscopic images comprising a left image channel, called the left channel, and a right image channel, called the right channel, asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues (22a, 22b) and illuminated by a light source, characterized in that it comprises the following steps: • a step (104) of receiving the video stream of stereoscopic images, • a step (106) of saving, in at least one memory device, a plurality of images of each channel, • a step (110) of determining a divergence score between a most recent image of one channel with the plurality of saved images of the other channel according to a method (200) of determining a divergence score according to one of claims 1 to 4, • a step (112) of determining the time shift of one channel relative to the other channel as a function of the divergence score calculated between the most recent image of one channel and the images of the other channel, • a step (114) of synchronizing or resynchronizing the channels by applying to one of the channels the determined time shift so that the channels are synchronized.

6. Synchronization or resynchronization method according to claim 5, characterized in that the step (112) of determining the time offset relative to the other channel comprises: • a sub-step (112a) of comparing the divergence scores between the most recent image of one channel and the plurality of saved images of the other channel, • a sub-step (112b) of determining the saved image of one channel closest to the saved image of the other channel as a function of said comparison of the divergence scores, • a sub-step (112c) of calculating the time shift between the left channel and the right channel of the video stream of stereoscopic images by calculating the time shift between the image most recent from one channel and the image determined to be closest to the other channel.

7. Synchronization or resynchronization method according to one of claims 5 or 6, characterized in that the time shift is expressed in a number of shift images in the video stream between the most recent image of one channel and the image determined as the closest to the other channel.

8. A computer program product for determining a divergence score between two stereoscopic images of a scene comprising organ tissue and illuminated by a light source, the two images being a left image and a right image, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computer, steps of a method comprising: • a step of detecting in each image, by a trained machine learning model, a plurality of specular spots corresponding to a plurality of specular reflections of the light source on organ tissue, • a step of defining a descriptor for each specular spot detected in each image, • a step of matching each specular spot of at least one of the two images with at least one specular spot of the other image,by searching for the specular spot closest to each specular spot to be matched, • a step of calculating a divergence score based on the distances between the descriptors of the matched specular spots and / or the number of matched spots.,

9. Device for determining a divergence score between two stereoscopic images of a scene comprising organ tissues and illuminated by a light source, characterized in that it comprises: • a detection module in each image, by a trained machine learning model, of a plurality of spots specular corresponding to a plurality of specular reflections of the light source on an organ tissue, • a module for defining a descriptor for each specular spot detected in each image, • a module for matching each specular spot of at least one of the two images with at least one specular spot of the other image, by searching for the specular spot closest to each specular spot to be matched, • a module for calculating a divergence score based on the distances between the descriptors of the paired specular spots and / or the number of paired spots.

10. Stereoscopic endoscopic imaging system comprising a stereoscopic endoscope (12), a light source (14), and a memory device, characterized in that it comprises a system (32) for synchronizing or resynchronizing a video stream of stereoscopic images transmitted by the stereoscopic endoscope and comprising a left image channel, called the left channel, and a right image channel, called the right channel, which are asynchronized or desynchronized, the video stream comprising a succession of images representing a scene comprising organ tissues and illuminated by the light source, said synchronization or resynchronization system comprising: • a module for receiving the video stream of stereoscopic images, • a module for saving, in said at least one memory device, a plurality of images of each channel, • a device for determining a divergence score according to claim 9, configured to determine a divergence score between a most recent image of one channel with the plurality of saved images of the other channel, • a module for determining the time shift of one channel in relation to the other channel based on the divergence score calculated between the most recent image of one channel and the images of the other channel, • a channel synchronization or resynchronization module by applying the time shift to one of the channels determined so that the channels are synchronized.

Images