Intracavitary dynamic panoramic imaging methods, devices, equipment and media

Abstract

This disclosure provides a method, apparatus, device, and storage medium for intracavitary dynamic panoramic imaging. The method includes: performing a panoramic scan of the lesion area and its surrounding area using a laparoscope to acquire multiple frames of original images; stitching the multiple frames of original images together using an image stitching algorithm to obtain a panoramic image; calculating the pose relationship between the current surgical field image and the panoramic image, and outlining the region corresponding to the current surgical field image in the panoramic image; removing surgical instruments from the current surgical field image, and fusing the surgical field image after removing the surgical instruments with the corresponding region in the panoramic image to obtain an intraoperative panoramic image. This method can stitch together a panoramic image of the lesion area and its surrounding area, and during the operation, the region corresponding to the current surgical field is selected in real time in the panoramic image, and the panoramic image is dynamically updated as the operation progresses.

Description

Technical Field

[0001] This disclosure relates to the field of medical technology, and in particular to a method, device, electronic device and medium for intracavitary dynamic panoramic imaging. Background Technology

[0002] With the development of medical technology, minimally invasive surgery has become very common. Minimally invasive surgery not only reduces surgical risks but also significantly shortens postoperative recovery time. Endoscopes are widely used in clinical minimally invasive surgery. Traditional endoscopic surgery captures two-dimensional images of the local surgical area. These images provide a small surgical field, and when surgeons need to observe the area surrounding the lesion, they must move the endoscope to observe the surrounding area area by area, which is inefficient and hinders the surgeon's overall understanding of the area surrounding the lesion during the procedure. Summary of the Invention

[0003] In view of the above problems, the present invention provides an intracavitary dynamic panoramic imaging method to solve the above technical problems.

[0004] The first aspect of this disclosure provides a method for intracavitary dynamic panoramic imaging, comprising: performing a panoramic scan of the lesion area and its surrounding area using a laparoscope to acquire multiple frames of original images; stitching the multiple frames of original images together based on an image stitching algorithm to obtain a panoramic image; calculating the pose relationship between the current surgical field image and the panoramic image, and outlining the region corresponding to the current surgical field image in the panoramic image; removing surgical instruments from the current surgical field image, and fusing the current surgical field image after removing surgical instruments with the corresponding region in the panoramic image to obtain an intraoperative panoramic image.

[0005] According to an embodiment of this disclosure, the step of stitching together the multiple original images based on the image stitching algorithm to obtain a panoramic image includes: calculating a first homography matrix between each frame of the original images; generating a maximum spanning tree based on the first homography matrix to obtain a center image with the largest association weight among the multiple original images; projecting the other images in the multiple original images except the center image onto the plane where the center image is located, and fusing them with the center image to obtain the panoramic image.

[0006] According to an embodiment of this disclosure, before calculating the first homography matrix between the original images in each frame, the method further includes: calculating the pose relationship between the original images in each frame based on a feature matching algorithm; and removing redundant images and images whose quality does not meet preset conditions from the multiple original images based on the pose relationship.

[0007] According to an embodiment of this disclosure, calculating the pose relationship between the current surgical field image and the panoramic image, and outlining the region corresponding to the current surgical field image in the panoramic image includes: calculating the pose relationship between the current surgical field image and each of the original images, and obtaining a second homography matrix between the current surgical field image and the panoramic image; calculating the coordinates of the vertex coordinates of the current surgical field image mapped to the coordinates in the panoramic image based on the second homography matrix, and outlining the region corresponding to the current surgical field image in the panoramic image using the position of the coordinates as the vertex.

[0008] According to an embodiment of this disclosure, removing surgical instruments from the current surgical field image includes: acquiring surgical field images from two time points before and after the current time, wherein the positions of the surgical instruments in the surgical field images are different from those in the current surgical field image; segmenting the current surgical field image and the surgical instruments in the surgical field images using an image segmentation algorithm based on a convolutional neural network; and fusing the current surgical field image after segmenting the surgical instruments and the surgical field image to obtain the current surgical field image after removing the surgical instruments.

[0009] According to an embodiment of this disclosure, the method further includes: segmenting the instrument from the current surgical field image and setting the pixel value of the region where the instrument is located in the current surgical field image to 0; calculating the pose transformation relationship between the current surgical field image and each frame of the original image, and stitching them together to obtain an updated panoramic image.

[0010] According to embodiments of this disclosure, the method further includes: if the original image is covered by other original images in the updated panoramic image, then replacing the original image with the current surgical field image.

[0011] A second aspect of this disclosure provides an intracavitary dynamic panoramic imaging device, comprising: an original image acquisition module for performing panoramic scanning of the lesion area and its surrounding area using a laparoscope to acquire multiple frames of original images; an image stitching module for stitching the multiple frames of original images based on an image stitching algorithm to obtain a panoramic image; a position calibration module for calculating the pose relationship between the current surgical field image and the panoramic image, and outlining the region corresponding to the current surgical field image in the panoramic image; and an image fusion module for removing surgical instruments from the current surgical field image and fusing the current surgical field image after removing surgical instruments with the corresponding region in the panoramic image to obtain an intraoperative panoramic image.

[0012] A third aspect of this disclosure provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the various steps of the intracavitary dynamic panoramic imaging method.

[0013] A fourth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the various steps of the intracavitary dynamic panoramic imaging method.

[0014] The above-described at least one technical solution adopted in the embodiments of this disclosure can achieve the following beneficial effects:

[0015] The present disclosure provides an intracavitary dynamic panoramic imaging method that can stitch together a panoramic image of the lesion area and its surrounding area. During the operation, the area corresponding to the current surgical field is selected in real time in the panoramic image, and the panoramic image is dynamically updated as the operation progresses. Attached Figure Description

[0016] To gain a more complete understanding of this disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, wherein:

[0017] Figure 1 The illustration shows a schematic diagram of an intracavitary dynamic panoramic imaging method provided in an embodiment of the present disclosure;

[0018] Figure 2 This illustration schematically shows a diagram of an image coordinate mapping provided by an embodiment of the present disclosure;

[0019] Figure 3 This illustration schematically shows an image stitching diagram provided by an embodiment of the present disclosure;

[0020] Figure 4 This schematic diagram illustrates a structural block diagram of an intracavitary dynamic panoramic imaging device provided in an embodiment of the present disclosure;

[0021] Figure 5 The schematic diagram illustrates a structural block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0022] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] The accompanying drawings show some block diagrams and / or flowcharts. It should be understood that some blocks or combinations thereof in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create means for implementing the functions / operations described in these block diagrams and / or flowcharts.

[0026] Therefore, the technology disclosed herein can be implemented in hardware and / or software (including firmware, microcode, etc.). Additionally, the technology disclosed herein can take the form of a computer program product stored on a computer-readable medium, which can be used by or in conjunction with an instruction execution system. In the context of this disclosure, a computer-readable medium can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, a computer-readable medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, apparatuses, or propagation media. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tape or hard disk drives (HDDs); optical storage devices, such as optical discs (CD-ROMs); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0027] The intracavitary dynamic panoramic imaging method provided in this embodiment can stitch together a panoramic image of the lesion area and its surrounding area. During the operation, the area corresponding to the current surgical field is selected in real time in the panoramic image, and the panoramic image is dynamically updated as the operation progresses.

[0028] Figure 1 The illustration shows a schematic diagram of an intracavitary dynamic panoramic imaging method provided by an embodiment of the present disclosure.

[0029] like Figure 1 As shown, this embodiment of the present disclosure provides an intracavitary dynamic panoramic imaging method, including operations S110 to S140.

[0030] The S110 uses a laparoscope to perform a panoramic scan of the lesion area and its surrounding area, acquiring multiple frames of raw images.

[0031] At the start of the surgery, a panoramic scan of the lesion area and its surrounding area is performed using a laparoscope, and each frame of the original image is recorded in real time during the scan.

[0032] After acquiring the original images, redundant images and images whose quality does not meet preset conditions can be removed from multiple original images based on pose relationships. Specifically, feature extraction algorithms such as SIFT, SURF, and ORB can be applied to extract feature points from each frame of the original images, and feature matching algorithms can be applied to calculate the pose relationships of each frame. Redundant images and images with poor quality are removed based on pose relationships, and the final selected photos are denoted as I1, I2, ..., I... n .

[0033] S120 uses an image stitching algorithm to stitch together multiple original images to obtain a panoramic image.

[0034] S120 may specifically include S121 to S123.

[0035] S121, calculate the first homography matrix between the original images of each frame.

[0036] S122, generate the maximum spanning tree based on the first homography matrix, and obtain the center image with the largest correlation weight among the multiple original images.

[0037] S123 projects all images except the central image from multiple original images onto the plane where the central image is located, and merges them with the central image to obtain a panoramic image.

[0038] Assuming image I j Feature points of a plane converted to image I i The transformation matrix of the plane is denoted as H. ij .

[0039] pi = H ij pj;

[0040] Where pi represents I i The pixel coordinates of the plane are (u, v), and pj represents I. j The pixel coordinates (u', v') of the plane. H ij It is a 3x3 matrix.

[0041]

[0042] Calculation yields I1~I n Transformation matrix H between original images ij Then, a maximum spanning tree is generated based on the matching relationships between images, and the center image with the highest association weight with other images is obtained, denoted as I. c .

[0043] Remove I c outside I1~I n All original images are projected onto I cThe image is then processed in a plane, and an image fusion algorithm is applied to generate the final panoramic image I. g , among which, I g with I c They are on the same plane.

[0044] S130, calculate the pose relationship between the current surgical field image and the panoramic image, and outline the region corresponding to the current surgical field image in the panoramic image.

[0045] During surgery, a panoramic image is typically displayed on one monitor, while a real-time local image of the current surgical field is displayed on another monitor. The current surgical field image (I) is calculated in real-time during the procedure. m With each original image I1~I n The pose relationship between them is determined, and the current surgical field image I is obtained. m With the plane I where the panoramic image is located g The second homography matrix H between gm .

[0046] Figure 2 The illustration shows a schematic diagram of an image coordinate mapping provided by an embodiment of the present disclosure.

[0047] like Figure 2 As shown, based on the second homography matrix H gm Calculate the vertex coordinates p1, p2, p3, and p4 of the current surgical field image and map them to the coordinates p1', p2', p3', and p4' in the panoramic image. Then, draw a bounding box with the positions of coordinates p1', p2', p3', and p4' as vertices to obtain the region corresponding to the current surgical field image in the panoramic image.

[0048] S140, Remove surgical instruments from the current surgical field image, and fuse the current surgical field image after removing surgical instruments with the corresponding area in the panoramic image to obtain the intraoperative panoramic image.

[0049] In this embodiment, the panoramic image can be dynamically updated intraoperatively. The first panoramic image obtained at the start of the surgery does not contain any instruments, because the surgery has just begun, and the instruments can be removed from the lesion area before panoramic scanning. However, during the surgery, the surgeon operates the surgical instruments, and the instruments will always be present in the current surgical field image. Therefore, when updating the current surgical field image to the panoramic image, it is necessary to first eliminate the influence of the instruments in the image.

[0050] In this embodiment, an image segmentation algorithm based on a convolutional neural network (such as the UET model) is applied to segment the instruments in the current surgical field image before panoramic stitching. Specifically, removing surgical instruments from the current surgical field image includes steps S141 to S143.

[0051] S141, acquire surgical field images of the two moments before and after the current moment, where the positions of surgical instruments differ between the surgical field images and the current surgical field image.

[0052] S142, the current surgical field image and the surgical instruments in the surgical field image are segmented using an image segmentation algorithm based on a convolutional neural network.

[0053] S143, merge the current surgical field image after segmenting the surgical instruments with the current surgical field image to obtain the current surgical field image after removing the surgical instruments.

[0054] During surgery, as the instruments are manipulated, their positions change continuously, thus obscuring different lesion tissues, such as... Figure 3 As shown, the positions of the instruments are represented at the current time *ti* and at the preceding and following times *ti-1* and *ti+1*, respectively. These three times correspond to three surgical field images, from which the instruments are segmented. The pixel values ​​of the region where the instruments are located in the current surgical field image are set to 0. The pose transformation relationship between the current surgical field image and each frame of the original image is calculated. By stitching and fusing these three images, a panoramic image without the instruments can be obtained.

[0055] For panoramic image I g The update first starts from the current surgical field image I m The instrument is segmented from the data, and the pixel values ​​of the region containing the instrument are set to 0. I is then calculated. m with I1~I n The pose transformation relationships of all original images are then analyzed, and a new panoramic image is stitched together. The original panoramic images are listed in I1 to I2. n If an image does not contribute to the new panoramic image (i.e., the area covered by this image is covered by other original images in the updated panoramic image), then the current surgical field image replaces the original image. In this case, the image is removed from the list of original images for the panoramic image, and the current image I is set to... m Add the original images to the list of the latest panoramic images, denoted as I1, I2, ..., I... k .

[0056] Figure 4 The schematic diagram illustrates a structural block diagram of an intracavitary dynamic panoramic imaging device provided in an embodiment of the present disclosure.

[0057] like Figure 4 As shown, this embodiment of the present disclosure provides an intracavitary dynamic panoramic imaging device, including: an original image acquisition module 410, an image stitching module 420, a position calibration module 430, and an image fusion module 440.

[0058] The original image acquisition module 410 is used to perform panoramic scanning of the lesion area and its surrounding area through a laparoscope to acquire multiple frames of original images.

[0059] The image stitching module 420 is used to stitch together multiple original images based on an image stitching algorithm to obtain a panoramic image.

[0060] The position calibration module 430 is used to calculate the pose relationship between the current surgical field image and the panoramic image, and to outline the region corresponding to the current surgical field image in the panoramic image.

[0061] The image fusion module 440 is used to remove surgical instruments from the current surgical field image and fuse the current surgical field image after removing surgical instruments with the corresponding area in the panoramic image to obtain an intraoperative panoramic image.

[0062] It is understood that the original image acquisition module 410, the image stitching module 420, the position calibration module 430, and the image fusion module 440 can be implemented in one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the original image acquisition module 410, the image stitching module 420, the position calibration module 430, and the image fusion module 440 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in a suitable combination of software, hardware, and firmware implementations. Alternatively, at least one of the original image acquisition module 410, image stitching module 420, position calibration module 430, and image fusion module 440 can be at least partially implemented as a computer program module, which can perform the functions of the corresponding module when the program is run by a computer.

[0063] Figure 5 The schematic diagram illustrates a structural block diagram of an electronic device provided in an embodiment of the present disclosure.

[0064] like Figure 5 As shown, the electronic device described in this embodiment includes: electronic device 500 including processor 510 and computer-readable storage medium 520. This electronic device 500 can perform the functions described above. Figure 1 The described method enables the detection of specific operations.

[0065] Specifically, processor 510 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 510 may also include onboard memory for caching purposes. Processor 510 may be used for executing reference... Figure 1 The method flow described according to embodiments of this disclosure refers to a single processing unit or multiple processing units performing different actions.

[0066] Computer-readable storage medium 520 can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, readable storage media can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, apparatuses, or propagation media. Specific examples of readable storage media include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); memories such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0067] Computer-readable storage medium 520 may include computer program 521, which may include code / computer-executable instructions that, when executed by processor 510, cause processor 510 to perform, for example, the above-described combination. Figure 1 The described method and any variations thereof.

[0068] Computer program 521 can be configured to have computer program code, for example, including computer program modules. For example, in an exemplary embodiment, the code in computer program 521 may include one or more program modules, such as 521A, module 521B, ... It should be noted that the division and number of modules are not fixed. Those skilled in the art can use appropriate program modules or combinations of program modules according to the actual situation. When these combinations of program modules are executed by processor 510, the processor 510 can perform, for example, the above-described combinations... Figure 1 The described method and any variations thereof.

[0069] According to an embodiment of the present invention, at least one of the original image acquisition module 410, the image stitching module 420, the position calibration module 430, and the image fusion module 440 can be implemented as a reference. Figure 5 The described computer program module, when executed by processor 510, can perform the corresponding operations described above.

[0070] This disclosure also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0071] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0072] Although this disclosure has been shown and described with reference to specific exemplary embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made to this disclosure without departing from the spirit and scope of the disclosure as defined by the appended claims and their equivalents. Therefore, the scope of this disclosure should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents.

Claims

1. An intracavitary dynamic panoramic imaging device, characterized in that, include: The raw image acquisition module is used to perform panoramic scanning of the lesion area and its surrounding area through the endoscope to acquire multiple frames of raw images. The image stitching module is used to stitch together the multiple original images based on an image stitching algorithm to obtain a panoramic image; The position calibration module is used to calculate the pose relationship between the current surgical field image and the panoramic image, and to outline the region corresponding to the current surgical field image in the panoramic image; An image fusion module is used to remove surgical instruments from the current surgical field image and fuse the current surgical field image after removing surgical instruments with the corresponding region in the panoramic image to obtain an intraoperative panoramic image. The removal of surgical instruments from the current surgical field image includes: Acquire surgical field images of two moments before and after the current moment, wherein the positions of surgical instruments in the surgical field images of the two moments before and after the current moment are different from those in the current surgical field image; The surgical instruments in the current surgical field image and the surgical field images of the two moments before and after the current moment are segmented using an image segmentation algorithm based on a convolutional neural network. The current surgical field image after the surgical instruments have been segmented is fused with the surgical field images from two moments before and after the current moment to obtain the current surgical field image after the surgical instruments have been removed.

2. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the computer program is executed by the processor, it performs the following steps: A panoramic scan of the lesion area and its surrounding area is performed using a laparoscope to obtain multiple frames of raw images. The multiple original images are stitched together using an image stitching algorithm to obtain a panoramic image; Calculate the pose relationship between the current surgical field image and the panoramic image, and outline the region corresponding to the current surgical field image in the panoramic image; The surgical instruments are removed from the current surgical field image, and the current surgical field image after removing the surgical instruments is fused with the corresponding region in the panoramic image to obtain an intraoperative panoramic image. The removal of surgical instruments from the current surgical field image includes: Acquire surgical field images of two moments before and after the current moment, wherein the positions of surgical instruments in the surgical field images of the two moments before and after the current moment are different from those in the current surgical field image; The surgical instruments in the current surgical field image and the surgical field images of the two moments before and after the current moment are segmented using an image segmentation algorithm based on a convolutional neural network. The current surgical field image after the surgical instruments have been segmented is fused with the surgical field images from two moments before and after the current moment to obtain the current surgical field image after the surgical instruments have been removed.

3. The electronic device according to claim 2, characterized in that, The image stitching algorithm stitches together the multiple original images to obtain a panoramic image, including: Calculate the first homography matrix between the original images in each frame; A maximum spanning tree is generated based on the first homography matrix to obtain the center image with the largest association weight in the multi-frame original images; The images other than the central image in the multiple original images are projected onto the plane where the central image is located and fused with the central image to obtain the panoramic image.

4. The electronic device according to claim 3, characterized in that, Before calculating the first homography matrix between the original images in each frame, the electronic device is further configured to: The pose relationship between the original images in each frame is calculated based on the feature matching algorithm; Based on the pose relationship, redundant images and images whose quality does not meet the preset conditions are removed from the multi-frame original images.

5. The electronic device according to claim 2, characterized in that, The step of calculating the pose relationship between the current surgical field image and the panoramic image, and outlining the region corresponding to the current surgical field image in the panoramic image, includes: Calculate the pose relationship between the current surgical field image and each of the original images, and obtain the second homography matrix between the current surgical field image and the panoramic image; Based on the second homography matrix, the vertex coordinates of the current surgical field image are mapped to the coordinates in the panoramic image, and the position of the coordinates is used as the vertex to draw a frame, thereby obtaining the region corresponding to the current surgical field image in the panoramic image.

6. The electronic device according to claim 2, characterized in that, The electronic device is also configured to: The instrument is segmented from the current surgical field image, and the pixel value of the region where the instrument is located in the current surgical field image is set to 0; The pose transformation relationship between the current surgical field image and each frame of the original image is calculated, and the images are stitched together to obtain an updated panoramic image.

7. The electronic device according to claim 6, characterized in that, The electronic device is also configured to: If the original image is covered by another original image in the updated panoramic image, then the current surgical field image replaces the original image.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the following steps: A panoramic scan of the lesion area and its surrounding area is performed using a laparoscope to obtain multiple frames of raw images. The multiple original images are stitched together using an image stitching algorithm to obtain a panoramic image; Calculate the pose relationship between the current surgical field image and the panoramic image, and outline the region corresponding to the current surgical field image in the panoramic image; The surgical instruments are removed from the current surgical field image, and the current surgical field image after removing the surgical instruments is fused with the corresponding region in the panoramic image to obtain an intraoperative panoramic image. The removal of surgical instruments from the current surgical field image includes: Acquire surgical field images of two moments before and after the current moment, wherein the positions of surgical instruments in the surgical field images of the two moments before and after the current moment are different from those in the current surgical field image; The surgical instruments in the current surgical field image and the surgical field images of the two moments before and after the current moment are segmented using an image segmentation algorithm based on a convolutional neural network. The current surgical field image after the surgical instruments have been segmented is fused with the surgical field images from two moments before and after the current moment to obtain the current surgical field image after the surgical instruments have been removed.

Images