A method and apparatus for real-time correction of endoscope image distortion

Abstract

This invention provides a real-time correction method for endoscopic image distortion, comprising: placing an improved rigid endoscope system at a target position and setting operating conditions; in each frame of the image of the biological tissue acquired by the endoscope, firstly performing laser gating projection correction, and calculating a homogeneous transformation matrix between the projection distortion correction line image and the correction line reference image; then, acquiring the current endoscopic distortion image acquired by the CCD camera again, processing it using the homogeneous transformation matrix to obtain the endoscopic corrected image, and displaying the corrected image on a monitor for human observation, thus eliminating the distortion. Because the period of the pulsed laser projecting the correction line reference image is extremely short, much shorter than the human eye's visual persistence period, the observer will not perceive the laser gating projection correction process, ensuring the dynamic and real-time nature of image correction; moreover, the instantaneous exposure of the weak laser will not damage biological tissue, ensuring the medical safety of image correction.

Description

Technical Field

[0001] This invention belongs to the field of medical devices, specifically relating to a real-time correction method for endoscopic image distortion. Background Technology

[0002] An endoscope is a commonly used medical device that is inserted into the organ to be examined through natural openings in the body or small incisions made during minimally invasive surgery, enabling the diagnosis of diseases. Endoscopes are essential tools for detailed examinations in internal and surgical fields and for minimally invasive surgical treatments, and their diagnostic and therapeutic advantages are widely recognized in the medical community.

[0003] A rigid medical endoscope is an endoscope whose insertion section cannot be bent during operation. It is widely used for examinations and minimally invasive treatments of the ear, nose, throat, abdomen, uterus, chest, joints, and urinary tract. To improve the observation range of the endoscope, a large field of view (generally 120°) is required. However, the diameter of the endoscope lens is limited by the body cavities, so the overall size of the imaging system cannot be very large, and its imaging optical system cannot be very complex. Therefore, endoscopic optical imaging systems generally suffer from significant optical distortion, which can affect the doctor's accurate judgment of lesion locations.

[0004] Lens distortion is mainly classified into two categories: radial distortion and tangential distortion. Radial distortion is distortion distributed along the radius of the lens. It occurs because light bends more near the center of the lens than near the center. This distortion is more pronounced in ordinary, inexpensive lenses. Radial distortion mainly includes barrel distortion and pincushion distortion. Tangential distortion is caused by the lens itself not being parallel to the camera sensor plane or image plane. This is often due to installation misalignment when the lens is mounted on the lens module. Furthermore, for rigid medical endoscopes, the surface structure of the internal biological tissues being observed is complex, and it is impossible to guarantee that the lens will always be parallel to the image. The spatial position between the two changes in real time, resulting in real-time image distortion due to the three-dimensional imaging of biological tissues. Therefore, image distortion correction for endoscope systems can effectively improve image distortion and accurately reflect lesion sites, which is of great significance for rapid and precise lesion localization. However, current traditional distortion correction methods cannot achieve real-time, dynamic, and precise distortion correction. Summary of the Invention

[0005] To address the shortcomings of the prior art, this invention provides a real-time correction method for endoscopic image distortion. This method can perform real-time, dynamic, and precise correction of endoscopic image distortion, effectively improving image distortion.

[0006] This invention is achieved through the following technical solution:

[0007] A real-time correction method for endoscopic image distortion includes the following steps:

[0008] (1) Place the improved rigid endoscope system in the target position and set the working conditions;

[0009] The improved rigid endoscope system includes an endoscope tube, an outer endoscope tube, a laser gating projection correction device, an optical imaging system, and an illumination fiber optic cable.

[0010] Both the endoscope tube and the outer endoscope tube are hollow tubes. The endoscope tube is placed inside the cavity of the outer endoscope tube, and there is a radial gap between the endoscope tube and the outer endoscope tube.

[0011] The optical imaging system includes a display screen, an image processing unit, a CCD camera, a cylindrical lens group, and an objective lens. Both the cylindrical lens group and the objective lens are housed within the internal cavity of the endoscope tube. The objective lens is located at one end of the endoscope tube, and the other end is connected to the CCD camera. An objective lens glass is mounted outside the objective lens, and the objective lens glass is tightly welded or bonded to the endoscope tube cavity to seal it. The image acquired by the objective lens is transmitted through the cylindrical lens group and then imaged by the CCD camera. The CCD camera is connected to the image processing unit, and the processed image is displayed on the display screen.

[0012] The illumination fiber is disposed in the gap between the endoscope tube and the outer endoscope tube, and the gap is sealed at the objective end by a transparent glass. A light cone for receiving the illumination fiber is provided on the side wall of the outer endoscope tube. One end of the illumination fiber is connected to the light cone provided on the side wall of the outer endoscope tube, and the other end is connected to the objective end. Light is emitted from the objective end of the endoscope to illuminate the body.

[0013] The laser gating projection correction device includes a pulsed laser, a beam shaper, a semi-transparent mirror, and a synchronization controller. The synchronization controller is connected to the CCD camera and the pulsed laser by a control circuit to synchronize their working cycles.

[0014] The semi-transparent and semi-reflective mirror is fixedly installed in the gap between the CCD camera and the cylindrical lens group of the endoscope tube. Both the endoscope tube and the outer endoscope tube are provided with laser entrance holes for receiving pulsed laser beams. The laser entrance holes of the semi-transparent and semi-reflective mirror are positioned opposite each other, which can reflect the incident light in the horizontal direction into vertical coaxial light, while transmitting the incident light in the vertical direction.

[0015] The horizontal laser beam emitted by the pulsed laser is shaped into a geometrically linear beam by a beam shaper, called the correction line reference image. The geometrically linear beam is horizontally incident on a semi-transparent mirror through the laser entrance aperture. After being reflected by the semi-transparent mirror, it is vertically incident on the transparent glass at the objective lens end. It passes through the transparent glass and illuminates the surface of the target biological tissue, where a linear light pattern will appear, called the projection distortion correction line image. The projection distortion correction line image reflected from the surface of the target biological tissue will return to the semi-transparent mirror. After being transmitted through the semi-transparent mirror, it is incident on the CCD camera. Thus, the projection distortion correction line image formed by the laser reflected back from the surface of the target biological tissue can be obtained.

[0016] (2) In each frame of the image of biological tissue acquired by the endoscope, laser gating projection correction is first performed with a period of less than 10ms. That is, the synchronous controller of the laser gating projection correction device sends a synchronous start signal, so that the pulsed laser projects the reference image of the correction line, and the CCD camera acquires the image of the projection distortion correction line. The homogeneous transformation matrix is ​​calculated between the image of the projection distortion correction line and the reference image of the correction line. Then, the current endoscope distortion image acquired by the CCD camera is acquired again, and the homogeneous transformation matrix obtained earlier is used for calculation to obtain the endoscope correction image. The corrected image is displayed on the monitor for human observation, which is the image with the distortion eliminated.

[0017] Furthermore, the laser beam emitted by the pulsed laser is required to be non-harmful to the surface of human biological tissues, with a power of less than 50mW, a pulse width of less than 10ms, and a laser color in the visible light band.

[0018] Furthermore, the horizontal laser beam emitted by the pulsed laser is shaped into a crisscross pattern by a beam shaper.

[0019] The present invention also provides an improved rigid endoscope system, including an endoscope tube, an outer endoscope tube, a laser gating projection correction device, an optical imaging system, and an illumination optical fiber;

[0020] Both the endoscope tube and the outer endoscope tube are hollow tubes. The endoscope tube is placed inside the cavity of the outer endoscope tube, and there is a radial gap between the endoscope tube and the outer endoscope tube.

[0021] The optical imaging system includes a display screen, an image processing unit, a CCD camera, a cylindrical lens group, and an objective lens. Both the cylindrical lens group and the objective lens are housed within the internal cavity of the endoscope tube. The objective lens is located at one end of the endoscope tube, and the other end is connected to the CCD camera. An objective lens glass is mounted outside the objective lens, and the objective lens glass is tightly welded or bonded to the endoscope tube cavity to seal it. The image acquired by the objective lens is transmitted through the cylindrical lens group and then imaged by the CCD camera. The CCD camera is connected to the image processing unit, and the processed image is displayed on the display screen.

[0022] The illumination fiber is disposed in the gap between the endoscope tube and the outer endoscope tube, and the gap is sealed at the objective end by a transparent glass. A light cone for receiving the illumination fiber is provided on the side wall of the outer endoscope tube. One end of the illumination fiber is connected to the light cone provided on the side wall of the outer endoscope tube, and the other end is connected to the objective end. Light is emitted from the objective end of the endoscope to illuminate the body.

[0023] The laser gating projection correction device includes a pulsed laser, a beam shaper, a semi-transparent mirror, and a synchronization controller. The synchronization controller is connected to the CCD camera and the pulsed laser by a control circuit to synchronize their working cycles. The semi-transparent mirror is fixedly installed in the gap between the CCD camera and the cylindrical lens group in the endoscope tube. Both the endoscope tube and the outer endoscope tube are provided with laser entrance holes for receiving the pulsed laser beam. The laser entrance holes of the semi-transparent mirror are positioned opposite each other, which can reflect the incident light in the horizontal direction into vertical coaxial light, while transmitting the incident light in the vertical direction.

[0024] The present invention has the following beneficial effects:

[0025] This invention improves upon existing rigid endoscopes by introducing a laser-gated projection correction device. A designed correction line reference image is projected onto the surface of biological tissue, acquiring a projection distortion correction line image that comprehensively reflects the superposition of current lens distortion and the distortion of the observed tissue surface. A homogeneous transformation matrix between the projection distortion correction line image and the correction line reference image is then calculated. Based on this homogeneous transformation matrix, it is applied to endoscope distortion images with severe optical distortion acquired by an optical imaging system, achieving real-time, dynamic, and precise correction of endoscopic distortion images.

[0026] Because the pulsed laser of the laser gating projection correction device projects the reference image of the correction line in a very short period (less than 10ms), which is much shorter than the persistence of vision of the human eye, the observer will not be aware of the laser gating projection correction process. This ensures the dynamic and real-time nature of image correction. Moreover, the instantaneous exposure of weak laser of less than 50mW (less than 10ms) will not damage biological tissue, ensuring the medical safety of image correction. Attached Figure Description

[0027] Figure 1 This is a structural diagram of the improved rigid endoscope system described in this invention. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0029] This invention provides a real-time correction method for endoscopic image distortion, comprising the following steps:

[0030] (1) Place the improved rigid endoscope system in the target position and set the working conditions;

[0031] The improved rigid endoscope system includes an endoscope tube 1, an outer endoscope tube 2, a laser gating projection correction device, an optical imaging system, and an illumination fiber optic cable 3.

[0032] Both the endoscope tube 1 and the external endoscope tube 2 are hollow tubes. The endoscope tube 1 is placed inside the cavity of the external endoscope tube 2, and there is a radial gap between the endoscope tube 1 and the external endoscope tube 2.

[0033] The optical imaging system includes a display screen, an image processing unit, a CCD camera 4, a cylindrical lens group 7, and an objective lens 6. Both the cylindrical lens group 7 and the objective lens 6 are housed within the internal cavity of the endoscope tube 1. The objective lens 6 is located at one end of the endoscope tube 1 (referred to as the objective end), and the other end of the endoscope tube 1 is connected to the CCD camera 4. An objective lens glass is disposed outside the objective lens 6, and the objective lens glass is tightly welded or glued to the endoscope tube 1 to seal the cavity of the endoscope tube 1. The image acquired by the objective lens 6 is transmitted through the cylindrical lens group 7 and then imaged by the CCD camera 4. The CCD camera 4 is connected to the image processing unit and displays the processed image on the display screen.

[0034] The illumination fiber 3 is disposed in the gap between the endoscope tube 1 and the outer endoscope tube 2, and the gap is sealed at the objective end by a transparent glass. A light cone 5 is provided on the side wall of the outer endoscope tube 2 for connecting the illumination fiber 3. The illumination fiber 3 is used to conduct light; one end of the illumination fiber 3 is connected to the light cone 5 on the side wall of the outer endoscope tube 2, and the other end is connected to the objective end. Light exits the endoscope from the objective end, illuminating the body.

[0035] The laser gating projection correction device includes a pulsed laser, a beam shaper, a semi-transparent mirror 8, and a synchronization controller. The synchronization controller is connected to the CCD camera 4 and the pulsed laser by a control circuit to synchronize their working cycles.

[0036] A semi-transparent mirror 8 is fixedly installed in the gap between the CCD camera 4 and the cylindrical lens group 7 of the endoscope tube 1. Both the endoscope tube 1 and the outer endoscope tube 2 are provided with laser entrance holes for receiving pulsed laser beams. The semi-transparent mirror 8 is positioned opposite to the laser entrance holes, and can reflect horizontal incident light into vertical coaxial light while transmitting vertical incident light.

[0037] The laser beam emitted by the pulsed laser is required to be non-harmful to the surface of human biological tissues, i.e., the power is less than 50mW, the pulse width is less than 10ms, and the laser color can be in the visible light band for easy observation.

[0038] The horizontal laser beam emitted by the pulsed laser is shaped into a geometric linear beam (a grid shape is recommended) by the beam shaper, which can be called the calibration line reference image. It is horizontally incident on the semi-transparent mirror 8 through the laser entrance aperture. After being reflected by the semi-transparent mirror 8, it is vertically incident on the light-transmitting glass at the objective lens end. It passes through the light-transmitting glass and illuminates the surface of the target biological tissue. A linear light pattern will appear on the surface of the target biological tissue, which can be called the projection distortion correction line image. The projection distortion correction line image reflected by the surface of the target biological tissue will return to the semi-transparent mirror 8. After being transmitted by the semi-transparent mirror 8, it is incident on the CCD camera 4. Thus, the projection distortion correction line image formed by the laser reflected back from the surface of the target biological tissue can be obtained.

[0039] (2) In each frame of the biological tissue image acquired by the endoscope, laser gating projection correction is first performed with a period of less than 10ms. That is, the synchronous controller of the laser gating projection correction device sends a synchronous start signal, so that the pulsed laser projects the correction line reference image, and the CCD camera 4 acquires the projection distortion correction line image. The projection distortion correction line image and the correction line reference image are subjected to homogeneous transformation matrix calculation (one of the basic principles of computer graphics, which uses three-dimensional scaling, shearing, rotation, displacement and other geometric transformation operations to make the distortion correction line in the projection distortion correction line image coincide with the correction line reference in the correction line reference image within a certain allowable deviation range). Then, the current endoscope distortion image acquired by the CCD camera 4 is acquired again, and the homogeneous transformation matrix obtained earlier is used for calculation to obtain the endoscope correction image. The corrected image is displayed on the monitor for human observation, which is the image with the distortion eliminated.

[0040] Because the pulsed laser of the laser gating projection correction device projects the reference image of the correction line in a very short period (less than 10ms), which is much shorter than the persistence of vision of the human eye, the observer will not be aware of the laser gating projection correction process. This ensures the dynamic and real-time nature of image correction. Moreover, the instantaneous exposure of weak laser of less than 50mW (less than 10ms) will not damage biological tissue, ensuring the medical safety of image correction.

[0041] It will be apparent to those skilled in the art that the present invention can be modified in various ways, and such modifications are not considered to depart from the scope of the invention. All such modifications that are obvious to those skilled in the art are included within the scope of the claims.

Claims

1. A real-time correction method for endoscopic image distortion, characterized in that, Includes the following steps: Place the improved rigid endoscope system in the target location and set the operating conditions; The improved rigid endoscope system includes an endoscope tube (1), an outer endoscope tube (2), a laser gating projection correction device, an optical imaging system, and an illumination fiber (3). Both the endoscope tube (1) and the external endoscope tube (2) are hollow tubes. The endoscope tube (1) is placed inside the cavity of the external endoscope tube (2). There is a radial gap between the endoscope tube (1) and the external endoscope tube (2). The optical imaging system includes a display screen, an image processing unit, a CCD camera (4), a cylindrical lens group (7), and an objective lens (6). The cylindrical lens group (7) and the objective lens (6) are both located in the internal cavity of the endoscope tube (1). The objective lens (6) is located at one end of the endoscope tube (1), and the other end of the endoscope tube (1) is connected to the CCD camera (4). An objective lens glass is provided outside the objective lens (6), and the objective lens glass is sealed to the endoscope tube (1) cavity by tight welding or bonding. The image acquired by the objective lens (6) is transmitted through the cylindrical lens group (7) and then imaged by the CCD camera (4). The CCD camera (4) is connected to the image processing unit and displays the processed image on the display screen. The illumination fiber (3) is disposed in the gap between the endoscope tube (1) and the outer endoscope tube (2), and the gap is sealed at the objective end by a transparent glass; a light cone (5) for connecting the illumination fiber (3) is provided on the side wall of the outer endoscope tube (2); one end of the illumination fiber (3) is connected to the light cone (5) provided on the side wall of the outer endoscope tube (2), and the other end is connected to the objective end, and light is emitted from the objective end of the endoscope to illuminate the body; The laser gating projection correction device includes a pulsed laser, a beam shaper, a semi-transparent mirror (8) and a synchronization controller. The synchronization controller is connected to the CCD camera (4) and the pulsed laser by a control line to synchronize their working cycles. The semi-transparent and semi-reflective mirror (8) is fixedly installed in the gap between the CCD camera (4) and the cylindrical lens group (7) of the endoscope tube (1). Both the endoscope tube (1) and the outer endoscope tube (2) are provided with laser entrance holes for receiving pulsed laser beams. The semi-transparent and semi-reflective mirror (8) is positioned opposite to the two laser entrance holes, and can reflect the incident light in the horizontal direction into vertical coaxial light, while transmitting the incident light in the vertical direction. The horizontal laser beam emitted by the pulsed laser is shaped into a geometric linear beam by the beam shaper, which is called the correction line reference image. The geometric linear beam is horizontally incident on the semi-transparent mirror (8) through the laser entrance hole. After being reflected by the semi-transparent mirror (8), it is vertically incident on the light-transmitting glass at the objective lens end. It passes through the light-transmitting glass and irradiates the surface of the target biological tissue. A linear light pattern will appear on the surface of the target biological tissue, which is called the projection distortion correction line image. The projection distortion correction line image reflected by the surface of the target biological tissue will return to the semi-transparent mirror (8) along the same path. After being transmitted by the semi-transparent mirror (8), it is incident on the CCD camera (4). Thus, the projection distortion correction line image formed by the laser reflected back from the surface of the target biological tissue can be obtained. In each frame of the biological tissue image acquired by the endoscope, laser gating projection correction is first performed with a cycle of less than 10ms. That is, the synchronous controller of the laser gating projection correction device sends a synchronous start signal, so that the pulsed laser projects the reference image of the correction line, and at the same time the CCD camera (4) acquires the projection distortion correction line image; the projection distortion correction line image and the reference image of the correction line are used to calculate the homogeneous transformation matrix; then, the current endoscope distortion image acquired by the CCD camera (4) is acquired again, and the homogeneous transformation matrix obtained earlier is used for calculation and processing to obtain the endoscope correction image. The corrected image is displayed on the monitor for human observation, which is the image that has eliminated the distortion.

2. The real-time correction method for endoscopic image distortion according to claim 1, characterized in that, The laser beam emitted by the pulsed laser is required to be non-harmful to the surface of human biological tissues, with a power of less than 50mW, a pulse width of less than 10ms, and a laser color in the visible light band.

3. The real-time correction method for endoscopic image distortion according to claim 1, characterized in that, The horizontal laser beam emitted by the pulsed laser is shaped into a grid-shaped beam by a beam shaper.

4. An improved rigid endoscope system, characterized in that, It includes an endoscope tube (1), an exoscope tube (2), a laser gating projection correction device, an optical imaging system, and an illumination optical fiber (3); Both the endoscope tube (1) and the external endoscope tube (2) are hollow tubes. The endoscope tube (1) is placed inside the cavity of the external endoscope tube (2). There is a radial gap between the endoscope tube (1) and the external endoscope tube (2). The optical imaging system includes a display screen, an image processing unit, a CCD camera (4), a cylindrical lens group (7), and an objective lens (6). The cylindrical lens group (7) and the objective lens (6) are both located in the internal cavity of the endoscope tube (1). The objective lens (6) is located at one end of the endoscope tube (1), and the other end of the endoscope tube (1) is connected to the CCD camera (4). An objective lens glass is provided outside the objective lens (6), and the objective lens glass is sealed to the endoscope tube (1) cavity by tight welding or bonding. The image acquired by the objective lens (6) is transmitted through the cylindrical lens group (7) and then imaged by the CCD camera (4). The CCD camera (4) is connected to the image processing unit and displays the processed image on the display screen. The illumination fiber (3) is disposed in the gap between the endoscope tube (1) and the outer endoscope tube (2), and the gap is sealed at the objective end by a transparent glass; a light cone (5) for connecting the illumination fiber (3) is provided on the side wall of the outer endoscope tube (2); one end of the illumination fiber (3) is connected to the light cone (5) provided on the side wall of the outer endoscope tube (2), and the other end is connected to the objective end, and light is emitted from the objective end of the endoscope to illuminate the body; The laser gating projection correction device includes a pulsed laser, a beam shaper, a semi-transparent mirror (8), and a synchronization controller. The synchronization controller is connected to the CCD camera (4) and the pulsed laser by a control line to synchronize their working cycles. The semi-transparent mirror (8) is fixedly installed in the gap between the CCD camera (4) and the cylindrical lens group (7) of the endoscope tube (1). Both the endoscope tube (1) and the outer endoscope tube (2) are provided with laser entrance holes for receiving the pulsed laser beam. The semi-transparent mirror (8) is positioned opposite to the two laser entrance holes and can reflect the incident light in the horizontal direction into vertical coaxial light while transmitting the incident light in the vertical direction.

Images