Single-drive endoscope image splicing positioning actuator and image splicing method

Abstract

The application discloses a single-drive endoscope image splicing positioning execution mechanism and an image splicing method, which comprise a casing and an endoscope camera; the casing is supported with a coaxial spring, a positioning pressure disc, a rotating positioning rod and a motor; the positioning pressure disc is provided with a limiting pit; the rotating positioning rod is provided with a positioning convex ball; the rotating positioning rod is further provided with a fish-eye bearing at the tail end; the endoscope camera is fixed through a connecting rod connected to the fish-eye bearing and a universal joint supported on the casing; the motor drives the rotating positioning rod to rotate, and the positioning convex ball on the rotating positioning rod rotates to the position of the limiting pit on the positioning pressure disc; under the pressure of the spring, the convex ball is pressed into the limiting pit, the positioning of the rotating positioning rod is realized, the rotating positioning rod drives the connected endoscope camera to adjust the collection angle, and the reliability and accuracy of the collection endoscope image splicing are ensured.

Description

Technical Field

[0001] This invention belongs to the field of endoscope structure technology, specifically relating to a single-drive endoscope image stitching and positioning actuator and an image stitching method. Background Technology

[0002] Endoscopes, as a new type of medical diagnostic instrument, can be used to help doctors examine various mucosal lesions of hollow organs. In the context of this article, oral endoscopes are also included within hollow organs.

[0003] Because endoscopes need to observe the internal structures of hollow organs in the human body, they require large-view images or even panoramic images at short distances. However, cameras typically cannot capture panoramic images, and large-view images are prone to severe distortion. Therefore, distortion removal and stitching of oral endoscope images are necessary. Furthermore, image stitching requires objects in the stitched images to be in a static state (rigid bodies). If feature pixels in the image are active, such as during the peristalsis of hollow organs, the image algorithm for feature point matching will malfunction, leading to image distortion and stitching failure. Additionally, feature point matching requires a significant overlap between the images to be stitched to obtain multiple feature points for matching. Due to this overlap, the viewing angle cannot be very large, often requiring more images to achieve sufficient overlap and obtain enough feature points for computation.

[0004] Furthermore, because endoscopes need to be small enough to ensure comfortable use by users during examinations, the camera often needs to move in two image coordinate systems, x and y, during image stitching. Therefore, two drive actuators (usually stepper motors) are required, along with displacement measurement sensors on two directional axes, resulting in a large endoscope size and causing discomfort for users. Summary of the Invention

[0005] In view of the above, the purpose of this invention is to provide a single-drive endoscopic image stitching positioning actuator and image stitching method, which realizes the acquisition of the posture of each fixed position of the endoscopic camera, so as to obtain the homography matrix required for image stitching from each perspective before shooting, and ensure the reliability and accuracy of the acquired endoscopic image stitching.

[0006] To achieve the above-mentioned objectives, an embodiment provides a single-drive endoscopic image stitching and positioning actuator, including a housing and an endoscopic camera;

[0007] The housing supports a coaxial spring, a positioning plate, a rotating positioning rod, and a motor. The positioning plate has a limit recess, the rotating positioning rod has a positioning protruding ball, and the end of the rotating positioning rod is also equipped with a fisheye bearing.

[0008] The endoscope camera is secured by a connecting rod that is attached to the fisheye bearing and a universal joint that is supported on the housing;

[0009] The motor rotates, causing the rotating positioning rod to rotate. The positioning protrusion ball on the rotating positioning rod rotates to the position where the positioning pressure plate has a limiting recess. Under the pressure of the spring, the protrusion ball is pressed into the limiting recess, realizing the positioning of the rotating positioning rod. The rotating positioning rod drives the endoscope camera connected to it to adjust the acquisition angle.

[0010] Before shooting, the homography matrix required for stitching endoscopic images from various perspectives is obtained by taking calibration photos of the fixed positions corresponding to the limiting recesses.

[0011] Preferably, there are multiple limiting recesses, which are evenly distributed on the circumferential surface of the positioning pressure plate.

[0012] Preferably, there are multiple positioning protruding beads, which are centrally symmetrically distributed on the rotating positioning rod.

[0013] Preferably, a sensor is also installed at the axis of the rotating positioning rod to determine the position and angle of the rotating positioning rod.

[0014] Preferably, the positioning pressure plate is fixed and does not rotate, but moves along the axial direction.

[0015] Preferably, the universal joint is fixed to the front end of the endoscope camera, and the connecting rod is fixed to the rear end of the endoscope camera. The universal joint and the connecting rod connected to the fisheye bearing work together to make the optical axis of the endoscope camera always rotate in a conical shape around the intersection of the universal joint.

[0016] Preferably, the camera is calibrated at the factory using a checkerboard pattern to mark the corresponding positions of each limiting recess, and the homography matrix between the camera positions is calculated. This homography matrix can be generated at the factory before shooting and is used for subsequent image stitching calculations.

[0017] Preferably, the motor is a stepper motor.

[0018] The embodiment also provides an endoscopic image stitching method, the method comprising the following steps:

[0019] Using the aforementioned single-drive endoscopic image stitching positioning actuator, before shooting, the homography matrix required for stitching images from various perspectives is obtained by taking a calibration photograph of the fixed position corresponding to the limiting recess; during shooting, endoscopic images from various perspectives are acquired.

[0020] Endoscopic images from various perspectives are stitched together based on the homography matrix required for image stitching from each perspective.

[0021] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0022] The single-drive endoscopic image stitching and positioning actuator provided by this invention realizes single-volume sensing and positioning. Through the cooperation of the positioning pressure plate, the rotating positioning rod and the limiting recess and positioning protruding ball on it, it can realize image stitching of non-fixed feature points (peristaltic hollow organs), thereby achieving distortion-free image stitching.

[0023] When using the aforementioned single-drive endoscopic image stitching and positioning actuator for image acquisition and stitching, there is no need to calculate the feature points and point matching of the stitched image in real time on site, nor is there a need to calculate the homography matrix in real time on site. This reduces the requirements for the image processing chip, greatly reduces the computing power requirements, and significantly reduces power consumption.

[0024] The single-drive endoscopic image stitching and positioning actuator described above can process images of mucosa with motion (peristalsis) changes without stitching failure and does not require a specific area of ​​overlapping parts in the stitched images. This reduces the number of images to be taken, decreases the computational power required for stitching image processing, and increases real-time performance.

[0025] The single-drive endoscopic image stitching and positioning actuator described above, by employing a positioning hole method, eliminates errors from the actuator (stepper motor, 1.8-degree error) and the angle sensor, reducing the complexity of the control algorithm. Furthermore, the single-drive method reduces the device size, making it more suitable for endoscopes. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a three-dimensional structural schematic diagram of the single-drive endoscopic image stitching and positioning actuator provided in the embodiment;

[0028] Figure 2 The distribution of positioning protruding beads on the rotating positioning rod of the single-drive endoscope image stitching and positioning actuator provided in the embodiment;

[0029] Figure 3 This describes the distribution of the upper limit recesses on the positioning pressure plate of the single-drive endoscopic image stitching and positioning actuator provided in the embodiment.

[0030] Figure 4 This is a schematic diagram of the raised ball being pressed into the limiting recess provided in the embodiment;

[0031] Figure 5This is a schematic diagram of the observation angle of the endoscope camera under the positioning actuator provided in the embodiment;

[0032] In the diagram: 1-House, 2-Endoscope camera, 3-Spring, 4-Positioning pressure plate, 5-Rotating positioning rod, 6-Motor, 7-Fisheye bearing, 8-Universal joint, 9-Connecting rod, 10-Limiting recess, 11-Positioning protrusion ball. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0034] like Figures 1-3 As shown, the single-drive endoscope image stitching and positioning actuator provided in the embodiment includes a housing 1, an endoscope camera 2, a spring 3, a positioning pressure plate 4, a rotating positioning rod 5, a motor 6, a fisheye bearing 7, a universal joint 8, and a connecting rod 9.

[0035] A coaxial spring 3, positioning pressure plate 4, rotating positioning rod 5, and motor 6 are supported on the housing 1, with spring 3 pressing against the rotating positioning rod 5. Specifically, the spring 3, positioning pressure plate 4, rotating positioning rod 5, and motor 6 supported on the housing 1 are suspended in the air. A fisheye bearing 7 is located at the end of the rotating positioning rod 5 and is connected to the rear end of the endoscope camera 2 via a connecting rod 9. The front end of the endoscope camera 2 is connected to a universal joint 8, which is also fixed to the housing 1. Figure 3 As shown, the positioning pressure plate 4 is provided with a limiting recess 10, such as Figure 2 As shown, the rotating positioning rod 5 is provided with positioning protruding beads 11. Specifically, there are multiple limiting recesses 10, such as four, which are evenly distributed on the circumferential surface of the positioning pressure plate 4. There are multiple positioning protruding beads 11, such as two, which are centrally symmetrically distributed on the rotating positioning rod 5.

[0036] like Figure 4 As shown, the rotating positioning rod 5 is driven by the motor 6. The motor 6 rotates, causing the rotating positioning rod 5 to rotate. The positioning protrusion 11 on the rotating positioning rod 5 rotates to the position where the positioning pressure plate has a limiting recess 10. Under the pressure of the spring, the positioning protrusion 11 is pressed into the limiting recess 10, as shown. Figure 4 As shown in Figure A, the rotating positioning rod 5 is positioned, and the rotating positioning rod 5 drives the endoscope camera 2 connected to it to adjust the acquisition angle.

[0037] A sensor is also installed at the axis of the rotating positioning rod 5 to determine its position. Together with the positioning pressure plate 4, the rotating positioning rod 5, and its limiting recesses 10 and positioning protruding balls 11, the rotating positioning rod 5 is temporarily fixed in these positions under the drive of the motor 6. Therefore, the attitude (six quantities: displacement x, y, z; angle: pitch yaw, left and right rotation pitch, roll) of the endoscope camera 2 at the temporary fixed position (recess) is also fixed. Thus, each temporary fixed position can be calibrated using a checkerboard pattern, thereby calculating the homography matrix between two adjacent temporary fixed positions. In the actual stitching process, only one rotation of the motor 6 is needed to take a picture at the temporary fixed position. Then, using the pre-calibrated and calculated homography matrix, the pictures can be stitched together.

[0038] In this embodiment, the universal joint 8 serves as a component for transmitting power at varying angles. Its rotation axis center is coaxial with the optical axis of the endoscope camera 2. Furthermore, the universal joint 8 and the connecting rod connected to the fisheye bearing work together to ensure that the optical axis of the endoscope camera always rotates in a conical shape around the intersection of the universal joints. Figure 5 This is a stitched image of the camera's viewpoint in the x-direction (the same applies to the y-direction). It can be seen that even a small amount of overlap (even just one line overlap between the sensors) is sufficient to stitch together a large image. Here, the area between the gray lines represents the area captured when the camera is tilted up to its maximum angle, the area between the black lines represents the area captured when the camera is tilted down to its maximum angle, and the beige area represents the overlapping area. The overlapping area should be as small as possible. Furthermore, it can be observed that the tilt angle of the endoscope camera 2 can be approximately equal to the x and y field of view (FOV), minimizing the stitching overlap.

[0039] The embodiment also provides an endoscopic image stitching method, including the following steps:

[0040] Using the aforementioned single-drive endoscopic image stitching positioning actuator, before shooting, the homography matrix required for stitching images from various perspectives is obtained by taking a calibration photograph of the fixed position corresponding to the limiting recess; during shooting, endoscopic images from various perspectives are acquired.

[0041] Endoscopic images from various perspectives are stitched together based on the homography matrix required for image stitching from each perspective.

[0042] When using the aforementioned single-drive endoscopic image stitching and positioning actuator for image acquisition and stitching, there is no need to calculate the feature points and point matching of the stitched image in real time on site, nor is there a need to calculate the homography matrix in real time on site. This reduces the requirements for the image processing chip, greatly reduces the computing power requirements, and significantly reduces power consumption.

[0043] The single-drive endoscopic image stitching and positioning actuator described above can process images of mucosa with motion (peristalsis) changes without stitching failure and does not require a specific area of ​​overlapping parts in the stitched images. This reduces the number of images to be taken, decreases the computational power required for stitching image processing, and increases real-time performance.

[0044] The single-drive endoscopic image stitching and positioning actuator described above, by employing a positioning hole method, eliminates errors from the actuator (stepper motor, 1.8-degree error) and the angle sensor, reducing the complexity of the control algorithm. Furthermore, the single-drive method reduces the device size, making it more suitable for endoscopes.

[0045] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A single-drive endoscopic image stitching and positioning actuator, characterized in that, Including the casing and the endoscope camera; The housing supports a coaxial spring, a positioning pressure plate, a rotating positioning rod, and a motor. The positioning pressure plate has a limit recess, the rotating positioning rod has a positioning protrusion ball, and the end of the rotating positioning rod is also equipped with a fisheye bearing. A sensor is also installed at the axis of the rotating positioning rod to determine the position and angle of the rotating positioning rod. The endoscope camera is fixed by a connecting rod connected to the fisheye bearing and a universal joint supported on the housing. The universal joint is fixed at the front end of the endoscope camera, and the connecting rod is fixed at the rear end of the endoscope camera. The universal joint and the connecting rod connected to the fisheye bearing work together to make the optical axis of the endoscope camera always rotate in a conical shape around the intersection of the universal joints. The motor rotates, causing the rotating positioning rod to rotate. The positioning protrusion ball on the rotating positioning rod rotates to the position where the positioning pressure plate has a limiting recess. Under the pressure of the spring, the protrusion ball is pressed into the limiting recess, realizing the positioning of the rotating positioning rod. The rotating positioning rod drives the endoscope camera connected to it to adjust the acquisition angle. The camera is calibrated at the corresponding positioning position of each limiting pit by using a checkerboard pattern, and the homography matrix between the cameras at the positioning position is calculated. This homography matrix is ​​used for subsequent image stitching calculation. Before shooting, the homography matrix required for stitching endoscopic images from various perspectives is obtained by taking calibration photos of the fixed positions corresponding to the limiting recesses.

2. The single-drive endoscopic image stitching and positioning actuator according to claim 1, characterized in that, The limiting recesses are multiple and are evenly distributed on the circumferential surface of the positioning pressure plate.

3. The single-drive endoscopic image stitching and positioning actuator according to claim 1, characterized in that, The positioning protruding beads are multiple and are symmetrically distributed on the rotating positioning rod.

4. The single-drive endoscopic image stitching and positioning actuator according to claim 1, characterized in that, The positioning pressure plate is fixed and does not rotate, but it moves along the axial direction.

5. The single-drive endoscopic image stitching and positioning actuator according to claim 1, characterized in that, The motor is a stepper motor.

6. A method for stitching endoscopic images, characterized in that, The method includes the following steps: The single-drive endoscopic image stitching positioning actuator according to any one of claims 1-5 is used to obtain the homography matrix required for image stitching from each perspective by taking a calibration photo of the fixed position corresponding to the limiting recess before shooting, and to collect endoscopic images from each perspective during shooting. Endoscopic images from various perspectives are stitched together based on the homography matrix required for image stitching from each perspective.

Images