A visual bending forceps image anti-shake method and system

Abstract

The present invention discloses a visual curved forceps image anti-shake method and system, which relates to the field of image communication technology, including an image acquisition module, an image processing module, a data transmission module, a display module, etc. The electronic anti-shake technology is adopted, and a high-performance image acquisition module is integrated in the limited clamp body space of the visual curved forceps, so as to give full play to the performance of the camera, achieve the effect of response delay time less than 20ms, stable output of 25 frames of images per second, and each frame of image is 512×512 pixels; the camera working state is divided into a mobile working state and a stationary working state, and image anti-shake calculation is only performed in the stationary working state, accurately matching the scene requirements, saving computing power, and making the image display more scientific and efficient; abandoning the complex feature point matching algorithm, ingeniously introducing an infrared light source, and calculating the distance from the camera to the photographed part in real time through the infrared light source, accurately realizing the anti-shake calculation, and at the same time, improving the tissue development effect in the stationary working state.

Description

Technical Field

[0001] The present invention relates to the field of image communication technology, and in particular to a visual curved forceps image anti-shake method and system. Background Art

[0002] The image stabilization technology of the visualized curved forceps is mainly used to reduce the image jitter caused by external interference or movement of surgical instruments during surgery. Existing image stabilization technologies can be divided into two categories: "optical image stabilization" and "electronic image stabilization". Optical image stabilization adjusts the optical path to offset external jitter by adding a moving lens group or compensation lens inside the camera according to the real-time collected motion data (such as acceleration, angular velocity). The advantages of optical image stabilization are strong real-time performance, excellent compensation effect for tiny jitters, and can effectively maintain high image quality. At the same time, it does not rely on image post-processing, the original image is not distorted, and complete details are retained. However, the application volume of optical image stabilization technology is large, and it is not suitable for embedding into the end of the curved forceps with limited size. Its hardware is also relatively complex, with high manufacturing costs and difficult maintenance. At the same time, the compensation effect for large-scale and rapid movements is limited.

[0003] Electronic image stabilization is based on the data of motion sensors (such as BMI270) and uses algorithms to digitally correct images. For example, the amount of jitter is calculated using angular velocity and acceleration data, and then compensated by rotating, translating, and cropping the image. Compared with optical image stabilization, it does not require additional mechanical structures and is suitable for miniaturized devices. At the same time, it is low-cost and easy to integrate into existing camera modules. The algorithm can also be optimized through software upgrades, and it is highly adaptable. Summary of the invention

[0004] The purpose of the present invention is to provide an image "electronic anti-shake" method and system for use in a narrow installation space (caliper body with an outer diameter of 5mm-8mm), with a response delay time of less than 20ms, a stable output of 25 frames of images per second, and each frame of image being 512×512 pixels.

[0005] In view of the above technical problems, the technical solution adopted by the present invention is: a visual bending forceps image anti-shake method, comprising the following steps:

[0006] Step 1: Regularly collect images with a pixel size of 1024×1024 through an image acquisition module, which includes a camera, a pair of infrared light sources, a pair of visible light sources, and a posture sensor;

[0007] Step 2: The visible light source is always on during use, and the motion parameters of the camera are collected and fed back through the posture sensor at regular intervals. The collection and feedback are collected every 40ms, including three rotation angular velocity data around the XYZ axis and three movement acceleration data along the XYZ axis. The camera's motion parameters are used to determine whether the camera is in a stationary working state;

[0008] Step 3: When the camera is in a static state, it starts collecting images from 0ms and collects them every 40ms. The pixels of the collected images are 1024×1024. i +1 image data storage matrix is ,in, i =0, 1, ..., N;

[0009] Step 4: The posture sensor starts collecting motion parameter data from 0ms and collects it every 40ms. i When taking +1 image, the optical center coordinate system of the camera is the dynamic coordinate system. When taking the first image, the dynamic coordinate system of the camera is also the fixed coordinate system. i +1 The data collected is used to calculate the rotation transformation matrix relative to the fixed coordinate system , the translation transformation matrix relative to the fixed coordinate system ;

[0010] Step 5: When the camera is in a stationary state, the two infrared light sources are always on, and the camera starts collecting infrared images from 20ms and collects them every 40ms. The distance between the camera and the object being photographed is calculated by the number of infrared pixels in the infrared image. ;

[0011] Step 6: Construct the image frame matrix ; Stored The data of the central 512×512 pixel area, The row coordinate of the pixel value is m, and the column coordinate is n; starting from the second picture taken, through the i +1 The coordinate transformation relationship between the moving coordinate system and the fixed coordinate system of the image, and the distance between the camera and the object being photographed and the physical parameters of the camera itself, mapping m and n to In and ;like and If both are integers, then Middle OK, Replace the pixel values ​​of the column The pixel value of the mth row and nth column in or is not an integer, then The pixel value to be replaced is calculated by bilinear interpolation, and the pixel value to be replaced is replaced The pixel value of the mth row and nth column in ; repeat the replacement operation until After all pixel values ​​in are replaced, we get ;

[0012] Step 7: Calculate and update the image frame matrix in real time , construct an image video with 25 frames per second; when calculating any image frame matrix Previously, the camera's motion parameters were used to determine whether the camera was in a stationary operating state. If it was determined that the camera had entered a moving operating state, the display mode would be switched to the moving operating state. In the moving operating state, the infrared light source was normally closed.

[0013] Step 8: When the camera is in the mobile working state, it collects a 1024×1024 pixel image every 40ms, and crops the image of the 512×512 pixel area in the center of the image as the final image to be displayed. The image video with 25 frames per second is constructed according to the time sequence from the final image to be displayed.

[0014] Preferably, the posture sensor is i The three rotation angular velocity data of the camera around the XYZ axis collected and fed back are , and , then i The angular change relationship between the moving coordinate system of +1 image and the coordinate axis of the fixed coordinate system is:

[0015] ;

[0016] In the formula, For the posture sensor s The angular velocity of the camera collected this time around the X-axis of the current moving coordinate system, in rad / s; For the posture sensor s The angular velocity of the camera collected this time rotating around the Y axis of the current moving coordinate system, in rad / s; For the posture sensor s The angular velocity of the camera collected this time rotating around the Z axis of the current moving coordinate system, in rad / s; is the data collection time interval of the posture sensor, i.e. 0.04s; For the i +1 The angle change of the moving coordinate system X axis relative to the fixed coordinate system X axis of the image, in rad; For the i +1 The angular change of the moving coordinate system Y axis relative to the fixed coordinate system Y axis of the image, in rad; For the i +1 The angular change of the moving coordinate system Z axis relative to the fixed coordinate system Z axis of the image, in rad.

[0017] Preferably, the posture sensor i +1 The data collected is used to calculate the rotation transformation matrix relative to the fixed coordinate system The expression is

[0018] .

[0019] Preferably, the posture sensor is i The three moving acceleration data of the camera around the XYZ axis collected and fed back are: , and , then i The displacement relationship between the moving coordinate system of +1 image and the coordinate axis of the fixed coordinate system is:

[0020] ;

[0021] In the formula, For the posture sensor s The acceleration of the camera along the X-axis of the current moving coordinate system, in mm / s 2 ; For the posture sensor s The acceleration of the camera's displacement around the Y axis of the current moving coordinate system, in mm / s 2 ; For the posture sensor s The acceleration of the camera's displacement around the Z axis of the current moving coordinate system, in mm / s 2 ; is the data collection time interval of the posture sensor, i.e. 0.04s; For the i +1 image's displacement change of the moving coordinate system X-axis relative to the fixed coordinate system X-axis, in mm; For the i +1 image's displacement change of the moving coordinate system Y axis relative to the fixed coordinate system Y axis, in mm; For the i +1 image's displacement change of the moving coordinate system Z axis relative to the fixed coordinate system Z axis, in mm; For the posture sensor s The speed of the camera displacement along the X-axis of the current moving coordinate system, in mm / s; For the posture sensor s The speed of the camera displacement around the Y axis of the current moving coordinate system, in mm / s; For the posture sensor s The speed of the camera collected this time around the Z axis of the current moving coordinate system, in mm / s;s =1, there is

[0022] ;

[0023] when s When it is greater than 1, there is

[0024] ;

[0025] By the posture sensor i +1 The collected data is used to calculate the movement transformation matrix relative to the fixed coordinate system The expression is

[0026] .

[0027] Preferably, the distance between the camera and the object being photographed is The calculation formula is

[0028] ;

[0029] In the formula, is the focal length of the camera, in mm; A The actual cross-sectional area of ​​the light beam emitted by the infrared light source, in mm 2 ; p is the pixel size of the camera, in mm; For the i +1 Total number of infrared pixels in the infrared image.

[0030] Preferably, construct a mapping matrix ,have

[0031] ;

[0032] In the formula, is the focal length of the camera in the X direction, in mm; is the focal length of the camera in the Y direction, in mm.

[0033] Preferably, In and The calculation method is

[0034] ;

[0035] In the formula, and is the pixel position of the camera's principal point, and its value is 512.

[0036] Preferably, The formula for bilinear interpolation calculation is:

[0037] ;

[0038] In the formula, and is the scale factor, , , ,and is the neighboring pixel operator,

[0039] .

[0040] Preferably, when the posture sensor i +1 collection of the three rotational angular velocity data of the camera around the XYZ axis , and When the sum of the squares of is greater than the angular velocity threshold, the camera is determined to enter the mobile operation state; the angular velocity threshold range is 0.0004 to 0.25.

[0041] The present invention also proposes an anti-shake system based on the visual bending pliers image anti-shake method, comprising an image acquisition module, an image processing module, a data transmission module, and a display module; the pliers of the visual bending pliers are composed of an upper pliers body and a lower pliers body; the image acquisition module is arranged on the lower pliers body; the image acquisition module also includes a focusing tube; the camera is fixedly installed at the center of the lower pliers body; an infrared light source is fixedly installed on each side of the camera on the lower pliers body; a visible light source is also fixedly installed on each side of the camera on the lower pliers body; the two visible light sources are adjacent to the upper pliers body; a focusing tube is fixedly installed on each infrared light source; the focusing tube is used to make the infrared light source emit a cylindrical infrared light; the posture sensor is fixedly installed in the lower pliers body; the image processing module and the data transmission module are both arranged inside the visual bending pliers; the display module is independently arranged outside the visual bending pliers; the image acquisition module and the image processing module exchange data through a wired connection; the image processing module and the data transmission module exchange data through a wired connection; the data transmission module and the display module exchange data through two connection forms, wired or wireless.

[0042] Compared with the prior art, the present invention has the following beneficial effects: (1) the camera working state is divided into a moving working state and a stationary working state, and image anti-shake calculation is performed only in the stationary working state, which accurately matches the scene requirements, saves computing power, and makes image display more scientific and efficient; (2) the complex feature point matching algorithm is abandoned, and an infrared light source is cleverly introduced. The distance from the camera to the photographed part is calculated in real time through the infrared light source, thereby accurately realizing anti-shake calculation; (3) the infrared light source is always on only in the stationary working state of the camera, which realizes the ranging function while also improving the tissue development effect in the stationary working state, facilitating decision-making and operation; (4) the performance of the camera is fully utilized in a limited space. The camera captures a total of 50 frames of images per second, of which 25 frames are used to display the final image, and the other 25 frames are infrared imaging used to realize the ranging function. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The figure is a flow chart of the image anti-shake method of the present invention.

[0044] Figure 2 It is a structural schematic diagram of the image acquisition module of the present invention.

[0045] Figure 3 It is a module diagram of the image stabilization system of the present invention.

[0046] Figure 4 In the static working state of the embodiment of the present invention The corresponding test image.

[0047] Figure 5 for Figure 4 middle The corresponding comparison chart of the test image before and after anti-shake processing.

[0048] In the figure: 1-image acquisition module; 2-image processing module; 3-data transmission module; 4-display module; 101-camera; 102-visible light source; 103-infrared light source; 104-focusing tube; 105-lower clamp body; 106-upper clamp body. DETAILED DESCRIPTION

[0049] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings and through specific implementation methods, wherein the accompanying drawings are only used for exemplary descriptions and represent only schematic diagrams rather than actual drawings, and should not be understood as limiting the present invention; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted.

[0050] Figures 1 to 5 It is a preferred embodiment of the present invention.

[0051] like Figures 1 to 3 As shown, a visual bending forceps image anti-shake method comprises the following steps:

[0052] Step 1: regularly collect images with a pixel size of 1024×1024 through the image acquisition module 1, where the image acquisition module 1 includes a camera 101, a pair of infrared light sources 103, a pair of visible light sources 102 and a posture sensor;

[0053] Step 2: The visible light source 102 is always on during use, and the motion parameters of the camera 101 are regularly collected and fed back through the posture sensor. The collection and feedback are collected every 40ms, including three rotation angular velocity data around the XYZ axis and three movement acceleration data along the XYZ axis. The motion parameters of the camera 101 are used to determine whether the camera 101 is in a stationary working state;

[0054] Step 3: When the camera 101 is in a stationary working state, it starts collecting images from 0ms and collects them once every 40ms. The pixels of the collected images are 1024×1024. i +1 image data storage matrix is ,in, i =0, 1, ..., N;

[0055] Step 4: The posture sensor starts collecting motion parameter data from 0ms and collects it every 40ms. i When taking the first image, the optical center coordinate system of the camera 101 is a moving coordinate system. When taking the first image, the moving coordinate system of the camera 101 is also a fixed coordinate system. i +1 The data collected is used to calculate the rotation transformation matrix relative to the fixed coordinate system , the translation transformation matrix relative to the fixed coordinate system ; Assume that the posture sensor i The three rotation angular velocity data of the camera 101 around the XYZ axis collected and fed back are respectively , and , then i The angular change relationship between the moving coordinate system of +1 image and the coordinate axis of the fixed coordinate system is:

[0056] ;

[0057] In the formula, For the posture sensor s The angular velocity of the camera 101 collected this time rotating around the X-axis of the current moving coordinate system, in rad / s; For the posture sensor sThe angular velocity of the camera 101 collected this time rotating around the Y axis of the current moving coordinate system, in rad / s; For the posture sensor s The angular velocity of the camera 101 collected this time rotating around the Z axis of the current moving coordinate system, in rad / s; is the data collection time interval of the posture sensor, i.e. 0.04s; For the i +1 The angle change of the moving coordinate system X axis relative to the fixed coordinate system X axis of the image, in rad; For the i +1 The angular change of the moving coordinate system Y axis relative to the fixed coordinate system Y axis of the image, in rad; For the i +1 image's moving coordinate system Z axis relative to the fixed coordinate system Z axis angular change, unit is rad; from the position sensor i +1 The data collected is used to calculate the rotation transformation matrix relative to the fixed coordinate system The expression is

[0058] ;

[0059] Assume that the posture sensor i The three moving acceleration data of the camera 101 around the XYZ axis collected and fed back are: , and , then i The displacement relationship between the moving coordinate system of +1 image and the coordinate axis of the fixed coordinate system is:

[0060] ;

[0061] In the formula, For the posture sensor s The acceleration of the camera 101 along the X-axis of the current moving coordinate system, in mm / s 2 ; For the posture sensor s The acceleration of the camera 101 around the Y axis of the current moving coordinate system collected this time, in mm / s 2 ; For the posture sensor s The acceleration of the camera 101 around the Z axis of the current moving coordinate system collected this time, in mm / s 2 ; is the data collection time interval of the posture sensor, i.e. 0.04s; For the i+1 image's displacement change of the moving coordinate system X-axis relative to the fixed coordinate system X-axis, in mm; For the i +1 image's displacement change of the moving coordinate system Y axis relative to the fixed coordinate system Y axis, in mm; For the i +1 image's displacement change of the moving coordinate system Z axis relative to the fixed coordinate system Z axis, in mm; For the posture sensor s The speed of the camera 101 along the X-axis of the current moving coordinate system collected this time, in mm / s; For the posture sensor s The speed of the camera 101 collected this time around the Y axis of the current moving coordinate system, in mm / s; For the posture sensor s The speed of the camera 101 collected for the second time around the Z axis of the current moving coordinate system is in mm / s; s =1, there is

[0062] ;

[0063] when s When it is greater than 1, there is

[0064] ;

[0065] By the posture sensor i +1 The collected data is used to calculate the movement transformation matrix relative to the fixed coordinate system The expression is

[0066] ;

[0067] Step 5: When the camera 101 is in a stationary working state, the two infrared light sources 103 are always on, and the camera 101 starts to collect infrared images from 20ms and collects them every 40ms. The distance between the camera 101 and the photographed object is calculated by the number of infrared pixels in the infrared image. , the calculation formula is

[0068] ;

[0069] In the formula, is the focal length of the camera 101, in mm; A is the actual cross-sectional area of ​​the light beam emitted by the infrared light source 103, in mm 2 ; p is the pixel size of the camera 101, in mm; For the i+1 Total number of infrared pixels in an infrared image;

[0070] Step 6: Construct the image frame matrix ; Stored The data of the central 512×512 pixel area, The row coordinate of the pixel value is m, and the column coordinate is n; starting from the second picture taken, through the i +1 The coordinate transformation relationship between the moving coordinate system and the fixed coordinate system of the image, the distance between the camera 101 and the object being photographed and the physical parameters of the camera 101 itself, mapping m and n to In and ;like and If both are integers, then Middle OK, Replace the pixel values ​​of the column The pixel value of the mth row and nth column in or is not an integer, then The pixel value to be replaced is calculated by bilinear interpolation, and the pixel value to be replaced is replaced The pixel value of the mth row and nth column in ; repeat the replacement operation until After all pixel values ​​in are replaced, we get ; Specifically, construct the mapping matrix ,have

[0071] ;

[0072] In the formula, is the focal length of the camera 101 in the X direction, in mm; is the focal length of the camera 101 in the Y direction, in mm; In and The calculation method is

[0073] ;

[0074] In the formula, and is the pixel position of the principal point of the camera 101, and its value is 512; The formula for bilinear interpolation calculation is:

[0075] ;

[0076] In the formula, and is the scale factor, , , ,and is the neighboring pixel operator,

[0077] ;

[0078] Step 7: Calculate and update the image frame matrix in real time , construct an image video with 25 frames per second; when calculating any image frame matrix Previously, the motion parameters of the camera 101 were used to determine whether the camera 101 was in a stationary operating state. If it was determined that the camera 101 had entered a moving operating state, the display mode would be switched to the moving operating state. In the moving operating state, the infrared light source 103 was normally closed. Specifically, when the posture sensor i +1 collection of feedback of the three rotation angular velocity data of the camera 101 around the XYZ axis , and When the sum of the squares of is greater than the angular velocity threshold, it is determined that the camera 101 enters the mobile operation state; the angular velocity threshold value ranges from 0.0004 to 0.25; in this embodiment, the angular velocity threshold value is 0.16;

[0079] Step 8: When the camera 101 is in a mobile working state, it collects a 1024×1024 pixel image every 40ms, and crops the image of the 512×512 pixel area in the center of the image as the final image to be displayed, and constructs a 25 frame per second image video according to the time sequence from the final image to be displayed.

[0080] like Figure 3As shown, the anti-shake system based on the visual bending forceps image anti-shake method includes an image acquisition module 1, an image processing module 2, a data transmission module 3, and a display module 4; the jaws of the visual bending forceps are composed of an upper jaw body 106 and a lower jaw body 105; the image acquisition module 1 is arranged on the lower jaw body 105; the image acquisition module 1 also includes a focusing tube 104; a camera 101 is fixedly installed at the center of the lower jaw body 105; an infrared light source 103 is fixedly installed on the lower jaw body 105 on both sides of the camera 101; a visible light source 102 is also fixedly installed on the lower jaw body 105 on both sides of the camera 101; the two visible light sources 102 are connected to the upper jaw body 105 6 are adjacent; a focusing tube 104 is fixedly installed on each infrared light source 103; the focusing tube 104 is used to make the infrared light source 103 emit a cylindrical infrared light; the posture sensor is fixedly installed in the lower clamp body 105; the image processing module 2 and the data transmission module 3 are both arranged inside the visual bending forceps; the display module 4 is independently arranged outside the visual bending forceps; the image acquisition module 1 and the image processing module 2 exchange data through a wired connection; the image processing module 2 and the data transmission module 3 exchange data through a wired connection; the data transmission module 3 and the display module 4 exchange data through two connection forms: wired or wireless.

[0081] In this embodiment, the camera 101 model is Awaiba NanEyeC, which has a very small package size (about 1mm×1mm), suitable for minimally invasive medical devices, a resolution of 1024×1024 (1 million pixels), a pixel size of 3μm×3μm, an optical format of 1 / 18 inch, and an interface using digital video output (single-ended LVDS). 2mm, and The values ​​are all 666.67; the posture sensor model is Bosch BMI270; the visible light source 102 model is Nichia 157 series, with a size of 1.0mm×1.0mm×0.2mm; the infrared light source 103 model is OSRAM SFH 4053, with a wavelength of 850nm (near infrared), a size of 1.0mm×0.8mm×0.6mm, and a luminous intensity of 10mW / sr; combined with the above hardware information, it is known that

[0082] ;

[0083] After calculation, we get

[0084] ;

[0085] If Figure 4 As shown, the static working state of this embodiment is given. The corresponding pixel is a 1024×1024 test image; if the image of the 512×512 pixel area in the center of the screen is directly displayed, Figure 5 The picture corresponding to "before anti-shake processing" in the figure is processed by the visual bending forceps image anti-shake method of the present invention, and the image is obtained. Figure 5 The picture corresponding to “After anti-shake processing”.

Claims

1. A visual bending forceps image anti-shake method, characterized in that: The following steps are involved: Step 1: regularly collecting images with a pixel size of 1024×1024 through an image acquisition module (1), wherein the image acquisition module (1) comprises a camera (101), a pair of infrared light sources (103), a pair of visible light sources (102) and a posture sensor; Step 2: The visible light source (102) is always on during use, and the motion parameters of the camera (101) are regularly collected and fed back through the posture sensor. The collection and feedback are performed every 40 ms, including three rotation angular velocity data around the XYZ axis and three movement acceleration data along the XYZ axis. Whether the camera (101) is in a stationary operating state is determined through the motion parameters of the camera (101); Step 3: When the camera (101) is in a stationary working state, it starts collecting images from 0ms and collects images every 40ms. The pixels of the collected images are 1024×1024. i +1 image data storage matrix is ,in, i =0, 1, ..., N; Step 4: The posture sensor starts collecting motion parameter data from 0ms and collects it every 40ms. i When the first image is taken, the optical center coordinate system of the camera (101) is a moving coordinate system. When the first image is taken, the moving coordinate system of the camera (101) is also a fixed coordinate system. i +1 The data collected is used to calculate the rotation transformation matrix relative to the fixed coordinate system , the translation transformation matrix relative to the fixed coordinate system ; Step 5: When the camera (101) is in a stationary working state, the two infrared light sources (103) are always on, and the camera (101) starts to collect infrared images from 20ms and collects them once every 40ms. The distance between the camera (101) and the object being photographed is calculated by the number of infrared pixels in the infrared image. ; Step 6: Construct the image frame matrix ; Stored The data of the central 512×512 pixel area, The row coordinate of the pixel value is m, and the column coordinate is n; starting from the second picture taken, through the i +1 The coordinate transformation relationship between the moving coordinate system and the fixed coordinate system of the image, and the distance between the camera (101) and the object being photographed and the physical parameters of the camera (101) itself, mapping m and n to In and ;like and If both are integers, then Middle OK, Replace the pixel values ​​of the column The pixel value of the mth row and nth column in or is not an integer, then The pixel value to be replaced is calculated by bilinear interpolation, and the pixel value to be replaced is replaced The pixel value of the mth row and nth column in ; repeat the replacement operation until After all pixel values ​​in are replaced, we get ; Step 7: Calculate and update the image frame matrix in real time , construct an image video with 25 frames per second; when calculating any image frame matrix Previously, it was determined whether the camera (101) was in a stationary operating state by using the motion parameters of the camera (101). If it was determined that the camera (101) had entered a moving operating state, the display mode was switched to the moving operating state. In the moving operating state, the infrared light source (103) was normally closed. Step 8: When the camera (101) is in a mobile operation state, an image with 1024×1024 pixels is collected every 40 ms, and an image with a 512×512 pixel area in the center of the image is cropped out as the final image to be displayed, and a video with 25 frames per second is constructed from the final image to be displayed in a time sequence.

2. A visual curved forceps image anti-shake method as claimed in claim 1, characterized in that: Assume that the posture sensor i The three rotation angular velocity data of the camera (101) around the XYZ axis of the +1 collection feedback are respectively , and , then i The angular change relationship between the moving coordinate system of +1 image and the coordinate axis of the fixed coordinate system is: ； In the formula, For the posture sensor s The angular velocity of the camera (101) collected this time rotating around the X-axis of the current moving coordinate system, in rad / s; For the posture sensor s The angular velocity of the camera (101) collected this time rotating around the Y axis of the current moving coordinate system, in rad / s; For the posture sensor s The angular velocity of the camera (101) collected this time rotating around the Z axis of the current moving coordinate system, in rad / s; is the data collection time interval of the posture sensor, i.e. 0.04s; For the i +1 The angle change of the moving coordinate system X axis relative to the fixed coordinate system X axis of the image, in rad; For the i +1 The angular change of the moving coordinate system Y axis relative to the fixed coordinate system Y axis of the image, in rad; For the i +1 The angular change of the moving coordinate system Z axis relative to the fixed coordinate system Z axis of the image, in rad.

3. A visual curved forceps image anti-shake method as claimed in claim 2, characterized in that: By the posture sensor i +1 The data collected is used to calculate the rotation transformation matrix relative to the fixed coordinate system The expression is 。 4. A visual curved forceps image anti-shake method as claimed in claim 3, characterized in that: Assume that the posture sensor i The three moving acceleration data of the camera (101) collected +1 feedback around the XYZ axis are , and , then i The displacement relationship between the moving coordinate system of +1 image and the coordinate axis of the fixed coordinate system is: ； In the formula, For the posture sensor s The acceleration of the camera (101) collected this time along the X-axis of the current moving coordinate system, in mm / s 2 ; For the posture sensor s The acceleration of the camera (101) collected this time around the Y axis of the current moving coordinate system, in mm / s 2 ; For the posture sensor s The acceleration of the camera (101) collected this time around the Z axis of the current moving coordinate system, in mm / s 2 ; is the data collection time interval of the posture sensor, i.e. 0.04s; For the i +1 image's displacement change of the moving coordinate system X-axis relative to the fixed coordinate system X-axis, in mm; For the i +1 image's displacement change of the moving coordinate system Y axis relative to the fixed coordinate system Y axis, in mm; For the i +1 image's displacement change of the moving coordinate system Z axis relative to the fixed coordinate system Z axis, in mm; For the posture sensor s The speed of the camera (101) collected this time along the X-axis of the current moving coordinate system, in units of mm / s; For the posture sensor s The speed of the camera (101) collected this time around the Y axis of the current moving coordinate system, in units of mm / s; For the posture sensor s The speed of the camera (101) collected this time around the Z axis of the current moving coordinate system, in units of mm / s; when s =1, there is ； when s When it is greater than 1, there is ； By the posture sensor i +1 The collected data is used to calculate the movement transformation matrix relative to the fixed coordinate system The expression is 。 5. A visual curved forceps image anti-shake method as claimed in claim 4, characterized in that: The distance between the camera (101) and the object being photographed The calculation formula is ； In the formula, is the focal length of the camera (101), in mm; A is the actual cross-sectional area of ​​the light beam emitted by the infrared light source (103), in mm 2 ; p is the pixel size of the camera (101), in mm; For the i +1 Total number of infrared pixels in the infrared image.

6. A visual curved forceps image anti-shake method as claimed in claim 5, characterized in that: Constructing the mapping matrix ,have ； In the formula, is the focal length of the camera (101) in the X direction, in mm; is the focal length of the camera (101) in the Y direction, in mm.

7. A visual curved forceps image anti-shake method as claimed in claim 6, characterized in that: In and The calculation method is ； In the formula, and is the pixel position of the principal point of the camera (101), and its value is 512.

8. A visual curved forceps image anti-shake method as claimed in claim 7, characterized in that: exist The formula for bilinear interpolation calculation is: ； In the formula, and is the scale factor, , , and is the neighboring pixel operator, 。 9. A visual curved forceps image anti-shake method as claimed in claim 8, characterized in that: When the posture sensor i +1 collection of feedback of the camera (101) three rotation angular velocity data around the XYZ axis , and When the sum of the squares of is greater than an angular velocity threshold, it is determined that the camera (101) enters a moving operation state; the angular velocity threshold value ranges from 0.0004 to 0.

25.

10. An anti-shake system based on the visual curved forceps image anti-shake method according to any one of claims 1 to 9, comprising an image acquisition module (1), an image processing module (2), a data transmission module (3), and a display module (4), characterized in that: The pliers of the visualization bending forceps are composed of an upper pliers body (106) and a lower pliers body (105); the image acquisition module (1) is arranged on the lower pliers body (105); the image acquisition module (1) also includes a focusing tube (104); the camera (101) is fixedly mounted at the center of the lower pliers body (105); an infrared light source (103) is fixedly mounted on the lower pliers body (105) on both sides of the camera (101); a visible light source (102) is also fixedly mounted on the lower pliers body (105) on both sides of the camera (101); the two visible light sources (102) are adjacent to the upper pliers body (106); each infrared light source (103) is fixedly mounted with a focusing tube (104); The light tube (104) is used to make the infrared light source (103) emit a cylindrical infrared light; the posture sensor is fixedly installed in the lower clamp body (105); the image processing module (2) and the data transmission module (3) are both arranged inside the visual bending clamp; the display module (4) is independently arranged outside the visual bending clamp; the image acquisition module (1) and the image processing module (2) exchange data through a wired connection; the image processing module (2) and the data transmission module (3) exchange data through a wired connection; the data transmission module (3) and the display module (4) exchange data through two connection forms, wired or wireless.

Images