An apparatus tremor detection method, system and readable storage medium

Abstract

The application relates to a tremor detection method of an instrument, receiving images taken during instrument operation, the images being sorted according to the instrument operation; using a neural network to identify the instrument in the images, including a tube body and an execution element installed on the tube body; obtaining an inclination angle theta0 of the instrument in any one of the first M images; estimating an inclination angle theta n n in the nth image to obtain the end position of the execution element in the nth image, obtaining the axis of the execution element in each image; obtaining the average axis of the instrument in all images according to the axis of the execution element in each image; and calculating the distance from the end position of the execution element in any image to the average axis to obtain the tremor distance of the instrument in the corresponding image. The position of the invisible needle tip is inferred, and the tremor of the needle tip can be accurately detected.

Description

Technical Field

[0001] This invention relates to the field of image detection and recognition, and more specifically, to a method, system, and readable storage medium for detecting instrument tremors. Background Technology

[0002] For retinal surgical robots, instrument tremor is a critical safety indicator, and obtaining tremor data helps compare the performance of the robot with that of a human hand. Detecting instrument tremor requires knowing the position of the instrument's actuator tip. However, because the actuator tip is located deep beneath the retina, it is invisible. Existing needle tip detection methods primarily focus on detecting the target, mainly using convolutional neural networks and spatiotemporal frames to detect the needle tip. Related technologies include: using convolutional neural networks to identify surgical needles in situations with blur or occlusion; or using spatiotemporal frames to detect needle tip displacement and thus obtain needle positioning. These methods all require detecting the visible position of the actuator tip to achieve detection, therefore they cannot be used when the position information of the actuator tip cannot be directly obtained, and thus cannot detect instrument tremor. Summary of the Invention

[0003] To overcome the problem in the prior art that the position of the actuator end cannot be detected when the actuator end is invisible, the present invention provides a device vibration detection method that can estimate the position of the invisible actuator end and then detect the vibration of the actuator end.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for detecting instrument tremor, comprising the following steps:

[0005] Step 1: Receive several images captured during the working process of the instrument, and arrange the images according to the working process of the instrument;

[0006] Step 2: Use a neural network to identify the instruments in the image. The extraction method is to input the image into a keypoint network to identify the key points of the instruments from the image. The instruments include a tube and an actuator mounted on the tube. The key points of the identified instruments can be located.

[0007] Step 3: Identify the position of the start end and the position of the visible point of the end end of the actuator in each image, and obtain the position A of the start end of the actuator in the nth image. n (x an ,y an ) and the visible point position B at the end of the actuator n (x bn ,y bn), and obtain the identifiable slope k of the actuator in the nth image. n and recognizable intercept b n ; Calculate the pixel length l of the actuator based on the position information of the visible points at the beginning and end of the actuator in any of the first M images. p The process involves obtaining the tilt angle θ0 of the device in any one of the first M images. During operation, the movement of the actuator tip gradually deepens. Once a certain depth is reached, the true position of the actuator tip cannot be accurately identified; only the visible point position can be determined. However, before reaching a certain depth, the position information of the actuator tip can be identified—that is, the true position of the actuator tip in the first M images. Therefore, any one of the first M images where the actuator tip can still be identified is selected to obtain the position information of the actuator tip and its end. A smaller value for M is better. The position information is obtained by labeling the points to obtain their pixel coordinates. After obtaining the two coordinate points, the distance between them can be calculated, which is the pixel length l of the actuator. p The tilt angle θ0 of the instrument is essentially the angle between the instrument and the camera's shooting plane when the instrument is working. This angle changes as the instrument moves. It can be obtained through external hardware that controls the instrument's movement, or through image recognition. The first end of the actuator refers to the end where the actuator is installed in the connecting tube, and the last end of the actuator refers to the end of the actuator away from the tube.

[0008] Step 4: For each image, perform the following processing: fit a straight line to the outer contour points on both sides of the pipe axis, which are respectively the first pipe contour line and the second pipe contour line; obtain the distance dis from the intersection of the first pipe contour line and the second pipe contour line to the beginning of the pipe in each image except the first M images. cp The distance between the intersection of the first and second pipe contour lines and the beginning of the pipe in any one of the first M images is compared. cp0 The tilt angle θ in the nth image is estimated by combining the cotangent and arctangent functions, specifically as follows:

[0009]

[0010] During the instrument's movement, vibrations or other factors can cause changes in its tilt angle. These changes affect the estimation of the actuator's end effector position. Therefore, by first calculating the instrument's tilt angle in each image, and then calculating the actuator's end effector position P in each image, a more precise calculation is made. end The obtained position information of the end position of the actuator is more accurate.

[0011] Step 5: Based on the information obtained in Step 4, obtain the position p of the end of the actuator in the nth image. endn Specifically:

[0012]

[0013]

[0014] In the formula, l pm The pixel length of the execution element in the nth image; direct n Select the direction for the actuator, which is 1 or -1;

[0015] Step 6: Based on the position p of the end of the actuator in the nth image endn and the position A of the start end of the actuator n (x an ,y an ), obtain the axis of the actuator in the nth image, with slope and intercept k and b respectively; based on the axis of the actuator in each image, obtain the average axis of the instrument in all images, and its slope-intercept equation is:

[0016]

[0017] In the formula, It is the average slope k of the axis of the actuator in each image. It is the average value of the intercept b of the axis of the actuator in each image;

[0018] Step 7: Calculate the end position P of the actuator in the nth image. endn The distance to the average axis is used to obtain the jitter distance of the instrument in the corresponding image. If the images are continuously input and acquired, in step six above, the jitter distance of the last image can be calculated using the average axis of the previously acquired images as the jitter distance of the actuator of the instrument at the current moment, thereby achieving real-time detection of instrument jitter.

[0019] Preferably, in step three, the process for obtaining the tilt angle of the instrument in any one of the first M images is as follows:

[0020] S3.1: Obtain the actual length l of the execution element mapped to the plane in any one of the first M images. rm0 ;

[0021] S3.2: Obtain the actual length l of the actuator. r ;

[0022] S3.3: Calculate the tilt angle θ0, specifically:

[0023]

[0024] Based on the projection relationship, the actual length of the actuator corresponds to the hypotenuse of a triangle, and the length of the actuator mapped onto the plane corresponds to the adjacent side of the triangle. The tilt angle is obtained through this trigonometric relationship. This method is achieved through image recognition, which avoids the need to add hardware to the instrument compared to obtaining it through external hardware. Since surgical instruments are delicate and precise instruments, adding external hardware will affect the movement accuracy of the instrument, requiring additional calculation compensation, and the adverse effects on the instrument are significant.

[0025] Preferably, the specific process of step S3.1 is as follows:

[0026] Obtain any two non-overlapping contour points of the tube body and the midpoint of the contour of the actuator in any one of the first M images;

[0027] Calculate the pixel distance between the tube diameters at two contour points; obtain the actual size of the instrument diameter at the two contour points;

[0028] The actual pixel ratio at two contour points of the tube is calculated based on the pixel distance of the tube diameter and the actual size of the instrument diameter; the actual pixel ratio refers to the ratio of the actual distance to the pixel distance.

[0029] Based on the actual pixel ratio at the two contour points and the pixel distances from the midpoint of the actuator's contour to the two contour points, the actual pixel ratio r at the midpoint of the actuator's contour is obtained. m ;

[0030] In practice, the actuating element has a length l mapped to the plane. rm0 The actual pixel ratio r at the midpoint of the actuator's outline m The pixel length l of the actuator p The product of.

[0031] Preferably, the positions of the first and last ends of the actuator, the midpoint of the actuator's contour, and two non-overlapping contour points on the tube are all identified through a keypoint network. The keypoint network is trained using data with labeled keypoints. The two non-overlapping contour points are preferably contour points located in the middle section of the tube, referred to as mid-section points.

[0032] In another preferred embodiment, the positions of the first and last ends of the actuator, the midpoint of the actuator's outline, and the two non-overlapping outline points on the tube can all be obtained through key point annotation. This method first extracts the instrument through semantic segmentation to obtain the outer outline points of the instrument, and then annotates the required key points based on the points passed by the outer wheel or obtains them through calculation. Specifically, the two non-overlapping outline points are the midpoints located in the middle section of the tube, and the specific method for obtaining them is as follows:

[0033] The skeleton of the axis is extracted from the tube body using the skeleton extraction method, and then the skeleton is fitted into a straight line as the axis of the tube body.

[0034] Select any two auxiliary points on the tube frame. Based on the positions of the two auxiliary points and the vertical slope perpendicular to the tube axis, obtain the first intersection line and the second intersection line that intersect the first tube outline and the second tube outline, respectively. The farther apart the auxiliary points are, the more accurate the result will be. If the two auxiliary points are too close, the error will be too large. For example, a distance of 5 pixels will result in an error of 1 pixel, while a distance of 50 pixels will also result in an error of only 1 pixel.

[0035] The first intersecting line intersects with the first pipe body outline and the second pipe body outline respectively, and obtains the first intersection point and the second intersection point;

[0036] The second intersecting line intersects with the first pipe body outline and the second pipe body outline respectively, and obtains the third intersection point and the fourth intersection point;

[0037] The midpoint between the first and second intersection points is taken as one of the midpoints of the tube body, and the third and fourth intersection points are taken as the other midpoint of the tube body.

[0038] Another preferred method for obtaining the position of the actuator's head end is as follows:

[0039] Calculate the projection points of the contour points on the axis of the tube body on both sides of the connection with the needle tip to obtain a number of contour point projection points;

[0040] Calculate the mean of all the projection points of the contour points to obtain the root point of the needle tip, which is the position of the head end of the actuator.

[0041] Preferably, any one of the first M images is the first image.

[0042] Preferably, in step six, the process for obtaining the average axis is as follows:

[0043] Based on the position P of the actuator end in the nth image endn and the position A of the start end of the actuator n (x an ,yan ), obtain the axis of the actuator in the nth image, with slope and intercept k and b respectively; based on the axis of the actuator in each image, obtain the average axis of the instrument in all images, and its slope-intercept equation is:

[0044]

[0045] In the formula, It is the average slope k of the axis of the actuator in each image. It is the average value of the intercept b of the axis of the actuator in each image;

[0046] Preferably, in step seven, the tremor distance corresponding to all images is calculated to obtain the tremor amplitude.

[0047] Preferably, in step one, the device's working process video is received and each frame in the video is broken down to obtain several images.

[0048] Preferably, in steps three and four, the position A of the identified actuator head is... n Position B of the visible point at the end of the actuator n Slope k can be identified n and tilt angle θ n Perform smoothing. The specific smoothing method is as follows:

[0049]

[0050]

[0051]

[0052] In the formula, A iori k iori and A respectively n k n and θ n Data before smoothing; w i Let w be the weight, and each weight satisfies w. n-1 =αw n , where α∈(0,1);

[0053] When n < 5, that is, when there are fewer than 5 images, no smoothing is performed.

[0054] Preferably, based on the position P of the actuator end in the nth image end Position B of the visible point at the end of the actuator n and the tilt angle θ in the nth image nThe injection depth of the actuator in the nth image is obtained. n Specifically:

[0055]

[0056] In the formula, This represents the ratio of the actual distance between the end position of the actuator and the pixel. It is the position P of the end of the actuator in the nth image. endn X-axis coordinates Is it the position P of the end of the actuator in the nth image? endn The Y-axis coordinate.

[0057] A computer-readable storage medium for storing a computer program that, when executed by a processor, implements the above-described method for detecting instrument tremors.

[0058] Compared with existing technologies, the beneficial effects of this invention are: by combining images of the instrument's usage process with the instrument's actual parameters, the position of the invisible needle tip under the retina can be inferred, thereby accurately detecting needle tip tremors. This allows the stability and safety of the instrument during retinal surgery to be verified, contributing to improved stability and safety of the surgical robot. Furthermore, compared to other existing needle tip positioning methods, this method allows for the selection of microneedles with smaller diameters, sometimes even one-tenth or less than the diameter of other needles. Attached Figure Description

[0059] Figure 1 This is a flowchart of a device tremor detection method according to the present invention;

[0060] Figure 2 This is a schematic diagram of the tilt angle θ0 of the present invention. Detailed Implementation

[0061] The accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. To better illustrate this embodiment, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0062] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "long," and "short" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0063] The technical solution of the present invention will be further described in detail below through specific embodiments and in conjunction with the accompanying drawings:

[0064] Example 1

[0065] like Figure 1 The following is an embodiment of a method for detecting instrument tremor, comprising the following steps:

[0066] Step 1: Receive several images captured during the working process of the instrument. The images are ordered according to the working process of the instrument. The images can be images or videos obtained by connecting to a microscope, or they can be images or videos directly input.

[0067] Step 2: Use a neural network to identify the instruments in the image. The extraction method is to input the image into a keypoint network to identify the key points of the instruments from the image. The instruments include a tube and an actuator mounted on the tube. The key points of the identified instruments can be located.

[0068] Step 3: Identify the position of the start end and the position of the visible point of the end end of the actuator in each image, and obtain the position A of the start end of the actuator in the nth image. n (x an ,y an ) and the visible point position B at the end of the actuator n (x bn ,y bn ), and obtain the identifiable slope k of the actuator in the nth image. n and recognizable intercept b n ; Calculate the pixel length l of the actuator based on the position information of the visible points at the beginning and end of the actuator in any of the first M images. pThe device's tilt angle θ0 is obtained from any one of the first M images. During operation, the movement of the actuator tip gradually deepens. After reaching a certain depth, the true position of the actuator tip cannot be accurately identified. However, before reaching a certain depth, the position information of the actuator tip can be identified, which is the true position of the actuator tip in the first M images. This is the visible point position of the actuator tip. Therefore, any one of the first M images where the actuator tip can still be identified is selected to obtain the position information of the actuator tip and the actuator tip. The smaller the value of M, the better. In this embodiment, the first image is selected. In this embodiment, the actuator tip needs to be obtained by projecting the point where the tube body and the actuator connect onto the axis of the tube body and taking the average point. The position information is obtained by labeling the pixel coordinates of the labeled points. After obtaining two coordinate points, the distance between the two coordinate points can be calculated, which is the pixel length l of the actuator. p In this embodiment, the tilt angle θ0 of the device in the first image is calculated by combining the ratio of the actual length to the pixel length and the actual length of the actuator to calculate the pixel length of the actuator, and then using the inverse cosine function to derive the tilt angle θ0 of the device in the first frame. The specific process is as follows:

[0069] The process for obtaining the tilt angle of the instrument in the first image is as follows:

[0070] S3.1: Obtain the length k of the actuator mapped to the plane in practice. rm0 The specific process is as follows:

[0071] Obtain any two non-overlapping contour points of the tube in the first image. These points are located in the middle section of the tube and are called mid-section points. The mid-section points are P1 and P2. ef and P gh and the midpoint P of the profile of the actuator. ab ;

[0072] Calculate the pixel distance between the tube diameters at two midpoints. and Obtain the actual diameter φ of the instrument at two midpoints of the tube. r ;

[0073] Calculate the actual pixel ratio at the two midpoints of the tube based on the pixel distance of the tube diameter and the actual size of the instrument diameter.

[0074] Based on the actual pixel ratio at the two midpoints and the pixel distances from the midpoint of the actuator's outline to the two midpoints, the actual pixel ratio r at the midpoint of the actuator's outline is obtained. m ;

[0075] In practice, the actuating element has a length l mapped to the plane. rm0 The actual pixel ratio r at the midpoint of the actuator's outline m The pixel length l of the actuator p The product of l rm0 =l p ·r m ;

[0076] S3.2: Obtain the actual length l of the actuator. r ;

[0077] S3.3: Calculate the tilt angle θ0, specifically:

[0078]

[0079] According to the projection relationship, such as Figure 2 As shown, the actual length of the actuator corresponds to the hypotenuse of a triangle, and the length of the actuator mapped onto the plane corresponds to the adjacent side of the triangle. The tilt angle is obtained through trigonometric relationships. This method is achieved through image recognition, which avoids the need to add hardware to the instrument compared to obtaining it through external hardware. Since surgical instruments are delicate and precise instruments, adding external hardware will affect the movement accuracy of the instrument, requiring additional calculation compensation, and the adverse effects on the instrument are significant.

[0080] Step 4: For each image, perform the following processing: fit a straight line to the outer contour points on both sides of the pipe axis, which are respectively the first pipe contour line and the second pipe contour line; obtain the distance dis from the intersection of the first pipe contour line and the second pipe contour line to the beginning of the pipe in each image except the first M images. cp The distance between the intersection of the first and second pipe contour lines and the beginning of the pipe in any one of the first M images is compared. cp0 The tilt angle θ in the nth image is estimated by combining the cotangent and arctangent functions, specifically as follows:

[0081]

[0082] During the instrument's movement, vibrations or other factors can cause changes in its tilt angle. These changes affect the estimation of the actuator's end effector position. Therefore, by first calculating the instrument's tilt angle in each image, and then calculating the actuator's end effector position P in each image, a more precise calculation is made. end The obtained position information of the end position of the actuator is more accurate.

[0083] Step 5: Based on the information obtained in Step 4, obtain the position P of the end of the actuator in the nth image. endn Specifically:

[0084]

[0085]

[0086] In the formula, l pm The pixel length of the execution element in the nth image; direct n Select the direction for the actuator, which is 1 or -1;

[0087] Step 6: Based on the position P of the end of the actuator in the nth image endn and the position A of the start end of the actuator n (x an ,y an ), obtain the axis of the actuator in the nth image, with slope and intercept k and b respectively; based on the axis of the actuator in each image, obtain the average axis of the instrument in all images, and its slope-intercept equation is:

[0088]

[0089] In the formula, It is the average slope k of the axis of the actuator in each image. It is the average value of the intercept b of the axis of the actuator in each image;

[0090] Step 7: Calculate the end position P of the actuator in the nth image. endn The distance to the average axis is used to obtain the vibration distance of the instrument in the corresponding image.

[0091] Based on the position P of the actuator end in the nth image end Position B of the visible point at the end of the actuator n and the tilt angle θ in the nth image n The injection depth of the actuator in the nth image is obtained. n Specifically:

[0092]

[0093] Calculate the position P of the actuator end in the nth image. endn The distance to the axis of the actuator in the nth image is used as the jitter data, specifically:

[0094]

[0095] If the images are continuously input and continuously acquired, in step six above, the average axis of the previously acquired images can be used to calculate the jitter distance of the last image as the jitter distance of the actuator of the instrument at the current moment, thereby achieving real-time detection of instrument jitter.

[0096] In this embodiment, the positions of the beginning and end of the actuator, as well as the midpoint and mid-segment points of the actuator's contour, are all obtained through a keypoint network. The end position of the actuator is clearly visible in the first M images, especially the first image, and therefore can be identified. The keypoint network identifies the corresponding points by using images labeled with the positions of the beginning and end of the actuator, as well as the midpoint and mid-segment points of the actuator's contour, as a training set. After training the keypoint network using this training set, the trained keypoint network is used to identify the images, thus obtaining the corresponding positional information.

[0097] One of the midpoints of the tube, P ef Another midpoint P of the tube body gh .

[0098]

[0099]

[0100] The above two needle pixel distances Specifically:

[0101]

[0102]

[0103] The actual pixel ratios at the two midpoints are as follows:

[0104]

[0105]

[0106] The midpoint P of the outline of the aforementioned actuator ab for The midpoint of the actuator's outline is used to calculate the pixel ratio, so the points used are taken from any one of the first M images.

[0107] The actual pixel ratio r at the midpoint of the contour of the actuator m for:

[0108]

[0109] The beneficial effects of this embodiment are as follows: By combining images of the instrument's usage process with the instrument's actual parameters, the position of the invisible needle tip under the retina can be inferred, thereby accurately detecting needle tip tremors. This allows the stability and safety of the instrument during retinal surgery to be verified, contributing to improved stability and safety of the surgical robot. Furthermore, compared to other existing needle tip positioning methods, this method allows for the selection of microneedles with smaller diameters, sometimes even one-tenth or less than the diameter of other needles.

[0110] Example 2

[0111] Example 2 of a device tremor detection method differs from Example 1 in that the position A of the identified actuator head is... n Slope k can be identified n and tilt angle θ n Perform smoothing. The specific smoothing method is as follows:

[0112]

[0113]

[0114]

[0115] In the formula, A iori k iori and A respectively m k m and θ n Data before smoothing; w i Let w be the weight, and each weight satisfies w. n-1 =αw n , where α∈(0,1);

[0116] When n < 5, that is, when there are fewer than 5 images, no smoothing is performed.

[0117] Example 3

[0118] An instrument tremor detection system, for implementing the method of Embodiment 1 or 2 above, includes an image processing module for receiving and processing instrument images and an estimation module for calculating needle tip position. The image processing module includes an image recognition module and a key point recognition module; the image recognition module is used to decompose video into individual images; the key point recognition module is used to identify the instrument outline in the image frames and locate key points based on the instrument outline.

[0119] In this embodiment, a video or image is input to the image processing module, and the image splitting module decomposes the video into individual images; the key point recognition module identifies the device contour in each image frame. The key point information identified by the key point recognition module is as follows: the position A of the beginning of the actuator in each image, the position B of the visible point of the end of the actuator, and the midpoint of the contour of any two non-overlapping contour points of the tube in the image and the actuator.

[0120] The estimation module is used to calculate the pixel length l of the actuator. p Slope k can be identified n The tilt angle θ0 of the instrument in any one of the first M images; the intersection point of the first tube contour line and the second tube contour line in each image excluding the first M images; the intersection point of the first tube contour line and the second tube contour line in any one of the first M images; and the distance dis. cp Distance comparison cp0 Tilt angle θ, actuator end position P end The actual length l of the actuator mapped to the plane in any one of the first M images. rm0 The actual pixel ratio r at the midpoint of the outline of the actuator. m Contour point projection point and needle tip root point.

[0121] Example 4

[0122] A computer-readable storage medium for storing a computer program that, when executed by a processor, implements the instrument tremor detection method of Embodiment 1 or Embodiment 2.

[0123] The remaining features and working principles of this embodiment are the same as those of Embodiment 1 or Embodiment 2.

[0124] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting instrument tremor, characterized in that, Includes the following steps: Step 1: Receive several images captured during the working process of the instrument, and arrange the images according to the working process of the instrument; Step 2: Use a neural network to identify the device in the image, the device including a tube and an actuator mounted on the tube; Step 3: Identify the position of the start end and the position of the visible point of the end end of the actuator in each image to obtain the position of the start end of the actuator in the nth image. and the visible point position at the end of the actuator And obtain the identifiable slope of the actuator in the nth image. The pixel length of the actuator is calculated based on the position information of the visible points at the beginning and end of the actuator in any one of the first M images that can be recognized at the actuator end. ; Obtain the tilt angle of the instrument in any one of the first M images. ; Step 4: For each image, perform the following processing: fit a straight line to the outer contour points on both sides of the pipe axis, which are respectively the first pipe contour line and the second pipe contour line; obtain the distance from the intersection point of the first pipe contour line and the second pipe contour line to the beginning of the pipe in the nth image (excluding any one of the first M images). The distance between the intersection of the first pipe outline and the second pipe outline in any one of the first M images and the beginning of the pipe is compared. The tilt angle in the nth image is estimated by combining the cotangent and arctangent functions. Specifically: Step 5: Based on the information obtained in Step 4, determine the position of the end of the actuator in the nth image. Specifically: In the formula, The pixel length of the execution element in the nth image, excluding any one of the first M images; For the selected direction of the actuator, or ; Step Six: Calculate the average axis of the instrument across all images; Step 7: Calculate the end position of the actuator in the nth image. The distance to the average axis of the instrument is used to obtain the vibration distance of the instrument in the corresponding image.

2. The method for detecting instrument tremor according to claim 1, characterized in that, In step three, the process for obtaining the tilt angle of the instrument in any one of the first M images is as follows: S3.1: Obtain the actual length of the execution element mapped to the plane in any one of the first M images. ; S3.2: Obtain the actual length of the actuator. ; S3.3: Calculate the tilt angle Specifically: 。 3. The method for detecting instrument tremor according to claim 2, characterized in that, The specific process of step S3.1 is as follows: Obtain any two non-overlapping contour points of the tube body and the midpoint of the contour of the actuator in any one of the first M images; Calculate the pixel distance between the tube diameters at two contour points; obtain the actual size of the instrument diameter at the two contour points; Calculate the actual pixel ratio at the two contour points of the tube body based on the pixel distance of the tube body diameter and the actual size of the instrument diameter; The actual pixel ratio at the midpoint of the actuator's contour is obtained based on the actual pixel ratio at the two midpoints and the pixel distances from the midpoint of the actuator's contour to the two contour points. ; In practice, the length of the actuator mapped to the plane It is the actual pixel ratio at the midpoint of the outline of the actuator. Pixel length of the actuator The product of.

4. The method for detecting instrument tremor according to claim 3, characterized in that, The positions of the visible points at the beginning and end of the actuator, the midpoint of the actuator's outline, and the two non-overlapping outline points on the tube are all identified through a key point network.

5. The method for detecting instrument tremor according to claim 3, characterized in that, In step six, the average axis is obtained in the following way: Based on the position of the actuator end in the nth image and the position of the head end of the actuator Obtain the axis of the actuator in the nth image, with its slope and intercept being respectively... The average axis of the instrument across all images is obtained from the axis of the actuator in each image, and its slope-intercept form equation is: In the formula, It is the slope of the axis of the actuator in each image. The average value, It is the intercept of the axis of the actuator in each image. The average value.

6. The method for detecting instrument tremor according to claim 1, characterized in that, In steps three and four, the position of the identified actuator head is determined. Visible point position at the end of the actuator Slope can be identified and tilt angle Perform smoothing processing.

7. The method for detecting instrument tremor according to claim 6, characterized in that, The smoothing process is as follows: In the formula, , and They are respectively , and Data before smoothing; Represented as weights, each weight satisfies ,in .

8. The method for detecting instrument tremor according to any one of claims 1-7, characterized in that, Based on the position of the actuator end in the nth image Visible point position at the end of the actuator and the tilt angle in the nth image The injection depth of the actuator in the nth image is obtained. Specifically: In the formula, This represents the ratio of the actual distance to the end position of the actuator to the pixel distance. It is the position of the end of the actuator in the nth image. X-axis coordinates It is the position of the end of the actuator in the nth image. The Y-axis coordinate.

9. A detection system for implementing the instrument tremor detection method according to any one of claims 1-8, comprising an image processing module for receiving and processing instrument images and an estimation module for calculating needle tip position; the image processing module includes an image recognition module and a key point recognition module; the image recognition module is used to decompose video into individual images; the key point recognition module is used to identify the instrument outline in the image frame and locate key points based on the instrument outline.

10. A computer-readable storage medium for storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the instrument tremor detection method according to any one of claims 1-8.

Images