Methods, systems, and devices for needle attitude estimation and navigation positioning in retinal surgery

Abstract

This disclosure relates to the field of computer medical technology, specifically to a method, system, and device for estimating the attitude of a retinal surgical needle, as well as a navigation and positioning method. The attitude estimation method includes: at the current moment, processing a two-dimensional RGB image of the retinal surgical needle acquired by a camera; creating a two-dimensional mask to obtain a foreground mask; extracting multiple connected component sequences from the foreground mask; matching the multiple connected component sequences with multiple three-dimensional reference point sequences to obtain multiple matching pairs; for each matching pair, iteratively applying the RANSAC algorithm and the PnP algorithm to obtain optimal attitude estimation information; and obtaining the attitude estimation information of the retinal surgical needle at the current moment based on the optimal attitude estimation information. This disclosure can accurately estimate the attitude of the retinal surgical needle in real time, helping operators make accurate judgments and adjustments, thereby improving the safety and success rate of retinal surgery.

Description

Technical Field

[0001] This disclosure relates to the field of computer medical technology, specifically to a method, system, and device for estimating the attitude of a retinal surgical needle, as well as a navigation and positioning method. Background Technology

[0002] In retinal surgery, accurate needle tip positioning and navigation are crucial for a successful procedure. During the operation, changes in the needle's posture can affect the safety and outcome of the surgery. By continuously monitoring the needle's posture, the position and angle of the needle relative to the target area (such as the retinal lesion) can be calculated more accurately, allowing for timely detection and correction of potential errors. This leads to more precise surgical procedures and reduces surgical risks.

[0003] Due to the complex structure of the fundus and the limited surgical space, traditional positioning methods typically rely on manual operation or simple mechanical devices, which are insufficient to meet the demands for high precision and efficiency. Therefore, there is an urgent need for automated and intelligent positioning needles for fundus surgery. Summary of the Invention

[0004] To address the problems in the related technologies, embodiments of this disclosure provide a method, system, and device for estimating the attitude of a fundus surgical needle, as well as a navigation and positioning method.

[0005] In a first aspect, this disclosure provides a method for estimating the posture of a fundus surgical needle. The fundus surgical needle has a positioning region, which includes multiple positioning patterns. The positions of the multiple positioning patterns are used to determine multiple three-dimensional reference points of the fundus surgical needle. The multiple three-dimensional reference points are divided into multiple three-dimensional reference point sequences. The area in the positioning region other than the multiple positioning patterns has a specified color. The method includes:

[0006] At the current moment, acquire a two-dimensional RGB image of the surgical needle for fundus surgery captured by the camera;

[0007] Convert the two-dimensional RGB image into a two-dimensional HSV image;

[0008] A two-dimensional mask is created using a set color threshold, and the two-dimensional HSV image is segmented using the two-dimensional mask to obtain a foreground mask, wherein the color threshold is related to the specified color;

[0009] Extract multiple connected components from the foreground mask, divide the multiple connected components, and obtain multiple connected component sequences that correspond to the multiple three-dimensional reference point sequences respectively;

[0010] For each of the multiple connected component sequences, calculate the center coordinates of each connected component in the camera coordinate system and set the label information, and create a feature list for each connected component in the connected component sequence. The feature list includes the center coordinates and label information of the corresponding connected component.

[0011] Based on the feature list of each connected component in the multiple connected component sequences, a one-to-one match is performed between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. Each matching pair includes a three-dimensional reference point and a center coordinate.

[0012] For the multiple matching pairs, the Random Sample Consensus (RANSAC) algorithm is used to perform multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The Perspective n-point PnP algorithm is used to estimate the camera pose of the preset number of matching pairs and calculate the number of inliers. The camera pose estimation result obtained in the iteration with the most inliers is used as the pose information of the camera relative to the fundus surgical needle.

[0013] By performing inverse camera pose transformation, the pose information of the camera relative to the retinal surgical needle is converted into the pose estimation information of the retinal surgical needle in the world coordinate system at the current moment.

[0014] According to embodiments of this disclosure, each connected component sequence includes a portion of the plurality of connected components;

[0015] Each three-dimensional reference point sequence includes a portion of the three-dimensional reference points from the plurality of three-dimensional reference points.

[0016] According to embodiments of this disclosure, converting the two-dimensional RGB image into a two-dimensional HSV image includes performing the image conversion using the following formula:

[0017]

[0018] Where H represents hue, R represents red luminance value, G represents green luminance value, B represents blue luminance value, max(R,G,B) is the maximum value among red, green and blue luminance values, and min(R,G,B) is the minimum value among red, green and blue luminance values.

[0019] According to embodiments of this disclosure, each connected component sequence is provided with a preset range, and the method further includes:

[0020] Check whether the connected components in each connected component sequence are within the corresponding preset range;

[0021] If so, then the connected component sequence is determined to be a valid connected component sequence;

[0022] If not, the connected component sequence is determined to be an abnormal connected component sequence, and the abnormal connected component sequence is not used for camera pose estimation.

[0023] According to embodiments of this disclosure, each connected component sequence is provided with a preset threshold for the number of interior points, and the method further includes:

[0024] Calculate the number of interior points in each connected component sequence;

[0025] Check whether the number of interior points in each connected component sequence is within the corresponding preset interior point number threshold;

[0026] If so, then the connected component sequence is determined to be a valid connected component sequence;

[0027] If not, the connected component sequence is determined to be an abnormal connected component sequence, and the abnormal connected component sequence is not used for camera pose estimation.

[0028] According to embodiments of this disclosure, the RANSAC algorithm has a set number of iterations;

[0029] The step of using the perspective n-point PnP algorithm to estimate the camera pose of the preset number of matching pairs and calculate the number of interior points includes:

[0030] The PnP algorithm is used to estimate the camera pose of the preset number of matching pairs to obtain the corresponding camera pose estimation results.

[0031] The three-dimensional reference points of the remaining matching pairs (excluding the preset number of matching pairs) are projected onto the camera coordinate system using the corresponding camera pose estimation results to obtain multiple three-dimensional reprojection results.

[0032] By comparing the multiple 3D reprojection results with the center coordinates in the remaining matching pairs, the reprojection error of each matching pair in the remaining matching pairs is obtained;

[0033] Matching pairs whose reprojection error is less than the reprojection error threshold are taken as interior points.

[0034] According to embodiments of this disclosure, the method further includes:

[0035] Obtain the positional relationship information between the positioning area and the tip of the fundus surgical needle in the world coordinate system;

[0036] Based on the positional relationship information and the attitude estimation information of the fundus surgical needle in the world coordinate system at the current moment, the attitude estimation information of the needle tip in the world coordinate system is located.

[0037] Secondly, this disclosure provides a navigation and positioning method for a fundus surgical needle, the method comprising:

[0038] At the current moment, acquire a two-dimensional RGB image of the surgical needle for fundus surgery captured by the camera;

[0039] Convert the two-dimensional RGB image into a two-dimensional HSV image;

[0040] A two-dimensional mask is created using a set color threshold, and the two-dimensional HSV image is segmented using the two-dimensional mask to obtain a foreground mask;

[0041] Extract multiple connected components from the foreground mask, divide the multiple connected components, and obtain multiple connected component sequences that correspond to multiple three-dimensional reference point sequences respectively;

[0042] For each of the multiple connected component sequences, calculate the center coordinates of each connected component in the camera coordinate system and set the label information, and create a feature list for each connected component in the connected component sequence. The feature list includes the center coordinates and label information of the corresponding connected component.

[0043] Based on the feature list of each connected component in the multiple connected component sequences, a one-to-one match is performed between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. Each matching pair includes a three-dimensional reference point and a center coordinate.

[0044] For the multiple matching pairs, the Random Sample Consensus (RANSAC) algorithm is used to perform multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The PnP algorithm is used to estimate the camera pose of the preset number of matching pairs and calculate the number of inliers. The camera pose estimation result obtained in the iteration with the most inliers is used as the pose information of the camera relative to the fundus surgical needle.

[0045] By performing inverse camera pose transformation, the pose information of the camera relative to the fundus surgical needle is converted into the pose estimation information of the fundus surgical needle in the world coordinate system at the current moment;

[0046] For any time after the current time, repeat the above steps to obtain the real-time attitude estimation information of the fundus surgical needle, thereby realizing the navigation and positioning of the fundus surgical needle.

[0047] The real-time attitude estimation information of the fundus surgical needle is sent to the display screen for display.

[0048] Thirdly, this disclosure provides a posture estimation system for a fundus surgical needle, including:

[0049] The image acquisition module is configured to acquire a two-dimensional RGB image of the fundus surgical needle captured by the camera at the current moment;

[0050] The image preprocessing module is configured to convert the two-dimensional RGB image into a two-dimensional HSV image, create a two-dimensional mask using a set color threshold, and segment the two-dimensional HSV image using the two-dimensional mask to obtain a foreground mask; extract multiple connected components from the foreground mask, divide the multiple connected components to obtain multiple connected component sequences corresponding to multiple three-dimensional reference point sequences, calculate the center coordinates of each connected component in the camera coordinate system and set labeling information for each connected component in the multiple connected component sequences, and create a feature list for each connected component in the connected component sequence, the feature list including the center coordinates and labeling information of the corresponding connected component;

[0051] The feature matching module is configured to perform a one-to-one matching between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence, based on the feature list of each connected component in the multiple connected component sequences, to obtain multiple matching pairs, each matching pair including a three-dimensional reference point and a center coordinate;

[0052] The attitude calculation module is configured to perform multiple RANSAC iterations using the Random Sample Consensus (RANSAC) algorithm for the multiple matching pairs. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The PnP algorithm is used to estimate the camera attitude of the preset number of matching pairs and calculate the number of inliers. The camera attitude estimation result obtained in the iteration with the most inliers is used as the attitude information of the camera relative to the fundus surgical needle.

[0053] The attitude estimation module is configured to convert the attitude information of the camera relative to the fundus surgical needle into attitude estimation information of the fundus surgical needle in the world coordinate system at the current moment through inverse camera attitude transformation.

[0054] Fourthly, this disclosure provides a posture estimation device for a fundus surgical needle, comprising: a camera, a computing device, and a microscope, wherein the camera is communicatively connected to the computing device, and the computing device is communicatively connected to the microscope; wherein,

[0055] The microscope is used to display the surgical area of ​​the fundus surgical needle;

[0056] The camera is used to acquire a two-dimensional RGB image of the fundus surgical needle and send the two-dimensional RGB image to the computing device;

[0057] The computing device is used to obtain the pose estimation information of the fundus surgical needle based on the two-dimensional RGB image, wherein the computing device includes a memory and a processor, the memory being used to store computer instructions, the computer instructions being executed by the processor to implement the method as described in any one of the first or second aspects.

[0058] According to the technical solution provided in this disclosure, at the current moment, after preprocessing the two-dimensional RGB image of the fundus surgical needle acquired by the camera, a two-dimensional mask is created to obtain a foreground mask, and multiple connected components in the obtained foreground mask are extracted. These connected components are then divided into connected component sequences corresponding to multiple three-dimensional reference point sequences. Each three-dimensional reference point in each three-dimensional reference point sequence is matched with the center coordinates of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. For each matching pair, the RANSAC algorithm and PnP algorithm are used iteratively to obtain the optimal pose estimation information, and the pose estimation information of the fundus surgical needle at the current moment is obtained based on the optimal pose estimation information. This disclosure enables real-time automated high-precision positioning of the fundus surgical needle's pose. The entire processing flow is real-time, efficient, and highly automated, thereby helping operators make accurate judgments and adjustments, and ultimately improving the safety and success rate of the surgery.

[0059] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0060] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0061] Figure 1 A flowchart illustrating a method for estimating the pose of a fundus surgical needle according to an embodiment of the present disclosure is shown;

[0062] Figure 2 A cross-sectional view of a fundus surgical needle according to an embodiment of the present disclosure is shown;

[0063] Figure 3 A perspective view of a fundus surgical needle according to an embodiment of the present disclosure is shown;

[0064] Figure 4This diagram illustrates the partitioning of a plurality of connected components in a foreground mask according to an embodiment of the present disclosure;

[0065] Figure 5 A logic flowchart of a method for estimating the pose of a surgical needle for fundus surgery according to an embodiment of the present disclosure is shown.

[0066] Figure 6 A flowchart illustrating a navigation and positioning method for a fundus surgical needle according to an embodiment of the present disclosure is shown;

[0067] Figure 7 A structural block diagram of a retinal surgical needle attitude estimation system according to an embodiment of the present disclosure is shown;

[0068] Figure 8 A structural block diagram of a posture estimation device for a fundus surgical needle according to an embodiment of the present disclosure is shown. Detailed Implementation

[0069] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of exemplary embodiments have been omitted from the drawings.

[0070] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0071] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0072] In this disclosure, any operation involving the acquisition of user information or user data, or the display of user information or user data to others, is an operation authorized or confirmed by the user, or actively selected by the user.

[0073] As mentioned earlier, due to the complex structure of the fundus and the limited surgical space, traditional positioning methods typically rely on manual operation or simple mechanical devices, which are insufficient to meet the demands for high precision and efficiency. The inventors discovered that the position and orientation of the retinal surgical needle constantly change during surgery. However, limitations in the precision of manual operation or using robotic arms to manipulate the needle result in a discrepancy between the actual orientation of the needle and the operator's desired orientation. This discrepancy is precisely what needs to be minimized in retinal surgery. By estimating the orientation of the retinal surgical needle in real time, its position and orientation in three-dimensional space can be dynamically tracked, which is crucial for precise control and guidance of the surgery.

[0074] This disclosure provides a method for estimating the pose of a fundus surgical needle. The fundus surgical needle has a positioning region, which includes multiple positioning patterns. The positions of the multiple positioning patterns are used to determine multiple three-dimensional reference points of the fundus surgical needle. The multiple three-dimensional reference points are divided into multiple three-dimensional reference point sequences, and the area in the positioning region other than the multiple positioning patterns has a specified color. The method includes:

[0075] At the current moment, acquire a two-dimensional RGB image of the fundus surgical needle captured by a camera; convert the two-dimensional RGB image into a two-dimensional HSV image; create a two-dimensional mask using a set color threshold, and segment the two-dimensional HSV image using the two-dimensional mask to obtain a foreground mask, wherein the color threshold is related to the specified color; extract multiple connected components from the foreground mask, divide the multiple connected components, and obtain multiple connected component sequences corresponding to the multiple three-dimensional reference point sequences; for each connected component sequence in the multiple connected component sequences, calculate the center coordinates of each connected component in the camera coordinate system and set label information, create a feature list for each connected component in the connected component sequence, the feature list including the center coordinates and label information of the corresponding connected component; based on the feature list of each connected component in the multiple connected component sequences, perform the following steps on the three-dimensional reference point sequences: In the 3D reference point sequence, each 3D reference point is matched one-to-one with the center coordinate of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. Each matching pair includes a 3D reference point and a center coordinate. For the multiple matching pairs, the Random Sample Consensus (RANSAC) algorithm is used to perform multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The perspective n-point PnP algorithm is used to estimate the camera pose of the preset number of matching pairs and calculate the number of inliers. The camera pose estimation result obtained in the iteration with the most inliers is used as the pose information of the camera relative to the fundus surgical needle. Through inverse camera pose transformation, the pose information of the camera relative to the fundus surgical needle is converted into the pose estimation information of the fundus surgical needle in the world coordinate system at the current time.

[0076] This disclosure enables automated and high-precision positioning of the needle in retinal surgery in real time. The entire process is real-time, efficient, and highly automated, thereby helping operators make accurate judgments and adjustments, and thus improving the safety and success rate of retinal surgery.

[0077] Figure 1 A flowchart illustrating a method for estimating the pose of a surgical needle for fundus surgery according to an embodiment of this disclosure is shown. Figure 1 As shown, the attitude estimation method includes the following steps S101-S108:

[0078] In this disclosure, the fundus surgical needle is provided with a positioning area, which includes multiple positioning patterns. The positions of the multiple positioning patterns are used to determine multiple three-dimensional reference points of the fundus surgical needle. The multiple three-dimensional reference points are divided into multiple three-dimensional reference point sequences. The area of ​​the positioning area other than the multiple positioning patterns has a specified color. In a specific embodiment, the positioning area is implemented by attaching a positioning sticker to the handle of the fundus surgical needle.

[0079] The positions of multiple positioning patterns in the positioning area can be the coordinate positions of the multiple positioning patterns measured in three-dimensional space. The coordinate positions of the multiple positioning patterns in the world coordinate system are used as multiple three-dimensional reference points for the fundus surgical needle. The division of the three-dimensional reference points can be based on the pattern features of each three-dimensional reference point or the distance between the three-dimensional reference points. Each three-dimensional reference point sequence includes a portion of the multiple three-dimensional reference points.

[0080] The inventors noted that although the coordinates of multiple positioning patterns on the retinal surgical needle in the world coordinate system can be directly obtained through measurement and other technical means, in practical applications, due to various factors (such as camera calibration errors, image distortion, and minor movements during surgery), directly using these points for surgical positioning may introduce errors. Therefore, this disclosure uses the coordinates of the positioning patterns obtained by measurement as three-dimensional reference points and uses these three-dimensional reference points for attitude estimation. This allows for the use of more information (such as image features, camera internal parameters, etc.) to reduce these errors and improve the accuracy of retinal surgical needle positioning.

[0081] The following is combined with Figure 2 and Figure 3 The positioning area of ​​the fundus surgical needle disclosed herein is described in detail:

[0082] Figure 2 A cross-sectional view of a fundus surgical needle according to an embodiment of the present disclosure is shown. Figure 3 A perspective view of a fundus surgical needle according to an embodiment of the present disclosure is shown. Figure 2 and Figure 3 The positioning area shown in the example is merely one specific implementation of the positioning area of ​​this disclosure. Those skilled in the art will understand that the location of the positioning area and the way the positioning pattern is set in this specific implementation are not intended to limit the scope of protection of this disclosure. For example, besides… Figure 2 and Figure 3 The ellipse shown can also be a rectangle, a polygon, etc.

[0083] like Figure 2 and Figure 3 As shown, a positioning area is provided at the needle handle position of the fundus surgical needle. Within this positioning area, there are various types of elliptical positioning patterns. In addition to the area with the elliptical positioning patterns, the remaining area in this positioning area is green, so that there is a clear dividing line between the positioning patterns and the area outside the positioning patterns in this positioning area.

[0084] In step S101, at the current moment, a two-dimensional RGB image of the fundus surgical needle is acquired by the camera.

[0085] The two-dimensional RGB image is a digital image synthesized based on the three basic color channels of red, green, and blue. It can be directly obtained by taking a picture with a camera. Those skilled in the art should understand that the camera is only one way to obtain the two-dimensional RGB image. For example, the two-dimensional RGB image of the fundus surgical needle can also be obtained by a video camera, a mobile phone camera device, etc.

[0086] In step S102, the two-dimensional RGB image is converted into a two-dimensional HSV image.

[0087] According to embodiments of this disclosure, image conversion is achieved using the following formula:

[0088]

[0089] Where H represents hue, R represents red luminance value, G represents green luminance value, B represents blue luminance value, max(R,G,B) is the maximum value among red, green and blue luminance values, and min(R,G,B) is the minimum value among red, green and blue luminance values.

[0090] In a two-dimensional HSV image, H represents hue, S represents saturation, and V represents value. Therefore, converting a two-dimensional RGB image to a two-dimensional HSV image makes it easier to select specific colors by setting the hue value.

[0091] According to embodiments of this disclosure, before converting the two-dimensional RGB image into a two-dimensional HSV image, the method further includes: loading the camera's intrinsic parameter matrix and distortion coefficients, and performing distortion correction processing on the RGB image. Distortion correction processing can correct distorted portions in the image, improve the detail representation of image edges and corners, and lay the foundation for further image processing.

[0092] In step S103, a two-dimensional mask is created using a set color threshold, and the two-dimensional HSV image is segmented using the two-dimensional mask to obtain a foreground mask, wherein the color threshold is related to the specified color.

[0093] Furthermore, pixels in the two-dimensional HSV image that meet the set color threshold are set to white, and the remaining pixels are set to black, thereby obtaining the foreground mask.

[0094] For example, in Figure 2 and Figure 3In the example shown, since the specified color is green, the color threshold is set to [35°, 85°]. This color threshold can be used to create a two-dimensional mask to segment the positioning region and positioning pattern in a two-dimensional RGB image. For another example, if the specified color is set to yellow, the color threshold can be set to [15°, 35°].

[0095] In step S104, multiple connected components are extracted from the foreground mask, the multiple connected components are divided, and multiple connected component sequences corresponding to the multiple three-dimensional reference point sequences are obtained.

[0096] In this context, a connected component is a set of interconnected pixels in an image. In this disclosure, each connected component is the outline of each positioning pattern in the positioning region.

[0097] Similar to the principle of dividing the multiple three-dimensional reference points, the division of the multiple connected components can be matched according to the distance or pattern features between the connected components. Figure 4 This diagram illustrates the partitioning of a plurality of connected components in a foreground mask according to an embodiment of the present disclosure.

[0098] exist Figure 4 In the example shown, based on the pattern features of each connected component, the multiple connected components in the positioning area can be divided into 5 groups: connected component sequence 1 to connected component sequence 5.

[0099] In step S105, for each of the plurality of connected component sequences, the center coordinates of each connected component in the camera coordinate system are calculated and the labeling information is set. A feature list for each connected component in the connected component sequence is created, and the feature list includes the center coordinates and labeling information of the corresponding connected component.

[0100] The feature list of each connected component includes the center coordinates (geometric center of the positioning pattern) and marking information of the connected component. Each center coordinate is marked with marking information, which facilitates marking and identification when selecting each center coordinate in the future.

[0101] Furthermore, each connected component sequence includes a portion of the multiple connected components.

[0102] In step S106, based on the feature list of each connected component in the multiple connected component sequences, a one-to-one matching is performed between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. Each matching pair includes a three-dimensional reference point and a center coordinate.

[0103] Furthermore, since multiple three-dimensional reference point sequences and corresponding multiple connected component sequences have been obtained, the three-dimensional reference point sequences and connected component sequences can be matched. For example, in a specific embodiment, multiple three-dimensional reference points are divided into 3 three-dimensional reference point sequences, and multiple connected components are divided into 3 connected component sequences. Then, based on the positional relationship features between each three-dimensional reference point in the first three-dimensional reference point sequence, a matching connected component sequence can be found.

[0104] The inventors noticed that after dividing multiple connected components to obtain multiple connected component sequences, there may be some abnormal sequences that do not meet the requirements, so it is necessary to filter and remove these abnormal sequences.

[0105] In one specific embodiment, each connected component sequence is set with a preset range, and the method further includes: checking whether the connected components in each connected component sequence are within the corresponding preset range; if so, the connected component sequence is determined to be a valid connected component sequence; if not, the connected component sequence is determined to be an abnormal connected component sequence, and the abnormal connected component sequence is not used for camera pose estimation.

[0106] In another specific embodiment, each connected component sequence is provided with a preset threshold for the number of interior points, and the method further includes: calculating the number of interior points in each connected component sequence; checking whether the number of interior points in each connected component sequence is within the corresponding preset threshold for the number of interior points; if yes, then the connected component sequence is determined to be a valid connected component sequence; if no, then the connected component sequence is determined to be an abnormal connected component sequence, and the abnormal connected component sequence is not used for camera pose estimation.

[0107] In each connected component sequence, the interior point represents a connected component that meets the requirements of that connected component sequence (which can be set according to needs).

[0108] RANSAC is an uncertain algorithm that assumes data consists of "interiors" and "outsides," where interiors are data points that conform to the model parameters, and outsides are data points that do not. For example, in a two-dimensional plane, a linear model assumes data points should be distributed along a straight line. RANSAC randomly selects two points as its sample set and uses these two points to compute the equation of the line. Then, RANSAC evaluates the distance of each data point to the line and considers data points with a distance less than a certain threshold as interiors.

[0109] In this disclosure, the preset range can be determined based on the selected sample, and the preset threshold for the number of interior points can be customized according to user needs.

[0110] In step S107, for the multiple matching pairs, the RANSAC algorithm is used for multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs, and the PnP algorithm is used to estimate the camera pose for the preset number of matching pairs and calculate the number of inliers. The camera pose estimation result obtained in the iteration process with the largest number of inliers is used as the pose information of the camera relative to the fundus surgical needle.

[0111] According to an embodiment of the present disclosure, the camera pose estimation result includes a rotation angle vector and a translation vector.

[0112] Among them, the RANSAC algorithm has a set number of iterations, which defines the number of iterations for the multiple matching pairs. The setting of this number of iterations can also be set according to user requirements.

[0113] The following will detail the use of the RANSAC algorithm for multiple RANSAC iterations on the multiple matching pairs (3D reference points - center coordinates of connected components):

[0114] Use the PnP algorithm to estimate the camera pose for the preset number of matching pairs to obtain the corresponding camera pose estimation result; project the 3D reference points in the remaining matching pairs except the preset number of matching pairs among the multiple matching pairs into the camera coordinate system using the corresponding camera pose estimation result to obtain multiple 3D reprojection results; compare the multiple 3D reprojection results with the center coordinates in the remaining matching pairs to obtain the reprojection error of each matching pair in the remaining matching pairs; and use the matching pairs with a reprojection error less than the reprojection error threshold as inliers.

[0115] For example, assume that the number of iterations of the RANSAC algorithm is M times and there are N matching pairs. Then in the first RANSAC iteration, randomly select n matching pairs, use the PnP algorithm for camera pose estimation and obtain the corresponding camera pose estimation result, where the preset number n is the minimum number of samples required for camera pose estimation, and n < N. Then project the 3D reference points in the remaining N - n matching pairs into the camera coordinate system using the obtained camera pose estimation result to obtain multiple 3D reprojection results; compare these multiple 3D reprojection results with the corresponding center coordinates in the remaining N - n matching pairs, then obtain the reprojection error of each matching pair in the remaining N - n matching pairs, and use the matching pairs with a reprojection error less than the reprojection error threshold in the remaining N - n matching pairs as inliers.

[0116] Specifically, if for any matching pair (X i , x i ) in the multiple matching pairs, the 3D reference point Xi =[X i ,Y i Z i ] T The center coordinates x of the connected components i =[u i ,v i ] T Then, when using the PnP algorithm to estimate the camera pose for each of the randomly selected n matching pairs, the following projection model formula is used:

[0117]

[0118] in, Let be the projection point (the center coordinates in the matching pair), K be the camera intrinsic parameter matrix, R be the rotation matrix of the camera to be determined, and t be the translation vector of the camera to be determined.

[0119] Furthermore, the rotation matrix R can be calculated based on the camera intrinsic parameter matrix K, as shown in the following formula:

[0120]

[0121] Where I is the identity matrix.

[0122] Combining the rotation matrix R, first use the formula Calculate the camera's rotation angle θ, and then use the formula Calculate the camera's rotation axis u, and finally calculate the camera's rotation angle vector r = θu, where trace(R) is the trace of R. ij Let be the element in the i-th row and j-th column of the rotation matrix R.

[0123] When calculating the reprojection error, assume that any of the remaining matching pairs is (X j ,x j ), then according to x j =K(RX) j +t), calculate the reprojection error Among them, X j Let x be the coordinates of the three-dimensional reference point of any of the matching pairs. j Let x be the center coordinate of any of the matching pairs. j [2] is x j The scale factor.

[0124] Calculate the reprojection error of all matching pairs in the remaining matching pairs, and count the number of inliers using the following formula:

[0125]

[0126] Wherein, ∈ is the reprojection error threshold, and δ is the indicator function. The indicator function is usually defined as follows: when the condition is true, the function value is 1; when the condition is false, the function value is 0. Specifically, in this disclosure, when the reprojection error is less than the reprojection error threshold ∈, it returns 1; otherwise, it returns 0.

[0127] In step S108, the camera pose inverse transformation is used to convert the pose information of the camera relative to the fundus surgical needle into the pose estimation information of the fundus surgical needle in the world coordinate system at the current moment.

[0128] Those skilled in the art will understand that the orientation of the camera relative to the retinal surgical needle (three-dimensional reference point) is the orientation of the retinal surgical needle in the camera coordinate system.

[0129] Specifically, the positional relationship information between the positioning area and the tip of the fundus surgical needle in the world coordinate system is obtained; based on the positional relationship information and the attitude estimation information of the fundus surgical needle in the world coordinate system at the current time, the attitude estimation information of the needle tip in the world coordinate system is located.

[0130] This disclosure establishes a positioning region on a retinal surgical needle and combines it with a positioning pattern set on that region. Then, it utilizes the PnP algorithm and the RANSAC algorithm. The PnP algorithm is used to derive the position and orientation in three-dimensional space from two-dimensional image points, while the RANSAC algorithm improves the robustness of the estimation results through iterative optimization, effectively reducing the impact of noise and outliers. This achieves high-precision needle tip orientation estimation, helping operators make accurate judgments and enabling them to more accurately control the surgical procedure, promptly detect and correct potential operational errors, thereby improving the safety and success rate of retinal surgery.

[0131] For ease of understanding, the flow of the retinal surgical needle attitude estimation method of this disclosure will be described in detail below with a specific embodiment.

[0132] Figure 5 A logic flowchart of a method for estimating the posture of a surgical needle for fundus surgery according to an embodiment of the present disclosure is shown.

[0133] like Figure 5As shown, the position of the retinal surgical needle is first calibrated to determine multiple three-dimensional reference points based on multiple positioning patterns in the positioning area on the retinal surgical needle. Then, a two-dimensional RGB image of the retinal surgical needle is acquired by taking a picture with a camera. The two-dimensional RGB image is converted into a two-dimensional HSV image. Then, a two-dimensional mask is created according to the specified color of the positioning area, thereby segmenting the two-dimensional HSV image to obtain a foreground mask. After obtaining the foreground mask, multiple connected components are extracted from the foreground mask. Multiple connected components are divided to obtain multiple connected component sequences corresponding to multiple three-dimensional reference point sequences. Then, the center coordinates of each connected component in the connected component sequence are calculated, and multiple three-dimensional reference points are... In the point sequence, each 3D reference point is matched one-to-one with the center coordinates of each connected group in the corresponding connected component sequence to obtain multiple matching pairs. For multiple matching pairs, the RANSAC algorithm is used to perform multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs, and the PnP algorithm is used to estimate the camera pose and calculate the number of inliers. The camera pose estimation result obtained in the iteration with the most inliers is used as the pose information of the camera relative to the retinal surgical needle. Finally, through inverse camera pose transformation, the pose information of the camera relative to the retinal surgical needle is converted into the pose estimation information of the retinal surgical needle in the world coordinate system.

[0134] This disclosure not only boasts a high degree of automation and robustness, but also ensures robustness and stability under different environments by using various image processing techniques, such as connected component analysis, sequence grouping, and abnormal sequence removal. Furthermore, it has a wide range of applications, not only suitable for the positioning of surgical needles in the fundus, but also applicable to other fields such as medical devices, industrial robots, and augmented reality systems that require high-precision positioning and attitude estimation, thus having broad application prospects.

[0135] Figure 6 A flowchart illustrating a navigation and positioning method for a fundus surgical needle according to an embodiment of the present disclosure is shown. Figure 6 The method shown is in Figure 1 Steps S601-S602 are added to the method shown:

[0136] In step S101, at the current moment, a two-dimensional RGB image of the fundus surgical needle is acquired by the camera.

[0137] In step S102, the two-dimensional RGB image is converted into a two-dimensional HSV image.

[0138] In step S103, a two-dimensional mask is created using a set color threshold, and the two-dimensional HSV image is segmented using the two-dimensional mask to obtain a foreground mask, wherein the color threshold is related to the specified color.

[0139] In step S104, multiple connected components are extracted from the foreground mask, the multiple connected components are divided, and multiple connected component sequences corresponding to multiple three-dimensional reference point sequences are obtained.

[0140] In step S105, for each of the plurality of connected component sequences, the center coordinates of each connected component in the camera coordinate system are calculated and the labeling information is set. A feature list for each connected component in the connected component sequence is created, and the feature list includes the center coordinates and labeling information of the corresponding connected component.

[0141] In step S106, based on the feature list of each connected component in the multiple connected component sequences, a one-to-one matching is performed between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. Each matching pair includes a three-dimensional reference point and a center coordinate.

[0142] In step S107, for the multiple matching pairs, the Random Sample Consensus (RANSAC) algorithm is used to perform multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The Perspective n-point PnP algorithm is used to estimate the camera pose of the preset number of matching pairs and calculate the number of inliers. The camera pose estimation result obtained in the iteration with the most inliers is used as the pose information of the camera relative to the fundus surgical needle.

[0143] In step S108, the camera pose inverse transformation is used to convert the pose information of the camera relative to the fundus surgical needle into the pose estimation information of the fundus surgical needle in the world coordinate system at the current moment.

[0144] like Figure 6 As shown, in step S601, for any time after the current time, the above steps are repeated to obtain the real-time attitude estimation information of the fundus surgical needle, thereby realizing the navigation and positioning of the fundus surgical needle.

[0145] In step S602, the real-time attitude estimation information of the fundus surgical needle is sent to the display screen for display in real time.

[0146] This disclosure employs efficient algorithms and optimized image processing workflows, enabling real-time operation during surgery to provide timely positioning and attitude estimation results, meeting the real-time requirements of surgery. It also offers intuitive result visualization, displaying needle tip positioning information and attitude changes to help operators make accurate judgments and adjustments, improving surgical safety and success rates. Furthermore, it reduces reliance on specialized equipment, allowing integration with conventional surgical microscopes and ordinary cameras, thus lowering costs.

[0147] Figure 7 A structural block diagram of a retinal surgical needle attitude estimation system according to an embodiment of the present disclosure is shown.

[0148] like Figure 7 As shown, the system 700 includes: an image acquisition module 710, an image preprocessing module 720, a feature matching module 730, a pose calculation module 740, and a pose estimation module 750.

[0149] The image acquisition module 710 is configured to acquire a two-dimensional RGB image of the fundus surgical needle captured by a camera at the current moment;

[0150] The image preprocessing module 720 is configured to convert the two-dimensional RGB image into a two-dimensional HSV image, create a two-dimensional mask using a set color threshold, and segment the two-dimensional HSV image using the two-dimensional mask to obtain a foreground mask; extract multiple connected components from the foreground mask, divide the multiple connected components to obtain multiple connected component sequences corresponding to multiple three-dimensional reference point sequences, calculate the center coordinates of each connected component in the camera coordinate system and set label information for each connected component in the multiple connected component sequences, and create a feature list for each connected component in the connected component sequence, the feature list including the center coordinates and label information of the corresponding connected component;

[0151] The feature matching module 730 is configured to perform a one-to-one matching between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence, based on the feature list of each connected component in the multiple connected component sequences, to obtain multiple matching pairs, each matching pair including a three-dimensional reference point and a center coordinate;

[0152] The attitude calculation module 740 is configured to perform multiple RANSAC iterations using the Random Sample Consensus (RANSAC) algorithm for the plurality of matching pairs. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the plurality of matching pairs, and the PnP algorithm is used to estimate the camera attitude and calculate the number of inliers for the preset number of matching pairs. The camera attitude estimation result obtained in the iteration with the largest number of inliers is used as the attitude information of the camera relative to the fundus surgical needle.

[0153] The attitude estimation module 750 is configured to convert the attitude information of the camera relative to the fundus surgical needle into attitude estimation information of the fundus surgical needle in the world coordinate system at the current moment through camera attitude inverse transformation.

[0154] Figure 8 A structural block diagram of a posture estimation device for a fundus surgical needle according to an embodiment of the present disclosure is shown.

[0155] like Figure 8 As shown, the device includes: a camera, a computing device, and a microscope, wherein the camera is communicatively connected to the computing device, and the computing device is communicatively connected to the microscope;

[0156] The microscope is used to display the surgical area of ​​the fundus surgical needle;

[0157] The camera is used to acquire a two-dimensional RGB image of the fundus surgical needle and send the two-dimensional RGB image to the computing device;

[0158] The computing device is used to obtain the pose estimation information of the fundus surgical needle based on the two-dimensional RGB image. The computing device includes a memory and a processor. The memory is used to store computer instructions, which are executed by the processor to implement the method described above.

[0159] This disclosure not only provides timely positioning and attitude estimation results to meet the real-time requirements of surgery, but also reduces the reliance on specialized equipment. It can be achieved by combining conventional surgical microscopes and ordinary cameras, thus reducing costs.

[0160] In particular, according to embodiments of this disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium.

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0162] The units or modules described in the embodiments of this disclosure can be implemented in software or programmable hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0163] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system described above; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to perform the methods described in this disclosure.

[0164] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for estimating the posture of a surgical needle in retinal surgery, characterized in that, The fundus surgical needle is provided with a positioning area, which includes multiple positioning patterns. The positions of the multiple positioning patterns are used to determine multiple three-dimensional reference points of the fundus surgical needle. The multiple three-dimensional reference points are divided into multiple three-dimensional reference point sequences. The area in the positioning area other than the multiple positioning patterns has a specified color. The method includes: At the current moment, acquire a two-dimensional RGB image of the surgical needle for fundus surgery captured by the camera; Convert the two-dimensional RGB image into a two-dimensional HSV image; A two-dimensional mask is created using a set color threshold, and the two-dimensional HSV image is segmented using the two-dimensional mask to obtain a foreground mask, wherein the color threshold is related to the specified color; Extract multiple connected components from the foreground mask, divide the multiple connected components, and obtain multiple connected component sequences that correspond to the multiple three-dimensional reference point sequences respectively; For each of the multiple connected component sequences, calculate the center coordinates of each connected component in the camera coordinate system and set the label information, and create a feature list for each connected component in the connected component sequence. The feature list includes the center coordinates and label information of the corresponding connected component. Based on the feature list of each connected component in the multiple connected component sequences, a one-to-one match is performed between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. Each matching pair includes a three-dimensional reference point and a center coordinate. For the multiple matching pairs, the Random Sample Consensus (RANSAC) algorithm is used to perform multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The Perspective n-point PnP algorithm is used to estimate the camera pose of the preset number of matching pairs and calculate the number of inliers. The camera pose estimation result obtained in the iteration with the most inliers is used as the pose information of the camera relative to the fundus surgical needle. By performing inverse camera pose transformation, the pose information of the camera relative to the retinal surgical needle is converted into the pose estimation information of the retinal surgical needle in the world coordinate system at the current moment.

2. The method according to claim 1, characterized in that: Each connected component sequence includes a portion of the multiple connected components; Each three-dimensional reference point sequence includes a portion of the three-dimensional reference points from the plurality of three-dimensional reference points.

3. The method according to claim 1, characterized in that, The conversion of the two-dimensional RGB image to a two-dimensional HSV image includes using the following formula to perform the image conversion: Where H represents hue, R represents red luminance value, G represents green luminance value, B represents blue luminance value, max(R,G,B) is the maximum value among red, green and blue luminance values, and min(R,G,B) is the minimum value among red, green and blue luminance values.

4. The method according to claim 1, characterized in that, Each connected component sequence is set with a preset range, and the method further includes: Check whether the connected components in each connected component sequence are within the corresponding preset range; If so, then the connected component sequence is determined to be a valid connected component sequence; If not, the connected component sequence is determined to be an abnormal connected component sequence, and the abnormal connected component sequence is not used for camera pose estimation.

5. The method according to claim 1, characterized in that, Each connected component sequence is set with a preset threshold for the number of interior points, and the method further includes: Calculate the number of interior points in each connected component sequence; Check whether the number of interior points in each connected component sequence is within the corresponding preset interior point number threshold; If so, then the connected component sequence is determined to be a valid connected component sequence; If not, the connected component sequence is determined to be an abnormal connected component sequence, and the abnormal connected component sequence is not used for camera pose estimation.

6. The method according to claim 1, characterized in that, The RANSAC algorithm has a set number of iterations; The step of using the perspective n-point PnP algorithm to estimate the camera pose of the preset number of matching pairs and calculate the number of interior points includes: The PnP algorithm is used to estimate the camera pose of the preset number of matching pairs to obtain the corresponding camera pose estimation results. The three-dimensional reference points of the remaining matching pairs (excluding the preset number of matching pairs) are projected onto the camera coordinate system using the corresponding camera pose estimation results to obtain multiple three-dimensional reprojection results. By comparing the multiple 3D reprojection results with the center coordinates in the remaining matching pairs, the reprojection error of each matching pair in the remaining matching pairs is obtained; Matching pairs whose reprojection error is less than the reprojection error threshold are taken as interior points.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the positional relationship information between the positioning area and the tip of the fundus surgical needle in the world coordinate system; Based on the positional relationship information and the attitude estimation information of the fundus surgical needle in the world coordinate system at the current moment, the attitude estimation information of the needle tip in the world coordinate system is located.

8. A navigation and positioning method for a fundus surgical needle, characterized in that, The method includes: At the current moment, acquire a two-dimensional RGB image of the surgical needle for fundus surgery captured by the camera; Convert the two-dimensional RGB image into a two-dimensional HSV image; A two-dimensional mask is created using a set color threshold, and the two-dimensional HSV image is segmented using the two-dimensional mask to obtain a foreground mask; Extract multiple connected components from the foreground mask, divide the multiple connected components, and obtain multiple connected component sequences that correspond to multiple three-dimensional reference point sequences respectively; For each of the multiple connected component sequences, calculate the center coordinates of each connected component in the camera coordinate system and set the label information, and create a feature list for each connected component in the connected component sequence. The feature list includes the center coordinates and label information of the corresponding connected component. Based on the feature list of each connected component in the multiple connected component sequences, a one-to-one match is performed between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence to obtain multiple matching pairs. Each matching pair includes a three-dimensional reference point and a center coordinate. For the multiple matching pairs, the Random Sample Consensus (RANSAC) algorithm is used to perform multiple RANSAC iterations. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The PnP algorithm is used to estimate the camera pose of the preset number of matching pairs and calculate the number of inliers. The camera pose estimation result obtained in the iteration with the most inliers is used as the pose information of the camera relative to the fundus surgical needle. By performing inverse camera pose transformation, the pose information of the camera relative to the fundus surgical needle is converted into the pose estimation information of the fundus surgical needle in the world coordinate system at the current moment; For any time after the current time, repeat the above steps to obtain the real-time attitude estimation information of the fundus surgical needle, thereby realizing the navigation and positioning of the fundus surgical needle. The real-time attitude estimation information of the fundus surgical needle is sent to the display screen for display.

9. A posture estimation system for a fundus surgical needle, characterized in that, include: The image acquisition module is configured to acquire a two-dimensional RGB image of the fundus surgical needle captured by the camera at the current moment; The image preprocessing module is configured to convert the two-dimensional RGB image into a two-dimensional HSV image, create a two-dimensional mask using a set color threshold, and segment the two-dimensional HSV image using the two-dimensional mask to obtain a foreground mask; extract multiple connected components from the foreground mask, divide the multiple connected components to obtain multiple connected component sequences corresponding to multiple three-dimensional reference point sequences, calculate the center coordinates of each connected component in the camera coordinate system and set labeling information for each connected component in the multiple connected component sequences, and create a feature list for each connected component in the connected component sequence, the feature list including the center coordinates and labeling information of the corresponding connected component; The feature matching module is configured to perform a one-to-one matching between each three-dimensional reference point in the three-dimensional reference point sequence and the center coordinate of each connected component in the corresponding connected component sequence, based on the feature list of each connected component in the multiple connected component sequences, to obtain multiple matching pairs, each matching pair including a three-dimensional reference point and a center coordinate; The attitude calculation module is configured to perform multiple RANSAC iterations using the Random Sample Consensus (RANSAC) algorithm for the multiple matching pairs. In each RANSAC iteration, a preset number of matching pairs are randomly selected from the multiple matching pairs. The PnP algorithm is used to estimate the camera attitude of the preset number of matching pairs and calculate the number of inliers. The camera attitude estimation result obtained in the iteration with the most inliers is used as the attitude information of the camera relative to the fundus surgical needle. The attitude estimation module is configured to convert the attitude information of the camera relative to the fundus surgical needle into attitude estimation information of the fundus surgical needle in the world coordinate system at the current moment through inverse camera attitude transformation.

10. A device for estimating the posture of a surgical needle in retinal surgery, characterized in that, include: The system includes a camera, a computing device, and a microscope, wherein the camera is communicatively connected to the computing device, and the computing device is communicatively connected to the microscope; wherein... The microscope is used to display the surgical area of ​​the fundus surgical needle; The camera is used to acquire a two-dimensional RGB image of the fundus surgical needle and send the two-dimensional RGB image to the computing device; The computing device is used to obtain the pose estimation information of the fundus surgical needle based on the two-dimensional RGB image, wherein the computing device includes a memory and a processor, the memory is used to store computer instructions, and the computer instructions are executed by the processor to implement the method of any one of claims 1 to 8.

Images