Surgical instrument pose recognition methods, devices, equipment and storage media

Abstract

This invention relates to the field of medical device technology, and provides a method, apparatus, device, and storage medium for surgical instrument pose recognition. The method includes: acquiring a target image of the surgical instrument to be identified using a preset positioning camera; the surgical instrument has a positioning mark, and the target image contains the positioning mark; determining a first pose transformation relationship between the positioning camera and the positioning mark based on the target image; determining a second pose transformation relationship between the positioning camera and the surgical instrument using the first pose transformation relationship as a reference; and identifying the target pose of the surgical instrument according to the second pose transformation relationship. By setting a positioning mark on the surgical instrument and identifying the pose of the surgical instrument based on the pose transformation relationship between the positioning camera and the positioning mark, the installation flexibility of the positioning camera can be improved, with no specific requirements for the installation location or method of the positioning camera, thus reducing the equipment cost of the surgical instrument.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a method, apparatus, device and storage medium for surgical instrument position recognition. Background Technology

[0002] With the development of information technology and the integration of medical devices with information technology, the application of surgical robots is becoming increasingly widespread. Most existing surgical robot navigation and positioning systems use infrared positioners, which require specific camera equipment to track and locate the surgical robot. Furthermore, there are specific requirements for the installation location and method of the positioning camera, resulting in high equipment costs. Summary of the Invention

[0003] This invention provides a surgical instrument pose recognition method, device, equipment, and storage medium to address the shortcomings of existing surgical instruments using infrared positioners, which have specific requirements for the installation position and method of the positioning camera, resulting in high equipment costs.

[0004] This invention provides a method for surgical instrument pose recognition, comprising:

[0005] A target image of a surgical instrument to be identified is acquired using a preset positioning camera; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark;

[0006] Determine the first pose transformation relationship between the positioning camera and the positioning marker based on the target image;

[0007] Using the first pose transformation relationship as a reference, a second pose transformation relationship between the positioning camera and the surgical instrument is determined;

[0008] The target pose of the surgical instrument is identified based on the second pose transformation relationship.

[0009] According to the surgical instrument pose recognition method provided by the present invention, the positioning identifier includes multiple positioning codes; the step of determining the first pose transformation relationship between the positioning camera and the positioning identifier based on the target image includes:

[0010] Based on the target image, determine the first conversion relationship between each positioning code in the positioning identifier and the positioning camera;

[0011] The positioning identifier is converted to the camera coordinate system of the positioning camera according to the first conversion relationship, so as to determine the conversion error between any two positioning codes in the positioning identifier in the camera coordinate system.

[0012] Based on the conversion error, a target positioning code is selected from the positioning identifiers, and based on the target conversion relationship between the target positioning code and the positioning camera, the first pose conversion relationship between the positioning camera and the positioning identifier is determined.

[0013] According to the surgical instrument pose recognition method provided by the present invention, the step of converting the positioning identifier to the camera coordinate system of the positioning camera based on the first conversion relationship, so as to determine the conversion error between any two positioning codes in the positioning identifier in the camera coordinate system, includes:

[0014] Based on the first conversion relationship, the positioning identifier is converted to the camera coordinate system of the positioning camera, and a second conversion relationship is determined between each pair of positioning codes in the positioning identifier in the camera coordinate system.

[0015] Obtain the preset third conversion relationship between each pair of positioning codes in the positioning identifier;

[0016] Based on the second and third transformation relationships, the transformation error between any two positioning codes in the positioning identifier is determined in the camera coordinate system.

[0017] According to the surgical instrument pose recognition method provided by the present invention, the step of selecting a target positioning code from the positioning identifier based on the conversion error includes:

[0018] Obtain the number of frames in the target image;

[0019] If the number of frames is less than a preset threshold, return and execute the step of acquiring the target image of the surgical instrument to be identified using a preset positioning camera until the number of frames is greater than or equal to the preset threshold, and select the target positioning code with the smallest error from the positioning identifiers according to the conversion error.

[0020] According to the surgical instrument pose recognition method provided by the present invention, the step of determining the first pose transformation relationship between the positioning camera and the positioning identifier based on the target transformation relationship between the target positioning code and the positioning camera includes:

[0021] The error value of the target positioning code is determined based on the conversion error;

[0022] If the error value of the target positioning code is greater than the preset error threshold, then return and execute the step of acquiring the target image of the surgical instrument to be identified using the preset positioning camera, until the error value of the target positioning code is less than or equal to the error threshold. Then, based on the target conversion relationship between the target positioning code and the positioning camera, determine the first pose conversion relationship between the positioning camera and the positioning identifier.

[0023] According to the surgical instrument pose recognition method provided by the present invention, the positioning mark is a three-dimensional positioning panel or a positioning polyhedron; the three-dimensional positioning panel includes a central panel, each side of the central panel is not perpendicularly connected to a side panel, and the central panel and each of the side panels are provided with positioning codes.

[0024] The first plane containing the side panel and the second plane containing the center panel form a predetermined angle.

[0025] According to the surgical instrument pose recognition method provided by the present invention, both the first pose transformation relationship and the second pose transformation relationship include translation transformation relationship and rotation transformation relationship.

[0026] The present invention also provides a surgical instrument position recognition device, comprising:

[0027] An image acquisition module is used to acquire a target image of a surgical instrument to be identified using a preset positioning camera; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark;

[0028] The first pose conversion module is used to determine the first pose conversion relationship between the positioning camera and the positioning identifier based on the target image;

[0029] The second pose conversion module is used to determine the second pose conversion relationship between the positioning camera and the surgical instrument with reference to the first pose conversion relationship;

[0030] The pose recognition module is used to identify the target pose of the surgical instrument based on the second pose transformation relationship.

[0031] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the surgical instrument pose recognition method as described above.

[0032] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the surgical instrument pose recognition method as described above.

[0033] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the surgical instrument pose recognition method as described above.

[0034] The present invention provides a surgical instrument pose recognition method, apparatus, device, and storage medium. By setting a positioning mark on the surgical instrument, a target image containing the positioning mark is acquired using a positioning camera. A first pose transformation relationship between the positioning camera and the positioning mark is determined based on the target image. Using the first pose transformation relationship as a reference, a second pose transformation relationship between the positioning camera and the surgical instrument is determined. The target pose of the surgical instrument is then identified based on the second pose transformation relationship. By setting a positioning mark on the surgical instrument and identifying the pose of the surgical instrument based on the pose transformation relationship between the positioning camera and the positioning mark, the installation flexibility of the positioning camera can be improved, with no specific requirements on the installation location or method, thus reducing the equipment cost of the surgical instrument. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the surgical instrument pose recognition method provided by the present invention;

[0037] Figure 2 This is one of the structural schematic diagrams of the positioning mark provided by the present invention;

[0038] Figure 3 This is the second structural schematic diagram of the positioning mark provided by the present invention;

[0039] Figure 4 This is another flowchart illustrating the surgical instrument pose recognition method provided by the present invention;

[0040] Figure 5 This is a schematic diagram of the surgical instrument pose recognition device provided by the present invention;

[0041] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] The following is combined Figures 1-6 The present invention describes a surgical instrument pose recognition method, apparatus, device, and storage medium.

[0044] The surgical instrument pose recognition method provided by this invention is applied to surgical instruments for positioning and pose recognition, where pose refers to both position and orientation. Specifically, refer to... Figure 1 , Figure 1 This is a flowchart illustrating the surgical instrument pose recognition method provided in an embodiment of the present invention, based on... Figure 1 The surgical instrument pose recognition method provided in this embodiment of the invention includes:

[0045] Step 100: Use a preset positioning camera to acquire a target image of the surgical instrument to be identified; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark;

[0046] First, a target image of the surgical instrument to be identified is acquired using a preset positioning camera. In this embodiment, the surgical instrument to be identified can be the entire surgical robot body or a part of the surgical robot body, such as the execution part of the surgical robot's end effector arm, etc., without specific limitations. A positioning mark is provided on the surgical instrument to be identified. This positioning mark is a reference for the pose recognition of the surgical instrument, and the acquired target image contains this positioning mark.

[0047] Step 200: Determine the first pose transformation relationship between the positioning camera and the positioning identifier based on the target image;

[0048] When performing pose recognition on surgical instruments, the pose transformation relationship between the positioning camera and the positioning marker is first determined based on the target image; this is the first pose transformation relationship. Specifically, the camera coordinate system of the positioning camera and the coordinate system of the positioning marker are obtained, and the first pose transformation relationship between the positioning camera and the positioning marker is the transformation relationship of a point on the positioning marker from its own coordinate system to the camera coordinate system. The coordinate system of the positioning marker can be the world coordinate system or the tool coordinate system of the surgical instrument; no specific limitation is made here.

[0049] Step 300: Using the first pose transformation relationship as a reference, determine the second pose transformation relationship between the positioning camera and the surgical instrument;

[0050] Using the first pose transformation relationship between the positioning camera and the positioning marker as a reference, the second pose transformation relationship between the positioning camera and the surgical instruments is determined. In the tool coordinate system, the transformation relationship between the positioning marker and the surgical instruments is fixed and known, and can be determined based on the position of the positioning marker on the surgical instruments. Therefore, by using the pose transformation relationship between the positioning camera and the positioning marker as a reference, the second pose transformation relationship between the positioning camera and the surgical instruments can be determined.

[0051] Step 400: Identify the target pose of the surgical instrument based on the second pose transformation relationship.

[0052] Based on the second pose transformation relationship between the positioning camera and the surgical instrument, the surgical instrument can be transformed from its tool coordinate system to the camera coordinate system of the positioning camera. This allows for the identification of the target pose of the surgical instrument based on the camera coordinate system, thereby enabling the positioning, tracking, and pose recognition of the surgical instrument.

[0053] In this embodiment, by setting positioning markers on the surgical instrument, a positioning camera acquires a target image containing the positioning markers; a first pose transformation relationship between the positioning camera and the positioning markers is determined based on the target image; a second pose transformation relationship between the positioning camera and the surgical instrument is determined with reference to the first pose transformation relationship; and the target pose of the surgical instrument is identified according to the second pose transformation relationship. By setting positioning markers on the surgical instrument and identifying the pose of the surgical instrument based on the pose transformation relationship between the positioning camera and the positioning markers, the installation flexibility of the positioning camera can be improved, with no specific requirements on the installation position or method of the positioning camera, thus reducing the equipment cost of the surgical instrument.

[0054] Preferably, in this embodiment, the positioning identifier is a three-dimensional positioning panel or positioning polyhedron comprising multiple positioning codes. Specifically, when the positioning identifier is a three-dimensional positioning panel, the three-dimensional positioning panel includes a central panel, which is a polygonal panel, with each side edge of the central panel being non-perpendicularly connected to a side panel, and positioning codes are provided on the central panel and each side panel; furthermore, the first plane containing any side panel forms a preset angle with the second plane containing the central panel, such as 25°. By using the geometric relationships of the three-dimensional positioning panel or positioning polyhedron itself, the pose of the positioning codes identified in real time is filtered, avoiding the instability of single positioning code recognition and the problem of errors easily occurring in the Z direction of a single positioning code, thus ensuring the robustness of surgical instrument pose recognition.

[0055] Reference Figure 2 , Figure 2 This is one of the structural schematic diagrams of the three-dimensional positioning panel provided in this embodiment, such as... Figure 2As shown, the 3D positioning panel includes five positioning codes 1 and five positioning panels 2. Positioning panel 2 includes a central panel and four side panels. Each panel has a positioning code containing different numbering information. The central panel and side panels are at a 25° angle, ensuring that the five positioning codes are on five different planes. This guarantees that no two positioning codes will simultaneously coincide with the Z-axis of the camera coordinate system under any condition, and also ensures that the positioning camera simultaneously images the positioning codes. The positioning camera can be a depth camera. By imaging the positioning codes with the depth camera and using the data from the depth camera at the center point of the positioning codes as the Z-axis information, the system's accuracy and stability can be further improved.

[0056] Furthermore, referring to Figure 3 , Figure 3 This is one of the schematic diagrams of the mechanism for positioning the polyhedron provided in this embodiment. Figure 3 The positioning polyhedron shown employs a polyhedral structure, ensuring that no two positioning codes simultaneously coincide with the Z-axis of the camera coordinate system, regardless of the surgical instrument's position. This guarantees that the positioning camera can image and locate the positioning markers from any angle and can simultaneously calculate the pose of multiple positioning codes, improving the accuracy of surgical instrument pose recognition.

[0057] In a preferred embodiment, based on the design specifications of the positioning markers and their placement on the surgical instruments, the conversion relationship between the positioning markers and the surgical instruments can be determined. Furthermore, based on the design specifications of the positioning markers, the rotation and translation conversion relationship between any two positioning codes within the positioning markers can be determined. Where n and m represent two different positioning codes in the positioning identifier, A is a matrix used to represent the transformation relationship, R is a matrix representing the rotation transformation relationship, and T is a matrix representing the translation transformation relationship. This indicates the conversion relationship between the location codes n and m. This indicates the rotational transformation relationship between the positioning codes n and m. This represents the translation relationship between the positioning codes n and m.

[0058] Preferably, in step 200, determining the first pose transformation relationship between the positioning camera and the positioning marker based on the target image specifically includes:

[0059] Step 201: Determine the first conversion relationship between each positioning code in the positioning identifier and the positioning camera based on the target image;

[0060] Step 202: Convert the positioning identifier to the camera coordinate system of the positioning camera according to the first conversion relationship, so as to determine the conversion error between any two positioning codes in the positioning identifier in the camera coordinate system;

[0061] Step 203: Select a target positioning code from the positioning identifiers based on the conversion error, and determine the first pose conversion relationship between the positioning camera and the positioning identifiers based on the target conversion relationship between the target positioning code and the positioning camera.

[0062] First, based on the imaging principle of the positioning camera and its intrinsic parameters, the first conversion relationship between each positioning code in the positioning identifier and the positioning camera is determined based on the acquired target image. This represents the conversion relationship between the positioning camera and the positioning code n. This represents the rotational conversion relationship between the positioning camera and the positioning code n. This represents the translational conversion relationship between the positioning camera and the positioning code n.

[0063] Based on this first transformation relationship, the positioning identifier is transformed to the camera coordinate system of the positioning camera. That is, each positioning code in the positioning identifier is transformed from the coordinate system of the positioning identifier to the camera coordinate system, thereby determining the transformation error between any two positioning codes in the positioning identifier in the camera coordinate system. Based on this transformation error, a target positioning code is selected from the multiple positioning codes of the positioning identifier. Using this target positioning code as a reference, the first pose transformation relationship between the positioning camera and the positioning identifier is determined. Specifically, based on the first transformation relationship between the positioning camera and each positioning code, the first transformation relationship between the target positioning code and the positioning camera is used as the pose transformation relationship between the positioning camera and the positioning identifier.

[0064] By using the rigid transformation relationship between multiple positioning codes, a target positioning code can be selected from multiple positioning codes as a reference. This allows for the calibration of positioning codes whose poses differ significantly from other positioning codes. The redundancy check algorithm for multiple positioning codes increases the accuracy of pose recognition and solves the problem of calibration instability caused by the overlap of the 3D pose positioning code with the Z-axis of the camera coordinate system, further improving the accuracy of pose recognition.

[0065] It is understandable that the target positioning code can be selected each time the surgical instrument's pose is recognized, or it can be selected every preset time interval. Alternatively, it can be selected based on the initial target positioning code, and then reselected when the surgical instrument's pose changes exceed a set threshold, such as a rotation angle exceeding a preset angle threshold and / or a translation distance exceeding a preset distance threshold. When reselecting a target positioning code after the surgical instrument's pose changes exceed a set threshold, the set thresholds for different positioning codes can be the same or different. For the selected target positioning code, the set thresholds can be the same or different when the surgical instrument's rotation direction and / or translation direction are different; no specific limitations are imposed here.

[0066] Further, in step 202, the positioning identifier is converted to the camera coordinate system of the positioning camera according to the first conversion relationship, so as to determine the conversion error between any two positioning codes in the positioning identifier in the camera coordinate system, including:

[0067] Step 2021: Convert the positioning identifier to the camera coordinate system of the positioning camera according to the first conversion relationship, and determine the second conversion relationship between each pair of positioning codes in the positioning identifier in the camera coordinate system.

[0068] Step 2022: Obtain the preset third conversion relationship between each pair of positioning codes in the positioning identifier;

[0069] Step 2023: Based on the second transformation relationship and the third transformation relationship, determine the transformation error between any two positioning codes in the positioning identifier in the camera coordinate system.

[0070] Based on the first transformation relationship between the positioning camera and each positioning code, each positioning code in the positioning identifier is transformed to the camera coordinate system of the positioning camera, and the second transformation relationship between each pair of positioning codes in the positioning identifier is determined in the camera coordinate system. That is, by performing a rigid transformation on the positioning codes to transform them to the camera coordinate system, the rotation and translation transformation relationship between any two positioning codes in the positioning identifier can be determined from the perspective of the positioning camera.

[0071]

[0072] Obtain the preset third conversion relationship between each pair of positioning codes in the positioning identifier, i.e. The third conversion relationship is determined based on the design specifications of the positioning identifier, and the third conversion relationship between any two positioning codes in the positioning identifier is fixed and known. Based on the second and third conversion relationships, the conversion error between any two positioning codes in the positioning identifier from the perspective of the positioning camera can be determined: The conversion error includes rotation error and translation error, where the rotation error between any two positioning codes is... Translation error The results are obtained by calculating according to the following formulas 1 and 2 respectively:

[0073]

[0074]

[0075] Based on the calculated conversion error between any two positioning codes, the positioning code with the smallest error is selected as the target positioning code and used as a conversion reference for surgical instruments. By identifying the pose of the positioning code, the target pose of the surgical instruments can be determined based on the conversion relationship between the positioning code and the surgical instruments. Alternatively, based on the conversion relationship between the positioning camera and the target positioning code, as well as the preset conversion relationship between the positioning code and the surgical instruments, the conversion relationship between the positioning camera and the surgical instruments is determined. Based on this conversion relationship, the surgical instruments are converted to the camera coordinate system, thereby identifying the target pose of the surgical instruments.

[0076] In this embodiment, by setting redundant positioning codes, it is applicable to positioning cameras installed at different angles and / or positions. This allows for accurate pose recognition of surgical instruments even when the imaging angle of the positioning camera is poor, thus improving the installation flexibility of the positioning camera. Furthermore, compared to existing surgical instruments using infrared positioning devices, this method has no specific requirements for the positioning camera. The flexible and adaptable positioning markers can be adapted to different surgical instruments or the actuators of surgical robots, improving the applicability and flexibility of pose recognition.

[0077] Preferably, refer to Figure 4 Another flowchart of the surgical instrument pose recognition method shown in the figure illustrates that, in a preferred embodiment, to ensure the accuracy of pose recognition, it is necessary to control the number of target images acquired and the conversion accuracy of the selected positioning code.

[0078] Specifically, based on Figure 4 When identifying the target pose of surgical instruments, a target image containing positioning markers is first acquired using a positioning camera. Based on the acquired target image, the conversion relationship of each positioning code relative to the positioning camera is determined. According to this conversion relationship, the conversion relationship between any two positioning codes in the positioning marker is determined from the camera's perspective. Based on the design specifications of the positioning marker, the preset conversion relationship between each pair of positioning codes in the positioning marker is obtained. The preset conversion relationship between any two positioning codes is compared with its conversion relationship from the camera's perspective to determine the conversion error between the two positioning codes.

[0079] Based on the conversion error between any two positioning codes, a target positioning code with the smallest error is selected as the conversion reference for the surgical instrument. Specifically, step 203, selecting the target positioning code from the positioning identifiers based on the conversion error, may further include:

[0080] Step 2031: Obtain the number of frames in the target image;

[0081] Step 2032: If the number of frames is less than a preset threshold, return and execute the step of acquiring the target image of the surgical instrument to be identified using a preset positioning camera until the number of frames is greater than or equal to the preset threshold. Then, select the target positioning code with the smallest error from the positioning identifiers based on the conversion error.

[0082] When selecting a target positioning code, the number of frames in the target image is first obtained to determine if it meets a preset threshold. If the number of frames is less than the preset threshold, the process returns to and executes the step of acquiring the target image of the surgical instrument to be identified using a preset positioning camera, to reacquire the target image until a sufficient number of images are acquired, i.e., the number of frames in the target image is greater than or equal to the preset threshold. Based on the conversion error, the positioning code n with the smallest coordinate pose error in multiple consecutive frames is selected as the target positioning code from the positioning codes in the positioning identifier. For example, the preset threshold is 30. When a sufficient number of 30 target images are acquired, the positioning code with the smallest error is selected as the target positioning code from the positioning codes in the positioning identifier. The error of the positioning code is obtained based on the conversion error, for example, by weighting the conversion error. As an error in the positioning code.

[0083] Furthermore, for the selected target positioning code, it is necessary to determine whether the error of the target positioning code meets the requirements, thereby determining the usability of the target positioning code. Specifically, in step 203, based on the target transformation relationship between the target positioning code and the positioning camera, the first pose transformation relationship between the positioning camera and the positioning marker is determined, which also includes:

[0084] Step 2033: Determine the error value of the target positioning code based on the conversion error;

[0085] Step 2034: If the error value of the target positioning code is greater than a preset error threshold, then return and execute the step of acquiring the target image of the surgical instrument to be identified using a preset positioning camera, until the error value of the target positioning code is less than or equal to the error threshold. Then, based on the first conversion relationship between the target positioning code and the positioning camera, determine the first pose conversion relationship between the positioning camera and the positioning identifier.

[0086] Based on the conversion error between any two positioning codes in the positioning identifier, the error value of the target positioning code is determined. This error value is then compared to a preset error threshold to determine if the target positioning code's error meets the requirements. If the target positioning code's error value is greater than the preset error threshold, it does not meet the error requirements. The process then returns to and executes the step of acquiring the target image of the surgical instrument to be identified using a preset positioning camera. A new target image containing the positioning identifier is acquired for pose recognition of the surgical instrument until the target positioning code's error value is less than or equal to the preset error threshold, i.e., when the target positioning code's error meets the error requirements. Based on the target conversion relationship between the target positioning code and the positioning camera, the first pose conversion relationship between the positioning camera and the positioning identifier is determined. Finally, based on the preset conversion relationship between the target positioning code and the surgical instrument... Determine the second pose transformation relationship between the positioning camera and surgical instruments. The target pose of the surgical instruments is identified based on this second pose transformation relationship.

[0087] In this embodiment, by using the number of frames of the target image and the coordinate pose error of each positioning code in multiple consecutive target images, the positioning code with the smallest error is selected from multiple positioning codes as a reference for the conversion relationship between the positioning camera and the surgical instrument. Based on the redundancy check of the positioning code, the accuracy of the pose recognition of the surgical instrument is improved.

[0088] The surgical instrument pose recognition device provided by the present invention is described below. The surgical instrument pose recognition device described below and the surgical instrument pose recognition method described above can be referred to in correspondence.

[0089] Reference Figure 5 The surgical instrument pose recognition device provided in this embodiment of the invention includes:

[0090] Image acquisition module 10 is used to acquire target images of surgical instruments to be identified using a preset positioning camera; the surgical instruments are provided with positioning marks, and the target images contain the positioning marks;

[0091] The first pose conversion module 20 is used to determine the first pose conversion relationship between the positioning camera and the positioning identifier based on the target image;

[0092] The second pose conversion module 30 is used to determine the second pose conversion relationship between the positioning camera and the surgical instrument with reference to the first pose conversion relationship;

[0093] The pose recognition module 40 is used to identify the target pose of the surgical instrument based on the second pose transformation relationship.

[0094] In one embodiment, the positioning identifier includes multiple positioning codes, and the first pose conversion module 20 is further configured to:

[0095] Based on the target image, determine the first conversion relationship between each positioning code in the positioning identifier and the positioning camera;

[0096] The positioning identifier is converted to the camera coordinate system of the positioning camera according to the first conversion relationship, so as to determine the conversion error between any two positioning codes in the positioning identifier in the camera coordinate system.

[0097] Based on the conversion error, a target positioning code is selected from the positioning identifiers, and based on the target conversion relationship between the target positioning code and the positioning camera, the first pose conversion relationship between the positioning camera and the positioning identifier is determined.

[0098] In one embodiment, the first pose conversion module 20 is further configured to:

[0099] Based on the first conversion relationship, the positioning identifier is converted to the camera coordinate system of the positioning camera, and a second conversion relationship is determined between each pair of positioning codes in the positioning identifier in the camera coordinate system.

[0100] Obtain the preset third conversion relationship between each pair of positioning codes in the positioning identifier;

[0101] Based on the second and third transformation relationships, the transformation error between any two positioning codes in the positioning identifier is determined in the camera coordinate system.

[0102] In one embodiment, the first pose conversion module 20 is further configured to:

[0103] Obtain the number of frames in the target image;

[0104] If the number of frames is less than a preset threshold, return and execute the step of acquiring the target image of the surgical instrument to be identified using a preset positioning camera until the number of frames is greater than or equal to the preset threshold, and select the target positioning code with the smallest error from the positioning identifiers according to the conversion error.

[0105] In one embodiment, the first pose conversion module 20 is further configured to:

[0106] The error value of the target positioning code is determined based on the conversion error;

[0107] If the error value of the target positioning code is greater than the preset error threshold, then return and execute the step of acquiring the target image of the surgical instrument to be identified using the preset positioning camera, until the error value of the target positioning code is less than or equal to the error threshold. Then, based on the target conversion relationship between the target positioning code and the positioning camera, determine the first pose conversion relationship between the positioning camera and the positioning identifier.

[0108] In one embodiment, the positioning identifier is a three-dimensional positioning panel or a positioning polyhedron; the three-dimensional positioning panel includes a center panel, each side of the center panel is not perpendicularly connected to a side panel, and the center panel and each of the side panels are provided with positioning codes.

[0109] The first plane containing the side panel and the second plane containing the center panel form a predetermined angle.

[0110] In one embodiment, both the first pose transformation relationship and the second pose transformation relationship include a translation transformation relationship and a rotation transformation relationship.

[0111] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a surgical instrument pose recognition method, which includes:

[0112] A target image of a surgical instrument to be identified is acquired using a preset positioning camera; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark;

[0113] Determine the first pose transformation relationship between the positioning camera and the positioning marker based on the target image;

[0114] Using the first pose transformation relationship as a reference, a second pose transformation relationship between the positioning camera and the surgical instrument is determined;

[0115] The target pose of the surgical instrument is identified based on the second pose transformation relationship.

[0116] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the surgical instrument pose recognition method provided by the above methods, the method comprising:

[0118] A target image of a surgical instrument to be identified is acquired using a preset positioning camera; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark;

[0119] Determine the first pose transformation relationship between the positioning camera and the positioning marker based on the target image;

[0120] Using the first pose transformation relationship as a reference, a second pose transformation relationship between the positioning camera and the surgical instrument is determined;

[0121] The target pose of the surgical instrument is identified based on the second pose transformation relationship.

[0122] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the surgical instrument pose recognition method provided by the methods described above, the method comprising:

[0123] A target image of a surgical instrument to be identified is acquired using a preset positioning camera; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark;

[0124] Determine the first pose transformation relationship between the positioning camera and the positioning marker based on the target image;

[0125] Using the first pose transformation relationship as a reference, a second pose transformation relationship between the positioning camera and the surgical instrument is determined;

[0126] The target pose of the surgical instrument is identified based on the second pose transformation relationship.

[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surgical instrument pose recognition method, characterized in that, The method includes: A target image of a surgical instrument to be identified is acquired using a preset positioning camera; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark; the positioning mark includes multiple positioning codes. Determining the first pose transformation relationship between the positioning camera and the positioning identifier based on the target image includes: determining a first transformation relationship between each positioning code in the positioning identifier and the positioning camera based on the target image; transforming the positioning identifier to the camera coordinate system of the positioning camera according to the first transformation relationship, to determine the transformation error between any two positioning codes in the positioning identifier in the camera coordinate system; selecting a target positioning code from the positioning identifier according to the transformation error, and determining the first pose transformation relationship between the positioning camera and the positioning identifier according to the target positioning code and the positioning camera. Using the first pose transformation relationship as a reference, a second pose transformation relationship between the positioning camera and the surgical instrument is determined; The target pose of the surgical instrument is identified based on the second pose transformation relationship.

2. The surgical instrument pose recognition method according to claim 1, characterized in that, The step of converting the positioning identifier to the camera coordinate system of the positioning camera according to the first conversion relationship, to determine the conversion error between any two positioning codes in the positioning identifier in the camera coordinate system, includes: Based on the first conversion relationship, the positioning identifier is converted to the camera coordinate system of the positioning camera, and a second conversion relationship is determined between each pair of positioning codes in the positioning identifier in the camera coordinate system. Obtain the preset third conversion relationship between each pair of positioning codes in the positioning identifier; Based on the second and third transformation relationships, the transformation error between any two positioning codes in the positioning identifier is determined in the camera coordinate system.

3. The surgical instrument pose recognition method according to claim 1, characterized in that, The step of selecting the target location code from the location identifier based on the conversion error includes: Obtain the number of frames in the target image; If the number of frames is less than a preset threshold, return and execute the step of acquiring the target image of the surgical instrument to be identified using a preset positioning camera until the number of frames is greater than or equal to the preset threshold, and select the target positioning code with the smallest error from the positioning identifiers according to the conversion error.

4. The surgical instrument pose recognition method according to claim 1, characterized by, The step of determining the first pose transformation relationship between the positioning camera and the positioning identifier based on the target transformation relationship between the target positioning code and the positioning camera includes: The error value of the target positioning code is determined based on the conversion error; If the error value of the target positioning code is greater than the preset error threshold, then return and execute the step of acquiring the target image of the surgical instrument to be identified using the preset positioning camera, until the error value of the target positioning code is less than or equal to the error threshold. Then, based on the target conversion relationship between the target positioning code and the positioning camera, determine the first pose conversion relationship between the positioning camera and the positioning identifier.

5. The surgical instrument pose recognition method according to claim 1, characterized in that, The positioning identifier is a three-dimensional positioning panel or a positioning polyhedron; the three-dimensional positioning panel includes a center panel, each side of the center panel is not perpendicularly connected to a side panel, and the center panel and each of the side panels are provided with positioning codes. The first plane containing the side panel and the second plane containing the center panel form a predetermined angle.

6. The surgical instrument pose recognition method according to claim 1, characterized in that, Both the first pose transformation relationship and the second pose transformation relationship include translation transformation relationship and rotation transformation relationship.

7. A surgical instrument position recognition device, characterized in that, include: An image acquisition module is used to acquire a target image of a surgical instrument to be identified using a preset positioning camera; the surgical instrument is provided with a positioning mark, and the target image contains the positioning mark; the positioning mark includes multiple positioning codes; The first pose conversion module is used to determine the first pose conversion relationship between the positioning camera and the positioning identifier based on the target image, including: determining a first conversion relationship between each positioning code in the positioning identifier and the positioning camera based on the target image; converting the positioning identifier to the camera coordinate system of the positioning camera according to the first conversion relationship, so as to determine the conversion error between any two positioning codes in the positioning identifier in the camera coordinate system; selecting a target positioning code from the positioning identifier according to the conversion error, and determining the first pose conversion relationship between the positioning camera and the positioning identifier according to the target positioning code and the positioning camera. The second pose conversion module is used to determine the second pose conversion relationship between the positioning camera and the surgical instrument with reference to the first pose conversion relationship; The pose recognition module is used to identify the target pose of the surgical instrument based on the second pose transformation relationship.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the surgical instrument pose recognition method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the surgical instrument pose recognition method as described in any one of claims 1 to 6.

Images