Planting navigation system and computer equipment
Through binocular positioning equipment and processors to detect feature points on complex surfaces and perform three-dimensional coordinate corrections, the problem of inaccurate positioning of traditional navigation systems on complex surfaces is solved, and the safety and success rate of the surgery are improved.
Patent Information
- Application Number
- CN202411947196.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-05-13
AI Technical Summary
Traditional navigation systems are difficult to ensure the accuracy of feature point extraction and positioning accuracy on complex surfaces, which affects surgical safety.
The binocular positioning device and processor are used to improve the accuracy of feature point positioning on complex surfaces through feature point detection, initial three-dimensional coordinate calculation and geometric correction model.
Accurate positioning of feature points on complex surfaces is achieved, and the overall performance of the navigation system and the success rate and safety of the surgery are improved.
Smart Images
Figure CN119970227A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of medical image navigation technology, and in particular, relates to an implant navigation system and computer equipment. Background Art
[0002] In modern medical implant surgeries (such as dental implants, spinal implants, etc.), navigation and tracking technologies are widely used to help doctors accurately position implants and guide tools. High-precision navigation and tracking systems can significantly improve the safety and success rate of surgery, so they place high demands on the precise positioning of surgical tools and devices in space.
[0003] The current navigation system is generally composed of a tracking device (such as an optical tracker, a magnetic tracker) and a tracked device. The surface of the tracked device is usually attached with a number of marking points, which can be used as feature points. Through the position of the feature points and their relative relationship, the system can calculate the spatial position and posture of the tracked device in real time.
[0004] Therefore, the navigation system needs to locate the feature points on the surface of the tracked device. However, traditional feature point markers are usually designed and arranged on flat surfaces or regular geometric surfaces (such as spheres). When the surface of the tracked device has a complex curved shape (such as a cylinder, cone or other free-form surface), it is difficult for traditional markers to guarantee extraction accuracy and positioning accuracy. Summary of the invention
[0005] In view of this, an embodiment of the present application provides a planting navigation system and a computer device to improve the positioning accuracy of feature points on complex curved surfaces.
[0006] A first aspect of an embodiment of the present application provides a planting navigation system, including a tracking device, a binocular positioning device, and a processor, wherein the surface of the tracking device has a characteristic pattern, the binocular positioning device includes a left camera and a right camera, the left camera is used to collect a first two-dimensional image of the tracking device, the right camera is used to collect a second two-dimensional image of the tracking device, the first two-dimensional image and the second two-dimensional image include the characteristic pattern, and the characteristic pattern includes a plurality of characteristic points; the processor is used to:
[0007] Performing feature point detection on the first two-dimensional image and the second two-dimensional image, and respectively determining the two-dimensional coordinates of each of the feature points in the first two-dimensional image and the second two-dimensional image;
[0008] Determine the initial three-dimensional coordinates of each of the feature points according to the two-dimensional coordinates of each of the feature points;
[0009] Based on the surface shape of the tracking device, the initial three-dimensional coordinates are corrected to obtain the target three-dimensional coordinates of each feature point.
[0010] A second aspect of an embodiment of the present application provides a method for determining a position of a feature point, comprising:
[0011] Performing feature point detection on the first two-dimensional image and the second two-dimensional image, and respectively determining the two-dimensional coordinates of each of the feature points in the first two-dimensional image and the second two-dimensional image;
[0012] Determine the initial three-dimensional coordinates of each of the feature points according to the two-dimensional coordinates of each of the feature points;
[0013] Based on the surface shape of the tracking device, the initial three-dimensional coordinates are corrected to obtain the target three-dimensional coordinates of each feature point.
[0014] In a possible implementation manner, the two-dimensional coordinates of each of the feature points in the first two-dimensional image and the second two-dimensional image are determined by the following formula:
[0015]
[0016] in, is the two-dimensional coordinate of the i-th feature point, k1 is the coordinate of the center point of the camera imaging plane of the left camera / the right camera, L / R ,k2 L / R is the radial lens distortion coefficient of the left camera / the right camera, P1 L / R ,P2 L / R are the tangential lens distortion coefficients of the left camera and the right camera respectively.
[0017] In a possible implementation, it is characterized in that the initial three-dimensional coordinates of each of the feature points are determined according to the two-dimensional coordinates of each of the feature points by the following formula:
[0018] P L / R =K L / R [R L / R t L / R ]X
[0019] Among them, K L / R is the intrinsic parameter matrix of the left camera / the right camera, (R L / R ,t L / R )) is the external parameter matrix of the left camera / the right camera, X is the initial three-dimensional coordinate, P L / R It is the coordinates of the initial three-dimensional coordinates projected onto the camera imaging plane corresponding to the left camera / the camera imaging plane corresponding to the right camera.
[0020] In a possible implementation, the initial three-dimensional coordinates are corrected based on the surface shape of the tracking device to obtain the target three-dimensional coordinates of each feature point, including:
[0021] Determining a plurality of geometric constraint models corresponding to the surface shapes;
[0022] Determining a correction model function based on each of the geometric constraint models and the curvature constraint model, wherein the curvature constraint model is used to constrain the curvature of each of the feature points to remain consistent;
[0023] The initial three-dimensional coordinates are corrected according to the correction model function to obtain the target three-dimensional coordinates.
[0024] In a possible implementation, if the surface shape of the tracking device is a plane, the geometric constraint model is:
[0025]
[0026] in, is the plane normal vector of the plane where the surface of the tracking device is located, P0(x0, y0, z0) is any point on the plane where the surface of the tracking device is located, and Pi(xi, yi, zi) is the three-dimensional coordinate of the feature point.
[0027] In a possible implementation, if the surface shape of the tracking device is a cylindrical surface, the geometric constraint model is:
[0028] Γ2:x i 2 +y i 2 -r 2 =0,i=1,...,n
[0029] Among them, Pi (xi, yi, zi) is the three-dimensional coordinate of the feature point, and r is the radius of the cylinder corresponding to the tracking device.
[0030] In a possible implementation, if the surface shape of the tracking device is a conical surface, the geometric constraint model is:
[0031]
[0032] Among them, Pi (xi, yi, zi) is the three-dimensional coordinate of the feature point, and k is a constant used to characterize the cone opening angle corresponding to the tracking device.
[0033] In a possible implementation, the curvature constraint model is:
[0034] Ccurvature =‖P i-1 -2P i +P i+1 ‖2
[0035] Among them, P i-1 ,P i ,P i+1 are three consecutive feature points.
[0036] In a possible implementation, the modified model function is:
[0037]
[0038] Wherein, Pi(xi, yi, zi) is the feature point, λ1 is the weight coefficient of the geometric constraint model, λ2 is the weight coefficient of the curvature constraint model, Γ α is the αth geometric constraint model, and Xi is the corrected three-dimensional coordinate of the target.
[0039] A third aspect of an embodiment of the present application provides a schematic diagram of a device for determining a feature point position, wherein:
[0040] a detection module, configured to perform feature point detection on the first two-dimensional image and the second two-dimensional image, and respectively determine the two-dimensional coordinates of each of the feature points in the first two-dimensional image and the second two-dimensional image;
[0041] A determination module, used to determine the initial three-dimensional coordinates of each of the feature points according to the two-dimensional coordinates of each of the feature points;
[0042] The correction module is used to correct the initial three-dimensional coordinates based on the surface shape of the tracking device to obtain the target three-dimensional coordinates of each feature point.
[0043] A third aspect of an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is the processor described in the first aspect above.
[0044] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the second aspect above is implemented.
[0045] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes the method described in the second aspect.
[0046] Compared with the prior art, the embodiments of the present application have the following advantages:
[0047] In the implant navigation system provided by the embodiment of the present application, the surface of the tracking device can be a complex curved surface, and the precise positioning of the feature points on the surface of the tracking device can be achieved during the positioning process. The implant navigation system may include a tracking device, a binocular positioning device, and a processor. The surface of the tracking device has a feature pattern. The binocular positioning device includes a left camera and a right camera. The left camera and the right camera can be used to collect a two-dimensional image of the tracking device. The collected two-dimensional image contains a feature pattern, and the feature pattern has multiple feature points. The processor can extract the feature points in the two-dimensional image and determine the two-dimensional coordinates of each feature point in the two-dimensional image respectively; according to the two-dimensional coordinates of each feature point, the initial three-dimensional coordinates of each feature point can be determined; then based on the geometric shape of the tracking device, the processor corrects the initial three-dimensional coordinates to obtain the target three-dimensional coordinates of each feature point. Through the above system, when locating the feature point, the three-dimensional coordinates can be determined first according to the two-dimensional coordinates of the feature point, and based on the geometric shape of the tracking device, the three-dimensional coordinates can be corrected so that the target three-dimensional coordinates of the feature point can meet the geometric shape constraints of the tracking device, thereby improving the positioning accuracy of the feature points on the complex curved surface and ensuring the safety of the operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.
[0049] Figure 1 is a schematic diagram of a tracking device provided in an embodiment of the present application;
[0050] Figure 2 It is a schematic diagram of the steps of a method for determining a feature point position provided in an embodiment of the present application;
[0051] Figure 3 is a schematic diagram of a method for determining three-dimensional coordinates of a target provided in an embodiment of the present application;
[0052] Figure 4 is a schematic diagram of a feature point position determination device provided in an embodiment of the present application;
[0053] Figure 5 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0054] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the present application.
[0055] Implant surgery requires precise control and positioning of the position and posture of implants or surgical tools in space. Existing navigation systems usually rely on external tracking devices and feature points arranged on the surface of the tracked object. However, conventional feature points such as spherical or dot markers are usually only distributed on regular planes, and cannot adapt well to devices with complex surface shapes (such as cylinders, cones, or other curved surfaces). That is, in the prior art, when the surface of the tracking device is a complex curved surface, the positioning of the tracking device is inaccurate, thereby affecting the safety of the operation.
[0056] In the existing implant navigation system, when the feature pattern is attached to a complex surface, the position of the feature point will be deformed due to the curvature and bending of the tracking device surface. Especially on surfaces with large curvature, the complex surface will cause the extracted feature point position to be inconsistent with its actual position in space, thus affecting the positioning accuracy.
[0057] In addition, during the imaging process, feature points on devices with complex surfaces are affected by perspective projection and optical distortion, resulting in obvious nonlinear distortion in the two-dimensional image. When the feature points are far away from the camera or at a large angle, the distortion is more significant, which increases the difficulty of extracting feature points.
[0058] In addition, evenly arranging feature points on a complex surface (such as a cylinder) requires accurate calculation of their distribution positions in three-dimensional space, and stable imaging in two-dimensional images and meeting the geometric distribution requirements of feature points. The difficulty of this arrangement increases exponentially with the increase in surface complexity.
[0059] In the prior art, most methods for locating surface feature points are mainly based on simple image processing methods and geometric feature extraction algorithms, such as directly extracting corner points or line segment intersections in an image. These methods can achieve good results when applied to feature point positioning on regular planes, but on complex surfaces, affected by curvature, projection deformation and noise, feature points will produce large deformations and deviations, resulting in the extracted feature points being unable to correspond to their actual spatial positions. Therefore, there is a need for an optimization method that can accurately extract feature points on complex surfaces and introduce geometric deformations and constraints to improve positioning accuracy.
[0060] Based on this, an embodiment of the present application provides a planting navigation system, which can achieve accurate positioning of feature points.
[0061] The technical solution of the present application is described below through specific embodiments.
[0062] Figure 1 FIG. 1 is a schematic diagram of a tracking device provided by an embodiment of the present application. Figure 1 As shown, the surface of the tracking device may be a plane, a cylindrical surface or a conical surface. In order to achieve stable and accurate detection and tracking of the tracking device by the implant navigation system in a complex environment, a characteristic pattern may be attached to the surface of the tracking device. The characteristic pattern can be used to identify and locate the tracking device.
[0063] Feature patterns can include many types. Common surface features can include edges, intersections, dots, annular dots and edge features, etc. Feature patterns can be evenly distributed and cover key areas of the surface. At the same time, the influence of surface curvature on the distribution of feature points must be considered during design to ensure the uniformity of the distribution of feature points in three-dimensional space. Since the center positioning of circular and annular features depends on contour extraction, their performance on the curved surface is easily affected by distortion, so the use of circular and annular features affects the positioning accuracy. Based on this, setting corner points as feature points in the embodiment of the present application can solve the problem of accurate detection and positioning of corner points on the surface of the tracking device, so as to improve the detection accuracy of feature points and enhance the overall performance of the implant navigation system.
[0064] like Figure 1 As shown, as an example, the characteristic pattern may be a black and white checkerboard pattern. The intersections of the lines in the black and white checkerboard pattern may be used as characteristic points. The surface of the tracking device may have a plurality of characteristic points.
[0065] The tracking device can be used in a planting navigation system. An embodiment of the present application provides a planting navigation system, which may include a tracking device, a binocular positioning device, and a processor. The surface of the planting navigation system tracking device has a characteristic pattern. The binocular positioning device of the planting navigation system includes a left camera and a right camera. The left camera of the planting navigation system is used to collect a first two-dimensional image of the tracking device, and the right camera of the planting navigation system is used to collect a second two-dimensional image of the tracking device. The first two-dimensional image and the second two-dimensional image are both images of the tracking device. Therefore, the first two-dimensional image and the second two-dimensional image may include a characteristic pattern, and the characteristic pattern includes multiple characteristic points.
[0066] The processor in the implant navigation system can extract and locate feature points from the first two-dimensional image and the second two-dimensional image. Figure 2 The method shown is used to determine the position information of each feature point.
[0067] Reference Figure 2 , shows a schematic flow chart of a method for determining a feature point position provided in an embodiment of the present application, which may specifically include the following steps:
[0068] S201 : Perform feature point detection on the first two-dimensional image and the second two-dimensional image, and respectively determine the two-dimensional coordinates of each feature point in the first two-dimensional image and the second two-dimensional image.
[0069] The execution subject of this embodiment may be the above-mentioned processor, and the processor may exist in a computer device. This embodiment does not limit the specific type of the computer device.
[0070] In order to facilitate the detection of feature points, in the embodiment of the present application, the feature points used can be corner points obtained by the intersection of lines. Figure 1 In the tracking device shown, the feature point can be the intersection corner point formed by the checkerboard pattern. The corner point feature is formed by the intersection of mutually perpendicular straight lines; on the curved surface, the corner point is not formed by a straight line, but a curve that is strongly coupled with the curvature of the curved surface, and the corner point also undergoes nonlinear deformation, which makes it difficult to directly apply traditional corner point extraction algorithms such as the Harris corner point detection algorithm. This embodiment uses the intersection corner point formed by the checkerboard pattern as the feature point to explain the solution.
[0071] The processor may perform feature point detection on the first two-dimensional image and the second two-dimensional image. As an example, an image processing algorithm, such as a sub-pixel corner point detection algorithm, may be deployed in the navigation system. Based on the image recognition algorithm, the pixel position of the feature point on the image may be preliminarily extracted and recorded as i=1,...,n, where n represents the number of corner points, n is a positive integer, L and R are used to represent the left camera and the right camera respectively. The processor can extract each feature point from the two-dimensional image, perform lens distortion correction on the extracted feature points, and obtain the ideal feature point position.
[0072]
[0073] in, is the two-dimensional coordinate of the i-th feature point, k1 is the image center of the first two-dimensional image captured by the left camera / the image center of the second two-dimensional image captured by the right camera, L / R ,k2 L / R ,P1 L / R ,P2 l / R are the lens distortion coefficients of the left camera and the right camera respectively. L / R ,k2 L / R is the radial lens distortion coefficient of the left camera / right camera, P1 L / R ,P2 L / Rare the tangential lens distortion coefficients of the left camera and the right camera respectively.
[0074] In the above formula, in order to simplify the description, the method for determining the two-dimensional coordinates of the feature points in the first two-dimensional image of the left camera and the method for determining the two-dimensional coordinates of the feature points in the second two-dimensional image of the right camera are combined for description. They are described separately below.
[0075] The formula for determining the two-dimensional coordinates of the feature points in the first two-dimensional image of the left camera can be:
[0076]
[0077] in, is the two-dimensional coordinate of the i-th feature point, is the image center of the first 2D image captured by the left camera, k1 L ,k2 L is the radial lens distortion coefficient of the left camera, P1 L ,P2 L are the tangential lens distortion coefficients of the right camera, respectively.
[0078] The formula for determining the two-dimensional coordinates of the feature points in the first two-dimensional image of the left camera can be:
[0079]
[0080] in, is the two-dimensional coordinate of the i-th feature point, is the image center of the second 2D image captured by the right camera, k1 R ,k2 R is the radial lens distortion coefficient of the right camera, P1 R ,P2 R are the tangential lens distortion coefficients of the right camera, respectively.
[0081] S202: Determine the initial three-dimensional coordinates of each of the feature points according to the two-dimensional coordinates of each of the feature points.
[0082] The left camera and the right camera may have camera parameters, which may include camera intrinsics and camera extrinsics, which are important parameters used in computer vision to describe camera characteristics and positions. Camera intrinsics may include focal length, principal point coordinates, distortion coefficients, etc. Camera intrinsics are usually determined when the camera leaves the factory and remain unchanged during the use of the camera. The calibration of intrinsics can be done by professional camera calibration tools or software to ensure its accuracy. The calibration process of intrinsics includes shooting a specific calibration pattern and calculating the precise intrinsic values through algorithms.
[0083] The camera extrinsic parameters describe the position and attitude of the camera in the world coordinate system. The camera extrinsic parameters can include the rotation matrix and the translation vector. The rotation matrix describes the rotation of the camera coordinate system relative to the world coordinate system. The translation vector describes the translation of the camera coordinate system relative to the world coordinate system. The camera extrinsic parameters will change as the camera moves or the shooting angle changes. For example, in a stereo vision application, if the relative position or direction of the two cameras changes, the extrinsic parameters also need to be updated accordingly. The calibration of the extrinsic parameters is usually obtained by measuring the imaging results of the camera at different positions and attitudes, and calculating them through algorithms.
[0084] In an embodiment of the present application, the camera may be calibrated to determine the camera intrinsic parameters and the camera extrinsic parameters, so as to perform coordinate calculation based on the camera intrinsic parameters and the camera extrinsic parameters.
[0085] The initial three-dimensional coordinates of each of the feature points are determined according to the two-dimensional coordinates of each of the feature points by the following formula:
[0086] P L / R =K L / R [R L / R t L / R ]X
[0087] Among them, K L / R is the intrinsic parameter matrix of the left camera / right camera, (R L / R , t L / R )) is the external parameter matrix of the left camera / right camera, X is the initial three-dimensional coordinate, P L / R It is the coordinate of the initial 3D coordinate projected on the imaging plane corresponding to the left camera / right camera.
[0088] The above formula is equivalent to two formulas that can determine the left camera and the right camera:
[0089] P L =K L [R L t LR ]X
[0090] P R =K R [R R t / R ]X
[0091] Based on the above two formulas, by solving X, we can get the initial three-dimensional coordinates.
[0092] S203, based on the surface shape of the tracking device, correct the initial three-dimensional coordinates to obtain target three-dimensional coordinates of each feature point.
[0093] The geometric shapes of the tracking device may include various shapes, such as a cuboid, a cylinder, a cone, etc. The tracking device may be a single geometric shape or may be a combination of multiple geometric shapes. Based on different geometric shapes, the tracking device may have different surfaces. Figure 1 As shown, the surface of the tracking device can be a plane, a cylindrical surface, or a conical surface.
[0094] Different surfaces of the tracking device may cause different distortions of the feature points. Therefore, the initial three-dimensional coordinates may be corrected based on the surface shape of the tracking device.
[0095] S301, determining geometric constraint models corresponding to a plurality of surface shapes.
[0096] If the surface of the tracking device is a plane, the plane normal vector of the plane where the surface of the tracking device is located is perpendicular to any straight line in the plane, and the geometric constraint model can be:
[0097]
[0098] in, is the plane normal vector of the plane where the surface of the tracking device is located; P0(x0, y0, z0) is any point on the plane where the surface of the tracking device is located, which does not coincide with the coordinates of all feature points; Pi(xi, yi, zi) is the feature point. Among them, the plane normal vector can be estimated by the least squares method from each feature point on the plane. Generally, the origin and XY axis of the plane coordinate system are both located in the plane, so z i = z0 = 0. Based on P0 and Pi, a unique straight line can be determined, which is perpendicular to the plane normal vector, so the product is 0
[0099] If the surface shape of the tracking device is a cylinder, the feature point can be located on the edge of a circle. Based on the coordinate point features of the points on the edge of the circle, the geometric constraint model can be:
[0100] Γ2:x i 2 +y i 2 -r 2 =0,i=1,...,n
[0101] Among them, Pi (xi, yi, zi) is the feature point, r is the radius of the cylinder corresponding to the tracking device, and (xi, yi) is the Cartesian coordinate of the feature point on the surface of the cylinder. Generally, it is assumed that the axis direction of the cylinder is the Z axis direction.
[0102] If the surface shape of the tracking device is a cone, the geometric constraint model can be determined based on the cone surface equation as follows:
[0103]
[0104] Among them, Pi (xi, yi, zi) is the characteristic point; assuming that the vertex of the cone is the origin, the opening is along the Z axis, and k is a constant used to characterize the cone opening angle corresponding to the tracking device, which can be calculated from the cone opening angle. For example, the opening angle θ usually refers to the angle between the axis and the side of the cone, and k = tan (θ).
[0105] S302: Determine a correction model function based on each of the geometric constraint models and the curvature constraint model, wherein the curvature constraint model is used to constrain the curvature of each of the feature points to remain consistent.
[0106] In addition to the geometric modeling constraints of each surface, considering the curvature consistency of the corner points in space, a second-order smoothness constraint is introduced to characterize the curvature characteristics. i-1 ,P i ,P i+1 , the curvature constraint model can be defined as:
[0107] C curvature =‖P i-1 -2P i +P i+1 ‖2
[0108] Among them, P i-1 ,P i ,P i+1 are three consecutive feature points.
[0109] When correcting the three-dimensional coordinates, a correction model function can be determined based on the geometric constraint model and the curvature constraint model. In the correction model function, different weights can be given to the geometric constraint model and the curvature constraint model.
[0110] As an example, the modified model function is:
[0111]
[0112] Among them, Pi (xi, yi, zi) is the feature point, λ1 is the weight coefficient of the geometric constraint model, λ2 is the weight coefficient of the curvature constraint model, Γ α is the αth geometric constraint model, Xi is the corrected target 3D coordinate, and the weight coefficients λ1λ2 can be adjusted through experiments to obtain the best effect.
[0113] S303: Correct the initial three-dimensional coordinates according to the correction model function to obtain the target three-dimensional coordinates.
[0114] Based on the above correction model function, the initial three-dimensional coordinates of each feature point can be corrected. The above correction model function is a value function of nonlinear multi-constraint conditions, which can be iteratively optimized through the Levenberg-Marquardt algorithm to minimize the value function and finally obtain the high-precision three-dimensional coordinates Xi of the corner point.
[0115] After determining the target three-dimensional coordinates of each feature point, the implant navigation system can determine the spatial position and posture of the tracking device, thereby assisting the smooth progress of the implant surgery.
[0116] In the embodiment of the present application, in order to ensure the accuracy of the position of the feature points, a feature pattern can be attached to the surface of the tracking device, and the checkerboard corner points or other corner points in the feature pattern can be used as feature points, so as to facilitate the identification and positioning of the feature points. In the embodiment of the present application, the initial feature points can be extracted from the two-dimensional image by an image processing method and lens distortion correction can be performed to ensure the accuracy of the two-dimensional coordinates of the feature points; the initial three-dimensional spatial coordinates of the feature points are obtained based on the triangulation positioning principle of the binocular implant navigation system; a variety of constraints are introduced to correct the corner point position in combination with the tracker surface geometry modeling and surface characteristics; the deformation constraint conditions can ensure that the corner point position conforms to the geometric model of the device surface, such as a cylindrical surface model, a cone surface model, etc.; the output of the precisely positioned three-dimensional feature point set is used in the implant navigation system.
[0117] The surface feature pattern design and feature point precise positioning method of the tracking device for implant navigation tracking provided in the embodiment of the present application can effectively improve the positioning accuracy of feature points on complex curved surfaces. Compared with the traditional planar corner point extraction method, the method in the embodiment of the present application introduces surface model constraints and multiple deformation constraints, which can adapt to more complex surface shapes and reduce corner point extraction errors caused by factors such as curvature and projection deformation.
[0118] The feature point design and precise positioning method of the embodiment of the present application can be applied to the navigation system of various implant surgeries to improve the positioning accuracy and stability of implants or surgical tools in space, thereby improving the success rate and safety of the surgery.
[0119] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0120] Reference Figure 4 , shows a schematic diagram of a device for determining a feature point position provided in an embodiment of the present application, which may specifically include a detection module 41, a determination module 42 and a correction module 43, wherein:
[0121] A detection module 41, configured to perform feature point detection on the first two-dimensional image and the second two-dimensional image, and respectively determine the two-dimensional coordinates of each feature point in the first two-dimensional image and the second two-dimensional image;
[0122] A determination module 42, configured to determine the initial three-dimensional coordinates of each of the feature points according to the two-dimensional coordinates of each of the feature points;
[0123] The correction module 43 is used to correct the initial three-dimensional coordinates based on the surface shape of the tracking device to obtain the target three-dimensional coordinates of each feature point.
[0124] In a possible implementation manner, the two-dimensional coordinates of each of the feature points in the first two-dimensional image and the second two-dimensional image are determined by the following formula:
[0125]
[0126]
[0127] in, is the two-dimensional coordinate of the i-th feature point, k1 is the coordinate of the center point of the camera imaging plane of the left camera / the right camera, L / R ,k2 L / R is the radial lens distortion coefficient of the left camera / the right camera, P1 L / R ,P2 L / R are the tangential lens distortion coefficients of the left camera and the right camera respectively.
[0128] In a possible implementation, it is characterized in that the initial three-dimensional coordinates of each of the feature points are determined according to the two-dimensional coordinates of each of the feature points by the following formula:
[0129] P L / R =K L / R [R l / R t L / R ]X
[0130] Among them, K L / R is the intrinsic parameter matrix of the left camera / the right camera, (R L / R ,t L / R )) is the external parameter matrix of the left camera / the right camera, X is the initial three-dimensional coordinate, P L / R It is the coordinates of the initial three-dimensional coordinates projected onto the camera imaging plane corresponding to the left camera / the camera imaging plane corresponding to the right camera.
[0131] In a possible implementation, the initial three-dimensional coordinates are corrected based on the surface shape of the tracking device to obtain the target three-dimensional coordinates of each feature point, including:
[0132] Determining a plurality of geometric constraint models corresponding to the surface shapes;
[0133] Determining a correction model function based on each of the geometric constraint models and the curvature constraint model, wherein the curvature constraint model is used to constrain the curvature of each of the feature points to remain consistent;
[0134] The initial three-dimensional coordinates are corrected according to the correction model function to obtain the target three-dimensional coordinates.
[0135] In a possible implementation, if the surface shape of the tracking device is a plane, the geometric constraint model is:
[0136]
[0137] in, is the plane normal vector of the plane where the surface of the tracking device is located, P0(x0, y0, z0) is any point on the plane where the surface of the tracking device is located, and Pi(xi, yi, zi) is the three-dimensional coordinate of the feature point.
[0138] In a possible implementation, if the surface shape of the tracking device is a cylindrical surface, the geometric constraint model is:
[0139] Γ2:x i 2 +y i 2 -r 2 =0,i=1,...,n
[0140] Among them, Pi (xi, yi, zi) is the three-dimensional coordinate of the feature point, and r is the radius of the cylinder corresponding to the tracking device.
[0141] In a possible implementation, if the surface shape of the tracking device is a conical surface, the geometric constraint model is:
[0142]
[0143] Among them, Pi (xi, yi, zi) is the three-dimensional coordinate of the feature point, and k is a constant used to characterize the cone opening angle corresponding to the tracking device.
[0144] In a possible implementation, the curvature constraint model is:
[0145] C curvature =‖Pi-1 -2P i +P i+1 ‖2
[0146] Among them, P i-1 ,P i ,P i+1 are three consecutive feature points.
[0147] In a possible implementation, the modified model function is:
[0148]
[0149] Wherein, Pi(xi, yi, zi) is the feature point, λ1 is the weight coefficient of the geometric constraint model, λ2 is the weight coefficient of the curvature constraint model, Γ α is the αth geometric constraint model, and Xi is the corrected three-dimensional coordinate of the target.
[0150] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.
[0151] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 5 As shown, the computer device 5 of this embodiment includes: at least one processor 50 ( Figure 5 Only one is shown in the figure), a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, wherein the processor 50 implements the steps of any of the above-mentioned method embodiments when executing the computer program 52.
[0152] The computer device 5 may be a desktop computer, a notebook computer, a PDA, a cloud computing device, etc. The computer device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of the computer device 5 and does not constitute a limitation on the computer device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.
[0153] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0154] In some embodiments, the memory 51 may be an internal storage unit of the computer device 5, such as a hard disk or memory of the computer device 5. In other embodiments, the memory 51 may also be an external storage device of the computer device 5, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 5. Further, the memory 51 may also include both an internal storage unit of the computer device 5 and an external storage device. The memory 51 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 51 may also be used to temporarily store data that has been output or is to be output.
[0155] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0156] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer device.
[0157] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A planting navigation system, characterized in that: The invention comprises a tracking device, a binocular positioning device and a processor, wherein the surface of the tracking device has a characteristic pattern, the binocular positioning device comprises a left camera and a right camera, the left camera is used to collect a first two-dimensional image of the tracking device, the right camera is used to collect a second two-dimensional image of the tracking device, the first two-dimensional image and the second two-dimensional image include the characteristic pattern, and the characteristic pattern includes a plurality of characteristic points; the processor is used to: Performing feature point detection on the first two-dimensional image and the second two-dimensional image, and respectively determining the two-dimensional coordinates of each of the feature points in the first two-dimensional image and the second two-dimensional image; Determine the initial three-dimensional coordinates of each of the feature points according to the two-dimensional coordinates of each of the feature points; Based on the surface shape of the tracking device, the initial three-dimensional coordinates are corrected to obtain the target three-dimensional coordinates of each feature point.
2. The system according to claim 1, characterized in that The two-dimensional coordinates of each of the feature points in the first two-dimensional image and the second two-dimensional image are determined by the following formula: in, is the two-dimensional coordinate of the i-th feature point, k1 is the coordinate of the center point of the camera imaging plane of the left camera / the right camera, L / R ,k2 L / R is the radial lens distortion coefficient of the left camera / the right camera, P1 L / R ,P2 L / R are the tangential lens distortion coefficients of the left camera and the right camera respectively.
3. The system according to claim 1, characterized in that The initial three-dimensional coordinates of each feature point are determined according to the two-dimensional coordinates of each feature point by the following formula: P L / R =K L / R [R l / R t L / R ]X Among them, K l / R is the intrinsic parameter matrix of the left camera / the right camera, (R L / R ,t L / R ) is the external parameter matrix of the left camera / the right camera, X is the initial three-dimensional coordinate, P L / R It is the coordinates of the initial three-dimensional coordinates projected onto the camera imaging plane corresponding to the left camera / the camera imaging plane corresponding to the right camera.
4. The system according to any one of claims 1 to 3, characterized in that: The step of correcting the initial three-dimensional coordinates based on the surface shape of the tracking device to obtain the target three-dimensional coordinates of each feature point includes: Determining a plurality of geometric constraint models corresponding to the surface shapes; Determining a correction model function based on each of the geometric constraint models and the curvature constraint model, wherein the curvature constraint model is used to constrain the curvature of each of the feature points to remain consistent; The initial three-dimensional coordinates are corrected according to the correction model function to obtain the target three-dimensional coordinates.
5. The system according to claim 4, characterized in that If the surface shape of the tracking device is a plane, the geometric constraint model is: in, is the plane normal vector of the plane where the surface of the tracking device is located, P0(x0, y0, z0) is any point on the plane where the surface of the tracking device is located, and Pi(xi, yi, zi) is the three-dimensional coordinate of the feature point.
6. The system according to claim 4, characterized in that If the surface shape of the tracking device is a cylindrical surface, the geometric constraint model is: Γ2:x i 2 +y i 2 -r 2 =0,i=1,...,n Among them, Pi (xi, yi, zi) is the three-dimensional coordinate of the feature point, and r is the radius of the cylinder corresponding to the tracking device.
7. The system according to claim 4, characterized in that If the surface shape of the tracking device is a conical surface, the geometric constraint model is: Among them, Pi (xi, yi, zi) is the three-dimensional coordinate of the feature point, and k is a constant used to characterize the cone opening angle corresponding to the tracking device.
8. The system according to claim 4, characterized in that The curvature constraint model is: C curvature =‖P i-1 -2P i +P i+1 ‖2 Among them, P i-1 ,P i ,P i+1 are three consecutive feature points.
9. The system according to any one of claims 5 to 8, characterized in that: The modified model function is: Wherein, Pi(xi, yi, zi) is the feature point, λ1 is the weight coefficient of the geometric constraint model, λ2 is the weight coefficient of the curvature constraint model, Γ α is the αth geometric constraint model, and Xi is the corrected three-dimensional coordinate of the target.
10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor is the processor according to any one of claims 1 to 9.