A method, system and storage medium for calibrating the relative position of a navigation plate and teeth in dental surgery navigation

Abstract

The present invention belongs to the field of surgical navigation technology, and specifically relates to a method, system and storage medium for calibrating the relative position of a navigation board and teeth in dental surgery. The method of the present invention comprises the following steps: obtaining CT data of the patient's teeth; converting the CT data into a point cloud; after the patient installs the navigation board before surgery, collecting images of the patient's teeth and the navigation board; segmenting the image using a deep learning model, segmenting the tooth image and the navigation board image, and calculating the contour of the tooth model using an edge detection operator; obtaining the position and posture of the navigation board in the world coordinate system using a two-dimensional code positioning algorithm; obtaining the position and posture of the point cloud in the world coordinate system through a registration algorithm; and obtaining the relative position and posture relationship between the navigation board and the teeth through a transformation matrix calculation. The present invention further provides a system for implementing the above method. The present invention has the advantages of simple operation, low cost and high accuracy, and has good application prospects.

Description

Technical Field

[0001] The invention belongs to the technical field of surgical operation navigation, and in particular relates to a relative position calibration method, system and storage medium for a navigation plate and teeth in dental operation navigation. Background Art

[0002] In modern medicine, surgical navigation can improve surgical accuracy, reduce surgical difficulty, and reduce patient pain. It is a very important technical field and plays an important role in the field of medical and health care. Existing technologies such as CN109864820A and CN209996471U have proposed a universal surgical navigation system and method, which is based on CT three-dimensional model mapping to achieve positioning navigation, can very intuitively obtain mixed reality fusion, play the effect of seeing through the human body, can safely and effectively avoid risks, and bring better prospects for use. The possibility of further expansion and improvement should be explored, especially in dynamic oral implant surgery scenarios.

[0003] In dynamic oral implant surgery navigation, optical sensors or electromagnetic sensors are traditionally used as sensors. These sensors determine the position of surgical tools by tracking specific marker points. Since these sensors cannot directly observe the surgical position, a dedicated radioactive baffle is required for auxiliary positioning. The baffle is rigidly connected to the patient's alveolar bone. After the patient wears the baffle to take a CBCT, the baffle shape is matched with the jaw relationship through software. During the operation, under the observation of the sensor, the relative position relationship between the jaw, the baffle, and the navigation board is established by clicking the baffle shape marker points with special tools. After the sensor determines the position of the navigation board, the position of the patient's alveolar bone can be calculated by the relative position of the navigation board and the teeth. The determination of the relative position of the navigation board and the patient's alveolar bone after the rigid connection is the calibration method of surgical navigation. This relative position directly affects the accuracy of the operation and is very important.

[0004] In the prior art, the main steps of the relative position calibration method between the navigation plate and the teeth include:

[0005] 1. Preoperative evaluation: Assess the patient's dental condition and alveolar bone structure.

[0006] 2. Make a personalized radiation barrier: Design and make a special radiation barrier according to the patient's oral structure.

[0007] 3. Positioning of the radiation shield: Fix the radiation shield to the patient's alveolar bone and ensure its stability.

[0008] 4. CT scan: After the radiation shield is fixed, a CT scan is performed to obtain three-dimensional image data of the patient's oral cavity.

[0009] 5. Data processing and calibration: Use software to process CT data and calibrate the relative position of the radiation shield and teeth.

[0010] 6. Intraoperative calibration: Fix the navigation board in the mouth, and under the observation of the sensor, use special tools to select the external markers of the radiation-blocking board to establish the positional relationship between the radiation-blocking board and the navigation board, thereby establishing the positional relationship between the navigation board and the jawbone.

[0011] The main problems of existing calibration methods are:

[0012] 1. After the radiation shield is fixed to the patient's alveolar bone, the patient needs to take a CT scan, which makes the operation time-consuming and expensive;

[0013] 2. After obtaining the CT scan, manual calibration is required based on the CT scan before surgery, resulting in low accuracy and cumbersome operations.

[0014] In addition, it is also possible to directly shoot CBCT without using a radiation blocking plate, design the dental anatomical landmarks for registration before the operation, fix the navigation plate during the operation, and use special tools to select the dental anatomical landmarks for registration in sequence under sensor observation to establish the positional relationship between the navigation plate and the jaw. However, this method also has the problems of limited result accuracy and cumbersome operation.

[0015] Artificial intelligence technology has been widely used in fields such as medical image processing. If artificial intelligence technology can be applied to the relative position calibration of navigation plates and teeth, it will effectively reduce manual operations, improve the accuracy of calibration results, make the method simpler, and reduce technical sensitivity and learning costs. At present, there is still a lack of relevant technical solutions for the data processing process in the relative position calibration task of navigation plates and teeth. Summary of the invention

[0016] In view of the problems in the prior art, the present invention provides a method, system and storage medium for calibrating the relative position of a navigation plate and teeth.

[0017] The present invention provides a surgical navigation system for calibrating the relative position of a navigation plate and teeth in dental surgery, and is characterized by comprising:

[0018] a camera for acquiring visible light images of the patient's teeth and the navigation plate;

[0019] An input component, used for inputting CT data of a patient's teeth;

[0020] A calculation component, used to obtain the relative position and posture relationship between the navigation plate and the teeth;

[0021] The computing components include:

[0022] A tooth contour detection module, used to segment the visible light image using a deep learning model, segment the tooth image and the navigation plate image, and calculate the tooth contour using an edge detection operator;

[0023] The world coordinate acquisition module is used to obtain the position and posture of the navigation board in the world coordinate system by using the QR code positioning algorithm, and obtain the position and posture of the point cloud in the world coordinate system by using the registration algorithm;

[0024] The coordinate conversion module is used to calculate the relative position and posture between the navigation plate and the teeth through the transformation matrix.

[0025] Preferably, the camera is a high-precision visible light camera.

[0026] Preferably, the deep learning model is selected from U-Net, U2-Net, U-Net++, SegNet or DeepLabv3+;

[0027] And / or, the edge detection operator is selected from a Canny operator, a Laplacian operator, a Sobel operator or a Roberts operator.

[0028] The present invention also provides a method for calibrating the relative position of a navigation plate and teeth, comprising the following steps:

[0029] Step 1, obtaining CT data of a patient's teeth; converting the CT data into a point cloud;

[0030] Step 2: After the patient has the navigation plate installed before surgery, images of the patient's teeth and the navigation plate are collected;

[0031] Step 3, segmenting the image using a deep learning model to segment the tooth image and the navigation plate image, and calculating the contour of the tooth model using an edge detection operator;

[0032] Step 4: Use the QR code positioning algorithm to obtain the position and posture of the navigation board in the world coordinate system; use the registration algorithm to obtain the position and posture of the point cloud in the world coordinate system;

[0033] Step 5: Obtain the relative position and posture relationship between the navigation plate and the teeth through transformation matrix calculation.

[0034] Preferably, in step 2, a high-precision visible light camera is used to capture images of the patient's teeth and the navigation plate.

[0035] Preferably, in step 3, the algorithm of the deep learning model is selected from U-Net, U2-Net, U-Net++, SegNet or DeepLabv3+;

[0036] And / or, the edge detection operator is selected from a Canny operator, a Laplacian operator, a Sobel operator or a Roberts operator.

[0037] Preferably, in step 4, the two-dimensional code positioning algorithm is selected from estimatePoseSingleMarkers, pnp or epipolar geometry;

[0038] And / or, the registration algorithm is selected from a registration method based on deep learning, a three-dimensional registration method based on ICP, or a registration algorithm based on key point features.

[0039] Preferably, in step 4, the process of the two-dimensional code positioning algorithm includes:

[0040] Step a, perform camera calibration to obtain the camera's intrinsic parameter matrix and distortion parameters;

[0041] Step b, measuring the actual size of the QR code on the navigation board;

[0042] Step c, extracting feature points from the two-dimensional code image;

[0043] Step d, call the estimatePoseSingleMarkers function in the OpenCV library, input the image coordinates of the feature points, the corresponding world coordinates, the camera intrinsic parameter matrix and the distortion parameters, and calculate the rotation vector and translation vector;

[0044] Step e: Convert the rotation vector into a rotation matrix and combine it with the translation vector to obtain the position and attitude of the navigation board in the world coordinate system.

[0045] Preferably, in step 4, the registration algorithm is based on a registration algorithm of key points and image features, and the process includes:

[0046] Step A: Project the point cloud onto the imaging plane to obtain the projection image of the point cloud, and use the random numbers generated by numpy to initialize the position and posture of the point cloud;

[0047] Step B, coarse registration stage: use Gaussian variation as the driver of the algorithm, use the mean square error loss between the segmented image and the projected image and the IOU Loss of the boungding box as a guide to move the point cloud to the corresponding spatial area;

[0048] Step C, refined registration stage: fine-tune the position and posture of the point cloud until the loss reaches below 0.01, and output the position and posture of the point cloud.

[0049] The present invention also provides a computer-readable storage medium, on which is stored: a computer program for implementing the relative position calibration method between the navigation plate and the teeth, or a computer program for implementing the system.

[0050] The present invention aims at the purpose of calibrating the relative position of the navigation plate and the teeth, and constructs a technical solution combining deep learning and a three-stage automatic registration algorithm. The original data required by the scheme include the CT data of the patient's teeth and the image obtained by scanning with a camera before the operation. Since the patient's teeth will be scanned by CT during the diagnosis stage, the method of the present invention does not require the patient to take additional CT data before the operation. Instead, the existing CT data and the image taken by the camera (which is very easy to obtain) are aligned, which can effectively reduce the patient's radiation dose and economic expenditure. In addition, there is no process of manually processing data in the technical solution of the present invention. Therefore, the present invention has the advantages of simple operation and reduced surgical costs. In addition, since the manual operation steps are avoided, the calibration method of the present invention has good accuracy. In summary, the calibration method of the present invention has a good application prospect.

[0051] Obviously, according to the above contents of the present invention, in accordance with common technical knowledge and customary means in the art, without departing from the above basic technical ideas of the present invention, other various forms of modification, replacement or change may be made.

[0052] The above contents of the present invention are further described in detail below through specific implementation methods in the form of embodiments. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following examples. All technologies realized based on the above contents of the present invention belong to the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a schematic diagram of the navigation panel in Example 1.

[0054] Figure 2 Schematic diagram of image segmentation by the image segmentation network in step 3 of Example 1.

[0055] Figure 3 This is a schematic diagram of the image after feature extraction in step 3 of Example 1.

[0056] Figure 4 This is the rough alignment of the position and attitude in step 4 of Example 1. Wherein, (a) is the rough alignment of the spatial position, and (b) is the rough alignment of the spatial attitude.

[0057] Figure 5 This is a schematic diagram of the refined registration and automatic registration results in step 4 of Example 1. DETAILED DESCRIPTION

[0058] It should be noted that the algorithms of data collection, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structure, circuit connection, etc. not specifically described can all be implemented through the contents disclosed in the prior art.

[0059] Embodiment 1 Method for calibrating the relative position between the navigation plate and the teeth

[0060] This embodiment includes the following steps:

[0061] Step 1, obtain CT data of the patient's teeth; convert the CT data into a point cloud; it should be noted that this step can directly use the CT data taken during the patient's diagnosis, so there is no need to take a CT scan again before the operation.

[0062] Step 2: Patients are fitted with a navigation board before surgery (e.g. Figure 1 After that, a high-precision visible light camera is used to collect images of the patient's teeth and the navigation plate; the navigation plate has a camera QR code that can be recognized by visible light;

[0063] Step 3: Use the deep learning model to segment the tooth image and navigation plate image, and use the edge detection operator to extract the teeth for the automated registration in step 4. The segmented image example is as follows: Figure 2 As shown, the example after segmentation and feature extraction is as follows Figure 3 shown.

[0064] The deep learning model refers to the use of deep learning technology to segment images, including but not limited to: U-Net, U2-Net, U-Net++, SegNet, DeepLabv3+, etc. Edge detection operators include but are not limited to: canny operator, Laplacian operator, Sobel operator, Roberts operator, etc. In this embodiment, the algorithm of the deep learning model is the U-Net image segmentation network, and the edge detection operator selects the canny operator of OpenCV.

[0065] Step 4: First, use the QR code positioning algorithm to obtain the position and posture of the navigation board in the world coordinate system. Then, take the segmented image and point cloud as input and enter the three-stage registration algorithm to output the position and posture of the point cloud in the world coordinate system.

[0066] The two-dimensional code positioning algorithm includes but is not limited to: estimatePoseSingleMarkers, pnp, and epipolar geometry. The registration algorithm includes but is not limited to: a registration method based on deep learning, a three-dimensional registration method based on ICP, a registration algorithm based on key point features, etc. In this embodiment, the two-dimensional code positioning algorithm selects estimatePoseSingleMarkers. Specifically, the algorithm flow of the two-dimensional code positioning is as follows:

[0067] First, the camera is calibrated to obtain the camera's intrinsic matrix and distortion parameters, which are the basis for subsequent calculations. Then, the actual size of the QR code on the navigation board is measured to locate it in the world coordinate system, usually assuming that the QR code is a plane with a z-axis coordinate of 0. Next, feature points, such as corner points, are extracted from the QR code image, and these image coordinate points correspond to points in the world coordinate system. By calling the estimatePoseSingleMarkers function in the OpenCV library and inputting the image coordinates of the feature points, the corresponding world coordinates, the camera intrinsic matrix, and the distortion parameters, the algorithm will calculate the rotation vector and translation vector. Finally, the rotation vector is converted into a rotation matrix and combined with the translation vector to obtain the three-dimensional spatial position of the QR code relative to the camera (i.e., the position and posture of the navigation board in the world coordinate system). This series of steps provides accurate data support for further image processing and robot navigation.

[0068] The three-stage registration algorithm is as follows: First, the initialization stage. In this stage, a virtual camera needs to be constructed to simulate a real camera, and the point cloud is projected onto the imaging plane to obtain a projection image of the point cloud. In this stage, the random numbers generated by numpy are also used to initialize the position and posture of the point cloud. In the coarse registration stage, Gaussian variation is used as the driver of the algorithm, and the mean square error loss between the segmented image and the projected image and the IOULoss (Intersection over Union Loss) of the boungding box of the two are used as guidance to move the point cloud to the corresponding spatial area. An example of this step is shown in Figure 4 As shown. Next, we enter the fine-tuning phase to fine-tune the position and posture of the point cloud until the loss is below 0.01, and then the fine-tuning is completed. The algorithm will output the position and posture of the point cloud. Among them, "numpy" is a scientific computing library in the Python programming language, which is used for large-scale numerical calculations. "Bounding box" is a rectangular box used to define the boundaries of an object in image processing. An example of this step is shown in Figure 5 shown.

[0069] Step 5: After obtaining the position and posture of the navigation board and the point cloud, multiply the transformation matrices of the two to obtain the relative posture relationship between the navigation board and the point cloud model, that is, the relative posture relationship with the teeth. The final mean square error is less than 0.001.

[0070] Embodiment 2: Relative position calibration system between navigation plate and teeth

[0071] This embodiment provides a system for implementing the calibration method of embodiment 1, including:

[0072] The camera is used to collect images of the patient's teeth and the navigation plate. In this embodiment, the camera is a high-precision visible light camera.

[0073] An input module, used for inputting CT data of the patient's teeth, and inputting images of the patient's teeth and a navigation plate;

[0074] The calculation module is used to obtain the relationship between the relative position and posture between the navigation plate and the teeth according to the relative position calibration method between the navigation plate and the teeth in Example 1.

[0075] Through the above embodiments, it can be seen that the present invention combines deep learning, QR code positioning algorithm and registration algorithm, and can use CT data without navigation board and preoperative images of teeth and navigation board to calculate the relative position of navigation board and teeth. The present invention has the advantages of simple operation, low cost and high accuracy, and has good application prospects.

Claims

1. A surgical navigation system for calibrating the relative position of a navigation plate and teeth in dental surgery, characterized in that: include: a camera for acquiring visible light images of the patient's teeth and the navigation plate; An input component, used for inputting CT data of a patient's teeth; A calculation component, used to obtain the relative position and posture relationship between the navigation plate and the teeth; The computing components include: A tooth contour detection module, used to segment the visible light image using a deep learning model, segment the tooth image and the navigation plate image, and calculate the tooth contour using an edge detection operator; The world coordinate acquisition module is used to obtain the position and posture of the navigation board in the world coordinate system by using the QR code positioning algorithm, and obtain the position and posture of the point cloud in the world coordinate system by using the registration algorithm; A coordinate conversion module is used to calculate the relative position and posture between the navigation plate and the teeth through a transformation matrix; The process of the two-dimensional code positioning algorithm includes: Step a, perform camera calibration to obtain the camera's intrinsic parameter matrix and distortion parameters; Step b, measuring the actual size of the QR code on the navigation board; Step c, extracting feature points from the two-dimensional code image; Step d, call the estimatePoseSingleMarkers function in the OpenCV library, input the image coordinates of the feature points, the corresponding world coordinates, the camera intrinsic parameter matrix and the distortion parameters, and calculate the rotation vector and translation vector; Step e, converting the rotation vector into a rotation matrix, and combining it with the translation vector to obtain the position and attitude of the navigation board in the world coordinate system; The registration algorithm is a registration algorithm based on key points and image features, and the process includes: Step A: Project the point cloud onto the imaging plane to obtain the projection image of the point cloud, and use the random numbers generated by numpy to initialize the position and posture of the point cloud; Step B, coarse registration stage: use Gaussian variation as the driver of the algorithm, use the mean square error loss between the segmented image and the projected image and the IOU Loss of the boungding box as a guide to move the point cloud to the corresponding spatial area; Step C, refined registration stage: fine-tune the position and posture of the point cloud until the loss reaches below 0.01, and output the position and posture of the point cloud.

2. The surgical navigation system of claim 1, wherein the camera is a high-precision visible light camera.

3. The surgical navigation system according to claim 1, wherein the deep learning model is selected from U-Net, U2-Net, U-Net++, SegNet or DeepLabv3+; And / or, the edge detection operator is selected from a Canny operator, a Laplacian operator, a Sobel operator or a Roberts operator.

4. A method for calibrating the relative position of a navigation plate and teeth in dental surgery navigation, characterized in that: The steps include: Step 1, obtaining CT data of a patient's teeth; converting the CT data into a point cloud; Step 2: After the patient has the navigation plate installed before surgery, visible light images of the patient's teeth and the navigation plate are collected; Step 3, using a deep learning model to segment the visible light image, segmenting the tooth image and the navigation plate image, and using an edge detection operator to calculate the contour of the tooth model; Step 4: Use the QR code positioning algorithm to obtain the position and posture of the navigation board in the world coordinate system; The position and posture of the point cloud in the world coordinate system are obtained through the registration algorithm; Step 5, obtaining the relative position and posture relationship between the navigation plate and the teeth through transformation matrix calculation; In step 4, the process of the two-dimensional code positioning algorithm includes: Step a, perform camera calibration to obtain the camera's intrinsic parameter matrix and distortion parameters; Step b, measuring the actual size of the QR code on the navigation board; Step c, extracting feature points from the two-dimensional code image; Step d, call the estimatePoseSingleMarkers function in the OpenCV library, input the image coordinates of the feature points, the corresponding world coordinates, the camera intrinsic parameter matrix and the distortion parameters, and calculate the rotation vector and translation vector; Step e, converting the rotation vector into a rotation matrix, and combining it with the translation vector to obtain the position and attitude of the navigation board in the world coordinate system; The registration algorithm is a registration algorithm based on key points and image features, and the process includes: Step A: Project the point cloud onto the imaging plane to obtain the projection image of the point cloud, and use the random numbers generated by numpy to initialize the position and posture of the point cloud; Step B, coarse registration stage: use Gaussian variation as the driver of the algorithm, use the mean square error loss between the segmented image and the projected image and the IOU Loss of the boungding box as a guide to move the point cloud to the corresponding spatial area; Step C, refined registration stage: fine-tune the position and posture of the point cloud until the loss reaches below 0.01, and output the position and posture of the point cloud.

5. The method for calibrating the relative position of the navigation plate and teeth according to claim 4, characterized in that: In step 2, a high-precision visible light camera is used to capture images of the patient's teeth and the navigation plate.

6. The method for calibrating the relative position of the navigation plate and teeth according to claim 4, characterized in that: In step 3, the algorithm of the deep learning model is selected from U-Net, U2-Net, U-Net++, SegNet or DeepLabv3+; And / or, the edge detection operator is selected from a Canny operator, a Laplacian operator, a Sobel operator or a Roberts operator.

7. The method for calibrating the relative position of the navigation plate and teeth according to claim 4, characterized in that: In step 4, the two-dimensional code positioning algorithm is selected from estimatePoseSingleMarkers, pnp or epipolar geometry; And / or, the registration algorithm is selected from a registration method based on deep learning, a three-dimensional registration method based on ICP, or a registration algorithm based on key point features.

8. A computer-readable storage medium, characterized in that: Stored thereon are: a computer program for implementing the relative position calibration method of the navigation plate and the teeth in dental surgery navigation as described in any one of claims 4-7, or a computer program for implementing the surgical navigation system as described in any one of claims 1-3.

Images