Tooth oral cavity endoscope two-dimensional image and CBCT three-dimensional model image registration method

Abstract

The invention discloses a registration method for a tooth and oral cavity endoscope two-dimensional image and a CBCT (cone beam computed tomography) three-dimensional model image, relates to the technical field of image registration, and aims at the tooth and oral cavity endoscope two-dimensional image and the CBCT three-dimensional model image. Image registration is carried out by using real tooth edge contour points in the oral cavity endoscope two-dimensional image and three-dimensional model projection contour points as registration feature points; according to the technical key points, an oral cavity internal three-dimensional model is obtained through oral cavity optical scanning, and tooth gingival margin line feature information is extracted; a YOLOv5s target detection algorithm and a Canny edge detection algorithm are combined to carry out target tooth identification and segmentation and edge contour extraction on a first frame image input by an oral cavity endoscope, so that the image registration efficiency is improved; contour point set registration is carried out in a one-to-many mode through a coherent point drift algorithm, and the rotation angle around each coordinate axis and a registration transformation matrix which should be applied when registration of the three-dimensional model is achieved are rapidly obtained; according to the method, the three-dimensional model is subjected to dimensionality reduction projection, so that the dimensionality of the image to be registered is unified, the real tooth edge contour points in the two-dimensional image of the oral cavity endoscope and the projection contour points of the three-dimensional model are taken as the registration feature points, and the efficiency and accuracy of image registration can be improved.

Description

Technical Field

[0001] The patent of this invention relates to a method for aligning a two-dimensional image of a dental oral endoscope with a three-dimensional CBCT model image, and belongs to the field of image alignment technology. Background Art

[0002] Image registration technology is widely used in medical imaging, computer vision, augmented reality and other fields, and the accuracy and efficiency of image registration have become crucial. However, in the field of oral surgery, there is still no good solution for the registration of oral endoscope images with three-dimensional models reconstructed from CBCT images. Moreover, because the dental endoscope is a monocular camera, only two-dimensional images of the affected teeth can be obtained. Because the two-dimensional images of the affected teeth cannot provide effective scene depth information, the two-dimensional images of the affected teeth only have planar features such as corners and edges, while the three-dimensional volume data model reconstructed from the CBCT image has spatial features such as curvature and inflection points. The difference in feature dimensions makes it difficult to realize image registration between oral endoscope images and three-dimensional models reconstructed from CBCT images. The two registration objects must first be unified in one dimension for matching; in order to achieve "dimensionality upgrading" of oral endoscope images, it is necessary to use the oral endoscope to simulate the human eye to take images from different angles for depth of field reconstruction, but during oral surgery, the oral endoscope is fixed in a certain position from the beginning of the operation, which makes it impossible to obtain images from two different angles; compared with the "dimensionality upgrading" method, the "dimensionality reduction" processing has lower computing power requirements, making the entire registration process less time-consuming, and because the contour data contains clear and accurate feature points, the registration accuracy can also be guaranteed; in summary, the current field of image registration technology urgently needs a method that can accurately align the two-dimensional image of the dental oral endoscope with the three-dimensional model reconstructed based on the CBCT image. Summary of the Invention

[0003] To address the above problems, the present invention proposes a method for aligning a dental oral endoscope two-dimensional image with a CBCT three-dimensional model image, which solves the problem in the current field of image registration technology that there is a lack of image alignment methods for oral endoscope two-dimensional images and three-dimensional models reconstructed based on CBCT images, and realizes accurate image alignment between oral endoscope two-dimensional images and CBCT three-dimensional models.

[0004] A method for registering a dental oral endoscope two-dimensional image with a CBCT three-dimensional model image is characterized in that the specific implementation process of the method is as follows:

[0005] Step 1: 3D reconstruction of the target tooth, target tooth root canal, and target tooth's left adjacent tooth CBCT images:

[0006] Mimics medical imaging control software was used to segment the target tooth, target tooth root canal, and target tooth left adjacent teeth in the patient's oral CBCT images in DICOM format based on the region growing algorithm. The segmented target tooth, target tooth root canal, and target tooth left adjacent teeth CBCT images were reconstructed three-dimensionally based on the isosurface extraction algorithm to obtain the target tooth three-dimensional model M. t , target tooth root canal 3D model M r And the 3D model M of the adjacent tooth on the left side of the target tooth tl ; The target tooth 3D model M t , target tooth root canal 3D model M r And the 3D model M of the adjacent tooth on the left side of the target tooth tl Assemble in Solidworks software and place the target tooth 3D model M t The transparency is adjusted to 60% to show the target tooth root canal 3D model M contained inside. r After the assembly is completed, the three-dimensional model M with root canal information reconstructed based on the CBCT image is obtained. a ;

[0007] Step 2: Extract feature information of the target tooth and the adjacent gingival margin on the left side of the target tooth:

[0008] The intraoral three-dimensional model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth and the crown information is obtained by oral optical scanning. o ; Based on the three-dimensional model of the oral cavity M o The right boundary of the target tooth, the left boundary of the adjacent tooth on the left side of the target tooth, and the gingival margin line of the target tooth and the adjacent tooth on the left side of the target tooth are used to establish a segmentation line, and the three-dimensional model M of the oral cavity is constructed. o Segmentation is performed to obtain a three-dimensional crown model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth c ; Using ICP algorithm, the crown three-dimensional model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth is transformed into c Registered to the 3D model M with root canal information reconstructed from the CBCT image a In the coordinate system of the two models, the three-dimensional model M with root canal information reconstructed based on the CBCT image is realized. a The initial pose in the coordinate space of remains consistent;

[0009] Step 3: 3D model of the crown with gingival margin feature information M c Projection contour point set extraction:

[0010] A three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth cThe center point is the origin, the right is the positive direction of the x-axis, the upward is the positive direction of the y-axis, and according to the right-hand rule, the three-dimensional coordinate system w is established perpendicular to the screen and outward is the positive direction of the z-axis. m Under the premise of ensuring that the real target tooth appears in the field of view of the oral endoscope, the oral endoscope is positioned in the three-dimensional coordinate system w in the oral space. m The range of rotation around the x-axis at the origin is Around the three-dimensional coordinate system w m The rotation range of the y-axis is Around the three-dimensional coordinate system w m The rotation range of the z-axis is The three-dimensional coordinate system w of the intraoral camera in the oral space m The rotation angle range around the x-axis, y-axis and z-axis at the origin of the tooth is fed back to the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth in a three-layer nested loop manner. c , as a three-dimensional model of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w m The rotation angle range of the x-axis, y-axis and z-axis is set to plus 1° for each layer and each cycle. Each rotation is relative to the crown 3D model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth. c The initial pose is used; the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is converted using the Visualization Toolkit. c Set the color to white and the background of the space to black; set the shooting direction of the virtual camera along the three-dimensional coordinate system w m The negative direction of the z-axis is fixed, and each time the three-dimensional crown model M with the gingival margin line information of the target tooth and the adjacent teeth on the left side of the target tooth is c After the rotation is completed, the parallel projection is used to obtain the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth in the current posture. c The two-dimensional projection image I i,j,k , using the color space contour extraction algorithm in the two-dimensional projection image I i,j,k The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c Projected contour line at the current rotation angle The projected contour line at the current rotation angle Convert to the projection contour point set P at the current rotation angle i,j,k , P i,j,k The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system wm The projected contour point set obtained when the x-axis rotation angle is i°, the y-axis rotation angle is j°, and the z-axis rotation angle is k°, where the value range of i is The value range of j is The value range of k is The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w m The lth contour point in the projected contour point set obtained when the x-axis rotation angle is i°, the y-axis rotation angle is j° and the z-axis rotation angle is k° is relative to the three-dimensional coordinate system w m The position information of the xoy plane, the value range of l is 1≤l≤n, where, is the lth contour point in the three-dimensional coordinate system w m The x-axis coordinate in the xoy plane, is the lth contour point in the three-dimensional coordinate system w m The y-axis coordinate in the xoy plane; the three-dimensional model of the crown with gingival margin information M c The specific process of projected contour point set extraction is as follows:

[0011] a) Jump to step 3b);

[0012] b) Judgment Whether it is established, specifically:

[0013] like If established, jump to step 3c);

[0014] like If not, jump to step 3m);

[0015] c) Jump to step 3d);

[0016] d) Judgment Whether it is established, specifically:

[0017] like If established, jump to step 3e);

[0018] like If not, jump to step 31);

[0019] e) Jump to step 3f);

[0020] f) Judgment Whether it is established, specifically:

[0021] like If established, jump to step 3g);

[0022] like If not, jump to step 3k);

[0023] g) The three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is c Initialized as a 3D model M with root canal information reconstructed from CBCT images a Initial pose in the coordinate space; after initialization, jump to step 3h);

[0024] h) A three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c In the three-dimensional coordinate system w m Rotate by i° around the x-axis, j° around the y-axis, and k° around the z-axis; after the rotation is complete, jump to step 3i);

[0025] i) Obtain a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c In the three-dimensional coordinate system w m The projection contour line when rotating i° around the x-axis, j° around the y-axis, and k° around the z-axis Projecting the contours Convert to projection contour point set P i,j,k ; After acquisition, jump to step 3j);

[0026] j) k=k+1, jump to step 3f);

[0027] k) j = j + 1, jump to step 3 d);

[0028] 1) i=i+1, jump to step 3b);

[0029] m) The cycle ends, and the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth has been obtained. c The projected contour point set at all rotation angles;

[0030] Step 4: Rapid extraction of the true tooth edge contour point set in the oral endoscope 2D image:

[0031] The first frame image I input by the oral endoscope f Capture the first frame image I f The input is used to identify the target tooth and the adjacent teeth on the left side of the target tooth in the endoscopic two-dimensional image; the YOLOv5s target detection algorithm will input the first frame image I fGenerate a target bounding box for the target tooth and the left adjacent tooth of the target tooth, and use the first frame image I as the input f The upper left corner is the origin, the right is the positive direction of the x-axis, and the downward is the positive direction of the y-axis. The two-dimensional target detection coordinate system w is established. d , output the target bounding box in the two-dimensional target detection coordinate system w d The upper left corner coordinates and lower right corner coordinates in the target tooth target bounding box are in the two-dimensional target detection coordinate system w d The coordinate of the upper left corner is P tl (x tl ,y tl ), the target tooth target bounding box is in the two-dimensional target detection coordinate system w d The coordinates of the lower right corner are P tr (x tr ,y tr ); the target tooth's left adjacent tooth target bounding box is in the two-dimensional target detection coordinate system w d The coordinate of the upper left corner is P al (x al ,y al ), the target tooth’s left neighboring tooth’s target bounding box is in the two-dimensional target detection coordinate system w d The coordinates of the lower right corner are P ar (x ar ,y ar ); In the two-dimensional target detection coordinate system w d Make a line through the point (min(x tl ,x al ),0) and perpendicular to the x-axis dividing line l1, in the two-dimensional target detection coordinate system w d Make a line through the point (max(x tr ,x ar ),0) and perpendicular to the x-axis dividing line l2, in the two-dimensional target detection coordinate system w d Make a line through the point (0, min (y tl ,y al )) and perpendicular to the y-axis dividing line l3, in the two-dimensional target detection coordinate system w d Make a line through the point (0, max(y tr ,y ar )) and perpendicular to the y-axis dividing line l4, with dividing lines l1, l2, l3 and l4 as dividing borders in the first frame image I f The image I containing the target tooth and the adjacent teeth on the left side of the target tooth is segmented s ; Use the Canny edge detection algorithm to segment the image I containing the target tooth and the adjacent teeth on the left side of the target tooth s Perform edge detection to segment the image I containing the target tooth and the adjacent teeth on the left side of the target tooth.s The lower left corner is the origin, the right is the positive direction of the x-axis, and the upward is the positive direction of the y-axis. The two-dimensional edge detection coordinate system w is established. e ; Extract the tooth contour point set of the target tooth and the adjacent tooth on the left side of the target tooth The mth contour point in the extracted target tooth and the adjacent tooth on the left of the target tooth is relative to the two-dimensional edge detection coordinate system w e The position information of m is in the range of 1≤m≤p, where For the mth contour point in the two-dimensional edge detection coordinate system w e The x-axis coordinate in For the mth contour point in the two-dimensional edge detection coordinate system w e The y-axis coordinate in;

[0032] Step 5: Use the coherent point drift algorithm to perform contour point set registration in a one-to-many manner:

[0033] The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope are transformed into the target tooth and the left adjacent tooth contour point set P using the coherent point drift algorithm. Ie The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c The projected contour point set P at all rotation angles i,j,k The rigid registration is performed in a one-to-many form, where the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera are Ie The three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth remains unchanged as a fixed point set during the registration process. c The projected contour point set P i,j,k The position transformation is performed as a moving point set to complete the registration. After each registration is completed, a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is obtained. c The projected contour point set P i,j,k The corresponding two-dimensional registration transformation matrix The two-dimensional registration transformation matrix Convert to 3D registration transformation matrix And the 3D registration transformation matrix Stored in the three-dimensional registration transformation matrix dataset T, in:

[0034]

[0035] The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth after rigid registration using the K nearest neighbor method c The projected contour point set P i,j,kIn the process, we search for the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope. Ie Middle contour point matching points Specifically: contour points The query point is set to 1, and the parameter K is set to 1. That is, after the rigid registration is completed, the crown 3D model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is c The projected contour point set P i,j,k Finding and querying points The closest point to the Euclidean distance point That is, the contour points of the two-dimensional image contour curve Matching points; define the contour points of the two-dimensional image contour curve with matching points The distance between the contour points is the distance between the adjacent points, and the symbol d m express,

[0036] Calculate the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth at each rotation angle posture c The projected contour point set P i,j,k The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera Ie The rigid registration error e i,j,k ,in:

[0037]

[0038] The rigid registration error e obtained after each registration is i,j,k Stored in the registration error dataset E, The specific process of contour point set registration in a one-to-many form using the coherent point drift algorithm is as follows:

[0039] a) Jump to step 5b);

[0040] b) Judgment Whether it is established, specifically:

[0041] like If established, jump to step 5c);

[0042] like If not, jump to step 5m);

[0043] c) Jump to step 5d);

[0044] d) Judgment Whether it is established, specifically:

[0045] like If established, jump to step 5e);

[0046] like If not, jump to step 51);

[0047] e) Jump to step 5f);

[0048] f) Judgment Whether it is established, specifically:

[0049] like If established, jump to step 5g);

[0050] like If not, jump to step 5k);

[0051] g) The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera Ie The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c Projection contour point set P i,j,k Perform rigid registration; after registration is completed, jump to step 5h);

[0052] h) After the registration is completed, a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c The projected contour point set P i,j,k The corresponding two-dimensional registration transformation matrix The two-dimensional registration transformation matrix Convert to 3D registration transformation matrix And the 3D registration transformation matrix Store in the 3D registration transformation matrix data set T; after acquisition, jump to step 5i)

[0053] i) The three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth after rigid registration using the K nearest neighbor method c The projected contour point set P i,j,k In the process, we search for the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope. Ie Middle contour point matching points Calculate the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c The projected contour point set P i,j,k The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera IeThe rigid registration error e i,j,k , and the rigid registration error e i,j,k Store in the registration error data set E; after point set registration is completed, jump to step 5j);

[0054] j) k=k+1, jump to step 5f);

[0055] k) j = j + 1, jump to step 5 d);

[0056] 1) i=i+1, jump to step 5b);

[0057] m) The cycle ends, and the three-dimensional crown model M of all the target teeth and the gingival margin information of the adjacent teeth on the left side of the target teeth has been obtained. c The projected contour point set P i,j,k The corresponding three-dimensional registration transformation matrix and the rigid registration error e i,j,k ;

[0058] Step 6: Registration of dental endoscope 2D images with CBCT 3D model images:

[0059] Get the minimum value in the registration error dataset E, The corresponding i, j, k values are the i, j, k values corresponding to the minimum registration error; the three-dimensional model M with root canal information reconstructed based on the CBCT image is a In the three-dimensional coordinate system w m Rotate i° around the x-axis, j° around the y-axis, and k° around the z-axis, and then transform the corresponding three-dimensional registration transformation matrix into Applied to the 3D model M with root canal information reconstructed from CBCT images a , the image registration process is completed.

[0060] The beneficial effects of the present invention are:

[0061] 1. Based on the image characteristics of two-dimensional images from an oral endoscope and three-dimensional models reconstructed from CBCT images, the present invention proposes for the first time an image registration method applicable to two-dimensional images from an oral endoscope and three-dimensional models reconstructed from CBCT images. In addition, this image registration method is also applicable to the image registration of two-dimensional images and three-dimensional models in other fields.

[0062] 2. When performing image registration, the present invention proposes a registration method for performing dimensionality reduction projection on the three-dimensional model to unify the dimensions of the registered image.

[0063] 3. The present invention proposes to use the YOLOv5s target detection algorithm and the Canny edge detection algorithm to quickly extract the real tooth edge contour point set in the two-dimensional image of the oral endoscope.

[0064] 4. The present invention proposes to use the coherent point drift algorithm to extract the target tooth and the target tooth's left adjacent tooth contour point set P from the oral endoscope two-dimensional image in a one-to-many manner. Ie The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c The projected contour point set P at all rotation angles i,j,k A method for contour point set registration is proposed. By finding the projection pose of the 3D model that matches the real teeth in the 2D image of the oral endoscope, the correct spatial pose of the 3D model is determined to achieve the registration process. This method improves the efficiency and accuracy of the image registration between the 2D image of the oral endoscope and the 3D model reconstructed based on the CBCT image. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] For ease of explanation, the present invention is described in detail with reference to the following specific implementations and accompanying drawings.

[0066] Figure 1 The three-dimensional model M of the crown with gingival margin information c Projection contour point set extraction flow chart;

[0067] Figure 2 Flowchart for contour point set registration in a one-to-many manner using the coherent point drift algorithm;

[0068] Figure 3 Schematic diagram of the three-dimensional model of the target tooth, the target tooth root canal, and the target tooth's left adjacent tooth reconstructed based on the CBCT image;

[0069] Figure 4 The three-dimensional model M of the tooth crown with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth c Schematic diagram;

[0070] Figure 5 The three-dimensional model M of the crown with gingival margin feature information c Schematic diagram of projected contour point set extraction;

[0071] Figure 6 The first frame image I is input by the YOLOv5s target detection algorithm in the oral endoscope camera f The image I containing the target tooth and the adjacent teeth on the left side of the target tooth is segmented s Schematic diagram;

[0072] Figure 7 The image I containing the target tooth and the adjacent teeth on the left side of the target tooth is segmented using the Canny edge detection algorithm. s Schematic diagram of edge contour extraction;

[0073] Figure 8Schematic diagram of contour point set registration using coherent point drift algorithm;

[0074] Figure 9 This is a schematic diagram of the registration effect between the dental oral endoscope 2D image and the CBCT 3D model image; DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described below through the specific embodiments shown in the accompanying drawings. However, it should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present invention.

[0076] Implementation Example 1: Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 、 Figure 9 As shown, this specific embodiment adopts the following technical solution: a method for registering a dental oral endoscope two-dimensional image with a CBCT three-dimensional model image, the specific implementation process of the method is as follows:

[0077] Step 1: 3D reconstruction of the target tooth, target tooth root canal, and target tooth's left adjacent tooth CBCT images:

[0078] Mimics medical imaging control software was used to segment the target tooth, target tooth root canal, and target tooth left adjacent teeth in the patient's oral CBCT images in DICOM format based on the region growing algorithm. The segmented target tooth, target tooth root canal, and target tooth left adjacent teeth CBCT images were reconstructed three-dimensionally based on the isosurface extraction algorithm to obtain the target tooth three-dimensional model M. t , target tooth root canal 3D model M r And the 3D model M of the adjacent tooth on the left side of the target tooth tl ; The target tooth 3D model M t , target tooth root canal 3D model M r And the 3D model M of the adjacent tooth on the left side of the target tooth tl Assemble in Solidworks software and place the target tooth 3D model M t The transparency is adjusted to 60% to show the target tooth root canal 3D model M contained inside. r After the assembly is completed, the three-dimensional model M with root canal information reconstructed based on the CBCT image is obtained. a ;

[0079] Step 2: Extract feature information of the target tooth and the adjacent gingival margin on the left side of the target tooth:

[0080] The intraoral three-dimensional model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth and the crown information is obtained by oral optical scanning. o ; Based on the three-dimensional model of the oral cavity M o The right boundary of the target tooth, the left boundary of the adjacent tooth on the left side of the target tooth, and the gingival margin line of the target tooth and the adjacent tooth on the left side of the target tooth are used to establish a segmentation line, and the three-dimensional model M of the oral cavity is constructed. o Segmentation is performed to obtain a three-dimensional crown model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth c ; Using ICP algorithm, the crown three-dimensional model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth is transformed into c Registered to the 3D model M with root canal information reconstructed from the CBCT image a In the coordinate system of the two models, the three-dimensional model M with root canal information reconstructed based on the CBCT image is realized. a The initial pose in the coordinate space of remains consistent;

[0081] Step 3: 3D model of the crown with gingival margin feature information M c Projection contour point set extraction:

[0082] A three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c The center point is the origin, the right is the positive direction of the x-axis, the upward is the positive direction of the y-axis, and according to the right-hand rule, the three-dimensional coordinate system w is established perpendicular to the screen and outward is the positive direction of the z-axis. m Under the premise of ensuring that the real target tooth appears in the field of view of the oral endoscope, the oral endoscope is positioned in the three-dimensional coordinate system w in the oral space. m The range of rotation around the x-axis at the origin is Around the three-dimensional coordinate system w m The rotation range of the y-axis is Around the three-dimensional coordinate system w m The rotation range of the z-axis is The three-dimensional coordinate system w of the intraoral camera in the oral space m The rotation angle range around the x-axis, y-axis and z-axis at the origin of the tooth is fed back to the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth in a three-layer nested loop manner. c , as a three-dimensional model of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w mThe rotation angle range of the x-axis, y-axis and z-axis is set to plus 1° for each layer and each cycle. Each rotation is relative to the crown 3D model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth. c The initial pose is used; the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is converted using the Visualization Toolkit. c Set the color to white and the background of the space to black; set the shooting direction of the virtual camera along the three-dimensional coordinate system w m The negative direction of the z-axis is fixed, and each time the three-dimensional crown model M with the gingival margin line information of the target tooth and the adjacent teeth on the left side of the target tooth is c After the rotation is completed, the parallel projection is used to obtain the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth in the current posture. c The two-dimensional projection image I i,j,k , using the color space contour extraction algorithm in the two-dimensional projection image I i,j,k The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c Projected contour line at the current rotation angle The projected contour line at the current rotation angle Convert to the projection contour point set P at the current rotation angle i,j,k , P i,j,k The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w m The projected contour point set obtained when the x-axis rotation angle is i°, the y-axis rotation angle is j°, and the z-axis rotation angle is k°, where the value range of i is The value range of j is The value range of k is The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w m The lth contour point in the projected contour point set obtained when the x-axis rotation angle is i°, the y-axis rotation angle is j° and the z-axis rotation angle is k° is relative to the three-dimensional coordinate system w m The position information of the xoy plane, the value range of l is 1≤l≤n, where, is the lth contour point in the three-dimensional coordinate system w m The x-axis coordinate in the xoy plane, is the lth contour point in the three-dimensional coordinate system w m The y-axis coordinate in the xoy plane; the three-dimensional model of the crown with gingival margin information Mc The specific process of projected contour point set extraction is as follows:

[0083] a) Jump to step 3b);

[0084] b) Judgment Whether it is established, specifically:

[0085] like If established, jump to step 3c);

[0086] like If not, jump to step 3m);

[0087] c) Jump to step 3d);

[0088] d) Judgment Whether it is established, specifically:

[0089] like If established, jump to step 3e);

[0090] like If not, jump to step 31);

[0091] e) Jump to step 3f);

[0092] f) Judgment Whether it is established, specifically:

[0093] like If established, jump to step 3g);

[0094] like If not, jump to step 3k);

[0095] g) The three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is c Initialized as a 3D model M with root canal information reconstructed from CBCT images a Initial pose in the coordinate space; after initialization, jump to step 3h);

[0096] h) A three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c In the three-dimensional coordinate system w m Rotate by i° around the x-axis, j° around the y-axis, and k° around the z-axis; after the rotation is complete, jump to step 3i);

[0097] i) Obtain a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth cIn the three-dimensional coordinate system w m The projection contour line when rotating i° around the x-axis, j° around the y-axis, and k° around the z-axis Projecting the contours Convert to projection contour point set P i,j,k ; After acquisition, jump to step 3j);

[0098] j) k=k+1, jump to step 3f);

[0099] k) j = j + 1, jump to step 3 d);

[0100] 1) i=i+1, jump to step 3b);

[0101] m) The cycle ends, and the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth has been obtained. c The projected contour point set at all rotation angles;

[0102] Step 4: Rapid extraction of the true tooth edge contour point set in the oral endoscope 2D image:

[0103] The first frame image I input by the oral endoscope f Capture the first frame image I f The input is used to identify the target tooth and the adjacent teeth on the left side of the target tooth in the endoscopic two-dimensional image; the YOLOv5s target detection algorithm will input the first frame image I f Generate a target bounding box for the target tooth and the left adjacent tooth of the target tooth, and use the first frame image I as the input f The upper left corner is the origin, the right is the positive direction of the x-axis, and the downward is the positive direction of the y-axis. The two-dimensional target detection coordinate system w is established. d , output the target bounding box in the two-dimensional target detection coordinate system w d The upper left corner coordinates and lower right corner coordinates in the target tooth target bounding box are in the two-dimensional target detection coordinate system w d The coordinate of the upper left corner is P tl (x tl ,y tl ), the target tooth target bounding box is in the two-dimensional target detection coordinate system w d The coordinates of the lower right corner are P tr (x tr ,y tr ); the target tooth's left adjacent tooth target bounding box is in the two-dimensional target detection coordinate system w d The coordinate of the upper left corner is P al (x al ,y al ), the target tooth’s left neighboring tooth’s target bounding box is in the two-dimensional target detection coordinate system wd The coordinates of the lower right corner are P ar (x ar ,y ar ); In the two-dimensional target detection coordinate system w d Make a line through the point (min(x tl ,x al ),0) and perpendicular to the x-axis dividing line l1, in the two-dimensional target detection coordinate system w d Make a line through the point (max(x tr ,x ar ),0) and perpendicular to the x-axis dividing line l2, in the two-dimensional target detection coordinate system w d Make a line through the point (0, min (y tl ,y al )) and perpendicular to the y-axis dividing line l3, in the two-dimensional target detection coordinate system w d Make a line through the point (0, max(y tr ,y ar )) and perpendicular to the y-axis dividing line l4, with dividing lines l1, l2, l3 and l4 as dividing borders in the first frame image I f The image I containing the target tooth and the adjacent teeth on the left side of the target tooth is segmented s ; Use the Canny edge detection algorithm to segment the image I containing the target tooth and the adjacent teeth on the left side of the target tooth s Perform edge detection to segment the image I containing the target tooth and the adjacent teeth on the left side of the target tooth. s The lower left corner is the origin, the right is the positive direction of the x-axis, and the upward is the positive direction of the y-axis. The two-dimensional edge detection coordinate system w is established. e ; Extract the tooth contour point set of the target tooth and the adjacent tooth on the left side of the target tooth The mth contour point in the extracted target tooth and the adjacent tooth on the left of the target tooth is relative to the two-dimensional edge detection coordinate system w e The position information of m is in the range of 1≤m≤p, where For the mth contour point in the two-dimensional edge detection coordinate system w e The x-axis coordinate in For the mth contour point in the two-dimensional edge detection coordinate system w e The y-axis coordinate in;

[0104] Step 5: Use the coherent point drift algorithm to perform contour point set registration in a one-to-many manner:

[0105] The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope are transformed into the target tooth and the left adjacent tooth contour point set P using the coherent point drift algorithm. IeThe three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c The projected contour point set P at all rotation angles i,j,k The rigid registration is performed in a one-to-many form, where the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera are Ie The three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth remains unchanged as a fixed point set during the registration process. c The projected contour point set P i,j,k The position transformation is performed as a moving point set to complete the registration. After each registration is completed, a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is obtained. c The projected contour point set P i,j,k The corresponding two-dimensional registration transformation matrix The two-dimensional registration transformation matrix Convert to 3D registration transformation matrix And the 3D registration transformation matrix Stored in the three-dimensional registration transformation matrix dataset T, in:

[0106]

[0107] The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth after rigid registration using the K nearest neighbor method c The projected contour point set P i,j,k In the process, we search for the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope. Ie Middle contour point matching points Specifically: contour points The query point is set to 1, and the parameter K is set to 1. That is, after the rigid registration is completed, the crown 3D model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is c The projected contour point set P i,j,k Finding and querying points The closest point to the Euclidean distance point That is, the contour points of the two-dimensional image contour curve Matching points; define the contour points of the two-dimensional image contour curve with matching points The distance between the contour points is the distance between the adjacent points, and the symbol d m express,

[0108] Calculate the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth at each rotation angle posture c The projected contour point set P i,j,k The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera Ie The rigid registration error e i,j,k ,in:

[0109]

[0110] The rigid registration error e obtained after each registration is i,j,k Stored in the registration error dataset E, The specific process of contour point set registration in a one-to-many form using the coherent point drift algorithm is as follows:

[0111] a) Jump to step 5b);

[0112] b) Judgment Whether it is established, specifically:

[0113] like If established, jump to step 5c);

[0114] like If not, jump to step 5m);

[0115] c) Jump to step 5d);

[0116] d) Judgment Whether it is established, specifically:

[0117] like If established, jump to step 5e);

[0118] like If not, jump to step 51);

[0119] e) Jump to step 5f);

[0120] f) Judgment Whether it is established, specifically:

[0121] like If established, jump to step 5g);

[0122] like If not, jump to step 5k);

[0123] g) The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera IeThe three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c Projection contour point set P i,j,k Perform rigid registration; after registration is completed, jump to step 5h);

[0124] h) After the registration is completed, a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c The projected contour point set P i,j,k The corresponding two-dimensional registration transformation matrix The two-dimensional registration transformation matrix Convert to 3D registration transformation matrix And the 3D registration transformation matrix Store in the 3D registration transformation matrix data set T; after acquisition, jump to step 5i)

[0125] i) The three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth after rigid registration using the K nearest neighbor method c The projected contour point set P i,j,k In the process, we search for the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope. Ie Middle contour point matching points Calculate the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c The projected contour point set P i,j,k The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera Ie The rigid registration error e i,j,k , and the rigid registration error e i,j,k Store in the registration error data set E; after point set registration is completed, jump to step 5j);

[0126] j) k=k+1, jump to step 5f);

[0127] k) j = j + 1, jump to step 5 d);

[0128] 1) i=i+1, jump to step 5b);

[0129] m) The cycle ends, and the three-dimensional crown model M of all the target teeth and the gingival margin information of the adjacent teeth on the left side of the target teeth has been obtained. c The projected contour point set P i,j,k The corresponding three-dimensional registration transformation matrix and the rigid registration error e i,j,k ;

[0130] Step 6: Registration of dental endoscope 2D images with CBCT 3D model images:

[0131] Get the minimum value in the registration error dataset E, The corresponding i, j, k values are the i, j, k values corresponding to the minimum registration error; the three-dimensional model M with root canal information reconstructed based on the CBCT image is a In the three-dimensional coordinate system w m Rotate i° around the x-axis, j° around the y-axis, and k° around the z-axis, and then transform the corresponding three-dimensional registration transformation matrix into Applied to the 3D model M with root canal information reconstructed from CBCT images a , the image registration process is completed.

Claims

1. A method for registering a dental intraoral camera 2D image with a CBCT 3D model image, characterized by: The specific implementation process of the method is as follows: Step 1: 3D reconstruction of the target tooth, target tooth root canal, and target tooth's left adjacent tooth CBCT images: Mimics medical imaging control software was used to segment the target tooth, target tooth root canal, and target tooth left adjacent teeth in the patient's oral CBCT images in DICOM format based on the region growing algorithm. The segmented target tooth, target tooth root canal, and target tooth left adjacent teeth CBCT images were reconstructed three-dimensionally based on the isosurface extraction algorithm to obtain the target tooth three-dimensional model M. t , target tooth root canal 3D model M r And the 3D model M of the adjacent tooth on the left side of the target tooth tl ; The target tooth 3D model M t , target tooth root canal 3D model M r And the 3D model M of the adjacent tooth on the left side of the target tooth tl Assemble in Solidworks software and place the target tooth 3D model M t The transparency is adjusted to 60% to show the target tooth root canal 3D model M contained inside. r After the assembly is completed, the three-dimensional model M with root canal information reconstructed based on the CBCT image is obtained. a ; Step 2: Extract feature information of the target tooth and the adjacent gingival margin on the left side of the target tooth: The intraoral three-dimensional model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth and the crown information is obtained by oral optical scanning. o ; Based on the three-dimensional model of the oral cavity M o The right boundary of the target tooth, the left boundary of the adjacent tooth on the left side of the target tooth, and the gingival margin line of the target tooth and the adjacent tooth on the left side of the target tooth are used to establish a segmentation line, and the three-dimensional model M of the oral cavity is constructed. o Segmentation is performed to obtain a three-dimensional crown model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth c ; Using ICP algorithm, the crown three-dimensional model M with the gingival margin feature information of the target tooth and the adjacent teeth on the left side of the target tooth is transformed into c Registered to the 3D model M with root canal information reconstructed from the CBCT image a In the coordinate system of the two models, the three-dimensional model M with root canal information reconstructed based on the CBCT image is realized. a The initial pose in the coordinate space of remains consistent; Step 3: 3D model of the crown with gingival margin feature information M c Projection contour point set extraction: A three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c The center point is the origin, the right is the positive direction of the x-axis, the upward is the positive direction of the y-axis, and according to the right-hand rule, the three-dimensional coordinate system w is established perpendicular to the screen and outward is the positive direction of the z-axis. m Under the premise of ensuring that the real target tooth appears in the field of view of the oral endoscope, the oral endoscope is positioned in the three-dimensional coordinate system w in the oral space. m The range of rotation around the x-axis at the origin is Around the three-dimensional coordinate system w m The rotation range of the y-axis is Around the three-dimensional coordinate system w m The rotation range of the z-axis is The three-dimensional coordinate system w of the intraoral camera in the oral space m The rotation angle range around the x-axis, y-axis and z-axis at the origin of the tooth is fed back to the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth in a three-layer nested loop manner. c , as a three-dimensional model of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w m The rotation angle range of the x-axis, y-axis and z-axis is set to plus 1° for each layer and each cycle. Each rotation is relative to the crown 3D model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth. c The initial pose is used; the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is converted using the Visualization Toolkit. c Set the color to white and the background of the space to black; set the shooting direction of the virtual camera along the three-dimensional coordinate system w m The negative direction of the z-axis is fixed, and each time the three-dimensional crown model M with the gingival margin line information of the target tooth and the adjacent teeth on the left side of the target tooth is c After the rotation is completed, the parallel projection is used to obtain the three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth in the current posture. c The two-dimensional projection image I i,j,k , using the color space contour extraction algorithm in the two-dimensional projection image I i,j,k The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c Projected contour line at the current rotation angle The projected contour line at the current rotation angle Convert to the projection contour point set P under the current rotation angle i,j,k , P i,j,k The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w m The projected contour point set obtained when the x-axis rotation angle is i°, the y-axis rotation angle is j, and the z-axis rotation angle is k°, where the value range of i is The value range of j is The value range of k is The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c Around the three-dimensional coordinate system w m The lth contour point in the projected contour point set obtained when the x-axis rotation angle is i°, the y-axis rotation angle is j, and the z-axis rotation angle is k° is relative to the three-dimensional coordinate system w m The position information of the xoy plane, the value range of l is 1≤l≤n, where, is the lth contour point in the three-dimensional coordinate system w m The x-axis coordinate in the xoy plane, is the lth contour point in the three-dimensional coordinate system w m The y-axis coordinate in the xoy plane; the three-dimensional model of the crown with gingival margin information M c The specific process of projected contour point set extraction is as follows: a) Jump to step 3b); b) Judgment Whether it is established, specifically: like If established, jump to step 3c); like If not, jump to step 3m); c) Jump to step 3d); d) Judgment Whether it is established, specifically: like If established, jump to step 3e); like If not, jump to step 31); e) Jump to step 3f); f) Judgment Whether it is established, specifically: like If established, jump to step 3g); like If not, jump to step 3k); g) The three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is c Initialized as a 3D model M with root canal information reconstructed from CBCT images a Initial pose in the coordinate space; after initialization, jump to step 3h); h) A three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c In the three-dimensional coordinate system w m Rotate by i° around the x-axis, j° around the y-axis, and k° around the z-axis; after the rotation is complete, jump to step 3i); i) Obtain a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c In the three-dimensional coordinate system w m The projection contour line when rotating i° around the x-axis, j° around the y-axis, and k around the z-axis Projecting the contours Convert to projection contour point set P i,j,k ; After acquisition, jump to step 3j); j) k=k+1, jump to step 3f); k) j = j + 1, jump to step 3 d); 1) i=i+1, jump to step 3b); m) The cycle ends, and the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth has been obtained. c The projected contour point set at all rotation angles; Step 4: Rapid extraction of the true tooth edge contour point set in the oral endoscope 2D image: The first frame image I input by the oral endoscope f Capture the first frame image I f The input is used to identify the target tooth and the adjacent teeth on the left side of the target tooth in the endoscopic two-dimensional image; the YOLOv5s target detection algorithm will input the first frame image I f Generate a target bounding box for the target tooth and the left adjacent tooth of the target tooth, and use the first frame image I as the input f The upper left corner is the origin, the right is the positive direction of the x-axis, and the downward is the positive direction of the y-axis. The two-dimensional target detection coordinate system w is established. d , output the target bounding box in the two-dimensional target detection coordinate system w d The upper left corner coordinates and lower right corner coordinates in the target tooth target bounding box are in the two-dimensional target detection coordinate system w d The upper left corner coordinate is Ptl(xtl,ytl), and the target tooth target bounding box is in the two-dimensional target detection coordinate system w d The coordinates of the lower right corner in the image are Ptr(xtr,ytr); the target tooth’s left neighboring tooth’s target bounding box is in the two-dimensional target detection coordinate system w d The upper left corner coordinate is Pal(xal,yal), and the target tooth’s left neighboring tooth’s target bounding box is in the two-dimensional target detection coordinate system w d The coordinates of the lower right corner are Par(xar,yar); in the two-dimensional target detection coordinate system w d Make a dividing line l1 passing through the point (min(xtl,xal),0) and perpendicular to the x-axis in the two-dimensional target detection coordinate system w d Make a dividing line l2 passing through the point (max(xtr,xar),0) and perpendicular to the x-axis in the two-dimensional target detection coordinate system w d Make a dividing line l3 passing through the point (0, min(ytl, yal)) and perpendicular to the y-axis in the two-dimensional target detection coordinate system w d Make a line through the point (0, max(y tr ,y ar )) and perpendicular to the y-axis dividing line l4, with dividing lines l1, l2, l3 and l4 as dividing borders in the first frame image I f The image I containing the target tooth and the adjacent teeth on the left side of the target tooth is segmented s ; Use the Canny edge detection algorithm to segment the image I containing the target tooth and the adjacent teeth on the left side of the target tooth s Perform edge detection to segment the image I containing the target tooth and the adjacent teeth on the left side of the target tooth. s The lower left corner is the origin, the right is the positive direction of the x-axis, and the upward is the positive direction of the y-axis. The two-dimensional edge detection coordinate system w is established. e ; Extract the tooth contour point set of the target tooth and the adjacent tooth on the left side of the target tooth The mth contour point in the extracted target tooth and the adjacent tooth on the left of the target tooth is relative to the two-dimensional edge detection coordinate system w e The position information of m is in the range of 1≤m≤p, where For the mth contour point in the two-dimensional edge detection coordinate system w e The x-axis coordinate in For the mth contour point in the two-dimensional edge detection coordinate system w e The y-axis coordinate in; Step 5: Use the coherent point drift algorithm to perform contour point set registration in a one-to-many manner: The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope are transformed into the target tooth and the left adjacent tooth contour point set P using the coherent point drift algorithm. Ie The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c The projected contour point set P at all rotation angles i,j,k The rigid registration is performed in a one-to-many form, where the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera are Ie The three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth remains unchanged as a fixed point set during the registration process. c The projected contour point set P i,j,k The position transformation is performed as a moving point set to complete the registration. After each registration is completed, a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is obtained. c The projected contour point set P i,j,k The corresponding two-dimensional registration transformation matrix The two-dimensional registration transformation matrix Convert to 3D registration transformation matrix And the 3D registration transformation matrix Stored in the three-dimensional registration transformation matrix dataset T, in: The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth after rigid registration using the K nearest neighbor method c The projected contour point set P i,j,k In the process, we search for the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope. Ie Middle contour point matching points Specifically: contour points The query point is set to 1, and the parameter K is set to 1. That is, after the rigid registration is completed, the crown 3D model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth is c The projected contour point set P i,j,k Finding and querying points The closest point to the Euclidean distance point That is, the contour points of the two-dimensional image contour curve Matching points; define the contour points of the two-dimensional image contour curve with matching points The distance between the contour points is the distance between the adjacent points, and the symbol d m express, Calculate the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth at each rotation angle posture c The projected contour point set P i,j,k The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera Ie The rigid registration error e i,j,k ,in: The rigid registration error e obtained after each registration is i,j,k Stored in the registration error dataset E, The specific process of contour point set registration in a one-to-many form using the coherent point drift algorithm is as follows: a) Jump to step 5b); b) Judgment Whether it is established, specifically: like If established, jump to step 5c); like If not, jump to step 5m); c) Jump to step 5d); d) Judgment Whether it is established, specifically: like If established, jump to step 5e); like If not, jump to step 51); e) Jump to step 5f); f) Judgment Whether it is established, specifically: like If established, jump to step 5g); like If not, jump to step 5k); g) The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera Ie The three-dimensional model M of the tooth crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c Projection contour point set P i,j,k Perform rigid registration; after registration is completed, jump to step 5h); h) After the registration is completed, a three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth is obtained. c The projected contour point set P i,j,k The corresponding two-dimensional registration transformation matrix The two-dimensional registration transformation matrix Convert to 3D registration transformation matrix And the 3D registration transformation matrix Store in the 3D registration transformation matrix data set T; after acquisition, jump to step 5i) i) The three-dimensional crown model M with the gingival margin information of the target tooth and the adjacent teeth on the left of the target tooth after rigid registration using the K nearest neighbor method c The projected contour point set P i,j,k In the process, we search for the target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the oral endoscope. Ie Middle contour point matching points Calculate the three-dimensional model M of the crown with the gingival margin information of the target tooth and the adjacent teeth on the left side of the target tooth c The projected contour point set P i,j,k The target tooth and the left adjacent tooth contour point set P extracted from the two-dimensional image of the intraoral camera Ie The rigid registration error e i,j,k , and the rigid registration error e i,j,k Store in the registration error data set E; after point set registration is completed, jump to step 5j); j) k=k+1, jump to step 5f); k) j = j + 1, jump to step 5 d); 1) i=i+1, jump to step 5b); m) The cycle ends, and the three-dimensional crown model M of all the target teeth and the gingival margin information of the adjacent teeth on the left side of the target teeth has been obtained. c The projected contour point set P i,j,k The corresponding three-dimensional registration transformation matrix and the rigid registration error e i,j,k ; Step 6: Registration of dental endoscope 2D images with CBCT 3D model images: Get the minimum value in the registration error dataset E, The corresponding i, j, k values are the i, j, k values corresponding to the minimum registration error; the three-dimensional model M with root canal information reconstructed based on the CBCT image is a In the three-dimensional coordinate system w m Rotate i° around the x-axis, j° around the y-axis, and k° around the z-axis, and then transform the corresponding three-dimensional registration transformation matrix into Applied to the 3D model M with root canal information reconstructed from CBCT images a , the image registration process is completed.

Images