Computer vision-based automatic extraction of anterior tooth feature parameters and evaluation system

Abstract

This invention discloses an automated anterior tooth feature parameter extraction method and evaluation system based on computer vision, comprising: acquiring and visualizing a full-mouth scan image of a patient to be processed; cropping the corresponding tooth portion image; using Gaussian blur to eliminate specular reflection; using color segmentation to set thresholds for the H, S, and V channels respectively; separating the teeth and gingiva in each tooth portion image and performing morphological processing on each; extracting the tooth and gingiva portion contours using contour extraction methods; further refining the segmentation accuracy using the Chan-Vese active contour model algorithm and the energy function within the tooth portion contour; extracting corner points using the Shi-Tomasi corner detection method and then clustering them; extracting other important evaluation feature points through curve fitting and extreme value finding methods; and finally calculating relevant objective indicators for tooth evaluation and automatically generating the corresponding tooth score, thus completing the automated anterior tooth evaluation.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and oral evaluation technology, and in particular relates to an automatic anterior tooth feature parameter extraction method and evaluation system based on computer vision. Background Technology

[0002] The anterior teeth occupy a prominent position in the oral cavity. Accurate and rapid assessment of these teeth provides dentists with a basis for developing more precise treatment plans, helping patients better understand their oral health and improving satisfaction. For example, in restorative treatment planning, dentists can use automated scoring results to quickly understand the patient's needs, select appropriate restorative options, or assist dental technicians in monitoring results and thus better aligning teeth. Therefore, accurate and rapid evaluation of the anterior teeth is crucial for dental diagnosis and restoration. Traditional anterior tooth evaluation mainly relies on the subjective experience of professional dentists, which suffers from high subjectivity, inconsistent standards, and low efficiency. Quantitative evaluation methods used in previous studies, such as manually measuring tooth characteristic parameters with measuring tools, are cumbersome and inefficient. In recent years, computer technology has been widely applied in the medical field, such as in medical image analysis and automated diagnosis. These technologies offer advantages such as high objectivity and automation, making intelligentization in the medical field possible. However, in the area of ​​automated dental assessment, most mainstream methods based on convolutional neural networks (CNNs) are still limited to recognizing and labeling standard oral models. In reality, human teeth are often varied in shape and arrangement, making it difficult to apply CNN methods in practice. Computer vision technology, on the other hand, can identify and extract data from real dental scans, resulting in more accurate automated scoring and better clinical application prospects. It allows for more objective and convenient evaluation of clinical efficacy and research. Currently, the objectivity of mainstream PES / WES scoring largely depends on the evaluator's technical sensitivity. Promoting the widespread application of this method will help achieve a unified objective evaluation standard worldwide.

[0003] Therefore, developing an automated anterior tooth evaluation method based on computer vision can overcome the limitations of existing evaluation methods, improve the objectivity, standardization and efficiency of the evaluation, and has important clinical application value. Summary of the Invention

[0004] The purpose of this invention is to address the problems of existing tooth evaluation methods and the challenges of automated anterior tooth evaluation based on oral scan images, and to provide an automatic anterior tooth feature parameter extraction method and evaluation system based on computer vision.

[0005] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution:

[0006] In a first aspect, the present invention provides an automatic anterior tooth feature parameter extraction method based on computer vision, which includes the following steps:

[0007] S1. Obtain a full-mouth scan image of the patient to be processed, and visualize the full-mouth scan image of the patient so that it is displayed in front of the user;

[0008] S2. When the user clicks on the tooth that needs to be scored, the corresponding tooth image is automatically extracted from the patient's full-mouth scan image based on the click location. Each tooth image includes the tooth part and the gum part.

[0009] S3. Apply Gaussian blur to each tooth portion of the image to eliminate specular reflections, resulting in a Gaussian blurred image;

[0010] S4. Based on the HSV value of the image after Gaussian blur, the color segmentation method is used to set thresholds for the H, S, and V channels respectively, and the teeth and gums in each tooth part image are separated, and each tooth part image obtains an initial tooth image and an initial gum image.

[0011] S5. Perform morphological processing on the initial tooth image and the initial gingival image respectively to obtain the morphologically processed tooth image and the morphologically processed gingival image.

[0012] S6. Use contour extraction methods to extract the tooth contour from the morphologically processed tooth image and the gingival contour from the morphologically processed gingival image, and add conditional rules based on user feedback to adjust the tooth contour and gingival contour.

[0013] S7. The final gingival contour and tooth bottom edge contour are obtained by the Chan-Vese active contour model algorithm. The gradient value of each pixel is calculated point by point on the final gingival contour and tooth bottom edge contour according to the corresponding contour edge curve. Combined with the actual position of each pixel in the Gaussian blurred image, the gingival papilla vertex and gingival vertex are obtained. The pixel with the smallest gradient value on the edge curve of the tooth bottom edge contour is taken as the boundary point on both sides of the tooth bottom. The intersection point of the tooth contour and the gingival contour and the tooth bottom boundary point are connected to obtain the boundary line between the teeth.

[0014] S8. Further refine and improve the accuracy of the tooth contour by using the energy function within the tooth contour, making the transition between adjacent edges of the teeth smoother;

[0015] S9. The Shi-Tomasi corner detection method is used to extract the corner points in the contour of the gingiva and the contour of the teeth respectively. The extracted corner points of the teeth are clustered using the K-means clustering algorithm to obtain the center point of the teeth after clustering. The extracted corner points of the gingiva are clustered using the K-means clustering algorithm to obtain the center point of the gingiva after clustering.

[0016] S10. Extract other important evaluation feature points in the tooth contour and gingival contour by curve fitting and finding extreme values, wherein the other important evaluation feature points include the highest and lowest points of the tooth and the extreme points of the tooth in the horizontal direction.

[0017] S11. The clustered center points of the teeth, the clustered center points of the gingiva, and other important evaluation feature points are used as final feature points. The coordinates of all final feature points are obtained. Based on the coordinates of the final feature points, the relevant objective indicators for tooth evaluation are calculated, and the automatic extraction of anterior tooth feature parameters based on computer vision is completed. The relevant objective indicators for tooth evaluation include the length and width of the teeth and the length-to-width ratio, the height of the gingival apex, and the ratio of the height of the gingival papilla to the length of the crown.

[0018] Based on the above scheme, each step can be implemented in the following preferred manner.

[0019] As a preferred embodiment of the first aspect above, in step S5, the morphological processing includes opening and closing operations, wherein the opening operation is used to remove small dots in the tooth image or gingival image, and the closing operation is used to fill holes in the tooth image or gingival image.

[0020] As a preferred embodiment of the first aspect, in step S6, the specific process of contour extraction using the contour extraction method is as follows: the initial tooth contour is extracted using Gaussian filtering and Canny edge detection, and then the initial tooth contour is enhanced and refined using image dilation and image skeletonization; the initial gingival contour is extracted using Gaussian filtering and Canny edge detection, and then the initial gingival contour is enhanced and refined using image dilation and image skeletonization.

[0021] As a preferred embodiment of the first aspect mentioned above, the specific process of obtaining the final gingival contour and the tooth base edge contour using the Chan-Vese active contour model algorithm in step S7 is as follows:

[0022] For the gingival contour, the gingival contour is used as the initial value of the Chan-Vese active contour model algorithm. In each iteration, an inner region and an outer region are formed with the gingival contour as the boundary. The gray mean of the inner region and the gray mean of the outer region are calculated respectively. The gradient descent method is used to minimize the difference between the gray mean of the inner region and the inner region, and to minimize the difference between the gray mean of the outer region and the outer region, so as to guide the updating of the gingival contour. The update amplitude is controlled to ensure that the update is only performed near the boundary of the gingival contour. The iteration is repeated continuously to gradually adjust the gingival contour until the preset maximum number of iterations is reached or the change amplitude of the gingival contour is less than the preset contour threshold, so as to obtain the final gingival contour.

[0023] For the tooth contour, the background portion at the bottom of the tooth is extracted from the morphologically processed tooth image and used as the initial value for the Chan-Vese active contour model algorithm. This process is repeated iteratively to obtain the bottom edge contour of the tooth.

[0024] As a preferred embodiment of the first aspect mentioned above, in step S8, the energy function The specific format is as follows:

[0025]

[0026] Among them, E penal E represents the penalty term for limiting the complexity of the contour; grad E represents the gradient term; color λ0, λ1, and λ2 represent the color change terms; λ0, λ1, and λ2 represent the contribution values ​​of each energy function term.

[0027] As a preferred embodiment of the first aspect, in step S11, the specific method for calculating the relevant objective indicators of tooth evaluation based on the coordinates of the final feature points is as follows: Taking the upper left corner of the Gaussian blurred image as the origin, the tangent direction of the gingival contour edge curve at any gingival vertex is taken as the x-axis, and the direction perpendicular to it is taken as the y-axis. The coordinate y-value of the highest point of the tooth is subtracted from the coordinate y-value of the lowest point of the tooth. The absolute value of the difference in y-values ​​is multiplied by the scale of the patient's full-mouth scan image to obtain the length value of the tooth. The coordinate x-value of the extreme point in the horizontal direction of the tooth is subtracted from the coordinate x-value of the tooth. The absolute value of the difference in y-values ​​of the tooth is multiplied by the scale of the patient's full-mouth scan image to obtain the width value of the tooth. The length value of the tooth is divided by the width value of the tooth to obtain the length-to-width ratio of the tooth. The coordinate y-value of the highest point of the gingiva is subtracted from the coordinate y-value of the lowest point of the gingiva. The absolute value of the difference in y-values ​​of the gingiva is multiplied by the scale of the patient's full-mouth scan image to obtain the height of the gingival vertex. The height of the gingival vertex is divided by the length value of the tooth to obtain the ratio of the height of the gingival papilla to the length of the crown.

[0028] Secondly, the present invention provides an automated anterior tooth evaluation system based on computer vision, comprising:

[0029] The data acquisition module is used to acquire mesial gingival papilla height, distal gingival papilla height, gingival origin level, gingival color, gingival contour, tooth shape, tooth color, tooth transparency, and tooth outline.

[0030] The evaluation module uses mesial gingival papilla height, distal gingival papilla height, gingival origin level, gingival color, and gingival contour as evaluation indicators to reflect the degree of gingival condition. It also uses tooth morphology, tooth color, tooth transparency, and tooth outline as evaluation indicators to reflect the degree of tooth condition. The module objectively calculates the characteristics of gingival color, height, and outline, as well as tooth color, height, transparency, and outline. It automatically generates corresponding tooth scores based on the specific differences between the gingival teeth and the contralateral corresponding teeth, thus completing the automated evaluation of anterior teeth.

[0031] In the computer vision-based automated anterior tooth evaluation system, the evaluation process for each evaluation indicator is as follows:

[0032] For mesial papilla height, the height difference between the current tooth's mesial papilla height and the contralateral tooth's mesial papilla height is taken as the first difference. The mesial papilla height is scored based on the first difference and the preset first scoring criteria.

[0033] Furthermore, if the absolute value of the first difference is greater than or equal to 0 and less than or equal to a1, then 2 points are awarded; if the absolute value of the first difference is greater than a1 and less than or equal to b1, then 1 point is awarded; otherwise, 0 points are awarded. Here, a1 and b1 represent the preset first threshold and second threshold, respectively.

[0034] For the distal gingival papilla height, the height difference between the distal gingival papilla height of the current tooth and the distal gingival papilla height of the contralateral tooth of the same name is used as the second difference. The distal gingival papilla height is scored based on the second difference and the preset second scoring criteria.

[0035] Furthermore, if the absolute value of the second difference is greater than or equal to 0 and less than or equal to a2, then 2 points are awarded; if the absolute value of the first difference is greater than a2 and less than or equal to b2, then 1 point is awarded; otherwise, 0 points are awarded. Here, a2 and b2 represent the preset third threshold and fourth threshold, respectively.

[0036] For the gingival origin level, the height difference between the gingival apex height of the current tooth and the gingival apex height of the contralateral tooth with the same name is used as the third difference. The gingival origin level is scored based on the third difference and the preset third scoring criteria.

[0037] Furthermore, if the absolute value of the third difference is greater than or equal to 0 and less than or equal to a3, then 2 points are awarded; if the absolute value of the third difference is greater than a3 and less than or equal to b3, then 1 point is awarded; otherwise, 0 points are awarded. Here, a3 and b3 represent the preset fifth threshold and sixth threshold, respectively.

[0038] For gingival color, a first processed image and a second processed image are obtained. The first processed image is a portion of the gingiva of the current tooth between the mesial and distal gingival papilla and within a preset distance above the gingival contour, cropped from the Gaussian blurred image. The second processed image is a portion of the gingiva of the contralateral tooth of the current tooth between the mesial and distal gingival papilla and within a preset distance above the gingival contour, cropped from the Gaussian blurred image. The difference between the RGB average values ​​of the gingival portions in the first and second processed images is taken as a fourth difference value. The gingival color is scored based on the fourth difference value and a preset fourth scoring standard.

[0039] Furthermore, if the absolute value of the fourth difference is greater than or equal to 0 and less than or equal to a4, then 2 points are awarded; if the absolute value of the fourth difference is greater than a4 and less than or equal to b4, then 1 point is awarded; otherwise, 0 points are awarded. Here, a4 and b4 represent the preset seventh threshold and eighth threshold, respectively.

[0040] For the gingival contour, the gingival contour of the current tooth between the mesial and distal gingival papilla is obtained as the first gingival contour. The gingival contour of the contralateral tooth of the same name between the mesial and distal gingival papilla is obtained as the second gingival contour. m points are taken on the first and second gingival contours respectively to calculate the length and relative center position of the gingival contour. The principal direction is obtained by using principal component analysis. The difference in length, the difference in relative center position, and the difference in principal direction of the first and second gingival contours are calculated. The weighted average of the three differences is taken as the final difference. The gingival contour is scored according to the final difference and the preset fifth scoring standard.

[0041] Furthermore, if the final difference is greater than or equal to 0 and less than or equal to a5, 2 points are awarded; if the final difference is greater than a5 and less than or equal to b5, 1 point is awarded; otherwise, 0 points are awarded. Here, a5 and b5 represent the preset ninth and tenth thresholds, respectively.

[0042] For tooth morphology, the length-to-width ratio of the current tooth and the length-to-width ratio of the corresponding tooth on the opposite side are calculated. The smaller of the two values ​​is divided by the larger value and the quotient is taken. The tooth morphology is scored based on this quotient and the preset sixth scoring standard.

[0043] Furthermore, if the quotient is greater than or equal to a6 and less than or equal to 1, it is scored as 2 points; if the quotient is greater than or equal to b6 and less than a6, it is scored as 1 point; otherwise, it is scored as 0 points. Here, a6 and b6 represent the preset eleventh and twelfth thresholds, respectively.

[0044] For tooth color, a third processed image and a fourth processed image are obtained. The third processed image is a partial image of the current tooth cropped from the image after Gaussian blurring. The fourth processed image is a partial image of the tooth of the same name on the opposite side of the current tooth cropped from the image after Gaussian blurring. The difference between the RGB average values ​​in the third processed image and the fourth processed image is used as the fifth difference value. The tooth color is scored based on the fifth difference value and the preset seventh scoring standard.

[0045] Furthermore, if the absolute value of the fifth difference is greater than or equal to 0 and less than or equal to a7, then 2 points are awarded; if the absolute value of the fifth difference is greater than a7 and less than or equal to b7, then 1 point is awarded; otherwise, 0 points are awarded. Here, a7 and b7 represent the preset thirteenth and fourteenth thresholds, respectively.

[0046] For tooth transparency, the difference between the average A-values ​​in the third and fourth processed images is used as the sixth difference. Tooth transparency is scored based on this sixth difference and a preset eighth scoring standard. Further, if the absolute value of the sixth difference is greater than or equal to 0 and less than or equal to a8, 2 points are awarded; if the absolute value of the sixth difference is greater than a8 and less than or equal to b8, 1 point is awarded; otherwise, 0 points are awarded. Here, a8 and b8 represent the preset fifteenth and sixteenth thresholds, respectively.

[0047] For the tooth outline, the tooth portion outline of the current tooth is taken as the first tooth outline, and the tooth portion outline of the tooth with the same name on the opposite side of the current tooth is taken as the second tooth outline. n points are taken on the first tooth outline and the second tooth outline respectively to calculate the polar coordinate histogram of the two tooth outlines. Then, the Euclidean distance is used to compare the two histograms to obtain the shape difference between the two tooth outlines. The tooth outline is scored according to the shape difference and the preset ninth scoring standard.

[0048] Furthermore, if the shape difference is greater than or equal to 0 and less than or equal to a9, 2 points are awarded; if the shape difference is greater than a9 and less than or equal to b9, 1 point is awarded; otherwise, 0 points are awarded. Here, a9 and b9 represent the preset seventeenth and eighteenth thresholds, respectively.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] By utilizing computer vision technology, this invention enables automatic segmentation and recognition of teeth and gums, as well as automatic measurement and scoring of relevant indicators, significantly improving the efficiency and standardization of evaluation and achieving automation and intelligence in anterior tooth evaluation. This invention not only calculates objective morphological features of teeth but also automatically outputs relevant objective indicators using a computer, greatly simplifying the process of manually using EXO / CAD software. Furthermore, there is no significant difference between the data and the results calculated manually by the software. In addition, this invention also verified the automated red-white score against scores given by experienced prosthodontists, confirming the comprehensiveness and reliability of the evaluation. Simultaneously, this invention employs a human-computer interaction-based visual tooth position selection method, making the evaluation process more intuitive, simple, and easy to operate, greatly improving the user experience and enhancing user engagement. This automated evaluation system can be applied to the formulation of various dental treatment and restoration plans, providing dentists with objective and standardized assessment criteria, improving the targeting and effectiveness of treatment, and providing valuable technical support for clinical applications. Attached Figure Description

[0051] Figure 1 This is a flowchart of the steps of the present invention;

[0052] Figure 2 This is a schematic diagram of a patient's full-mouth oral scan image according to the present invention;

[0053] Figure 3 This is a schematic diagram of the teeth and gums image captured based on the user's click location according to the present invention;

[0054] Figure 4 This is a comparative schematic diagram of the grayscale image of a tooth before and after the user clicks on it; wherein, (a) is a schematic diagram of the grayscale image of a tooth before the user clicks on it and (b) is a schematic diagram of the grayscale image of a tooth after the user clicks on it.

[0055] Figure 5 This is a schematic diagram of the contour obtained by the present invention using the color segmentation method and the Chan-Vese active contour model.

[0056] Figure 6 This is a schematic diagram of key feature points of the gingiva and teeth obtained by the present invention.

[0057] Figure 7 This is a system block diagram of the present invention. Detailed Implementation

[0058] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0059] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0060] In a preferred embodiment of the present invention, an automatic anterior tooth feature parameter extraction method based on computer vision is provided, which can be used to automate and intelligently evaluate anterior teeth, and improve the problems of traditional methods such as strong subjectivity, inconsistent standards, low efficiency, and the inability of convolutional neural network-based methods to cope with real messy teeth images.

[0061] like Figure 1 As shown, the above-mentioned automatic anterior tooth feature parameter extraction method based on computer vision includes the following steps S1 to S11, and their specific implementation processes will be described in detail below.

[0062] S1. Acquire a full-mouth scan image of the patient to be processed, and visualize the full-mouth scan image to present it to the user.

[0063] It should be noted that in this invention, there are multiple ways to acquire the patient's full-mouth scan image. In this embodiment, the above-mentioned patient's full-mouth scan image is exported by 3Shape Unite software.

[0064] S2. When the user clicks on the tooth that needs to be scored, the corresponding tooth image is automatically extracted from the patient's full-mouth scan image based on the click location. Each tooth image includes the tooth part and the gum part.

[0065] It should be noted that in this invention, the number of teeth to be scored can be two or more. In this embodiment, two teeth to be scored are selected. When the user clicks on the two teeth to be scored with the mouse, the corresponding tooth portion image will be automatically captured and displayed based on the clicked location.

[0066] S3. Apply Gaussian blur to each tooth portion of the image to eliminate specular reflections, resulting in a Gaussian blurred image.

[0067] It should be noted that the implementation of the Gaussian blur method in this invention is prior art and will not be described in detail here.

[0068] S4. Based on the HSV value of the image after Gaussian blurring, the color segmentation method is used to set thresholds for the H, S, and V channels respectively, and the teeth and gums in each tooth part image are separated, resulting in an initial tooth image and an initial gum image for each tooth part image.

[0069] It should be noted that in this invention, the color segmentation method distinguishes between the teeth and gums by the HSV value of the image, and accurately extracts the images of the tooth region and the gum region respectively.

[0070] S5. Perform morphological processing on the initial tooth image and the initial gingival image respectively to obtain the morphologically processed tooth image and the morphologically processed gingival image.

[0071] It should be noted that, in this invention, the above-mentioned morphological processing includes opening and closing operations, wherein the opening operation is used to remove small dots in the tooth image or gum image, and the closing operation is used to fill holes in the tooth image or gum image.

[0072] S6. Use contour extraction methods to extract the tooth contour from the morphologically processed tooth image and the gingival contour from the morphologically processed gingival image, and add conditional rules based on user feedback to adjust the tooth and gingival contours.

[0073] It should be noted that, in this invention, the specific process of contour extraction using the contour extraction method is as follows: the initial tooth contour is extracted using Gaussian filtering and Canny edge detection, and then the initial tooth contour is enhanced and refined using image dilation and image skeletonization; the initial gingival contour is extracted using Gaussian filtering and Canny edge detection, and then the initial gingival contour is enhanced and refined using image dilation and image skeletonization.

[0074] S7. The final gingival contour and tooth base edge contour are obtained by the Chan-Vese active contour model algorithm. The gradient value of each pixel is calculated point by point on the final gingival contour and tooth base edge contour according to the corresponding contour edge curve. Combined with the actual position of each pixel in the Gaussian blurred image, the gingival papilla vertex and gingival vertex are obtained. The pixel with the smallest gradient value on the tooth base edge contour curve is taken as the boundary point on both sides of the tooth base. The intersection point of the tooth contour and the gingival contour and the tooth base boundary point are connected to obtain the boundary line between the teeth.

[0075] It should be noted that, in this invention, the dividing line between teeth is used to distinguish adjacent teeth from each other, avoiding confusion in contour extraction.

[0076] It should be noted that, in this invention, the specific process of obtaining the final gingival contour and the tooth base edge contour using the Chan-Vese active contour model algorithm is as follows:

[0077] For the gingival contour, the gingival contour is used as the initial value of the Chan-Vese active contour model algorithm. In each iteration, an inner region and an outer region are formed with the gingival contour as the boundary. The gray mean of the inner region and the gray mean of the outer region are calculated respectively. The gradient descent method is used to minimize the difference between the gray mean of the inner region and the inner region, and to minimize the difference between the gray mean of the outer region and the outer region, so as to guide the updating of the gingival contour. The update amplitude is controlled to ensure that the update is only performed near the boundary of the gingival contour. The iteration is repeated continuously to gradually adjust the gingival contour until the preset maximum number of iterations is reached or the change amplitude of the gingival contour is less than the preset contour threshold, so as to obtain the final gingival contour.

[0078] For the tooth contour, the background portion at the bottom of the morphologically processed tooth image is extracted (in this embodiment, the background portion at the bottom of the morphologically processed tooth image with a height of 5 pixels is selected) and used as the initial value of the Chan-Vese active contour model algorithm. This process is repeated iteratively to obtain the bottom edge contour of the tooth for subsequent feature point search.

[0079] S8. The accuracy of the tooth contour is further refined and improved by using the energy function within the tooth contour, resulting in a smoother transition between adjacent edges of the teeth.

[0080] It should be noted that in this invention, a three-term energy function is used to fine-tune the contour curve to make it more consistent with the actual situation. The specific form of the energy function is as follows:

[0081]

[0082] in, E represents the energy function; penal This represents a penalty term that limits the complexity of the contour, preventing excessive contour curvature or irregular shape; E grad E represents the gradient term, used to encourage the contour to update in the direction of large gradient change, that is, to make the contour closer to the edge; color The color change term is used to drive the contour to iterate towards the region with a large color change amplitude; λ0, λ1, and λ2 represent the contribution values ​​of each energy function term, and the larger the contribution value, the more the algorithm pays attention to this aspect.

[0083] S9. The Shi-Tomasi corner detection method is used to extract the corner points in the contours of the gingiva and the teeth respectively. The extracted corner points of the teeth are clustered using the K-means clustering algorithm to obtain the center points of the teeth after clustering. The extracted corner points of the gingiva are clustered using the K-means clustering algorithm to obtain the center points of the gingiva after clustering.

[0084] It should be noted that in this invention, the feature points in the gingival contour and the tooth contour are used as the basis for image feature points, and then the K-means clustering algorithm is used to cluster the feature points to reduce the number of feature points.

[0085] S10. Extract other important evaluation feature points in the tooth contour and gingival contour by curve fitting and finding extreme values; the other important evaluation feature points include the highest and lowest points of the tooth and the extreme points of the tooth in the horizontal direction.

[0086] S11. The clustered center points of the teeth, the clustered center points of the gingiva, and other important evaluation feature points are used as final feature points. The coordinates of all final feature points are obtained. Based on the coordinates of the final feature points, the relevant objective indicators for tooth evaluation are calculated, and the automatic extraction of anterior tooth feature parameters based on computer vision is completed. The relevant objective indicators for tooth evaluation include the length and width of the teeth and the length-to-width ratio, the height of the gingival apex, and the ratio of the height of the gingival papilla to the length of the crown.

[0087] It should be noted that, in step S11 of this invention, the specific method for calculating the relevant objective indicators of tooth evaluation based on the coordinates of the final feature points is as follows: Taking the upper left corner of the Gaussian blurred image as the origin, the tangent direction of the gingival contour edge curve at any gingival vertex is taken as the x-axis, and the direction perpendicular to it is taken as the y-axis. The coordinate y-value of the highest point of the tooth is subtracted from the coordinate y-value of the lowest point of the tooth. The absolute value of the difference in y-values ​​is multiplied by the scale of the patient's full-mouth scan image to obtain the length value of the tooth. The coordinate x-value of the extreme point in the horizontal direction of the tooth is subtracted from the coordinate x-value. The absolute value of the difference in y-values ​​of the tooth is multiplied by the scale of the patient's full-mouth scan image to obtain the width value of the tooth. The length value of the tooth is divided by the width value of the tooth to obtain the length-to-width ratio of the tooth. The coordinate y-value of the highest point of the gingiva is subtracted from the coordinate y-value of the lowest point of the gingiva. The absolute value of the difference in y-values ​​of the gingiva is multiplied by the scale of the patient's full-mouth scan image to obtain the height of the gingival vertex. The height of the gingival vertex is divided by the length value of the tooth to obtain the ratio of the height of the gingival papilla to the length of the crown.

[0088] It should be noted that in step S11 of the present invention, in order to evaluate the calculated objective indicators related to tooth evaluation, the above-mentioned objective indicators related to tooth evaluation can be obtained manually by using 3Shape Unite software in a manual fixed-point manner. Then, the objective indicators related to tooth evaluation calculated by the present invention are compared with the objective indicators related to tooth evaluation obtained by 3Shape Unite software to observe whether there are significant differences.

[0089] In embodiments of the present invention, an automated anterior tooth evaluation system based on computer vision is also provided, such as... Figure 7 As shown, it includes:

[0090] The data acquisition module is used to acquire mesial gingival papilla height, distal gingival papilla height, gingival origin level, gingival color, gingival contour, tooth shape, tooth color, tooth transparency, and tooth outline.

[0091] The evaluation module uses mesial gingival papilla height, distal gingival papilla height, gingival origin level, gingival color, and gingival contour as evaluation indicators to reflect the degree of gingival condition. It also uses tooth morphology, tooth color, tooth transparency, and tooth outline as evaluation indicators to reflect the degree of tooth condition. The module objectively calculates the characteristics of gingival color, height, and outline, as well as tooth color, height, transparency, and outline. It automatically generates corresponding tooth scores based on the specific differences between the gingival teeth and the contralateral corresponding teeth, thus completing the automated evaluation of anterior teeth.

[0092] It should be noted that, in the computer vision-based automated anterior tooth evaluation system of the present invention, the evaluation process for each evaluation index is as follows:

[0093] For mesial papilla height, the height difference between the current tooth's mesial papilla height and the contralateral tooth's mesial papilla height is taken as the first difference. The mesial papilla height is scored based on the first difference and the preset first scoring criteria.

[0094] Furthermore, if the absolute value of the first difference is greater than or equal to 0 and less than or equal to a1, then 2 points are awarded; if the absolute value of the first difference is greater than a1 and less than or equal to b1, then 1 point is awarded; otherwise, 0 points are awarded. Here, a1 and b1 represent the preset first threshold and second threshold, respectively.

[0095] For the distal gingival papilla height, the height difference between the distal gingival papilla height of the current tooth and the distal gingival papilla height of the contralateral tooth of the same name is used as the second difference. The distal gingival papilla height is scored based on the second difference and the preset second scoring criteria.

[0096] Furthermore, if the absolute value of the second difference is greater than or equal to 0 and less than or equal to a2, then 2 points are awarded; if the absolute value of the first difference is greater than a2 and less than or equal to b2, then 1 point is awarded; otherwise, 0 points are awarded. Here, a2 and b2 represent the preset third threshold and fourth threshold, respectively.

[0097] For the gingival origin level, the height difference between the gingival apex height of the current tooth and the gingival apex height of the contralateral tooth with the same name is used as the third difference. The gingival origin level is scored based on the third difference and the preset third scoring criteria.

[0098] Furthermore, if the absolute value of the third difference is greater than or equal to 0 and less than or equal to a3, then 2 points are awarded; if the absolute value of the third difference is greater than a3 and less than or equal to b3, then 1 point is awarded; otherwise, 0 points are awarded. Here, a3 and b3 represent the preset fifth threshold and sixth threshold, respectively.

[0099] For gingival color, a first processed image and a second processed image are obtained. The first processed image is a portion of the gingiva of the current tooth between the mesial and distal gingival papilla and within a preset distance above the gingival contour, cropped from the Gaussian blurred image. The second processed image is a portion of the gingiva of the contralateral tooth of the current tooth between the mesial and distal gingival papilla and within a preset distance above the gingival contour, cropped from the Gaussian blurred image. The difference between the RGB average values ​​of the gingival portions in the first and second processed images is taken as a fourth difference value. The gingival color is scored based on the fourth difference value and a preset fourth scoring standard.

[0100] Furthermore, if the absolute value of the fourth difference is greater than or equal to 0 and less than or equal to a4, then 2 points are awarded; if the absolute value of the fourth difference is greater than a4 and less than or equal to b4, then 1 point is awarded; otherwise, 0 points are awarded. Here, a4 and b4 represent the preset seventh threshold and eighth threshold, respectively.

[0101] For the gingival contour, the gingival contour of the current tooth between the mesial and distal gingival papilla is obtained as the first gingival contour. The gingival contour of the contralateral tooth of the same name between the mesial and distal gingival papilla is obtained as the second gingival contour. m points are taken on the first and second gingival contours respectively to calculate the length and relative center position of the gingival contour. The principal direction is obtained by using principal component analysis. The difference in length, the difference in relative center position, and the difference in principal direction of the first and second gingival contours are calculated. The weighted average of the three differences is taken as the final difference. The gingival contour is scored according to the final difference and the preset fifth scoring standard.

[0102] Furthermore, if the final difference is greater than or equal to 0 and less than or equal to a5, 2 points are awarded; if the final difference is greater than a5 and less than or equal to b5, 1 point is awarded; otherwise, 0 points are awarded. Here, a5 and b5 represent the preset ninth and tenth thresholds, respectively.

[0103] For tooth morphology, the length-to-width ratio of the current tooth and the length-to-width ratio of the corresponding tooth on the opposite side are calculated. The smaller of the two values ​​is divided by the larger value and the quotient is taken. The tooth morphology is scored based on this quotient and the preset sixth scoring standard.

[0104] Furthermore, if the quotient is greater than or equal to a6 and less than or equal to 1, it is scored as 2 points; if the quotient is greater than or equal to b6 and less than a6, it is scored as 1 point; otherwise, it is scored as 0 points. Here, a6 and b6 represent the preset eleventh and twelfth thresholds, respectively.

[0105] For tooth color, a third processed image and a fourth processed image are obtained. The third processed image is a partial image of the current tooth cropped from the image after Gaussian blurring. The fourth processed image is a partial image of the tooth of the same name on the opposite side of the current tooth cropped from the image after Gaussian blurring. The difference between the RGB average values ​​in the third processed image and the fourth processed image is used as the fifth difference value. The tooth color is scored based on the fifth difference value and the preset seventh scoring standard.

[0106] Furthermore, if the absolute value of the fifth difference is greater than or equal to 0 and less than or equal to a7, then 2 points are awarded; if the absolute value of the fifth difference is greater than a7 and less than or equal to b7, then 1 point is awarded; otherwise, 0 points are awarded. Here, a7 and b7 represent the preset thirteenth and fourteenth thresholds, respectively.

[0107] For tooth transparency, the difference between the average A-values ​​in the third and fourth processed images is used as the sixth difference. Tooth transparency is scored based on this sixth difference and a preset eighth scoring standard. Further, if the absolute value of the sixth difference is greater than or equal to 0 and less than or equal to a8, 2 points are awarded; if the absolute value of the sixth difference is greater than a8 and less than or equal to b8, 1 point is awarded; otherwise, 0 points are awarded. Here, a8 and b8 represent the preset fifteenth and sixteenth thresholds, respectively.

[0108] For the tooth outline, the tooth portion outline of the current tooth is taken as the first tooth outline, and the tooth portion outline of the tooth with the same name on the opposite side of the current tooth is taken as the second tooth outline. n points are taken on the first tooth outline and the second tooth outline respectively to calculate the polar coordinate histogram of the two tooth outlines. Then, the Euclidean distance is used to compare the two histograms to obtain the shape difference between the two tooth outlines. The tooth outline is scored according to the shape difference and the preset ninth scoring standard.

[0109] Furthermore, if the shape difference is greater than or equal to 0 and less than or equal to a9, 2 points are awarded; if the shape difference is greater than a9 and less than or equal to b9, 1 point is awarded; otherwise, 0 points are awarded. Here, a9 and b9 represent the preset seventeenth and eighteenth thresholds, respectively.

[0110] It should be noted that in the automated anterior tooth evaluation system of this invention, thresholds are set for the aforementioned evaluation indicators based on the evaluations of 20 teeth by dentists with extensive clinical experience and the differences in the objective values ​​of each indicator, to reflect the condition of the teeth and gums. To evaluate the automatically generated tooth scores, they can be compared with the scores from experienced prosthodontists to observe whether there are significant differences.

[0111] Figure 2 This is a schematic diagram of a real patient's full-mouth scan image. This embodiment uses this image for practice, according to... Figure 1 The flowchart shown illustrates images of teeth and gums captured based on the user's click location. Figure 3 As shown, with Figure 3 Taking a middle tooth as an example, the comparison between the grayscale image of that tooth before and after de-reflection is shown below. Figure 4 As shown. This invention uses two methods to obtain the contour diagram, as follows. Figure 5 As shown. Figure 6 The invention demonstrates the tooth and gingival feature points found by this invention. Based on the coordinates of these feature points, subsequent objective index calculations and automated scoring are performed, and the results are shown in Table 1.

[0112] Table 1. Automated scoring results

[0113] Tooth number 13 12 11 21 22 23 Aspect Ratio 2.13 1.36 1.32 1.21 1.28 1.94 PES near the mid-gingival papilla 2 2 2 2 2 2 PES distal gingival papilla 1 2 2 2 2 1 PES Gingival Source Level 1 0 1 1 0 1 PES soft tissue contour 2 0 1 1 0 2 PES soft tissue color 1 2 0 0 2 1 WES restoration color 2 1 0 0 1 2 WES restoration morphology 2 2 2 2 2 2 WES restoration outline 1 1 2 2 1 1 WES restoration transparency 1 0 2 2 0 1

[0114] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for automatic anterior tooth feature parameter extraction based on computer vision, characterized in that, Includes the following steps: S1. Obtain a full-mouth scan image of the patient to be processed, and visualize the full-mouth scan image of the patient so that it is displayed in front of the user; S2. When the user clicks on the tooth that needs to be scored, the corresponding tooth image is automatically extracted from the patient's full-mouth scan image based on the click location. Each tooth image includes the tooth part and the gum part. S3. Apply Gaussian blur to each tooth portion of the image to eliminate specular reflections, resulting in a Gaussian blurred image; S4. Based on the HSV value of the image after Gaussian blur, the color segmentation method is used to set thresholds for the H, S, and V channels respectively, and the teeth and gums in each tooth part image are separated, and each tooth part image obtains an initial tooth image and an initial gum image. S5. Perform morphological processing on the initial tooth image and the initial gingival image respectively to obtain the morphologically processed tooth image and the morphologically processed gingival image. S6. Use contour extraction methods to extract the tooth contour from the morphologically processed tooth image and the gingival contour from the morphologically processed gingival image, and add conditional rules based on user feedback to adjust the tooth contour and gingival contour. S7. The final gingival contour and tooth bottom edge contour are obtained by the Chan-Vese active contour model algorithm. The gradient value of each pixel is calculated point by point on the final gingival contour and tooth bottom edge contour according to the corresponding contour edge curve. Combined with the actual position of each pixel in the Gaussian blurred image, the gingival papilla vertex and gingival vertex are obtained. The pixel with the smallest gradient value on the edge curve of the tooth bottom edge contour is taken as the boundary point on both sides of the tooth bottom. The intersection point of the tooth contour and the gingival contour and the tooth bottom boundary point are connected to obtain the boundary line between the teeth. S8. Further refine and improve the accuracy of the tooth contour by using the energy function within the tooth contour, making the transition between adjacent edges of the teeth smoother; S9. The Shi-Tomasi corner detection method is used to extract the corner points in the contour of the gingiva and the contour of the teeth respectively. The extracted corner points of the teeth are clustered using the K-means clustering algorithm to obtain the center point of the teeth after clustering. The extracted corner points of the gingiva are clustered using the K-means clustering algorithm to obtain the center point of the gingiva after clustering. S10. Extract other important evaluation feature points in the tooth contour and gingival contour by curve fitting and finding extreme values, wherein the other important evaluation feature points include the highest and lowest points of the tooth and the extreme points of the tooth in the horizontal direction. S11. The clustered center points of the teeth, the clustered center points of the gingiva, and other important evaluation feature points are used as final feature points. The coordinates of all final feature points are obtained. Based on the coordinates of the final feature points, the relevant objective indicators for tooth evaluation are calculated, and the automatic extraction of anterior tooth feature parameters based on computer vision is completed. The relevant objective indicators for tooth evaluation include the length and width of the teeth and the length-to-width ratio, the height of the gingival apex, and the ratio of the height of the gingival papilla to the length of the crown. In step S8, the energy function The specific format is as follows: ； in, This represents a penalty term that limits the complexity of the outline; Represents the gradient term; Indicates color variation items; These represent the contribution value of each energy function; In step S11, the specific method for calculating the relevant objective indicators of tooth evaluation based on the coordinates of the final feature points is as follows: Taking the upper left corner of the Gaussian blurred image as the origin, the tangent direction of the gingival contour edge curve at any gingival vertex is taken as the x-axis, and the direction perpendicular to it is taken as the y-axis. The coordinate y-value of the highest point of the tooth is subtracted from the coordinate y-value of the lowest point of the tooth. The absolute value of the difference in y-values ​​is multiplied by the scale of the patient's full-mouth scan image to obtain the length value of the tooth. The coordinate x-value of the extreme points in the horizontal direction of the tooth is subtracted from the coordinate x-value of the tooth. The absolute value of the difference in y-values ​​of the tooth is multiplied by the scale of the patient's full-mouth scan image to obtain the width value of the tooth. The length value of the tooth is divided by the width value of the tooth to obtain the length-to-width ratio of the tooth. The coordinate y-value of the highest point of the gingiva is subtracted from the coordinate y-value of the lowest point of the gingiva. The absolute value of the difference in y-values ​​of the gingiva is multiplied by the scale of the patient's full-mouth scan image to obtain the height of the gingival vertex. The height of the gingival vertex is divided by the length value of the tooth to obtain the ratio of the height of the gingival papilla to the length of the crown.

2. The automatic anterior tooth feature parameter extraction method based on computer vision as described in claim 1, characterized in that, In step S5, the morphological processing includes opening and closing operations, wherein the opening operation is used to remove small dots in the tooth image or gum image, and the closing operation is used to fill holes in the tooth image or gum image.

3. The automatic anterior tooth feature parameter extraction method based on computer vision as described in claim 1, characterized in that, In step S6, the specific process of contour extraction using the contour extraction method is as follows: the initial tooth contour is extracted using Gaussian filtering and Canny edge detection, and then the initial tooth contour is enhanced and refined using image dilation and image skeletonization; the initial gingival contour is extracted using Gaussian filtering and Canny edge detection, and then the initial gingival contour is enhanced and refined using image dilation and image skeletonization.

4. The automatic anterior tooth feature parameter extraction method based on computer vision as described in claim 1, characterized in that, In step S7, the specific process of obtaining the final gingival contour and the bottom edge contour of the teeth using the Chan-Vese active contour model algorithm is as follows: For the gingival contour, the gingival contour is used as the initial value of the Chan-Vese active contour model algorithm. In each iteration, an inner region and an outer region are formed with the gingival contour as the boundary. The gray mean of the inner region and the gray mean of the outer region are calculated respectively. The gradient descent method is used to minimize the gray mean of the inner region and the gray mean of the outer region and minimize the difference between the inner region and the outer region to guide the updating of the gingival contour. The update amplitude is controlled to ensure that the update is only performed near the boundary of the gingival contour. The process is repeated iteratively, gradually adjusting the gingival contour until the preset maximum number of iterations is reached or the change in the gingival contour is less than the preset contour threshold, thus obtaining the final gingival contour. For the tooth contour, the background portion at the bottom of the tooth is extracted from the morphologically processed tooth image and used as the initial value for the Chan-Vese active contour model algorithm. This process is repeated iteratively to obtain the bottom edge contour of the tooth.

5. An automated anterior tooth evaluation system based on computer vision, characterized in that, include: The data acquisition module is used to acquire the mesial gingival papilla height, distal gingival papilla height, gingival origin level, gingival color, gingival contour, tooth shape, tooth color, tooth transparency and tooth outline obtained by the automatic anterior tooth feature parameter extraction method based on computer vision as described in any one of claims 1-4. The evaluation module uses mesial gingival papilla height, distal gingival papilla height, gingival origin level, gingival color, and gingival contour as evaluation indicators to reflect the degree of gingival condition. It also uses tooth morphology, tooth color, tooth transparency, and tooth outline as evaluation indicators to reflect the degree of tooth condition. The module objectively calculates the characteristics of gingival color, height, and outline, as well as tooth color, height, transparency, and outline. It automatically generates corresponding tooth scores based on the specific differences between the gingival teeth and the contralateral corresponding teeth, thus completing the automated evaluation of anterior teeth.

6. The automated anterior tooth evaluation system based on computer vision as described in claim 5, characterized in that, The evaluation process for each evaluation indicator is as follows: For the height of the mesial gingival papilla, the height difference between the height of the mesial gingival papilla of the current tooth and the height of the mesial gingival papilla of the contralateral tooth of the same name is taken as the first difference. The height of the mesial gingival papilla is scored based on the first difference and the preset first scoring standard. For the height of the distal gingival papilla, the height difference between the height of the distal gingival papilla of the current tooth and the height of the distal gingival papilla of the contralateral tooth of the same name is used as the second difference. The height of the distal gingival papilla is scored based on the second difference and the preset second scoring criteria. For the gingival origin level, the height difference between the gingival apex height of the current tooth and the gingival apex height of the contralateral tooth of the same name is used as the third difference. The gingival origin level is scored based on the third difference and the preset third scoring criteria. For gingival color, a first processed image and a second processed image are obtained. The first processed image is a gingival portion of the current tooth between the mesial and distal gingival papilla and within a preset distance above the gingival contour, cropped from the Gaussian blurred image. The second processed image is a gingival portion of the contralateral tooth of the current tooth between the mesial and distal gingival papilla and within a preset distance above the gingival contour, cropped from the Gaussian blurred image. The difference between the RGB average values ​​of the gingival portion in the first processed image and the second processed image is taken as a fourth difference value. The gingival color is scored based on the fourth difference value and a preset fourth scoring standard. For the gingival contour, the gingival contour of the current tooth between the mesial and distal gingival papilla is obtained as the first gingival contour. The gingival contour of the contralateral tooth with the same name between the mesial and distal gingival papilla is obtained as the second gingival contour. m points are taken on the first and second gingival contours respectively to calculate the length and relative center position of the gingival contour. The principal component analysis method is used to obtain the principal direction. The length difference, relative center position difference and principal direction difference of the first and second gingival contours are calculated. The weighted average of the three differences is the final difference. The gingival contour is scored according to the final difference and the preset fifth scoring standard. For tooth morphology, calculate the length-to-width ratio of the current tooth and the length-to-width ratio of the corresponding tooth on the opposite side, divide the smaller of the two by the larger value and take the quotient, and score the tooth morphology based on this quotient and the preset sixth scoring standard. For tooth color, a third processed image and a fourth processed image are obtained. The third processed image is a partial image of the current tooth cropped from the image after Gaussian blurring. The fourth processed image is a partial image of the tooth of the same name on the opposite side of the current tooth cropped from the image after Gaussian blurring. The difference between the RGB average values ​​in the third processed image and the fourth processed image is used as the fifth difference. The tooth color is scored based on the fifth difference and the preset seventh scoring standard. For tooth transparency, the difference between the average A value in the third and fourth processed images is used as the sixth difference. The tooth transparency is scored based on the sixth difference and the preset eighth scoring standard. For the tooth outline, the tooth portion outline of the current tooth is taken as the first tooth outline, and the tooth portion outline of the tooth with the same name on the opposite side of the current tooth is taken as the second tooth outline. n points are taken on the first tooth outline and the second tooth outline respectively to calculate the polar coordinate histogram of the two tooth outlines. Then, the Euclidean distance is used to compare the two histograms to obtain the shape difference between the two tooth outlines. The tooth outline is scored according to the shape difference and the preset ninth scoring standard.

Images