Automatic anterior tooth characteristic parameter extraction method and evaluation system based on computer vision

Abstract

The invention discloses an automatic anterior tooth characteristic parameter extraction method and evaluation system based on computer vision, and the method comprises the steps: obtaining and visualizing a full oral cavity scanning image of a patient to be processed, intercepting a tooth part image corresponding to the full oral cavity scanning image, eliminating specular reflection by using a Gaussian blur method, and obtaining an anterior tooth part image; threshold values are set for the H channel, the S channel and the V channel respectively through a color segmentation method, teeth and gingiva in each tooth part image are separated and then subjected to morphological processing respectively, and tooth part contours and gingiva part contours are extracted through a contour extraction method; the segmentation precision is further refined through a Chan-Vese active contour model algorithm and an energy function in a tooth part contour, angular points are extracted through a Shii-Tomasi angular point detection method and then clustered, and other evaluation important feature points are extracted through curve fitting and an extreme value searching method. And finally, measuring and calculating tooth evaluation related objective indexes and automatically generating a corresponding tooth score, thereby completing automatic anterior tooth evaluation.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and oral evaluation, and in particular relates to an automatic front teeth feature parameter extraction method and an evaluation system based on computer vision. Background Art

[0002] The front teeth are located in a prominent position in the oral cavity. Accurate and rapid evaluation of the front teeth can provide a basis for dentists to develop more accurate diagnosis and treatment plans, help patients better understand their oral conditions, and improve satisfaction. For example, in restorative treatment planning, doctors can use automated scoring results to quickly understand patients' needs and choose appropriate restoration plans, or assist dental technicians to focus on the effects, so as to better arrange teeth. Therefore, accurate and rapid evaluation of the front teeth is crucial for dental diagnosis and restoration. Traditional evaluation of the front teeth mainly relies on the subjective experience and judgment of professional dentists, which has problems such as strong subjectivity, inconsistent standards, and low efficiency. The quantitative evaluation methods used in previous studies, such as manually measuring tooth characteristic parameters using measuring tools, are cumbersome and inefficient. In recent years, computer technology has been widely used in the medical field, such as medical image analysis and automated diagnosis. These technologies have the advantages of strong objectivity and high degree of automation, which provide possibilities for the intelligentization of the medical field. However, in terms of automated evaluation of teeth, the current mainstream methods based on convolutional neural networks are still limited to the recognition and marking of standard oral models. However, in reality, human teeth are often varied in shape and arranged differently, which makes it difficult to apply the convolutional neural network method in practical applications. Computer vision technology can identify and extract oral scans of real teeth, so as to obtain more accurate automated scores and have better clinical application prospects. In clinical efficacy evaluation and clinical research, the effect can be evaluated more objectively and conveniently. The objectivity of the current mainstream PES / WES scores depends largely on the evaluator's technical sensitivity. In the future, promoting the widespread application of this method will help to achieve a unified objective evaluation standard worldwide.

[0003] Therefore, developing an automated anterior teeth evaluation method based on computer vision can overcome the limitations of existing evaluation methods, improve the objectivity, standardization and efficiency of the evaluation, and has important clinical application value. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of tooth evaluation methods existing in the prior art and the challenges of automated evaluation of anterior teeth based on oral scan images, and to provide an automatic anterior teeth feature parameter extraction method and evaluation system based on computer vision.

[0005] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for automatically extracting anterior teeth feature parameters based on computer vision, which comprises the following steps:

[0007] S1. Obtain a full-mouth oral scan image of a patient to be processed, and visualize the full-mouth oral scan image of the patient to display it in front of a user;

[0008] S2. When the user clicks on the tooth to be scored, the corresponding tooth part image is automatically captured in the patient's full-mouth oral scan image according to the click position, and each tooth part image includes the tooth part and the gum part;

[0009] S3. Applying Gaussian blur method to each tooth partial image to eliminate the specular reflection therein, and obtaining a Gaussian blurred image;

[0010] S4. Using the color segmentation method based on the HSV value of the image after Gaussian blur, set thresholds for the three channels H, S, and V respectively, separate the teeth from the gums in each tooth partial image, and obtain an initial tooth image and an initial gum image for each tooth partial image;

[0011] S5. Performing morphological processing on the initial tooth image and the initial gum image to obtain a morphologically processed tooth image and a morphologically processed gum image;

[0012] S6. extracting the tooth part contour from the morphologically processed tooth image and the gingival part contour from the morphologically processed gingival image using a contour extraction method, and adding conditional rules to adjust the tooth part contour and the gingival part contour according to user feedback;

[0013] S7. The final gingival contour and tooth bottom edge contour are obtained by using the Chan-Vese active contour model algorithm. The gradient value of each pixel point is calculated point by point on the final gingival contour and tooth bottom edge contour according to the direction of the corresponding contour edge curve. The gingival papilla vertex and the gingival vertex are obtained in combination with the actual position of each pixel point in the image after Gaussian blur. The pixel point with the smallest gradient value on the edge curve of the tooth bottom edge contour is used as the boundary point on both sides of the tooth bottom. The intersection point of the tooth contour and the gingival contour and the tooth bottom boundary point are connected to obtain the boundary line between the teeth.

[0014] S8. Further refine and improve the accuracy of the tooth contour through the energy function within the tooth contour, so that the transition between adjacent edges of the teeth is smoother;

[0015] S9. Apply Shi-Tomasi corner point detection method to extract corner points in the outline of gum part and tooth part respectively, cluster the extracted tooth part corner points using K-means clustering algorithm to obtain the clustered center point of tooth part, cluster the extracted gum part corner points using K-means clustering algorithm to obtain the clustered center point of gum part;

[0016] S10. extracting other important feature points for evaluation in the tooth contour and the gum contour by curve fitting and extreme value finding methods, wherein the other important feature points for evaluation include the highest point and the lowest point of the tooth and the extreme value point of the tooth in the horizontal direction;

[0017] S11. Take the center point of the clustered tooth part, the center point of the clustered gum part and other important feature points for evaluation as the final feature points, obtain the coordinates of all the final feature points, calculate the objective indicators related to tooth evaluation based on the coordinates of the final feature points, and complete the automatic anterior tooth feature parameter extraction based on computer vision; the objective indicators related to tooth evaluation include the length and width value and length-to-width ratio of the teeth, the gingival apex height and the ratio of the gingival papilla height to the crown length.

[0018] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0019] As a preferred embodiment of the first aspect, in step S5, the morphological processing includes an opening operation and a closing operation, wherein the opening operation is used to remove small dots in the tooth image or the gum image, and the closing operation is used to fill holes in the tooth image or the gum image.

[0020] As a preferred embodiment of the above-mentioned first aspect, in step S6, the specific process of contour extraction using the contour extraction method is: using Gaussian filtering and Canny edge detection method to extract the initial tooth part contour, and then using image expansion and image skeletonization method to enhance and refine the initial tooth part contour; using Gaussian filtering and Canny edge detection method to extract the initial gum part contour, and then using image expansion and image skeletonization method to enhance and refine the initial gum part contour.

[0021] As a preferred embodiment of the first aspect, in step S7, the specific process of obtaining the final gingival part contour and the tooth bottom edge contour by the Chan-Vese active contour model algorithm is as follows:

[0022] For the gingival contour, the gingival contour is used as the initial value of the Chan-Vese active contour model algorithm. In each iteration, the inner region and the outer region are formed with the gingival contour as the boundary, and the grayscale mean of the inner region and the grayscale mean of the outer region are calculated respectively. The gradient descent method is used to minimize the difference between the grayscale mean of the inner region and the inner region and minimize the difference between the grayscale mean of the outer region and the outer region to guide the gingival contour to be updated, and the update amplitude is controlled to meet the requirement of updating only near the boundary of the gingival contour; the iteration is repeated continuously, and the gingival contour is gradually adjusted until the preset maximum number of iterations is reached or the change amplitude of the gingival contour is less than the preset contour threshold, so as to obtain the final gingival contour;

[0023] For the partial tooth contour, the background part at the bottom is extracted from the morphologically processed tooth image and used as the initial value of the Chan-Vese active contour model algorithm. The iteration is repeated to obtain the bottom edge contour of the tooth.

[0024] As a preferred embodiment of the first aspect, in step S8, the energy function The specific form is as follows:

[0025]

[0026] Among them, E penal represents the penalty term that limits the complexity of the contour; E grad represents the gradient term; E color represents the color change term; λ0, λ1, λ2 represent the contribution value of each energy function respectively.

[0027] As a preferred embodiment of the above-mentioned first aspect, in step S11, the specific method for calculating the objective indicators related to tooth evaluation based on the coordinates of the final feature points is as follows: taking the upper left corner of the image after Gaussian blur as the origin, the tangent direction of the edge curve of the gingival part contour at any gingival vertex is taken as the x-axis, and the direction perpendicular to it is taken as the y-axis, subtracting the coordinate y value of the highest point of the tooth from the coordinate y value of the lowest point of the tooth, multiplying the absolute value of the difference in the y values ​​by the scale of the patient's full-mouth oral scan image to obtain the length value of the tooth, subtracting the coordinate x value of the extreme point in the horizontal direction of the tooth, multiplying the absolute value of the difference in the x values ​​of the tooth by the scale of the patient's full-mouth oral scan image to obtain the width value of the tooth, dividing the length value of the tooth by the width value of the tooth to obtain the aspect ratio of the tooth, subtracting the coordinate y value of the highest point of the gum from the coordinate y value of the lowest point of the gum, multiplying the absolute value of the difference in the y values ​​of the gum by the scale of the patient's full-mouth oral scan image to obtain the gingival apex height, and dividing the gingival apex height by the length value of the tooth to obtain the ratio of the gingival papilla height to the crown length.

[0028] In a second aspect, the present invention provides an automated anterior teeth evaluation system based on computer vision, comprising:

[0029] A data acquisition module is used to obtain the mesial gingival papilla height, distal gingival papilla height, gingival gingival source level, gingival color, gingival contour, tooth morphology, tooth color, tooth transparency and tooth shape contour;

[0030] The evaluation module is used to take the mesial gingival papilla height, distal gingival papilla height, gingival gingival source level, gingival color and gingival contour as evaluation indicators reflecting the degree of gingival condition, and take the tooth morphology, tooth color, tooth transparency and tooth shape contour as evaluation indicators reflecting the degree of tooth condition, objectively calculate the gingival color, height and contour characteristics as well as the tooth color, height, transparency and contour characteristics, automatically generate the corresponding tooth score through the specific difference values ​​between the gingival teeth and the opposite teeth with the same name, and complete the automated evaluation of the anterior teeth.

[0031] In the automated anterior teeth evaluation system based on computer vision, the evaluation process of each evaluation index is as follows:

[0032] For the mesial papilla height, the height difference between the mesial papilla height of the current tooth and the mesial papilla height of the same tooth on the opposite side is taken as the first difference, and the mesial papilla height is scored based on the first difference and the preset first scoring standard.

[0033] Furthermore, if the absolute value of the first difference is greater than or equal to 0 and less than or equal to a1, 2 points are scored; if the absolute value of the first difference is greater than a1 and less than or equal to b1, 1 point is scored; otherwise 0 points are scored; wherein a1 and b1 represent the preset first threshold and second threshold, respectively.

[0034] For the distal gingival papilla height, the height difference between the distal gingival papilla height of the current tooth and the distal gingival papilla height of the same tooth on the opposite side is taken as the second difference, and the distal gingival papilla height is scored based on the second difference and the preset second scoring standard.

[0035] Furthermore, if the absolute value of the second difference is greater than or equal to 0 and less than or equal to a2, 2 points are scored; if the absolute value of the first difference is greater than a2 and less than or equal to b2, 1 point is scored; otherwise 0 points are scored; wherein a2 and b2 represent the preset third threshold and fourth threshold, respectively.

[0036] For the gingival origin level, the height difference between the gingival apex height of the current tooth and the gingival apex height of the same tooth on the opposite side is taken as the third difference, and the gingival origin level is scored based on the third difference and a preset third scoring standard.

[0037] Further, if the absolute value of the third difference is greater than or equal to 0 and less than or equal to a3, 2 points are scored; if the absolute value of the third difference is greater than a3 and less than or equal to b3, 1 point is scored; otherwise 0 points are scored; wherein a3 and b3 represent the preset fifth threshold and sixth threshold, respectively.

[0038] For gum color, a first processed image and a second processed image are obtained, wherein the first processed image is a partial image of the gums of the current tooth captured in the image after Gaussian blur, between the mesial gum papilla and the distal gum papilla and within a preset distance above the gum contour, and the second processed image is a partial image of the gums of the same tooth on the opposite side of the current tooth captured in the image after Gaussian blur, between the mesial gum papilla and the distal gum papilla and within a preset distance above the gum contour. The difference between the RGB average values ​​within the gum part in the first processed image and the second processed image is taken as the fourth difference, and the gum color is scored based on the fourth difference and the preset fourth scoring standard.

[0039] Further, if the absolute value of the fourth difference is greater than or equal to 0 and less than or equal to a4, 2 points are scored; if the absolute value of the fourth difference is greater than a4 and less than or equal to b4, 1 point is scored; otherwise 0 points are scored; wherein a4 and b4 represent the preset seventh threshold and eighth threshold, respectively.

[0040] For the gingival contour, the gingival contour of the current tooth between the mesial gingival papilla and the distal gingival papilla is obtained and used as the first gingival contour, and the gingival contour of the tooth with the same name on the opposite side of the current tooth between the mesial gingival papilla and the distal gingival papilla is obtained and used as the second gingival contour. m points are taken on the first gingival contour and the second gingival contour respectively to calculate the length and relative center position of the gingival contour, and the principal component analysis method is used to obtain the main direction. The length difference, relative center position difference and main direction difference between the first gingival contour and the second gingival contour are calculated, and the weighted average of the three differences is taken as the final difference. The gingival contour is scored according to the final difference and the preset fifth scoring standard.

[0041] Furthermore, if the final difference is greater than or equal to 0 and less than or equal to a5, 2 points are scored; if the final difference is greater than a5 and less than or equal to b5, 1 point is scored; otherwise 0 points are scored; wherein a5 and b5 represent the preset ninth threshold and tenth threshold, respectively.

[0042] For tooth morphology, the length-to-width ratio of the current tooth and the length-to-width ratio of the tooth with the same name on the opposite side are calculated, the smaller value is divided by the larger value and the quotient is taken, and the tooth morphology is scored based on this quotient and the preset sixth scoring standard.

[0043] Furthermore, if the quotient is greater than or equal to a6 and less than or equal to 1, 2 points are scored; if the quotient is greater than or equal to b6 and less than a6, 1 point is scored; otherwise, 0 points are scored; wherein a6 and b6 represent the preset eleventh threshold and twelfth threshold, respectively.

[0044] For tooth color, a third processed image and a fourth processed image are obtained, wherein the third processed image is a partial image of the current tooth captured in the image after Gaussian blur, and the fourth processed image is a partial image of the tooth with the same name on the opposite side of the current tooth captured in the image after Gaussian blur. The difference between the RGB average values ​​in the third processed image and the fourth processed image is taken as the fifth difference, and the tooth color is scored based on the fifth difference and the preset seventh scoring standard.

[0045] Further, if the absolute value of the fifth difference is greater than or equal to 0 and less than or equal to a7, 2 points are scored; if the absolute value of the fifth difference is greater than a7 and less than or equal to b7, 1 point is scored; otherwise 0 points are scored; wherein a7 and b7 represent the preset thirteenth threshold and fourteenth threshold, respectively.

[0046] For tooth transparency, the difference between the average values ​​of the A values ​​in the third processed image and the fourth processed image is taken as the sixth difference, and the tooth transparency is scored according to the sixth difference and the preset eighth scoring standard. Further, if the absolute value of the sixth difference is greater than or equal to 0 and less than or equal to a8, 2 points are scored, if the absolute value of the sixth difference is greater than a8 and less than or equal to b8, 1 point is scored, otherwise 0 points are scored; wherein a8 and b8 represent the preset fifteenth threshold and sixteenth threshold, respectively.

[0047] For the tooth contour, the partial tooth contour of the current tooth is taken as the first tooth contour, and the partial tooth contour of the tooth with the same name on the opposite side of the current tooth is taken as the second tooth contour, and n points are taken on the first tooth contour and the second tooth contour respectively to calculate the histogram of the polar coordinates of the two tooth contours, and then the Euclidean distance is used to compare the two histograms to obtain the shape difference of the two tooth contours, and the tooth contour is scored based on the shape difference and the preset ninth scoring standard.

[0048] Furthermore, if the shape difference is greater than or equal to 0 and less than or equal to a9, 2 points are scored; if the shape difference is greater than a9 and less than or equal to b9, 1 point is scored; otherwise, 0 points are scored; wherein a9 and b9 represent the preset seventeenth and eighteenth thresholds, respectively.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] Through computer vision technology, the automatic segmentation and recognition of teeth and gums, as well as the automatic measurement and scoring of relevant indicators are realized, which greatly improves the efficiency and standardization of the evaluation, and realizes the automation and intelligence of the anterior teeth evaluation. The present invention not only measures the objective tooth morphological characteristic indicators, but also uses the computer to automatically output the relevant objective indicators, which greatly simplifies the process of manual use of EXO / CAD software, and there is no significant difference between the data and the results of manual fixed-point measurement by the software. In addition, the present invention also verifies the red and white scores obtained by automation with the scores performed by experienced prosthodontists, confirming the comprehensiveness and reliability of the evaluation. At the same time, the present invention adopts a visual tooth position selection method based on human-computer interaction, which makes the evaluation process more intuitive, simple and easy to operate, greatly improves the user experience, and enhances the user experience and participation. The automated evaluation system can be applied to the formulation of various dental diagnosis and treatment and restoration plans, providing dentists with objective and standardized evaluation basis, improving the pertinence and effect of diagnosis and treatment, and providing valuable technical support for clinical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the steps of the present invention;

[0052] Figure 2 This is a schematic diagram of a full-mouth oral scan image of a patient according to the present invention;

[0053] Figure 3 This is a schematic diagram of the teeth and gums image captured according to the user's click position in the present invention;

[0054] Figure 4 Schematic diagrams comparing the grayscale image of a tooth clicked by a user before and after de-reflection; (a) is a schematic diagram of the grayscale image of a tooth clicked by a user before de-reflection, and (b) is a schematic diagram of the grayscale image of a tooth clicked by a user after de-reflection;

[0055] Figure 5 It is a schematic diagram of the contour obtained by the color segmentation method and the Chan-Vese active contour model of the present invention;

[0056] Figure 6 This is a schematic diagram of the key characteristic points of gums and teeth obtained by the present invention;

[0057] Figure 7 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0059] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0060] In a preferred implementation of the present invention, a method for automatic anterior teeth feature parameter extraction based on computer vision is provided, which can be used to realize the automation and intelligence of anterior teeth evaluation, improve the problems of traditional methods such as strong subjectivity, inconsistent standards, low efficiency, and the inability of methods based on convolutional neural networks to cope with real messy teeth images.

[0061] like Figure 1 As shown, the above-mentioned method for automatic anterior teeth feature parameter extraction based on computer vision includes the following steps S1 to S11, and the specific implementation processes thereof are described in detail below.

[0062] S1. Obtain a full-mouth oral scan image of a patient to be processed, and visualize the full-mouth oral scan image of the patient to display it in front of a user.

[0063] It should be noted that, in the present invention, there are multiple ways to obtain the patient's full-mouth oral scan image. In this embodiment, the patient's full-mouth oral scan image is exported by 3Shape Unite software.

[0064] S2. When the user clicks on the tooth that needs to be scored, the corresponding tooth partial image is automatically captured in the patient's full-mouth oral scan image according to the click position, and each tooth partial image includes a tooth part and a gum part.

[0065] It should be noted that in the present invention, the number of teeth that need to be scored can be two or more. In this embodiment, two teeth that need to be scored are selected. When the user clicks the position of the two teeth that need to be scored with the mouse, the image of the corresponding tooth part can be automatically captured and displayed according to the click position.

[0066] S3. Use the Gaussian blur method to eliminate the mirror reflection of each tooth partial image to obtain the Gaussian blurred image.

[0067] It should be noted that, in the present invention, the implementation of the Gaussian blur method belongs to the prior art and will not be described in detail.

[0068] S4. Based on the HSV value of the image after Gaussian blur, the color segmentation method is used to set thresholds for the three channels H, S, and V respectively, and the teeth and gums in each tooth partial image are separated. Each tooth partial image obtains an initial tooth image and an initial gum image.

[0069] It should be noted that, in the present invention, the color segmentation method distinguishes the teeth and gums by the HSV value of the image, and accurately extracts the teeth area image and the gum area image respectively.

[0070] S5. Perform morphological processing on the initial tooth image and the initial gum image respectively to obtain a morphologically processed tooth image and a morphologically processed gum image.

[0071] It should be noted that, in the present invention, the above-mentioned morphological processing includes an opening operation and a closing operation, wherein the opening operation is used to remove small dots in the tooth image or the gum image, and the closing operation is used to fill holes in the tooth image or the gum image.

[0072] S6. Extract the tooth part contour from the morphologically processed tooth image and extract the gum part contour from the morphologically processed gum image using a contour extraction method, and add conditional rules to adjust the tooth part contour and the gum part contour according to user feedback.

[0073] It should be noted that, in the present invention, the specific process of contour extraction using the contour extraction method is: using Gaussian filtering and Canny edge detection methods to extract the initial tooth part contour, and then using image expansion and image skeletonization methods to enhance and refine the initial tooth part contour; using Gaussian filtering and Canny edge detection methods to extract the initial gum part contour, and then using image expansion and image skeletonization methods to enhance and refine the initial gum part contour.

[0074] S7. The final gingival contour and tooth bottom edge contour are obtained by the Chan-Vese active contour model algorithm. The gradient value of each pixel point is calculated point by point on the final gingival contour and tooth bottom edge contour according to the direction of the corresponding contour edge curve. The gingival papilla vertex and gingival vertex are obtained in combination with the actual position of each pixel point in the image after Gaussian blur. The pixel point with the smallest gradient value on the edge curve of the tooth bottom edge contour is used as the boundary point on both sides of the tooth bottom. The intersection point of the tooth contour and the gingival contour and the tooth bottom boundary point are connected to obtain the boundary line between the teeth.

[0075] It should be noted that, in the present invention, the boundary line between teeth is used to distinguish adjacent teeth from each other to avoid confusion in contour extraction.

[0076] It should be noted that, in the present invention, the specific process of obtaining the final gingival part contour and the bottom edge contour of the tooth by the Chan-Vese active contour model algorithm is as follows:

[0077] For the gingival contour, the gingival contour is used as the initial value of the Chan-Vese active contour model algorithm. In each iteration, the inner region and the outer region are formed with the gingival contour as the boundary, and the grayscale mean of the inner region and the grayscale mean of the outer region are calculated respectively. The gradient descent method is used to minimize the difference between the grayscale mean of the inner region and the inner region and minimize the difference between the grayscale mean of the outer region and the outer region to guide the gingival contour to be updated, and the update amplitude is controlled to meet the requirement of updating only near the boundary of the gingival contour; the iteration is repeated continuously, and the gingival contour is gradually adjusted until the preset maximum number of iterations is reached or the change amplitude of the gingival contour is less than the preset contour threshold, so as to obtain the final gingival contour;

[0078] For the partial tooth contour, the background part at the bottom is extracted from the morphologically processed tooth image (in this embodiment, the background part with a bottom height of 5 pixels of the morphologically processed tooth image is selected) and used as the initial value of the Chan-Vese active contour model algorithm. The iteration is repeated to obtain the bottom edge contour of the tooth for subsequent use in finding feature points.

[0079] S8. The energy function within the tooth contour is used to further refine and improve the accuracy of the tooth contour, so that the transition between adjacent edges of the teeth is smoother.

[0080] It should be noted that, in the present invention, a three-term energy function is used to fine-tune the contour curve to make it more in line with the actual situation. The form of the energy function is as follows:

[0081]

[0082] in, represents the energy function; E penal represents the penalty term that limits the complexity of the contour, preventing the contour curvature from being too large or the shape from being irregular; E grad represents the gradient term, which is used to encourage the contour to update in the direction of large gradient change, that is, to make the contour closer to the edge; E color Represents the color change term, which is used to drive the contour to iterate to the area with larger color change; λ0, λ1, λ2, respectively represent the contribution value of each energy function. The larger the contribution value, the more attention the algorithm pays to this aspect.

[0083] S9. Apply Shi-Tomasi corner point detection method to extract corner points in the gum part contour and the tooth part contour respectively, cluster the extracted tooth part corner points using K-means clustering algorithm to obtain the clustered tooth part center point, and cluster the extracted gum part corner points using K-means clustering algorithm to obtain the clustered gum part center point.

[0084] It should be noted that, in the present invention, the feature points in the gum part contour and the tooth part contour are used as the basis of the image feature points, and then the K-means clustering algorithm is used to cluster the feature points to reduce the number of feature points.

[0085] S10. Extract other important feature points for evaluation in the tooth and gum contours by curve fitting and extreme value finding methods; the other important feature points for evaluation include the highest point and the lowest point of the tooth and the extreme value points of the tooth in the horizontal direction.

[0086] S11. Take the center point of the clustered tooth part, the center point of the clustered gum part and other important feature points for evaluation as the final feature points, obtain the coordinates of all the final feature points, calculate the objective indicators related to tooth evaluation based on the coordinates of the final feature points, and complete the automatic anterior tooth feature parameter extraction based on computer vision; the objective indicators related to tooth evaluation include the length and width value and length-to-width ratio of the teeth, the gingival apex height and the ratio of the gingival papilla height to the crown length.

[0087] It should be noted that, in step S11 of the present invention, the specific method for calculating the objective indicators related to tooth evaluation based on the coordinates of the final feature points is as follows: taking the upper left corner of the image after Gaussian blur as the origin, the tangent direction of the edge curve of the gingival part contour at any gingival vertex is taken as the x-axis, and the direction perpendicular to it is taken as the y-axis, the coordinate y value of the highest point of the tooth is subtracted from the coordinate y value of the lowest point of the tooth, and the absolute value of the difference in the y values ​​is multiplied by the scale of the patient's full-mouth oral scan image to obtain the length value of the tooth, the coordinate x value of the extreme point in the horizontal direction of the tooth is subtracted, and the absolute value of the difference in the x values ​​of the tooth is multiplied by the scale of the patient's full-mouth oral scan image to obtain the width value of the tooth, and the length value of the tooth is divided by the width value of the tooth to obtain the aspect ratio of the tooth, the coordinate y value of the highest point of the gum is subtracted from the coordinate y value of the lowest point of the gum, and the absolute value of the difference in the y values ​​of the gum is multiplied by the scale of the patient's full-mouth oral scan image to obtain the gingival apex height, and the gingival apex height is divided by the length value of the tooth to obtain the ratio of the gingival papilla height to the crown length.

[0088] It should be noted that in step S11 of the embodiment of the present invention, in order to evaluate the calculated objective indicators related to tooth evaluation, the above-mentioned objective indicators related to tooth evaluation can be obtained manually by using the 3Shape Unite software in an artificial fixed-point manner, and then the objective indicators related to tooth evaluation calculated by the present invention are compared with the objective indicators related to tooth evaluation obtained by the 3Shape Unite software to observe whether there are significant differences.

[0089] In an embodiment of the present invention, an automated anterior teeth evaluation system based on computer vision is also provided. Figure 7 As shown, it includes:

[0090] A data acquisition module is used to obtain the mesial gingival papilla height, distal gingival papilla height, gingival gingival source level, gingival color, gingival contour, tooth morphology, tooth color, tooth transparency and tooth shape contour;

[0091] The evaluation module is used to take the mesial gingival papilla height, distal gingival papilla height, gingival gingival source level, gingival color and gingival contour as evaluation indicators reflecting the degree of gingival condition, and take the tooth morphology, tooth color, tooth transparency and tooth shape contour as evaluation indicators reflecting the degree of tooth condition, objectively calculate the gingival color, height and contour characteristics as well as the tooth color, height, transparency and contour characteristics, automatically generate the corresponding tooth score through the specific difference values ​​between the gingival teeth and the opposite teeth with the same name, and complete the automated evaluation of the anterior teeth.

[0092] It should be noted that in the computer vision-based automated anterior teeth evaluation system of the present invention, the evaluation process of each evaluation index is as follows:

[0093] For the mesial papilla height, the height difference between the mesial papilla height of the current tooth and the mesial papilla height of the same tooth on the opposite side is taken as the first difference, and the mesial papilla height is scored based on the first difference and the preset first scoring standard.

[0094] Furthermore, if the absolute value of the first difference is greater than or equal to 0 and less than or equal to a1, 2 points are scored; if the absolute value of the first difference is greater than a1 and less than or equal to b1, 1 point is scored; otherwise 0 points are scored; wherein a1 and b1 represent the preset first threshold and second threshold, respectively.

[0095] For the distal gingival papilla height, the height difference between the distal gingival papilla height of the current tooth and the distal gingival papilla height of the same tooth on the opposite side is taken as the second difference, and the distal gingival papilla height is scored based on the second difference and the preset second scoring standard.

[0096] Furthermore, if the absolute value of the second difference is greater than or equal to 0 and less than or equal to a2, 2 points are scored; if the absolute value of the first difference is greater than a2 and less than or equal to b2, 1 point is scored; otherwise 0 points are scored; wherein a2 and b2 represent the preset third threshold and fourth threshold, respectively.

[0097] For the gingival origin level, the height difference between the gingival apex height of the current tooth and the gingival apex height of the same tooth on the opposite side is taken as the third difference, and the gingival origin level is scored based on the third difference and a preset third scoring standard.

[0098] Further, if the absolute value of the third difference is greater than or equal to 0 and less than or equal to a3, 2 points are scored; if the absolute value of the third difference is greater than a3 and less than or equal to b3, 1 point is scored; otherwise 0 points are scored; wherein a3 and b3 represent the preset fifth threshold and sixth threshold, respectively.

[0099] For gum color, a first processed image and a second processed image are obtained, wherein the first processed image is a partial image of the gums of the current tooth captured in the image after Gaussian blur, between the mesial gum papilla and the distal gum papilla and within a preset distance above the gum contour, and the second processed image is a partial image of the gums of the same tooth on the opposite side of the current tooth captured in the image after Gaussian blur, between the mesial gum papilla and the distal gum papilla and within a preset distance above the gum contour. The difference between the RGB average values ​​within the gum part in the first processed image and the second processed image is taken as the fourth difference, and the gum color is scored based on the fourth difference and the preset fourth scoring standard.

[0100] Further, if the absolute value of the fourth difference is greater than or equal to 0 and less than or equal to a4, 2 points are scored; if the absolute value of the fourth difference is greater than a4 and less than or equal to b4, 1 point is scored; otherwise 0 points are scored; wherein a4 and b4 represent the preset seventh threshold and eighth threshold, respectively.

[0101] For the gingival contour, the gingival contour of the current tooth between the mesial gingival papilla and the distal gingival papilla is obtained and used as the first gingival contour, and the gingival contour of the tooth with the same name on the opposite side of the current tooth between the mesial gingival papilla and the distal gingival papilla is obtained and used as the second gingival contour. m points are taken on the first gingival contour and the second gingival contour respectively to calculate the length and relative center position of the gingival contour, and the principal component analysis method is used to obtain the main direction. The length difference, relative center position difference and main direction difference between the first gingival contour and the second gingival contour are calculated, and the weighted average of the three differences is taken as the final difference. The gingival contour is scored according to the final difference and the preset fifth scoring standard.

[0102] Furthermore, if the final difference is greater than or equal to 0 and less than or equal to a5, 2 points are scored; if the final difference is greater than a5 and less than or equal to b5, 1 point is scored; otherwise 0 points are scored; wherein a5 and b5 represent the preset ninth threshold and tenth threshold, respectively.

[0103] For tooth morphology, the length-to-width ratio of the current tooth and the length-to-width ratio of the tooth with the same name on the opposite side are calculated, the smaller value is divided by the larger value and the quotient is taken, and the tooth morphology is scored based on this quotient and the preset sixth scoring standard.

[0104] Furthermore, if the quotient is greater than or equal to a6 and less than or equal to 1, 2 points are scored; if the quotient is greater than or equal to b6 and less than a6, 1 point is scored; otherwise, 0 points are scored; wherein a6 and b6 represent the preset eleventh threshold and twelfth threshold, respectively.

[0105] For tooth color, a third processed image and a fourth processed image are obtained, wherein the third processed image is a partial image of the current tooth captured in the image after Gaussian blur, and the fourth processed image is a partial image of the tooth with the same name on the opposite side of the current tooth captured in the image after Gaussian blur. The difference between the RGB average values ​​in the third processed image and the fourth processed image is taken as the fifth difference, and the tooth color is scored based on the fifth difference and the preset seventh scoring standard.

[0106] Further, if the absolute value of the fifth difference is greater than or equal to 0 and less than or equal to a7, 2 points are scored; if the absolute value of the fifth difference is greater than a7 and less than or equal to b7, 1 point is scored; otherwise 0 points are scored; wherein a7 and b7 represent the preset thirteenth threshold and fourteenth threshold, respectively.

[0107] For tooth transparency, the difference between the average values ​​of the A values ​​in the third processed image and the fourth processed image is taken as the sixth difference, and the tooth transparency is scored according to the sixth difference and the preset eighth scoring standard. Further, if the absolute value of the sixth difference is greater than or equal to 0 and less than or equal to a8, 2 points are scored, if the absolute value of the sixth difference is greater than a8 and less than or equal to b8, 1 point is scored, otherwise 0 points are scored; wherein a8 and b8 represent the preset fifteenth threshold and sixteenth threshold, respectively.

[0108] For the tooth contour, the partial tooth contour of the current tooth is taken as the first tooth contour, and the partial tooth contour of the tooth with the same name on the opposite side of the current tooth is taken as the second tooth contour, and n points are taken on the first tooth contour and the second tooth contour respectively to calculate the histogram of the polar coordinates of the two tooth contours, and then the Euclidean distance is used to compare the two histograms to obtain the shape difference of the two tooth contours, and the tooth contour is scored based on the shape difference and the preset ninth scoring standard.

[0109] Furthermore, if the shape difference is greater than or equal to 0 and less than or equal to a9, 2 points are scored; if the shape difference is greater than a9 and less than or equal to b9, 1 point is scored; otherwise, 0 points are scored; wherein a9 and b9 represent the preset seventeenth and eighteenth thresholds, respectively.

[0110] It should be noted that in the automated anterior teeth evaluation system of the embodiment of the present invention, thresholds are set for the above evaluation indicators based on the evaluation of 20 teeth by doctors with senior clinical experience and the difference in the objective values ​​of each indicator to reflect the status of teeth and gums. In order to evaluate the automatically generated tooth scores, they can be compared with the scores of experienced prosthodontists to observe whether there are significant differences.

[0111] Figure 2 This is a schematic diagram of a real patient's full-mouth oral scan image. This embodiment uses this image for practice. Figure 1 As shown in the flowchart, the teeth and gum images captured according to the user's clicked position are as follows Figure 3 As shown, Figure 3 Take the middle tooth as an example, the grayscale image of the tooth before and after de-reflection is compared. Figure 4 The present invention uses two methods to obtain the outline diagram, as shown in Figure 5 shown. Figure 6 The tooth feature points and gum feature points found by the present invention are displayed, and the subsequent objective index measurement and automatic scoring are completed according to the coordinates of these feature points. The results are shown in Table 1.

[0112] Table 1. Automated scoring results

[0113] Tooth number 13 12 11 21 22 23 Aspect Ratio 2.13 1.36 1.32 1.21 1.28 1.94 PES mesiogingival papilla 2 2 2 2 2 2 PES distal papilla 1 2 2 2 2 1 PES gingival source level 1 0 1 1 0 1 PES Soft Tissue Contouring 2 0 1 1 0 2 PES soft tissue color 1 2 0 0 2 1 WES restoration color 2 1 0 0 1 2 WES restoration morphology 2 2 2 2 2 2 WES restoration contour 1 1 2 2 1 1 WES restoration transparency 1 0 2 2 0 1

[0114] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for automatic anterior teeth feature parameter extraction based on computer vision, characterized in that: The following steps are involved: S1. Obtain a full-mouth oral scan image of a patient to be processed, and visualize the full-mouth oral scan image of the patient to display it in front of a user; S2. When the user clicks on the tooth to be scored, the corresponding tooth part image is automatically captured in the patient's full-mouth oral scan image according to the click position, and each tooth part image includes the tooth part and the gum part; S3. Applying Gaussian blur method to each tooth partial image to eliminate the specular reflection therein, and obtaining a Gaussian blurred image; S4. Using the color segmentation method based on the HSV value of the image after Gaussian blur, set thresholds for the three channels H, S, and V respectively, separate the teeth from the gums in each tooth partial image, and obtain an initial tooth image and an initial gum image for each tooth partial image; S5. Performing morphological processing on the initial tooth image and the initial gum image to obtain a morphologically processed tooth image and a morphologically processed gum image; S6. extracting the tooth part contour from the morphologically processed tooth image and the gingival part contour from the morphologically processed gingival image using a contour extraction method, and adding conditional rules to adjust the tooth part contour and the gingival part contour according to user feedback; S7. The final gingival contour and tooth bottom edge contour are obtained by using the Chan-Vese active contour model algorithm. The gradient value of each pixel point is calculated point by point on the final gingival contour and tooth bottom edge contour according to the direction of the corresponding contour edge curve. The gingival papilla vertex and the gingival vertex are obtained in combination with the actual position of each pixel point in the image after Gaussian blur. The pixel point with the smallest gradient value on the edge curve of the tooth bottom edge contour is used as the boundary point on both sides of the tooth bottom. The intersection point of the tooth contour and the gingival contour and the tooth bottom boundary point are connected to obtain the boundary line between the teeth. S8. Further refine and improve the accuracy of the tooth contour through the energy function within the tooth contour, so that the transition between adjacent edges of the teeth is smoother; S9. Apply Shi-Tomasi corner point detection method to extract corner points in the outline of gum part and tooth part respectively, cluster the extracted tooth part corner points using K-means clustering algorithm to obtain the clustered center point of tooth part, cluster the extracted gum part corner points using K-means clustering algorithm to obtain the clustered center point of gum part; S10. extracting other important feature points for evaluation in the tooth contour and the gum contour by curve fitting and extreme value finding methods, wherein the other important feature points for evaluation include the highest point and the lowest point of the tooth and the extreme value point of the tooth in the horizontal direction; S11. Taking the center point of the clustered tooth part, the center point of the clustered gum part and other important evaluation feature points as the final feature points, obtaining the coordinates of all the final feature points, calculating the objective indicators related to tooth evaluation based on the coordinates of the final feature points, and completing the automatic anterior tooth feature parameter extraction based on computer vision; The objective indicators related to tooth evaluation include the length and width value and length-to-width ratio of the tooth, the height of the gingival apex, and the ratio of the gingival papilla height to the crown length.

2. The method for automatic anterior teeth feature parameter extraction based on computer vision as claimed in claim 1, characterized in that: In step S5, the morphological processing includes an opening operation and a closing operation, wherein the opening operation is used to remove small dots in the tooth image or the gum image, and the closing operation is used to fill holes in the tooth image or the gum image.

3. The method for automatic anterior teeth feature parameter extraction based on computer vision as claimed in claim 1, characterized in that: In step S6, the specific process of contour extraction using the contour extraction method is: using Gaussian filtering and Canny edge detection methods to extract the initial tooth part contour, and then using image expansion and image skeletonization methods to enhance and refine the initial tooth part contour; using Gaussian filtering and Canny edge detection methods to extract the initial gum part contour, and then using image expansion and image skeletonization methods to enhance and refine the initial gum part contour.

4. The method for automatic anterior teeth feature parameter extraction based on computer vision as claimed in claim 1, characterized in that: In step S7, the specific process of obtaining the final gingival part contour and the bottom edge contour of the tooth by the Chan-Vese active contour model algorithm is as follows: For the gingival contour, the gingival contour is used as the initial value of the Chan-Vese active contour model algorithm. In each iteration, the inner region and the outer region are formed with the gingival contour as the boundary. The grayscale mean of the inner region and the grayscale mean of the outer region are calculated respectively. The gradient descent method is used to minimize the grayscale mean of the inner region and the grayscale mean of the outer region and minimize the difference between the inner region and the outer region to guide the update of the gingival contour, and the update amplitude is controlled to meet the update only near the boundary of the gingival contour. Repeat the iterations continuously and gradually adjust the gingival contour until a preset maximum number of iterations is reached or the change in the gingival contour is less than a preset contour threshold, thereby obtaining a final gingival contour; For the partial tooth contour, the background part at the bottom is extracted from the morphologically processed tooth image and used as the initial value of the Chan-Vese active contour model algorithm. The iteration is repeated to obtain the bottom edge contour of the tooth.

5. The method for automatic anterior teeth feature parameter extraction based on computer vision as claimed in claim 1, characterized in that: In step S8, the energy function The specific form is as follows: Among them, E penal represents the penalty term that limits the complexity of the contour; E grad represents the gradient term; E color represents the color change term; λ0, λ1, λ2 represent the contribution value of each energy function respectively.

6. The method for automatic anterior teeth feature parameter extraction based on computer vision as claimed in claim 1, characterized in that: In step S11, the specific method for calculating the objective indicators related to tooth evaluation based on the coordinates of the final feature points is as follows: taking the upper left corner of the image after Gaussian blur as the origin, the tangent direction of the edge curve of the gingival part contour at any gingival vertex is taken as the x-axis, and the direction perpendicular to it is taken as the y-axis, subtracting the coordinate y value of the highest point of the tooth from the coordinate y value of the lowest point of the tooth, multiplying the absolute value of the difference in the y values ​​by the scale of the patient's full-mouth oral scan image to obtain the length value of the tooth, subtracting the coordinate x value of the extreme point in the horizontal direction of the tooth, multiplying the absolute value of the difference in the x values ​​of the tooth by the scale of the patient's full-mouth oral scan image to obtain the width value of the tooth, dividing the length value of the tooth by the width value of the tooth to obtain the aspect ratio of the tooth, subtracting the coordinate y value of the highest point of the gum from the coordinate y value of the lowest point of the gum, multiplying the absolute value of the difference in the y values ​​of the gum by the scale of the patient's full-mouth oral scan image to obtain the gingival apex height, and dividing the gingival apex height by the length value of the tooth to obtain the ratio of the gingival papilla height to the crown length.

7. An automated anterior teeth evaluation system based on computer vision, characterized in that: include: A data acquisition module is used to obtain the mesial gingival papilla height, distal gingival papilla height, gingival gingival source level, gingival color, gingival contour, tooth morphology, tooth color, tooth transparency and tooth shape contour; The evaluation module is used to take the mesial gingival papilla height, distal gingival papilla height, gingival gingival source level, gingival color and gingival contour as evaluation indicators reflecting the degree of gingival condition, and take the tooth morphology, tooth color, tooth transparency and tooth shape contour as evaluation indicators reflecting the degree of tooth condition, objectively calculate the gingival color, height and contour characteristics as well as the tooth color, height, transparency and contour characteristics, automatically generate the corresponding tooth score through the specific difference values ​​between the gingival teeth and the opposite teeth with the same name, and complete the automated evaluation of the anterior teeth.

8. The computer vision-based automated anterior teeth evaluation system according to claim 7, characterized in that: The evaluation process of each evaluation indicator is as follows: For the mesial papilla height, the height difference between the mesial papilla height of the current tooth and the mesial papilla height of the tooth with the same name on the opposite side is taken as the first difference, and the mesial papilla height is scored according to the first difference and the preset first scoring standard; For the distal gingival papilla height, the height difference between the distal gingival papilla height of the current tooth and the distal gingival papilla height of the tooth with the same name on the opposite side is taken as the second difference, and the distal gingival papilla height is scored according to the second difference and the preset second scoring standard; For the gingival origin level, the height difference between the gingival apex height of the current tooth and the gingival apex height of the tooth with the same name on the opposite side is taken as the third difference, and the gingival origin level is scored according to the third difference and the preset third scoring standard; For gum color, a first processed image and a second processed image are obtained, wherein the first processed image is a partial image of the gums of the current tooth captured in the image after Gaussian blurring, between the mesial and distal gum papillae and within a preset distance above the gum contour; the second processed image is a partial image of the gums of the tooth with the same name on the opposite side of the current tooth captured in the image after Gaussian blurring, between the mesial and distal gum papillae and within a preset distance above the gum contour; the difference between the RGB average values ​​within the gum part in the first processed image and the second processed image is taken as a fourth difference, and the gum color is scored according to the fourth difference and a preset fourth scoring standard; For the gingival contour, the gingival contour of the current tooth between the mesial gingival papilla and the distal gingival papilla is obtained as the first gingival contour, the gingival contour of the tooth with the same name on the opposite side of the current tooth between the mesial gingival papilla and the distal gingival papilla is obtained as the second gingival contour, m points are respectively taken on the first gingival contour and the second gingival contour to calculate the length and relative center position of the gingival contour, and the principal direction is obtained by using the principal component analysis method, and the length difference, relative center position difference and principal direction difference between the first gingival contour and the second gingival contour are calculated, and the weighted average of the three differences is taken as the final difference, and the gingival contour is scored according to the final difference and the preset fifth scoring standard; For tooth morphology, the length-width ratio of the current tooth and the length-width ratio of the tooth with the same name on the opposite side are calculated, the smaller value of the two is divided by the larger value and the quotient is taken, and the tooth morphology is scored based on the quotient and the preset sixth scoring standard; For tooth color, a third processed image and a fourth processed image are obtained, wherein the third processed image is a tooth partial image of the current tooth captured in the image after Gaussian blur, and the fourth processed image is a tooth partial image of the tooth with the same name on the opposite side of the current tooth captured in the image after Gaussian blur, and the difference between the RGB average values ​​in the third processed image and the fourth processed image is used as a fifth difference, and the tooth color is scored according to the fifth difference and a preset seventh scoring standard; For tooth transparency, the difference between the average values ​​of the A values ​​in the third processed image and the fourth processed image is taken as the sixth difference value, and the tooth transparency is scored according to the sixth difference value and the preset eighth scoring standard; For the tooth contour, the partial tooth contour of the current tooth is taken as the first tooth contour, and the partial tooth contour of the tooth with the same name on the opposite side of the current tooth is taken as the second tooth contour, and n points are taken on the first tooth contour and the second tooth contour respectively to calculate the histogram of the polar coordinates of the two tooth contours, and then the Euclidean distance is used to compare the two histograms to obtain the shape difference of the two tooth contours, and the tooth contour is scored based on the shape difference and the preset ninth scoring standard.

Images