Dental caries early warning system based on image recognition

Abstract

The invention relates to a decayed tooth early warning system based on image recognition in the technical field of oral examination. An image preprocessing module; the feature extraction module comprises a textural feature extraction module, a color feature extraction module and a morphological feature extraction module which are used for extracting textural features, color features and morphological features of each extracted tooth image; the decayed tooth detection module is used for identifying a decayed tooth area according to the extracted texture features, color features and morphological features through a decayed tooth detection model based on a convolutional neural network, and judging whether the type of the decayed tooth is light decayed tooth, medium decayed tooth or deep decayed tooth; the data acquisition module is used for acquiring user data; the risk assessment module is used for predicting the risk probability that the user suffers from dental caries within 1-3 months in the future through a dental caries risk model and giving a corresponding risk grade prompt; and the result generation module is used for generating oral care suggestions according to the comparison result. According to the scheme, early-stage accurate detection and risk early warning of decayed teeth are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of oral examination, in particular to a caries early warning system based on image recognition. BACKGROUND

[0002] Caries is a common oral disease, if not discovered and treated in time, it will lead to tooth pain, infection and even tooth loss. Traditional caries detection methods mainly rely on clinical examination and X-ray by dentists. Clinical examination is greatly influenced by the subjective experience of the doctor, and for early caries, its symptoms are not obvious, and it is easy to miss diagnosis. Although X-ray can assist in detection, it has radiation risk, high equipment cost and inconvenience in use, and is not suitable for daily frequent oral health monitoring. With the popularity of smart phones and image processing technology, caries detection using oral images has become a new research direction. However, the existing image-based caries detection technology still has many problems, such as low detection accuracy, poor adaptability to complex oral environment, and inability to achieve early and accurate warning. In the patent with publication number CN 118983086 A, a caries recognition analysis and early warning method and system based on tooth images are disclosed, which relates to the technical field of caries recognition analysis and early warning. User tooth data is extracted from the user's tooth image to generate a user tooth feature vector value. The user tooth feature vector value is analyzed with the caries feature vector threshold to mark and evaluate whether the user has caries risk. For the condition that the marked user has caries risk, the caries risk level is further identified, and an alarm is given according to the caries risk level. For the condition that the user does not have caries risk, the user's historical tooth data is obtained, and the user's historical tooth data is analyzed in time sequence by using a recurrent neural network to predict the probability of the user having caries risk, and finally a caries risk detection report of the user is generated. Personalized caries risk assessment and real-time early warning are realized, and comprehensive caries management suggestions are provided through historical data analysis and prediction. However, the features of early caries are often very weak, such as local enamel demineralization in the early stage of shallow caries, which may only show slight color change and texture change. The pre-processing method in the above patent, which adjusts image brightness and contrast, image denoising and image size normalization, cannot highlight the local enamel demineralization feature in the early stage of shallow caries, so it may lead to inaccurate early warning of early shallow caries. SUMMARY

[0003] The present application aims to provide a caries early warning system based on image recognition to realize early and accurate detection and risk warning of caries.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solution: a caries early warning system based on image recognition, comprising a server, the server comprising: an image acquisition module for acquiring tooth images; An image preprocessing module for extracting a tooth region image in the oral cavity and performing image processing of enhancing overall contrast and denoising the tooth image; A feature extraction module including a texture feature extraction module, a color feature extraction module and a morphological feature extraction module for extracting texture features, color features and morphological features of each tooth image extracted; The texture feature extraction module is configured to calculate texture features thereof by using a gray level co-occurrence matrix algorithm, and the texture features include smoothness of tooth enamel and texture changes formed by plaque adhesion; The color feature extraction module is configured to convert the tooth image from an RGB color space to an HSV color space, analyze color distribution of three channels of hue H, saturation S and brightness V respectively, and count and calculate proportions of pixel numbers of different color values in each channel. The morphological feature extraction module is configured to obtain a contour of the tooth by an edge detection algorithm, and calculate a contour perimeter, an area and a circularity. The caries detection module is configured to identify a caries region in the tooth image by using a caries detection model based on a convolutional neural network and judge a severity of the caries as shallow caries, moderate caries or deep caries. The data collection module is configured to collect user data, and the user data includes age, eating habits, oral hygiene habits and historical detection data of the teeth. The risk assessment module is configured to use a caries risk model based on a logistic regression algorithm to predict a risk probability of the user suffering from caries in the future 1-3 months and give a corresponding risk level prompt. The result generation module is configured to compare a detection result with pre-recorded risk results and oral care suggestions corresponding to the risk results, and generate oral care suggestions according to a comparison result.

[0005] Preferably, as an improvement, the tooth region in the oral cavity is extracted by the following steps: S1, inputting an oral cavity image, the oral cavity image being a 3-channel RGB image with a size of 512x512 pixels, and gradually extracting deep features of the image through convolution layers and max-pooling layers; S2, convolution layer calculation, each convolution operation being performed through a preset convolution kernel with a size of 3x3 and a sliding window convolution with an input feature map, and a formula being: F I

[0006] wherein W is a convolution kernel weight, I is an input image region, b is a bias term, and an output feature map F is used to capture local features such as edges and textures.

[0007] S3, pooling layer calculation: through max pooling, the maximum value within a 2×2 window is taken, reducing the feature map size, preserving key features while reducing computation, enhancing the network's robustness to image scale changes, and obtaining a high-dimensional feature map. S4, the high-dimensional feature map is gradually restored to the input image size through deconvolution layers, while shallow features from the corresponding layers in the encoding stage are fused to compensate for the spatial location information lost during pooling. The deconvolution calculation formula is as follows: F' ·F ' in Step size, To output position coordinates, ensuring that the feature map accurately locates the tooth region during the recovery process; S5, finally, through a 1×1 convolutional layer and a Softmax activation function, classifies each pixel in the image into two categories: teeth and background, and outputs a segmentation mask with the same size as the input image. P

[0008] in For pixels The logits value belonging to category c, C=2 (teeth / background), when P(teeth) When the threshold is greater than the threshold, the pixel is marked as a tooth region.

[0009] Preferably, as an improvement, texture feature extraction includes the following steps: A1, Gray Level Compression, compresses the gray levels from 0-255 of the original image to 16 levels. The formula is: G 压缩 =⌊ ⌋ Among them G 原始 G represents the original grayscale value. 压缩 The value is the compressed grayscale value, which ranges from 0 to 15. ⌊·⌋ indicates rounding down. A2, construct the GLCM matrix by selecting four directions (0°, 45°, 90°, 135°) and distances of 1-2 pixels, and statistically analyze the grayscale pairs in the image. The co-occurrence probability P ; A3 calculates texture feature values ​​and energy, reflecting texture uniformity: Formula for calculating texture uniformity: E= ; Texture complexity calculation formula: H=- , The value of this item is 0 at that time; Texture definition formula: C=

[0010] A4, Local Binary Pattern (LBP) extraction: A401, define the field, take a 3x3 neighborhood centered on each pixel, the center pixel gray value is gc , the neighborhood pixel gray value is g 0~7; g A402, calculate the LBP value, generate a binary number through threshold comparison, and get the LBP value after conversion to decimal: LBP=

[0011] Where s

[0012] A403, generate LBP histogram, count the distribution of LBP values of all pixels in the image to form a histogram as a texture feature.

[0013] Preferably, as an improvement, the tooth image is converted from the RGB color space to the HSV color space, and the formula is: R, G, B ∈ [0, 1]; Brightness V = max (R, G, B); Saturation S= ; Hue H= ; Statistical histogram distribution of H channel, calculate the mean and standard deviation of S channel, analyze the brightness change of V channel, and identify color abnormalities caused by caries.

[0014] Preferably, as an improvement, the morphological feature extraction includes the following steps: B1, calculate the contour perimeter, first get the contour coordinates, get the tooth contour through Canny edge detection, record the pixel point coordinates ,… ; Accumulate the Euclidean distance of adjacent pixels, the formula is: L= + ; B2, calculate the area, based on the contour coordinates, the area calculation formula is: S= Where = ; B3, calculate the circularity, combine the perimeter and area, the formula is: C=​ ; B4, calculate the position relationship of adjacent teeth, first obtain the center point of each tooth through the contour centroid formula and , calculate the distance between the center points of adjacent teeth, the formula is: D= .

[0015] Preferably, as an improvement, the dietary habit is the frequency of sugar intake; the oral hygiene habit includes the frequency of brushing teeth, and whether to use dental floss.

[0016] Preferably, as an improvement, the image acquisition module includes a smart phone or an oral cavity special camera connected to the terminal device by an electrical signal.

[0017] The beneficial effects of the present application are: 1. Compared with the prior art, the present scheme greatly improves the accuracy of caries detection by combining various image processing and analysis techniques and training a deep learning model, can accurately identify the subtle features of early caries, and reduce missed diagnosis and misdiagnosis.

[0018] 2. Not only can it detect whether there is caries at present, but also can predict the risk of users suffering from caries in advance based on a multi-factor risk assessment model, realize early warning, and let users take preventive measures before caries occurs.

[0019] 3. Support using common smart phones and other devices for image acquisition, users can monitor their oral health at any time and anywhere, without going to the hospital or professional institutions, saving time and cost.

[0020] 4. According to the personal information and detection history of the user, personalized early warning and oral care suggestions are provided for the user, improving the user's oral health management effect.

[0021] 5. Traditional visual inspection relies on doctors' naked eye observation, and it is difficult to detect subtle texture changes in enamel, with a missed diagnosis rate of early caries as high as 40%. In the present scheme, the texture features are extracted to quantify the microscopic texture through algorithm, which can capture the invisible demineralization signs, and the detection rate of early caries is improved to more than 85%, which saves time for caries intervention.

[0022] 6. Through color feature extraction, it helps to judge the caries stage, shallow caries only shows slight color shift, medium caries exposes dentin, color deepens, and deep caries is accompanied by pulp infection, color is brown-black. Although traditional X-ray film can show the depth of caries, it cannot quantify color changes, and color features can complement X-ray film and improve the grading accuracy.

[0023] 7、In the traditional examination, the interproximal caries is difficult to be accurately judged by naked eyes and X-ray film due to the hidden position. In the scheme, the extraction of morphological features can find the change of the gap between teeth caused by interproximal caries through the calculation of the relative position relationship of teeth, such as the distance between the center points of adjacent teeth: the normal tooth spacing is about 20-30 pixels, and the spacing may be expanded to 40-50 pixels due to the tooth defect caused by interproximal caries, which can effectively identify interproximal caries and improve the detection rate by more than 35%. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The logic block diagram of the caries early warning system based on image recognition. DETAILED DESCRIPTION

[0025] The following will be further described in detail through specific embodiments: The embodiment is basically as shown in the accompanying drawings: Figure 1 The caries early warning system based on image recognition comprises: An image acquisition module, which comprises a smart phone or an oral cavity special camera connected with a terminal device by an electrical signal, in use, the user directs the camera of the phone to the oral cavity, adjusts the angle to make the teeth completely enter the shooting picture, ensures that the light uniformly illuminates the oral cavity, and clicks the shooting button to obtain the oral cavity image.

[0026] An image preprocessing module, which is used for processing the tooth image, enhancing the overall contrast of the tooth image, performing denoising processing, and then accurately segmenting the tooth region in the oral cavity image from the background cavity mucosa and tongue through a semantic segmentation algorithm based on deep learning; Specifically comprising the following steps: S1, inputting an oral cavity image, a 3-channel RGB image, the size of which is 512*512 pixels, gradually extracting deep features of the image through convolution layers and maximum pooling layers; S2, convolution layer calculation, each convolution operation is performed through a preset convolution kernel with a size of 3*3 and a sliding window convolution with an input feature map, and the formula is: F I

[0027] Wherein, W is the convolution kernel weight, I is the input image region, b is the bias term, and the output feature map F is used to capture local features such as edges and textures.

[0028] S3, pooling layer calculation, through maximum pooling, the maximum value in a 2*2 window is taken, the feature map size is reduced, the calculation amount is reduced while the key features are reserved, the robustness of the network to image scale changes is enhanced, and a high-dimensional feature map is obtained; ​S4, the high-dimensional feature map is gradually restored to the input image size through the deconvolution layer, while the shallow features of the corresponding level in the encoding stage are fused to make up for the loss of spatial position information in the pooling process, and the deconvolution calculation formula is: F' ·F ' wherein is the step size, is the output position coordinate, ensuring accurate positioning of the tooth region during feature map restoration; S5, finally, through the 1x1 convolution layer and the Softmax activation function, each pixel in the image is classified, including two categories of teeth and background, and a segmentation mask with the same size as the input image is output; P

[0029] wherein is the logits value of pixel belonging to class c, C=2 (teeth / background), when P (teeth ) > threshold, the pixel is marked as a tooth region.

[0030] The feature extraction module is configured to extract texture features, color features, and morphological features from each segmented tooth image.

[0031] The texture feature extraction module is configured to calculate texture features of the tooth using a gray level co-occurrence matrix algorithm, wherein the texture features include smoothness of tooth enamel and texture changes caused by plaque adhesion. The texture feature extraction includes the following steps: A1, compress the gray level, compress the original image 0-255 gray value to 16 levels, the formula is: G 压缩 =⌊ ⌋ wherein G 原始 is the original gray value, G 压缩 is the compressed gray value, the gray value range is (0-15), and ⌊·⌋ represents rounding down; A2, construct GLCM matrix, select 0°, 45°, 90°, 135° four directions and 1-2 pixel distance, and count the co-occurrence probability P of the gray value pair in the image; A3, calculate the texture feature value, energy, which reflects the texture uniformity: Texture uniformity calculation formula: E= ; ​Texture complexity calculation formula: H=- , 0 when the item is; Texture clarity calculation formula: C=

[0032] A4, Local Binary Pattern (LBP) extraction: A401, define the field, take 3x3 neighborhood centered on each pixel, the center pixel gray value is gc , the neighborhood pixel gray value is g 0~ 7; g A402, calculate the LBP value, generate binary number through threshold comparison, convert to decimal to get LBP value: LBP=

[0033] Where s

[0034] A403, generate LBP histogram, count the distribution of LBP values of all pixels in the image to form a histogram as a texture feature.

[0035] Color feature extraction module, used to convert the tooth image from RGB color space to HSV color space, analyze the color distribution of hue H, saturation S, and brightness V three channels respectively, and count the number and proportion of pixels of different color values in each channel; Convert the tooth image from RGB color space to HSV color space, the calculation formula is: R, G, B∈[0,1]; Brightness V = max (R, G, B); Saturation S= ; Hue H= ; Statistical histogram distribution of H channel, calculate the mean and standard deviation of S channel, analyze the brightness change of V channel, and identify the color anomaly caused by dental caries.

[0036] Morphological feature extraction module, used to obtain the contour of the tooth through edge detection algorithm, and then calculate the contour perimeter, area, circularity and adjacent tooth position relationship, the specific steps include: B1, calculate the contour perimeter, first get the contour coordinates, get the tooth contour through Canny edge detection, record the pixel point coordinates ,… ; Accumulate the Euclidean distance of adjacent pixel points, the formula is:​ L= + ; B2, calculate the area based on the contour coordinates, the area calculation formula is: S= wherein = ; B3, calculate the circularity, combine the circumference and the area, the formula is: C= ; B4, calculate the position relationship of adjacent teeth, first get the center point of each tooth through the contour centroid formula and , calculate the distance between the center points of adjacent teeth, the formula is: D= .

[0037] The caries detection module is configured to identify a caries region in the tooth image by using the extracted texture features, color features and morphological features through a caries detection model based on a convolutional neural network, and determine the severity of the caries as shallow caries, medium caries or deep caries.

[0038] The data acquisition module is configured to acquire user age, sugar intake frequency, tooth brushing frequency, whether to use dental floss and historical detection data of teeth, and use a caries risk model based on a logistic regression algorithm to predict the risk probability of the user suffering from caries in a future period of time by combining the caries risk model with the acquired user age, sugar intake frequency, tooth brushing frequency, whether to use dental floss and historical detection data of teeth, and give a corresponding risk level prompt. The result generation module is configured to compare the detection result with pre-recorded risk results and corresponding oral care suggestions based on the comparison result to generate oral care suggestions.

[0039] The above is only an embodiment of the present application, and the specific technical solutions and / or common knowledge of the scheme are not described in detail. It should be noted that for those skilled in the art, without departing from the technical solutions of the present application, some modifications and improvements can be made, which should also be considered as the protection scope of the present application, which will not affect the effect and practicality of the present application. The protection scope of the present application should be subject to the content of its claims, and the specific embodiments in the specification can be used to explain the content of the claims.

Claims

1. A caries warning system based on image recognition, characterized in that, The server comprises: An image acquisition module for acquiring tooth images; An image preprocessing module for extracting tooth region images in the oral cavity and performing image processing of enhancing overall contrast and removing noise on the tooth images; A feature extraction module comprising a texture feature extraction module, a color feature extraction module and a morphological feature extraction module for extracting texture features, color features and morphological features of each tooth image extracted; The texture feature extraction module is configured to calculate texture features of the tooth images by using a gray level co-occurrence matrix algorithm, wherein the texture features include smoothness of tooth enamel and texture changes caused by plaque adhesion; The color feature extraction module is configured to convert the tooth images from an RGB color space to an HSV color space, analyze color distribution of three channels of hue H, saturation S and brightness V respectively, and count pixel numbers and proportions of different color values in each channel; The morphological feature extraction module is configured to obtain a contour of the tooth by using an edge detection algorithm, and calculate a contour perimeter, an area and a circularity of the tooth; A caries detection module for identifying a caries region in the tooth images by using a caries detection model based on a convolutional neural network, and judging a severity of the caries as shallow caries, moderate caries or deep caries; A data acquisition module for acquiring user data, wherein the user data includes age, eating habits, oral hygiene habits and historical detection data of the teeth; A risk assessment module for predicting a risk probability of the user suffering from caries in the future 1-3 months by using a caries risk model based on a logistic regression algorithm and combining the user data, and giving a corresponding risk level prompt; A result generation module for comparing a detection result with pre-recorded risk results and oral care suggestions corresponding to the risk results, and generating oral care suggestions according to a comparison result.

2. The image recognition based early warning system for dental caries as claimed in claim 1 wherein: The tooth region in the oral cavity is extracted by the following steps: S1, inputting an oral image, wherein the oral image is a 3-channel RGB image with a size of 512*512 pixels, and deep features of the image are gradually extracted through convolution layers and maximum pooling layers; S2, convolution layer calculation, wherein each convolution operation is performed through a preset convolution kernel with a size of 3*3 and a sliding window convolution with an input feature map, and a formula is as follows: F I wherein, W is a convolution kernel weight, I is an input image region, b is a bias term, and an output feature map F is used to capture local features such as edges and textures. S3, pooling layer calculation, wherein a maximum value in a 2*2 window is taken through maximum pooling, a feature map size is reduced, key features are reserved, a calculation amount is reduced, robustness of a network to image scale changes is enhanced, and a high-dimensional feature map is obtained; S4, the high-dimensional feature map is gradually restored to an input image size through a deconvolution layer, shallow features of a corresponding level in an encoding stage are fused, spatial position information lost in a pooling process is made up, and a deconvolution calculation formula is as follows: F • F ' wherein is a step size, is an output position coordinate, ensuring that the feature map is accurately positioned in the tooth region during the restoration process; S5, finally, 1*1 convolution layers and a Softmax activation function are used to classify each pixel in the image, classes include two classes of teeth and background, and a segmentation mask with the same size as the input image is outputted; P wherein is a pixel logits value belonging to class c, C = 2 (tooth / background), when P(tooth ) > threshold, the pixel is labeled as tooth region.

3. The image recognition based early warning system for dental caries as claimed in claim 2 wherein: The texture feature extraction includes the following steps: A1, compress the gray level, compress the original image 0-255 gray value to 16 levels, the formula is: G 压缩 =⌊ ⌋ where G 原始 is the original gray value, G 压缩 is the compressed gray value, the gray value is in the range of 0-15, and ⌊·⌋ represents rounding down; A2, the GLCM matrix is constructed by selecting four directions of 0°, 45°, 90° and 135° and a distance of 1-2 pixels, and the co-occurrence probability P of the gray value pairs in the image is counted ;​ A3, calculate the texture feature value, energy, reflect the texture uniformity: Texture uniformity calculation formula: E= ; Texture complexity calculation formula: H=- , when this term is 0; Texture clarity calculation formula: C= A4, local binary pattern (LBP) extraction: A401, define the field, take 3x3 neighborhood, center pixel gray value is gc 0~7 g , neighborhood pixel gray value is g 0~7 A402, calculate the LBP value, generate binary number through threshold comparison, convert to decimal to get LBP value: LBP= wherein s A403, generate LBP histogram, count the distribution of LBP value of all pixels in the image, form the histogram as texture feature.

4. The image recognition based early warning system for dental caries as claimed in claim 3 wherein: Convert the tooth image from RGB color space to HSV color space, the calculation formula is: R, G, B∈[0,1]; Brightness V=max(R,G,B); Saturation S= ; Hue H = 0 ; Statistical H channel histogram distribution, calculate the mean and standard deviation of S channel, analyze the brightness change of V channel, identify the color anomaly caused by dental caries.

5. The image recognition based early warning system for dental caries as claimed in claim 4 wherein: The morphological feature extraction includes the following steps: B1, calculate the contour perimeter, first get the contour coordinates, get the tooth contour through Canny edge detection, record the pixel point coordinates ,... ; Accumulate the Euclidean distance of adjacent pixels, the formula is: L= + ； B2, calculate the area, based on the contour coordinates, the area calculation formula is: S= wherein = ; B3, calculate the circularity, combine the perimeter and area, the formula is: C= ； B4, calculate the adjacent tooth position relationship, first get the center point of each tooth by the contour centroid formula and , calculate the adjacent tooth center point distance, the formula is: D= 。 6. The image recognition based early warning system for dental caries as claimed in claim 5 wherein: The eating habits are sugar intake frequency; The oral hygiene habits include tooth brushing frequency, whether to use dental floss.

7. The image recognition based early warning system for dental caries as claimed in claim 6 wherein: The image acquisition module includes a smartphone or an oral cavity special camera electrically connected with the terminal device.

Images