Multi-feature fusion segmentation and quantitative analysis method and system for oral cavity fluorescence image, terminal and medium

Abstract

The invention discloses a multi-feature fusion segmentation and quantitative analysis method and system for an oral cavity fluorescence image, a terminal and a medium, and the method comprises the steps: obtaining the oral cavity fluorescence image, carrying out the preprocessing of the oral cavity fluorescence image, carrying out the region segmentation of the preprocessed oral cavity fluorescence image based on a semantic segmentation network, obtaining a tooth area in the oral cavity fluorescence image; determining a suspected dental plaque region, extracting multi-dimensional features of the suspected dental plaque region, and inputting the multi-dimensional features to the dental plaque classification neural network to obtain a classification result of the suspected dental plaque region, the multi-dimensional features including color features, texture features, morphological features and spatial features; and performing morphological post-processing on the classification result, determining a dental plaque distribution diagram according to the classification result, and outputting a quantitative analysis result. The dental plaque detection system and method are suitable for household daily oral care monitoring, dental clinic auxiliary diagnosis and other scenes, and rapid, accurate and non-invasive detection of dental plaque and quantitative evaluation of tooth health conditions are achieved.

Description

Technical Field

[0001] This invention relates to the field of oral health detection technology, and in particular to a method, system, terminal, and medium for multi-feature fusion segmentation and quantitative analysis of oral fluorescence images. Background Technology

[0002] Dental plaque is a bacterial biofilm mainly composed of oral bacteria, mucoproteins in saliva, and food debris. Its accumulation on the tooth surface is a core cause of oral diseases such as tooth decay and periodontitis. Early detection and timely removal of dental plaque are key to preventing oral diseases.

[0003] Currently, plaque detection methods mainly include visual inspection, staining agent testing, and specialized equipment testing. Visual inspection relies on professional dentists or the user to visually inspect the tooth surface to determine the presence of plaque. This method is highly subjective and has extremely low accuracy in identifying early-stage or hidden plaque (such as between teeth or along the gum line), and it cannot differentiate between the age of the plaque. Staining agent testing uses staining agents such as methylene blue to stain teeth, giving plaque a specific color for observation. However, staining agents can irritate the oral mucosa, and immediate cleaning is required after staining, making the process cumbersome and unsuitable for frequent daily checks. While specialized equipment such as laser fluorescence detectors used in dental clinics can achieve accurate detection, these devices are bulky and expensive, requiring specialized facilities and failing to meet the needs of users for real-time home monitoring.

[0004] Therefore, existing technologies still have shortcomings. Summary of the Invention

[0005] To address the aforementioned deficiencies in existing technologies, this invention provides a multi-feature fusion segmentation and quantification analysis system, method, terminal, and medium for oral fluorescence images. The technical solution adopted by this invention is as follows: In a first aspect, the present invention provides a method for multi-feature fusion segmentation and quantification analysis of oral fluorescence images, the method comprising: An oral fluorescence image is acquired, the oral fluorescence image is preprocessed to obtain a preprocessed oral fluorescence image, and the preprocessed oral fluorescence image is segmented into regions based on a semantic segmentation network to obtain the tooth regions in the oral fluorescence image. Based on the tooth region, a suspected dental plaque region is identified, and multi-dimensional features of the suspected dental plaque region are extracted. The multi-dimensional features are then input into a plaque classification neural network to obtain the classification result of the suspected dental plaque region. The multi-dimensional features include color features, texture features, morphological features, and spatial features. The classification results are subjected to morphological post-processing, and a dental plaque distribution map is determined based on the classification results, outputting quantitative analysis results.

[0006] In one implementation, preprocessing the oral fluorescence image to obtain a preprocessed oral fluorescence image includes: Convert the oral fluorescence image from the RGB color space to the HSV color space or the CIELab color space; Gaussian filtering and median filtering are used to remove image noise, and top-hat transformation is used to correct uneven image illumination caused by the curved surface of the teeth.

[0007] In one implementation, a semantic segmentation network is used to perform region segmentation on the preprocessed oral fluorescence image to obtain the tooth region in the oral fluorescence image, including: The semantic segmentation network is pre-trained, which is obtained by training the U-Net network based on a first labeled training image dataset, wherein the first training image dataset is labeled with tooth region, gum region and oral cavity background; The preprocessed oral fluorescence image is input into the trained semantic segmentation network, which outputs the tooth region segmentation result to obtain the tooth region in the oral fluorescence image.

[0008] In one implementation, determining a suspected plaque area based on the tooth region includes: Determine the color value of each pixel in the tooth region; The color value of each pixel is matched with a preset typical color of dental plaque to obtain a set of successfully matched pixels, and the suspected dental plaque area is determined based on the set of pixels.

[0009] In one implementation, multi-dimensional features of the suspected dental plaque region are extracted, and the multi-dimensional features are input into a plaque classification neural network to obtain a classification result for the suspected dental plaque region, including: Extract the color features, texture features, morphological features, and spatial features of the suspected dental plaque area; The color features, texture features, morphological features, and spatial features are fused to obtain a feature vector; The feature vector is input into a trained plaque classification neural network, and the classification result of the suspected dental plaque region is output. The plaque classification neural network is a deep neural network trained in advance based on a second set of labeled training images, in which new plaques, old plaques and non-plaques are labeled.

[0010] In one implementation, the classification result undergoes morphological post-processing, including: New or old spots in the classification results are selected, and plaque noise and plaque areas with a plaque area smaller than a preset value in the new or old spots are eliminated. Fill the small holes inside the area corresponding to the new or old spots.

[0011] In one implementation, a plaque distribution map is determined based on the classification results, and quantitative analysis results are output, including: The classification results are overlaid on the original oral fluorescence image with different colors to generate a dental plaque distribution map; Based on the plaque distribution map, the plaque coverage rate, plaque type ratio, and plaque coverage rate in high-risk areas were calculated to obtain quantitative analysis results.

[0012] Secondly, embodiments of the present invention also provide a multi-feature fusion segmentation and quantization analysis system for oral fluorescence images, wherein the system is used to implement the steps of the multi-feature fusion segmentation and quantization analysis method for oral fluorescence images described in any of the above claims, and the system includes: A tooth region segmentation module is used to acquire an oral fluorescence image, preprocess the oral fluorescence image to obtain a preprocessed oral fluorescence image, and perform region segmentation on the preprocessed oral fluorescence image based on a semantic segmentation network to obtain the tooth region in the oral fluorescence image. The dental plaque classification module is used to identify suspected dental plaque areas based on the tooth area, extract multi-dimensional features of the suspected dental plaque areas, input the multi-dimensional features into the plaque classification neural network, and obtain the classification result of the suspected dental plaque areas. The multi-dimensional features include color features, texture features, morphological features and spatial features. The quantitative analysis module is used to perform morphological post-processing on the classification results, determine the dental plaque distribution map based on the classification results, and output the quantitative analysis results.

[0013] Thirdly, embodiments of the present invention also provide a terminal, wherein the terminal includes a memory, a processor, and a multi-feature fusion segmentation and quantization analysis program for oral fluorescence images stored in the memory and executable on the processor. When the processor executes the multi-feature fusion segmentation and quantization analysis program for oral fluorescence images, it implements the steps of the multi-feature fusion segmentation and quantization analysis method for oral fluorescence images of any of the above-mentioned schemes.

[0014] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a multi-feature fusion segmentation and quantization analysis program for oral fluorescence images, the multi-feature fusion segmentation and quantization analysis program for oral fluorescence images implementing the steps of the multi-feature fusion segmentation and quantization analysis method for oral fluorescence images as described in any of the above schemes on the computer-readable storage medium.

[0015] Beneficial Effects: Compared with existing technologies, this invention provides a multi-feature fusion segmentation and quantitative analysis method for oral fluorescence images. First, an oral fluorescence image is acquired and preprocessed to obtain a preprocessed oral fluorescence image. Then, based on a semantic segmentation network, the preprocessed oral fluorescence image is segmented into regions to obtain the tooth regions. Next, based on the tooth regions, suspected plaque regions are identified, and multi-dimensional features of these suspected plaque regions are extracted. These multi-dimensional features are input into a plaque classification neural network to obtain the classification results for the suspected plaque regions. The multi-dimensional features include color features, texture features, morphological features, and spatial features. Then, morphological post-processing is performed on the classification results, and a plaque distribution map is determined based on the classification results, outputting quantitative analysis results. This invention is applicable to scenarios such as home daily oral care monitoring and dental clinic auxiliary diagnosis, enabling rapid, accurate, and non-invasive detection of dental plaque and quantitative assessment of dental health. Attached Figure Description

[0016] Figure 1 This is a flowchart of a preferred embodiment of the multi-feature fusion segmentation and quantification analysis method for oral fluorescence images provided by the present invention.

[0017] Figure 2 This is a schematic diagram of the principle framework of the multi-feature fusion segmentation and quantification analysis system for oral fluorescence images provided in an embodiment of the present invention.

[0018] Figure 3 A schematic diagram of the terminal provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content, operations, or steps, nor does it require execution in the described order. For example, some operations or steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0021] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. For example, "first control information" and "second control information" are only used to distinguish different control information and do not limit their order.

[0023] Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.

[0024] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] To address the problems of existing technologies, this embodiment provides a multi-feature fusion segmentation and quantitative analysis method for oral fluorescence images. Based on this method, real-time, accurate, and quantitative detection of dental plaque is achieved, while providing intuitive detection reports and personalized health suggestions, solving the problems of strong subjectivity, cumbersome operation, low accuracy, and poor applicability of existing detection methods. In specific application, this embodiment first acquires an oral fluorescence image, preprocesses it to obtain a preprocessed oral fluorescence image, and then performs region segmentation on the preprocessed oral fluorescence image based on a semantic segmentation network to obtain the tooth regions in the oral fluorescence image. Then, based on the tooth regions, suspected dental plaque regions are identified, and multi-dimensional features of the suspected dental plaque regions are extracted. These multi-dimensional features are input into a plaque classification neural network to obtain the classification results of the suspected dental plaque regions. The multi-dimensional features include color features, texture features, morphological features, and spatial features. Next, morphological post-processing is performed on the classification results, and a dental plaque distribution map is determined based on the classification results, outputting quantitative analysis results.

[0026] The multi-feature fusion segmentation and quantification analysis method for oral fluorescence images in this embodiment can be applied to a terminal, including intelligent products such as computers. The terminal and the handheld fluorescence detection device communicate via WiFi. Specifically, as shown... Figure 1As shown in the figure, the multi-feature fusion segmentation and quantization analysis method for oral fluorescence images in this embodiment includes the following steps: Step S100: Obtain an oral fluorescence image, preprocess the oral fluorescence image to obtain a preprocessed oral fluorescence image, and perform region segmentation on the preprocessed oral fluorescence image based on a semantic segmentation network to obtain the tooth region in the oral fluorescence image.

[0027] The core of this embodiment lies in its end-to-end technical solution, encompassing hardware acquisition, image preprocessing, region segmentation, precise classification, and quantitative evaluation, to achieve real-time, accurate detection and health assessment of dental plaque. In practical applications, the handheld fluorescence detection device of this embodiment includes a laser light source, a filter, and a camera sensor. The laser light source in this embodiment can be a 405nm near-ultraviolet LED light source with a power of 5-10W and a wavelength error ≤±5nm, ensuring effective excitation of porphyrin-like substances in dental plaque to produce orange-red fluorescence (wavelength 550-650nm). The laser light source can adopt a ring design with an inner diameter of 8-10mm and an outer diameter of 12-15mm, installed around the camera lens to form 360° uniform illumination, avoiding shadows on the tooth surface and ensuring consistent image illumination. The LED light source uses a constant current drive, has a lifespan of ≥5000 hours, low power consumption, and low heat generation, making it suitable for long-term use with handheld devices. The filter in this embodiment can be a 430nm long-pass filter made of optical glass, with a transmittance ≥90% (wavelength ≥430nm) and a cutoff rate ≥99% (wavelength <430nm). It completely blocks 405nm excitation light and interference from ultraviolet and blue light in the environment, allowing only fluorescence from teeth (green fluorescence, wavelength 480-520nm) and dental plaque (orange-red fluorescence, wavelength 550-650nm) to pass through, significantly improving the image signal-to-noise ratio. In practical applications, the filter is threaded and installed on the front of the camera lens for easy disassembly, replacement, and maintenance. The camera sensor in this embodiment can be a 1 / 2.3-inch monochrome CMOS camera sensor with ≥12 million effective pixels, a pixel size of 1.4μm × 1.4μm, a quantum efficiency ≥70% (wavelength 400-700nm), a maximum frame rate of 30fps, and supports real-time video acquisition. Compared to color cameras, monochrome cameras do not require color filter array (CFA) interpolation and can directly capture full-spectrum light signals. They are more sensitive to weak fluorescence and can acquire richer fluorescence details. Therefore, oral fluorescence images can be acquired through this camera sensor, providing high-quality raw data for subsequent algorithm processing.

[0028] In another implementation, the handheld fluorescence detection device of this embodiment also includes auxiliary components, such as a WiFi communication module, a supplementary light adjustment module, and human-machine interaction buttons. The WiFi communication module supports the 802.11 b / g / n protocol, which can transmit the acquired real-time video stream and image data to a remote terminal for remote display and processing. The supplementary light adjustment module has a built-in light sensor that can detect ambient light intensity in real time. When the ambient light is too strong, it automatically adjusts the brightness of the laser light source to avoid interference from ambient light in the acquisition of oral fluorescence images. The human-machine interaction buttons include a power button, a shooting button, and a mode switching button, which are simple and intuitive to operate. In addition, the handheld fluorescence detection device of this embodiment is made of ABS engineering plastic shell with a non-slip surface. The grip is designed in an arc shape to fit the contour of the palm, making it easy to operate with one hand. The front of the device is the detection head (this area integrates a ring-shaped laser light source, a filter, and a camera sensor), the middle is the grip area (setting buttons), and the rear is the battery compartment and USB-C interface. Furthermore, the detection head in this embodiment can rotate ±30°, allowing users to easily adjust the shooting angle and adapt to the detection needs of different areas of the oral cavity (including front teeth, back teeth, and interdental spaces).

[0029] In practical applications, after the user turns on the handheld fluorescence detection device, the device automatically completes the initial calibration of the light source and camera, and establishes a connection between the WiFi communication module and the terminal (such as a smartphone APP). The user inserts the detection head into the oral cavity, aligns it with the area of ​​the tooth to be detected, and can also preview the fluorescence image in real time through the terminal's APP, adjust the shooting angle and distance (recommended shooting distance 5-10cm), and after ensuring that the entire tooth area is in the viewfinder, press the shooting button. The camera sensor acquires the oral fluorescence image and transmits it to the remote terminal via WiFi.

[0030] Furthermore, this embodiment preprocesses the acquired oral fluorescence images. The core purpose of preprocessing is to improve image quality and enhance the characteristic differences between dental plaque and healthy tooth tissue, providing high-quality image data for subsequent region segmentation and classification. The preprocessing process in this embodiment includes: converting the oral fluorescence images from the RGB color space to the HSV color space or the CIELab color space; removing image noise using Gaussian filtering or median filtering; and correcting uneven image illumination caused by the curved surface of the teeth using top-hat transformation.

[0031] Specifically, since the original oral fluorescence images are stored in the RGB color space, and brightness, hue, and saturation are coupled in the RGB space, it is not conducive to the extraction of fluorescence features. Therefore, this embodiment converts the oral fluorescence images from the RGB color space to the HSV and CIELab dual color spaces, and enhances the fluorescence signal of dental plaque through multi-space feature fusion. Compared with the RGB color space, the HSV color space can separate color information from brightness information. The fluorescence features of dental plaque are mainly reflected in the hue and saturation channels. The differences between healthy teeth and dental plaque in these two channels are more significant, which facilitates subsequent feature extraction. The conversion formula is as follows: Let the three channel values ​​of the oral fluorescence image be R, G, and B (range 0-255). First, calculate the maximum value Max = max(R,G,B), the minimum value Min = min(R,G,B), and the difference Δ = Max - Min. The conversion formula for hue H is: when Δ = 0, H = 0; when Max = R, H = 60° × [(GB) / Δ mod 6]; when Max = G, H = 60° × [(BR) / Δ + 2]; when Max = B, H = 60° × [(RG) / Δ + 4]. The conversion formula for saturation S is: when Max = 0, S = 0; otherwise, S = Δ / Max × 100% (range 0%-100%). The conversion formula for lightness V is: V = Max / 255 × 100% (range 0%-100%). In this embodiment, the fluorescence of dental plaque is represented in HSV space as H∈[15°,45°] (orange-red range), S∈[60%,90%], V∈[30%,70%], while the fluorescence of healthy teeth is represented as H∈[120°,150°] (green range), S∈[40%,80%], V∈[50%,90%]. Through color space conversion, the characteristics of the two types of regions can be initially separated.

[0032] In another implementation, the CIELab color space is designed based on human visual perception. The L channel represents luminance, the a channel represents the red-green components (red is positive, green is negative), and the b channel represents the yellow-blue components (yellow is positive, blue is negative). Dental plaque, appearing orange-red under fluorescence, shows a high positive value in the a channel and a moderate positive value in the b channel. In contrast, the fluorescence of healthy teeth shows a negative or low positive value in the a channel and a low negative value in the b channel, with a significant difference between the two. The conversion process is as follows: First, the RGB space of the oral fluorescence image is converted to the XYZ space. The conversion matrix adopts the standard matrix recommended by the International Commission on Illumination (CIE): [X]=[0.4124564 0.3575761 0.1804375][R]; [Y]=[0.2126729 0.7151522 0.0721750][G]; [Z]=[0.0193339 0.1191920 0.9503041][B]; where R, G, and B are the original RGB pixel values ​​(0-255).

[0033] Then, the values ​​in the XYZ space are normalized, and Xn=X / X0, Yn=Y / Y0, Zn=Z / Z0 are calculated, where X0=95.047, Y0=100.000, and Z0=108.883 (tristimulus values ​​of standard light source D65). Next, calculate the L, a, and b values ​​in the CIELab space: If Yn > 0.008856, then L = 116 × Yn^(1 / 3) - 16; otherwise, L = 903.3 × Yn. If Xn > 0.008856, then a = 500 × ( If Zn>0.008856, then b=200×(Yn^(1 / 3)-Zn^(1 / 3)); otherwise b=200×[Yn^(1 / 3) -(7.787×Zn)+16 / 116].

[0034] By using dual color space conversion, this embodiment can capture the color differences between dental plaque and healthy tooth tissue from different dimensions, providing richer information for subsequent feature extraction.

[0035] Furthermore, since Gaussian noise and salt-and-pepper noise may be introduced during the camera sensor acquisition process, affecting image quality, this embodiment also employs Gaussian filtering and median filtering to remove image noise. Specifically, Gaussian filtering involves applying a Gaussian filter to the converted image, with the Gaussian kernel size set to 5×5 and the standard deviation σ=1.0. Gaussian filtering effectively suppresses Gaussian noise and preserves image edge details relatively well. The filtering formula is as follows: Where (x,y) are the pixel coordinates within the Gaussian kernel, and G(x,y) are the corresponding weight values. The filtered pixel values ​​are calculated using a weighted average. Median filtering: The Gaussian-filtered image is then subjected to median filtering, with a window size of 3×3. Median filtering effectively suppresses salt-and-pepper noise, preventing noise from interfering with subsequent feature extraction. Specifically, the pixel values ​​within the window are sorted by size, and the median value is taken as the output value of the center pixel of the window.

[0036] Because teeth have a curved surface and the oral cavity has a narrow space, illumination from a light source can create shadow areas, leading to uneven lighting in the image. This manifests as low brightness at the tooth edges and high brightness in the center, which affects the accuracy of subsequent color feature extraction. This embodiment uses Top-Hat Transform to correct for uneven lighting. Top-Hat Transform is a morphological image enhancement method that highlights details brighter than their surroundings while suppressing background interference caused by uneven lighting. For example, this embodiment first selects a 3×3 rectangular structuring element. Then, an opening operation (erosion followed by dilation) is performed on the V channel (luminance channel) of the transformed HSV image to obtain the background image. The background image is then subtracted from the original V channel image to obtain the V channel image after lighting homogenization. Finally, the corrected V channel is recombined with the original H and S channels to obtain the HSV image after lighting homogenization.

[0037] After the above preprocessing steps, high-quality oral fluorescence images are obtained, with significantly reduced noise, uniform illumination, and further highlighted differences in fluorescence characteristics between dental plaque and healthy tooth tissue.

[0038] Furthermore, in this embodiment, the semantic segmentation network can be pre-trained. This semantic segmentation network is pre-trained on a U-Net network based on a pre-labeled first training image dataset, where the first training image dataset consists of several oral images labeled with tooth regions, gingival regions, and oral background. Then, the pre-processed oral fluorescence image is input into the trained semantic segmentation network, which outputs the tooth region segmentation result, obtaining the tooth region from the oral fluorescence image. In practical applications, the first training image dataset collects multiple oral fluorescence images containing tooth regions, gingival regions, and oral background regions. These images are then labeled by a professional dentist, marking the outlines of the tooth regions. When the pre-processed oral fluorescence image is input into the trained semantic segmentation network, a segmentation map of the same size as the input image is output, where each pixel is labeled as either "tooth" (label 1) or "non-tooth" (label 0), achieving accurate segmentation of the tooth region.

[0039] In another implementation, when splitting the tooth region, the present embodiment may also adopt a splitting method based on a brightness threshold. During specific implementation, first, the preprocessed oral fluorescence image is converted into a grayscale image. Then, the oral fluorescence image is split into several sub-regions, and the brightness value of each sub-region is determined. Then, based on the brightness value of each sub-region, the brightness distribution of the oral fluorescence image is determined. Since the brightness value (i.e., grayscale value) of teeth in the oral fluorescence image is significantly higher than that of regions such as gums and oral backgrounds. For example, healthy teeth have high fluorescence brightness, and the grayscale value is usually ≥180, the grayscale value of gums is ≤120, and the grayscale value of the background is ≤80. Based on this, combined with the above brightness distribution of the oral fluorescence image, the present embodiment can mark the sub-regions in the grayscale image with a grayscale value ≥ T (brightness threshold, such as 180) as the foreground (i.e., tooth region), and the sub-regions with a grayscale value < T as the oral background (including the gum region), so as to distinguish the tooth region in the oral fluorescence image.

[0040] Step S200: Based on the tooth region, determine the suspected dental plaque region, extract the multi-dimensional features of the suspected dental plaque region, and input the multi-dimensional features into the plaque classification neural network to obtain the classification result of the suspected dental plaque region. The multi-dimensional features include color features, texture features, morphological features, and spatial features.

[0041] Specifically, after splitting the tooth region, the present embodiment first extracts the suspected dental plaque region based on the fluorescence color feature. Dental plaque produces orange-red fluorescence under the excitation of 405nm ultraviolet light, and healthy teeth produce green fluorescence. There are significant differences in the hue (H) and saturation (S) channels of the HSV color space between the two. Based on this, the present embodiment determines the color value of each pixel in the tooth region (i.e., the hue H, saturation S, and value V in the HSV color space); then matches the color value of each pixel with the preset typical color of dental plaque to obtain the set of successfully matched pixels, and based on the set of pixels, determines the suspected dental plaque region. In practical applications, by matching the color value of each pixel with the preset typical color of dental plaque, the set of pixels of the suspected dental plaque region screened out is: Hue H ∈ [15°, 45°] (the hue range corresponding to orange-red fluorescence); Saturation S ∈ [60%, 90%] (the saturation of dental plaque fluorescence is higher than that of healthy teeth); Value V ∈ [30%, 70%] (excluding overly bright or overly dark interfering pixels).

[0042] This embodiment identifies suspected plaque regions that cover over 95% of plaque pixels while effectively excluding interfering pixels such as healthy teeth and food debris. Furthermore, this embodiment performs connectivity analysis on the set of suspected plaque pixels, merging adjacent (8-neighbor) suspected plaque pixels into connected regions, with each connected region considered a single suspected plaque region. A connected region area threshold of ≥50 pixels is set, and small connected regions with an area <50 pixels (considered noise) are removed, ultimately yielding multiple suspected plaque regions.

[0043] Furthermore, this embodiment extracts the color features, texture features, morphological features, and spatial features of the suspected dental plaque region. Specifically, when extracting color features, this embodiment calculates the statistical features of the suspected dental plaque region in the RGB, HSV, and Lab color spaces, including the mean, standard deviation, maximum value, minimum value, and median of each channel, for a total of 3 × 5 = 15 color features. Since the fluorescent image texture of dental plaque (which is unevenly granular) differs from the texture of food debris and tooth staining (which is mostly uniform or regular), this embodiment extracts Local Binary Pattern (LBP) features and Gray-Level Co-occurrence Matrix (GLCM) features when extracting texture features. The LBP operator with radius R = 3 and number of sampling points P = 8 is used to calculate the LBP histogram of the suspected dental plaque region. The histogram is divided into 16 intervals, and the frequency of each interval is used as a feature, for a total of 16 LBP features. The gray-level co-occurrence matrix features are obtained by calculating the GLCM of suspected dental plaque regions (distance d=1, angle θ=0°, 45°, 90°, 135°) and extracting four texture indices: contrast, correlation, energy, and homogeneity, for a total of 4×4=16 GLCM features.

[0044] When extracting morphological features, this embodiment calculates four morphological parameters for the suspected plaque region: area, perimeter, roundness, and aspect ratio. Since plaque is more likely to be distributed in cleaning dead zones such as the gingival margin and interdental spaces, when extracting spatial features, this embodiment first calculates the pixel distance from the center of the suspected plaque region to the gingival line. Specifically, the gingival line can be determined by the intersection of the segmented tooth region edge and the gingival region, resulting in a distance feature. Then, the tooth region is divided into four areas: anterior teeth, posterior teeth, interdental spaces, and the gingival margin. The location of the suspected plaque region is determined based on its center coordinates and represented using one-hot encoding, resulting in four location features.

[0045] Next, in this embodiment, the above-mentioned color features, texture features, morphological features and spatial features are fused to obtain a feature vector. That is, 15 color features, 16 LBP features, 16 GLCM features, 4 morphological features, 1 distance feature and 4 position features can be extracted from the suspected dental plaque area. These features are converted into feature vectors to obtain a 56-dimensional feature vector.

[0046] The feature vector is input into a trained plaque classification neural network, which outputs the classification result of the suspected dental plaque region. This plaque classification neural network is pre-trained on a deep neural network based on a pre-labeled second training image dataset, which consists of several oral images. Because the fluorescence characteristics of new plaque (formation time ≤ 24 hours) and old plaque (formation time > 24 hours) differ—new plaque has a lower porphyrin content, weaker fluorescence intensity, and a lighter orange-red color; old plaque has a higher porphyrin content, stronger fluorescence intensity, and a darker orange-red color—the second training image dataset in this embodiment is labeled with new plaque, old plaque, and non-plaque. Thus, inputting the feature vector into the trained plaque classification neural network outputs a classification result of: new plaque, old plaque, or non-plaque. This not only allows for the rapid identification of the actual dental plaque region but also directly determines whether the actual plaque region is new or old plaque.

[0047] In another implementation, this embodiment can also utilize a binary classification model to identify the feature vector obtained by fusing the aforementioned color features, texture features, morphological features, and spatial features, thereby obtaining the classification result corresponding to the suspected dental plaque area. The classification result is either plaque or non-plaque, thus identifying the actual dental plaque area. In practical applications, the binary classification model in this embodiment can employ a support vector machine (SVM) model. When training the SVM model, several dental plaque area samples can be used, and the aforementioned color features, texture features, morphological features, and spatial features can be extracted from each plaque area sample for training. The trained binary classification model can accurately analyze the classification result of the suspected dental plaque area, i.e., whether it is plaque or non-plaque. Next, this embodiment further calculates the average fluorescence intensity of the actual plaque area, for example, the average pixel value of the V channel in the HSV color space. Then, the average fluorescence intensity of the actual plaque area is compared with a preset threshold range. If the average fluorescence intensity ∈ [30%, 50%], it is determined to be a new plaque; if the average fluorescence intensity ∈ [50%, 70%], it is determined to be an old plaque. In this way, it is determined whether the actual dental plaque area is a new plaque or an old plaque.

[0048] In addition, this embodiment can also combine the morphological characteristics of the actual plaque area to assist in the judgment. Specifically, new plaques are mostly irregular patches with a roundness of ≤0.5; old plaques are mostly dense patches with a roundness of ≥0.6, which further improves the accuracy of plaque type judgment.

[0049] Step S300: Perform morphological post-processing on the classification results, determine the dental plaque distribution map based on the classification results, and output the quantitative analysis results.

[0050] Since the classification results may contain a small amount of noise (such as isolated misjudged pixels, small false positive areas) and small holes (such as blank pixels inside the plaque area), which affect the accuracy of the final detection results, this embodiment performs post-processing through morphological operations and connected component analysis to screen out new or old plaques in the classification results, and eliminate plaque noise and plaque areas with a plaque area smaller than a preset value in the new or old plaques; and fill the small holes inside the corresponding areas of the new or old plaques.

[0051] Specifically, the morphological operations in this embodiment include opening and closing operations. The opening operation is used to eliminate small false positive areas, that is, plaque areas with an area smaller than a preset value, which are actually non-plaques but misclassified as new or old plaques. This embodiment uses a 2×2 square structuring element, first performing an erosion operation (removing small pixels at the edges) on the image of the classification result, and then performing a dilation operation (restoring the main outline of the region). For example, if a classification result image contains an isolated new plaque area with an area of ​​5 pixels, this area is eliminated after the opening operation. The closing operation is used to fill small holes inside the corresponding regions of new or old plaques. This embodiment uses a 2×2 square structuring element, first performing a dilation operation (filling holes) on the image of the classification result, and then performing an erosion operation (restoring the original boundaries of the region). For example, if an old plaque area contains a hole with an area of ​​3 pixels, this hole is filled after the closing operation.

[0052] Furthermore, this embodiment also performs connected component analysis on the classification results image after morphological operations, filtering out new and old spots with an area ≥ 15 pixels (areas < 15 pixels are considered noise), and removing areas that overlap with the tooth region boundary (to avoid edge misjudgment). For example, a new spot with an area of ​​2 pixels is removed after filtering; another old spot with an area of ​​20 pixels, which does not overlap with the tooth region boundary, is retained.

[0053] Furthermore, after obtaining the above classification results, this embodiment can generate a dental plaque distribution map by overlaying the classification results onto the original oral fluorescence image with different colors based on plaque type. This dental plaque distribution map uses different colors to represent: new plaque, old plaque, and non-plaque. Then, based on the plaque distribution map, the dental plaque coverage rate, plaque type proportion, and dental plaque coverage rate in high-risk areas are calculated to obtain quantitative analysis results. Specifically, this embodiment first calculates the plaque coverage rate using the following formula: Plaque coverage per tooth = (Total number of pixels in the actual plaque area on a single tooth / Total number of pixels in the area of ​​a single tooth) × 100%; Overall plaque coverage = (total number of pixels in actual plaque areas on all teeth / total number of pixels in all tooth areas) × 100%.

[0054] Further calculation of the percentage of plaque types is performed using the following formula: New plaque percentage = (total number of pixels in the new plaque area / total number of pixels in the actual plaque area) × 100%; Percentage of old plaque = (Total number of pixels in the old plaque area / Total number of pixels in the actual plaque area) × 100%.

[0055] Next, the tooth area was divided into 5 high-risk areas, including: interdental area, gingival margin area, medial area of ​​posterior teeth, medial area of ​​anterior teeth, and occlusal surface of teeth. The plaque coverage rate of each high-risk area was calculated using the formula: Plaque coverage rate of high-risk area = (number of pixels of actual plaque in the area / total number of pixels in the area) × 100%.

[0056] Furthermore, this embodiment can also output a dental health assessment report based on the overall plaque coverage, the proportion of old plaque, and the plaque coverage in high-risk areas. The dental health assessment report in this embodiment includes at least the following: basic detection information, a plaque distribution heatmap, quantitative analysis results (overall plaque coverage, plaque coverage per tooth, proportion of new / old plaque, plaque coverage in each high-risk area), and personalized suggestions. For example, if the plaque coverage in the interdental area is high: it is recommended to use dental floss and a water flosser to clean the interdental spaces; if the proportion of old plaque is high: it is recommended to increase brushing time (≥2 minutes each time) and use the Bass brushing technique. The dental health report in this embodiment can be displayed in real time on the terminal's APP for easy viewing by the user. This invention is applicable to scenarios such as daily oral care monitoring at home and auxiliary diagnosis in dental clinics, achieving rapid, accurate, and non-invasive detection of dental plaque and quantitative assessment of dental health status.

[0057] Based on the above embodiments, the present invention also provides a multi-feature fusion segmentation and quantization analysis system for oral fluorescence images, the system being used to implement the method steps in the above embodiments. Specifically, as Figure 2As shown in the diagram, the system in this embodiment includes: a tooth region segmentation module 10, a dental plaque classification module 20, and a quantitative analysis module 30. Specifically, the tooth region segmentation module 10 is used to acquire an oral fluorescence image, preprocess the oral fluorescence image to obtain a preprocessed oral fluorescence image, and perform region segmentation on the preprocessed oral fluorescence image based on a semantic segmentation network to obtain the tooth regions in the oral fluorescence image. The dental plaque classification module 20 is used to determine suspected dental plaque regions based on the tooth regions, extract multi-dimensional features of the suspected dental plaque regions, input the multi-dimensional features into a plaque classification neural network, and obtain the classification result of the suspected dental plaque regions. The multi-dimensional features include color features, texture features, and spatial features. The quantitative analysis module 30 is used to perform morphological post-processing on the classification results, determine the dental plaque distribution map based on the classification results, and output the quantitative analysis result.

[0058] The multi-feature fusion segmentation and quantification analysis system for oral fluorescence images in this embodiment is based on the same principle as the steps in the above method embodiments, and will not be elaborated further here.

[0059] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 3 As shown. The terminal may include one or more processors 100 ( Figure 3 (Only one is shown in the image), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100. For example, a multi-feature fusion segmentation and quantization analysis program for oral fluorescence images. When one or more processors 100 execute computer program 102, they can implement the various steps in the embodiments of the multi-feature fusion segmentation and quantization analysis method for oral fluorescence images. Alternatively, when one or more processors 100 execute computer program 102, they can implement the functions of each module / unit in the embodiments of the multi-feature fusion segmentation and quantization analysis system for oral fluorescence images, which is not limited here.

[0060] In one embodiment, the processor 100 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0061] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or RAM. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital Card (SD), or Flash Card. Furthermore, memory 101 may include both internal and external storage units. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 can also be used to temporarily store data that has been output or will be output.

[0062] Those skilled in the art will understand that Figure 3 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0063] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct memory bus RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-feature fusion segmentation and quantitative analysis method of oral fluorescence images, characterized in that, The method comprises: obtaining an oral cavity fluorescence image, preprocessing the oral cavity fluorescence image to obtain a preprocessed oral cavity fluorescence image, and performing region segmentation on the preprocessed oral cavity fluorescence image based on a semantic segmentation network to obtain a tooth region in the oral cavity fluorescence image; based on the tooth region, determining a suspected dental plaque region, and extracting multi-dimensional features of the suspected dental plaque region, inputting the multi-dimensional features into a plaque classification neural network to obtain a classification result of the suspected dental plaque region, wherein the multi-dimensional features include color features, texture features, morphological features, and spatial features; morphological post-processing is performed on the classification result, and a dental plaque distribution map is determined according to the classification result, and a quantitative analysis result is output.

2. The multi-feature fusion segmentation and quantitative analysis method of the oral fluorescence image according to claim 1, characterized in that, The preprocessed oral cavity fluorescence image is obtained by preprocessing the oral cavity fluorescence image, which comprises: converting the oral cavity fluorescence image from an RGB color space to an HSV color space or a CIELab color space; using Gaussian filtering and median filtering to remove image noise, and correcting uneven image illumination caused by tooth curvature through top-hat transformation.

3. The multi-feature fusion segmentation and quantitative analysis method of oral fluorescence images according to claim 2, characterized in that, The tooth region in the oral cavity fluorescence image is obtained by performing region segmentation on the preprocessed oral cavity fluorescence image based on a semantic segmentation network, which comprises: The semantic segmentation network is pre-trained, and the semantic segmentation network is obtained by pre-training a U-Net network based on a labeled first training image dataset, wherein the tooth region, gum region and oral cavity background are labeled in the first training image dataset; the preprocessed oral cavity fluorescence image is input into the trained semantic segmentation network to output a tooth region segmentation result, and the tooth region in the oral cavity fluorescence image is obtained.

4. The multi-feature fusion segmentation and quantitative analysis method of oral fluorescence images according to claim 3, characterized in that, Based on the tooth region, the suspected dental plaque region is determined, which comprises: determining the color value of each pixel in the tooth region; matching the color value of each pixel with a preset typical dental plaque color to obtain a set of matching successful pixels, and determining the suspected dental plaque region based on the set of pixels.

5. The multi-feature fusion segmentation and quantitative analysis method of oral fluorescence images according to claim 4, characterized in that, The multi-dimensional features of the suspected dental plaque region are extracted, and the multi-dimensional features are input into a plaque classification neural network to obtain a classification result of the suspected dental plaque region, which comprises: extracting color features, texture features, morphological features and spatial features of the suspected dental plaque region; fusing the color features, texture features, morphological features and spatial features to obtain a feature vector; the feature vector is input into the trained plaque classification neural network to output the classification result of the suspected dental plaque region, wherein the plaque classification neural network is obtained by pre-training a deep neural network based on a labeled second training image dataset, and the second training image dataset labels new plaque, old plaque and non-plaque.

6. The multi-feature fusion segmentation and quantitative analysis method of the oral fluorescence image according to claim 5, characterized in that, The morphological post-processing of the classification result comprises: filtering out new plaque or old plaque in the classification result, and eliminating dental plaque noise and dental plaque regions with an area smaller than a preset value in the new plaque or old plaque; filling small holes in the region corresponding to the new plaque or old plaque.

7. The multi-feature fusion segmentation and quantitative analysis method of oral fluorescence images according to claim 6, characterized in that, According to the classification result, a dental plaque distribution map is determined, and a quantitative analysis result is output, including: The classification result is overlaid on the original oral fluorescence image according to different colors to generate a dental plaque distribution map; According to the plaque distribution map, the plaque coverage rate, plaque type proportion, and high-risk area plaque coverage rate are calculated to obtain the quantitative analysis result.

8. A multi-feature fusion segmentation and quantitative analysis system of oral fluorescence images, characterized in that, The system is used to implement the steps of the oral fluorescence image multi-feature fusion segmentation and quantitative analysis method according to any one of claims 1-7, and the system comprises: A tooth region segmentation module is configured to acquire an oral fluorescence image, pre-process the oral fluorescence image to obtain a pre-processed oral fluorescence image, and perform region segmentation on the pre-processed oral fluorescence image based on a semantic segmentation network to obtain a tooth region in the oral fluorescence image. A dental plaque classification module is configured to determine a suspected dental plaque region based on the tooth region, extract multi-dimensional features of the suspected dental plaque region, input the multi-dimensional features into a plaque classification neural network, and obtain a classification result of the suspected dental plaque region, wherein the multi-dimensional features include color features, texture features, morphological features, and spatial features. A quantitative analysis module is configured to perform morphological post-processing on the classification result, determine a dental plaque distribution map according to the classification result, and output a quantitative analysis result.

9. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and an oral fluorescence image multi-feature fusion segmentation and quantitative analysis program stored in the memory and executable on the processor. When the processor executes the oral fluorescence image multi-feature fusion segmentation and quantitative analysis program, the steps of the oral fluorescence image multi-feature fusion segmentation and quantitative analysis method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an oral fluorescence image multi-feature fusion segmentation and quantitative analysis program. The oral fluorescence image multi-feature fusion segmentation and quantitative analysis program implements the steps of the oral fluorescence image multi-feature fusion segmentation and quantitative analysis method according to any one of claims 1-7 on the computer-readable storage medium.

Images