Oral cavity image multi-tissue full-automatic segmentation method

Abstract

The invention discloses an oral cavity image multi-tissue full-automatic segmentation method, which belongs to the technical field of oral cavity image multi-tissue segmentation, combines multi-stage classification with image processing, and improves the accuracy of image recognition, enhances the depth of image understanding and improves the efficiency of image processing through the cooperative operation between the multi-stage classification and the image processing. By means of the multi-stage classifier, images can be classified from local to global and from rough to fine, the classification mode is beneficial to more accurately recognizing target objects or features in the images, in the image processing process, the region of interest can be gradually reduced through multi-stage classification, the target judgment capacity is enhanced in each stage, and the target recognition efficiency is improved. In this way, efficient and accurate target detection is achieved, in some cases, multi-level classification can be combined with other image processing tasks for learning, and the efficiency and accuracy of the model are improved by sharing the feature extractor and the model parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of oral image multi-tissue segmentation, and specifically to a fully automatic multi-tissue segmentation method for oral images. Background Art

[0002] Oral images refer to images used to observe and analyze the internal structure of the oral cavity. Oral image segmentation refers to the technology of segmenting different structures or tissues (such as teeth, gums, tongue, etc.) in oral images to achieve applications such as oral disease diagnosis, computer-aided dental design, and oral surgery planning;

[0003] At present, great progress has been made in the application of deep learning technology to oral CT / MRI image segmentation. Automatic segmentation of multiple regions such as teeth, maxilla, and soft tissues can be achieved. Structures such as the convolutional neural network UNet and Attention UNet have been successfully applied to oral segmentation tasks, and the segmentation accuracy has been continuously improved. There is already a certain scale of open-source oral image datasets;

[0004] However, compared with other medical images, it is still relatively small. Although automatic segmentation products for oral images have been developed for auxiliary diagnostic analysis, there are still challenges in wide clinical applications. Moreover, the segmentation quality of different oral anatomical structures varies, and fine structures are easily occluded and cannot be segmented, resulting in problems such as over-segmentation and missed segmentation. There is still room for improvement in post-processing. At the same time, due to the influence of interference factors such as light intensity, distance, and relative position on the collected oral images, the segmentation accuracy of oral images is affected. Therefore, a fully automatic multi-tissue segmentation method for oral images is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a fully automatic multi-tissue segmentation method for oral images to solve the problems raised in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A fully automatic multi-tissue segmentation method for oral images, including the following steps:

[0007] S1. Acquisition and segmentation of images: Collect oral image information and segment the acquired oral image information;

[0008] S2. Feature extraction: Extract features for each segmented region or object;

[0009] S3. Multi-level classification: Input the extracted features into a multi-level classification model;

[0010] S4. Result output and post-processing: Post-process the classification results and output the post-processed classification results.

[0011] As a further preference of this technical solution: In S1, the image segmentation runs, including the following steps:

[0012] A1. Image preprocessing: Preliminarily process the input image to improve the image quality and facilitate subsequent regional feature extraction;

[0013] A2. Region generation: After preprocessing, generate initial regions or superpixels using the attributes of the image and the similarity between pixels;

[0014] A3. Region merging: According to the similarity criterion, merge adjacent regions with similar attributes until the stopping criterion is met;

[0015] A4. Postprocessing: After running A3, perform further postprocessing operations on the segmentation result to obtain a more accurate segmentation result.

[0016] As a further preference of this technical solution: In A1, the preprocessing operations include noise reduction, smoothing, and color space conversion. Image preprocessing helps reduce noise and unnecessary details in the image while enhancing the saliency of regional features;

[0017] In A2, the image attributes include color, texture, and grayscale. By region generation, the image is divided into a series of pixel sets with similar attributes, providing a basis for subsequent region merging;

[0018] In A3, the merging process is implemented using the watershed algorithm or K-means clustering. By region merging, pixel regions with similar features are further aggregated to form a more continuous and accurate segmentation result;

[0019] In A4, the postprocessing operations include boundary refinement, denoising, and filling holes. Postprocessing helps eliminate discontinuities and noise in the segmentation result, improving the accuracy and visualization effect of the segmentation.

[0020] As a further preference of this technical solution: In S2, the feature extraction includes three main features: teeth, gums, and tongue;

[0021] Among them, the tooth features include multiple subordinate features such as tooth size, tooth volume, tooth texture, and tooth number. The gum features include multiple subordinate features such as gum color, gum growth edge, and alveolar coverage area. The tongue features include multiple subordinate features such as tongue length size and tongue thickness size.

[0022] As a further preference of this technical solution: In S3, the model classifies each region or object in the image step by step according to the input features until the required classification accuracy and hierarchy are achieved;

[0023] Among them, in multi-level classification, there is usually one or more intermediate layers for more detailed classification of images. The intermediate layers are used to capture more details and features in the images, thereby improving the accuracy of classification.

[0024] As a further preference of this technical solution: in S3, the multi-level classification model runs, including the following steps:

[0025] B1. Data preparation: First, collect and prepare the data set for training the decision tree;

[0026] B2. Feature selection: By calculating the information gain, Gini index or other relevant metrics of each feature, select the feature that can best distinguish the samples as the splitting criterion for the current node;

[0027] B3. Data partitioning: According to the values of the selected features, divide the data set into multiple subsets, and each of the multiple subsets corresponds to a branch in the decision tree;

[0028] B4. Recursively construct subtrees: For each partitioned subset, repeat the steps of B2 and B3 above until the stopping condition is met;

[0029] B5. Pruning: To avoid overfitting of the decision tree, prune the generated decision tree;

[0030] B6. Prediction and classification: When the decision tree is constructed, it can be used to classify or predict new data points.

[0031] As a further preference of this technical solution: in B1, the data set contains the features of the samples and the corresponding labels or categories;

[0032] In B2, the aim is to find the feature that has the greatest impact on the classification result, so as to facilitate dividing the data set into purer subsets;

[0033] In B3, the data set is gradually refined, making the samples in the multiple subsets more homogeneous;

[0034] In B4, the stopping conditions include that the number of samples in the subset is less than a predetermined threshold, all the samples in the subset belong to the same category, or the desired prediction accuracy is reached, and through recursively constructing subtrees, the decision tree gradually grows into a complete tree structure;

[0035] In B5, the purpose of pruning is to simplify the structure of the decision tree and improve the generalization ability of the model, and the pruning strategies include pre-pruning and post-pruning;

[0036] Among them, pre-pruning is carried out during the generation of the decision tree, and overfitting is avoided by stopping the growth of the tree in advance;

[0037] Among them, post-pruning is carried out after the decision tree is generated, and the structure of the tree is simplified by removing unnecessary nodes;

[0038] In B6, starting from the root node of the decision tree, move down along the tree structure according to the feature values of the data points until reaching a leaf node. At this time, the class or predicted value represented by this leaf node is the classification or prediction result of the decision tree for the data points.

[0039] As a further optimization of this technical solution: in S4, the classification result output forms include image labels, bounding boxes or segmentation masks, and post-processing is used to improve the accuracy and visualization effect of the classification results.

[0040] As a further optimization of this technical solution: in S4, post-processing operations include improving the signal-to-noise ratio and contrast. Among them, the signal-to-noise ratio calculation formula is:

[0041]

[0042] In formula (1) thereof, ISNR represents the image signal-to-noise ratio, P S represents the signal power of the image, and P n represents the noise power of the image.

[0043] As a further optimization of this technical solution: the signal-to-noise ratio refers to the ratio of the image signal to the noise signal. A high signal-to-noise ratio means less noise in the image, higher image quality, and the image signal-to-noise ratio is an important indicator to measure image quality and the ratio of signal to noise;

[0044] Among them, the larger the ISNR of the image, the better the image quality, the higher the ratio of signal to noise. A high signal-to-noise ratio means less noise in the image, a clean picture, and clear details, while a low signal-to-noise ratio may lead to a large amount of noise interference (such as "grains" and "snowflakes") in the image, affecting the visual perception and quality of the image.

[0045] As a further optimization of this technical solution: the contrast calculation formula is:

[0046] V = 127 + (V' - 127) × (1 + d)(2)

[0047] In formula (2) thereof, V represents the adjusted color value, V' represents the original color value, 127 represents the reference, that is, the middle value of brightness, and d represents the contrast adjustment coefficient, whose value range is -1 ≤ d ≤ 1. The set formula (2) is used to adjust the contrast of the image, and the contrast is enhanced or weakened by increasing the larger color value and decreasing the smaller color value. Among them, the contrast refers to the difference degree between the bright and dark parts of the image. High contrast can enhance the sense of hierarchy and three-dimensionality of the image, making the image more vivid and lifelike.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] 1. In the present invention, multi-level classification is combined with image processing. By the collaborative operation between multi-level classification and image processing, the accuracy of image recognition is improved, the depth of image understanding is enhanced, and the efficiency of image processing is increased, thereby providing strong support for the development of fields such as computer vision and medical image processing. With the help of the multi-level classifier, the image can be classified from local to global and from rough to fine. This classification method helps to more accurately identify the target object or feature in the image. Moreover, during the image processing process, through multi-level classification, the region of interest can be gradually narrowed down, and the discrimination ability for the target can be enhanced at each stage, thus achieving efficient and accurate target detection;

[0050] 2. In the present invention, multi-level classification helps to achieve a deeper understanding of complex scenes. In the image segmentation task, using a multi-level classifier can classify different regions, thereby realizing pixel-level scene segmentation, further enhancing the understanding and description of the image semantics. At the same time, the design of the multi-level classifier can optimize the image processing process. By classifying and screening at different levels, unnecessary calculations and processing time can be reduced. In some cases, multi-level classification can also perform joint learning with other image processing tasks (such as feature extraction, image segmentation, etc.). By sharing feature extractors and model parameters, the efficiency and accuracy of the model can be improved;

[0051] 3. In the present invention, by combining multi-level classification and image processing technologies, image information from different levels or modalities can be integrated to provide a more comprehensive and reliable image representation, thereby enhancing the understanding of the image content;

[0052] 4. In the present invention, through image segmentation, pixel regions with similar attributes are gradually aggregated, and finally accurate segmentation of the image is achieved. Moreover, the set multi-level classification model is used to perform classification and regression tasks on the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is the operation flow chart of the multi-tissue full-automatic segmentation method for oral images of the present invention;

[0054] Figure 2 is the operation flow chart of the model construction module in the multi-tissue full-automatic segmentation method for oral images of the present invention;

[0055] Figure 3 is the operation flow chart of the image processing module in the multi-tissue full-automatic segmentation method for oral images of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0057] Embodiment

[0058] Please refer to Figures 1 - 3 , the present invention provides a technical solution: an automatic multi-tissue segmentation method for oral images, including the following steps:

[0059] S1. Acquisition and segmentation of images: Collect oral image information and segment the acquired oral image information.

[0060] S2. Feature extraction: Extract features for each segmented region or object.

[0061] S3. Multi-level classification: Input the extracted features into a multi-level classification model.

[0062] S4. Result output and post-processing: Post-process the classification results and output the post-processed classification results.

[0063] In this embodiment, specifically: in S1, the image segmentation operation includes the following steps:

[0064] A1. Image preprocessing: Preliminarily process the input image to improve the image quality and facilitate subsequent regional feature extraction.

[0065] A2. Region generation: After preprocessing, use the attributes of the image and the similarity between pixels to generate initial regions or superpixels.

[0066] A3. Region merging: According to the similarity criterion, merge adjacent regions with similar attributes until the stop criterion is met.

[0067] A4. Post-processing: After running A3, perform further post-processing operations on the segmentation results to obtain more accurate segmentation results.

[0068] In this embodiment, specifically: in A1, the preprocessing operations include noise reduction, smoothing, color space conversion, etc. Image preprocessing helps to reduce noise and unnecessary details in the image, and at the same time enhance the significance of regional features.

[0069] In A2, the image attributes include color, texture, grayscale, etc. Through region generation, the image is divided into a series of pixel sets with similar attributes, providing a basis for subsequent region merging.

[0070] In A3, the merging process is implemented using the watershed algorithm or K-means clustering. Through region merging, pixel regions with similar features are further aggregated to form a more continuous and accurate segmentation result;

[0071] In A4, the post-processing operations include boundary refinement, denoising, hole filling, etc. Post-processing helps to eliminate discontinuities and noise in the segmentation result, improving the accuracy and visualization effect of the segmentation.

[0072] In this embodiment, specifically: in S2, feature extraction includes three main features: teeth, gums, and tongue;

[0073] Among them, the tooth features include multiple subordinate features such as tooth size, tooth volume, tooth texture, and tooth count. The gum features include multiple subordinate features such as gum color, gum growth edge, and alveolar coverage area. The tongue features include multiple subordinate features such as tongue length size and tongue thickness size.

[0074] In this embodiment, specifically: in S3, the model classifies each region or object in the image step by step according to the input features until the required classification accuracy and hierarchy are achieved;

[0075] Among them, in multi-level classification, there is usually one or more intermediate layers for more detailed classification of the image. The intermediate layers are used to capture more details and features in the image, thereby improving the classification accuracy.

[0076] In this embodiment, specifically: in S3, the multi-level classification model runs, including the following steps:

[0077] B1. Data preparation: First, collect and prepare the dataset for training the decision tree;

[0078] B2. Feature selection: By calculating the information gain, Gini index, or other relevant metrics of each feature, select the feature that can best distinguish the samples as the splitting criterion for the current node;

[0079] B3. Data partitioning: According to the values of the selected features, divide the dataset into multiple subsets, and each subset corresponds to a branch in the decision tree;

[0080] B4. Recursively construct subtrees: For each partitioned subset, repeat the steps of B2 and B3 until the stopping condition is met;

[0081] B5. Pruning: To avoid overfitting of the decision tree, prune the generated decision tree;

[0082] B6. Prediction and classification: When the decision tree is constructed, it can be used to classify or predict new data points.

[0083] In this embodiment, specifically: in B1, the data set contains the features of the samples and the corresponding labels or categories;

[0084] In B2, the aim is to find the features that have the greatest impact on the classification results, so as to facilitate the division of the data set into purer subsets;

[0085] In B3, the data set is gradually refined so that the samples in multiple subsets are more homogeneous;

[0086] In B4, the stopping conditions include that the number of samples in the subset is less than a predetermined threshold, all the samples in the subset belong to the same category, or the desired prediction accuracy is achieved, and through recursive construction of subtrees, the decision tree gradually grows into a complete tree structure;

[0087] In B5, the purpose of pruning is to simplify the structure of the decision tree and improve the generalization ability of the model, and the pruning strategies include pre-pruning and post-pruning;

[0088] Among them, pre-pruning is carried out during the generation of the decision tree, and overfitting is avoided by stopping the growth of the tree in advance;

[0089] Among them, post-pruning is carried out after the generation of the decision tree is completed, and the structure of the tree is simplified by removing unnecessary nodes;

[0090] In B6, starting from the root node of the decision tree, move down along the tree structure according to the feature values of the data points until reaching a leaf node. At this time, the category or prediction value represented by this leaf node is the classification or prediction result of the decision tree for the data point.

[0091] In this embodiment, specifically: in S4, the classification result output forms include image labels, bounding boxes, or segmentation masks, etc., and post-processing is used to improve the accuracy and visualization effect of the classification results.

[0092] In this embodiment, specifically: in S4, the post-processing operation includes improving the signal-to-noise ratio and contrast. Among them, the signal-to-noise ratio calculation formula is:

[0093]

[0094] In formula (1) thereof, ISNR represents the image signal-to-noise ratio, P S represents the signal power of the image, P n represents the noise power of the image.

[0095] In this embodiment, specifically: the signal-to-noise ratio refers to the ratio of the image signal to the noise signal. A high signal-to-noise ratio means less noise in the image and higher image quality. The image signal-to-noise ratio is an important indicator for measuring image quality and the ratio of signal to noise;

[0096] Among them, the larger the ISNR of the image, the better the image quality, and the higher the ratio of signal to noise. A high signal-to-noise ratio means less noise in the image, a clean picture, and clear details. On the other hand, a low signal-to-noise ratio may result in a large amount of noise interference (such as "granules" and "snowflakes") in the image, affecting the visual perception and quality of the image.

[0097] In this embodiment, specifically, the contrast calculation formula is:

[0098] V = 127 + (V' - 127) × (1 + d)(2)

[0099] In formula (2), V represents the adjusted color value, V' represents the original color value, 127 represents the reference, that is, the middle value of brightness, and d represents the contrast adjustment coefficient, and its value range is -1 ≤ d ≤ 1.

[0100] In this embodiment, specifically, formula (2) is set to adjust the contrast of the image. By increasing the larger color value and decreasing the smaller color value, the enhancement or weakening of the contrast is achieved. Among them, the contrast refers to the difference degree between the bright and dark parts of the image. High contrast can enhance the sense of hierarchy and three-dimensionality of the image, making the image more vivid and realistic.

[0101] Working principle or structural principle: When in use and operation, first collect oral image information, and segment the obtained oral image information. Image segmentation performs preliminary processing on the input image to improve the image quality and facilitate subsequent regional feature extraction. After preprocessing, using the attributes of the image and the similarity between pixels, initial regions or superpixels are generated, and according to the similarity criterion, adjacent regions with similar attributes are merged until the stop criterion is met. And after the region merging operation, further post-processing operations are performed on the segmentation result to obtain a more accurate segmentation result. After that, feature extraction is performed on each segmented region or object;

[0102] Input the extracted features into a multi-level classification model. First, collect and prepare the dataset for training the decision tree. By calculating the information gain, Gini index or other relevant metrics of each feature, select the feature that can best distinguish the samples as the splitting criterion for the current node, and divide the dataset into multiple subsets according to the value of the selected feature. And multiple subsets all correspond to a branch in the decision tree. Among them, for each divided subset, repeat the above running steps until the stop condition is met. And to avoid overfitting of the decision tree, pruning is performed on the generated decision tree. When the decision tree is constructed, it can be used to classify or predict new data points. Finally, post-process the classification results and output the post-processed classification results.

[0103] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fully automatic multi-tissue segmentation method for oral images, characterized in that: The following steps are involved: S1. Image acquisition and segmentation: collecting oral image information and segmenting the acquired oral image information; S2, feature extraction: extract features from each segmented region or object; S3, multi-class classification: the extracted features are input into the multi-class classification model; S4. Result output and post-processing: post-process the classification results and output the classification results after post-processing.

2. The method for fully automatic segmentation of multiple tissues of oral images according to claim 1, characterized in that: In S1, the image segmentation is run, which includes the following steps: A1. Image preprocessing: Perform preliminary processing on the input image to improve image quality and facilitate subsequent regional feature extraction; A2. Region generation: After preprocessing, the initial regions or superpixels are generated by using the image attributes and the similarity between pixels; A3. Region merging: According to the similarity criterion, adjacent regions with similar attributes are merged until the stopping criterion is met; A4, post-processing: After A3 is run, the segmentation results are further post-processed to obtain more accurate segmentation results.

3. The method for fully automatic segmentation of multiple tissues of oral images according to claim 2, characterized in that: In A1, preprocessing operations include noise reduction, smoothing, and color space conversion. Image preprocessing helps reduce noise and unnecessary details in the image while enhancing the significance of regional features. In A2, image attributes include color, texture, and grayscale. Region generation is used to divide the image into a series of pixel sets with similar attributes, thus providing a basis for subsequent region merging. In A3, the merging process is implemented using the watershed algorithm or K-means clustering, and pixel regions with similar features are further aggregated through region merging to form a more continuous and accurate segmentation result; In A4, post-processing operations include boundary refinement, denoising, and hole filling. Post-processing helps eliminate discontinuities and noise in the segmentation results and improves segmentation accuracy and visualization.

4. The method for fully automatic segmentation of multiple tissues of oral images according to claim 1, characterized in that: In S2, feature extraction includes three main features: teeth, gums, and tongue; Among them, tooth characteristics include multiple subordinate characteristics such as tooth size, tooth volume, tooth texture and tooth number; gum characteristics include multiple subordinate characteristics such as gum color, gum growth edge and gum coverage area; tongue characteristics include multiple subordinate characteristics such as tongue length and tongue thickness.

5. The method for fully automatic segmentation of multiple tissues of oral images according to claim 1, characterized in that: In S3, the model classifies each region or object in the image step by step according to the input features until the required classification accuracy and level are achieved; Among them, in multi-level classification, there are usually one or more intermediate layers for more detailed classification of images. The intermediate layers are used to capture more details and features in the image, thereby improving the accuracy of classification.

6. The method for fully automatic segmentation of multiple tissues of oral images according to claim 1, characterized in that: In S3, the multi-class classification model is run, which includes the following steps: B1. Data preparation: First, collect and prepare the data set for training the decision tree; B2. Feature selection: By calculating the information gain, Gini index or other related indicators of each feature, the feature that can best distinguish the samples is selected as the splitting criterion for the current node; B3. Data partitioning: According to the value of the selected feature, the data set is divided into multiple subsets, and each subset corresponds to a branch in the decision tree; B4, recursively construct subtrees: for each divided subset, repeat the above steps B2 and B3 until the stopping condition is met; B5. Pruning: To avoid overfitting of the decision tree, prune the generated decision tree; B6. Prediction and classification: Once the decision tree is built, it can be used to classify or predict new data points.

7. The method for fully automatic segmentation of multiple tissues of oral images according to claim 6, characterized in that: In B1, the dataset contains the features of the samples and the corresponding labels or categories; In B2, the goal is to find the features that have the greatest impact on the classification results, so as to facilitate the division of the data set into purer subsets; In B3, the data set is gradually refined to make the samples in multiple subsets more homogeneous; In B4, the stopping conditions include that the number of samples in the subset is less than a predetermined threshold, that all samples in the subset belong to the same category, or that the desired prediction accuracy is achieved, and that the decision tree gradually grows into a complete tree structure by recursively constructing subtrees; In B5, the purpose of pruning is to simplify the structure of the decision tree and improve the generalization ability of the model, and the pruning strategies include pre-pruning and post-pruning; Among them, pre-pruning is performed during the decision tree generation process to avoid overfitting by stopping the growth of the tree in advance; Among them, post-pruning is performed after the decision tree is generated, simplifying the tree structure by removing unnecessary nodes; In B6, starting from the root node of the decision tree, we move down along the tree structure according to the feature value of the data point until we reach a leaf node. At this time, the category or prediction value represented by the leaf node is the classification or prediction result of the decision tree for the data point.

8. The method for fully automatic segmentation of multiple tissues of oral images according to claim 1, characterized in that: In S4, the classification results are output in the form of image labels, bounding boxes or segmentation masks, and post-processing is used to improve the accuracy and visualization of the classification results.

9. The method for fully automatic segmentation of multiple tissues of oral images according to claim 8, characterized in that: In S4, the post-processing operation includes improving the signal-to-noise ratio and contrast, wherein the signal-to-noise ratio is calculated as: In formula (1), ISNR is the image signal-to-noise ratio, P S Expressed as the signal power of the image, P n Expressed as the noise power of the image.

10. The method for fully automatic segmentation of multiple tissues of oral images according to claim 9, characterized in that: The contrast ratio is calculated as: V=127+(V'-127)×(1+d) (2) In formula (2), V represents the adjusted color value, V' represents the original color value, 127 represents the reference, that is, the middle value of the brightness, and d represents the contrast adjustment coefficient, whose value range is -1≤d≤1.

Images