A dental image segmentation method

Abstract

This invention discloses a dental image segmentation method, belonging to the fields of image processing and machine learning, including the following steps: S1, constructing a dental image segmentation dataset and data augmentation; S2, designing a dental image segmentation algorithm; S3, training the dental image segmentation algorithm; S4, inferring the dental image segmentation algorithm. This invention establishes a large amount of data that is easy to collect, and can fully utilize the data to construct an effective tooth feature model, thereby further improving the efficiency of oral health management, that is, improving the service quality and level in the industry; and can effectively solve the problem that existing methods still have the problem of incomplete or missing tooth segments in the extracted tooth CT images, effectively improving the feature expression ability.

Description

Technical Field

[0001] This invention relates to the fields of image processing and machine learning, specifically a dental image segmentation method. Background Technology

[0002] With economic and social development and the improvement of people's living standards, oral health has gradually become a concern. The health status of oral cavity also indirectly reflects the level of health and civilization of a country's residents. In China, the oral health and dental orthodontics industry has gradually become one of the industries with the greatest development potential. Promoting the digitalization and informatization of the oral health industry is now imperative. Dental image segmentation methods mainly process CT oral images to segment the distribution of all teeth in the oral cavity, and finally combine the specific distribution of teeth in the oral cavity to carry out comprehensive treatment of affected teeth.

[0003] Artificial intelligence and image recognition technologies have demonstrated their powerful vitality across various industries. Using computer vision to detect oral health problems can effectively reduce the workload of dental specialists, improve the efficiency of oral health management, and enhance the service quality and level in the industry. However, oral image data is scarce and difficult to collect. Constructing an effective tooth feature model using limited data is key to effectively utilizing image recognition technology. Furthermore, existing methods still suffer from incomplete or missing tooth segmentation images during the extraction of dental CT images. Improving the expressive power of features is also an urgent problem to be solved. To address these issues, we propose a dental image segmentation method. Summary of the Invention

[0004] The purpose of this invention is to provide a dental image segmentation method that can generate a large amount of data that is easy to collect, and can fully utilize the data to construct an effective tooth feature model, thereby further improving the efficiency of oral health management, that is, improving the service quality and level in the industry; and can effectively solve the problem that existing methods still have the problem of incomplete or missing tooth segments in the extracted tooth CT images, effectively improving the feature expression ability.

[0005] To solve the above problems, the present invention adopts the following technical solution:

[0006] A dental image segmentation method includes the following steps:

[0007] S1. Constructing a dental image segmentation dataset and data augmentation: Annotating the collected CT oral image data to obtain the shape annotation information and category information of each tooth;

[0008] S2. Design of a dental image segmentation algorithm: Considering the characteristics of blurred boundaries and complex gradients in oral CT images, the segmentation algorithm needs to acquire a large amount of high-resolution information. Furthermore, the internal structure of the oral cavity is relatively fixed, and the distribution of the teeth to be segmented in the human image is very regular, with simple and clear semantics. Low-resolution information can effectively locate them. Therefore, the overall algorithm network structure adopts a classic U-shaped network structure design, which mainly consists of a feature encoding network and a feature decoding network.

[0009] The feature encoding network uses a lightweight feature extraction operator module, and the feature encoding network is mainly composed of a convolutional neural network combined with a transformer-based attention network, which enables the extracted features to have powerful feature representation capabilities while possessing global context information. The operator adopts a feature information stacking form to simplify the computational load when the convolutional neural network generates redundant information during the information extraction process.

[0010] The specific implementation steps are as follows:

[0011] A. First, extract the full set of features f' from the input features f using conventional convolution;

[0012] B. The full set of features f' is subjected to identity mapping operation using depthwise convolution to obtain the identity mapping information f”;

[0013] C. Divide the full set of features into two parts and superimpose them onto the features of the identity mapping, f”. This is equivalent to obtaining the entire feature information by performing only one regular convolution.

[0014] D. Repeat operations A and B once on the output feature in C and output the final combined feature;

[0015] S3. Dental image segmentation algorithm training: The ADMA optimization algorithm is selected as the optimizer during training iteration. The data is divided into training set and validation set in a 9:1 ratio. Cross-validation is used for training. The number of iterations during training is e, and the initial learning rate is set to l.

[0016] S4. Dental Image Segmentation Algorithm Inference: Input the oral CT image into the algorithm prediction interface to obtain the segmented images of 32 teeth output by the algorithm.

[0017] As a preferred embodiment of the present invention, the decoding network in S2 mainly utilizes spatial multi-scale perceptual convolution to enhance the output information of the previous transformer features, and then uses feature fusion to obtain the features of the previous layer with a larger receptive field to enhance the pixel classification effect, thereby improving the accuracy of the algorithm in obtaining the segmentation results of 32 teeth.

[0018] As a preferred embodiment of the present invention, the spatial multi-scale perceptual convolution operation in S2 is specifically performed as follows:

[0019] a. The feature size output by the transformer network is [patch, D], where D represents the dimension of the output feature of the transformer network, and patch is... First, perform a scaling operation on the features. We obtain the feature x, and then use a feature attention mechanism to obtain the weight importance vector R that is applied to the convolution attention calculation.

[0020] b. Then, the weights within a set of convolutional kernels with kernel size k = [[k1,k1],[k2,k2],[k3,k3]] are adjusted. The weights of convolutions of different sizes are weighted by the values ​​in the weight importance vector at the corresponding index positions to obtain a new set of convolutional weight parameters [[k′1,k′1],[k′2,k′2],[k′3,k′3]]. Then, the newly generated convolutions are used to calculate the input features respectively. Finally, the features extracted by all convolutions are scaled and fused to output as the enhanced features.

[0021] c. Finally, transform the feature scale extracted by convolutions of different sizes in the convolutional group to... The features obtained by the convolution of the three are fused to enhance the information deficiency of the transformer network in local feature extraction.

[0022] In a preferred embodiment of the present invention, the operators in S2 are embedded in the residual structure module, wherein N lightweight feature extraction operator modules are cascaded, allowing the CT oral cavity images to be downsampled by 2x, 4x, 8x, and 16x in the network, and finally the output feature map is segmented into The feature encoding task is completed after the feature blocks are input into the transformer network.

[0023] As a preferred embodiment of the present invention, the feature decoding steps in S2 are as follows: the transformer network first processes the features output from the feature encoding network... The features are fed into a spatial multi-scale perceptual convolution operator to enhance the local information representation capability of the features in the previous transformer layer, and then the features are upsampled to the feature size. The size of the downsampled feature map in the previous feature encoding network is... The features are fused, and the fused features are then upsampled to a feature size of [value missing] after a convolution operation. The feature map of the same size as the previous feature encoding network is fused together, and the above operation is repeated until the feature map is restored to the original size. Finally, after a fully connected operation, the feature map [w, h, 32] is output, where 32 represents the teeth segmented from 32 instances.

[0024] As a preferred embodiment of the present invention, the weight importance vector R assigned to the convolutional attention is calculated as follows:

[0025] R=[r1,..,rd]=softmax(Conv(Relu(Conv(avg(x))))) (1)

[0026] Where d represents the dimension of the weight importance vector, x represents the input features, avg represents the mean pooling operation, Conv represents the 1x1 convolution operation, ReLU represents the activation function, and softmax represents the softmax function.

[0027] As a preferred embodiment of the present invention, the weights of the spatial multi-scale perceptual convolution... The calculation is as follows:

[0028]

[0029] Where r i This indicates that the i-th weight in the weight importance vector R is used, w i This represents the original weights of the convolution.

[0030] Compared with the prior art, the advantages of this invention are:

[0031] This solution establishes a large amount of data that is easy to collect, and can fully utilize the data to construct an effective tooth feature model, thereby further improving the efficiency of oral health management, which can improve the service quality and level in the industry; and it can effectively solve the problem that existing methods still have incomplete or missing tooth segmentation images when extracting tooth CT images, effectively improving the feature expression ability. Attached Figure Description

[0032] Figure 1 This is a network structure diagram of the oral cavity segmentation algorithm of the present invention.

[0033] Figure 2 This is a flowchart of the lightweight feature extraction operator module of the present invention.

[0034] Figure 3 This is a flowchart illustrating the spatial multi-scale perceptual convolution operation of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0036] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "top / bottom," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0037] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "sleeved / connected," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0038] Example:

[0039] Please see Figure 1-3 A dental image segmentation method includes the following steps:

[0040] S1. Constructing a dental image segmentation dataset and data augmentation: Annotating the collected CT oral image data to obtain the shape annotation information and category information of each tooth;

[0041] S2. Design of a dental image segmentation algorithm: Considering the characteristics of blurred boundaries and complex gradients in oral CT images, the segmentation algorithm needs to acquire a large amount of high-resolution information. Furthermore, the internal structure of the oral cavity is relatively fixed, and the distribution of the teeth to be segmented in the human image is very regular, with simple and clear semantics. Low-resolution information can effectively locate them. Therefore, the overall algorithm network structure adopts a classic U-shaped network structure design, which mainly consists of a feature encoding network and a feature decoding network.

[0042] The feature encoding network uses a lightweight feature extraction operator module, and the feature encoding network is mainly composed of a convolutional neural network combined with a transformer-based attention network, which enables the extracted features to have powerful feature representation capabilities while possessing global context information. The operator adopts a feature information stacking form to simplify the computational load when the convolutional neural network generates redundant information during the information extraction process.

[0043] The specific implementation steps are as follows:

[0044] A. First, extract the full set of features f' from the input features f using conventional convolution;

[0045] B. The full set of features f' is subjected to identity mapping operation using depthwise convolution to obtain the identity mapping information f”;

[0046] C. Divide the full set of features into two parts and superimpose them onto the features of the identity mapping, f”. This is equivalent to obtaining the entire feature information by performing only one regular convolution.

[0047] D. Repeat operations A and B once on the output feature in C and output the final combined feature;

[0048] S3. Dental image segmentation algorithm training: The ADMA optimization algorithm is selected as the optimizer during training iteration. The data is divided into training set and validation set in a 9:1 ratio. Cross-validation is used for training. The number of iterations during training is e, and the initial learning rate is set to l.

[0049] S4. Dental Image Segmentation Algorithm Inference: Input the oral CT image into the algorithm prediction interface to obtain the segmented images of 32 teeth output by the algorithm.

[0050] Specifically, the decoding network in S2 mainly uses spatial multi-scale perceptual convolution to enhance the output information of the previous transformer features, and then uses feature fusion to obtain the features of the previous layer with a larger receptive field to enhance the pixel classification effect, ultimately improving the accuracy of the algorithm in obtaining the segmentation results of 32 teeth.

[0051] Specifically, the spatial multi-scale perceptual convolution operation in S2 is as follows:

[0052] a. The feature size output by the transformer network is [patch, D], where D represents the dimension of the output feature of the transformer network, and patch is... First, perform a scaling operation on the features. We obtain the feature x, and then use a feature attention mechanism to obtain the weight importance vector R that is applied to the convolution attention calculation.

[0053] b. Then, the weights within a set of convolutional kernels with kernel size k = [[k1,k1],[k2,k2],[k3,k3]] are adjusted. The weights of convolutions of different sizes are weighted by the values ​​in the weight importance vector at the corresponding index positions to obtain a new set of convolutional weight parameters [[k′1,k′1],[k′2,k′2],[k′3,k′3]]. Then, the newly generated convolutions are used to calculate the input features respectively. Finally, the features extracted by all convolutions are scaled and fused to output as the enhanced features.

[0054] c. Finally, transform the feature scale extracted by convolutions of different sizes in the convolutional group to... The features obtained by the convolution of the three are fused to enhance the information deficiency of the transformer network in local feature extraction.

[0055] Specifically, the operators in S2 are embedded into the residual structure modules, where N lightweight feature extraction operator modules are cascaded, allowing CT oral cavity images to be downsampled by 2x, 4x, 8x, and 16x in the network. Finally, the output feature map is segmented into... The feature encoding task is completed after the feature blocks are input into the transformer network.

[0056] Specifically, the feature decoding steps in S2 are as follows: the transformer network first converts the features output by the feature encoding network into... The features are fed into a spatial multi-scale perceptual convolution operator to enhance the local information representation capability of the features in the previous transformer layer, and then the features are upsampled to the feature size. The size of the downsampled feature map in the previous feature encoding network is... The features are fused, and the fused features are then upsampled to a feature size of [value missing] after a convolution operation. The feature map of the same size as the previous feature encoding network is fused together, and the above operation is repeated until the feature map is restored to the original size. Finally, after a fully connected operation, the feature map [w, h, 32] is output, where 32 represents the teeth segmented from 32 instances.

[0057] Specifically, the weight importance vector R assigned to the convolution attention is calculated as follows:

[0058] R = [r1,..,r d ]=softmax(Conv(Relu(Conv(avg(x))))) (1)

[0059] Where d represents the dimension of the weight importance vector, x represents the input features, avg represents the mean pooling operation, Conv represents the 1x1 convolution operation, ReLU represents the activation function, and softmax represents the softmax function.

[0060] Specifically, the weights of spatial multi-scale perceptual convolutions The calculation is as follows:

[0061]

[0062] Where r i This indicates that the i-th weight in the weight importance vector R is used, w i This represents the original weights of the convolution.

[0063] Working principle:

[0064] S1. Constructing a dental image segmentation dataset and data augmentation: Annotating the collected CT oral image data to obtain the shape annotation information and category information of each tooth;

[0065] S2. Design of dental image segmentation algorithm: Considering that oral CT images have blurred boundaries and complex gradients, the segmentation algorithm needs to acquire more high-resolution information. In addition, the internal structure of the oral cavity is relatively fixed, and the distribution of the teeth to be segmented in the human body image is very regular, with simple and clear semantics. Low-resolution information can be used to effectively locate them. Therefore, the overall algorithm network structure adopts the classic U-shaped network structure design.

[0066] The algorithm mainly consists of a feature encoding network and a decoding network. The feature encoding network is mainly composed of a convolutional neural network combined with a transformer-based attention network, which enables the extracted features to have strong feature representation capabilities while also possessing global context information. The decoding network mainly uses spatial multi-scale perceptual convolution to enhance the output information of the previous transformer features, and then uses feature fusion to obtain the features of the previous layer with a larger receptive field to enhance the pixel classification effect, ultimately improving the accuracy of the algorithm in obtaining the segmentation results of 32 teeth.

[0067] Feature Encoding: Due to the powerful feature extraction performance of convolutional neural networks, some redundant information will inevitably be introduced into the network. This paper uses a lightweight feature extraction operator module in the feature encoding network. The operator adopts a feature information stacking form to simplify the computational load when the convolutional neural network generates redundant information during the information extraction process.

[0068] The specific implementation steps are as follows:

[0069] A. First, extract the full set of features f' from the input features f using conventional convolution;

[0070] B. The full set of features f' is subjected to identity mapping operation using depthwise convolution to obtain the identity mapping information f”;

[0071] C. Divide the full set of features into two parts and superimpose them onto the features of the identity mapping, f”. This is equivalent to obtaining the entire feature information by performing only one regular convolution.

[0072] D. Repeat the A and B operations on the output feature of C once, and then output the final combined feature.

[0073] The aforementioned lightweight operators are embedded within the residual structure modules. N lightweight feature extraction operator modules are cascaded, allowing CT oral cavity images to be downsampled by 2x, 4x, 8x, and 16x in the network. Finally, the output feature map is segmented into... After being input into the transformer network in the form of small feature blocks, the feature encoding task is completed.

[0074] Feature decoding: The transformer network first decodes the features output in (1) The features are fed into a spatial multi-scale perceptual convolution operator to enhance the local information representation capability of the features in the previous transformer layer, and then the features are upsampled to the feature size. The size of the downsampled feature map in the previous feature encoding network is... The features are fused, and the fused features are then upsampled to a feature size of [value missing] after a convolution operation. The feature map of the same size as the previous feature encoding network is fused together, and the above operation is repeated until the feature map is restored to the original size. Finally, after a fully connected operation, the feature map [w,h,32] is output, where 32 represents the teeth segmented from 32 instances.

[0075] The specific operations of spatial multi-scale perceptual convolution are as follows:

[0076] The feature size output by the transformer network is [patch, D], where D represents the dimension of the output feature of the transformer network, and patch is... First, perform a scaling operation on the features. We obtain the feature x, and then use a feature attention mechanism to obtain the weight importance vector R that is applied to the convolution attention calculation.

[0077] The weight importance vector R, which assigns attention to the convolution, is calculated as follows:

[0078] R = [r1,..,r d]=softmax(Conv(Relu(Conv(avg(x))))) (1)

[0079] Where d represents the dimension of the weight importance vector, x represents the input features, avg represents the mean pooling operation, Conv represents the 1x1 convolution operation, ReLU represents the activation function, and softmax represents the softmax function.

[0080] Next, the weights within a set of convolutional kernels with kernel size k = [[k1,k1],[k2,k2],[k3,k3]] are adjusted. The weights of convolutions of different sizes are weighted by the values ​​in the weight importance vector at the corresponding index positions to obtain a new set of convolutional weight parameters [[k′1,k′1],[k′2,k′2],[k′3,k′3]]. Then, the newly generated convolutions are used to calculate the input features respectively. Finally, the features extracted by all convolutions are scaled and fused to output as the enhanced features.

[0081] Weights of spatial multi-scale perceptual convolution The calculation is as follows:

[0082]

[0083] Where r i This indicates that the i-th weight in the weight importance vector R is used, w i This represents the original weights of the convolution;

[0084] Finally, the feature scales extracted from convolutions of different sizes in the convolutional group are transformed to... The features obtained by the convolution of the three are fused to enhance the information deficiency of the transformer network in local feature extraction;

[0085] S3. Dental image segmentation algorithm training: The ADMA optimization algorithm is selected as the optimizer during training iteration. The data is divided into training set and validation set in a 9:1 ratio. Cross-validation is used for training. The number of iterations during training is e, and the initial learning rate is set to l.

[0086] S4. Dental Image Segmentation Algorithm Inference: Input the oral CT image into the algorithm prediction interface to obtain the segmented images of 32 teeth output by the algorithm.

[0087] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and its improved concept, should be covered within the scope of protection of the present invention.

Claims

1. A dental image segmentation method, characterized in that, Includes the following steps: S1. Constructing a dental image segmentation dataset and data augmentation: Annotating the collected CT oral image data to obtain the shape annotation information and category information of each tooth; S2. Design of a dental image segmentation algorithm: Considering the characteristics of blurred boundaries and complex gradients in oral CT images, the segmentation algorithm needs to acquire a large amount of high-resolution information. Furthermore, the internal structure of the oral cavity is relatively fixed, and the distribution of the teeth to be segmented in the human image is very regular, with simple and clear semantics. Low-resolution information can effectively locate them. Therefore, the overall algorithm network structure adopts a classic U-shaped network structure design, which mainly consists of a feature encoding network and a feature decoding network. The feature encoding network uses a lightweight feature extraction operator module, and the feature encoding network is mainly composed of a convolutional neural network combined with a transformer-based attention network, which enables the extracted features to have powerful feature representation capabilities while possessing global context information. The operator adopts a feature information stacking form to simplify the computational load when the convolutional neural network generates redundant information during the information extraction process. The specific implementation steps are as follows: A. First, extract the full set of features f' from the input features f using conventional convolution; B. The full set of features f' is subjected to identity mapping operation using depthwise convolution to obtain the identity mapping information f”; C. Divide the full set of features into two parts and superimpose them onto the features of the identity mapping, f”. This is equivalent to obtaining the entire feature information by performing only one regular convolution. D. Repeat operations A and B once on the output feature in C and output the final combined feature; S3. Dental image segmentation algorithm training: The ADMA optimization algorithm is selected as the optimizer during training iteration. The data is divided into training set and validation set in a 9:1 ratio. Cross-validation is used for training. The number of iterations during training is e, and the initial learning rate is set to l. S4. Dental Image Segmentation Algorithm Inference: Input the oral CT image into the algorithm prediction interface to obtain the segmented images of 32 teeth output by the algorithm.

2. The dental image segmentation method according to claim 1, characterized in that: In S2, the decoding network mainly uses spatial multi-scale perceptual convolution to enhance the output information of the previous transformer features, and then uses feature fusion to obtain features with a larger receptive field in the previous layer to enhance the pixel classification effect, ultimately improving the accuracy of the algorithm in obtaining the segmentation results of 32 teeth.

3. The dental image segmentation method according to claim 1, characterized in that: The specific operations of spatial multi-scale perceptual convolution in S2 are as follows: a. The feature size output by the transformer network is [patch, D], where D represents the dimension of the output feature of the transformer network, and patch is... First, perform a scaling operation on the features. We obtain the feature x, and then use a feature attention mechanism to obtain the weight importance vector R that is applied to the convolution attention calculation. b. Then, the weights within a set of convolutional kernels with kernel size k = [[k1,k1],[k2,k2],[k3,k3]] are adjusted. The weights of convolutions of different sizes are weighted by the values ​​in the weight importance vector at the corresponding index positions to obtain a new set of convolutional weight parameters [[k′1,k′1],[k′2,k′2],[k′3,k′3]]. Then, the newly generated convolutions are used to calculate the input features respectively. Finally, the features extracted by all convolutions are scaled and fused to output as the enhanced features. c. Finally, transform the feature scale extracted by convolutions of different sizes in the convolutional group to... The features obtained by the convolution of the three are fused to enhance the information deficiency of the transformer network in local feature extraction.

4. The dental image segmentation method according to claim 1, characterized in that: The operators in S2 are embedded into the residual structure modules, where N lightweight feature extraction operator modules are cascaded to downsample CT oral images by 2x, 4x, 8x, and 16x. Finally, the output feature maps are segmented into... The feature encoding task is completed after the feature blocks are input into the transformer network.

5. The dental image segmentation method according to claim 1, characterized in that: The working steps of the feature decoding network in S2 are as follows: The transformer network first processes the features output from the feature encoding network... The features are fed into a spatial multi-scale perceptual convolution operator to enhance the local information representation capability of the features in the previous transformer layer, and then the features are upsampled to the feature size. The size of the downsampled feature map in the previous feature encoding network is... The features are fused, and the fused features are then upsampled to a feature size of [value missing] after a convolution operation. The feature map of the same size as the previous feature encoding network is fused together, and the above operation is repeated until the feature map is restored to the original size. Finally, after a fully connected operation, the feature map [w, h, 32] is output, where 32 represents the teeth segmented from 32 instances.

6. The dental image segmentation method according to claim 3, characterized in that: The weight importance vector R assigned to the convolution attention is calculated as follows: R=[r1,..,r d ]=softmax(Conv(Relu(Conv(avg(x))))) (1) Where d represents the dimension of the weight importance vector, x represents the input features, avg represents the mean pooling operation, Conv represents the 1x1 convolution operation, ReLU represents the activation function, and softmax represents the softmax function.

7. The dental image segmentation method according to claim 3, characterized in that: Weights of spatial multi-scale perceptual convolution The calculation is as follows: Where r i This indicates that the i-th weight in the weight importance vector R is used, w i This represents the original weights of the convolution.

Images