A method and system for automatic segmentation of dental plaque in two-dimensional images

Abstract

The present invention discloses a method and system for automatic segmentation of dental plaque in a two-dimensional image, and relates to the technical fields of computer vision and image processing. The context information of the input image is encoded and sent to an encoder module composed of a self-attention mechanism and Kmeans clustering. After the encoded image context information is input into the encoder module, a self-attention map and a clustering feature map are obtained, and the two are combined to obtain the encoded data. The data encoded by the encoder will be sent to a decoder composed of a multi-layer perception mechanism. The decoder passes the encoded data through a convolutional layer and a fully connected layer, and outputs the final segmentation mask. The image is input into the network for training until the training converges, and finally a complete model is obtained. The present invention can automatically divide the area of ​​teeth and dental plaque, has high recognition accuracy, and solves the problems of poor model convergence and poor edge detail segmentation effect.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and more particularly to a method and system for automatically segmenting dental plaque from a two-dimensional image. Background Art

[0002] Currently, dental plaque is the primary causative factor for both periodontal disease and dental caries. These two diseases are collectively referred to as plaque diseases. Plaque is a widespread, unmineralized bacterial deposit that adheres firmly to tooth surfaces and restorations. It consists of a sticky matrix and embedded bacteria. As plaque accumulates on the tooth surface, the bacteria and their metabolites affect various tooth components, causing decalcification and even dental caries.

[0003] The traditional method of detecting dental plaque is to identify the boundaries of diseased tissue through clinical examination and probing by doctors. However, this method is highly dependent on the doctor's experience. Different doctors have different subjective evaluation standards. Even the diagnosis of the same doctor can vary due to the influence of different environments and instruments. Existing caries and dental plaque detection equipment abroad utilizes the fluorescence effect of teeth and adopts quantitative light-induced fluorescence (QLF) technology. A special camera is used to receive reflected fluorescence images, and the fluorescence loss is obtained by reconstructing the fluorescence of healthy enamel. The percentage difference between the actual surface and the reconstructed surface is calculated to determine the reduction in fluorescence. Any area with a fluorescence reduction of more than 5% is considered a lesion. Existing caries and dental plaque detection equipment is not only expensive, but also has no intelligent diagnostic output and can only be used as an auxiliary device. Human intervention is still required to make an effective diagnosis.

[0004] Therefore, how to provide an intelligent automatic dental plaque segmentation method that can automatically divide the area of ​​teeth and dental plaque is a problem that those skilled in the art urgently need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a two-dimensional image dental plaque automatic segmentation method and system, which can automatically divide the area of ​​teeth and dental plaque with high recognition accuracy, and solve the problems of poor model convergence and poor edge detail segmentation effect.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for automatic segmentation of dental plaque from a two-dimensional image, comprising:

[0007] Obtain two-dimensional image information of teeth;

[0008] Encoding based on contextual information of the two-dimensional image;

[0009] The encoded image information is introduced into the self-attention mechanism to obtain the self-attention map;

[0010] At the same time, the encoded image information is used to extract image features and cluster similar features to obtain a clustering feature map of the data;

[0011] Combining the self-attention map and the clustering feature map to obtain final encoded data;

[0012] The encoded data is input into a linear decoding network, and segmentation mask information of a two-dimensional image is obtained after being processed by the linear decoding network.

[0013] Specifically, by inputting the context information of the two-dimensional image into an encoder, the encoder encodes the three-channel image information into specific data.

[0014] The three channels are RGB image information, and the RGB image information of the original two-dimensional image is deepened to 32 using convolution, and the size is reduced to 128*128 for encoding.

[0015] Specifically, the steps for obtaining the self-attention map are as follows:

[0016] The encoded image information is introduced into the multi-head self-attention layer composed of the self-attention mechanism; the construction function of the multi-head self-attention layer is as follows:

[0017]

[0018] Among them, Q, K, and V are obtained by linear transformation of the input image code, d head Indicates the number of heads in the multi-head self-attention layer. It can be used to obtain the self-attention map of the model for the input image.

[0019] Among them, Q represents the query vector, K represents the construction vector, and V represents the value vector. Q, K, and V are all obtained through linear transformation based on the context information of the two-dimensional image and can be used as representatives of the data.

[0020] Specifically, the steps of obtaining the cluster feature map of the data are as follows:

[0021] The encoded image information is input into the convolutional network and the clustering network composed of Kmeans to extract image features and cluster similar features;

[0022] The construction function of the clustering network is:

[0023]

[0024] where c (i) represents the class that is closest to the sample i among the k classes, c(i) The value is one from 1 to k, the centroid μ j Represents the guess of the center point of samples belonging to the same category, μ i represents the cluster center before updating, x (i) For each sample to be clustered, m represents the total number of samples to be clustered, and j represents the jth cluster. This step can obtain the corresponding pixel clustering features from the original encoded image.

[0025] Specifically, the construction function combining the self-attention map and the clustering feature map is:

[0026] Output(A,C)=Attention(A)+Cluster(C);

[0027] Where A represents the attention map being manipulated, C represents the clustering map being manipulated, and Output represents the features finally output by the encoder. This step combines the self-attention map with the clustering feature map to obtain a hybrid feature map.

[0028] Specifically, the linear decoding network is composed of a convolutional layer, a linear layer, and a multi-layer perceptron; the linear decoding network construction function is:

[0029]

[0030] Among them, C represents the dimension of the data, Linear(C i ,C) means that C is transformed by linear transformation i The dimension is transformed into C dimension, Downsample means downsampling to a specific size, and Concat means concatenation operation; F i represents the characteristics of the encoder output, Represents the data after each layer of the decoder operation, F represents the intermediate variable of the decoder, M represents the final segmentation mask Mask, N cls Indicates the final segmentation category, C i Represents the dimension of the encoder output feature, H and W represent the height and width of the original input two-dimensional image respectively.

[0031] The segmentation categories include three categories: background, teeth and dental plaque; after the above steps, the feature map generated by the encoder can be converted into the final background, teeth and dental plaque segmentation masks.

[0032] Specifically, the linear decoder will merge feature information of large and small scales to achieve global field of view perception.

[0033] A two-dimensional image dental plaque automatic segmentation system, comprising:

[0034] An image acquisition module, used for acquiring a two-dimensional image of the teeth;

[0035] An encoder module, configured to obtain contextual information of the two-dimensional image, and combine a self-attention map obtained from the contextual information with a clustering feature map to generate final encoded data;

[0036] A decoder module, configured to decode the encoded data to ultimately obtain segmentation mask information of the two-dimensional image;

[0037] A generating module, configured to generate a two-dimensional segmented image with image semantic information according to the segmentation mask information;

[0038] Wherein, the encoder module includes an attention module and a clustering module;

[0039] The attention module is used to obtain a self-attention map based on the context information of the two-dimensional image;

[0040] The clustering module is used to obtain a clustering feature map according to the context information of the two-dimensional image.

[0041] It can be seen from the above technical solution that compared with the prior art, the present invention provides a two-dimensional image dental plaque automatic segmentation method and system, which encodes the context information of the input image and sends it to an encoder module composed of a self-attention mechanism and Kmeans clustering. After the encoded image context information is input into the encoder module, a self-attention map and a clustering feature map will be obtained, and the two will be combined to obtain the encoded data. The data encoded by the encoder will be sent to a decoder composed of a multi-layer perception mechanism. The decoder passes the encoded data through a convolutional layer and a fully connected layer, and outputs the final segmentation mask. The image is input into the network for training until the training converges, and finally a complete model is obtained. The present invention can automatically divide the areas of teeth and dental plaque, has high recognition accuracy, and solves the problems of poor model convergence and poor edge detail segmentation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0043] Figure 1 Schematic diagram of the method flow of the present invention;

[0044] Figure 2 This is a display of dental plaque segmentation results according to an embodiment of the present invention;

[0045] Figure 3 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] The embodiment of the present invention discloses a method for automatic segmentation of dental plaque in a two-dimensional image. Figure 1 Shown, including:

[0048] Obtain two-dimensional image information of teeth;

[0049] Encoding based on contextual information of the two-dimensional image;

[0050] The encoded image information is introduced into the self-attention mechanism to obtain the self-attention map;

[0051] At the same time, the encoded image information is used to extract image features and cluster similar features to obtain a clustering feature map of the data;

[0052] Combining the self-attention map and the clustering feature map to obtain final encoded data;

[0053] The encoded data is input into a linear decoding network, and segmentation mask information of a two-dimensional image is obtained after being processed by the linear decoding network.

[0054] Specifically, by inputting the context information of the two-dimensional image into an encoder, the encoder encodes the three-channel image information into specific data.

[0055] The three channels are RGB image information, and the RGB image information of the original two-dimensional image is deepened to 32 using convolution, and the size is reduced to 128*128 for encoding.

[0056] Specifically, the steps for obtaining the self-attention map are as follows:

[0057] The encoded image information is introduced into the multi-head self-attention layer composed of the self-attention mechanism; the construction function of the multi-head self-attention layer is as follows:

[0058]

[0059] Among them, Q, K, and V are obtained by linear transformation of the input image code, d headIndicates the number of heads in the multi-head self-attention layer. Obtains the self-attention map of the model for the input image.

[0060] Among them, Q represents the query vector, K represents the construction vector, and V represents the value vector. Q, K, and V are all obtained by linear transformation of the context information of the two-dimensional image and can be used as representatives of the data.

[0061] Specifically, the clustering module is a clustering network composed of a convolutional network and Kmeans, which is used to extract image features and cluster similar features; the construction function of the clustering network is:

[0062]

[0063] where c (i) represents the class that is closest to the sample i among the k classes, c (i) The value is one from 1 to k, the centroid μ j Represents the guess of the center point of samples belonging to the same category, μ i represents the cluster center before updating, x (i) For each sample to be clustered, m represents the total number of samples to be clustered, and j represents the jth cluster. This step can obtain the corresponding pixel clustering features from the original encoded image.

[0064] Specifically, the construction function combining the self-attention map and the clustering feature map is:

[0065] Output(A,C)=Attention(A)+Cluster(C);

[0066] Where A represents the manipulated attention map, C represents the manipulated cluster map, and Output represents the features finally output by the encoder. The self-attention map is combined with the cluster feature map to obtain a hybrid feature map.

[0067] Specifically, the linear decoding network is composed of a convolutional layer, a linear layer, and a multi-layer perceptron; the linear decoding network construction function is:

[0068]

[0069] Among them, C represents the dimension of the data, Linear(C i ,C) means that C is transformed by linear transformation i The dimension is transformed into C dimension, Downsample means downsampling to a specific size, and Concat means concatenation operation; F i represents the characteristics of the encoder output, Represents the data after each layer of the decoder operation, F represents the intermediate variable of the decoder, M represents the final segmentation mask Mask, Ncls Indicates the final segmentation category, C i Represents the dimension of the encoder output feature, H and W represent the height and width of the original input two-dimensional image respectively.

[0070] Among them, N cls Represents the final segmentation category, which includes three categories: background, teeth, and dental plaque. After the above steps, the feature map generated by the encoder can be converted into the final background, teeth, and dental plaque segmentation masks.

[0071] Specifically, the linear decoder will merge feature information of large and small scales to achieve global field of view perception.

[0072] In order to compare the proposed method with other high-performance semantic segmentation methods in a quantitative manner, we show the performance of the state-of-the-art methods in the prior art: Segformer and SegNeXt. The data and results are shown in Table 1.

[0073] Table 1. Comparison results of automatic segmentation of dental plaque in 2D images

[0074]

[0075]

[0076] As can be seen from the above table, the automatic segmentation method for dental plaque in a two-dimensional image described in the present invention has higher segmentation results for teeth and dental plaque than the segmentation methods in the prior art, such as Segformer and SegNeXt.

[0077] An automatic segmentation system for dental plaque in 2D images, such as Figure 3 Shown, including:

[0078] An image acquisition module, used for acquiring a two-dimensional image of the teeth;

[0079] An encoder module, configured to obtain contextual information of the two-dimensional image, and combine a self-attention map obtained from the contextual information with a clustering feature map to generate final encoded data;

[0080] A decoder module, configured to decode the encoded data to ultimately obtain segmentation mask information of the two-dimensional image;

[0081] A generating module, configured to generate a two-dimensional segmented image with image semantic information according to the segmentation mask information;

[0082] Wherein, the encoder module includes an attention module and a clustering module;

[0083] The attention module is used to obtain a self-attention map based on the context information of the two-dimensional image;

[0084] The clustering module is used to obtain a clustering feature map according to the context information of the two-dimensional image.

[0085] In one embodiment of the present invention, Figure 2 As shown, there are 6 tooth images of different shooting angles and different parts selected in the embodiment of the present invention; wherein, the first horizontal row is the original tooth two-dimensional image information obtained, the second horizontal row is the reference information, that is, the real dental plaque mark image; the third horizontal row is the dental plaque mark information detected by the original tooth image using the SegFormer semantic segmentation method in the prior art, and the fourth horizontal row is the dental plaque information detected by using a two-dimensional image dental plaque automatic segmentation method described in the present invention; in the embodiment of the present invention, the areas of teeth and plaque are automatically divided by only inputting the tooth image. When the computer program is executed by the processor, a dental plaque photo collected by a digital camera device is first prepared, and then the original RGB dental plaque image is input into the program. The program will be automatically executed and finally output the segmentation results of the teeth and dental plaque, wherein purple is the background, green is the teeth, yellow is the dental plaque, and the arrow points to the area where the dental plaque is located. By comparison, it can be found that compared with the existing technology, the two-dimensional image dental plaque automatic segmentation method described in the present invention can more accurately detect the distribution range of dental plaque, and will not miss the detected dental plaque location (semantic segmentation method); it can serve patients more efficiently.

[0086] The present invention provides a method and system for automatic segmentation of dental plaque in a two-dimensional image, which encodes the context information of the input image and feeds it into an encoder module composed of a self-attention mechanism and Kmeans clustering. After the encoded image context information is input into the encoder module, a self-attention map and a clustering feature map are obtained, and the two are combined to obtain encoded data. The data encoded by the encoder will be sent to a decoder composed of a multi-layer perception mechanism. The decoder passes the encoded data through a convolutional layer and a fully connected layer, and outputs a final segmentation mask. The image is input into the network for training until the training converges, and finally a complete model is obtained. The present invention can automatically divide the areas of teeth and dental plaque, has high recognition accuracy, and solves the problems of poor model convergence and poor edge detail segmentation effect.

[0087] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0088] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for automatic segmentation of dental plaque from a two-dimensional image, characterized in that: include: Obtain two-dimensional image information of teeth; Encoding based on contextual information of the two-dimensional image; The encoded image information is introduced into the self-attention mechanism to obtain the self-attention map; At the same time, the encoded image information is used to extract image features and cluster similar features to obtain a clustering feature map of the data; Combining the self-attention map and the clustering feature map to obtain final encoded data; Inputting the encoded data into a linear decoding network, and obtaining segmentation mask information of the two-dimensional image after being processed by the linear decoding network; The specific steps of obtaining the self-attention map are as follows: The encoded image information is introduced into the multi-head self-attention layer composed of the self-attention mechanism; the construction function of the multi-head self-attention layer is as follows: Among them, Q represents the query vector, K represents the construction vector, and V represents the value vector. Q, K, and V are obtained by linear transformation of the input image code. head Indicates the number of heads in the multi-head self-attention layer; The specific steps of obtaining the clustering feature map of the data are as follows: The encoded image information is input into the convolutional network and the clustering network composed of Kmeans to extract image features and cluster similar features; The construction function of the clustering network is: where c (i) represents the class that is closest to the sample i among the k classes, c (i) The value is one from 1 to k, the centroid μ j Represents the guess of the center point of samples belonging to the same category, μ i represents the cluster center before updating, x (i) For each sample to be clustered, m represents the total number of samples to be clustered, and j represents the jth cluster.

2. The method for automatic segmentation of dental plaque from a two-dimensional image according to claim 1, characterized in that: By inputting context information of the two-dimensional image into an encoder, the encoder encodes three-channel image information into specific data.

3. The method for automatic segmentation of dental plaque from a two-dimensional image according to claim 1, characterized in that: The construction function of the combination of the self-attention map and the cluster feature map is: Output(A,C)=Attention(A)+Cluster(C); Among them, A represents the operated attention map, C represents the operated clustering map, and Output represents the final output feature of the encoder.

4. The method for automatic segmentation of dental plaque from a two-dimensional image according to claim 1, characterized in that: The linear decoding network is composed of a convolutional layer, a linear layer, and a multi-layer perceptron; the linear decoding network construction function is: Among them, C represents the dimension of the data, Linear(C i ,C) means that C is transformed by linear transformation i The dimension is transformed into C dimension, Downsample means downsampling to a specific size, and Concat means concatenation operation; F i represents the characteristics of the encoder output, Represents the data after each layer of the decoder operation, F represents the intermediate variable of the decoder, M represents the final segmentation mask Mask, N cls Represents the final segmentation category, C i Represents the dimension of the encoder input feature, H and W represent the height and width of the original input two-dimensional image respectively.

5. A two-dimensional image dental plaque automatic segmentation system, characterized in that: include: An image acquisition module, used for acquiring a two-dimensional image of the teeth; An encoder module, configured to obtain contextual information of the two-dimensional image, and combine a self-attention map obtained from the contextual information with a clustering feature map to generate final encoded data; A decoder module, configured to decode the encoded data to ultimately obtain segmentation mask information of the two-dimensional image; A generating module, configured to generate a two-dimensional segmented image with image semantic information according to the segmentation mask information; Wherein, the encoder module includes an attention module and a clustering module; The attention module is used to obtain a self-attention map based on the context information of the two-dimensional image; The clustering module is used to obtain a clustering feature map according to the context information of the two-dimensional image; The specific steps to obtain the self-attention map are as follows: The encoded image information is introduced into the multi-head self-attention layer composed of the self-attention mechanism; the construction function of the multi-head self-attention layer is as follows: Among them, Q represents the query vector, K represents the construction vector, and V represents the value vector. Q, K, and V are obtained by linear transformation of the input image code. head Indicates the number of heads in the multi-head self-attention layer; The specific steps to obtain the clustering feature map of the data are as follows: The encoded image information is input into the convolutional network and the clustering network composed of Kmeans to extract image features and cluster similar features; The construction function of the clustering network is: where c (i) represents the class that is closest to the sample i among the k classes, c (i) The value is one from 1 to k, the centroid μ j Represents the guess of the center point of samples belonging to the same category, μ i represents the cluster center before updating, x (i) For each sample to be clustered, m represents the total number of samples to be clustered, and j represents the jth cluster.

Images