Method and device for automatically detecting multiple teeth, storage medium and electronic equipment

Abstract

The invention discloses a multi-generation tooth automatic detection method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring a three-dimensional image, and preprocessing the three-dimensional image to obtain a plurality of image blocks; inputting the image blocks into the trained two-stage deep attention network for identification to obtain a probability value that the three-dimensional image contains multiple teeth; wherein the two-stage deep attention network comprises a positioning sub-network and a classification sub-network; and inputting the image blocks into the trained two-stage deep attention network for identification: inputting the image blocks into a positioning sub-network to obtain a rectangular positioning frame, and inputting the rectangular positioning frame into a classification sub-network to obtain a probability value that the image blocks contain multiple raw teeth. According to the invention, the detection speed of multi-generation teeth can be increased, and the classification precision is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, storage medium and electronic device for automatically detecting supernumerary teeth. Background Art

[0002] Teeth that exceed the normal number of teeth in the dentition are called supernumerary teeth. They are a common developmental abnormality of teeth, often seen in the maxillary anterior region. Their morphologies include conical, tuberous, supplementary, and odontoma. Supernumerary teeth often cause problems such as abnormal eruption of adjacent teeth and root resorption. Supernumerary teeth in the anterior teeth often cause crowding or abnormal interdental spaces, while severe misalignment and deformity may also lead to caries, periodontal disease, and trauma. Therefore, the diagnosis and treatment of supernumerary teeth requires early diagnosis based on clinical and radiographic examinations, and supernumerary tooth positioning is the key to assisting in the treatment of supernumerary teeth.

[0003] In recent years, deep learning systems based on convolutional neural networks have been introduced into the field of oral and maxillofacial diagnostic imaging, showing great potential in the application of supernumerary tooth detection and diagnosis. For example, methods using several commonly used deep learning models to detect supernumerary teeth in the maxillary anterior region of curved tomographic images have achieved high classification accuracy, but this method is limited to the permanent dentition. To address this limitation, methods using models such as SqueezeNet, ResNet-18, ResNet-101, and Inception-ResNet-V2 have achieved high-accuracy classification of supernumerary teeth in mixed dentition, enabling more accurate and rapid diagnosis for clinicians with limited clinical experience. However, this method also suffers from the limitation of requiring manual annotation of the maxillary anterior region. In the DeeplabV3plus model, the Inception-ResNet-V2 model is used to classify supernumerary teeth, achieving a fully automated supernumerary tooth detection process. These detection methods are all based on two-dimensional curved tomographic images, which may not provide sufficient information for further diagnosis and treatment of complex anatomical structures or lesions. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, storage medium and electronic device for automatic detection of supernumerary teeth, which provide rich 3D spatial information through three-dimensional images, avoid the influence of spatial overlap of oral anatomical structures in two-dimensional curved surface layer images on recognition, provide more accurate three-dimensional spatial position, and accelerate the detection speed of supernumerary teeth and improve classification accuracy.

[0005] The present application provides a method for automatically detecting supernumerary teeth, comprising: Acquire a three-dimensional image, and pre-process the three-dimensional image to obtain a plurality of image blocks; Inputting the image block into a trained two-stage deep attention network for recognition, and obtaining a probability value of the supernumerary tooth being contained in the three-dimensional image; In which, the two-stage deep attention network includes a positioning subnetwork and a classification subnetwork; the inputting of the image block into the trained two-stage deep attention network for recognition includes: inputting the image block into the positioning subnetwork to obtain a rectangular positioning frame, and inputting the rectangular positioning frame into the classification subnetwork to obtain a probability value of the image block containing supernumerary teeth.

[0006] Furthermore, in the above-mentioned supernumerary tooth automatic detection method, the positioning sub-network includes an encoder, and the encoder includes a pooling layer, a CGR module and a plurality of feature extraction branches; Inputting the image block into the positioning sub-network to obtain the rectangular positioning frame includes: Input the image block into two CGR modules to obtain a low-level semantic feature map; Inputting the low-level semantic feature map into the first feature extraction branch and the second feature extraction branch to obtain a mid-level semantic feature map; Inputting the middle-level semantic feature map into the first feature extraction branch and the second feature extraction branch to obtain a first high-level semantic feature map; The first high-level semantic feature map is input into the pooling layer and a CGR module to obtain a low-resolution high-level semantic feature map.

[0007] Furthermore, in the above-mentioned supernumerary tooth automatic detection method, the positioning sub-network further includes a decoder, and the decoder includes an upsampling layer, a CGR module, and a three-dimensional convolutional layer; The step of inputting the image block into the positioning sub-network to obtain the rectangular positioning frame further includes: Passing the low-resolution high-level semantic feature map through a CGR module and an upsampling layer and then splicing it with the first high-level semantic feature map to obtain a small-size fused feature map; Passing the small-size fused feature map through two CGR modules and an upsampling layer and then concatenating it with the middle-layer semantic feature map to obtain a medium-size fused feature map; Passing the medium-sized fused feature map through two CGR modules and an upsampling layer and then splicing it with the low-resolution high-level semantic feature map to obtain a large-sized fused feature map; The large-size fusion feature map is input into two CGR modules and a three-dimensional convolution layer to obtain a segmentation result, and the rectangular positioning frame is obtained based on the segmentation result.

[0008] Furthermore, in the above-mentioned supernumerary tooth automatic detection method, obtaining the rectangular positioning frame based on the segmentation result includes: Obtaining a probability value of each pixel point in the segmentation result belonging to the maxillary anterior teeth area, and when the probability value is greater than a preset probability threshold, using the corresponding pixel point as a candidate point of the region of interest; Combining adjacent candidate points of the region of interest to obtain multiple three-dimensional connected domains; The volume and circumscribed rectangular frame of each connected domain are obtained through the connected domain labeling algorithm; The circumscribed rectangular frame is used as a candidate detection frame, and connected domains with a volume smaller than a preset volume threshold are screened out to obtain a candidate detection frame of the target; Merge the circumscribed rectangular frame of the largest connected domain and the candidate detection frames of the nearest neighbors to obtain a rectangular positioning frame; The coordinate values ​​of the rectangular positioning frame in the image block are calculated, the coordinate values ​​are used as boundary range coordinate values, and the distance between the center point of the rectangular positioning frame and the fixed point position is calculated based on the boundary range coordinate values.

[0009] Furthermore, in the above-mentioned supernumerary tooth automatic detection method, the classification subnetwork includes a CBR module, a maximum pooling layer, multiple residual modules, an average pooling layer, and a fully connected layer; Inputting the rectangular positioning frame into the classification subnetwork to obtain a probability value of the image block containing supernumerary teeth includes: Input the rectangular positioning box into a CBR module, a maximum pooling layer and four residual modules to obtain a high-order feature map; The high-order feature map is input into the average pooling layer and the fully connected layer to obtain a probability value of the image block containing supernumerary teeth.

[0010] Furthermore, in the above-mentioned supernumerary tooth automatic detection method, the training process of the two-stage deep attention network includes: Acquire a three-dimensional image, obtain a position mark of a maxillary anterior teeth region and supernumerary tooth category information of the maxillary anterior teeth region in the three-dimensional image, and form an image data set; Preprocessing the images in the image data set by a three-dimensional sliding block algorithm to obtain a plurality of image blocks; Inputting the preprocessed image dataset into the positioning sub-network to obtain a rectangular positioning frame, and training based on the position annotation of the maxillary anterior region, the cross entropy loss function of the segmentation result, and the hybrid loss function of the distance between the center point and the vertex position of the rectangular positioning frame; The rectangular positioning frame is input into the classification subnetwork to obtain a probability value of containing supernumerary teeth, and training is performed based on the supernumerary tooth category information of the maxillary anterior teeth area and a cross entropy loss function.

[0011] Furthermore, in the above-mentioned supernumerary tooth automatic detection method, the preprocessing of the images in the image dataset by a three-dimensional sliding block algorithm comprises: For the image dataset, the size is D × W × H For a three-dimensional image, set a size of d × w × h The sliding window has a moving step size of ( sd,sw,sh ), generate the slice range corresponding to each image block in the complete image according to the window and step size: The number of slices in the Z direction is

[0012] The number of slices in the Y direction is:

[0013] The number of slices in the X direction is:

[0014] For slices , calculate the coordinate range of each slice:

[0015] Among them, the integer . The present application also provides an automatic supernumerary tooth detection device, comprising: a preprocessing module, configured to acquire a three-dimensional image and preprocess the three-dimensional image to obtain a plurality of image blocks; A prediction module, configured to input the image block into a trained two-stage deep attention network for recognition, and obtain a probability value of the supernumerary tooth being contained in the three-dimensional image; In which, the two-stage deep attention network includes a positioning subnetwork and a classification subnetwork; the inputting of the image block into the trained two-stage deep attention network for recognition includes: inputting the image block into the positioning subnetwork to obtain a rectangular positioning frame, and inputting the rectangular positioning frame into the classification subnetwork to obtain a probability value of the image block containing supernumerary teeth.

[0016] An embodiment of the present application also provides a computer-readable storage medium, in which a plurality of instructions are stored. The instructions are suitable for being loaded by a processor to execute any of the above-mentioned methods for automatically detecting supernumerary teeth.

[0017] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to perform the steps in any of the above-mentioned methods for automatically detecting supernumerary teeth.

[0018] The present application provides a method, device, storage medium and electronic device for automatic detection of supernumerary teeth. The present application obtains the probability value of supernumerary teeth by detecting three-dimensional images, provides rich 3D spatial information through three-dimensional images, avoids the influence of spatial overlap of oral anatomical structures in two-dimensional curved surface slice images on recognition, and can provide more accurate three-dimensional spatial positions. The two-stage deep attention network of the present invention includes a positioning subnetwork and a classification subnetwork. The positioning subnetwork is used to segment the rectangular positioning frame, and the classification subnetwork is used to identify and calculate the probability value of supernumerary teeth. The positioning subnetwork is based on the segmentation network framework. Compared with the traditional detection network, it does not need to process a large number of anchor points and anchor frames, which speeds up the detection reasoning speed. The classification subnetwork integrates the original input information features, optimizes the feature expression efficiency, enhances the network generalization ability, and can improve the classification accuracy. The present application obtains the probability value of supernumerary teeth in the image by recognizing the three-dimensional image, which allows users to read the image and the probability value recognized by the image more intuitively, and can assist users in making accurate judgments on supernumerary teeth. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The following detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings will make the technical solutions and other beneficial effects of the present application apparent.

[0020] Figure 1 This is a flow chart of the method for automatically detecting supernumerary teeth provided in an embodiment of the present application.

[0021] Figure 2 Another flow chart of the method for automatically detecting supernumerary teeth provided in an embodiment of the present application.

[0022] Figure 3 A schematic diagram of the structure of a two-stage deep attention network provided in an embodiment of the present application.

[0023] Figure 4 A schematic diagram of a process for obtaining a rectangular positioning frame is provided in an embodiment of the present application.

[0024] Figure 5 A schematic diagram of a rectangular positioning frame provided in an embodiment of the present application.

[0025] Figure 6 This is a schematic diagram of the structure of the automatic detection device for supernumerary teeth provided in an embodiment of the present application.

[0026] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0028] Embodiments of the present application provide a method, device, storage medium, and electronic device for automatically detecting supernumerary teeth. The device for automatically detecting supernumerary teeth provided in embodiments of the present application can be integrated into an electronic device, such as a terminal or server. The terminal may include a tablet computer, laptop computer, personal computer (PC), microprocessor, or other device.

[0029] See also Figure 1 and Figure 2 , Figure 1 This is a flow chart of the method for automatically detecting supernumerary teeth provided in an embodiment of the present application. Figure 2 Another flowchart of the method for automatically detecting supernumerary teeth provided in an embodiment of the present application, which is applied to an electronic device, includes the following steps: S1, obtaining a three-dimensional image, and preprocessing the three-dimensional image to obtain multiple image blocks.

[0030] Specifically, the three-dimensional image is a three-dimensional cone-beam CT image.

[0031] S2, input the image block into the trained two-stage deep attention network for recognition, and obtain the probability value of the supernumerary tooth in the three-dimensional image; Among them, the two-stage deep attention network includes a positioning subnetwork and a classification subnetwork; the image block is input into the trained two-stage deep attention network for recognition, including: inputting the image block into the positioning subnetwork to obtain a rectangular positioning box of the image block, and inputting the rectangular positioning box into the classification subnetwork to obtain the probability value of the image block containing supernumerary teeth.

[0032] In one embodiment, Figure 3 For a schematic diagram of the structure of the two-stage deep attention network provided in the embodiment of the present application, please refer to Figure 3 The positioning sub-network includes an encoder and a decoder. In step S2, the image block is input into the positioning sub-network to obtain a rectangular positioning frame of the image block, specifically including: S211, input the image block into two CGR modules to obtain a low-level semantic feature map; S212, inputting the low-level semantic feature map into the first feature extraction branch and the second feature extraction branch to obtain a mid-level semantic feature map; S213, inputting the middle-level semantic feature map into the first feature extraction branch and the second feature extraction branch to obtain a first high-level semantic feature map; S214: Input the first high-level semantic feature map into a maximum pooling layer and a CGR module to obtain a low-resolution high-level semantic feature map.

[0033] Specifically, the encoder first receives a 3D image patch as input. The input passes through two layers of 3D convolution, group normalization, and activation function combination modules, namely two CGR (Convolution+GroupNorm+ReLU) modules, to output a low-level semantic feature map with the same resolution as the input 3D image. The low-level semantic feature map is then fed into two feature extraction branches. Specifically, the low-level semantic feature map is fed into the first feature extraction branch and then skip-connected to a feature map at the same resolution of the decoder. This skip-connected feature map is then fed into the second feature extraction branch, which includes a max pooling layer and two CGR modules. This produces a mid-level semantic feature map with half the original resolution. The above two branches are repeated for the mid-level semantic feature map, outputting a high-level semantic feature map. The high-level semantic feature map is then fed through a max pooling layer and a CGR module to produce a low-resolution high-level semantic feature map.

[0034] Furthermore, the step of inputting the image block into the positioning subnetwork to obtain a rectangular positioning frame of the image block in step S2 further includes: S215, passing the low-resolution high-level semantic feature map through a CGR module and an upsampling layer and then splicing it with the first high-level semantic feature map to obtain a small-size fused feature map; S216, passing the small-size fused feature map through two CGR modules and an upsampling layer and then concatenating it with the middle-layer semantic feature map to obtain a medium-size fused feature map; S217, the medium-sized fused feature map is passed through two CGR modules and an upsampling layer and then concatenated with the low-resolution high-level semantic feature map to obtain a large-sized fused feature map; S218: Input the large-size fusion feature map into two CGR modules and a three-dimensional convolution layer to obtain a segmentation result, and obtain a rectangular positioning box based on the segmentation result.

[0035] Specifically, the low-resolution high-level semantic feature map output by the encoder is first used as the input of the decoder. After passing through a CGR module and an upsampling layer to restore the resolution to the same as that of the high-level semantic feature map, the high-level semantic feature map from the encoder jump connection branch is received, and the two are spliced ​​channel by channel to increase the dimension of the feature to obtain a small-sized fused feature map. The small-sized fused feature map is then passed through two CGR modules and an upsampling layer again, and spliced ​​with the middle-level semantic feature map to output a medium-sized fused feature map. The above operations are repeated on the medium-sized fused feature map in sequence to output a large-sized fused feature map. The large-sized fused feature map passes through two CGR modules and then a 1×1×1 three-dimensional convolutional layer to obtain a segmentation result consistent with the size of the input three-dimensional image. Each value of the result represents the probability value of the pixel point at the corresponding position belonging to the maxillary anterior tooth area.

[0036] Furthermore, step S218 may include the following steps: S2181, obtaining a probability value of each pixel point in the segmentation result belonging to the maxillary anterior teeth area, and when the probability value is greater than a preset probability threshold, taking the corresponding pixel point as a candidate point of the region of interest; S2182, combining adjacent candidate points of the region of interest to obtain multiple three-dimensional connected domains; S2183, obtaining the volume and circumscribed rectangular box of each connected domain through a connected domain labeling algorithm; S2184: Using the bounding rectangle as a candidate detection frame, filtering out connected domains whose volume is smaller than a preset volume threshold to obtain a candidate detection frame for the target; S2185: Merge the bounding rectangle of the largest connected domain and the candidate detection boxes of the nearest neighbors to obtain a rectangular positioning box. S2186, calculating the coordinate values ​​of the rectangular positioning frame in the image block, and using the coordinate values ​​as the boundary range coordinate values; S2187: Calculate the distance between the center point and the vertex position of the rectangular positioning frame based on the boundary range coordinate values.

[0037] Specifically, the voxel points at the corresponding positions of the 3D cone beam CT image are obtained through the positioning sub-network p n The probability value of belonging to the maxillary anterior area P n , when the probability value P n Greater than the preset probability threshold P When , it is regarded as the foreground, that is, the candidate point of the region of interest, and the adjacent candidate points are combined to form m A three-dimensional connected domain { A1, A2...Am}, using the connected domain labeling algorithm to obtain the volume and circumscribed rectangular box of each connected domain as a candidate detection frame, filter out the connected domains whose volume is less than the preset volume threshold, and obtain a set of candidate detection frames that meet the requirements { Box1, Box2...Boxm}, then the nearest neighbor non-connected domain is merged, based on the maximum connected domain Amax The bounding rectangle of Boxn , the candidate detection box of the nearest neighbor { Boxi...Boxj}and Boxn Merge to get the final predicted rectangular positioning box DetBox , its position coordinates are ( xmin, xmax, ymin, ymax, zmin, zmax ). Finally, according to the scaling ratio Scale calculate DetBox The coordinate values ​​in the original resolution image ( xmin, xmax, ymin, ymax, zmin,zmax ) as the boundary range coordinate value of the rectangular positioning frame, and further calculate the center point and vertex coordinates of the rectangular positioning frame.

[0038] In one embodiment, Figure 4 The schematic diagram of the process of obtaining the rectangular positioning frame provided in the embodiment of the present application can be found in Figure 4 , combined with Figure 4 Describe the specific steps of the nearest neighbor non-connected domain merging algorithm: For the maximum connected domain Amax The bounding rectangle of Boxn , whose coordinates are ( xmin n , xmax n , ymin n , ymax n ， zmin n , zmax n ), for the detection boxes of other connected domains { Box1, Box2...Boxm} in a box Boxi , whose coordinates are ( xmin i , xmax i , ymin i , ymax i , zmin i , zmax i ),calculate Boxn and Boxi The minimum distance in different directions in three-dimensional space is as follows:

[0039] Similarly:

[0040]

[0041] dx i 、 dy i 、 dz i for X 、 Y 、 Z The minimum distance in the direction, take the maximum value dmax :

[0042] When the maximum value does not exceed the preset distance threshold dt When Boxi Put it into the set of neighbor detection frames, repeat the above operation, and get a series of neighbor detection frames { Boxi...Boxj}, the neighbor detection box and Boxn Merge into a rectangular positioning box DetBox , where the coordinates of the merged detection box are calculated as follows, Boxk ϵ{ Boxi...Boxj}:

[0043]

[0044] By the same process, we obtain DetBox Coordinate values ​​of ymin 、 ymax 、 zmin and zmax .

[0045] See Figure 3 The classification subnetwork includes a CBR module, a maximum pooling layer, multiple residual modules, an average pooling layer, and a fully connected layer. In step S2, the step of inputting the rectangular positioning box into the classification subnetwork to obtain the probability value of the image block containing supernumerary teeth specifically includes: S221, input the rectangular positioning frame into a CBR module, a maximum pooling layer and four residual modules to obtain a high-order feature map; Among them, the feature extraction part contains 4 residual modules, each of which is composed of 2 identical or similar basic residual modules.

[0046] The first residual module consists of two basic residual modules based on the attention mechanism with the same structure. The basic residual module includes a CBR module, a three-dimensional convolution, a batch normalization, and an SE (Squeeze-and-Excitation) attention module connected in sequence. The rectangular positioning box is sequentially passed through the CBR module, three-dimensional convolution, and batch normalization to obtain the first feature map. The first feature is passed through the SE (Squeeze-and-Excitation) attention module to obtain the second feature map. The first feature map and the second feature map are added channel by channel, and then after the activation function, the feature-enhanced fusion feature map is obtained as the output of the first basic residual module; the feature-enhanced fusion feature map is again input into the second basic residual module with the same structure as above to obtain the first fusion feature.

[0047] The second residual module consists of two basic residual modules with the same structure. Unlike the first residual module, there is no attention module. In the second residual module, the first fused feature is passed through two CBR modules in sequence to obtain a first intermediate feature map, which is then fused with the first fused feature map. The fused feature map is then repeatedly operated to obtain a second fused feature map.

[0048] The third residual module consists of two basic residual modules with similar structures, differing in whether downsampling is performed. Specifically, the second fused feature map is sequentially passed through two CBR modules to obtain a second intermediate feature map, which is then input into the maximum pooling layer to obtain a third intermediate feature map. The second intermediate feature map and the third intermediate feature map are concatenated to obtain a third fused feature map. The third fused feature map is sequentially passed through two CBR modules to obtain a third intermediate feature map, which is then concatenated with the third fused feature map to obtain a fourth fused feature map.

[0049] The structure of the fourth residual module is consistent with that of the third residual module. After receiving the fourth fusion feature map, the fourth residual module passes through the downsampling basic residual module (containing the maximum pooling layer) and the non-downsampling basic residual module (excluding the maximum pooling layer) in sequence to obtain the high-order feature map of the segmented image block.

[0050] In the attention module, a global average pooling operation is applied to the feature map with an input size of D×H×W×C to obtain a compressed feature map with a size of 1×1×1×C. The compressed feature map passes through two fully connected layers to obtain a weight coefficient with a size of 1×1×1×C. The weight coefficient is used to reflect the importance between C different channels. The weight coefficient is then multiplied by the attention module input feature map channel by channel to obtain a feature map with enhanced channel features.

[0051] S222: Input the high-order feature map into the average pooling layer and the fully connected layer to obtain the probability value of the image block containing supernumerary teeth.

[0052] Specifically, the classification subnetwork consists of a feature extraction part and a classification prediction part. The feature extraction part is responsible for extracting the features of the maxillary anterior teeth area. It receives a rectangular positioning frame as input, and sequentially passes through a three-dimensional convolution, batch normalization, and ReLU activation function combination module, namely the CBR combination module. Then, after passing through a maximum pooling layer, a feature map with a resolution reduced to half of the input image is obtained. The downsampled feature map is then passed through four residual modules in sequence to output a high-order feature map of the maxillary anterior teeth area image block. The classification prediction part takes the high-order feature map obtained by the feature extraction part as input, and sequentially passes through average pooling and fully connected layers, and finally outputs the confidence level of the maxillary anterior teeth area containing central supernumerary teeth, that is, the probability value of predicting the presence or absence of supernumerary teeth.

[0053] Figure 5 A schematic diagram of a rectangular positioning frame provided in an embodiment of the present application is shown in FIG. Figure 5 As shown in the figure, each column from top to bottom is the transverse section, sagittal section and coronal section of the same three-dimensional image. Figure 5 The rectangular box in the figure is the maxillary anterior tooth area where the algorithm of the present invention detects the possible presence of centric supernumerary teeth in the three-dimensional cone-beam CT image. The label "nst" above the rectangular box represents the absence of centric supernumerary teeth, and the label "st" represents the presence of centric supernumerary teeth. The numbers represent the confidence level of the absence or presence of supernumerary teeth.

[0054] The above is the application process of the two-stage deep attention network. The following introduces the training process of the two-stage deep attention network: SA1, collects three-dimensional images, obtains the position annotation of the maxillary anterior tooth area and the supernumerary tooth category information of the maxillary anterior tooth area in the three-dimensional image, and forms an image data set.

[0055] The position annotation method is as follows: the maxillary anterior teeth area is used as the region of interest (ROI) and marked with a three-dimensional rectangular frame; the center point coordinates of the ROI in the three-dimensional image are extracted ( x, y, z ) and the corresponding width and height ( w, h, d ) as the location annotation information of the maxillary anterior teeth area required for training.

[0056] The category labeling method is as follows: using three-dimensional images, the tooth position and morphology in the maxillary anterior area are analyzed from the cross-section and three-dimensional spatial model, and the presence of centrifugal supernumerary teeth in the maxillary anterior area is used as the classification labeling information required for training. If centrifugal supernumerary teeth exist in the selected area of ​​interest, it is labeled as 1, otherwise it is labeled as 0.

[0057] SA2, preprocesses the images in the image dataset through a three-dimensional sliding block algorithm to obtain multiple image blocks.

[0058] Before the 3D image is input into the localization subnetwork, a segmentation mask of the same size as the 3D image is generated, that is, the pixel value of the ROI area is 1 and the pixel value of the non-ROI area is 0. Then, a 3D sliding block algorithm is adopted to divide the cone-beam CT image and the segmentation mask into multiple image blocks and mask blocks with partial overlap using a 3D sliding window. These blocks are input into the network respectively. At the same time, according to the input mask block, the 3D coordinates of the 1 center point and 8 vertices of the ROI area in the block are obtained to facilitate the calculation of the subsequent block mixing loss.

[0059] Specifically, for an image dataset with a size of D × W × H For a three-dimensional image, set a size of d × w × h The sliding window moves with a step size of ( sd,sw,sh ), generate the slice range corresponding to each image block in the complete image according to the window and step size: The number of slices in the Z direction is

[0060] The number of slices in the Y direction is:

[0061] The number of slices in the X direction is:

[0062] For slices , calculate the coordinate range of each slice:

[0063] Among them, the integer .

[0064] According to the calculated slice range, the image blocks of the corresponding range are taken out from the three-dimensional image and input into the positioning sub-network.

[0065] SA3, the preprocessed image dataset is input into the positioning sub-network to obtain a rectangular positioning frame, which is trained based on the position annotation of the maxillary anterior area, the cross entropy loss function of the segmentation result, and the mixed loss function of the distance between the center point and the vertex position of the rectangular positioning frame.

[0066] Total loss function of the positioning sub-network The expression is:

[0067] Where, It is the binary cross entropy loss of the predicted segmentation result of the ROI region in the image block obtained by three-dimensional sliding block. is the segmentation result of the ROI area in the image block, and the distance loss between the center point of the rectangular positioning box and the vertex coordinate position obtained by merging the neighboring non-connected domains. is a hyperparameter that balances these two losses.

[0068] It can be expressed by the following formula:

[0069] in:

[0070] in, Represents the total number of voxel points in the three-dimensional image, represents the true label of each voxel point, Represents the probability value of each voxel point belonging to the maxillary anterior area after being predicted by the localization sub-network.

[0071] It can be expressed by the following formula:

[0072] in, and Represent the set of center points and vertices of the predicted rectangular positioning box and the real ROI area in the image block, respectively. Represents the set of center points and vertices of a 3D image block, represent middle Points and image patches The Euclidean distance between points, Represents the true annotation corresponding to the above predicted point middle Points and image patches Euclidean distance between points.

[0073] The vertex coordinates and center point of the rectangular positioning box in the image block are obtained through the nearest neighbor non-connected domain merging algorithm, and the loss function is calculated. During the continuous iterative training process, when the loss function value tends to converge, the training is completed.

[0074] SA4, inputs the rectangular positioning box into the classification subnetwork to obtain the probability value of containing supernumerary teeth, and trains based on the supernumerary tooth category information in the maxillary anterior area and the cross entropy loss function.

[0075] The loss function of the classification subnetwork adopts cross entropy loss, which can be expressed by the following formula:

[0076] in, represents the number of maxillary anterior teeth image blocks obtained by clipping the rectangular maxillary anterior teeth region position coordinates; Indicates the number of categories of samples. In this embodiment =2, that is, divided into two categories: no supernumerary teeth and supernumerary teeth; Indicates the i The true value of the probability of whether the image block contains a central supernumerary tooth; Indicates the model i The probability prediction value of whether an image block contains a central supernumerary tooth. During the iterative training process, when the loss function value converges, the training is completed.

[0077] Here is an example of a two-stage deep attention network training: (1) The dataset was divided into a training set and a validation set. The training set and validation set were set to 3D cone-beam CT images of 194 and 55 study cases, respectively.

[0078] (2) Position annotation was performed on the 3D cone-beam CT images of the study cases. The maxillary anterior teeth area was used as the region of interest (ROI) and marked with a 3D rectangular frame. The center point coordinates of the ROI in the 3D cone-beam CT images were extracted ( x, y, z ) and the corresponding width and height ( w, h, d ) as the regional position annotation information of the maxillary anterior teeth required for training; the 3D cone-beam CT images of the research cases were classified and annotated, and the tooth positions and morphologies in the maxillary anterior teeth area were analyzed based on the cross-section of the 3D cone-beam CT images and the 3D spatial model, and the situation of whether there were central supernumerary teeth in the maxillary anterior teeth area was used as the classification annotation information required for training. If there were central supernumerary teeth in the selected area of ​​interest, it was labeled as 1, otherwise it was labeled as 0; 137 samples with supernumerary teeth in the training set were labeled as positive samples, and 57 samples without supernumerary teeth were labeled as negative samples.

[0079] (3) Before the 3D cone-beam CT image is input into the 3D U-shaped localization sub-network, a segmentation mask of the same size as the image is generated, that is, the pixel value of the ROI area is 1 and the pixel value of the non-ROI area is 0. For a 3D cone-beam CT image of size 305×305×268, a 3D sliding block algorithm is adopted. A 3D sliding window of size 170×170×80 is used, and the step size is set to 40×40×20. The cone-beam CT image and the segmentation mask are divided into multiple image blocks and mask blocks with partial overlap. These blocks are input into the network respectively. At the same time, according to the input mask block, the 3D coordinates of the center point and 8 vertices of the ROI area in the block are obtained; according to the actual annotation box position, that is, the above-mentioned ROI area, the complete 3D cone-beam image is cropped to obtain the segmentation result containing only the maxillary anterior area, which is input into the classification sub-network as the training set.

[0080] (4) Set the initial values ​​of the positioning sub-network model hyperparameters, the initial learning rate is set to 0.0001, the number of training rounds is set to 80 rounds, the Adam optimizer is used, and the hyperparameters of the balanced coordinate distance loss in the loss function are Set to 0.01; set the initial values ​​of the hyperparameters of the three-dimensional classification sub-network model based on the attention mechanism, the initial learning rate is set to 0.00001, the number of training rounds is set to 150 rounds, and the Adam optimizer is used.

[0081] (5) Repeat steps (3) to (4) as the training phase of the two-stage 3D deep attention network model to obtain the final model weights.

[0082] (6) The two-stage three-dimensional deep attention network model trained in step (5) is used to test the newly acquired three-dimensional cone-beam CT image. The trained two-stage three-dimensional deep attention network model is used to take the image block obtained by three-dimensional sliding segmentation of the newly acquired three-dimensional cone-beam CT image as input. The segmentation result of the image block is output through the positioning subnetwork, and the image segmentation block is merged and restored to the original position and the segmented area is merged by the nearest non-connected domain to obtain the predicted coordinates of the rectangular maxillary anterior tooth area position. The maxillary anterior tooth area is cropped according to the coordinates of the rectangular maxillary anterior tooth area position, and the predicted category confidence is output through the classification subnetwork. Finally, the predicted maxillary anterior tooth area position in the newly acquired three-dimensional cone-beam CT image and the confidence of the presence of the central supernumerary tooth at this position are obtained.

[0083] The present invention uses three-dimensional images and the locations and categories of annotated central supernumerary teeth as model experimental data, providing rich 3D spatial information and avoiding the impact of spatial overlap of oral anatomical structures in two-dimensional curved slice images on recognition, thereby providing more accurate three-dimensional spatial locations for subsequent treatment. The present invention's localization subnetwork is based on a segmentation network framework. Compared with traditional detection networks, it does not require processing a large number of anchor points and anchor boxes, which speeds up detection and inference. The present invention uses cross-entropy loss and prediction box coordinate distance loss as loss functions in localization subnetwork training. The prediction box coordinate distance loss can effectively improve the detection accuracy of segmentation bounding boxes. The present invention's classification subnetwork combines an attention mechanism with a residual module. The attention mechanism can enhance the channel representation of useful information features in the feature map, while the residual module can integrate the original input information features, optimize feature expression efficiency, enhance network generalization ability, and ultimately achieve the purpose of improving classification accuracy. The present invention uses a three-dimensional sliding block algorithm to divide the entire three-dimensional image into several image blocks with partial overlap. These blocks are then input into the localization subnetwork for training and inference. This can effectively organize local features of the image, reduce computational overhead, and improve segmentation accuracy and stability. This invention uses a neighboring non-connected domain merging algorithm to transform the segmentation results from several non-connected neighboring regions into target detection frames. This fully considers the fact that when segmentation replaces traditional detection algorithms, the foreground may be divided into multiple small regions, which may affect detection accuracy, and can further improve detection accuracy. The present invention inputs the test 3D image into a trained two-stage 3D deep attention network model for testing. The test phase requires no manual operation; only a single scan of the entire 3D image is required to quickly and automatically identify the presence of centric supernumerary teeth in the image, thereby helping to improve the efficiency of clinical diagnosis of centric supernumerary teeth.

[0084] According to the method described in the above embodiment, this embodiment will be further described from the perspective of the supernumerary tooth automatic detection device. The supernumerary tooth automatic detection device can be implemented as an independent entity or integrated into an electronic device. The electronic device can be a terminal, server and other devices. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0085] See also Figure 6 , Figure 6 The present invention specifically describes an automatic supernumerary tooth detection device provided in an embodiment of the present invention, which is applied to an electronic device. The automatic supernumerary tooth detection device may include: A preprocessing module is used to obtain a three-dimensional image and preprocess the three-dimensional image to obtain multiple image blocks; The prediction module is used to input the image block into the trained two-stage deep attention network for recognition and obtain the probability value of the supernumerary tooth in the 3D image; Among them, the two-stage deep attention network includes a positioning subnetwork and a classification subnetwork; the image block is input into the trained two-stage deep attention network for recognition, including: inputting the image block into the positioning subnetwork to obtain a rectangular positioning box, and inputting the rectangular positioning box into the classification subnetwork to obtain the probability value of the image block containing supernumerary teeth.

[0086] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also be found in the beneficial effects in the previous method embodiments, which will not be repeated here.

[0087] In addition, an embodiment of the present application also provides an electronic device, which may be a computer, a tablet computer, or other device. Figure 7 As shown, the electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

[0088] The processor 401 is the control center of the electronic device 400. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or loading applications stored in the memory 402 and calling data stored in the memory 402, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole.

[0089] In this embodiment, the processor 401 in the electronic device 400 loads instructions corresponding to one or more application processes into the memory 402 according to the following steps, and the processor 401 runs the application stored in the memory 402 to implement various functions: Acquire a three-dimensional image, and pre-process the three-dimensional image to obtain a plurality of image blocks; The image block is input into the trained two-stage deep attention network for recognition, and the probability value of the supernumerary tooth in the three-dimensional image is obtained; Among them, the two-stage deep attention network includes a positioning subnetwork and a classification subnetwork; the image block is input into the trained two-stage deep attention network for recognition, including: inputting the image block into the positioning subnetwork to obtain a rectangular positioning box, and inputting the rectangular positioning box into the classification subnetwork to obtain the probability value of the image block containing supernumerary teeth.

[0090] This electronic device can implement the steps of any embodiment of the supernumerary tooth automatic detection method provided in the embodiments of the present application. Therefore, it can achieve the beneficial effects that can be achieved by any supernumerary tooth automatic detection method provided in the embodiments of the present invention. Please refer to the previous embodiments for details and will not be repeated here.

[0091] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be accomplished through instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the embodiments of the automatic supernumerary tooth detection method provided in the embodiments of the present invention.

[0092] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0093] Since the instructions stored in the storage medium can execute the steps in any embodiment of the supernumerary tooth automatic detection method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any supernumerary tooth automatic detection method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0094] The above is a detailed introduction to the method, device, storage medium and electronic device for automatic detection of supernumerary teeth provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for automatically detecting supernumerary teeth, characterized in that: The method comprises: Acquire a three-dimensional image, and preprocess the three-dimensional image to obtain a plurality of image blocks; Inputting the image block into a trained two-stage deep attention network for recognition, and obtaining a probability value of containing supernumerary teeth in the three-dimensional image; Among them, the two-stage deep attention network includes a positioning subnetwork and a classification subnetwork; the inputting of the image block into the trained two-stage deep attention network for recognition includes: inputting the image block into the positioning subnetwork to obtain a rectangular positioning box, and inputting the rectangular positioning box into the classification subnetwork to obtain the probability value of the image block containing supernumerary teeth.

2. The method for automatically detecting supernumerary teeth according to claim 1, characterized in that: The positioning sub-network includes an encoder, and the encoder includes a pooling layer, a CGR module, and a plurality of feature extraction branches; The step of inputting the image block into the positioning subnetwork to obtain the rectangular positioning frame includes: Inputting the image block into two CGR modules to obtain a low-level semantic feature map; Inputting the low-level semantic feature map into the first feature extraction branch and the second feature extraction branch to obtain a mid-level semantic feature map; Inputting the middle-level semantic feature map into the first feature extraction branch and the second feature extraction branch to obtain a first high-level semantic feature map; The first high-level semantic feature map is input into the pooling layer and a CGR module to obtain a low-resolution high-level semantic feature map.

3. The method for automatically detecting supernumerary teeth according to claim 2, characterized in that: The positioning sub-network also includes a decoder, and the decoder includes an upsampling layer, a CGR module and a three-dimensional convolution layer; The step of inputting the image block into the positioning subnetwork to obtain the rectangular positioning frame further includes: The low-resolution high-level semantic feature map is passed through a CGR module and an upsampling layer, and then concatenated with the first high-level semantic feature map to obtain a small-size fused feature map; The small-size fused feature map is passed through two CGR modules and an upsampling layer, and then concatenated with the middle-layer semantic feature map to obtain a middle-size fused feature map; The medium-sized fused feature map is passed through two CGR modules and an upsampling layer, and then concatenated with the low-resolution high-level semantic feature map to obtain a large-sized fused feature map; The large-size fused feature map is input into two CGR modules and a three-dimensional convolutional layer to obtain a segmentation result, and the rectangular positioning frame is obtained based on the segmentation result.

4. The method for automatically detecting supernumerary teeth according to claim 3, characterized in that: The obtaining the rectangular positioning frame based on the segmentation result includes: Obtaining a probability value of each pixel point in the segmentation result belonging to the maxillary anterior teeth area, and when the probability value is greater than a preset probability threshold, taking the corresponding pixel point as a candidate point of the region of interest; Combining adjacent candidate points of the region of interest to obtain multiple three-dimensional connected domains; The volume and circumscribed rectangular frame of each connected domain are obtained through the connected domain labeling algorithm; The circumscribed rectangular frame is used as a candidate detection frame, and connected domains whose volumes are smaller than a preset volume threshold are screened out to obtain a target candidate detection frame; Merge the circumscribed rectangular frame of the maximum connected domain and the candidate detection frames of the neighbors to obtain a rectangular positioning frame; The coordinate values ​​of the rectangular positioning frame in the image block are calculated, the coordinate values ​​are used as the boundary range coordinate values, and the distance between the center point of the rectangular positioning frame and the fixed point position is calculated based on the boundary range coordinate values.

5. The method for automatically detecting supernumerary teeth according to claim 1, characterized in that: The classification subnetwork includes a CBR module, a maximum pooling layer, multiple residual modules, an average pooling layer and a fully connected layer; The step of inputting the rectangular positioning frame into the classification subnetwork to obtain a probability value of the image block containing supernumerary teeth includes: Inputting the rectangular positioning frame into a CBR module, a maximum pooling layer and four residual modules to obtain a high-order feature map; The high-order feature map is input into the average pooling layer and the fully connected layer to obtain the probability value of the image block containing supernumerary teeth.

6. The method for automatically detecting supernumerary teeth according to claim 1, characterized in that: The training process of the two-stage deep attention network includes: Acquire a three-dimensional image, obtain a position mark of a maxillary anterior teeth region in the three-dimensional image and information on the type of supernumerary teeth in the maxillary anterior teeth region, and form an image data set; Preprocessing the images in the image data set by a three-dimensional sliding block algorithm to obtain a plurality of image blocks; Inputting the preprocessed image data set into the positioning subnetwork to obtain a rectangular positioning frame, and training based on the position annotation of the maxillary anterior teeth area, the cross entropy loss function of the segmentation result, and the mixed loss function of the distance between the center point and the vertex position of the rectangular positioning frame; The rectangular positioning frame is input into the classification subnetwork to obtain a probability value of containing supernumerary teeth, and training is performed based on supernumerary tooth category information in the maxillary anterior teeth area and a cross entropy loss function.

7. The method for automatically detecting supernumerary teeth according to claim 6, characterized in that: The preprocessing of the images in the image data set by using a three-dimensional sliding block algorithm includes: For the image dataset, the size is D × W × H For a 3D image, set a size of d × w × h The sliding window has a moving step size of ( sd,sw,sh ), generate the slice range corresponding to each image block in the complete image according to the window and step size: The number of slices in the Z direction is: ； The number of slices in the Y direction is: ； The number of slices in the X direction is: ； For Slices , calculate the coordinate range of each slice: ； Among them, the integer , round The () function is used to round a number to a given number of digits. When the last significant digit is 5 or greater, it is rounded up; when the last significant digit is less than 5, it is rounded down.

8. An automatic detection device for supernumerary teeth, characterized in that: include: A preprocessing module, used for acquiring a three-dimensional image, and preprocessing the three-dimensional image to obtain a plurality of image blocks; A prediction module, used for inputting the image block into a trained two-stage deep attention network for recognition, and obtaining a probability value of the three-dimensional image containing supernumerary teeth; Among them, the two-stage deep attention network includes a positioning subnetwork and a classification subnetwork; the inputting of the image block into the trained two-stage deep attention network for recognition includes: inputting the image block into the positioning subnetwork to obtain a rectangular positioning box, and inputting the rectangular positioning box into the classification subnetwork to obtain the probability value of the image block containing supernumerary teeth.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method for automatically detecting supernumerary teeth as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It comprises a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the automatic detection method of supernumerary teeth as described in any one of claims 1 to 7.

Images