Image detection method and device, computer device and storage medium

Abstract

The application relates to an image detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring tooth image blocks corresponding to each tooth from a tooth image; inputting each tooth image block into a preset feature extraction model to perform feature extraction, to obtain features corresponding to each tooth image block; the features corresponding to the tooth image blocks comprise image features of the tooth image blocks and correlation features between the tooth image blocks; performing image detection on the tooth image according to the features corresponding to each tooth image block, to generate an image detection result of the tooth image. Compared with directly performing feature extraction on the entire tooth image, the method proposed in the application performs feature extraction based on tooth image blocks of each tooth, can improve the extraction accuracy of tooth features and correlation features between teeth, improve the detection effect of tooth lesions and tooth defects, and further improve the detection accuracy of various tooth defects such as tooth lesions and tooth defects.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image detection method, apparatus, computer device, and storage medium. Background Technology

[0002] Cone-beam computed tomography (CBCT) is an important dental examination technique performed before orthodontic treatment. CBCT scans of teeth generate CBCT images, which provide a comprehensive three-dimensional view of the oral cavity. Image analysis of these CBCT images can identify areas with tooth defects or lesions.

[0003] Traditionally, image detection is performed by inputting CBCT images of teeth into a 3D neural network model. This method can detect and identify areas with tooth defects or lesions in CBCT images. However, traditional methods cannot accurately detect images with tooth defects or lesions. Summary of the Invention

[0004] Therefore, it is necessary to provide an image detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of detecting tooth defects and lesions, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides an image detection method. The method includes:

[0006] Obtain the corresponding tooth image patch from the tooth image;

[0007] Each tooth image patch is input into a preset feature extraction model for feature extraction, resulting in features corresponding to each tooth image patch. The features corresponding to each tooth image patch include the image features of the tooth image patch and the correlation features between the tooth image patches.

[0008] Based on the features corresponding to each tooth image block, image detection is performed on the tooth image to generate the image detection results of the tooth image.

[0009] In one embodiment, the preset feature extraction model includes a multi-scale feature extraction module and an encoder module; each tooth image block is input into the preset feature extraction model for feature extraction to obtain the features corresponding to each tooth image block, including:

[0010] Each tooth image patch is input into the multi-scale feature extraction module for feature extraction, generating multi-scale image features for each tooth image patch;

[0011] The multi-scale image features of each tooth image patch are input into the encoder module for feature encoding to generate the encoded features of each tooth image patch; the encoded features include correlation features; the encoded features of the tooth image patch are used to characterize the features corresponding to the tooth image patch.

[0012] In one embodiment, if the encoder module includes a multi-head self-attention mechanism submodule, then the encoded features are self-attention encoded features; the multi-scale image features of each tooth image block are input into the encoder module for feature encoding to generate the encoded features of each tooth image block, including:

[0013] The multi-scale image features of each tooth image block are input into the multi-head self-attention mechanism submodule in the encoder module for feature encoding, generating self-attention encoded features for each tooth image block.

[0014] In one embodiment, the method further includes:

[0015] Obtain the positional encoding information of each tooth image block;

[0016] For each tooth image patch, the positional encoding information of the tooth image patch is fused with the multi-scale image features of the tooth image patch to generate the fused features of the tooth image patch;

[0017] Accordingly, the multi-scale image features of each tooth image patch are input into the encoder module for feature encoding to generate the encoded features of each tooth image patch, including:

[0018] The fused features of each tooth image block are input into the encoder module for feature encoding to generate the encoded features of each tooth image block.

[0019] In one embodiment, each tooth image patch is input into a multi-scale feature extraction module for feature extraction, generating multi-scale image features for each tooth image patch, including:

[0020] For each tooth image patch, the tooth image patch is input into the multi-scale feature extraction module for feature extraction, resulting in image features at multiple different scales;

[0021] By fusing image features at multiple different scales, multi-scale image features of tooth image blocks are obtained.

[0022] In one embodiment, image detection is performed on the tooth image based on the features corresponding to each tooth image block to generate image detection results for the tooth image, including:

[0023] The self-attention coding features and the initial detection box queue corresponding to each tooth image block are input into the decoder module for decoding, generating the predicted detection box queue corresponding to each tooth image block.

[0024] The predicted detection box queue is input into a fully connected network for processing to generate attribute information for the predicted detection box queue;

[0025] Based on the attribute information of each predicted detection box queue, image detection is performed on the tooth image to generate the image detection result of the tooth image.

[0026] In one embodiment, obtaining tooth image blocks corresponding to each tooth from a tooth image includes:

[0027] The dental images are registered according to the dentition to obtain the registered dental images; the registered dental images include multiple teeth arranged in a straight line.

[0028] The registered tooth image is segmented according to tooth particles to obtain the initial tooth image block corresponding to each tooth.

[0029] The window width and window level of the initial tooth image block corresponding to each tooth are adjusted to obtain the tooth image block corresponding to each tooth.

[0030] Secondly, this application also provides an image detection apparatus. The apparatus includes:

[0031] The first acquisition module is used to acquire tooth image blocks corresponding to each tooth from the tooth image;

[0032] The feature extraction module is used to input each tooth image patch into a preset feature extraction model for feature extraction, and obtain the features corresponding to each tooth image patch; the features corresponding to each tooth image patch include the image features of the tooth image patch and the correlation features between the tooth image patches.

[0033] The detection module is used to perform image detection on the tooth image based on the features corresponding to each tooth image block, and generate the image detection results of the tooth image.

[0034] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the image detection method described in the first aspect.

[0035] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the image detection method described in the first aspect.

[0036] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the image detection method described in the first aspect.

[0037] The aforementioned image detection method, apparatus, computer equipment, storage medium, and computer program product involve the computer equipment acquiring tooth image blocks corresponding to each tooth from a tooth image; inputting each tooth image block into a preset feature extraction model for feature extraction to obtain features corresponding to each tooth image block; wherein, the features corresponding to each tooth image block include image features of the tooth image block and correlation features between tooth image blocks; then, based on the features corresponding to each tooth image block, image detection is performed on the tooth image to generate image detection results for the tooth image. In other words, in this embodiment, the complete tooth image is segmented according to tooth particles to obtain individual tooth image blocks. Then, features are extracted from each tooth image block and the correlation information between them to obtain correlation features between the tooth image blocks. Based on the correlation features between the tooth image blocks and the image features of each tooth, tooth defect detection is performed to obtain tooth image detection results. Compared with directly extracting features from the entire tooth image, the method proposed in this application extracts features based on the tooth image blocks corresponding to each tooth, which can greatly improve the extraction accuracy of tooth features and correlation features between teeth, improve the detection effect of tooth lesions and tooth defects, and thus improve the detection accuracy of various tooth defects such as tooth lesions and tooth defects. Attached Figure Description

[0038] Figure 1 This is an application environment diagram of the image detection method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating an image detection method in one embodiment;

[0040] Figure 3 This is a flowchart illustrating the image detection method in another embodiment;

[0041] Figure 4 This is a flowchart illustrating the image detection method in another embodiment;

[0042] Figure 5 This is a flowchart illustrating the image detection method in another embodiment;

[0043] Figure 6 This is a flowchart illustrating the image detection method in another embodiment;

[0044] Figure 7 This is a schematic diagram of the overall process structure of an image detection method in one embodiment;

[0045] Figure 8 This is a structural block diagram of an image detection device in one embodiment;

[0046] Figure 9This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] Dental CBCT provides patients with a comprehensive three-dimensional view of the oral cavity and can also display lesion information, including tooth defects and artificial fillings. More and more patients are having dental CBCT scans to provide imaging references for pre-orthodontic surgical planning. Among these, computer-aided detection (CAD) of dental lesions (including proximal caries, impacted teeth, periapical shadows, wedge-shaped defects, etc.) is of great help to doctors in quickly interpreting images and providing diagnostic reports. Using a CAD system can reduce the number of lesions missed by doctors and improve the speed of image interpretation.

[0049] Traditional V-NET-based 3D image feature extraction backbones are capable of extracting local features, but dental lesions have longer-range semantic relationships: such as impacted teeth growing on both sides of the jaw, or wedge-shaped defects appearing in the same location due to brushing habits. Traditional convolution-based backbones cannot effectively extract these features, and the detection results overfit to local features, leading to false positives. Furthermore, dental restorative materials, root canal fillings, crowns, and even enamel can easily produce strong image artifacts and cause uneven brightness values ​​in different areas, affecting the model's judgment results.

[0050] Furthermore, traditional techniques mostly employ conventional 3D convolution and anchor-box-based detection mechanisms. These models struggle to model long-range semantic relationships effectively, easily leading to absurd false positives. Additionally, the detection framework, based on anchor-box classification and regression combined with complex post-processing to obtain the detection set, is sensitive to anchor-box parameters, and model training consumes significant time for adjusting and optimizing these cumbersome parameters.

[0051] Based on this, this application proposes an image detection method for detecting dental lesions, thereby improving the accuracy and completeness of dental lesion detection.

[0052] The image detection method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, computer device 102 can be either a terminal or a server. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0053] In one embodiment, such as Figure 2 As shown, an image detection method is provided, which is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0054] Step 201: Obtain the tooth image blocks corresponding to each tooth from the tooth image.

[0055] The tooth image can be a CBCT image of the tooth or a conventional tooth scan image; this application does not specifically limit the type of image.

[0056] Optionally, the computer device can acquire the tooth image locally or from other devices that are communicatively connected to the computer device. For example, the computer device can acquire the tooth image to be processed from a server, an image storage server, or an image storage database. This application does not specifically limit the method of acquiring the tooth image.

[0057] Next, after acquiring the tooth image, image processing operations can be performed on the tooth image to obtain the tooth image block corresponding to each tooth from the tooth image; optionally, the computer device can input the tooth image into a preset tooth segmentation model to segment the tooth image according to tooth particles, thereby obtaining the tooth image block corresponding to each tooth.

[0058] Step 202: Input each tooth image block into a preset feature extraction model for feature extraction to obtain the features corresponding to each tooth image block.

[0059] The features corresponding to each tooth image patch include the image features of the tooth image patch and the correlation features between tooth image patches. The preset feature extraction model is used to extract features from each tooth image patch and to extract the correlation features between tooth image patches, thereby obtaining the features corresponding to each tooth image patch. Optionally, the preset feature extraction model can be a traditional three-dimensional convolutional feature extraction model, a multi-scale feature extraction model, or a feature extraction model based on a self-attention mechanism, etc., and this application embodiment does not specifically limit this.

[0060] Optionally, the computer device can simultaneously input each tooth image into the preset feature extraction model for feature extraction, thereby obtaining the features corresponding to each tooth image block; alternatively, each tooth image can be preprocessed first, and then the preprocessed tooth images can be input into the preset feature extraction model for extraction of tooth features and correlation features between teeth, thereby obtaining the features corresponding to each tooth image.

[0061] Step 203: Based on the features corresponding to each tooth image block, perform image detection on the tooth image to generate the image detection result of the tooth image.

[0062] Optionally, after obtaining the features corresponding to each tooth image, image detection processing can be performed on the tooth image based on the features corresponding to each tooth image to obtain the image detection result corresponding to the tooth image; the image detection result includes the detection results of tooth defects corresponding to various tooth lesions existing in the tooth image, including but not limited to proximal caries, impacted teeth, periapical shadows, wedge-shaped defects, etc.

[0063] Optionally, the features corresponding to each tooth image can be input into a preset image detection model for image detection, so as to output the image detection result corresponding to the tooth image; wherein, the preset image detection model can be a fully connected network model, a normalized network model, a decoder network model, etc., and this application embodiment does not specifically limit it.

[0064] In the aforementioned image detection method, the computer device acquires tooth image blocks corresponding to each tooth from a tooth image; and inputs each tooth image block into a preset feature extraction model for feature extraction to obtain features corresponding to each tooth image block; wherein, the features corresponding to each tooth image block include the image features of the tooth image block and the correlation features between the tooth image blocks; then, based on the features corresponding to each tooth image block, image detection is performed on the tooth image to generate the image detection result of the tooth image. That is to say, in this embodiment, the complete tooth image is segmented according to tooth particles to obtain each tooth image block; then, features are extracted from each tooth image block and the correlation information between each tooth image block to obtain the correlation features between each tooth image block; and based on the correlation features between each tooth image block and the image features of each tooth, tooth defect detection is performed to obtain the tooth image detection result; compared with directly extracting features from the entire tooth image, the method proposed in this application extracts features based on the tooth image blocks corresponding to each tooth, which can greatly improve the extraction accuracy of tooth features and the correlation features between teeth, improve the detection effect of tooth lesions and tooth defects, and thus improve the detection accuracy of various tooth defects such as tooth lesions and tooth defects.

[0065] In one embodiment, the aforementioned preset feature extraction model may include a multi-scale feature extraction module and an encoder module. The multi-scale feature extraction module is used to extract multi-scale features from each tooth image block to obtain multi-scale features corresponding to each tooth image block. The encoder module is used to extract correlation features between each tooth image block based on the multi-scale features corresponding to each tooth image block to obtain a feature vector corresponding to each tooth image block, including its own tooth features and correlation features between teeth.

[0066] Optionally, for the multi-scale feature extraction module, the network structure of the multi-scale feature extraction model can be any network structure, such as concatenated convolutions, residual networks, fully connected networks, etc.; for the decoder module, the number of stacks it contains, i.e. the number of layers of the multi-layer encoder, can also be any number; the embodiments of this application do not specifically limit this.

[0067] Based on this, such as Figure 3 The diagram illustrates a flowchart of an image detection method. This embodiment relates to an optional implementation process where a computer device inputs each tooth image block into a preset feature extraction model for feature extraction, obtaining the features corresponding to each tooth image block. Based on the above embodiment, step 202 may include:

[0068] Step 301: Input each tooth image block into the multi-scale feature extraction module for feature extraction to generate multi-scale image features for each tooth image block.

[0069] For each tooth image patch, the tooth image patch is input into a multi-scale feature extraction model for multi-scale feature extraction, generating multi-scale image features corresponding to that tooth image patch. Optionally, for each tooth image patch, multi-scale transformation processing can be performed on the tooth image patch to obtain multiple images at different scales corresponding to that tooth image patch; then, the multiple images at different scales corresponding to that tooth image patch are input into a multi-scale feature extraction model for multi-scale feature extraction, outputting feature maps for each scale of the tooth image patch; then, feature fusion processing is performed on the feature maps of each scale image to generate multi-scale image features corresponding to that tooth image patch.

[0070] For example, for each tooth image block, the tooth image block can be input into a multi-scale feature extraction module for feature extraction to obtain multiple image features at different scales; then, the multiple image features at different scales can be fused to obtain multi-scale image features of the tooth image block; wherein, the multi-scale feature extraction module can perform feature extraction at multiple different scales on the tooth image block to output multiple image features at different scales.

[0071] Step 302: Input the multi-scale image features of each tooth image block into the encoder module for feature encoding to generate the encoded features of each tooth image block.

[0072] The coding features include correlation features; the coding features of the tooth image patch are used to characterize the features corresponding to the tooth image patch.

[0073] Furthermore, after obtaining the multi-scale image features corresponding to each tooth image block, the multi-scale image features of each tooth image block can be input into the encoder module to extract the inter-tooth correlation features, generating the encoded features corresponding to each tooth image block, including the inter-tooth correlation features.

[0074] Optionally, the encoder module may include a self-attention mechanism submodule, which can be used to effectively extract the correlation features between tooth image blocks and other tooth image blocks. Based on this, the computer device can input the multi-scale image features of each tooth image block into the self-attention mechanism submodule in the encoder module for feature encoding to generate attention-encoded features for each tooth image block.

[0075] Optionally, the encoder module may also include a multi-head self-attention mechanism submodule, in which case the encoded feature can be a self-attention encoded feature. In this case, the computer device can input the multi-scale image features of each tooth image block into the multi-head self-attention mechanism submodule in the encoder module for feature encoding to generate the self-attention encoded features of each tooth image block.

[0076] In this embodiment, the aforementioned preset feature extraction model includes a multi-scale feature extraction module and an encoder module. The computer device inputs each tooth image patch into the multi-scale feature extraction module for feature extraction, generating multi-scale image features for each tooth image patch. Then, the multi-scale image features of each tooth image patch are input into the encoder module for feature encoding, generating coded features for each tooth image patch. These coded features include correlation features. The coded features of the tooth image patch are used to characterize the features corresponding to the tooth image patch. In other words, in this embodiment, when extracting tooth features, not only are the multi-scale features of the teeth extracted, but also the correlation features between teeth are extracted, resulting in more complete and comprehensive tooth features. This enables effective detection of different types of dental defects and improves the accuracy of dental defect detection.

[0077] Figure 4 This is a flowchart illustrating an image detection method in another embodiment. This embodiment relates to an optional implementation process for coded feature extraction using a computer device based on multi-scale image features of each tooth image block and the positional coding information of each tooth image block. Based on the above embodiment, such as... Figure 4 As shown, the above method may further include:

[0078] Step 401: Obtain the positional encoding information of each tooth image block.

[0079] The positional encoding information of the tooth image blocks can be used to characterize the positional relationship between the teeth.

[0080] Optionally, the positional encoding information of each tooth image block can be determined based on the tooth image, or the teeth in the tooth image can be registered to a straight line according to the dentition, and then the positional encoding information of each tooth image block can be determined based on the registered tooth image. Optionally, the positional encoding information of each tooth image block can include basic positional encoding information and learnable positional encoding information; wherein, the basic positional encoding information can be the positional encoding of each tooth determined based on the initial tooth image or the tooth image registered according to the dentition, and the learnable positional encoding information can be the positional encoding of each tooth generated after positional learning based on the features of each tooth image block combined with a preset learning algorithm.

[0081] It should be noted that the positional coding information of each tooth image block may include only basic positional coding information, or only learnable positional coding information, or both basic positional coding information and learnable positional coding information; this application embodiment does not specifically limit this.

[0082] Step 402: For each tooth image block, the positional encoding information of the tooth image block is fused with the multi-scale image features of the tooth image block to generate the fused features of the tooth image block.

[0083] Optionally, for each tooth image patch, during feature fusion, the positional encoding information of the tooth image patch can be added to the multi-scale image features of the tooth image patch to generate the fusion features of the tooth image patch. That is, the positional encoding information of the tooth image patch and the multi-scale image features of the tooth image patch can be concatenated to obtain the fusion features of the tooth image patch; alternatively, the positional encoding information of the tooth image patch and the multi-scale image features of the tooth image patch can be weighted and fused to generate the fusion features of the tooth image patch.

[0084] Accordingly, the above-mentioned input of multi-scale image features of each tooth image patch into the encoder module for feature encoding to generate coded features for each tooth image patch may include:

[0085] Step 403: Input the fused features of each tooth image block into the encoder module for feature encoding to generate the encoded features of each tooth image block.

[0086] Optionally, the fused features of each tooth image patch can be input into the encoder module to learn the inter-tooth correlation features and generate the encoded features of each tooth image patch; alternatively, the fused features of each tooth image patch can also be input into the multi-head self-attention mechanism submodule in the encoder module for feature encoding and generate the self-attention encoded features of each tooth image patch.

[0087] In this embodiment, the positional encoding information of each tooth image patch is obtained. For each tooth image patch, the positional encoding information is fused with the multi-scale image features of the tooth image patch to generate a fused feature of the tooth image patch. Then, the fused features of each tooth image patch are input into the encoder module for feature encoding to generate the encoded features of each tooth image patch. In other words, in this embodiment, by combining the positional encoding information and the multi-scale image features of each tooth image patch, the correlation features between each tooth are learned. By fusing the positional information of each tooth, it is beneficial to obtain longer-distance semantic association information between tooth lesions, thereby improving the detection effect of various tooth lesions and the detection accuracy of tooth defects.

[0088] Figure 5 This is a flowchart illustrating an image detection method in another embodiment. This embodiment relates to an optional implementation process where a computer device performs image detection on a tooth image based on the features corresponding to each tooth image block, generating an image detection result for the tooth image. Based on the above embodiment, as... Figure 5 As shown, step 203 above may include:

[0089] Step 501: Input the self-attention coding features and initial detection box queue corresponding to each tooth image block into the decoder module for decoding to generate the prediction detection box queue corresponding to each tooth image block.

[0090] The initial detection box queue can include separate initial detection box queues for each tooth image block. Each initial detection box queue can include at least one preset detection box, which is used to detect and mark any type of tooth defect, such as tooth lesions or tooth defects. Each preset detection box can be randomly initialized. The number of detection boxes in the initial detection box queue is not specifically limited and can be flexibly set and adjusted according to the specific detection task in practical applications. For example, the initial detection box queue can be encoded in N queues, where N is the number of preset detection boxes, which can be set to 100. The decoder module can have the same or similar structure as the encoder module described above. It is used to decode the encoded features output by the encoder. That is, the decoder module re-encodes the initial detection box queue based on the encoded features of each tooth image block output by the encoder module to detect tooth defects in the tooth image and output various types of tooth defects in the tooth image. The various types of tooth defects output are detected and marked by the re-encoded detection boxes.

[0091] Optionally, if the encoder module includes a multi-head self-attention mechanism submodule, the encoder module can output the self-attention coding features corresponding to each tooth image block. Then, the computer device can input the self-attention coding features corresponding to each tooth image block output by the encoder module and the initial detection box queue into the decoder module for decoding processing. After performing multi-layer self-attention interactive calculations on the initial detection box queue and the self-attention coding features of each tooth image block, the preset detection boxes in the initial detection box queue are re-encoded to generate a predicted detection box queue corresponding to each tooth image block. The predicted detection box queue includes at least one detection box information for marking tooth defects, and each detection box information corresponds to a detection box that includes at least one tooth defect. If the predicted detection box queue includes information on multiple predicted detection boxes, it indicates that the tooth image includes multiple tooth defects.

[0092] Step 502: Input the predicted detection box queue into the fully connected network for processing to generate the attribute information of the predicted detection box queue.

[0093] The attribute information of the predicted detection box queue may include the number of predicted detection boxes, the position and size of each predicted detection box, and the defect type of the tooth defect corresponding to each predicted detection box.

[0094] Optionally, a fully connected network can be connected after the decoder module. This fully connected network can normalize the predicted detection box queues corresponding to each tooth image block output by the decoder module and output the predicted detection box information corresponding to each tooth image block.

[0095] Step 503: Based on the attribute information of each predicted detection box queue, perform image detection on the tooth image to generate the image detection result of the tooth image.

[0096] Furthermore, after determining the predicted detection box queues corresponding to each tooth image block, image detection can be performed on the tooth image based on the attribute information of each predicted detection box queue. That is, based on each predicted detection box in each predicted detection box queue, tooth defects in the tooth image are detected and marked, thereby generating an image detection result of the tooth image. The image detection result includes a tooth image with at least one tooth defect detection box.

[0097] In this embodiment, the computer device inputs the self-attention coding features corresponding to each tooth image block and the initial detection box queue into the decoder module for decoding, generating a predicted detection box queue for each tooth image block. Then, the predicted detection box queue is input into a fully connected network for processing, generating attribute information for the predicted detection box queue. Finally, based on the attribute information of each predicted detection box queue, image detection is performed on the tooth image to generate the image detection result. Using the method in this embodiment, accurate detection and localization of various dental defects in tooth images can be achieved, especially for dental defects with distant correlation, improving the detection accuracy of various dental defects, including dental lesions and tooth defects.

[0098] Figure 6 This is a flowchart illustrating an image detection method in another embodiment. This embodiment relates to an optional implementation process for a computer device to obtain tooth image blocks corresponding to each tooth from a tooth image. Based on the above embodiment, as... Figure 6 As shown, step 201 above includes:

[0099] Step 601: Perform image registration on the tooth image according to the dentition to obtain the registered tooth image.

[0100] In this context, the dentition refers to the arrangement of the roots of the upper and lower jaw teeth within their alveolar sockets, with their crowns connected to each other in a specific order, direction, and position, forming an arch. The registered dental image includes multiple teeth arranged sequentially in a straight line.

[0101] Optionally, the tooth image can be input into a preset registration algorithm for image registration processing according to the dentition row to obtain a registered tooth image. In this tooth image, all teeth can be arranged in a straight line, or the upper and lower rows of teeth can be arranged in separate rows. This application embodiment does not specifically limit this.

[0102] Step 602: The registered tooth image is segmented according to tooth particles to obtain the initial tooth image block corresponding to each tooth.

[0103] Optionally, for the registered tooth image, the connected components corresponding to each tooth can be determined according to the tooth particles. Based on the connected components corresponding to each tooth, image segmentation processing can be performed on each tooth in the tooth image to obtain the initial tooth image block corresponding to each tooth.

[0104] Step 603: Adjust the window width and window level of the initial tooth image block corresponding to each tooth to obtain the tooth image block corresponding to each tooth.

[0105] Window width refers to the range of CT values ​​selected when displaying an image, and its value affects the image contrast; a narrow window displays a small range of CT values ​​with high contrast, while a wide window displays a large range of CT values ​​with poor contrast. Window level refers to the average of the upper and lower limits of the window width CT values, and the window level affects the image brightness; a low window level results in a bright, white image, while a high window level results in a dark, black image.

[0106] For the initial tooth image blocks corresponding to each segmented tooth, problems such as poor imaging effect, poor contrast, or uneven brightness leading to artifacts may exist. Therefore, for each initial tooth image block, the window width and window level can be adaptively adjusted to obtain tooth image blocks corresponding to each tooth with better display effect and higher image quality. Optionally, when adjusting the window width and window level, adjustments can be made according to a preset fixed window width and window level, or the window width and window level corresponding to each initial tooth image block can be determined based on the determined window width and window level, and the initial tooth image block can be adaptively adjusted based on the determined window width and window level.

[0107] In this embodiment, the computer device performs image registration on a tooth image according to the dentition, obtaining a registered tooth image. The registered tooth image includes multiple teeth arranged sequentially in a straight line. Next, the registered tooth image is segmented according to tooth particles to obtain initial tooth image blocks corresponding to each tooth. The window width and window level of each initial tooth image block are adjusted to obtain a tooth image block for each tooth. That is, in this embodiment, arranging the arched teeth sequentially in a straight line through image registration facilitates tooth segmentation, improving segmentation efficiency and accuracy. Furthermore, adjusting the window width and window level of each initial tooth image block improves the image quality of each tooth image block, further enhancing the accuracy of subsequent detection of tooth defects based on each tooth image block.

[0108] In one embodiment, a specific implementation of tooth image detection is provided. Its overall flowchart can be found here. Figure 7 As shown, the input to the entire network is the processed original 3D image, and the output is an unordered lesion detection box. The specific steps include:

[0109] Step 1: Register the original tooth image to a straight line according to the dentition. Use a preset tooth segmentation model to segment the registered tooth image to obtain the tooth segmentation result. Based on the tooth segmentation result, crop out the tooth image block corresponding to each tooth.

[0110] Step 2: After adjusting the window width and window level of the cropped tooth image blocks, input them into the preset feature extraction network for feature extraction to obtain the features of each tooth image block.

[0111] Optionally, the preset feature extraction network in this embodiment can be similar to a general three-dimensional classification network, using traditional three-dimensional convolution to extract feature maps, and combining the Feature Pyramid Network (FPN) and VNET network, i.e., the FPN-VNET network structure, to extract features from each tooth image block after window width and window level adjustment, thereby obtaining multi-scale image features corresponding to each tooth image block.

[0112] Step 3: For each tooth image block, generate a basic position code for each tooth based on the tooth segmentation sequence number, and then generate a learnable position code based on the multi-scale image features corresponding to each tooth image block. The two codes are fused to generate complete position code information corresponding to each tooth image block.

[0113] Among them, the positional encoding can save the three-dimensional positional information of the tooth image block. Optionally, the positional encoding can also be generated using a fixed algorithm rather than a learnable one, such as a sine or cosine function.

[0114] Step 4: For each tooth image block, the positional encoding information of the tooth image block is fused with the multi-scale image features of the tooth image block output by the 3D convolution to generate the fused feature of the tooth image block, and the fused feature is used as the input of the Transformer encoder.

[0115] Step 5: The multi-layer Transformer encoder uses a multi-head self-attention mechanism to calculate the correlation between the fusion features of each tooth image block and generate a self-attention coding feature for each tooth image block. The generated self-attention coding feature of the tooth image block contains the fusion feature of the tooth image block and the correlation feature between the fusion feature and the fusion features of other tooth image blocks.

[0116] During the encoding calculation process, the position encoding information is simultaneously involved in the calculation of each layer of the encoder to enhance the position of the feature map.

[0117] Step 6: For each tooth image patch, generate a learnable initial detection box queue corresponding to the tooth image patch. This initial detection box queue contains the encoding information of at least one randomly initialized preset detection box. Each initial detection box queue and the self-attention encoded features of each tooth image patch generated by the Transformer encoder are simultaneously used as input to the Transformer decoder. The number of detection boxes in each initial detection box queue is the maximum number of preset detection boxes corresponding to each tooth image patch.

[0118] Step 7: The output of the Transformer decoder is the predicted detection box queue after the initial detection box queue has undergone attention calculation with the self-attention encoded features output by the encoder. That is, the Transformer decoder re-encodes each initial detection box queue based on the self-attention encoded features of each tooth to obtain multiple processed detection box queues. Each initial detection box queue performs attention interaction calculation with the self-attention encoded features of all tooth image blocks respectively, thereby obtaining the decoded detection box queue corresponding to each tooth image block.

[0119] Step 8: The predicted detection box queues corresponding to each tooth image block output by the decoder contain the encoding information of the predicted detection boxes. After each predicted detection box queue is calculated by a fully connected layer, the size coordinate information and classification information of the predicted detection box are generated. In other words, each predicted detection box queue output by the decoder is input into the fully connected layer for calculation, and the lesion detection box corresponding to each tooth is output.

[0120] In this embodiment, a network structure based on a self-attention mechanism first registers the images of oral teeth to a straight line. A segmentation network is then used to segment each tooth from the image, and the image is further divided into multiple tooth image blocks based on the segmentation results for each tooth. Next, the window width and level are adjusted for each cropped image block, and convolutions are performed to obtain a sequence of tooth feature maps. The encoder in the self-attention module then calculates the correlation between each feature map and encodes each image block. Finally, a decoder with a similar structure directly generates a set of predicted lesion detection boxes. This method alleviates the interference of uneven brightness caused by artifacts on the model, enhances the model's high-dimensional semantic understanding of the input image, uncovers the correlation between lesions at distant locations, and improves the semantic rationality of the detection results. Furthermore, the end-to-end detection structure avoids complex hyperparameter adjustments and post-processing, improving the detection efficiency of tooth images.

[0121] The tooth image detection method proposed in this embodiment has the following innovative features:

[0122] 1. Based on the registration of the tooth rows into straight lines, a feature map of each tooth is generated. This encoding method, which mimics the characteristics of the teeth themselves to generate a semantic structure, can better express the high-dimensional semantic relationships between teeth.

[0123] 2. Targeted window width and window level adjustments can be made to each tooth image block segmented and cropped based on the registered tooth image to alleviate the artifact effects of different tooth image blocks.

[0124] 3. It integrates positional coding based on tooth segmentation results with self-learned positional coding.

[0125] 4. Feature maps that integrate multi-scale image features.

[0126] 5. The correlation between image patches is calculated based on multi-head self-attention between high-dimensional feature maps. This self-attention calculation of high-dimensional feature maps can ensure that the neural network understands the high-dimensional semantic features of the overall information of the image, reducing the semantic errors that are prone to occur in the output detection box.

[0127] 6. The detection boxes are generated directly using an attention-based decoder. The end-to-end input and output avoids dependence on anchor box hyperparameters. In addition, previous techniques required complex post-processing of anchor boxes to ensure output stability. In this embodiment, the predicted detection boxes are directly output by the neural network without the need for additional post-processing.

[0128] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0129] Based on the same inventive concept, this application also provides an image detection apparatus for implementing the image detection method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image detection apparatus embodiments provided below can be found in the limitations of the image detection method described above, and will not be repeated here.

[0130] In one embodiment, such as Figure 8As shown, an image detection device is provided, comprising: a first acquisition module 801, a feature extraction module 802, and a detection module 803, wherein:

[0131] The first acquisition module 801 is used to acquire tooth image blocks corresponding to each tooth from the tooth image.

[0132] The feature extraction module 802 is used to input each tooth image block into a preset feature extraction model for feature extraction, and obtain the features corresponding to each tooth image block; the features corresponding to each tooth image block include the image features of the tooth image block and the correlation features between the tooth image blocks.

[0133] The detection module 803 is used to perform image detection on the tooth image based on the features corresponding to each tooth image block, and generate the image detection result of the tooth image.

[0134] In one embodiment, the preset feature extraction model includes a multi-scale feature extraction module and an encoder module; the feature extraction module 802 is used to input each tooth image block into the multi-scale feature extraction module for feature extraction, generating multi-scale image features of each tooth image block; input the multi-scale image features of each tooth image block into the encoder module for feature encoding, generating coded features of each tooth image block; the coded features include correlation features; the coded features of the tooth image block are used to characterize the features corresponding to the tooth image block.

[0135] In one embodiment, if the encoder module includes a multi-head self-attention mechanism submodule, the encoded features are self-attention encoded features; the feature extraction module 802 is used to input the multi-scale image features of each tooth image block into the multi-head self-attention mechanism submodule in the encoder module for feature encoding, thereby generating self-attention encoded features for each tooth image block.

[0136] In one embodiment, the device further includes: a second acquisition module and a feature fusion module; wherein, the second acquisition module is used to acquire the positional encoding information of each tooth image block; the feature fusion module is used to perform feature fusion of the positional encoding information of each tooth image block with the multi-scale image features of the tooth image block to generate fused features of the tooth image block; the aforementioned feature extraction module 802 is used to input the fused features of each tooth image block into the encoder module for feature encoding to generate coded features of each tooth image block.

[0137] In one embodiment, the feature extraction module 802 is used to input the tooth image block into the multi-scale feature extraction module for feature extraction to obtain multiple image features at different scales; and to fuse the multiple image features at different scales to obtain multi-scale image features of the tooth image block.

[0138] In one embodiment, the detection module 803 is used to input the self-attention coding features and the initial detection box queue corresponding to each tooth image block into the decoder module for decoding to generate a predicted detection box queue corresponding to each tooth image block; input the predicted detection box queue into a fully connected network for processing to generate attribute information of the predicted detection box queue; and perform image detection on the tooth image according to the attribute information of each predicted detection box queue to generate the image detection result of the tooth image.

[0139] In one embodiment, the first acquisition module 801 is used to perform image registration on the tooth image according to the dentition to obtain a registered tooth image; the registered tooth image includes multiple teeth arranged in a straight line; the registered tooth image is segmented according to tooth particles to obtain an initial tooth image block corresponding to each tooth; the window width and window level of the initial tooth image block corresponding to each tooth are adjusted to obtain a tooth image block corresponding to each tooth.

[0140] Each module in the aforementioned image detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0141] In one embodiment, a computer device is provided, which may be a terminal or a server, and its internal structure diagram may be as follows. Figure 9 As shown. The computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores the aforementioned preset feature extraction model. The network interface communicates with an external terminal via a network connection. When the computer program is executed by the processor, it implements an image detection method. Optionally, if the computer device is a terminal, it may also include a display screen and an input device. The display screen may be a liquid crystal display (LCD) or an electronic ink display (EIM). The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0142] Those skilled in the art will understand that Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the image detection methods in the various embodiments described above.

[0144] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the image detection methods described in the various embodiments above.

[0145] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the image detection methods described in the various embodiments above.

[0146] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0147] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0148] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image detection method, characterized in that, The method includes: The dental image is registered according to the dental arch to obtain the registered dental image; the registered dental image includes multiple teeth arranged in a straight line, the dental arch refers to the roots of the upper and lower jaw teeth growing in the alveolar sockets, and their crowns being adjacent to each other in a certain order, direction and position, arranged in an arch shape; From the registered tooth image, the initial tooth image block corresponding to each tooth is obtained. The window width and window level of the initial tooth image block corresponding to each tooth are adjusted to obtain the tooth image block corresponding to each tooth. The window width refers to the range of CT values ​​selected when displaying the image, and the window level refers to the average of the upper and lower limits of the window width CT values. Each of the tooth image blocks is input into a preset feature extraction model for feature extraction to obtain the features corresponding to each tooth image block. The preset feature extraction model includes a multi-scale feature extraction module and an encoder module. The process of inputting each of the tooth image blocks into the preset feature extraction model for feature extraction to obtain the features corresponding to each tooth image block includes: inputting each of the tooth image blocks into the multi-scale feature extraction module for feature extraction to generate multi-scale image features for each tooth image block; obtaining the positional encoding information of each tooth image block; for each tooth image block, fusing the positional encoding information with the multi-scale image features to generate fused features for the tooth image block; inputting the fused features of each tooth image block into the encoder module for feature encoding to generate coded features for each tooth image block. The coded features of the tooth image blocks are used to characterize the features corresponding to the tooth image blocks. The features corresponding to the tooth image blocks include the image features of the tooth image blocks and the correlation features between the tooth image blocks. Based on the features corresponding to each of the tooth image blocks, image detection is performed on the tooth image to generate the image detection result of the tooth image.

2. The method according to claim 1, characterized in that, If the encoder module includes a multi-head self-attention mechanism submodule, then the encoded features are self-attention encoded features; the step of inputting each of the tooth image blocks into a preset feature extraction model for feature extraction to obtain the features corresponding to each of the tooth image blocks includes: The multi-scale image features of each of the tooth image blocks are input into the multi-head self-attention mechanism submodule in the encoder module for feature encoding, thereby generating self-attention encoded features for each of the tooth image blocks.

3. The method according to claim 1, characterized in that, The step of inputting each of the tooth image blocks into the multi-scale feature extraction module for feature extraction, generating multi-scale image features for each of the tooth image blocks, includes: For each of the tooth image blocks, the tooth image blocks are input into the multi-scale feature extraction module for feature extraction to obtain image features at multiple different scales; The image features at multiple different scales are fused to obtain the multi-scale image features of the tooth image block.

4. The method according to claim 2, characterized in that, The step of performing image detection on the tooth image based on the features corresponding to each of the tooth image blocks, and generating the image detection result of the tooth image, includes: The self-attention coding features and the initial detection box queue corresponding to each tooth image block are input into the decoder module for decoding to generate the predicted detection box queue corresponding to each tooth image block. The predicted detection box queue is input into a fully connected network for processing to generate the attribute information of the predicted detection box queue; Based on the attribute information of each of the predicted detection box queues, image detection is performed on the tooth image to generate the image detection result of the tooth image.

5. The method according to claim 1, characterized in that, The step of obtaining the initial tooth image block corresponding to each tooth from the registered tooth image includes: The registered tooth images are segmented according to tooth particles to obtain initial tooth image blocks corresponding to each tooth.

6. An image detection device, characterized in that, The device includes: The first acquisition module is used to perform image registration on the tooth image according to the dentition to obtain the registered tooth image; the registered tooth image includes multiple teeth arranged in a straight line, and the dentition refers to the roots of the upper and lower jaw teeth growing in the alveolar sockets, and their crowns being adjacent to each other in a certain order, direction and position, arranged in an arc shape; From the registered tooth image, the initial tooth image block corresponding to each tooth is obtained. The window width and window level of the initial tooth image block corresponding to each tooth are adjusted to obtain the tooth image block corresponding to each tooth. The window width refers to the range of CT values ​​selected when displaying the image, and the window level refers to the average of the upper and lower limits of the window width CT values. A feature extraction module is used to input each of the tooth image blocks into a preset feature extraction model for feature extraction, thereby obtaining features corresponding to each of the tooth image blocks. The preset feature extraction model includes a multi-scale feature extraction module and an encoder module. The process of inputting each of the tooth image blocks into the preset feature extraction model for feature extraction to obtain features corresponding to each of the tooth image blocks includes: inputting each of the tooth image blocks into the multi-scale feature extraction module for feature extraction, generating multi-scale image features for each of the tooth image blocks; obtaining positional encoding information for each of the tooth image blocks; for each of the tooth image blocks, fusing the positional encoding information with the multi-scale image features to generate fused features for the tooth image blocks; inputting the fused features of each of the tooth image blocks into the encoder module for feature encoding, generating coded features for each of the tooth image blocks, wherein the coded features of the tooth image blocks are used to characterize the features corresponding to the tooth image blocks; the features corresponding to the tooth image blocks include the image features of the tooth image blocks and the correlation features between the tooth image blocks. The detection module is used to perform image detection on the tooth image based on the features corresponding to each tooth image block, and generate the image detection result of the tooth image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images