A method for detecting abnormal tooth areas

Abstract

An embodiment of the present invention provides a method for detecting abnormal tooth areas, which includes: obtaining a dental diagnosis and treatment image including at least one tooth; inputting the dental diagnosis and treatment image into a dental abnormality detection model to obtain an abnormal area mask image corresponding to the suspected abnormal area in the dental diagnosis and treatment image, abnormal area bounding box information, abnormal area classification results, and tooth position classification results to which the abnormal area belongs, wherein the dental abnormality detection model includes: a feature extraction network, a region of interest extraction network, a region feature alignment network for determining regional feature information of the region of interest, and an abnormal area evaluation network for determining abnormal area detection results based on the regional feature information of each region of interest, which can identify the mask image, bounding box, abnormal area classification results, and tooth position information corresponding to the abnormal area, thereby improving the detection accuracy, reliability, and efficiency of abnormal tooth areas.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting abnormal tooth areas. Background Art

[0002] In dental diagnosis and treatment, medical imaging is a commonly used diagnosis and treatment method. For example, small dental radiographs (periapical radiographs / bitewing radiographs) can be used to observe the roots of the teeth and the surrounding bone. Periapical radiograph: This type of X-ray shows the full length of a single or multiple teeth, including the crown, root and surrounding tissues, and is used to detect periapical disease, pulp disease, etc. Bitewing radiograph: This type of X-ray shows the occlusal surface of the upper and lower teeth, and is mainly used to detect periodontal disease and caries. Low-density shadows (such as caries, etc.) and high-density shadows (such as dentin hyperplasia or calcification deposition) are common abnormal manifestations on small dental radiographs, which may be related to periodontal disease, caries or other oral diseases.

[0003] Currently, during dental diagnosis and treatment, doctors typically detect dental abnormalities by observing low- and high-density shadows in medical images. However, this method relies heavily on the doctor's subjective judgment, is time-consuming, and is susceptible to factors such as doctor experience and fatigue. This leads to a high rate of missed detections, resulting in low efficiency, poor accuracy, and poor reliability in dental abnormality detection. Summary of the Invention

[0004] An embodiment of the present invention provides a method for detecting abnormal tooth areas to identify a mask image, a bounding box, a classification result of the abnormal area, and tooth position information corresponding to the abnormal area, thereby improving the accuracy, reliability, and efficiency of detecting abnormal tooth areas.

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting abnormal tooth areas, the method comprising:

[0006] Acquire a dental diagnosis image to be detected; wherein the dental diagnosis image includes at least one tooth;

[0007] Inputting the dental diagnosis and treatment image into a dental abnormality detection model to obtain abnormal area detection results corresponding to suspected abnormal areas in the dental diagnosis and treatment image; wherein the abnormal area detection results include an abnormal area mask image, abnormal area bounding box information, abnormal area classification results, and tooth position classification results to which the abnormal area belongs;

[0008] Among them, the dental abnormality detection model includes: a feature extraction network for extracting a dental diagnosis feature map based on the dental diagnosis image, a region of interest extraction network for extracting a region of interest from the dental diagnosis feature map, a region feature alignment network for determining the region feature information of the region of interest, and an abnormal region evaluation network for determining the abnormal region detection result based on the region feature information of each region of interest.

[0009] In a second aspect, an embodiment of the present invention further provides a device for detecting abnormal tooth areas, the device comprising:

[0010] A dental diagnosis and treatment image acquisition module, configured to acquire a dental diagnosis and treatment image to be detected; wherein the dental diagnosis and treatment image includes at least one tooth;

[0011] An abnormal result determination module is used to input the dental diagnosis and treatment image into a dental abnormality detection model to obtain an abnormal area detection result corresponding to the suspected abnormal area in the dental diagnosis and treatment image; wherein the abnormal area detection result includes an abnormal area mask image, abnormal area bounding box information, abnormal area classification result, and tooth position classification result to which the abnormal area belongs;

[0012] Among them, the dental abnormality detection model includes: a feature extraction network for extracting a dental diagnosis feature map based on the dental diagnosis image, a region of interest extraction network for extracting a region of interest from the dental diagnosis feature map, a region feature alignment network for determining the region feature information of the region of interest, and an abnormal region evaluation network for determining the abnormal region detection result based on the region feature information of each region of interest.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting abnormal tooth areas as described in any one of the embodiments of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the method for detecting abnormal tooth areas as described in any one of the embodiments of the present invention.

[0018] The technical solution of the embodiment of the present invention obtains a dental diagnosis image including at least one tooth, and then inputs the dental diagnosis image into a dental abnormality detection model to obtain an abnormal region mask image corresponding to the suspected abnormal region in the dental diagnosis image, abnormal region bounding box information, abnormal region classification results, and tooth position classification results to which the abnormal region belongs, wherein the dental abnormality detection model includes: a feature extraction network for extracting a dental diagnosis feature map based on the dental diagnosis image, a region of interest extraction network for extracting regions of interest from the dental diagnosis feature map, a region feature alignment network for determining regional feature information of the regions of interest, and an abnormal region evaluation network for determining abnormal region detection results based on the regional feature information of each region of interest. The technical solution provided by the embodiment of the present invention can automatically detect abnormal regions in dental diagnosis images through the dental abnormality detection model, and can not only identify the mask image, bounding box, and abnormal region classification results corresponding to the abnormal region, but also identify the tooth position information where the abnormal region is located, thereby improving the accuracy, reliability, and efficiency of detecting dental abnormal regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings introduced here only illustrate some of the embodiments to be described by the present invention, and are not exhaustive. A person skilled in the art can derive other drawings based on these drawings without inventive effort.

[0020] Figure 1 A schematic flow chart of a method for detecting abnormal tooth areas provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the arrangement of deciduous teeth and permanent teeth according to an embodiment of the present invention;

[0022] Figure 3 Schematic diagram of a tooth abnormality detection model according to an embodiment of the present invention;

[0023] Figure 4 A schematic flow chart of another method for detecting abnormal tooth areas provided by an embodiment of the present invention;

[0024] Figure 5 A schematic structural diagram of a device for detecting abnormal tooth areas provided by an embodiment of the present invention;

[0025] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0027] Figure 1 The present invention provides a flowchart of a method for detecting abnormal areas on teeth provided in an embodiment of the present invention. This embodiment is applicable to situations where automated detection of abnormal areas on teeth is required. The method can be performed by a device for detecting abnormal areas on teeth, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server.

[0028] like Figure 1 As shown, the method for detecting abnormal tooth areas includes:

[0029] S110: Acquire a dental diagnosis image to be detected.

[0030] Dental treatment images are X-ray images generated during the patient's dental treatment. These images include at least one tooth. For example, dental treatment images can be small dental films. These images are obtained using X-ray projection technology to show the shape, size, position, and relationship of the tooth to adjacent tissues. If conditions such as caries, pulpitis, and periapical periodontitis are present, these abnormalities may be visible on the images.

[0031] Specifically, when a patient visits a doctor, images of his teeth and surrounding tissues can be obtained through X-ray projection technology to obtain dental diagnostic images to be tested; in addition, dental diagnostic images taken of a certain subject in a historical period can also be obtained from a database.

[0032] In this embodiment, a possible abnormal area on a tooth of a certain object can be identified quickly and accurately by acquiring a dental diagnosis image and then automatically identifying the position and image appearance of the abnormal area in the dental diagnosis image. Finally, a dental inspection report can be automatically generated to assist the doctor in further analysis and decision-making.

[0033] In particular, in order to improve the detection accuracy of dental treatment images, the dental treatment images can be preprocessed after being obtained. The specific implementation methods of the preprocessing may include but are not limited to the following methods: Grayscale processing: converting the dental treatment image (this is because the dental treatment image can be an image with three RGB channels) into a grayscale image, for example, this can be achieved by the cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) method. Denoising processing: using median filtering to denoise the dental treatment image, for example, the cv2.medianBlur(image, ksize) function can be used, where ksize is set to 3. Normalization: Normalize the size of the dental treatment image, for example, adjust the dental treatment image to a fixed size (for example, 800x800 pixels); and use the minimum-maximum normalization method to normalize the pixel values of the dental treatment image to the range of [0, 1] or [-1, 1]. For example, this can be achieved by the formula normalized_image = (image - min_val) / (max_val -min_val).

[0034] S120. Input the dental diagnosis and treatment image into the dental abnormality detection model to obtain the abnormal area detection result corresponding to the suspected abnormal area in the dental diagnosis and treatment image, wherein the abnormal area detection result includes the abnormal area mask image, the abnormal area bounding box information, the abnormal area classification result and the tooth position classification result to which the abnormal area belongs.

[0035] The dental anomaly detection model is pre-trained. It detects abnormal regions in dental diagnostic images and determines the corresponding detailed information. This detailed information includes the abnormal region mask image, the abnormal region bounding box information, the abnormal region classification result, and the tooth position classification result to which the abnormal region belongs.

[0036] Specifically, an abnormal region mask image uses masking techniques to identify and highlight abnormal regions in an image. The mask is typically a binary image where abnormal regions are marked white (or higher grayscale values) and normal regions are marked black (or lower grayscale values). This allows the abnormal regions in an image to be clearly identified and isolated.

[0037] The abnormal area bounding box is a rectangular box that represents the abnormal area on the tooth. The abnormal area bounding box information refers to the location and size of this rectangular box in the dental image. For example, the abnormal area bounding box information can be expressed as (x, y, h, w), where x and y are the center coordinates of the bounding box, and w and h are the width and height of the bounding box.

[0038] The abnormal region classification result is used to characterize the abnormal category corresponding to the abnormal region. The abnormal region classification result can be at least one of the restoration region category, root filling region category, high-density lesion region category, low-density lesion region category, extra tooth region category, fracture line region category, caries region category, and vacant region category. The restoration region category refers to an area on the tooth that may be a restoration, such as a crown; the root filling region category refers to an area on the tooth that may be a root filling; the high-density lesion region category refers to an area in the tooth in the dental diagnosis image that presents a high-density morphology. This area may have pathological changes such as pulp calcification, pulp stone, diffuse calcification, etc.; the low-density lesion region category refers to an area in the tooth in the dental diagnosis image that presents a low-density morphology. This area may have pathological changes such as abnormal root canal morphology, external root resorption, irregular resorption, inflammation, apical resorption, apical periodontitis, apical abscess, apical granuloma, etc.; the extra tooth region category refers to an area on the tooth that may be a supernumerary tooth, that is, a tooth with a number of teeth exceeding the normal range. The fracture line area category refers to an area on the tooth that may be an alveolar fracture, root fracture, or fracture line. The caries area category refers to an area on the tooth that may be a caries site. The vacancy area category refers to an area on the tooth that may be abnormally empty, such as a tooth defect or tooth dislocation.

[0039] Next, the basic concept of tooth position is introduced. Tooth position refers to the position of teeth in the mouth. Clinically, a specific representation method is used to mark the position of each tooth. Each tooth can be represented by two Arabic numerals. A person has two sets of teeth in his life. The first set of teeth is called deciduous teeth, and the second set of teeth is called permanent teeth. Permanent teeth are teeth that grow after deciduous teeth fall out. Therefore, deciduous teeth are the first set of teeth in the human body, and permanent teeth are the second set of teeth. For a diagram of the arrangement of deciduous teeth and permanent teeth, please refer to Figure 2 There are usually 20 deciduous teeth, divided into four quadrants, with 5 teeth in each quadrant. For a diagram of the arrangement of deciduous teeth, see Figure 2 (a), Figure 2 The specific arrangement of the deciduous teeth shown in (a) is as follows: maxillary left side: 55, 54, 53, 52, 51; maxillary right side: 61, 62, 63, 64, 65; mandibular left side: 85, 84, 83, 82, 81; mandibular right side: 71, 72, 73, 74, 75. The permanent dentition usually consists of 32 teeth, divided into four quadrants, with 8 teeth in each quadrant. For a diagram of the permanent tooth arrangement, see Figure 2 (b), Figure 2(b) The specific arrangement of permanent teeth is as follows: maxillary left side: 18, 17, 16, 15, 14, 13, 12, 11; maxillary right side: 21, 22, 23, 24, 25, 26, 27, 28; mandibular left side: 48, 47, 46, 45, 44, 43, 42, 41; mandibular right side: 31, 32, 33, 34, 35, 36, 37, 38. In this embodiment, the tooth position classification result to which an abnormal region belongs refers to the tooth position information where the abnormal region is located. For example, if the tooth position classification result to which an abnormal region belongs is "75", it means that the abnormal region is located at tooth position 75.

[0040] In this embodiment, the tooth abnormality detection model diagram is shown in Figure 3 ,like Figure 3 As shown, the dental anomaly detection model includes a feature extraction network, a region of interest extraction network, a regional feature alignment network, and an abnormal region evaluation network. Among them, the feature extraction network is used to extract a dental treatment feature map based on the dental treatment image, the region of interest extraction network is used to extract the region of interest from the dental treatment feature map, the regional feature alignment network is used to determine the regional feature information of the region of interest, and the abnormal region evaluation network is used to determine the abnormal region detection result based on the regional feature information of each region of interest. More specifically, the feature extraction network can also include a backbone network and a feature pyramid network, the region of interest extraction network includes a candidate region extraction network and a candidate region screening network, and the abnormal region evaluation network includes a tooth position recognition network and an abnormal region recognition network.

[0041] In actual application, the dental diagnosis and treatment image is input into the dental abnormality detection model. The various network units in the dental abnormality detection model process and analyze the dental diagnosis and treatment image to obtain the abnormal area mask image corresponding to the suspected abnormal area in the dental diagnosis and treatment image, the abnormal area bounding box information, the abnormal area classification result and the tooth position classification result to which the abnormal area belongs.

[0042] More specifically, the specific implementation method of inputting the dental diagnosis image into the dental abnormality detection model to obtain the abnormal area detection result corresponding to the suspected abnormal area in the dental diagnosis image includes:

[0043] S1201: Input a dental diagnosis and treatment image into a feature extraction network to obtain at least one dental diagnosis and treatment feature map corresponding to the dental diagnosis and treatment image.

[0044] Among them, the dental diagnosis feature map refers to the feature image extracted from the dental diagnosis image through convolution and feature extraction operations.

[0045] In this embodiment, the feature extraction network may include a backbone network and a feature pyramid network. The backbone network may be a convolutional neural network. The Feature Pyramid Network (FPN) is used to generate multi-scale feature maps to better handle objects of varying sizes. The FPN fuses feature maps from different levels through a top-down pathway and lateral connections.

[0046] Specifically, the specific implementation method of inputting the dental diagnosis and treatment image into the feature extraction network to obtain at least one dental diagnosis and treatment feature map corresponding to the dental diagnosis and treatment image may include: inputting the dental diagnosis and treatment image into the backbone network to obtain at least one first feature map corresponding to the dental diagnosis and treatment image; inputting each first feature map into the feature pyramid network to obtain at least one dental diagnosis and treatment feature map corresponding to the dental diagnosis and treatment image.

[0047] In this embodiment, by inputting the dental diagnosis and treatment image into the backbone network, the backbone network extracts features of the dental diagnosis and treatment image to obtain one or more first feature maps; further, each first feature map can be input into the feature pyramid network, and the feature pyramid network processes each first feature map to obtain a multi-scale feature map, that is, a plurality of dental diagnosis and treatment feature maps.

[0048] For example, a Caffe-style ResNet-50 can be used as the backbone network, which contains 50 convolutional layers and is divided into four stages. Each stage generates feature maps of different scales. Stage 1: Contains a 7x7 convolution layer with a stride of 2, followed by a 3x3 maximum pooling layer with a stride of 2. Stages 2-4: Each stage contains multiple residual blocks. Each residual block consists of two or three convolutional layers, and the dimension of the feature map is maintained through skip connections. A feature pyramid is constructed based on the output feature map of ResNet-50, which includes a bottom-up path and a top-down path. Bottom-up path: Extract the first feature map (C2, C3, C4, C5) from each stage of ResNet-50, where C2, C3, C4, and C5 correspond to feature maps of different scales. Top-down path: High-level semantic information is transferred to lower levels through upsampling and lateral connections, generating new feature maps (P2, P3, P4, P5), the dental diagnosis feature maps. 1x1 convolutional layer: A 1x1 convolutional layer is used in each lateral connection to adjust the number of channels and maintain the consistent dimensionality of the feature maps.

[0049] S1202: For each dental diagnosis and treatment feature map, input the current dental diagnosis and treatment feature map into a region of interest extraction network to obtain at least one region of interest corresponding to the current dental diagnosis and treatment feature map.

[0050] Regions of Interest (ROIs) are areas of particular interest identified from the dental diagnosis and treatment feature map. These areas are typically marked using boxes, circles, ellipses, irregular polygons, and other methods to facilitate further processing and analysis. Specifically, in this embodiment, the ROIs include each tooth and any abnormal areas on the tooth.

[0051] In this embodiment, the region of interest extraction network includes a candidate region extraction network and a candidate region screening network. The candidate region extraction network can be a region proposal network (RPN), a key component in the Faster R-CNN network. Its primary function is to extract candidate regions (region proposals) from the input image. These candidate regions are then used for object detection and localization. The RPN generates a series of candidate regions by sliding a small network (called anchor boxes) across the feature map and scores these regions to determine whether they contain an object. The region proposal network is applied to the feature map generated by the FPN to generate candidate regions. The candidate region screening network can use non-maximum suppression (NSS) to screen multiple candidate regions to obtain one or more regions of interest. In this embodiment, the candidate region screening network can use non-maximum suppression (NMS) to remove overlapping candidate regions. NMS aims to retain the most likely candidate regions and remove regions that significantly overlap with high-confidence candidate regions.

[0052] Specifically, for each dental diagnosis and treatment feature map, the current dental diagnosis and treatment feature map is input into the region of interest extraction network to obtain at least one region of interest corresponding to the current dental diagnosis and treatment feature map. The specific implementation method may include: for each dental diagnosis and treatment feature map, the current dental diagnosis and treatment feature map is input into the candidate region extraction network to obtain at least one candidate region corresponding to the current dental diagnosis and treatment feature map; based on the candidate region screening network, the at least one candidate region corresponding to the current dental diagnosis and treatment feature map is subjected to region screening to obtain at least one region of interest corresponding to the current dental diagnosis and treatment feature map.

[0053] In this embodiment, the processing process for each dental treatment feature map is the same. Here, any one of the dental treatment feature maps is used as the current dental treatment feature map, and the current dental treatment feature map is used as an example for exemplary description. The current dental treatment feature map refers to the dental treatment feature map currently being processed. By inputting the current dental treatment feature map into the candidate region extraction network, the candidate region extraction network can generate multiple anchor boxes of different scales and aspect ratios on the current dental treatment feature map. Furthermore, each anchor point (i.e., each pixel within the anchor box) can be subjected to binary classification (foreground or background) and bounding box regression to generate candidate regions, and the confidence value corresponding to each candidate region is determined. Furthermore, the confidence value corresponding to each candidate region and the corresponding candidate region bounding box coordinates are input into the candidate region screening network. After the candidate region screening network performs region screening on these candidate regions, the one or more candidate regions obtained after screening are the regions of interest corresponding to the current dental treatment feature map.

[0054] S1203: Input the current dental diagnosis and treatment feature map and at least one region of interest corresponding to the current dental diagnosis and treatment feature map into a region feature alignment network to obtain region feature information corresponding to each region of interest.

[0055] The regional feature information refers to feature data corresponding to the region of interest.

[0056] In this embodiment, the region feature alignment network may be a region of interest alignment network (RoI Align). RoI Align is used to extract features in the region of interest and perform alignment operations to more accurately capture feature information of the target.

[0057] Specifically, the current dental treatment feature map (i.e., P2, P3, P4, or P5 in the above example) and the corresponding regions of interest (ROIs) can be input into the regional feature alignment network. The regional feature alignment network refines the ROIs using bilinear interpolation to obtain regional feature information corresponding to each ROI. Specifically, RoI Align retains more accurate positional information by interpolating the floating-point coordinate positions within the ROI, allowing this layer to better capture the precise contours and features of the target in object detection and instance segmentation tasks.

[0058] S1204. Input the regional feature information of at least one region of interest of each dental diagnosis and treatment feature map into the abnormal region evaluation network to obtain the abnormal region mask image, abnormal region bounding box information, abnormal region classification result and tooth position classification result to which the abnormal region belongs, corresponding to the suspected abnormal region in the dental diagnosis and treatment image.

[0059] In this embodiment, the abnormal area assessment network includes a tooth position recognition network and an abnormal area recognition network. The tooth position recognition network is used to determine the tooth position information in the dental diagnosis and treatment image based on the regional feature information of each region of interest in each dental diagnosis and treatment feature map. The purpose of determining the tooth position information is to provide the doctor with the tooth position information of the abnormal area so that the doctor can quickly locate the specific location of the abnormal part in the patient's mouth, which is conducive to improving the doctor's diagnosis and treatment efficiency. The abnormal area recognition network is used to determine the specific information corresponding to the suspected abnormal part in the dental diagnosis and treatment image based on the regional feature information of each region of interest in each dental diagnosis and treatment feature map.

[0060] Specifically, the specific implementation method of this step may include: inputting the regional feature information of at least one area of interest of each dental diagnosis and treatment feature map into the tooth position recognition network to obtain the tooth position bounding box information and tooth position classification results corresponding to each tooth in the dental diagnosis and treatment image; inputting the regional feature information of at least one area of interest of each dental diagnosis and treatment feature map into the abnormal area recognition network to obtain the abnormal area mask image, abnormal area bounding box information and abnormal area classification results corresponding to the suspected abnormal area in the dental diagnosis and treatment image; based on the abnormal area bounding box information and the tooth position bounding box information and tooth position classification results corresponding to each tooth, determine the tooth position classification result to which the abnormal area belongs.

[0061] The tooth position bounding box information represents the rectangular box of a certain tooth.

[0062] In this embodiment, the tooth position recognition network can specifically include a tooth bounding box regression subnetwork and a tooth position classification subnetwork. The regional feature information of each region of interest in each dental treatment feature image is input into the tooth position evaluation network. After the tooth bounding box regression subnetwork processes and analyzes these regional feature information, it can output the tooth position bounding box information corresponding to each tooth in the dental treatment image; after the tooth position classification subnetwork processes and analyzes these regional feature information, it can output the tooth position classification result corresponding to each tooth in the dental treatment image. It should be noted that the tooth position evaluation network can also include a tooth mask generation subnetwork. After the tooth mask generation subnetwork processes and analyzes these regional feature information, it can output a tooth mask image of the tooth position corresponding to each tooth in the dental treatment image.

[0063] The abnormal region identification network can specifically include an abnormal region bounding box regression subnetwork, an abnormal region classification subnetwork, and an abnormal region mask generation subnetwork. The regional feature information of each region of interest of each dental treatment feature image is input into the abnormal region evaluation network. After the abnormal region bounding box regression subnetwork processes and analyzes these regional feature information, it can output the abnormal region bounding box information of each abnormal region in the identified dental treatment image; after the abnormal region classification subnetwork processes and analyzes these regional feature information, it can output the abnormal region classification result of each abnormal region in the dental treatment image; after the abnormal region mask generation subnetwork processes and analyzes these regional feature information, it can output the abnormal region mask image of each abnormal region in the dental treatment image. For example, the abnormal region mask generation subnetwork can be a fully convolutional network, and a pixel-level segmentation mask can be generated through the fully convolutional network. The size of the abnormal region mask image can be the same as that of the dental treatment image.

[0064] Based on this, the target center coordinates corresponding to the abnormal area can be determined according to the abnormal area bounding box information. Through the target center coordinates and the tooth position bounding box information, it can be determined in which tooth position bounding box the target center coordinates are located. This tooth position bounding box can be determined as the target tooth position bounding box to which the suspected abnormal area belongs. Furthermore, the tooth position classification result corresponding to the target tooth position bounding box can be determined as the tooth position classification result to which the abnormal area belongs.

[0065] The technical solution of the embodiment of the present invention obtains a dental diagnosis image including at least one tooth, and then inputs the dental diagnosis image into a dental abnormality detection model to obtain an abnormal region mask image corresponding to the suspected abnormal region in the dental diagnosis image, abnormal region bounding box information, abnormal region classification results, and tooth position classification results to which the abnormal region belongs, wherein the dental abnormality detection model includes: a feature extraction network for extracting a dental diagnosis feature map based on the dental diagnosis image, a region of interest extraction network for extracting regions of interest from the dental diagnosis feature map, a region feature alignment network for determining regional feature information of the regions of interest, and an abnormal region evaluation network for determining abnormal region detection results based on the regional feature information of each region of interest. The technical solution provided by the embodiment of the present invention can automatically detect abnormal regions in dental diagnosis images through the dental abnormality detection model, and can not only identify the mask image, bounding box, and abnormal region classification results corresponding to the abnormal region, but also identify the tooth position information where the abnormal region is located, thereby improving the accuracy, reliability, and efficiency of detecting dental abnormal regions.

[0066] Example 2

[0067] Figure 4This is a flow chart of a method for detecting dental anomaly areas provided in Example 2 of the present invention. Based on the previous example, the training method for the dental anomaly detection model is further refined. Technical terms that are identical or corresponding to those in the previous example are not repeated here.

[0068] like Figure 4 As shown, the method includes:

[0069] S210: Acquire multiple sample images containing annotation information.

[0070] Among them, the annotation information includes the tooth annotation information corresponding to each tooth in the sample image and the abnormal area annotation information corresponding to the abnormal area on the tooth. The tooth annotation information includes the first annotation boundary box, the first annotation mask image and the tooth position classification label corresponding to each tooth. The abnormal area annotation information includes the second annotation boundary box, the second annotation mask image and the abnormal classification label corresponding to the abnormal area of the tooth.

[0071] In this embodiment, historical dental diagnosis and treatment images generated in historical periods may be obtained as sample images. The historical dental diagnosis and treatment images are dental X-ray images corresponding to patients with abnormal areas on their teeth.

[0072] You can use the annotation tool to annotate historical dental images and generate annotation files in JSON format. For each sample image, the annotation file includes a first annotated bounding box, a first annotated mask image, and a tooth classification label for each tooth; and a second annotated bounding box, a second annotated mask image, and an abnormality classification label for each abnormal tooth area.

[0073] S220: For each sample image, input the current sample image into the tooth abnormality detection model to be trained for image processing to obtain tooth prediction information corresponding to each tooth in the current sample image and abnormal area prediction information corresponding to the abnormal area on the tooth.

[0074] Among them, the tooth prediction information includes the first prediction bounding box, the first prediction mask image, and the predicted tooth position classification corresponding to each tooth, and the abnormal area prediction information includes the second prediction bounding box, the second prediction mask image, and the predicted abnormality classification corresponding to the tooth abnormal area.

[0075] In this embodiment, the model parameters of the tooth anomaly detection model to be trained are initial values, and the model parameters need to be corrected during the model training process to finally obtain the tooth anomaly detection model. The model structure of the tooth anomaly detection model to be trained is Figure 2 The model structure shown.

[0076] Specifically, the processing process for each sample image is the same. Here, any one of the sample images is used as the current sample image, and the current sample image is used as an example for illustrative description. The current sample image is the sample image currently being processed. The current sample image can be input into the tooth anomaly detection model to be trained. After the feature extraction network, the region of interest extraction network, and the region feature alignment network in the tooth anomaly detection model to be trained process the current sample image, the sample region feature information corresponding to each region of interest corresponding to the current sample image can be obtained; further, the tooth bounding box regression subnetwork, the tooth position classification subnetwork, and the tooth mask generation subnetwork in the tooth position recognition network respectively process and analyze these sample region feature information, and then output the first predicted bounding box, predicted tooth position classification, and first predicted mask image corresponding to each tooth in the current sample; the abnormal region bounding box regression subnetwork, the abnormal region classification subnetwork, and the abnormal region mask generation subnetwork in the abnormal region recognition network respectively process and analyze these sample region feature information, and then output the second predicted bounding box, predicted abnormality classification, and second predicted mask image corresponding to each suspected abnormal region in the current sample.

[0077] S230: Determine a tooth result loss value corresponding to the current sample image based on the tooth annotation information and the tooth prediction information.

[0078] Among them, the tooth result loss value is used to characterize the difference between the tooth prediction information and the tooth annotation information.

[0079] Specifically, based on the tooth marking information and the tooth prediction information, a specific implementation method for determining the tooth result loss value may include the following steps:

[0080] S1. Determine a tooth bounding box loss value corresponding to the current sample image based on the offset of the first annotated bounding box and the offset of the first predicted bounding box corresponding to each tooth. Determine a tooth mask loss value corresponding to the current sample image based on the first annotated mask image and the first predicted mask image corresponding to each tooth. Determine a tooth position classification loss value corresponding to the current sample image based on the tooth position classification label and the predicted tooth position classification corresponding to each tooth.

[0081] First, the tooth bounding box loss value is determined by:

[0082] In this embodiment, for each tooth in the current sample image, the bounding box loss value of each tooth is determined separately, and then the bounding box loss values are summed to obtain the tooth bounding box loss value corresponding to the current sample image. The formula for calculating the tooth bounding box loss value corresponding to the current sample image is as follows:

[0083] (1)

[0084] Where, represents the tooth bounding box loss value, Indicates the number of the current sample image The offset of the first annotation bounding box corresponding to the tooth, Indicates the number of the current sample image The offset of the first predicted bounding box corresponding to the tooth; is the smooth L1 loss function, defined as:

[0085] (2)

[0086] Among them, the offset of the first annotation bounding box is obtained from the labeled data. Assume we have a real labeled bounding box , are the center coordinates of the bounding box, and is the width and height of the bounding box. For each region of interest, the offset between it and the true bounding box can be calculated. Assume that the bounding box of the region of interest is , then the true bounding box offset It can be expressed as:

[0087] (3)

[0088] Where: and is the offset of the first annotation's bounding box center coordinates. and are the width and height offsets of the first annotation's bounding box.

[0089] The offset of the first predicted bounding box is predicted by the model through the tooth bounding box regression subnetwork. Assume that the bounding box predicted by the model is , then the offset of the first predicted bounding box is It can be expressed as:

[0090] (4)

[0091] Where: and is the offset of the center coordinates of the first predicted bounding box. and are the width offset and height offset of the first predicted bounding box.

[0092] Secondly, the methods for determining the tooth mask loss value mainly include:

[0093] In this embodiment, for each tooth in the current sample image, the mask loss value of each tooth is determined separately, and then the mask loss values are summed to obtain the mask loss value of the tooth corresponding to the current sample image. The formula for the mask loss value corresponding to a tooth is as follows:

[0094] (5)

[0095] Where, Indicates the first labeled mask image The mask pixel value (0 or 1) corresponding to the pixel, Indicates the first predicted mask image The mask pixel value (0 or 1) corresponding to the pixel, is the resolution of the mask. Based on this, the mask loss value corresponding to each tooth can be calculated in the same way, and these mask loss values are summed to obtain the tooth mask loss value.

[0096] Again, the methods for determining the tooth position classification loss value mainly include:

[0097] In this embodiment, for each tooth in the current sample image, the tooth position loss value of each tooth is determined separately, and then the tooth position loss values are summed to obtain the tooth position classification loss value corresponding to the current sample image. For each tooth in the current sample image, the predicted tooth position classification corresponding to the tooth can be represented as a 1×52-dimensional vector, where each element in the vector represents the probability value of the current tooth being in that tooth position; similarly, the tooth position classification label is also a 1×52-dimensional vector. The calculation formula for the tooth position classification loss value corresponding to a tooth is as follows:

[0098] (6)

[0099] Where, Indicates the total number of teeth (i.e. 52), Indicates the The tooth classification label of each tooth position, Indicates the Predicted tooth position classification of individual tooth positions.

[0100] S2. Determine a tooth position constraint loss value corresponding to the current sample image based on the predicted tooth position classification corresponding to each tooth, the first predicted bounding box, and a preset tooth position constraint function.

[0101] In this embodiment, taking into account the spatial distribution characteristics of tooth positions, a tooth position constraint loss value is introduced to ensure that the tooth position predicted by the model conforms to the actual arrangement of teeth.

[0102] Among them, the preset tooth position constraint function can be expressed as:

[0103] (7)

[0104] Where, Indicates the tooth position constraint loss value;

[0105] Represents the preset position constraint matrix, considering the adjacent relationship between permanent teeth and deciduous teeth and the mutual exclusion relationship between them, the preset position constraint matrix is predefined , the dimension of the preset position constraint matrix is 52x52, and the value of each element position in the matrix is used to represent the relationship between the corresponding two tooth positions. Figure 2 , tooth position 11 and tooth position 12 are adjacent, then ; 11 and 52 are adjacent, then ; 11 and 51 are mutually exclusive, then ; 11 and 14 are not adjacent, then ; In particular, the deciduous teeth 55, 54, 53, 52, 51 on the left side of the maxillary teeth are mutually exclusive with the permanent teeth 15, 14, 13, 12, 11, that is, permanent teeth and deciduous teeth cannot coexist in the same position; similarly, the deciduous teeth 61, 62, 63, 64, 65 on the right side of the maxillary teeth are mutually exclusive with the permanent teeth 21, 22, 23, 24, 25; the deciduous teeth 85, 84, 83, 82, 81 on the left side of the mandible are mutually exclusive with the permanent teeth 45, 44, 43, 42, 41; the deciduous teeth 71, 72, 73, 74, 75 on the right side of the mandible are mutually exclusive with the permanent teeth 31, 32, 33, 34, 35.

[0106] Indicates the The center coordinates of the first predicted bounding box of the tooth are The distance from the center coordinate of the first predicted bounding box of each tooth.

[0107] In this embodiment, based on obtaining the predicted tooth position classification corresponding to each tooth, this information is brought into the preset tooth position constraint function to obtain the tooth position constraint loss value.

[0108] In this embodiment, the tooth position constraint loss value is calculated by calculating the distance between the predicted positions and combining the relative position relationship in the position constraint matrix to punish the prediction that does not conform to the rules, thereby promoting the model to learn the rules of tooth arrangement and improving the robustness of the model.

[0109] S3. Determine the tooth result loss value corresponding to the current sample image based on the bounding box loss value, the tooth mask loss value, the tooth position classification loss value, and the tooth position constraint loss value.

[0110] In this embodiment, the weighted sum of the bounding box loss value, tooth mask loss value, tooth position classification loss value, and tooth position constraint loss value corresponding to each tooth may be determined as the tooth result loss value.

[0111] S240 : Determine the abnormal region result loss value corresponding to the current sample image based on the abnormal region labeling information and the abnormal region prediction information.

[0112] Among them, the abnormal area result loss value is used to characterize the difference between the abnormal area prediction information and the abnormal area annotation information.

[0113] Specifically, based on the abnormal area annotation information and the abnormal area prediction information, a specific implementation method for determining the abnormal area result loss value may include the following steps:

[0114] S1. For each abnormal area, based on the offset of the second annotated bounding box and the offset of the second predicted bounding box corresponding to the current abnormal area, determine a loss value of the abnormal area bounding box corresponding to the sample image.

[0115] In this embodiment, for each abnormal area in the current sample image, the bounding box loss value of each abnormal area is determined respectively, and then the bounding box loss values are summed to obtain the abnormal area bounding box loss value corresponding to the current sample image. The formula for calculating the abnormal area bounding box loss value corresponding to the sample image can be referred to formula (1). The difference from step S1 in S230 is that in this step, Represents the loss value of the abnormal area bounding box, Indicates the sample image The offset of the second annotation bounding box corresponding to the abnormal area, Indicates the sample image The offset of the second predicted bounding box corresponding to the abnormal area. It should be noted that the method for determining the offset of the second annotated bounding box can refer to the specific processing process of the offset of the first annotated bounding box, and the method for determining the offset of the second predicted bounding box can refer to the specific processing process of the offset of the first predicted bounding box, which will not be repeated here.

[0116] S2. Determine a mask loss value of an abnormal area corresponding to the sample image based on the second annotated mask image and the second predicted mask image.

[0117] In this embodiment, for each abnormal area in the current sample image, the mask loss value of each abnormal area is determined respectively, and then the mask loss values are summed to obtain the mask loss value of the abnormal area corresponding to the current sample image. The formula for the mask loss value corresponding to an abnormal area can be found in formula (5). The difference from step S1 in S230 is that in this step, Indicates the first The mask pixel value (0 or 1) corresponding to the pixel, Indicates the first predicted mask image The mask pixel value (0 or 1) corresponding to the pixel, is the resolution of the mask. Based on this, the mask loss value corresponding to each abnormal area can be calculated in the same way, and these mask loss values are summed to obtain the mask loss value of the abnormal area.

[0118] S3. Based on the abnormal classification label and the predicted abnormal classification, determine the abnormal region classification loss value corresponding to the sample image.

[0119] In this embodiment, for each abnormal region in the current sample image, the abnormal region loss value corresponding to the abnormal region can be represented as a 1×N-dimensional vector, where each element in the vector represents the probability value of the current abnormal region being classified as an abnormal region of that type. Similarly, the abnormal classification label is also a 1×N-dimensional vector. For example, if there are eight abnormal region classification results (i.e., at least one of the following: restoration region category, root filling region category, high-density lesion region category, low-density lesion region category, supernumerary tooth region category, fracture line region category, caries region category, and vacancy region category), the abnormal region loss value can be represented as a 1×8-dimensional vector.

[0120] The calculation formula for the tooth position classification loss value corresponding to the abnormal area is as follows:

[0121] (8)

[0122] Where, represents the total number of abnormal region classifications (i.e. 8), Indicates the The abnormal classification label corresponding to the abnormal area, Indicates the The predicted abnormal classification corresponding to the abnormal area.

[0123] S4. Determine the abnormal area result loss value corresponding to the current sample image based on the abnormal area bounding box loss value, abnormal area mask loss value, and abnormal area classification loss value corresponding to each abnormal area.

[0124] In this embodiment, the weighted sum of the abnormal region bounding box loss value, the abnormal region mask loss value, and the abnormal region classification loss value corresponding to each abnormal region can be determined as the abnormal region result loss value.

[0125] S250. Based on the tooth result loss value and abnormal area result loss value of each sample image, the tooth anomaly detection model to be trained is trained until the training is completed when the preset conditions are met, thereby obtaining a tooth anomaly detection model.

[0126] The preset condition is a pre-set condition. When training the tooth anomaly detection model to be trained, if the training result meets the preset condition, it indicates that the training of the tooth anomaly detection model to be trained is completed.

[0127] Optionally, the preset conditions include at least one of the following situations: the number of training iterations reaches a preset iteration threshold; the tooth result loss value and the abnormal area result loss value are respectively less than their corresponding loss thresholds; the comprehensive loss value is determined by the tooth result loss value and the abnormal area result loss value, and the comprehensive loss value is less than the comprehensive loss threshold.

[0128] In an embodiment of the present invention, the tooth result loss value and the abnormal area result loss value are used together to optimize the parameters of the dental anomaly detection model to be trained. Each time a sample image is input into a preset image processing model, a set of tooth result loss values and abnormal area result loss values can be determined. These determined tooth result loss values and abnormal area result loss values can be fed back into the dental anomaly detection model to be trained to optimize the parameters of the dental anomaly detection model to be trained. Furthermore, a comprehensive loss value can be determined from the tooth result loss values and abnormal area result loss values, and this comprehensive loss value can be fed back into the dental anomaly detection model to be trained to optimize the parameters of the dental anomaly detection model to be trained. Subsequently, the trained dental anomaly detection model, after model optimization, processes the next set of sample images. An iteration threshold can be pre-set. When the number of times the trained dental anomaly detection model is trained based on the sample images reaches the iteration threshold, and the tooth result loss value and abnormal area result loss value are each less than their corresponding loss thresholds, the training of the dental anomaly detection model is considered complete. In another embodiment, when the number of times the trained dental anomaly detection model is trained based on the sample images reaches the iteration threshold, and the comprehensive loss value is less than the loss threshold, the training of the dental anomaly detection model is considered complete.

[0129] In particular, training parameters can be pre-set before training the dental anomaly detection model. For example, the learning rate can be set to 0.02 (8 GPUs), the batch size can be set to 16, the optimizer can be set to Stochastic Gradient Descent (SGD), the momentum can be set to 0.9, and the weight decay can be set to 0.0001. Furthermore, during model training, data augmentation methods such as random horizontal flipping, random scaling (with a scale range of 0.8 to 1.2), and random color jittering can be applied to the sample images to increase the diversity and quantity of training data, thereby improving the generalization and robustness of the model and reducing overfitting.

[0130] S260: Acquire a dental diagnosis image to be detected; wherein the dental diagnosis image includes at least one tooth.

[0131] S270. Input the dental diagnosis and treatment image into the dental abnormality detection model to obtain the abnormal area detection result corresponding to the suspected abnormal area in the dental diagnosis and treatment image; wherein the abnormal area detection result includes the abnormal area mask image, the abnormal area bounding box information, the abnormal area classification result and the tooth position classification result to which the abnormal area belongs.

[0132] Among them, the dental abnormality detection model includes: a feature extraction network that extracts a dental diagnosis feature map based on a dental diagnosis image, a region of interest extraction network that extracts a region of interest from the dental diagnosis feature map, a region feature alignment network for determining the regional feature information of the region of interest, and an abnormal region evaluation network that determines the abnormal region detection result based on the regional feature information of each region of interest.

[0133] The technical solution of this embodiment is to obtain a plurality of sample images containing annotation information during the process of training the tooth abnormality detection model, wherein the annotation information includes the tooth annotation information corresponding to each tooth in the sample image and the abnormal area annotation information corresponding to the abnormal area on the tooth, the tooth annotation information includes the first annotation boundary box corresponding to each tooth, the first annotation mask image and the tooth position classification label, and the abnormal area annotation information includes the second annotation boundary box corresponding to the abnormal area of the tooth, the second annotation mask image and the abnormal classification label; then, for each sample image, the current sample image is input into the tooth abnormality detection model to be trained for image processing, and the tooth prediction information corresponding to each tooth in the current sample image and the abnormal area on the tooth are obtained. The abnormal area prediction information corresponding to the tooth prediction includes the first prediction bounding box, the first prediction mask image, and the predicted tooth position classification corresponding to each tooth, and the abnormal area prediction information includes the second prediction bounding box, the second prediction mask image, and the predicted abnormal classification corresponding to the tooth abnormal area; further, based on the tooth annotation information and the tooth prediction information, the tooth result loss value corresponding to the current sample image is determined, and based on the abnormal area annotation information and the abnormal area prediction information, the abnormal area result loss value corresponding to the current sample image is determined; thereby, based on the tooth result loss value and the abnormal area result loss value of each sample image, the tooth abnormality detection model to be trained is trained until the training ends when the preset conditions are met, and the tooth abnormality detection model is obtained. In this embodiment, the tooth abnormality detection model to be trained is trained by multi-dimensional loss values, thereby improving the performance of the tooth abnormality detection model. In particular, a tooth position constraint loss value is designed. The tooth position constraint loss value penalizes irregular predictions by calculating the distance between the predicted positions and combining the relative position relationship in the position constraint matrix, thereby promoting the model to learn the regularity of tooth arrangement and improving the robustness of the model. When the trained dental abnormality detection model is subsequently applied, more accurate tooth position information can be provided. By providing the doctor with the tooth position information of the abnormal area, the doctor can quickly locate the specific position of the abnormal part in the patient's mouth, which is conducive to improving the doctor's diagnosis and treatment efficiency.

[0134] Example 3

[0135] Figure 5 A schematic diagram of a tooth abnormality area detection device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device includes: a dental diagnosis and treatment image acquisition module 310 and an abnormal result determination module 320.

[0136] The dental diagnosis and treatment image acquisition module 310 is used to acquire a dental diagnosis and treatment image to be detected; wherein the dental diagnosis and treatment image includes at least one tooth;

[0137] The abnormal result determination module 320 is used to input the dental diagnosis image into the dental abnormality detection model to obtain abnormal area detection results corresponding to suspected abnormal areas in the dental diagnosis image; wherein the abnormal area detection results include abnormal area mask images, abnormal area bounding box information, abnormal area classification results, and tooth position classification results to which the abnormal area belongs;

[0138] Among them, the dental abnormality detection model includes: a feature extraction network that extracts a dental diagnosis feature map based on a dental diagnosis image, a region of interest extraction network that extracts a region of interest from the dental diagnosis feature map, a region feature alignment network for determining the regional feature information of the region of interest, and an abnormal region evaluation network that determines the abnormal region detection result based on the regional feature information of each region of interest.

[0139] Based on the above embodiment, optionally, the abnormal result determination module 320 includes:

[0140] a feature map determining unit, configured to input the dental diagnosis and treatment image into a feature extraction network to obtain at least one dental diagnosis and treatment feature map corresponding to the dental diagnosis and treatment image;

[0141] A region of interest determination unit is configured to input the current dental diagnosis and treatment feature map into a region of interest extraction network for each dental diagnosis and treatment feature map, and obtain at least one region of interest corresponding to the current dental diagnosis and treatment feature map;

[0142] A regional feature determination unit, configured to input the current dental diagnosis and treatment feature map and at least one region of interest corresponding to the current dental diagnosis and treatment feature map into a regional feature alignment network to obtain regional feature information corresponding to each region of interest;

[0143] The abnormal result determination unit is used to input the regional feature information of at least one area of interest of each dental diagnosis and treatment feature map into the abnormal area evaluation network, and obtain the abnormal area mask image, abnormal area bounding box information, abnormal area classification result and tooth position classification result to which the abnormal area belongs corresponding to the suspected abnormal area in the dental diagnosis and treatment image.

[0144] Based on the above embodiment, optionally, the feature extraction network includes a backbone network and a feature pyramid network, and the feature map determination unit includes:

[0145] A first feature map determining subunit, configured to input the dental diagnosis and treatment image into the backbone network to obtain at least one first feature map corresponding to the dental diagnosis and treatment image;

[0146] The feature map determining subunit is used to input each first feature map into a feature pyramid network to obtain at least one tooth diagnosis feature map corresponding to the tooth diagnosis image.

[0147] Based on the above embodiment, optionally, the region of interest extraction network includes a candidate region extraction network and a candidate region screening network, and the region of interest determination unit includes:

[0148] The candidate region determination subunit is used to input the current tooth diagnosis and treatment feature map into the candidate region extraction network for each tooth diagnosis and treatment feature map, and obtain at least one candidate region corresponding to the current tooth diagnosis and treatment feature map;

[0149] The region of interest determination subunit is used to perform region screening on at least one candidate region corresponding to the current dental diagnosis and treatment feature map based on the candidate region screening network to obtain at least one region of interest corresponding to the current dental diagnosis and treatment feature map.

[0150] Based on the above embodiment, optionally, the abnormal area assessment network includes a tooth position recognition network and an abnormal area recognition network, and the abnormal result determination unit includes:

[0151] a tooth position result determination subunit, configured to input regional feature information of at least one region of interest of each of the dental diagnosis and treatment feature images into the tooth position recognition network, and obtain tooth position bounding box information and tooth position classification results corresponding to each tooth in the dental diagnosis and treatment image;

[0152] an abnormal region result determination subunit, configured to input the region feature information of at least one region of interest of each of the dental diagnosis and treatment feature images into the abnormal region recognition network, and obtain an abnormal region mask image, abnormal region bounding box information, and abnormal region classification results corresponding to the suspected abnormal region in the dental diagnosis and treatment image;

[0153] The abnormal result determination subunit is used to determine the tooth position classification result to which the abnormal area belongs based on the abnormal area bounding box information, the tooth position bounding box information corresponding to each tooth, and the tooth position classification result.

[0154] Based on the above embodiment, optionally, the tooth abnormal area detection device further includes: a model training module; the model training module includes:

[0155] a sample image acquisition unit, configured to acquire a plurality of sample images containing annotation information; wherein the annotation information includes tooth annotation information corresponding to each tooth in the sample image and abnormal region annotation information corresponding to abnormal regions on the teeth, the tooth annotation information including a first annotation bounding box, a first annotation mask image, and a tooth position classification label corresponding to each tooth, and the abnormal region annotation information including a second annotation bounding box, a second annotation mask image, and an abnormal classification label corresponding to the abnormal region of the tooth;

[0156] A prediction information determination unit is configured to input the current sample image into a tooth abnormality detection model to be trained for image processing for each of the sample images, thereby obtaining tooth prediction information corresponding to each tooth in the current sample image and abnormal area prediction information corresponding to the abnormal area on the tooth; wherein the tooth prediction information includes a first prediction bounding box, a first prediction mask image, and a predicted tooth position classification corresponding to each tooth, and the abnormal area prediction information includes a second prediction bounding box, a second prediction mask image, and a predicted abnormality classification corresponding to the abnormal area of the tooth;

[0157] a tooth loss determining unit, configured to determine a tooth loss value corresponding to the current sample image based on the tooth annotation information and the tooth prediction information;

[0158] an abnormal region loss determining unit, configured to determine an abnormal region result loss value corresponding to the current sample image based on the abnormal region labeling information and the abnormal region prediction information;

[0159] The detection model training unit is used to train the tooth abnormality detection model to be trained based on the tooth result loss value and the abnormal area result loss value of each sample image, until the training is completed when the preset conditions are met, thereby obtaining the tooth abnormality detection model.

[0160] Based on the above embodiment, optionally, the tooth loss determination unit includes:

[0161] a first loss determination subunit, configured to determine a tooth bounding box loss value corresponding to the sample image based on an offset of the first annotated bounding box corresponding to each tooth and an offset of the first predicted bounding box, determine a tooth mask loss value corresponding to the sample image based on the first annotated mask image corresponding to each tooth and the first predicted mask image, and determine a tooth position classification loss value corresponding to the sample image based on the tooth position classification label corresponding to each tooth and the predicted tooth position classification;

[0162] a tooth position constraint loss determination subunit, configured to determine a tooth position constraint loss value corresponding to the sample image based on the predicted tooth position classification corresponding to each tooth, the first predicted bounding box, and a preset tooth position constraint function;

[0163] The tooth loss determination subunit is used to determine the tooth result loss value corresponding to the current sample image based on the bounding box loss value, the tooth mask loss value, the tooth position classification loss value and the tooth position constraint loss value.

[0164] Based on the above embodiment, optionally, the preset tooth position constraint function is:

[0165]

[0166] Where, Indicates the tooth position constraint loss value, represents the preset position constraint matrix, Indicates the The center coordinates of the first predicted bounding box of the tooth are The distance from the center coordinate of the first predicted bounding box of each tooth.

[0167] Based on the above embodiment, optionally, the abnormal area loss determination unit includes:

[0168] a bounding box loss determination subunit, configured to determine, for each of the abnormal regions, a bounding box loss value of the abnormal region corresponding to the sample image based on an offset of the second annotated bounding box corresponding to the current abnormal region and an offset of the second predicted bounding box;

[0169] a mask loss determination subunit, configured to determine a mask loss value of an abnormal region corresponding to the sample image based on the second annotated mask image and the second predicted mask image;

[0170] The region classification loss determination subunit is used to determine the abnormal region classification loss value corresponding to the sample image based on the abnormal classification label and the predicted abnormal classification.

[0171] The region result loss determination subunit is used to determine the abnormal region result loss value corresponding to the current sample image based on the abnormal region bounding box loss value, the abnormal region mask loss value, and the abnormal region classification loss value corresponding to each abnormal region.

[0172] Based on the above embodiment, optionally, the abnormal area classification results include: at least one of the restoration area category, root filling area category, high-density lesion area category, low-density lesion area category, extra tooth area category, fracture line area category, caries area category and vacant area category.

[0173] The technical solution of the embodiment of the present invention obtains a dental diagnosis image including at least one tooth, and then inputs the dental diagnosis image into a dental abnormality detection model to obtain an abnormal region mask image corresponding to the suspected abnormal region in the dental diagnosis image, abnormal region bounding box information, abnormal region classification results, and tooth position classification results to which the abnormal region belongs, wherein the dental abnormality detection model includes: a feature extraction network for extracting a dental diagnosis feature map based on the dental diagnosis image, a region of interest extraction network for extracting regions of interest from the dental diagnosis feature map, a region feature alignment network for determining regional feature information of the regions of interest, and an abnormal region evaluation network for determining abnormal region detection results based on the regional feature information of each region of interest. The technical solution provided by the embodiment of the present invention can automatically detect abnormal regions in dental diagnosis images through the dental abnormality detection model, and can not only identify the mask image, bounding box, and abnormal region classification results corresponding to the abnormal region, but also identify the tooth position information where the abnormal region is located, thereby improving the accuracy, reliability, and efficiency of detecting dental abnormal regions.

[0174] The tooth abnormality area detection device provided in the embodiment of the present invention can execute the tooth abnormality area detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0175] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of the present invention.

[0176] Example 4

[0177] Figure 6 The present invention provides a schematic structural diagram of an electronic device. Figure 6 A block diagram of an exemplary electronic device 40 suitable for implementing exemplary embodiments of the present invention is shown. Figure 6 The electronic device 40 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0178] like Figure 6 As shown, electronic device 40 is a general-purpose computing device. Components of electronic device 40 may include, but are not limited to, one or more processors or processing units 401, system memory 402, and a bus 403 connecting various system components (including system memory 402 and processing unit 401).

[0179] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0180] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0181] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 Not shown, usually called a "hard drive"). Although Figure 6 Although not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), as well as an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data media interfaces. System memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0182] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in system memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 407 generally implement the functions and / or methods of the embodiments described herein.

[0183] The electronic device 40 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 40, and / or any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 411. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 412. As shown, the network adapter 412 communicates with other modules of the electronic device 40 via the bus 403. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0184] The processing unit 401 executes various functional applications and page processing by running the programs stored in the system memory 402, such as implementing the tooth abnormality area detection method provided in the embodiment of the present invention.

[0185] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the I / O interface 411, or installed from the storage system 406. When the computer program is executed by the processing unit 401, the above-mentioned functions defined in the method of the embodiment of the present invention are performed.

[0186] Example 5

[0187] An embodiment of the present invention further provides a storage medium containing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform a method for detecting abnormal tooth areas, the method comprising:

[0188] Acquire a dental diagnosis image to be detected; wherein the dental diagnosis image includes at least one tooth;

[0189] Inputting the dental diagnosis and treatment image into a dental abnormality detection model to obtain abnormal area detection results corresponding to suspected abnormal areas in the dental diagnosis and treatment image; wherein the abnormal area detection results include an abnormal area mask image, abnormal area bounding box information, abnormal area classification results, and tooth position classification results to which the abnormal area belongs;

[0190] Among them, the dental abnormality detection model includes: a feature extraction network for extracting a dental diagnosis feature map based on the dental diagnosis image, a region of interest extraction network for extracting a region of interest from the dental diagnosis feature map, a region feature alignment network for determining the region feature information of the region of interest, and an abnormal region evaluation network for determining the abnormal region detection result based on the region feature information of each region of interest.

[0191] The computer storage media of the embodiments of the present invention may employ any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0192] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0193] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0194] Computer program code for performing the operations of embodiments of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0195] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for detecting abnormal tooth areas, characterized in that: include: Acquire a dental diagnosis image to be detected; wherein the dental diagnosis image includes at least one tooth; Inputting the dental diagnosis and treatment image into a dental abnormality detection model to obtain an abnormal region detection result corresponding to a suspected abnormal region in the dental diagnosis and treatment image; wherein the abnormal region detection result includes an abnormal region mask image, abnormal region bounding box information, an abnormal region classification result, and a tooth position classification result to which the abnormal region belongs; the tooth position classification result is the tooth position information of the tooth where the abnormal region is located; The dental abnormality detection model includes: a feature extraction network for extracting a dental diagnosis feature map based on the dental diagnosis image, a region of interest extraction network for extracting a region of interest from the dental diagnosis feature map, a region feature alignment network for determining regional feature information of the region of interest, and an abnormal region evaluation network for determining the abnormal region detection result based on the regional feature information of each region of interest; The abnormal area assessment network includes: a tooth position recognition network and an abnormal area recognition network; The tooth position recognition network is used to determine the tooth position classification result to which the abnormal area in the dental diagnosis and treatment image belongs based on the regional feature information of each region of interest in the dental diagnosis and treatment feature map; The abnormal area recognition network is used to determine the abnormal area mask image, the abnormal area bounding box information and the abnormal area classification result corresponding to the suspected abnormal part in the dental diagnosis and treatment image based on the area feature information of each area of interest in the dental diagnosis and treatment feature map.

2. The method according to claim 1, characterized in that The step of inputting the dental diagnosis and treatment image into a dental abnormality detection model to obtain abnormal area detection results corresponding to suspected abnormal areas in the dental diagnosis and treatment image includes: Inputting the dental treatment image into the feature extraction network to obtain at least one dental treatment feature map corresponding to the dental treatment image; For each of the dental diagnosis and treatment characteristic maps, inputting the current dental diagnosis and treatment characteristic map into the region of interest extraction network to obtain at least one region of interest corresponding to the current dental diagnosis and treatment characteristic map; Inputting the current dental diagnosis and treatment feature map and at least one region of interest corresponding to the current dental diagnosis and treatment feature map into a regional feature alignment network to obtain regional feature information corresponding to each region of interest; The regional feature information of at least one region of interest of each of the dental diagnosis and treatment feature images is input into the abnormal area evaluation network to obtain the abnormal area mask image, abnormal area bounding box information, abnormal area classification result and tooth position classification result to which the abnormal area belongs corresponding to the suspected abnormal area in the dental diagnosis and treatment image.

3. The method according to claim 2, characterized in that The feature extraction network includes a backbone network and a feature pyramid network. Inputting the dental diagnosis image into the feature extraction network to obtain at least one dental diagnosis feature map corresponding to the dental diagnosis image includes: Inputting the dental diagnosis and treatment image into the backbone network to obtain at least one first feature map corresponding to the dental diagnosis and treatment image; Each of the first feature maps is input into the feature pyramid network to obtain at least one tooth diagnosis feature map corresponding to the tooth diagnosis image.

4. The method according to claim 2, characterized in that The region of interest extraction network includes a candidate region extraction network and a candidate region screening network. For each of the dental diagnosis and treatment feature maps, the current dental diagnosis and treatment feature map is input into the region of interest extraction network to obtain at least one region of interest corresponding to the current dental diagnosis and treatment feature map, including: For each of the tooth diagnosis and treatment feature maps, inputting the current tooth diagnosis and treatment feature map into the candidate region extraction network to obtain at least one candidate region corresponding to the current tooth diagnosis and treatment feature map; Based on the candidate region screening network, at least one candidate region corresponding to the current dental diagnosis and treatment feature map is screened to obtain at least one region of interest corresponding to the current dental diagnosis and treatment feature map.

5. The method according to claim 2, characterized in that The abnormal region assessment network includes a tooth position recognition network and an abnormal region recognition network. The region feature information of at least one region of interest in each of the dental diagnosis and treatment feature images is input into the abnormal region assessment network to obtain an abnormal region mask image corresponding to the suspected abnormal region in the dental diagnosis and treatment image, abnormal region bounding box information, abnormal region classification results, and tooth position classification results to which the abnormal region belongs, including: Inputting regional feature information of at least one region of interest of each of the dental diagnosis and treatment feature images into the tooth position recognition network to obtain tooth position bounding box information and tooth position classification results corresponding to each tooth in the dental diagnosis and treatment image; Inputting regional feature information of at least one region of interest in each of the dental diagnosis and treatment feature images into the abnormal region recognition network to obtain an abnormal region mask image, abnormal region bounding box information, and abnormal region classification results corresponding to the suspected abnormal region in the dental diagnosis and treatment image; The tooth position classification result to which the abnormal region belongs is determined based on the abnormal region bounding box information, the tooth position bounding box information corresponding to each tooth, and the tooth position classification result.

6. The method according to claim 1, characterized in that Also includes: Acquire multiple sample images containing annotation information; wherein the annotation information includes tooth annotation information corresponding to each tooth in the sample image and abnormal region annotation information corresponding to abnormal regions on the teeth, the tooth annotation information includes a first annotation bounding box corresponding to each tooth, a first annotation mask image, and a tooth position classification label, and the abnormal region annotation information includes a second annotation bounding box corresponding to the abnormal region of the tooth, a second annotation mask image, and an abnormal classification label; For each of the sample images, the current sample image is input into the tooth abnormality detection model to be trained for image processing to obtain tooth prediction information corresponding to each tooth in the current sample image and abnormal area prediction information corresponding to the abnormal area on the tooth; wherein the tooth prediction information includes a first prediction bounding box, a first prediction mask image, and a predicted tooth position classification corresponding to each tooth, and the abnormal area prediction information includes a second prediction bounding box, a second prediction mask image, and a predicted abnormality classification corresponding to the tooth abnormal area; Determining a tooth result loss value corresponding to the current sample image based on the tooth annotation information and the tooth prediction information; Determining an abnormal region result loss value corresponding to the current sample image based on the abnormal region labeling information and the abnormal region prediction information; Based on the tooth result loss value and the abnormal area result loss value of each sample image, the tooth abnormality detection model to be trained is trained until the training is completed when a preset condition is met, thereby obtaining the tooth abnormality detection model.

7. The method according to claim 6, characterized in that The determining, based on the tooth annotation information and the tooth prediction information, a tooth result loss value corresponding to the current sample image includes: Determining a tooth bounding box loss value corresponding to the sample image based on the offset of the first labeled bounding box corresponding to each tooth and the offset of the first predicted bounding box; determining a tooth mask loss value corresponding to the sample image based on the first labeled mask image corresponding to each tooth and the first predicted mask image; and determining a tooth position classification loss value corresponding to the sample image based on the tooth position classification label corresponding to each tooth and the predicted tooth position classification; Determining a tooth position constraint loss value corresponding to the sample image based on the predicted tooth position classification corresponding to each tooth, the first predicted bounding box, and a preset tooth position constraint function; Based on the bounding box loss value, the tooth mask loss value, the tooth position classification loss value and the tooth position constraint loss value, a tooth result loss value corresponding to the current sample image is determined.

8. The method according to claim 7, characterized in that The preset tooth position constraint function is: Where, Indicates the tooth position constraint loss value, represents the preset position constraint matrix, Indicates the The center coordinates of the first predicted bounding box of the tooth are the same as the The distance from the center coordinates of the first predicted bounding box of the tooth.

9. The method according to claim 6, characterized in that The determining, based on the abnormal region labeling information and the abnormal region prediction information, a result loss value of the abnormal region corresponding to the current sample image includes: For each of the abnormal regions, determining a loss value of a bounding box of the abnormal region corresponding to the sample image based on an offset of the second annotated bounding box and an offset of the second predicted bounding box corresponding to the current abnormal region; determining a mask loss value of an abnormal area corresponding to the sample image based on the second annotated mask image and the second predicted mask image; Determining a classification loss value of an abnormal region corresponding to the sample image based on the abnormal classification label and the predicted abnormal classification; Based on the abnormal area bounding box loss value, the abnormal area mask loss value, and the abnormal area classification loss value corresponding to each abnormal area, the abnormal area result loss value corresponding to the current sample image is determined.

10. The method according to claim 1, characterized in that The abnormal area classification results include: at least one of: restoration area category, root filling area category, high-density lesion area category, low-density lesion area category, supernumerary tooth area category, fracture line area category, caries area category and vacant area category.

Images