Caries recognition method, device and storage medium

Abstract

The application discloses a caries recognition method and device and a storage medium, and is applied to the technical field of oral image processing. The method comprises the following steps: acquiring an oral full-mouth image and a corresponding image of each tooth; extracting a first image feature of each tooth image based on a trained tooth-level attention network; performing feature extraction on the oral full-mouth image to obtain an oral full-mouth image feature, and acquiring a second image feature of each tooth from the oral full-mouth image feature based on a tooth positioning result; arranging the second image feature of each tooth into a tooth feature node sequence based on a preset tooth position sorting rule; performing feature extraction on the tooth feature node sequence based on a trained case-level attention network to obtain a full-mouth context feature of the oral full-mouth image; and performing recognition on the first image feature and the full-mouth context feature based on a trained recognition network to obtain a caries recognition result, so that the recognition accuracy of complex scenes such as interproximal caries and multiple caries can be improved.

Description

Technical Field

[0001] This application belongs to the field of oral image processing technology, and in particular relates to a method, device and storage medium for dental caries identification. Background Technology

[0002] With the widespread application of artificial intelligence technology in medical image analysis, deep learning-based model-assisted diagnosis of oral diseases has become an important development direction in oral medical imaging. In the area of ​​caries identification, mainstream analysis methods typically begin by separating the region of interest (ROI) of a single tooth from a full-mouth image. Then, a convolutional neural network is used to independently extract features from the local image blocks of each tooth. Finally, a fully connected layer outputs the pathological classification result for each tooth.

[0003] However, when identifying cavities based on a single tooth, the accuracy is not high for complex scenarios such as proximal caries and multiple caries because it only focuses on the local image area of ​​the current tooth. Summary of the Invention

[0004] This application provides a method, device, and storage medium for identifying dental caries, which can improve the accuracy of identification in complex scenarios such as proximal caries and multiple caries.

[0005] In a first aspect, embodiments of this application provide a method for identifying dental caries, the method comprising: Obtain full-mouth images, as well as images of each tooth corresponding to the full-mouth images; Based on the trained tooth-level attention network, the first image features of each tooth image are extracted. Feature extraction is performed on the full-mouth images to obtain full-mouth image features, and based on the tooth localization results, the second image features of each tooth are obtained from the full-mouth image features. Based on the preset tooth position sorting rules, the second image features of each tooth are arranged into a tooth feature node sequence; Based on the trained case-level attention network, feature extraction is performed on the tooth feature node sequence to obtain the full-mouth context features of the whole mouth image. Based on the trained recognition network, the first image features and full mouth context features are recognized to obtain the caries recognition results.

[0006] In some possible implementations, a first image feature is extracted for each tooth image based on a trained tooth-level attention network, including: Multi-layer convolutional feature extraction is performed on each tooth image to obtain multiple initial image features; For each tooth image, spatial weights are assigned to the initial image features based on a spatial attention subnetwork to obtain spatially weighted image features; Based on the channel attention subnetwork, channel weights are assigned to spatially weighted image features to obtain the first image feature of each tooth image.

[0007] In some possible implementations, for each tooth image, spatial weights are assigned to the initial image features based on a spatial attention subnetwork to obtain spatially weighted image features, including: Multiple sets of convolution kernels of different shapes are used to convolve the initial image features to generate multiple first initial spatial attention maps; the convolution kernels include: vertical rectangular convolution kernel, square convolution kernel, horizontal rectangular convolution kernel and elliptical convolution kernel; A weighted summation is performed on multiple initial spatial attention maps to generate a first final spatial attention map. The first final spatial attention map is multiplied element-wise with the initial image features to obtain spatially weighted image features.

[0008] In some possible implementations, for each tooth image, spatial weights are assigned to the initial image features based on a spatial attention subnetwork to obtain spatially weighted image features, including: Multiple sets of convolution kernels with different directions are used to convolve the initial image features to generate multiple second initial spatial attention maps; the convolution kernels include: horizontal convolution kernels, vertical convolution kernels and diagonal convolution kernels; Based on the initial image features, the weights corresponding to multiple second initial spatial attention maps are determined, and the multiple second initial spatial attention maps are weighted and summed based on the weights to generate a second final spatial attention map. The second final spatial attention map is multiplied element-wise with the initial image features to obtain spatially weighted image features.

[0009] In some possible implementations, before arranging the image features of each tooth into a sequence of tooth feature nodes based on a preset tooth position sorting rule, the following steps are also included: Each tooth is coded to obtain a tooth position code; the tooth position code is determined based on at least one of the following: tooth position numbering system, tooth quadrant, tooth relative position index, maxillary and mandibular information, and left and right side information; The tooth position code of each tooth is fused with the corresponding image features to obtain the first fused image feature for each tooth. Based on the preset tooth position sorting rules, the features of the first fused image are arranged into a sequence of tooth feature nodes.

[0010] In some possible implementations, before arranging the image features of each tooth into a sequence of tooth feature nodes based on a preset tooth position sorting rule, the following steps are also included: Each tooth is coded for its occlusal status to obtain an occlusal status code; wherein, the occlusal status code is determined based on at least one of the following: the presence of the corresponding opposing tooth, the relative positional relationship of the opposing tooth, the degree of overlap with the opposing tooth, the change in occlusal support caused by missing teeth, and the state of tooth elongation or tilting. The occlusal state encoding of each tooth is fused with the corresponding image features to obtain the second fused image feature for each tooth. Based on the preset tooth position sorting rules, the features of the second fused image are arranged into a sequence of tooth feature nodes.

[0011] In some possible implementations, before identifying the first image features and full-mouth context features based on the trained recognition network to obtain the caries identification result, the following steps are also included: The first image features and the full-mouth context features are input into the gated fusion sub-network, and the fusion weights corresponding to the first image features and the full-mouth context features are calculated based on the gated fusion sub-network. Based on the fusion weight, feature fusion is performed on the first image features and the full mouth context features to obtain fused features; Based on the recognition network, the fused features are identified to obtain the caries recognition results.

[0012] In some possible implementations, before extracting the first image features of each tooth image based on the trained tooth-level attention network, the following steps are also included: Obtain a dental caries recognition training set, which includes multiple dental caries recognition training samples. Each dental caries recognition training sample includes: a whole mouth image sample, an image sample of each tooth corresponding to the whole mouth image, and a dental caries recognition result label for each tooth. Based on a tooth-level attention network, the predicted first image features of each tooth image sample are extracted. Feature extraction is performed on the whole mouth image samples to obtain the predicted whole mouth image features. Based on the tooth localization results, the predicted second image features of each tooth are obtained from the predicted whole mouth image features. Based on the preset tooth position sorting rules, the predicted second image features of each tooth are arranged into a predicted tooth feature node sequence; Based on a case-level attention network, feature extraction is performed on the predicted tooth feature node sequence to obtain the predicted whole-mouth context features of the whole-mouth image. Based on the recognition network, the predicted first image features and predicted whole mouth context features are identified to obtain the predicted caries recognition results; The classification loss is calculated based on the predicted caries identification results and the caries identification result labels; The regularization loss is calculated based on the parameters of the tooth-level attention network, the case-level attention network, and the recognition network. Based on classification loss and regularization loss, it is determined whether the preset training stopping condition is met. If not, the network parameters of at least one of the tooth-level attention network, case-level attention network, and recognition network are adjusted, and training continues using the caries recognition training set until the preset training stopping condition is met, resulting in the trained tooth-level attention network, the trained case-level attention network, and the trained recognition network.

[0013] Secondly, embodiments of this application provide a dental caries identification device, the device comprising: The acquisition module is used to acquire full-mouth images and images of each tooth corresponding to the full-mouth images; The extraction module is used to extract the first image features of each tooth image based on the trained tooth-level attention network. The first feature extraction module is used to extract features from the full oral cavity image to obtain the full oral cavity image features, and based on the tooth localization results, to obtain the second image features of each tooth from the full oral cavity image features. The sorting module is used to arrange the second image features of each tooth into a sequence of tooth feature nodes based on a preset tooth position sorting rule. The second feature extraction module is used to extract features from the tooth feature node sequence based on the trained case-level attention network to obtain the full-mouth context features of the oral cavity image. The recognition module is used to identify the first image features and the full mouth context features based on the trained recognition network to obtain the caries recognition result.

[0014] Thirdly, embodiments of this application provide an electronic device, the device comprising: A processor and a memory storing computer program instructions; a method for identifying dental caries that implements any one of the above when the processor executes the computer program instructions.

[0015] Fourthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement a dental caries identification method as described above.

[0016] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by the processor of an electronic device, enable the electronic device to perform any of the above-mentioned methods for identifying dental caries.

[0017] This application discloses a method, device, and storage medium for identifying dental caries. By using a trained tooth-level attention network, it extracts the first image features of a single tooth image, thereby accurately mining the local refined features such as the shape, density, and surface texture of a single tooth. Then, it extracts features from the entire oral cavity image to obtain the entire oral cavity image features. Based on the tooth localization results, it obtains the second image features of each tooth from the entire oral cavity image features. Based on a preset tooth position sorting rule, it arranges the second image features of each tooth into a tooth feature node sequence. Next, based on a trained case-level attention network, it extracts features from the tooth feature node sequence to obtain the entire oral cavity context features. The entire oral cavity context features of each tooth no longer only contain its own local information but also integrate information from all related teeth from a global perspective. Therefore, based on the trained recognition network, it identifies the first image features and the entire oral cavity context features to obtain the dental caries identification result. This enables the synergistic fusion of local detailed features and global context features, improving the accuracy of dental caries identification in complex scenarios such as proximal caries and multiple caries. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the first method for identifying dental caries provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating the second caries identification method provided in this application embodiment is shown; Figure 3 A flowchart illustrating the third method for identifying dental caries provided in this application is shown. Figure 4 A flowchart illustrating the fourth caries identification method provided in this application embodiment is shown; Figure 5 This illustration shows a structural schematic diagram of a dental caries identification device provided in an embodiment of this application; Figure 6 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0020] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0022] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0023] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0024] In existing technologies, since each tooth is not isolated, the interproximal areas between adjacent teeth and the occlusal relationship between the upper and lower teeth contain rich identification information. However, when identifying caries based on a single tooth, only the local image area of ​​the current tooth is focused on, resulting in low identification accuracy when dealing with complex clinical scenarios such as early identification of interproximal caries and comprehensive judgment of multiple caries.

[0025] To address the problems of existing technologies, this application provides a method, device, and storage medium for dental caries identification. To improve the accuracy of dental caries identification in complex scenarios such as interproximal caries and multiple caries, this application, based on the extraction of first image features from a single tooth image, extracts features from a full-mouth image to obtain full-mouth image features. Based on tooth localization results, second image features of each tooth are obtained from the full-mouth image features. Then, based on a preset tooth position sorting rule, the second image features of each tooth are arranged into a sequence of tooth feature nodes, such that nodes at adjacent positions in the sequence correspond to teeth with adjacent or similar anatomical positions. Next, based on a trained case-level attention network, features are extracted from the tooth feature node sequence to obtain the full-mouth context features of the full-mouth image. The full-mouth context features of each tooth no longer only contain its own local information but also integrate information from all related teeth from a global perspective. Therefore, when identifying dental caries, it is possible to achieve the collaborative fusion of local detail features and global context features, improving the accuracy of dental caries identification in complex scenarios such as interproximal caries and multiple caries.

[0026] The following is a description of a tooth decay identification method provided in the embodiments of this application.

[0027] Figure 1 A flowchart illustrating the first method for identifying dental caries provided in this application is shown. Figure 1 As shown, the method may include the following steps: S101: Acquire full-mouth images and images of each tooth corresponding to the full-mouth images.

[0028] In this embodiment, the full-mouth image refers to a panoramic radiographic image of the oral cavity, which completely covers the patient's entire dentition. Each tooth image refers to a region of interest image containing only a single tooth, separated from the full-mouth image. Each tooth image can be generated by locating each tooth in the full-mouth image using a tooth instance segmentation model or a target detection model, obtaining the bounding box coordinates or segmentation mask for each tooth, and then cropping the corresponding tooth region from the full-mouth image based on the obtained bounding box coordinates or segmentation mask to obtain several single-tooth images.

[0029] In one example, the differences in format, size, grayscale distribution, etc., of images acquired by different devices and with different scanning parameters are eliminated to ensure that the data input to the model has a uniform and stable numerical distribution. After acquiring the full mouth images, the full mouth images can be standardized to obtain standardized tooth images.

[0030] The standardization process includes: channel and grayscale standardization; converting the original image into a single-channel grayscale image, removing redundant information from the color channel, and normalizing the grayscale values ​​to a fixed range, such as [0,1] or [-1,1]. This operation ensures that images acquired by different devices have a consistent range of grayscale values. Size and scale standardization; scaling the full-mouth image to a uniform resolution size, such as scaling to a fixed height and width, ensuring consistency in the input size of subsequent networks and avoiding feature extraction deviations caused by differences in input size. Centering and geometric pose standardization; performing rotation correction and centering adjustment on the full-mouth image to ensure that the tooth area is not skewed or offset in the image, and that the image as a whole is not rotated or tilted, eliminating geometric pose differences caused by shooting positioning deviations. Anatomical integrity verification; checking whether the image completely covers the entire permanent or deciduous dentition and alveolar bone area, ensuring that no key anatomical structures are missing. For images with incomplete structures, appropriate completion or marking processing is performed. Gray-level and contrast standardization: Histogram equalization or contrast-limited adaptive histogram equalization is performed on the images to improve the discriminative power of subsequent feature extraction. Artifact and noise suppression: Filtering methods are used to reduce or eliminate scanning noise, metal artifacts, and other imaging artifacts in the images, reducing the adverse effects of interference factors on model judgment and improving image quality. Coordinate system and annotation standardization: A unified standard coordinate system is established for the preprocessed images. Tooth position numbering and region of interest division follow general specifications for oral medical imaging. If accompanying annotation data is available, ensure that the annotation boxes, segmentation masks, and standardized images are accurately matched without misalignment or deviation.

[0031] S102: Based on the trained tooth-level attention network, extract the first image features of each tooth image.

[0032] In this embodiment, a tooth-level attention network is used to extract local features from a single tooth image, thereby focusing on the carious area of ​​a single tooth. This allows for the precise extraction of the morphological, density, and surface texture features of a single tooth, enhancing the expression of caries-related local features.

[0033] In one example, the network structure of the tooth-level attention network can be a parallel structure of spatial attention and channel attention. In this parallel structure, the input single tooth image is first processed by a multi-layer convolutional neural network for preliminary feature extraction, resulting in multiple initial image features. Subsequently, the initial image features are fed into two parallel processing branches: spatial attention and channel attention. The channel attention mechanism optimizes the feature weight allocation for the proximal and occlusal surfaces where caries is prone to occur, resulting in channel-weighted image features. Simultaneously, the spatial attention mechanism identifies local lesion areas of the tooth, resulting in spatially weighted image features. Finally, the channel-weighted image features and the spatially weighted image features are fused to obtain the first image feature. This allows the network to highlight the features of high-incidence caries areas while improving computational efficiency through parallel execution.

[0034] S103: Extract features from the full-mouth image to obtain full-mouth image features, and based on the tooth localization results, obtain the second image features of each tooth from the full-mouth image features.

[0035] In this embodiment, the first image feature is extracted only from a single tooth image and cannot characterize global contextual information such as the adjacent positional relationship between teeth, the distribution of caries in the whole mouth, and the occlusal state. It is difficult to meet the feature recognition requirements of complex caries scenarios such as interproximal caries and multiple caries. Therefore, this application extracts features from the whole mouth image to obtain the whole mouth image features, and based on the tooth localization results, obtains the second image features of each tooth from the whole mouth image features. This enables the second image features to have the whole mouth spatial association attribute, providing support for the subsequent extraction of global contextual features by the case-level attention network.

[0036] In one example, features can be extracted from a full-mouth image using a backbone network to obtain the full-mouth image features.

[0037] In another example, the feature regions corresponding to each tooth in the whole mouth can be determined by the tooth detection or segmentation results. Based on the positioning coordinates and boundary range of the teeth, adaptive pooling or precise cropping operations are performed on the feature regions corresponding to each tooth to retain the global correlation information within the feature regions and obtain the second image features corresponding to each tooth.

[0038] S104: Based on the preset tooth position sorting rules, the second image features of each tooth are arranged into a tooth feature node sequence.

[0039] In this embodiment, the dispersed and independent second image features cannot reflect the anatomical positional relationship of teeth in the dental arch. The case-level attention network has difficulty directly modeling global contextual information such as adjacent tooth association and full mouth distribution for disordered features. Therefore, it is necessary to arrange the second image features in an orderly manner according to fixed rules to construct a structured tooth feature node sequence so that the case-level attention network can accurately perceive the spatial adjacency relationship between teeth.

[0040] The preset tooth position sorting rule refers to a predefined rule for sorting all teeth according to the anatomical arrangement of the dental arch or the tooth position numbering order. This rule ensures that adjacent nodes in the sequence correspond to teeth that are anatomically adjacent or close to each other. The tooth feature node sequence refers to an ordered feature sequence formed by arranging the second image features of each tooth sequentially according to the preset sorting rule. ,in, Represents a sequence of tooth feature nodes, with dimension 1. ; The second image feature representing the tooth at the Nth position in the sequence; N represents the number of teeth; The dimension representing the second image feature of each tooth.

[0041] In one example, the preset tooth position sorting rules include, but are not limited to, the following sorting methods: sorting by tooth position number, sorting by dental arch arrangement, sorting from left to right, or sorting from right to left.

[0042] The tooth position numbering order refers to the sorting according to the FDI tooth position numbers. The FDI tooth position numbers are represented by two digits, with the tens digit indicating the quadrant and the units digit indicating the tooth position order from the midline to the distal end within that quadrant. The dental arch arrangement order refers to the sorting of teeth according to their actual anatomical course within the dental arch. For example, it can start from the right maxillary third molar (FDI number 18), proceeding along the maxillary dental arch to the left to the left maxillary third molar (FDI number 28), then turning to the left mandibular third molar (FDI number 38), and then proceeding along the mandibular dental arch to the right to the right mandibular third molar (FDI number 48). This arrangement completely traverses the anatomical course of the entire maxillary and mandibular dental arches, ensuring that adjacent teeth in the sequence are anatomically adjacent.

[0043] S105: Based on the trained case-level attention network, feature extraction is performed on the tooth feature node sequence to obtain the full-mouth context features of the oral cavity image.

[0044] In this embodiment of the application, based on the trained case-level attention network, feature extraction is performed on the tooth feature node sequence to obtain the full-mouth context features of the full-mouth image. This enables the feature representation of each tooth to no longer be limited to the local image information of the tooth itself, but to integrate the information of all other teeth with anatomical association with it from the global perspective of the full-mouth panoramic image.

[0045] For example, in the scenario of diagnosing proximal caries, when the proximal features of a tooth are not clear enough, the case-level attention network can automatically focus on the corresponding proximal region of the adjacent tooth through a self-attention mechanism. If the corresponding region of the adjacent tooth has clear caries features, this information will be aggregated into the feature representation of the current tooth through attention weights, thereby enhancing the confidence in diagnosing proximal caries of the current tooth. In the scenario of diagnosing multiple caries, the common pathological patterns presented by multiple teeth throughout the entire mouth can be shared and propagated by each tooth node through a self-attention mechanism, enabling the network to perceive the global distribution trend of multiple caries from the perspective of the entire mouth and feed this global prior information back into the diagnosis process of each individual tooth, improving the accuracy of identifying early or atypical caries lesions.

[0046] In one example, the case-level attention network can be a neural network built on self-attention operations. Specifically, an attention matrix is ​​generated by performing self-attention computation on each tooth feature vector within the sequence of tooth feature nodes. ,in, This represents the attention weight of the i-th tooth to the j-th tooth. The second image features of all teeth are weighted and aggregated using the attention matrix to obtain the whole-mouth context features. It should be noted that the whole-mouth context features refer to the context features carried by each tooth. The whole-mouth context features of each tooth no longer contain its own local information, but rather integrate information from all related teeth from a global perspective of the whole-mouth image. The formula for calculating the whole-mouth context features is: in, This represents the full-mouth contextual features corresponding to the i-th tooth; The attention weight of the i-th tooth on the j-th tooth is represented by ; N represents the number of teeth. The second image feature represents the j-th tooth.

[0047] In another example, a tooth feature map structure can be constructed based on a predefined tooth position relationship. Each tooth is considered a node in the graph, and the node feature is the second image feature of that tooth. Edges are established between nodes according to the anatomical relationships between teeth, forming the graph structure. These edges can be constructed based on one or more of the following relationships: adjacency edges: undirected edges are established between corresponding nodes for teeth anatomically adjacent in the dental arch; symmetry edges: edges are established between corresponding nodes for teeth with symmetrical relationships in the upper and lower jaws or left and right sides; occlusal edges: edges are established between corresponding nodes for corresponding teeth in the upper and lower jaws that have occlusal contact relationships. Then, a case-level attention network can use a graph neural network to extract features from the tooth feature map structure, obtaining the full-oral contextual features of each tooth.

[0048] It should be noted that in full-mouth panoramic imaging, adjacent teeth share the same contiguous region. For example, the distal surface of tooth FDI number 14 and the mesial surface of tooth FDI number 15 are closely connected in the image, and the image content of this shared region is reflected in the feature vectors of both teeth. Therefore, the feature vectors of these two teeth naturally exhibit high correlation among the features extracted from the backbone network. In self-attention calculation, highly correlated feature vectors generate larger inner product values, thereby increasing the attention weight between adjacent teeth.

[0049] S106: Based on the trained recognition network, the first image features and full-mouth context features are recognized to obtain the caries recognition result.

[0050] In this embodiment, the first image features and whole-mouth context features corresponding to each tooth can be fused, and caries identification can be performed based on the fused features to obtain caries identification results. For example, the first image features and whole-mouth context features can be spliced ​​or fixedly weighted to obtain fused features. The caries identification results may include: tooth position, confidence level, and identification category.

[0051] In this embodiment, compared to existing technologies that only identify images of single teeth when performing caries identification, leading to low accuracy in complex scenarios such as interproximal caries and multiple caries, this application uses a trained tooth-level attention network to extract the first image features of a single tooth image, thereby accurately mining local refined features such as the shape, density, and surface texture of a single tooth. Furthermore, to ensure accurate identification of complex features such as interproximal caries and multiple caries, this application first extracts features from the whole-mouth image, obtaining whole-mouth image features. Based on the tooth localization results, the second image features of each tooth are obtained from the whole-mouth image features. These second image features originate from the global feature map of the whole-mouth image, encoding the spatial position information of each tooth within the whole-mouth range and its relative relationship with adjacent teeth, providing an informational foundation for modeling the relationships between teeth at the whole-mouth level. Then, based on a preset tooth position sorting rule, the second image features of each tooth are arranged into a tooth feature node sequence, ensuring that adjacent teeth in the sequence are aligned. The nodes at the location correspond to teeth that are adjacent or close to each other in anatomical position. Then, based on the trained case-level attention network, features are extracted from the sequence of tooth feature nodes to obtain the full-mouth context features of the whole-mouth image. The full-mouth context features of each tooth no longer only contain its own local information, but also integrate the information of all other teeth related to it from the global perspective of the whole mouth. Therefore, based on the trained recognition network, the first image features and the full-mouth context features are recognized to obtain the caries recognition result. It can achieve the synergistic fusion of local detail features and global context features, and improve the caries recognition accuracy in complex scenarios such as proximal caries and multiple caries.

[0052] In one example, a specific implementation of fusing the first image features and the full-mouth context features is as follows: The first image features and the full-mouth context features are input into the gated fusion subnetwork, and the fusion weights corresponding to the first image features and the full-mouth context features are calculated based on the gated fusion subnetwork.

[0053] In this embodiment, to dynamically adjust the fusion ratio of local refined features and global contextual features based on the actual characteristics of each tooth, a gated fusion sub-network is used to calculate the weights of the first image features and the whole-mouth contextual features to obtain fusion weights. Since the fusion weights are calculated in real time by the gated network based on the specific values ​​of the input features, rather than using a preset fixed fusion ratio, the network can adaptively adjust the fusion strategy when facing different clinical scenarios.

[0054] In one example, the first image feature and the whole-mouth context feature are extracted through two different network branches. The first image feature comes from a tooth-level attention network, and the whole-mouth context feature comes from a case-level attention network. Due to the structural differences and potential differences in output dimensions between the two network branches, the feature spaces of the two features may have different numerical distributions and scales. Directly fusing two features with different dimensions or significantly different distributions will lead to an unstable fusion process, which is detrimental to the learning of the gating network. Therefore, in order to unify the two features into the same feature space and eliminate scale differences, it is necessary to perform a linear mapping between the first image feature and the whole-mouth context feature. The formula for calculating the linear mapping is: in, Indicates the standard first image feature; A linear projection matrix representing the first image feature; Indicates the first image feature; The bias term represents the linear projection of the first image feature; Represents standard full-mouth context features; A linear projection matrix representing the full-context features; Represents the contextual features of the entire mouth; This represents the bias term for the linear projection of the full-mouth context features.

[0055] Then, the aligned first image features and full-mouth context features are concatenated, and the concatenation result is input into a gating network to calculate the fusion weights. The calculation formula is as follows: in, Indicates the fusion weights; This represents the fully connected weight matrix of the gated network; This represents the bias term of the gated network; This represents the Sigmoid activation function, which compresses the mapping result within any real number range to the interval between 0 and 1; This indicates that the first image feature is based on the standard. Features obtained by concatenating standard full-mouth context features.

[0056] Based on the fusion weight, the first image features and the full mouth context features are fused to obtain the fused features.

[0057] In this embodiment, standard first image features and standard full-mouth context features are fused based on fusion weights to obtain fused features, which can achieve an adaptive weighted combination of local refined information and global context information. Since the fusion weights are dynamically calculated based on the input content, the contribution ratio of the two types of information in the fused features can be automatically adjusted according to the specific clinical scenario. For isolated occlusal caries, the fused features are mainly composed of local features; for multiple proximal caries, the proportion of global context information in the fused features is significantly increased. The calculation formula for the fused features is as follows: in, Indicates fusion characteristics; Indicates the fusion weights; Indicates the standard first image feature; Represents standard full-mouth context features; This indicates an element-wise multiplication operation.

[0058] In one example, to ensure that local fine-grained information is not lost during the fusion process, the fused features are added to the standard first image features using residuals, and then layer normalization is performed to stabilize the feature distribution, even with the gate value. By assigning higher weights to global features in certain dimensions, information from the original local features can still be preserved in the final output through the residual path, avoiding the loss of local detail information due to over-reliance on global information. The calculation formula is as follows: (6) in, This represents the final output feature after residual connections and layer normalization; Indicates fusion characteristics; Indicates the standard first image feature; Presentation layer normalization operation.

[0059] Based on the recognition network, the fused features are identified to obtain the caries recognition results.

[0060] In this embodiment of the application, the final output feature vector after residual connection and layer normalization can be... The input is a classification network consisting of fully connected layers. After linear transformation and normalization using the Softmax function, the output is the probability distribution of the tooth belonging to each caries category. The category with the highest probability is output as the caries identification result for that tooth. Adaptive fusion of local and global features has been achieved through a gating fusion mechanism. The input features received by the classification network have higher discriminative power and scene adaptability. Therefore, it can make more accurate category judgments based on high-quality fused features, and the final output of dental caries recognition results has higher confidence and reliability.

[0061] In this embodiment, by inputting the first image feature and the whole-mouth context feature into the gated fusion sub-network, calculating the fusion weights corresponding to the first image feature and the whole-mouth context feature based on the gated fusion sub-network, and calculating the fusion feature based on the fusion weights, the contribution ratio of local fine information and global context information in the fusion feature can be dynamically and adaptively adjusted according to the actual feature content of each tooth. Therefore, by recognizing the fusion feature based on the recognition network, high-quality fusion feature input can be obtained in different clinical scenarios, enabling the recognition network to make category judgments based on the optimal combination of local and global information, thereby significantly improving the accuracy of caries recognition in various scenarios such as proximal caries and multiple caries.

[0062] In one example, to obtain the trained tooth-level attention network, the trained case-level attention network, and the trained recognition network, a model training step is included before step S102: Obtain a dental caries recognition training set, which includes multiple dental caries recognition training samples. Each dental caries recognition training sample includes: a full mouth image sample, an image sample of each tooth corresponding to the full mouth image, and a dental caries recognition result label for each tooth.

[0063] In this embodiment, to adapt to different caries identification task requirements, the caries identification result labels can adopt different annotation formats. In one example, the label can be a binary label, that is, indicating whether each tooth has caries, with a label value of 0 indicating a normal tooth and a label value of 1 indicating the presence of caries. In another example, the label can be a multi-class label, that is, further subdividing the type of caries, such as classifying caries into different categories such as pit and fissure caries, interproximal caries, and secondary caries. In yet another example, the label can also include caries severity grading information, such as superficial caries, moderate caries, and deep caries. Regardless of the label format used, the label is based on a single tooth as the annotation unit, with each full-mouth image corresponding to several annotation records, and each record corresponding to the classification label of one tooth.

[0064] In one example, to improve the diversity of training data and suppress model overfitting, data augmentation can be performed on the training samples during training. The data augmentation operation is performed online each time the training samples are read. By randomly selecting the augmentation method and generating augmented samples in real time, the samples encountered by the model in each round of training are somewhat different, thereby forcing the model to learn anatomical features that are essentially related to the diagnosis of dental caries, rather than relying on surface features under specific imaging conditions.

[0065] Specifically, data augmentation may include a combination of one or more of the following operations: Geometric transformation data augmentation includes: horizontal flip enhancement, which mirrors the standardized panoramic grayscale image along the vertical midline to expand data diversity by leveraging the physiological characteristics of left-right symmetry in the maxillofacial region and eliminating feature bias caused by positioning skew; vertical flip enhancement, which mirrors the image along the horizontal midline to enhance the model's robustness to vertical and horizontal changes, but this operation is not suitable for images containing text labels or orientation markers; and random rotation enhancement, which randomly rotates the image around its geometric center point at small angles, with the rotation angle limited within a safe threshold to avoid truncating key anatomical structures, used to simulate minute angular deviations in shooting positioning. The model is enhanced in several ways: First, rotational invariance is improved. Second, affine transformation enhancement involves using linear transformations to achieve a combination of rigid transformations (translation, scaling, and shearing) while maintaining the topological relationships of the anatomical structure. This is used to simulate imaging effects at different shooting distances and angles. Third, elastic deformation enhancement uses a smooth Gaussian random field to generate a local displacement field, applying nonlinear, non-rigid deformation to the image. This simulates the natural deformation of soft tissues and jawbones, improving the model's adaptability to local morphological changes. Fourth, mesh distortion enhancement involves dividing the image into a regular mesh, applying random, smooth displacements to the mesh nodes, and interpolating to achieve local flexible distortion. This simulates optical distortion of the equipment and shooting curvature deviation.

[0066] Grayscale and illumination enhancement: This includes grayscale jitter enhancement, which involves randomly adjusting the brightness and contrast parameters for single-channel grayscale images to simulate imaging differences between different devices and scanning parameters, eliminate model overfitting caused by differences in imaging conditions, and enhance the model's adaptability to different imaging conditions.

[0067] Noise and blur enhancement: This includes Gaussian blur enhancement, which uses a two-dimensional Gaussian kernel to perform convolution operations with the image to simulate low-quality imaging effects such as defocus and motion blur, forcing the model to learn global anatomical features and improving the ability to recognize low-resolution and blurred images; and Gaussian noise enhancement, which involves superimposing zero-mean Gaussian random noise onto a standardized grayscale image to simulate sensor noise and scanning quantization errors, achieving regularization constraints and improving robustness to low-quality, high-noise images.

[0068] Occlusion-related enhancements include random occlusion enhancements, which randomly select rectangular regions of interest in the image and fill them with the mean or fixed grayscale value. This is used to simulate interference scenarios such as local occlusion, metal artifacts, and foreign object occlusion, forcing the model to learn global anatomical features, reducing over-reliance on local regions, and improving the ability to process occluded and incomplete images.

[0069] In another example, to unify the format and numerical distribution of the input data and ensure that the data processing logic is consistent between the training and inference phases, the same standardized processing as in the application phase can be used to process the training set data.

[0070] Based on a tooth-level attention network, the predicted first image features of each tooth image sample are extracted; features are extracted from the whole-mouth image samples to obtain predicted whole-mouth image features, and based on the tooth localization results, the predicted second image features of each tooth are obtained from the predicted whole-mouth image features; based on a preset tooth position sorting rule, the predicted second image features of each tooth are arranged into a predicted tooth feature node sequence; based on a case-level attention network, features are extracted from the predicted tooth feature node sequence to obtain the predicted whole-mouth context features of the whole-mouth image; based on a recognition network, the predicted first image features and the predicted whole-mouth context features are recognized to obtain the predicted caries recognition result.

[0071] In this embodiment, the same forward propagation logic as the aforementioned inference stage is executed for each caries recognition training sample, that is, the caries recognition result is obtained by sequentially passing through the tooth-level attention network, the case-level attention network, and the recognition network.

[0072] The classification loss is calculated based on the predicted caries identification results and the caries identification result labels.

[0073] In this embodiment, classification loss refers to the loss function value that measures the degree of difference between the model's predicted result and the true label. By calculating the difference between the predicted result and the true label and using this difference as the loss value, the model can adjust the network parameters along the gradient direction through the backpropagation algorithm, gradually reduce the prediction error, and make the model output gradually approach the true label.

[0074] The regularization loss is calculated based on the parameters of the tooth-level attention network, the case-level attention network, and the recognition network.

[0075] In this embodiment, regularization loss refers to a loss term that imposes constraints on the magnitude of model parameters. Regularization loss punishes excessively large parameter values, thereby suppressing overfitting of the model to the training data and improving the model's generalization ability.

[0076] Based on classification loss and regularization loss, it is determined whether the preset training stopping condition is met. If not, the network parameters of at least one of the tooth-level attention network, case-level attention network, and recognition network are adjusted, and training continues using the caries recognition training set until the preset training stopping condition is met, resulting in the trained tooth-level attention network, the trained case-level attention network, and the trained recognition network.

[0077] In this embodiment, the preset training stopping condition can be: the current training epoch reaches the preset maximum training epoch, the performance metric on the validation set no longer improves in several consecutive training epochs, or the total loss value decreases below a preset threshold. When the preset training stopping condition is not met, the backpropagation algorithm is used, and the optimizer updates the network parameters based on the calculated gradients.

[0078] In one example, to constrain the semantic consistency between the first image features of a single tooth and the whole-mouth context features, and to avoid feature shifts in bi-branch feature extraction caused by structural differences and gradient update path separation between the tooth-level attention network and the case-level attention network, the loss value can also include a local-global feature consistency loss. This local-global feature consistency loss measures the degree of feature alignment between the first image features and the whole-mouth context features, ensuring that the local refined features of the same tooth and its enhanced feature representation in the whole-mouth context maintain similar semantic orientations in the feature space. Specifically, the cosine similarity between the first image features and the whole-mouth context features can be calculated, and the difference between this similarity and the ideal similarity (e.g., a similarity of 1) can be used as the local-global feature consistency loss. Therefore, when determining whether the preset training stopping condition is met, the method also includes: judging whether the preset training stopping condition is met based on the classification loss, regularization loss, and local-global feature consistency loss.

[0079] In this embodiment, by calculating the classification loss, the tooth-level attention network, case-level attention network, and recognition network can be jointly optimized end-to-end under the guidance of a unified supervision signal. The classification loss directly measures the difference between the predicted caries recognition result output by the model and the true label, driving the parameters of each network to be updated in the direction of reducing prediction error. This enables the tooth-level attention network to learn to extract local refined features with caries discrimination power, the case-level attention network to learn to model the spatial relationship between teeth at the whole mouth level and extract effective whole mouth context features, and the recognition network to learn to adaptively fuse the two types of features and make accurate category judgments. Furthermore, by calculating the regularization loss, numerical constraints are imposed on the parameters of the tooth-level attention network, the case-level attention network, and the recognition network to suppress the excessive growth of each network parameter during training, prevent model overfitting, and improve the model's generalization ability. Therefore, the trained tooth-level attention network, the trained case-level attention network, and the trained recognition network can maintain the accuracy of caries recognition on the training set while possessing good generalization performance and robustness.

[0080] Figure 2 A flowchart illustrating the second caries identification method provided in this application is shown. Figure 2 As shown above, in the above Figure 1Based on the illustrated embodiment, one specific implementation of step S102 is as follows: S201: Perform multi-layer convolution feature extraction on each tooth image to obtain multiple initial image features.

[0081] In this embodiment, the trained tooth-level attention network may include a multi-layer convolutional network, which extracts multi-layer convolutional features from each tooth image to obtain initial image features. ,in, Indicates the number of channels; Indicates the spatial height of the initial image features; This represents the spatial width of the initial image features.

[0082] In one example, an image of each tooth is fed into a pre-built multi-layer convolutional network, which can consist of multiple stacked convolutional modules. Each convolutional module may contain convolutional layers, batch normalization layers, and activation function layers. Pooling layers (such as max pooling or average pooling) can be inserted between the multiple convolutional modules to downsample the feature maps. Pooling operations can reduce the spatial size of the feature maps, reduce subsequent computation, and expand the effective receptive field of subsequent convolutional layers, enabling them to capture a wider range of contextual information.

[0083] S202: For each tooth image, spatial weights are assigned to the initial image features based on the spatial attention sub-network to obtain spatially weighted image features.

[0084] In this embodiment, a single tooth image contains multiple anatomical regions. The pits and fissures on the occlusal surface are common sites of pit and fissure caries, appearing as localized areas of decreased density. The proximal contact area is a common site of interproximal caries, appearing as a decreased density area extending along the contact surface into the tooth structure. However, the tooth image also includes areas such as the smooth surface of the crown, the cervical region, part of the root, and the surrounding alveolar bone. These areas have low direct correlation with caries diagnosis. If all regions in the entire image are given equal weight, information from irrelevant regions will reduce the discrimination signal of key regions, decreasing the efficiency of feature representation. Therefore, a spatial attention mechanism is needed to assign higher weights to key regions in the spatial dimension and suppress non-key regions.

[0085] In one example, to enable the spatial attention subnetwork to perform differentiated spatial attention on different morphological features of different anatomical regions such as the occlusal surface and proximal surface, step S202 obtains spatially weighted image features in the following specific way: Multiple sets of convolution kernels of different shapes are used to convolve the initial image features to generate multiple first initial spatial attention maps; the convolution kernels include: vertical rectangular convolution kernel, square convolution kernel, horizontal rectangular convolution kernel and elliptical convolution kernel.

[0086] In this embodiment, convolutional kernels of different shapes refer to two-dimensional convolutional filters with different height and width ratios. Convolutional kernels of different shapes have receptive fields of different shapes, which can specifically capture image features of different anatomical planes and orientations. Therefore, multiple sets of convolutional kernels of different shapes are used to convolve the initial image features to generate multiple first initial spatial attention maps. This can adapt to the spatial morphology of different anatomical regions of the tooth, cover the spatial features of high caries-prone areas such as proximal surfaces, occlusal surfaces, buccal surfaces, and cusps, and avoid feature extraction blind spots that exist with a single convolutional kernel.

[0087] In one example, the initial image features can be convolved using the four convolution kernels described above to generate four first initial spatial attention maps, the calculation formula of which is as follows: in, Let the k-th initial spatial attention map be , and its dimension be . ; This represents the convolution operation of the k-th convolution kernel, where each convolution kernel has a specific shape; Indicates initial image features; Indicates the group index number of the convolution kernel; This indicates the total number of convolution kernels.

[0088] For example, the size of a longitudinal rectangular convolutional kernel can be 5×16. The receptive field of this kernel extends along the long axis of the tooth (longitudinal direction), covering a narrow lateral width and a long longitudinal range. This shape design fits the radiographic features of proximal caries. Proximal caries typically begins at the proximal contact point and extends along the proximal surface towards the cervical region. Its density reduction area appears longitudinally along the long axis of the tooth on imaging. The longitudinal rectangular receptive field can extract features along this density gradient direction, effectively capturing the longitudinal density gradient features of the proximal contact area.

[0089] The square convolution kernel can be 16×16 in size, a shape designed to fit the radiographic features of pit and fissure caries on the occlusal surface. Pit and fissure caries on imaging manifest as reduced density in localized areas of the occlusal surface; the lesion area is relatively compact and lacks clear directionality. The square receptive field can completely cover the pit and fissure area, extracting regional texture and density statistical features without being affected by any specific directional bias.

[0090] The transverse rectangular convolution kernel can be 16×5 in size. This shape is designed to fit the radiographic features of certain caries lesions extending laterally along the cusp slope in the occlusal region. When the caries lesions extend laterally along the mesiodistal or buccal-lingual direction of the occlusal surface, the transverse rectangular receptive field can cover the range of density changes along this direction and capture the characteristics of lateral density changes.

[0091] An elliptical convolution kernel is a convolution kernel obtained by element-wise multiplication of a predefined elliptical binary mask matrix with a standard rectangular convolution kernel, resulting in an elliptical receptive field. The receptive field shape of this type of convolution kernel is approximately elliptical, and its major axis can be oriented along a specific direction. For example, an elliptical approximation convolution kernel with its major axis oriented along the tooth's major axis (longitudinal direction) can be used, giving its receptive field a longitudinally extending elliptical shape. Elliptical convolution kernels with their major axis oriented laterally or at other angles can also be used to adapt to different tooth anatomical orientations. This shape design further closely approximates the natural anatomical morphology of the proximal contact area. Under normal anatomical structure, the proximal contact area is not a regular rectangle but rather an approximately elliptical contact surface with a certain curvature. When proximal caries occurs, the caries lesion extends into the tooth along the contact surface, and its lesion outline also presents an arc shape. The elliptical receptive field can more accurately match the typical outline morphology of the proximal contact area and the proximal caries lesion, reducing interference from the background area and focusing more on the actual lesion area.

[0092] The first initial spatial attention map is generated by weighted summation of multiple initial spatial attention maps.

[0093] In the embodiments of this application, convolutional kernels of different shapes locate key regions from the perspective of different receptive field shapes. However, for a specific input tooth image, the importance of different receptive field shapes varies. For example, for a tooth image block mainly containing proximal surfaces, the spatial attention map generated by a longitudinally rectangular convolutional kernel may be more accurate and should be given higher weight. For a tooth image block mainly containing occlusal surfaces, the spatial attention map generated by a square convolutional kernel may be more accurate and should be given higher weight.

[0094] Therefore, by introducing learnable weight parameters, the network can automatically learn the importance of various receptive fields during training, achieving adaptive weighted fusion to obtain the first final spatial attention map. This first final spatial attention map integrates information from multiple first initial spatial attention maps, combining the advantages of receptive fields of different shapes. Its calculation formula is as follows: in, This represents the first final spatial attention map; This represents the learnable fusion weight parameters corresponding to the k-th initial spatial attention map; This represents the k-th initial spatial attention map; This represents the Sigmoid activation function.

[0095] The first final spatial attention map is multiplied element-wise with the initial image features to obtain spatially weighted image features.

[0096] In this embodiment, the first final spatial attention map identifies the importance of each spatial location in the initial image features. By performing element-wise multiplication of the first final spatial attention map with the initial image features, spatially weighted image features are obtained. This allows the feature vectors corresponding to key regions to be numerically amplified as a whole, making them dominant in subsequent channel attention processing and global pooling operations. The feature vectors corresponding to non-key regions are attenuated, and their contribution to the final feature representation is suppressed.

[0097] The formula for calculating spatially weighted image features is as follows: in, Indicates spatially weighted image features; Indicates initial image features; This represents the first final spatial attention map; This indicates an element-wise multiplication operation.

[0098] In one example, since the dimension of the first final spatial attention map is not equal to the channel dimension of the initial image features, before performing element-wise multiplication of the first final spatial attention map with the initial image features, it is necessary to broadcast the first final spatial attention map along the channel dimension to obtain an element-wise multiplication with the channel dimension of the initial image features.

[0099] In this embodiment, multiple sets of convolutional kernels of different shapes are used to convolve the initial image features, which can extract features from multiple angles simultaneously. This avoids the feature extraction blind spots that exist in single-shape convolutional kernels, enabling the spatial attention sub-network to generate spatial attention distributions suitable for different types of caries. This solves the problem that a single-shape receptive field cannot simultaneously adapt to the recognition needs of multiple types of caries. Then, by assigning learnable weight parameters to the output of each type of convolutional kernel, the network can automatically learn the optimal fusion ratio of various receptive fields during training based on the statistical characteristics of the training data and the actual task requirements. As a result, the final spatially weighted image features can perform differentiated spatial attention for different anatomical planes and different caries types, effectively improving the spatial localization accuracy of the first image features.

[0100] In one example, to enable the spatial attention subnetwork to perform differentiated spatial attention on different morphological features of different anatomical regions such as the occlusal surface and proximal surfaces, step S202 yields another specific implementation of spatially weighted image features as follows: Multiple sets of convolution kernels with different directions are used to convolve the initial image features to generate multiple second initial spatial attention maps; the convolution kernels include: horizontal convolution kernels, vertical convolution kernels and diagonal convolution kernels.

[0101] In the embodiments of the application, convolution kernels in different directions refer to two-dimensional convolutional filters with the same or similar size but different orientation angles. Convolution kernels in different directions have receptive fields extending along a specific direction, which can capture changes in image features along that direction. Due to differences in patient positioning during imaging, the natural curvature of the dental arch, and variations in individual tooth arrangement, the actual orientation of different teeth in the image may differ, and the tilt angle of the proximal contact area of ​​the same tooth may also differ in different individuals. Therefore, using multiple sets of convolution kernels in different directions to convolve the initial image features and generate multiple second initial spatial attention maps can accurately align the actual anatomical orientation of each tooth, ensuring that the receptive field is consistent with the main direction of the lesion area. This allows the final spatial attention distribution to dynamically match the current actual orientation of the tooth and the direction of lesion expansion.

[0102] It should be noted that a horizontal convolution kernel refers to a convolution kernel whose receptive field extends along the horizontal direction of the image, effectively capturing density change features along the horizontal direction; a vertical convolution kernel refers to a convolution kernel whose main response direction extends along the vertical direction of the image, effectively capturing density change features along the vertical direction; and a diagonal convolution kernel refers to a convolution kernel whose main response direction extends at a certain angle to both the horizontal and vertical axes of the image, effectively capturing density change features along that diagonal direction, such as 45 degrees or 135 degrees.

[0103] Although the direction of the convolution kernel is defined based on the image coordinate system, there is a relatively stable correspondence between the direction of the image coordinate system and the clinical anatomical direction because dental images usually follow certain anatomical alignment rules during cropping and standardization. For example, the vertical direction (longitudinal direction) of the image roughly corresponds to the long axis of the tooth, that is, the direction from the crown to the root apex, and the horizontal direction (lateral direction) of the image roughly corresponds to the mesiodistal direction or buccal-lingual direction of the tooth, that is, the direction perpendicular to the long axis of the tooth.

[0104] Therefore, based on the receptive field directions of the convolution kernels in different directions and the image coordinate system of the tooth image, it can be seen that: longitudinal convolution kernels are suitable for capturing the density gradient features of proximal caries extending along the long axis of the tooth towards the neck; transverse convolution kernels are suitable for capturing the caries features extending along the mesiodistal or buccal-lingual directions on the occlusal surface; and due to misalignment of teeth, deviation of shooting angle, or the natural tilting shape of the proximal contact area, the direction of expansion of proximal caries may not be strictly vertical, but rather exhibits a certain tilting angle. Oblique convolution kernels can match the density change features of this type of tilting direction.

[0105] Based on the initial image features, the weights corresponding to multiple second initial spatial attention maps are determined, and the multiple second initial spatial attention maps are weighted and summed based on the weights to generate a second final spatial attention map.

[0106] In this embodiment, the weights corresponding to multiple second initial spatial attention maps are determined based on the initial image features, and the multiple second initial spatial attention maps are weighted and summed based on the weights to generate a second final spatial attention map. Through the above processing, the network can dynamically calculate the fusion weights of the response maps in each direction according to the actual content of the current input image, realize directional adaptive spatial attention, and make the final spatial attention distribution accurately match the actual orientation of the current tooth and the direction of lesion expansion, thereby adapting to individual differences of different teeth.

[0107] In one example, the specific calculation process for determining the weights corresponding to multiple second initial spatial attention maps based on the initial image features is as follows: First, a global average pooling operation is performed on the initial image features to compress the spatial distribution information of each channel into a scalar, thereby obtaining a feature description vector of the same length as the number of channels. This vector summarizes the global feature statistics of the entire dental image. Then, this feature description vector is input into a lightweight orientation weight prediction network, which can consist of a fully connected layer to map the feature description vector into a set of raw weight scores equal to the number of orientation convolutional kernels. Next, the raw scores are normalized using the Softmax function to obtain the fusion weights corresponding to each orientation convolutional kernel. The sum of these weights is one, and the value of each weight falls between zero and one. Finally, this set of weights is used to perform a weighted summation on each second initial spatial attention map, and the values ​​are compressed using the Sigmoid activation function to generate the second final spatial attention map. In this process, the calculation of the orientation weights depends entirely on the content of the initial image features; different inputs will result in different weight assignments, thus achieving orientation adaptation based on global feature statistics.

[0108] In another example, weights can be predicted based on information from multiple initial second spatial attention maps. The specific calculation process is as follows: First, the initial second spatial attention maps are concatenated along the channel dimension to form a multi-channel concatenated feature map. This feature map preserves the response intensity information of each spatial location in different directions. Next, the concatenated feature map is input into a weight prediction network (e.g., composed of single-layer convolutional layers with a 1x1 kernel size), outputting a weight score map with the same number of channels as the number of orientation groups. Then, the weight score map is normalized using the Softmax function along the channel dimension to obtain a spatially adaptive orientation weight map. In this weight map, each spatial location has an independent set of orientation weights, and the sum of the weights in different directions at the same location is one. Finally, using this spatially adaptive orientation weight map, the response values ​​of each initial second spatial attention map at the corresponding spatial location are weighted and summed point-by-point, and then normalized using the Sigmoid activation function to generate the final second spatial attention map. This approach achieves spatial location-level orientation adaptation, enabling the network to automatically allocate the optimal combination of orientation weights based on the texture orientation of different anatomical regions in dental images. For example, it may emphasize longitudinal response in the proximal region and lateral or oblique response in the occlusal region, thereby more precisely matching the actual expansion direction of the lesion.

[0109] The second final spatial attention map is multiplied element-wise with the initial image features to obtain spatially weighted image features.

[0110] In this embodiment, the second final spatial attention map is generated through a direction-adaptive fusion mechanism. Since the direction weights in this fusion mechanism are dynamically calculated based on the actual content of the input tooth image, rather than using preset fixed weight parameters, the spatial weight distribution of the second final spatial attention map can more accurately match the actual directional orientation of the current tooth lesion region. Performing element-wise multiplication of the second final spatial attention map with the initial image features yields spatially weighted image features. This selectively enhances the image feature responses along the main direction of lesion expansion in the initial image features, while effectively suppressing the feature responses of background regions and non-lesion regions that are unrelated to or deviate from this direction. Consequently, the spatially weighted image features significantly reduce feature redundancy and noise interference caused by directional mismatch regions while preserving the complete information of the lesion region.

[0111] In this embodiment, multiple sets of convolutional kernels with different directions are used to convolve the initial image features. This allows for the simultaneous extraction of features from multiple directions, avoiding the problem that a single-direction convolutional kernel cannot adapt to the actual orientation differences of different teeth in the image due to its fixed receptive field orientation. Then, by determining the weights corresponding to multiple second initial spatial attention maps based on the initial image features, the allocation of fusion weights can be dynamically determined by the actual content of the input image. This allows the final spatially weighted image features to accurately match the actual orientation of the current tooth lesion area. That is, the image feature response along the main direction of lesion expansion is selectively amplified, while the feature response of the mismatched area is effectively suppressed.

[0112] It should be noted that the convolution kernels of different shapes and directions mentioned above can be processed in parallel, serially, or in combination with a serial-parallel approach to perform convolution operations in the feature extraction process of the spatial attention subnetwork. At the same time, convolution kernels of different shapes and directions can also be mixed and matched according to the actual feature distribution requirements of the tooth images to flexibly adapt to the feature extraction of tooth regions with different tooth positions, different caries morphologies, and different texture directions, further improving the adaptability and accuracy of spatial attention weighting.

[0113] S203: Based on the channel attention subnetwork, channel weights are assigned to spatially weighted image features to obtain the first image features of each tooth image.

[0114] In this embodiment, the channel attention subnetwork refers to the subnetwork in the tooth-level attention network that is responsible for filtering important feature channels in the channel dimension. It can optimize the feature channel weight allocation for the proximal and occlusal surfaces where caries are prone to occur, automatically strengthen the effective feature channels related to tooth morphology, density, and surface texture, weaken the interference of redundant and ineffective channels, and make the channel features highly matched with caries identification.

[0115] In one example, the specific structure of the channel attention subnetwork can be: Perform a global average pooling operation on each channel of the spatially weighted image features to obtain the channel description vector: in, Represents the channel description vector; Indicates spatially weighted image features; This indicates a global average pooling operation.

[0116] Bottleneck fully connected mapping involves sequentially passing the channel description vector through a dimensionality-reducing fully connected layer, a ReLU activation function, a dimensionality-increasing fully connected layer, and a Sigmoid activation function to output the channel attention weights. in, Indicates channel attention weights; This represents the weight matrix of the dimension-reduced fully connected layer; This represents the weight matrix of the upgraded fully connected layer. Indicates the number of channels; This represents the hyperparameter indicating the dimensionality reduction ratio; Represents the channel description vector; Represents the ReLU activation function; This represents the Sigmoid activation function.

[0117] The channel attention weights are broadcast along the spatial dimension to make them equal to the number of channels in the spatially weighted image features. Then, a principal component multiplication operation is performed with the spatially weighted image features to obtain the channel-weighted image features. in, Indicates channel-weighted image features; Indicates spatially weighted image features; This represents the channel attention weight.

[0118] Finally, the channel-weighted image features are subjected to global average pooling and flattened to obtain the first image features: in, Indicates the first image feature; Indicates channel-weighted image features; Indicates global average pooling; This indicates a flattening operation.

[0119] In this embodiment, when extracting the first image features of each tooth image, a spatial attention subnetwork and a channel attention subnetwork are cascaded. First, spatial attention weighting is performed, enabling the network to accurately locate key anatomical regions closely related to caries diagnosis, such as the occlusal surface and proximal contact area, within the initial image features. The network then selectively enhances the image feature responses of these regions while suppressing responses from background and non-critical regions. Next, channel attention weighting is performed, allowing the network to further filter out the feature channels most capable of discriminating caries, building upon the spatial feature enhancement. Channels responding to important feature patterns such as low-density caries lesions and blurred edges are enhanced, while redundant channels responding to normal tooth tissue or irrelevant textures are suppressed. This results in enhanced feature responses in key anatomical regions such as the occlusal surface and proximal contact area in the final extracted first image features, while simultaneously increasing the weight of discriminative feature channels related to caries diagnosis. This effectively improves the representation quality of caries-related features in individual tooth features.

[0120] Figure 3 A flowchart illustrating the third caries identification method provided in this application is shown. Figure 3 As shown above, in the above Figure 1 Based on the illustrated embodiment, before step S104, the following is further included: S301: Encode the position of each tooth to obtain a tooth position code; wherein the tooth position code is determined based on at least one of the tooth position numbering system, the quadrant in which the tooth is located, the relative position index of the tooth, the information of the upper and lower jaws, and the information of the left and right sides.

[0121] In this embodiment, tooth position encoding refers to a feature vector used to characterize the anatomical position information of teeth in the dental arch. This encoded vector explicitly carries the spatial positioning information of teeth, enabling the subsequent self-attention mechanism to consider the anatomical positional relationship between teeth when calculating the association weights between teeth, thereby superimposing the guiding effect of spatial proximity on top of content similarity.

[0122] Specifically, when the self-attention mechanism calculates the attention weight of the i-th tooth on the j-th tooth, it depends not only on the similarity of their visual features but also on the similarity of their positional encodings. Teeth that are anatomically adjacent have similar positional encoding representations, and their inner product values ​​are larger, thus giving adjacent teeth higher attention weights in the self-attention calculation. This mechanism guides the case-level attention network to focus biasedly on anatomically adjacent tooth pairs, effectively strengthening the feature associations between adjacent teeth.

[0123] In one example, the tooth numbering system refers to the internationally accepted FDI tooth numbering system, which uses two digits to represent each tooth. The tens digit indicates the quadrant in which the tooth is located, and the units digit indicates the tooth's position within that quadrant from the midline to the distal end. The quadrant in which a tooth is located refers to its position in the oral cavity, including the right maxillary quadrant, left maxillary quadrant, left mandibular quadrant, and right mandibular quadrant. The relative position index of a tooth refers to its sequential number along a certain direction in the dental arch, such as a number from the midline to the distal end or a continuous position index along the arc length of the dental arch. Maxillary / mandibular information refers to the jaw position information of the tooth, i.e., maxillary or mandibular. Lateral information refers to the left or right lateral information of the tooth, i.e., left or right side.

[0124] In one example, the encoding method can be determined using at least one of the following methods: Learnable embedding vector form: Each location attribute is treated as an independent categorical variable, and a learnable embedding vector is assigned to each possible attribute value. For example, for the FDI tooth position numbering system, an embedding matrix can be constructed, where the number of rows represents the maximum possible number of teeth, and the number of columns represents the preset encoding vector dimension. For each tooth, the corresponding embedding vector is obtained as the location code by looking up a table based on its FDI number. This embedding matrix, along with other model parameters, is optimized through backpropagation during network training, enabling the network to automatically learn the optimal location encoding representation according to the actual needs of the caries identification task.

[0125] One-hot encoding: This method converts the positional attributes of teeth into high-dimensional sparse vectors containing only zeros and ones. For example, for the quadrant where a tooth is located, a four-dimensional one-hot encoding can be used, corresponding to the four quadrants: right maxilla, left maxilla, left mandible, and right mandible. The dimension corresponding to the current tooth's quadrant has a value of one, while the other dimensions have values ​​of zero. For information on the maxilla and mandible and information on the left and right sides, two-dimensional one-hot encoding can be used respectively.

[0126] Sine-cosine coding form: For tooth relative position indices with a continuous sequential relationship, a combination of sine and cosine functions can be used to generate position codes.

[0127] S302: The tooth position code of each tooth is fused with the corresponding image features to obtain the first fused image feature for each tooth.

[0128] In this embodiment, the tooth position code of each tooth is fused with its corresponding image features to obtain the first fused image feature corresponding to each tooth. This not only preserves the visual semantic information of the original image, but also carries explicit anatomical position information, providing a complete information foundation for self-attention calculation.

[0129] In one example, the location encoding can be combined with image features through addition, splicing, mapping fusion, etc., to obtain the first fused image feature corresponding to the tooth.

[0130] S303: Based on the preset tooth position sorting rules, the features of the first fused image are arranged into a sequence of tooth feature nodes.

[0131] In this embodiment of the application, in order to further strengthen the spatial positional relationship constraints between teeth, although the first fused image features have already carried the anatomical position information of each tooth through positional coding injection, by arranging the first fused image features into an ordered sequence of tooth feature nodes according to the preset tooth position sorting rules, the nodes in adjacent positions in the sequence can also be anatomically adjacent to each other, thereby providing a sequence structure constraint that conforms to the direction of dental arch for the subsequent self-attention mechanism, which makes it easier for the case-level attention network to capture the adjacent tooth association relationship propagating along the direction of dental arch more efficiently.

[0132] In this embodiment, to further enhance the anatomical location association information and spatial arrangement features of the tooth feature node sequence, and to enable the whole-mouth context features extracted by the case-level attention network to more accurately fuse information of adjacent teeth in anatomical location, this embodiment performs position encoding on each tooth before arranging the image features of each tooth into a tooth feature node sequence. The tooth position encoding is then fused with the corresponding image features, thereby explicitly injecting the anatomical location information of the tooth in the dental arch into the second image features. Due to the injection of position encoding, the subsequent self-attention mechanism, when calculating the association weights between teeth, not only relies on the content similarity between image features but is also guided by the similarity of position encoding. Teeth with similar anatomical locations have similar position encoding representations, thus obtaining higher attention weights in the self-attention calculation. This allows the case-level attention network to adaptively strengthen the feature associations between adjacent teeth, effectively capturing the clinical pattern of interproximal caries occurring in pairs on the contact surfaces of adjacent teeth, and improving the recognition accuracy of complex scenarios such as interproximal caries and multiple caries.

[0133] Figure 4 A flowchart illustrating the fourth caries identification method provided in this application is shown. Figure 4 As shown above, in the above Figure 1 Based on the illustrated embodiment, before step S104, the following is further included: S401: Encode the occlusal status of each tooth to obtain an occlusal status code; wherein the occlusal status code is determined based on at least one of the following: the presence of the opposing tooth, the relative positional relationship of the opposing tooth, the degree of overlap with the opposing tooth, the change in occlusal support caused by missing teeth, and the state of tooth elongation or tilting.

[0134] In the embodiments of this application, the occurrence and development of dental caries are not only related to the horizontal position of teeth in the dental arch, but also closely related to the occlusal state between the upper and lower teeth. Occlusal state refers to the relationship between the upper and lower teeth during the occlusal contact process, including the tightness of the occlusal contact, the integrity of the occlusal correspondence, and changes in the occlusal relationship caused by tooth loss or abnormal position.

[0135] It's important to note that abnormal occlusion can significantly increase the risk of tooth decay. For example, when an opposing tooth is missing, the corresponding tooth may elongate due to the loss of occlusal contact. This elongated tooth creates an abnormal contact relationship with adjacent teeth, easily leading to food impaction and plaque retention, thus increasing the probability of interproximal caries. Similarly, when teeth tilt, the contact relationship between the tilted tooth and adjacent teeth changes, forming a triangular gap that is difficult to clean, also increasing the risk of caries. Therefore, visually encoding the occlusal status of each tooth can increase the accuracy of caries identification.

[0136] The presence or absence of the opposing tooth refers to whether a tooth actually exists in the position that has occlusal contact with the current tooth. This information can be represented as a binary flag, with a value of 1 indicating the presence of the opposing tooth and a value of 0 indicating the absence of the opposing tooth. Based on this binary flag, a corresponding encoding vector can be generated using a one-hot two-dimensional encoding method, i.e., the presence state is encoded as [1,0], and the absence state is encoded as [0,1].

[0137] The relative positional relationship of opposing teeth refers to the spatial relationship between the current tooth and its opposing tooth, including normal occlusal contact and abnormal malocclusion. In one example, one or more of the following indices can be calculated to represent the relative positional relationship of opposing teeth: Horizontal overlap ratio: the ratio of the horizontal overlap length of the two tooth bounding boxes to the width of the smaller bounding box; Vertical distance: the vertical distance between the center points of the two tooth bounding boxes, normalized to a preset range, reflecting the size of the occlusal gap in the vertical direction; Area overlap ratio: the ratio of the intersection area to the union area of ​​the two tooth bounding boxes, comprehensively reflecting the degree of overlap of the upper and lower teeth on the two-dimensional image plane. The calculation results of the above indices can be used as continuous values, and after normalization, directly as the dimensional components of the relative positional relationship encoding.

[0138] The degree of overlap with the opposing tooth refers to the extent to which the current tooth and its opposing tooth overlap in a full-mouth panoramic image. The magnitude of this overlap comprehensively reflects various factors, including the tightness of the occlusal contact, the vertical position of the tooth, and the shooting angle. In one example, this information can be obtained by calculating the pixel overlap between the current tooth region and its opposing tooth region in the image. Specifically, the segmentation masks of the current tooth and its opposing tooth are first obtained, and then the ratio of the intersection area of ​​the two masks to the area of ​​the current tooth mask is calculated. A higher degree of overlap indicates a tighter occlusal contact or that the tooth is closer to its normal vertical position. This ratio, as a continuous value, can be directly used as a dimension for overlapping degree encoding after normalization.

[0139] Changes in occlusal support due to missing teeth refer to the alteration of the occlusal support relationship of the remaining teeth caused by the loss of some teeth. For example, when a tooth is missing, its opposing tooth may elongate due to loss of occlusal contact, and adjacent teeth may tilt towards the gap. These cascading changes alter the occlusal state of multiple teeth, increasing the risk of secondary caries. The occlusal support changes caused by missing teeth can be encoded by constructing a propagation model of the effects of tooth loss. One approach is as follows: First, determine the location of the missing tooth in the entire dentition based on tooth positioning results. Then, using the missing tooth location as the center, propagate the influence weights along the dental arch and occlusal direction to adjacent and opposing teeth. The influence weights can be attenuated according to the distance from the missing tooth location; teeth closer to the missing tooth location are more significantly affected. For each existing tooth, the weighted sum of the effects of all missing teeth is used as a quantitative indicator of the occlusal support change. This indicator is then mapped into a fixed-dimensional encoding vector through a fully connected layer.

[0140] Tooth elongation or tilting refers to the state in which a tooth tilts towards the missing tooth or elongates towards the opposing tooth due to reasons such as the loss of adjacent teeth, the loss of opposing teeth, or periodontal disease. Specifically, for tilting, it can be determined by calculating the angle between the long axis of the tooth and the tangent of the normal dental arch curve at that position. The normal dental arch curve can be obtained by fitting the center points of all teeth in the mouth, and the long axis of the tooth can be extracted from the tooth segmentation mask through principal component analysis. When the angle exceeds a preset threshold, the tooth is determined to be tilted, and the angle value is used as a quantitative indicator of the degree of tilt. For elongation, it can be determined by comparing the difference between the height of the current tooth's cusp or occlusal surface and the corresponding height of the adjacent teeth. If the current tooth's vertical position is significantly higher than that of the adjacent teeth, and there is a missing opposing tooth, the tooth is determined to be elongated, and the height difference is used as a quantitative indicator of the degree of elongation. The tilt angle and elongation can be treated as continuous values, normalized, and directly used as dimensions for state coding.

[0141] S402: The occlusal state code of each tooth is fused with the corresponding image features to obtain the second fused image feature for each tooth.

[0142] In this embodiment, the occlusal state code of each tooth is fused with the corresponding image features to obtain a second fused image feature for each tooth. This retains the visual semantic information of the original image while also carrying explicit occlusal state information. This allows the network to not only model relationships based on the similarity of image content when calculating the association weights between teeth, but also to perceive the occlusal correspondence between the upper and lower jaw teeth and the potential risk associations caused by occlusal abnormalities. This enables the case-level attention network to explicitly capture the risk of secondary caries caused by occlusal abnormalities such as missing opposing teeth, tooth tilting, and tooth elongation, thereby improving the model's sensitivity to identifying caries related to occlusal abnormalities. The occlusal state code can be combined with image features through addition, splicing, mapping fusion, etc., to obtain the second fused image feature corresponding to the tooth.

[0143] S403: Based on the preset tooth position sorting rules, the features of the second fused image are arranged into a sequence of tooth feature nodes.

[0144] In this embodiment, to further enhance the expression of spatial topological relationships between teeth while preserving occlusal state information, the second fused image features are arranged into a sequence of tooth feature nodes according to a preset tooth position sorting rule. This allows for the establishment of clear constraints on the dental arch's direction of travel within the sequence, while simultaneously reflecting the occlusal correspondence between the maxillary and mandibular teeth in the sequence structure. The ordered arrangement ensures that nodes at adjacent positions in the sequence are anatomically close to each other, providing a clinically prior sequence structure basis for subsequent self-attention mechanisms to capture adjacent tooth relationships propagating along the dental arch direction. Simultaneously, since the second fused image features already carry occlusal state information, the case-level attention network can perceive the occlusal association between corresponding teeth in the maxilla and mandible while focusing on the same-maxillary adjacency relationship. This enables comprehensive modeling of both same-maxillary horizontal adjacency and transmaxillary vertical occlusal relationships across the entire mouth, effectively improving the ability of full-mouth contextual features to represent multi-dimensional interdental relationships.

[0145] In one example, to further strengthen the spatial positional constraints between teeth, tooth position encoding, occlusion state encoding, and corresponding image features can be fused simultaneously.

[0146] In this embodiment, by encoding the occlusal state of each tooth, the subsequent case-level attention network can explicitly perceive the occlusal correspondence between the upper and lower jaw teeth and the potential caries risk information caused by occlusal abnormalities such as missing opposing teeth, tooth tilting, and tooth elongation when calculating the inter-tooth association weights. Furthermore, by fusing the occlusal state code of each tooth with the corresponding image features, a second fused image feature corresponding to each tooth is obtained. According to a preset tooth position sorting rule, the second fused image feature is arranged into a tooth feature node sequence. This enables the case-level attention network to simultaneously perceive the horizontal adjacency relationship of the teeth in the same jaw and the vertical occlusal relationship of the upper and lower jaw teeth at the whole-mouth level. As a result, when a tooth is at a high risk of secondary caries due to occlusal abnormalities, this risk information can be transmitted to the feature representations of adjacent teeth and opposing teeth through a self-attention mechanism, thereby effectively improving the model's sensitivity and accuracy in identifying caries related to occlusal abnormalities.

[0147] Based on the dental caries identification method provided in the above embodiments, this application also provides a specific implementation of a dental caries identification device. Please refer to the following embodiments.

[0148] Figure 5 A schematic diagram of the structure of a dental caries identification device provided in an embodiment of this application is shown, as follows: Figure 5 As shown, the dental caries identification device 500 includes: The acquisition module 501 is used to acquire full-mouth images and images of each tooth corresponding to the full-mouth images; Extraction module 502 is used to extract the first image features of each tooth image based on the trained tooth-level attention network; The first feature extraction module 503 is used to extract features from the full oral cavity image to obtain the full oral cavity image features, and based on the tooth localization results, to obtain the second image features of each tooth from the full oral cavity image features. The sorting module 504 is used to arrange the second image features of each tooth into a tooth feature node sequence based on a preset tooth position sorting rule. The second feature extraction module 505 is used to extract features from the tooth feature node sequence based on the trained case-level attention network to obtain the full-mouth context features of the oral cavity full-mouth image. The recognition module 506 is used to recognize the first image features and the full mouth context features based on the trained recognition network to obtain the caries recognition result.

[0149] In one example, extraction module 502 includes: The feature extraction submodule is used to perform multi-layer convolutional feature extraction on each tooth image to obtain multiple initial image features; The spatial weight allocation submodule is used to allocate spatial weights to the initial image features for each tooth image based on the spatial attention subnetwork, so as to obtain spatially weighted image features. The channel weight allocation submodule is used to allocate channel weights to spatially weighted image features based on the channel attention subnetwork, so as to obtain the first image feature of each tooth image.

[0150] In one example, extraction module 502 includes: The convolution processing submodule is used to perform convolution processing on the initial image features using multiple sets of convolution kernels of different shapes to generate multiple first initial spatial attention maps; the convolution kernels include: vertical rectangular convolution kernel, square convolution kernel, horizontal rectangular convolution kernel and elliptical convolution kernel; The weighted summation submodule is used to perform weighted summation on multiple initial spatial attention maps to generate the first final spatial attention map; The multiplication submodule is used to perform element-wise multiplication of the first final spatial attention map with the initial image features to obtain spatially weighted image features.

[0151] In one example, extraction module 502 includes: The convolution processing submodule is also used to perform convolution processing on the initial image features using multiple sets of convolution kernels in different directions to generate multiple second initial spatial attention maps; the convolution kernels include: horizontal convolution kernels, vertical convolution kernels and diagonal convolution kernels; The weighted summation submodule is also used to determine the weights corresponding to multiple second initial spatial attention maps based on the initial image features, and to perform weighted summation on the multiple second initial spatial attention maps based on the weights to generate a second final spatial attention map; The multiplication submodule is also used to perform element-wise multiplication of the second final spatial attention map with the initial image features to obtain spatially weighted image features.

[0152] In one example, the tooth decay detection device 500 also includes: The coding module is used to encode the position of each tooth to obtain a tooth position code; wherein, the tooth position code is determined based on at least one of the tooth position numbering system, the quadrant in which the tooth is located, the relative position index of the tooth, the information of the upper and lower jaws, and the information of the left and right sides. The feature fusion module is used to fuse the tooth position code of each tooth with the corresponding image features to obtain the first fused image feature for each tooth. The arrangement module is used to arrange the features of the first fused image into a sequence of tooth feature nodes based on a preset tooth position sorting rule.

[0153] In one example, the tooth decay detection device 500 also includes: The coding module is also used to encode the occlusal state of each tooth to obtain an occlusal state code; wherein, the occlusal state code is determined based on at least one of the following: the presence of the opposing tooth, the relative positional relationship of the opposing tooth, the degree of overlap with the opposing tooth, the change in occlusal support caused by missing teeth, and the state of tooth elongation or tilting. The feature fusion module is also used to fuse the occlusal state code of each tooth with the corresponding image features to obtain the second fused image features corresponding to each tooth. The arrangement module is also used to arrange the second fused image features into a sequence of tooth feature nodes based on a preset tooth position sorting rule.

[0154] In one example, the tooth decay detection device 500 also includes: The linear mapping module is used to linearly map the first image features and the full-mouth context features to obtain the standard first image; The processing module is used to input the first image features and the full-mouth context features into the gated fusion sub-network, and calculate the fusion weights corresponding to the first image features and the full-mouth context features based on the gated fusion sub-network; The fusion module is also used to perform feature fusion on the first image features and the full-mouth context features based on the fusion weights to obtain fused features; The recognition module is also used to identify fused features based on the recognition network to obtain the caries recognition result.

[0155] In one example, the tooth decay detection device 500 also includes: The acquisition module is also used to acquire a dental caries recognition training set, which includes multiple dental caries recognition training samples. Each dental caries recognition training sample includes: a whole mouth image sample, an image sample of each tooth corresponding to the whole mouth image, and a dental caries recognition result label for each tooth. The extraction module is also used to extract the predicted first image features of each tooth image sample based on a tooth-level attention network. The first feature extraction module is also used to extract features from the whole mouth image samples to obtain predicted whole mouth image features, and based on the tooth localization results, to obtain the predicted second image features of each tooth from the predicted whole mouth image features. The sorting module is also used to arrange the predicted second image features of each tooth into a predicted tooth feature node sequence based on a preset tooth position sorting rule. The second feature extraction module is also used to extract features from the predicted tooth feature node sequence based on the case-level attention network to obtain the predicted full-mouth context features of the full-mouth image. The recognition module is also used to recognize the predicted first image features and the predicted whole mouth context features based on the recognition network to obtain the predicted caries recognition result; The calculation module is used to calculate the classification loss based on the predicted caries identification results and the caries identification result labels; The computation module is also used to calculate the regularization loss based on the parameters of the tooth-level attention network, the parameters of the case-level attention network, and the parameters of the recognition network. The training module is used to determine whether the preset training stopping condition is met based on the classification loss and regularization loss. If not, the network parameters of at least one of the tooth-level attention network, case-level attention network, and recognition network are adjusted, and training continues using the caries recognition training set until the preset training stopping condition is met, resulting in the trained tooth-level attention network, case-level attention network, and recognition network.

[0156] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application is shown. The electronic device 600 includes a processor 601 and a memory 602 storing computer program instructions.

[0157] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0158] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 602 may include removable or non-removable (or fixed) media, or memory 602 may be a non-volatile solid-state memory.

[0159] In one example, memory 602 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0160] Memory 602 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0161] The processor 601 implements a tooth decay identification method in the above-described embodiment by reading and executing computer program instructions stored in the memory 602.

[0162] In one example, the electronic device may also include a communication interface 603 and a bus 604. Wherein, as... Figure 6 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 604 and complete communication with each other.

[0163] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0164] Bus 604 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 604 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0165] In addition, in conjunction with the dental caries identification method in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the dental caries identification methods in the above embodiments.

[0166] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the tooth decay identification methods described above.

[0167] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0168] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0169] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0170] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0171] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and servers described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for identifying dental caries, characterized in that, include: Acquire full-mouth images and images of each tooth corresponding to the full-mouth images; Based on the trained tooth-level attention network, the first image features of each tooth image are extracted; Feature extraction is performed on the full-mouth image to obtain full-mouth image features, and based on the tooth localization results, second image features of each tooth are obtained from the full-mouth image features. Based on a preset tooth position sorting rule, the second image features of each tooth are arranged into a tooth feature node sequence; Based on the trained case-level attention network, feature extraction is performed on the tooth feature node sequence to obtain the full-mouth context features of the oral cavity full-mouth image; Based on the trained recognition network, the first image features and the full mouth context features are recognized to obtain the caries recognition result.

2. The method according to claim 1, characterized in that, The trained tooth-level attention network extracts the first image features of each tooth image, including: Multi-layer convolutional feature extraction is performed on each tooth image to obtain multiple initial image features; For each tooth image, spatial weights are assigned to the initial image features based on a spatial attention subnetwork to obtain spatially weighted image features; Based on the channel attention subnetwork, channel weights are assigned to the spatially weighted image features to obtain the first image features of each tooth image.

3. The method according to claim 2, characterized in that, For each tooth image, based on a spatial attention subnetwork, spatial weights are assigned to the initial image features to obtain spatially weighted image features, including: Multiple sets of convolution kernels of different shapes are used to convolve the initial image features to generate multiple first initial spatial attention maps; the convolution kernels include: vertical rectangular convolution kernels, square convolution kernels, horizontal rectangular convolution kernels, and elliptical convolution kernels; The multiple initial spatial attention maps are weighted and summed to generate a first final spatial attention map; The first final spatial attention map is multiplied element-wise with the initial image features to obtain the spatially weighted image features.

4. The method according to claim 2, characterized in that, For each tooth image, based on a spatial attention subnetwork, spatial weights are assigned to the initial image features to obtain spatially weighted image features, including: Multiple sets of convolution kernels with different directions are used to convolve the initial image features to generate multiple second initial spatial attention maps; the convolution kernels include: horizontal convolution kernels, vertical convolution kernels and diagonal convolution kernels; Based on the initial image features, the weights corresponding to the multiple second initial spatial attention maps are determined, and the multiple second initial spatial attention maps are weighted and summed based on the weights to generate a second final spatial attention map; The second final spatial attention map is multiplied element-wise with the initial image features to obtain the spatially weighted image features.

5. The method according to claim 1, characterized in that, Before arranging the image features of each tooth into a sequence of tooth feature nodes based on a preset tooth position sorting rule, the process also includes: Each tooth is coded to obtain a tooth position code; wherein the tooth position code is determined based on at least one of the following: tooth position numbering system, tooth quadrant, tooth relative position index, maxillary and mandibular information, and left and right side information; The tooth position code of each tooth is fused with the corresponding image features to obtain the first fused image feature corresponding to each tooth. Based on a preset tooth position sorting rule, the features of the first fused image are arranged into a sequence of tooth feature nodes.

6. The method according to claim 1, characterized in that, Before arranging the image features of each tooth into a sequence of tooth feature nodes based on a preset tooth position sorting rule, the process also includes: Each tooth is coded for its occlusal state to obtain an occlusal state code; wherein the occlusal state code is determined based on at least one of the following: the presence of the opposing tooth, the relative positional relationship of the opposing tooth, the degree of overlap with the opposing tooth, the change in occlusal support caused by missing teeth, and the state of tooth elongation or tilting. The occlusal state encoding of each tooth is fused with the corresponding image features to obtain the second fused image feature corresponding to each tooth. Based on the preset tooth position sorting rules, the features of the second fused image are arranged into a sequence of tooth feature nodes.

7. The method according to claim 1, characterized in that, Before obtaining the caries identification result by recognizing the first image features and the full-mouth context features based on the trained recognition network, the process further includes: The first image feature and the full-mouth context feature are input into the gated fusion sub-network, and the fusion weights corresponding to the first image feature and the full-mouth context feature are calculated based on the gated fusion sub-network. Based on the fusion weights, the first image features and the full-mouth context features are fused to obtain fused features; Based on the recognition network, the fused features are identified to obtain the dental caries recognition result.

8. The method according to claim 1, characterized in that, Before extracting the first image features of each tooth image based on the trained tooth-level attention network, the process further includes: Obtain a dental caries recognition training set, which includes multiple dental caries recognition training samples. Each dental caries recognition training sample includes: a whole mouth image sample, an image sample of each tooth corresponding to the whole mouth image, and a dental caries recognition result label for each tooth. Based on a tooth-level attention network, the predicted first image features of each tooth image sample are extracted. Feature extraction is performed on the oral cavity full mouth image samples to obtain predicted oral cavity full mouth image features, and based on the tooth localization results, predicted second image features of each tooth are obtained from the predicted oral cavity full mouth image features. Based on the preset tooth position sorting rules, the predicted second image features of each tooth are arranged into a predicted tooth feature node sequence; Based on the case-level attention network, feature extraction is performed on the predicted tooth feature node sequence to obtain the predicted whole-mouth context features of the oral cavity image; Based on the recognition network, the predicted first image features and the predicted full mouth context features are recognized to obtain the predicted caries recognition result; The classification loss is calculated based on the predicted caries identification results and the caries identification result labels. Based on the parameters of the tooth-level attention network, the parameters of the case-level attention network, and the parameters of the recognition network, the regularization loss is calculated; Based on the classification loss and the regularization loss, it is determined whether the preset training stopping condition is met; if not, the network parameters of at least one of the tooth-level attention network, the case-level attention network, and the recognition network are adjusted, and training continues using the caries recognition training set until the preset training stopping condition is met, thus obtaining the trained tooth-level attention network, the trained case-level attention network, and the trained recognition network.

9. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements a dental caries identification method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement a dental caries identification method as described in any one of claims 1-8.

Images