Dental caries identification method, device and equipment and storage medium

Abstract

The invention discloses a decayed tooth identification method and device, equipment and a storage medium. The method comprises the following steps: acquiring an oral image of a target object; performing feature extraction on the oral cavity image based on a feature extraction model to obtain multi-scale feature data corresponding to the oral cavity image; performing decayed tooth recognition on the multi-scale feature data through a decayed tooth recognition model to obtain a decayed tooth recognition result; the decayed tooth recognition model comprises a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model comprises a feature extraction module and a decision tree; performing feature extraction on the multi-scale feature data through a category prediction sub-model to obtain multi-dimensional feature data; inputting the multi-dimensional feature data into a decision tree to obtain category information of feature points in the oral cavity image; performing mask identification on the multi-scale feature data through a mask identification sub-model to obtain mask information corresponding to the oral cavity image; the decayed tooth recognition result is generated based on the category information and the mask information, and the decayed tooth recognition efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of oral medical technology, and in particular to a caries identification method, device, equipment and storage medium. Background Art

[0002] Dental caries, a common oral health problem, is the result of demineralization of tooth enamel due to acid produced by bacterial fermentation of dietary sugars. Untreated dental caries can lead to severe pain, infection, and tooth loss, so dental caries identification plays an important role in the diagnosis of oral problems.

[0003] Although traditional caries identification methods, such as visual-tactile inspection methods and radiographic imaging methods, are capable of identifying caries, they have the problems of low efficiency and accuracy in caries identification. Summary of the invention

[0004] The present invention provides a dental caries identification method, device, equipment and storage medium to solve the problems of low efficiency and low accuracy in dental caries identification.

[0005] According to one aspect of the present invention, a method for identifying dental caries is provided, the method comprising:

[0006] Acquire an oral image of a target subject;

[0007] Extract features from the oral image based on the feature extraction model to obtain multi-scale feature data corresponding to the oral image;

[0008] Through the caries recognition model, caries recognition is performed on the multi-scale feature data to obtain a caries recognition result; wherein the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree;

[0009] The multi-scale feature data is extracted by a feature extraction module to obtain multi-dimensional feature data; the multi-dimensional feature data is input into a decision tree to obtain category information of feature points in the oral image, the category information including caries category information;

[0010] The mask recognition sub-model is used to perform mask recognition on the multi-scale feature data to obtain the mask information corresponding to the oral image;

[0011] Generate caries recognition results based on category information and mask information.

[0012] According to another aspect of the present invention, a caries identification device is provided, the device comprising:

[0013] An oral image acquisition module, used to acquire an oral image of a target object;

[0014] A multi-scale feature data extraction module is used to extract features from oral images based on a feature extraction model to obtain multi-scale feature data corresponding to the oral images;

[0015] A caries recognition result determination module is used to perform caries recognition on multi-scale feature data through a caries recognition model to obtain a caries recognition result; wherein the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree;

[0016] The multi-scale feature data is extracted by a feature extraction module to obtain multi-dimensional feature data; the multi-dimensional feature data is input into a decision tree to obtain category information of feature points in the oral image, the category information including caries category information;

[0017] The mask recognition sub-model is used to perform mask recognition on the multi-scale feature data to obtain the mask information corresponding to the oral image;

[0018] Generate caries recognition results based on category information and mask information.

[0019] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0020] at least one processor; and

[0021] a memory communicatively connected to at least one processor; wherein,

[0022] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the caries identification method of any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the caries identification method of any embodiment of the present invention when executed.

[0024] The technical solution of the embodiment of the present invention provides data support for the analysis and processing of subsequent tasks by acquiring an oral image of a target object; extracts features from the oral image based on a feature extraction model to obtain multi-scale feature data corresponding to the oral image, and obtains feature data of different scales to determine local features and global features of the oral image; performs caries recognition on the multi-scale feature data through a caries recognition model to obtain a caries recognition result; and by performing caries recognition on the multi-scale feature data, recognition can be performed from local and global perspectives, thereby improving the accuracy of caries recognition; wherein the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree; the multi-scale feature data is identified by the category prediction sub-model; and the caries recognition result is obtained by performing caries recognition on the multi-scale feature data. Feature extraction is performed on multi-scale feature data to obtain multi-dimensional feature data. By extracting features from multi-scale feature data to obtain multi-dimensional feature data, feature data of different dimensions can be obtained, providing comprehensive data support for the analysis and processing of subsequent tasks; the multi-dimensional feature data is input into a decision tree to obtain category information of feature points in the oral image, and the category information includes caries category information; mask recognition is performed on the multi-scale feature data through a mask recognition sub-model to obtain mask information corresponding to the oral image; caries recognition results are generated based on the category information and mask information, and caries recognition results are generated based on the category information and mask information, thereby improving the accuracy of caries recognition, solving the problems of low efficiency and low accuracy in caries recognition, and improving the efficiency and accuracy of caries recognition.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 is a flow chart of a caries identification method provided in Embodiment 1 of the present invention;

[0028] Figure 2 is a structural schematic diagram of a caries identification model provided by an embodiment of the present invention;

[0029] Figure 3 A flowchart of performing caries recognition on multi-scale feature data based on a caries recognition model provided by an embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of the structure of a first prediction head provided by an embodiment of the present invention;

[0031] Figure 5 This is a flow chart of performing caries recognition on multi-scale feature data based on a caries recognition model provided by the second embodiment of the present invention;

[0032] Figure 6 is a structural schematic diagram of another caries recognition model provided by an embodiment of the present invention;

[0033] Figure 7 is a structural schematic diagram of a caries identification device provided in Embodiment 3 of the present invention;

[0034] Figure 8 It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Embodiment 1

[0038] Figure 1This is a flow chart of a caries identification method provided in the first embodiment of the present invention. This embodiment is applicable to the case of improving the efficiency and accuracy of caries identification. The method can be executed by a caries identification device. The caries identification device can be implemented in the form of hardware and / or software. The caries identification device can be configured in the electronic device provided in the embodiment of the present invention. The electronic device can be a server, a computer, a mobile terminal or a medical device, etc. For example, the mobile terminal can be a mobile phone, a tablet computer, etc. For example, the medical device can be a device equipped with a processor, such as an oral medical device, and the oral medical device can be equipped with an oral scanning device and / or an oral surgical instrument, etc. Figure 1 As shown, the method includes:

[0039] S110, obtaining an oral cavity image of the target object.

[0040] In this embodiment, the oral image can be understood as an image obtained by scanning the oral cavity through an oral scanning device, including multiple types of images, such as oral X-ray images and oral optical coherence tomography (OCT) images. Oral images can be used in oral medicine fields such as orthodontics, implantation, and restoration. Optionally, the oral scanning device can be an oral scanner. Optionally, the oral image can include a grayscale image. The oral image includes an image of the tooth area, an image of the jaw area, and an image of the background area, and the background area includes but is not limited to the gum area and the gum area. The oral image of the target object can be obtained by scanning the oral cavity of the target object in real time through an oral scanning device. The oral image of the target object can also be obtained by reading from a database, where historical oral images of multiple target objects are stored, and the target object's identity information is matched in the database. Under the condition that the identity information matches successfully, the corresponding historical oral image is read, and the corresponding historical oral image is determined as the oral image.

[0041] After acquiring the oral image of the target object, a preprocessing operation may be performed on the oral image, and the preprocessing operation may include one or more of image registration, labeling, noise removal, pixel value normalization, and image enhancement.

[0042] Since the oral image of the target object can be obtained at different locations through different oral scanning devices, even the oral images collected by the same oral scanning device may have differences in position or angle. In order to facilitate the analysis and processing of subsequent tasks, the oral image of the target object can be aligned to a standard coordinate system, and the oral image can be registered using an image registration algorithm. For example, a registration algorithm based on feature points is used to extract feature points of the oral image, and the correspondence between the feature points and the nearest point in the standard coordinate system is calculated to register the oral image to the standard coordinate system.

[0043] In order to clearly display the outline of the teeth in the oral image, the outline of the teeth in the oral image can be annotated. For example, the outline can be annotated through the outline annotation model. The oral image is input into the pre-trained outline annotation model for annotation. The outline of the tooth area is annotated through the contour line to obtain the oral outline annotated image. The tooth area and the jaw area are distinguished from the background area. The oral outline annotated image can be used for caries identification. The oral image can also be annotated by editing the script software. During the annotation process, it can be automatically annotated according to the historical oral images of the target object to improve the accuracy of the annotation.

[0044] In the process of oral image acquisition, factors such as noise interfere with the oral image. In order to improve the quality of oral images, the oral image can be denoised, and the oral image can be denoised by a filter. Taking the oral image as an oral X-ray image as an example, a median filter is used to remove the salt and pepper noise in the oral image. For each pixel point in the oral image, the median of the pixel values ​​corresponding to multiple pixels within a preset range of the pixel point is determined, and the median is used as the new pixel value of the pixel point. While removing the salt and pepper noise, the characteristics of the oral image are retained.

[0045] The pixel values ​​of the oral image are normalized to a preset range. The pixel values ​​of the oral image can be normalized to the range of [0,1] or [-1,1]. It can be set according to the needs and is not limited here. Taking the normalization of the pixel values ​​of the oral image to the range of [0,1] as an example, the maximum value, minimum value and the difference between the maximum value and the minimum value of all pixel values ​​in the oral image are calculated, and the pixel value of each pixel point in the oral image is subtracted from the minimum value, and then divided by the difference between the maximum value and the minimum value. All pixel values ​​in the oral image can be normalized to the range of [0,1].

[0046] When performing image enhancement operations on oral images, a histogram equalization method can be used to divide the oral images according to pixel values, and the areas with pixel values ​​greater than the first grayscale threshold are divided into tooth areas, the areas with pixel values ​​lower than the second grayscale threshold are divided into background areas, and the areas with pixel values ​​between the second grayscale threshold and the first grayscale threshold are divided into jaw areas. The tooth areas, jaw areas and background areas in the oral images are distinguished. For example, the image of the background area can be set to pure black, so that the grayscale value difference between the images of the tooth areas, jaw areas and background areas in the oral images is increased.

[0047] S120 , extracting features of the oral image based on the feature extraction model to obtain multi-scale feature data corresponding to the oral image.

[0048] In this embodiment, the feature extraction model may include a machine learning model and a deep learning model, for example, a convolutional neural network model and a transformer model, for extracting features from oral images. Optionally, the feature extraction model may be a feature pyramid network model. The feature pyramid network model may include one or more feature pyramid network modules, different feature pyramid network modules output feature data of different scales, and feature data of different scales output by at least one feature pyramid network module form multi-scale feature data.

[0049] The feature extraction model may include a feature pyramid network module, and may also include three feature pyramid network modules, which can be set according to needs and are not limited here. Optionally, the feature pyramid network model includes five feature pyramid network modules. The five feature pyramid network modules are connected in sequence, and the five feature pyramid network modules are respectively the first feature pyramid network module, the second feature pyramid network module, the third feature pyramid network module, the fourth feature pyramid network module and the fifth feature pyramid network module from bottom to top. The feature extraction model is used to extract multi-scale feature data of oral images, wherein the multi-scale feature data can be understood as feature data of different scales in oral images. Optionally, the multi-scale feature data includes feature data of multiple scales. Taking the feature extraction model composed of five feature pyramid network modules as an example, the size of the oral image input to the feature extraction model is 256×256. After feature extraction by the first feature pyramid network module, feature data with a size of 128×128 can be obtained. After feature extraction by the second feature pyramid network module, feature data with a size of 64×64 can be obtained. After feature extraction by the third feature pyramid network module, feature data with a size of 32×32 can be obtained. After feature extraction by the fourth feature pyramid network module, feature data with a size of 16×16 can be obtained. After feature extraction by the fifth feature pyramid network module, feature data with a size of 8×8 can be obtained. Feature data with sizes of 128×128, 64×64, 32×32, 16×16 and 8×8 respectively constitute multi-scale feature data.

[0050] By extracting features from oral images through the feature extraction model, multi-scale feature data corresponding to the oral images can be obtained. Feature data at different levels can be obtained to provide comprehensive data support for subsequent tasks.

[0051] S130, performing caries recognition on the multi-scale feature data through a caries recognition model to obtain a caries recognition result.

[0052] The caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree. Figure 2and Figure 3 , Figure 2 is a structural schematic diagram of a caries recognition model provided by an embodiment of the present invention, Figure 3 FIG. 1 is a flow chart of performing caries recognition on multi-scale feature data based on a caries recognition model provided by an embodiment of the present invention. Figure 3 As shown, caries recognition is performed on multi-scale feature data based on a caries recognition model, including:

[0053] S131. Perform feature extraction on the multi-scale feature data through a feature extraction module to obtain multi-dimensional feature data, input the multi-dimensional feature data into a decision tree, and obtain category information of feature points in the oral image, wherein the category information includes caries category information.

[0054] S132. Perform mask recognition on the multi-scale feature data through the mask recognition sub-model to obtain mask information corresponding to the oral cavity image.

[0055] S133. Generate caries recognition results based on the category information and the mask information.

[0056] In this embodiment, the caries recognition model may include a machine learning model and a deep learning model, for example, a convolutional neural network model, which may be used to perform caries recognition on multi-scale feature data to obtain a caries recognition result. Optionally, the caries recognition result may include whether the teeth in the tooth area are caries; optionally, the caries recognition result may include the tooth type in the tooth area and whether the teeth are caries, the tooth type includes central incisors, lateral incisors, canines, premolars and molars, etc., the tooth area may include multiple tooth positions, different tooth positions correspond to different tooth types, and the tooth area is set with different marks according to the tooth position, for example, the marks of different tooth positions in the tooth area may be digital marks. Exemplarily, when the upper and lower jaws are not distinguished, the corresponding tooth positions of the upper and lower jaws may be set with the same tooth position mark, and each tooth position mark may be 1-16; Exemplarily, when the upper and lower jaws are distinguished, each tooth position in the upper and lower jaws is set with different tooth position marks, and each tooth position mark may be 1-32. The marks of different tooth positions in the tooth area may be color marks. Exemplarily, when the upper and lower jaws are not distinguished, the corresponding tooth positions of the upper and lower jaws may be set with the same tooth position mark, and each tooth position mark may be 16 different colors; Exemplarily, when the upper and lower jaws are distinguished, different tooth position marks are set for each tooth position in the upper and lower jaws, and each tooth position mark may be 32 different colors. The marks of different tooth positions in the tooth area may be numbers and color marks. Exemplarily, when the upper and lower jaws are not distinguished, the corresponding tooth positions of the upper and lower jaws may be set with the same tooth position mark, and each tooth position mark may be 1-16, and each number corresponds to a color, for example, 1 corresponds to red; Exemplarily, when the upper and lower jaws are distinguished, different tooth position marks are set for each tooth position in the upper and lower jaws, and each tooth position mark may be 1-32, corresponding to 32 different colors.

[0057] The category prediction submodel can be understood as a model that extracts and discriminates features from multi-scale feature data. The category prediction submodel includes a feature extraction module and a decision tree. The feature extraction module extracts features from the multi-scale feature data to obtain multi-dimensional feature data. The multi-dimensional feature data is input into the decision tree to obtain category information of feature points in the oral image. Multi-dimensional feature data can be understood as feature data of different dimensions. Optionally, the multi-dimensional feature data includes texture feature data and morphological feature data. Texture feature data may include texture features of the tooth surface and their corresponding grayscale values, and morphological feature data may include one or more items such as the size, shape and position of the teeth. The feature extraction module can be understood as a module that extracts features from multi-scale feature data. By extracting features from the multi-scale feature data, feature data of different dimensions are obtained, thereby improving the comprehensiveness of the feature data.

[0058] The decision tree can discriminate the input multi-dimensional feature data to obtain the category information of the feature points in the oral image. The category information can be understood as the information characterizing the category of the feature points in the oral image, and the feature points can be understood as any pixel points in the oral image, or the feature points can be pixels located within the tooth contour in the oral contour annotation image. Optionally, the category information includes caries category information. Optionally, the category information also includes one or more of jaw category information, tooth type information and gum status information. The jaw category information can be understood as the information characterizing the jaw type, and the jaw category information can include the maxilla and the mandible. The tooth type information can be understood as the information characterizing the tooth type, and the tooth types include central incisors, lateral incisors, canines, premolars and molars. The gum status information can include whether the gums are swollen and whether there is a significant gap between the gums and the teeth.

[0059] Optionally, the decision tree includes multiple layers of decision nodes, each layer of decision nodes classifies preset features through a preset threshold, and the leaf nodes of the decision tree correspond to caries category information respectively.

[0060] Specifically, a decision node can be understood as a node that classifies the features input to the decision node. A decision tree may include multiple layers of decision nodes, and multiple layers of decision nodes can distinguish a variety of different features until the leaf node obtains the category information corresponding to the feature point. For example, a decision tree may include N layers of decision nodes, distinguish N different features, and finally determine whether the feature point is caries through the leaf node to obtain the category information corresponding to the feature point. The features corresponding to the decision nodes at different levels are not limited here. Exemplarily, the multi-dimensional feature data is input to the root node, and the jaw type corresponding to the multi-dimensional feature data, for example, the feature point can be divided into maxillary teeth and mandibular teeth. At the second-layer decision node, the division can be based on the tooth maxillofacial morphology. For example, when the tooth maxillofacial morphology is narrow and slender, the feature point can be divided into anterior teeth, otherwise it is divided into posterior teeth. At the third-layer decision node, the division can be based on the grayscale value corresponding to the texture feature. For example, when the grayscale value corresponding to the texture feature is lower than the preset grayscale threshold, the feature point can be divided into a candidate caries, and when the grayscale value corresponding to the texture feature is greater than or equal to the preset grayscale threshold, it can be divided into a normal tooth. The candidate caries is input to the fourth-layer leaf node, and the candidate caries can be classified based on the morphological feature data. For example, if the morphological feature data does not meet the preset morphology, the feature point can be classified as caries. And so on, until the category information of the feature point in the oral image is obtained.

[0061] Optionally, the category prediction sub-model includes multiple first prediction heads, and each first prediction head extracts features from feature data of a scale.

[0062] Specifically, the prediction head can be understood as a module for extracting features from multi-scale feature data. The category prediction sub-model can include three first prediction heads, and can also include five first prediction heads, which are set according to needs and are not limited here. Each first prediction head extracts features from feature data of one scale. Taking feature data of four different scales as an example, the first scale feature data may include feature data of a size of 1 / 4 of the oral image, the second scale feature data may include feature data of a size of 1 / 8 of the oral image, the third scale feature data may include feature data of a size of 1 / 16 of the oral image, and the fourth scale feature data may include feature data of a size of 1 / 32 of the oral image. The number of prediction heads is the same as the size type of the multi-scale feature data. For example, the multi-scale feature data includes feature data of M different scales, and M first prediction heads can be used to extract features from the multi-scale feature data, and each first prediction head extracts features from feature data of one scale. Taking four prediction heads as an example, the first prediction head 1 extracts features from the first scale feature data, the first prediction head 2 extracts features from the second scale feature data, the first prediction head 3 extracts features from the third scale feature data, and the first prediction head 4 extracts features from the fourth scale feature data. The scales of the first scale feature data, the second scale feature data, the third scale feature data, and the fourth scale feature data may be different.

[0063] For example, the size of the oral image is 256×256, the first scale feature data is the feature data of 64×64, the second scale feature data is the feature data of 32×32, the third scale feature data is the feature data of 16×16, and the fourth scale feature data is the feature data of 8×8. Therefore, the first prediction head 1 can extract features from the first scale feature data of 64×64, the first prediction head 2 can extract features from the second scale feature data of 32×32, the first prediction head 3 can extract features from the third scale feature data of 16×16, and the first prediction head 4 can extract features from the fourth scale feature data of 64×64. Each prediction head can include a neural network model.

[0064] Optionally, any first prediction head includes multiple convolution blocks, and shortcut connections are arranged between input ends and output ends of a preset number of adjacent convolution blocks.

[0065] Specifically, each first prediction head may include three convolution blocks, or may include six convolution blocks, which are set according to requirements and are not limited here. For example, each first prediction head includes six convolution blocks, each convolution block includes the same structure, and each convolution block may include a 3x3 convolution layer, a 32-group group normalization layer, and a ReLU activation layer. The shortcut connection can be understood as splicing the input of a preset number of adjacent convolution blocks with the output of a preset number of adjacent convolution blocks. The input and output of a preset number of adjacent convolution blocks can be spliced. For example, a first shortcut connection can be set between the first convolution block and the second convolution block, a second shortcut connection can be set between the third convolution block and the fourth convolution block, and a third shortcut connection can be set between the fifth convolution block and the sixth convolution block. In order to keep the number of channels of the multi-scale feature data unchanged, the first shortcut connection between the first convolution block and the second convolution block can be set to a 3x3 convolution layer, and the shortcut connections between the remaining convolution blocks remain unchanged. For example, see Figure 4 , Figure 4 It is a structural schematic diagram of a first prediction head provided by an embodiment of the present invention. The mask recognition sub-model can be understood as a model for performing mask recognition on multi-scale feature data. By performing mask recognition on multi-scale feature data, mask information corresponding to the oral image can be obtained. Optionally, the mask recognition sub-model may include a machine learning model and a deep learning model. For example, the mask recognition sub-model may include a convolutional neural network model and a transformer model. The mask information can be understood as information characterizing the tooth area in the oral image. The multi-scale feature data can be converted into mask information in the oral image in the form of a binary image. For example, the tooth area in the multi-scale feature data can be set to 1, and the jaw area and the background area can be set to 0 to obtain the mask information in the oral image. For example, the mask information can be obtained by a differentiable rasterization method. The mask information may include multiple tooth positions, and different tooth positions correspond to different tooth types. Different marks are set in the tooth area according to the tooth position. For example, the marks of different tooth positions in the tooth area can be color marks. Exemplarily, when the upper and lower jaws are not distinguished, the corresponding tooth positions of the upper and lower jaws can be set with the same tooth position mark, and each tooth position mark can be 16 different colors; Exemplarily, when the upper and lower jaws are distinguished, different tooth position marks are set for each tooth position in the upper and lower jaws, and each tooth position mark can be 32 different colors. The marks of different tooth positions in the tooth area can be digital and color marks. Exemplarily, when the upper and lower jaws are not distinguished, the corresponding tooth positions of the upper and lower jaws can be set with the same tooth position mark, and each tooth position mark can be 1-16, and each number corresponds to a color, for example, 1 corresponds to red; Exemplarily, when the upper and lower jaws are distinguished, different tooth position marks are set for each tooth position in the upper and lower jaws, and each tooth position mark can be 1-32, corresponding to 32 different colors.

[0066] The caries recognition result can be generated based on the category information and mask information. The caries recognition result is obtained by fusing the information of the feature point category in the oral image with the mask information. For example, the category information and the mask information can be in matrix form respectively, and the target matrix corresponding to the caries recognition result is obtained by matrix multiplication, and the target matrix is ​​converted into an image form to obtain the caries recognition result.

[0067] Optionally, the method also includes: obtaining a caries recognition model to be trained, performing model training on the caries recognition model to be trained based on sample data, and obtaining a trained caries recognition model; wherein the target loss function in the training process includes a preset type of loss function and preset items of Taylor expansion of the loss function.

[0068] Specifically, the sample data can be understood as data for training the caries recognition model to be trained, and can include oral images of multiple target objects. The target loss function can be understood as a function that characterizes the training effect of the caries recognition model. The smaller the target loss function, the better the training effect of the caries recognition model. When the target loss function reaches a convergence state, the training of the caries recognition model can be terminated to obtain a trained caries recognition model. Alternatively, when the target loss function is less than or equal to a preset threshold, the training of the caries recognition model can be terminated to obtain a trained caries recognition model.

[0069] The training process of the caries recognition model includes: obtaining multimodal oral images as sample data; extracting multidimensional feature data of the sample data and obtaining classification labels corresponding to the sample data; performing feature extraction on the sample data based on the feature extraction model to obtain multi-scale feature data corresponding to the sample data; performing caries recognition on the multi-scale feature data corresponding to the sample data through the caries recognition model to be trained to obtain multi-dimensional prediction feature data output by the category prediction sub-model in the caries recognition model to be trained and caries prediction results output by the caries recognition model; generating a first loss function based on the multi-scale feature data and multi-dimensional prediction feature data of the sample data; generating a second loss function based on the classification labels and caries prediction results corresponding to the sample data; and performing model training on the caries recognition model to be trained based on the first loss function and / or the second loss function.

[0070] Specifically, multimodal oral images can be understood as images obtained by scanning the oral cavity using different oral scanning devices, and may include oral X-ray images and oral OCT images. Classification labels may include tooth type labels and jaw type labels. Multidimensional predicted feature data may be understood as feature data of different dimensions predicted in the process of caries recognition performed on the multi-scale feature data of sample data by the caries recognition model to be trained. The caries prediction result may be understood as the prediction result of caries recognition performed on the multi-scale feature data of the sample data, for example, the caries prediction result may include a probability of 80% that the maxillary premolar is carious. The first loss function may be understood as the difference between the multi-scale feature data and the multi-dimensional predicted feature data characterizing the sample data, which may be obtained by calculating the difference between the multi-scale feature data and the multi-dimensional predicted feature data. The second loss function may be understood as the difference between the classification label corresponding to the sample data and the caries prediction result, for example, which may be obtained by calculating the cross entropy loss function.

[0071] The caries recognition model to be trained can be trained by the first loss function. The smaller the first loss function is, the better the training effect of the caries recognition model is. When the first loss function reaches a convergence state, the training of the caries recognition model can be ended to obtain a trained caries recognition model. Alternatively, when the first loss function is less than or equal to a preset threshold, the training of the caries recognition model can be ended to obtain a trained caries recognition model. The caries recognition model to be trained can also be trained by the second loss function. The smaller the second loss function is, the better the training effect of the caries recognition model is. When the second loss function reaches a convergence state, the training of the caries recognition model can be ended to obtain a trained caries recognition model. Alternatively, when the second loss function is less than or equal to a preset threshold, the training of the caries recognition model can be ended to obtain a trained caries recognition model. Exemplarily, taking the focal loss function as the second loss function as an example, the calculation formulas of the second loss function and the second loss function based on Taylor expansion are as follows:

[0072] L FL =-(1-P t ) γ log(P t );

[0073]

[0074] Among them, P trepresents the probability of the caries recognition model predicting a positive sample, γ represents the weight between the positive and negative samples, the positive sample represents that the tooth is carious, and the negative sample represents that the tooth is normal. The caries recognition model to be trained can also be trained by the first loss function and the second loss function. For example, when both the first loss function and the second loss function reach a convergence state, the training of the caries recognition model can be terminated to obtain a trained caries recognition model.

[0075] The technical solution of this embodiment provides data support for the analysis and processing of subsequent tasks by acquiring the oral image of the target object; extracts features from the oral image based on the feature extraction model to obtain multi-scale feature data corresponding to the oral image, and obtains feature data of different scales to determine the local features and global features of the oral image; performs caries recognition on the multi-scale feature data through the caries recognition model to obtain caries recognition results, and by performing caries recognition on the multi-scale feature data, recognition can be performed from local and global perspectives, thereby improving the accuracy of caries recognition; wherein the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree; the multi-scale feature data is identified by the category prediction sub-model Feature extraction is performed on feature data to obtain multi-dimensional feature data. By extracting features from multi-scale feature data to obtain multi-dimensional feature data, feature data of different dimensions can be obtained, providing comprehensive data support for the analysis and processing of subsequent tasks; the multi-dimensional feature data is input into a decision tree to obtain category information of feature points in the oral image, and the category information includes caries category information; mask recognition is performed on the multi-scale feature data through a mask recognition sub-model to obtain mask information corresponding to the oral image; caries recognition results are generated based on the category information and mask information, and caries recognition results are generated based on the category information and mask information, which improves the accuracy of caries recognition, solves the problems of low efficiency and low accuracy in caries recognition, and improves the efficiency and accuracy of caries recognition.

[0076] Embodiment 2

[0077] Figure 5 The second embodiment of the present invention provides a flow chart of performing caries recognition on multi-scale feature data based on a caries recognition model. This embodiment is an optimization of the above embodiment, specifically, an optimization of S130 in the above embodiment. The caries recognition model includes a category prediction sub-model and a mask recognition sub-model. The category prediction sub-model includes a feature extraction module and a decision tree; the mask recognition sub-model includes a contour prediction branch module, a kernel prediction branch module, a feature prediction branch module, and a contour deformation module. Figure 5 As shown, the caries recognition model is used to perform caries recognition on the multi-scale feature data to obtain a caries recognition result, which includes the following steps:

[0078] S134. Perform feature extraction on the multi-scale feature data through a feature extraction module to obtain multi-dimensional feature data, input the multi-dimensional feature data into a decision tree, and obtain category information of feature points in the oral image, wherein the category information includes caries category information.

[0079] S135. Perform contour extraction on the multi-scale feature data through the contour prediction branch module to obtain an initial tooth contour.

[0080] S136. Extract offset information from the multi-scale feature data through the kernel prediction branch module and the feature prediction branch module to obtain an offset matrix.

[0081] S137, fusing the initial tooth contour and the offset matrix through a contour deformation module to obtain a target tooth contour, and converting the target tooth contour into mask information corresponding to the oral image.

[0082] S138. Generate caries recognition results based on the category information and mask information.

[0083] In this embodiment, the mask recognition sub-model includes a contour prediction branch module, a kernel prediction branch module, a feature prediction branch module and a contour deformation module, and the contour prediction branch module, the kernel prediction branch module and the feature prediction branch module are executed in parallel.

[0084] The contour prediction branch module can be understood as a module for contour extraction of multi-scale feature data. By performing contour extraction on multi-scale feature data, the initial tooth contour can be obtained. The contour prediction branch module may include multiple second prediction heads, and each second prediction head includes the same structure. The multiple second prediction heads in the contour prediction branch module and the multiple first prediction heads in the category prediction sub-model may have the same structure or different structures, and may be set according to requirements, without limitation here. Each second prediction head includes six convolution blocks and three shortcut connections, each convolution block includes the same structure, each convolution block may include a 3x3 convolution layer, a 32-group group normalization layer and a ReLU activation layer, a first shortcut connection is set between the first convolution block and the second convolution block, a second shortcut connection is set between the third convolution block and the fourth convolution block, a third shortcut connection is set between the fifth convolution block and the sixth convolution block, the first shortcut connection between the first convolution block and the second convolution block is set to a 3x3 convolution layer, and the shortcut connections between the remaining convolution blocks remain unchanged. Multiple second prediction heads can predict multiple points for the multi-scale feature data. For example, four sample points can be predicted for the multi-scale feature data, and the four sample points are used as the four vertices of the multi-scale feature data contour. In order to ensure that the four sample points are evenly distributed in the multi-scale feature data contour, the directions of the four sample points can be constrained. For example, the coordinate system where the multi-scale feature data is located is a standard coordinate system, and two sample points are constrained to have the same horizontal coordinate as the multi-scale feature data center. The two sample points are located on both sides of the multi-scale feature data center, and the straight line formed by the two sample points is parallel to the longitudinal axis of the standard coordinate system; the remaining two sample points are constrained to have the same longitudinal coordinate as the multi-scale feature data center. The two sample points are located on both sides of the multi-scale feature data center, and the straight line formed by the two sample points is parallel to the transverse axis of the standard coordinate system. The contour formed by the four sample points is determined as the initial tooth contour.

[0085] The kernel prediction branch module may include multiple third prediction heads, each of which includes the same structure. The multiple third prediction heads can process the multi-scale feature data to obtain multiple dynamic convolution kernels. The dynamic convolution kernels can extract the feature data of different teeth in the multi-scale feature data, so that each tooth can be distinguished from the remaining teeth. For example, the dynamic convolution kernel can be 1x1xD, where D represents the number of channels of the multi-scale feature data. The multiple third prediction heads in the kernel prediction branch module can have the same structure as the multiple second prediction heads in the contour prediction branch module and the multiple first prediction heads in the category prediction sub-model, or they can have different structures. They can be set according to needs and are not limited here. Each third prediction head includes six convolution blocks and three shortcut connections, each convolution block includes the same structure, each convolution block may include a 3x3 convolution layer, a group normalization layer of 32 groups and a ReLU activation layer, a first shortcut connection is set between the first convolution block and the second convolution block, a second shortcut connection is set between the third convolution block and the fourth convolution block, a third shortcut connection is set between the fifth convolution block and the sixth convolution block, the first shortcut connection between the first convolution block and the second convolution block is set to a 3x3 convolution layer, and the shortcut connections between the remaining convolution blocks remain unchanged.

[0086] The feature prediction branch module may include a 3×3 convolution layer, a 32-group normalization layer, and a ReLU activation function layer. The feature prediction branch module is used to scale the multi-scale feature data to obtain multiple uniform-scale dilated feature data, and determine the fusion features of the multiple uniform-scale dilated feature data. The dilated feature data may be understood as feature data of the same size as the input multi-scale feature data. For example, the multi-scale feature data may be automatically filled without changing the multi-scale feature data to obtain uniform-scale dilated feature data. The fusion feature may be understood as a feature obtained by fusing multiple uniform-scale dilated feature data. For example, four uniform-scale dilated feature data may be spliced ​​in the channel dimension to obtain a fusion feature of uniform-scale dilated feature data.

[0087] The offset information of the multi-scale feature data is extracted by the kernel prediction branch module and the feature prediction branch module to obtain an offset matrix. The offset information can be understood as information characterizing the offset of the initial tooth contour, which may include the offset direction and offset size of the initial tooth contour. The offset matrix can be understood as a matrix characterizing the offset information of the initial tooth contour. Optionally, the fusion feature is convolved by a dynamic convolution kernel to obtain an offset matrix.

[0088] The contour deformation module can be understood as a module for contour deformation of the initial tooth contour, and can include multiple contour deformation submodules. Exemplarily, the initial tooth contour and the offset matrix are input into the first contour deformation submodule, and the first contour deformation submodule determines the offset direction and offset size of the four points in the initial tooth contour according to the offset direction and offset size in the offset matrix, and then determines the first tooth contour. Based on the first tooth contour, a random interpolation method is used to insert k sample points on each edge of the new tooth contour. The number of sample points can be set according to requirements and is not limited here. The first tooth contour is input into the second contour deformation submodule, and based on the sample points in the first contour deformation submodule, a sample point is selected from each edge of the first tooth contour in a counterclockwise order, and a second tooth contour is formed based on the selected new sample points, and then input into the third contour deformation submodule, and based on the sample points in the second contour deformation submodule, a sample point is selected from each edge of the second tooth contour in a counterclockwise order, and so on, until the last contour deformation submodule generates a new tooth contour. The tooth contours generated by the multiple contour deformation submodules are iteratively processed to obtain the target tooth contour, thereby improving the accuracy of the target tooth contour.

[0089] For example, see Figure 6 , Figure 6It is a structural schematic diagram of another caries recognition model provided by an embodiment of the present invention. The caries recognition model includes a category prediction submodel, a contour prediction branch module, a kernel prediction branch module, a feature prediction branch module and a contour deformation module, wherein the category prediction submodel includes a first prediction head and a decision tree, the contour prediction branch module includes a second prediction head, the kernel prediction branch module includes a third prediction head, and the feature prediction branch module includes a convolution layer, a group normalization layer and an activation function layer. The prediction heads in the category prediction submodel, the contour prediction branch module and the kernel prediction branch module may include one or more prediction heads, as shown in the figure. Multi-scale feature data are respectively input into the first prediction head, the second prediction head, the third prediction head and the convolution layer, and the multi-scale feature data are feature extracted by the first prediction head to obtain multi-dimensional feature data, and the multi-dimensional feature data are input into the decision tree to obtain category information of feature points in the oral image. The multi-scale feature data are contour extracted by the second prediction head to obtain an initial tooth contour. The multi-scale feature data are processed by the third prediction head to obtain multiple dynamic convolution kernels. The multi-scale feature data is processed in turn through a convolution layer, a group normalization layer, and an activation function layer to obtain the fusion features of multiple uniform-scale dilated feature data. The fusion features are convolved through a dynamic convolution kernel to obtain an offset matrix. The offset matrix and the initial tooth contour are input into the contour deformation module for processing to obtain the target tooth contour. The target tooth contour is masked to obtain the mask information of the oral image, and the caries recognition result is obtained based on the category information and the mask information.

[0090] The technical solution of this embodiment provides data support for the analysis and processing of subsequent tasks by acquiring the oral image of the target object; extracts features from the oral image based on the feature extraction model to obtain multi-scale feature data corresponding to the oral image, and obtains feature data of different scales to determine the local and global features of the oral image; performs caries recognition on the multi-scale feature data through the caries recognition model to obtain caries recognition results, and by performing caries recognition on the multi-scale feature data, recognition can be performed from local and global perspectives, thereby improving the accuracy of caries recognition; wherein, the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree; the mask recognition sub-model includes a contour prediction branch module, a kernel prediction branch module, a feature prediction branch module and a contour deformation module; extracts features from the multi-scale feature data through the category prediction sub-model to obtain multi-dimensional feature data, and obtains multi-dimensional feature data by performing feature extraction on the multi-scale feature data, so that feature data of different dimensions can be obtained for the analysis and processing of subsequent tasks. The method provides comprehensive data support for the recognition of caries; the multi-dimensional feature data is input into the decision tree to obtain the category information of the feature points in the oral image, and the category information includes caries category information; the contour of the multi-scale feature data is extracted through the contour prediction branch module to obtain the initial tooth contour; the offset information of the multi-scale feature data is extracted through the kernel prediction branch module and the feature prediction branch module to obtain the offset matrix, and the offset matrix is ​​obtained by extracting the offset information of the multi-scale feature data, thereby reducing the amount of calculation to improve the efficiency of caries recognition; the initial tooth contour and the offset matrix are fused through the contour deformation module to obtain the target tooth contour, and the target tooth contour is converted into the mask information corresponding to the oral image, and the accurate target tooth contour can be obtained by fusing the initial tooth contour and the offset matrix, thereby increasing the accuracy of the mask information; the caries recognition result is generated based on the category information and the mask information, and the caries recognition result is generated according to the category information and the mask information, thereby improving the accuracy of caries recognition, solving the problems of low efficiency and low accuracy of caries recognition, and improving the efficiency and accuracy of caries recognition.

[0091] Embodiment 3

[0092] Figure 7 Schematic diagram of the structure of a dental caries identification device provided in Embodiment 3 of the present invention. Figure 7 As shown, the device comprises:

[0093] An oral image acquisition module 310 is used to acquire an oral image of a target object;

[0094] A multi-scale feature data extraction module 320 is used to extract features from the oral image based on the feature extraction model to obtain multi-scale feature data corresponding to the oral image;

[0095] The caries identification result determination module 330 is used to perform caries identification on the multi-scale feature data through the caries identification model to obtain the caries identification result; wherein the caries identification model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree; feature extraction is performed on the multi-scale feature data through the category prediction sub-model to obtain multi-dimensional feature data; the multi-dimensional feature data is input into the decision tree to obtain category information of feature points in the oral image, and the category information includes caries category information; mask recognition is performed on the multi-scale feature data through the mask recognition sub-model to obtain mask information corresponding to the oral image; and the caries identification result is generated based on the category information and the mask information.

[0096] Based on the above embodiment, optionally, the multi-scale feature data includes feature data at multiple scales.

[0097] Optionally, the decision tree includes multiple layers of decision nodes, each layer of decision nodes classifies preset features through a preset threshold, and the leaf nodes of the decision tree correspond to caries category information respectively.

[0098] Optionally, the category information also includes one or more of jaw category information, tooth type information and gum status information.

[0099] Optionally, the category prediction sub-model includes multiple prediction heads, and each prediction head performs feature extraction on feature data of a scale.

[0100] Optionally, any prediction head includes multiple convolution blocks, and shortcut connections are set between input ends and output ends of a preset number of adjacent convolution blocks.

[0101] Optionally, the mask recognition sub-model includes: a contour prediction branch module, a kernel prediction branch module, a feature prediction branch module and a contour deformation module.

[0102] Optionally, the caries identification result determination module 330 is also used to: extract the contour of the multi-scale feature data through the contour prediction branch module to obtain the initial tooth contour; extract the offset information of the multi-scale feature data through the kernel prediction branch module and the feature prediction branch module to obtain the offset matrix; fuse the initial tooth contour and the offset matrix through the contour deformation module to obtain the target tooth contour, and convert the target tooth contour into the mask information corresponding to the oral image.

[0103] Optionally, the caries identification result determination module 330 is also used to: process the multi-scale feature data through the kernel prediction branch module to obtain a dynamic convolution kernel; scale the multi-scale feature data through the feature prediction branch module to obtain multiple uniform-scale expansion feature data, and determine the fusion features of the multiple uniform-scale expansion feature data; and convolve the fusion features through the dynamic convolution kernel to obtain an offset matrix.

[0104] Optionally, the device also includes a caries recognition model training module, which is used to: obtain a caries recognition model to be trained, perform model training on the caries recognition model to be trained based on sample data, and obtain a trained caries recognition model; wherein the target loss function in the training process includes a preset type of loss function and preset items of the Taylor expansion of the loss function.

[0105] The caries identification device provided in the embodiment of the present invention can execute the caries identification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0106] Embodiment 4

[0107] Figure 8 1 is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0108] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0109] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0110] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the XX method.

[0111] In some embodiments, the caries identification method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the caries identification method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the caries identification method in any other appropriate manner (e.g., by means of firmware).

[0112] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] The computer program for implementing the caries identification method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0114] Embodiment 5

[0115] Embodiment 5 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute a caries identification method, the method comprising:

[0116] Acquire an oral image of a target object; extract features from the oral image based on a feature extraction model to obtain multi-scale feature data corresponding to the oral image; perform caries recognition on the multi-scale feature data through a caries recognition model to obtain a caries recognition result; wherein the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree; extract features from the multi-scale feature data through the category prediction sub-model to obtain multi-dimensional feature data; input the multi-dimensional feature data into the decision tree to obtain category information of feature points in the oral image, the category information including caries category information; perform mask recognition on the multi-scale feature data through a mask recognition sub-model to obtain mask information corresponding to the oral image; generate a caries recognition result based on the category information and the mask information.

[0117] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0119] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0120] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0121] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0122] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying dental caries, characterized in that: include: Acquire an oral image of a target subject; Extracting features from the oral image based on a feature extraction model to obtain multi-scale feature data corresponding to the oral image; Caries recognition is performed on the multi-scale feature data through a caries recognition model to obtain a caries recognition result; wherein the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree; Performing feature extraction on the multi-scale feature data by the feature extraction module to obtain multi-dimensional feature data; inputting the multi-dimensional feature data into the decision tree to obtain category information of feature points in the oral image, wherein the category information includes caries category information; Performing mask recognition on the multi-scale feature data through the mask recognition sub-model to obtain mask information corresponding to the oral image; The caries identification result is generated based on the category information and the mask information.

2. The method according to claim 1, characterized in that The multi-dimensional feature data includes texture feature data and morphological feature data; The decision tree includes multiple layers of decision nodes, each layer of decision nodes classifies preset features through a preset threshold, and the leaf nodes of the decision tree correspond to the dental caries category information respectively.

3. The method according to claim 1 or 2, characterized in that: The category information also includes one or more of jaw category information, tooth type information and gum status information.

4. The method according to claim 1, characterized in that: The multi-scale feature data includes feature data of multiple scales; The category prediction sub-model includes a plurality of prediction heads, each of which extracts features from feature data of a scale; Any of the prediction heads includes a plurality of convolution blocks, and shortcut connections are arranged between input ends and output ends of a preset number of adjacent convolution blocks.

5. The method according to claim 1, characterized in that The mask recognition sub-model includes: a contour prediction branch module, a kernel prediction branch module, a feature prediction branch module and a contour deformation module; Performing mask recognition on the multi-scale feature data through the mask recognition sub-model to obtain mask information corresponding to the oral image includes: Performing contour extraction on the multi-scale feature data through the contour prediction branch module to obtain an initial tooth contour; Extracting offset information from the multi-scale feature data through the kernel prediction branch module and the feature prediction branch module to obtain an offset matrix; The initial tooth contour and the offset matrix are fused through the contour deformation module to obtain a target tooth contour, and the target tooth contour is converted into mask information corresponding to the oral image.

6. The method according to claim 5, characterized in that The extracting offset information from the multi-scale feature data by the kernel prediction branch module and the feature prediction branch module to obtain an offset matrix includes: Processing the multi-scale feature data through the kernel prediction branch module to obtain a dynamic convolution kernel; Performing scale transformation on the multi-scale feature data through a feature prediction branch module to obtain a plurality of uniform-scale dilated feature data, and determining fusion features of the plurality of uniform-scale dilated feature data; The fusion feature is convolved using the dynamic convolution kernel to obtain the offset matrix.

7. The method according to claim 1, characterized in that The method further comprises: Acquire a caries recognition model to be trained, and perform model training on the caries recognition model to be trained based on sample data to obtain a trained caries recognition model; Among them, the target loss function in the training process includes a preset type of loss function and a preset item of the Taylor expansion of the loss function.

8. A dental caries identification device, characterized in that: include: An oral image acquisition module, used to acquire an oral image of a target object; A multi-scale feature data extraction module, used to extract features from the oral image based on a feature extraction model to obtain multi-scale feature data corresponding to the oral image; A caries recognition result determination module, used to perform caries recognition on the multi-scale feature data through a caries recognition model to obtain a caries recognition result; wherein the caries recognition model includes a category prediction sub-model and a mask recognition sub-model; the category prediction sub-model includes a feature extraction module and a decision tree; Performing feature extraction on the multi-scale feature data by the feature extraction module to obtain multi-dimensional feature data; inputting the multi-dimensional feature data into the decision tree to obtain category information of feature points in the oral image, wherein the category information includes caries category information; Performing mask recognition on the multi-scale feature data through the mask recognition sub-model to obtain mask information corresponding to the oral image; The caries identification result is generated based on the category information and the mask information.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the caries identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the dental caries identification method according to any one of claims 1 to 7 when executed.

Images