An oral cavity detection method, apparatus and device

Abstract

The present disclosure relates to an oral cavity detection method, device and equipment, comprising: obtaining a three-dimensional tooth model by scanning; detecting a target lesion area of the tooth and its confidence level based on an oral cavity detection model under a current detection view angle; back-projecting the target lesion area to the three-dimensional tooth model to obtain a current lesion position of the three-dimensional tooth model; judging whether the area overlap degree of the current lesion position and a historical lesion position is higher than a preset threshold; if yes, selecting the one with high confidence level as the target lesion position from the current lesion position and the historical lesion position, and updating the mark of the target lesion position; if no, determining the current lesion position as a new lesion position, and adding a mark of the new lesion position; determining a next detection view angle and performing the above operation again until the operation under the last detection view angle is completed, and obtaining multiple lesion positions of the three-dimensional tooth model. The present disclosure can improve the oral cavity detection efficiency and lesion marking accuracy.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent stomatology, and in particular to an oral cavity detection method, device and equipment. BACKGROUND

[0002] In the oral cavity detection process, a doctor needs to determine oral diseases according to his own experience, and manually mark a model using a recognized lesion site. This way consumes a lot of time and effort of the oral doctor, is low in efficiency, and there is a situation of inaccurate lesion site detection and marking. SUMMARY

[0003] To solve the above technical problems, the present disclosure provides an oral cavity detection method, device and equipment.

[0004] According to an aspect of the present disclosure, an oral cavity detection method is provided, comprising:

[0005] obtaining a three-dimensional tooth model through scanning;

[0006] detecting a target lesion area of the tooth and a confidence of the target lesion area based on a pre-trained oral cavity detection model under a current detection view;

[0007] back-projecting the target lesion area to the three-dimensional tooth model to obtain a current lesion position of the three-dimensional tooth model;

[0008] judging whether a region overlap degree of the current lesion position and a historical lesion position on the three-dimensional tooth model is higher than a preset threshold; wherein the historical lesion position is a lesion position obtained under a detection view before the current detection view;

[0009] if yes, selecting a high-confidence one as a target lesion position from among the current lesion position and the historical lesion position, and updating a mark of the target lesion position on the three-dimensional tooth model;

[0010] if no, determining the current lesion position as a new lesion position, and adding a mark of the new lesion position on the three-dimensional tooth model;

[0011] determining a next detection view and performing the above operations again until the operation under the last detection view is completed, to obtain a plurality of lesion positions of the three-dimensional tooth model.

[0012] According to another aspect of the present disclosure, an oral cavity detection device is provided, comprising:

[0013] a three-dimensional model obtaining module configured to obtain a three-dimensional tooth model through scanning;

[0014] a lesion area detection module, configured to detect a target lesion area of a tooth and a confidence level of the target lesion area based on a pre-trained oral cavity detection model under a current detection view angle;

[0015] a projection module, configured to project the target lesion area to the three-dimensional tooth model to obtain a current lesion position of the three-dimensional tooth model;

[0016] a judgment module, configured to judge whether a region overlap degree of the current lesion position and a historical lesion position on the three-dimensional tooth model is higher than a preset threshold, wherein the historical lesion position is a lesion position obtained under a detection view angle before the current detection view angle;

[0017] a mark update module, configured to, in a case of yes, select a target lesion position with a high confidence level from among the current lesion position and the historical lesion position, and update a mark of the target lesion position on the three-dimensional tooth model;

[0018] a mark addition module, configured to, in a case of no, determine that the current lesion position is a new lesion position, and add a mark of the new lesion position on the three-dimensional tooth model;

[0019] a repeated execution module, configured to determine a next detection view angle and execute the above operations again until the operation under a last detection view angle is completed, to obtain a plurality of lesion positions of the three-dimensional tooth model.

[0020] According to another aspect of the present disclosure, an electronic device is provided, which includes:

[0021] a processor;

[0022] a memory for storing executable instructions of the processor;

[0023] the processor is configured to read the executable instructions from the memory and execute the instructions to implement the above-mentioned oral cavity detection method.

[0024] The technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art:

[0025] The oral cavity detection method, device and equipment provided by the embodiments of the present disclosure comprise the following steps: obtaining a three-dimensional tooth model through scanning; detecting a target lesion area of the tooth and a confidence level of the target lesion area based on a pre-trained oral cavity detection model under a current detection perspective; back-projecting the target lesion area to the three-dimensional tooth model to obtain a current lesion position of the three-dimensional tooth model; determining whether the area overlap of the current lesion position and a historical lesion position on the three-dimensional tooth model is higher than a preset threshold; if yes, selecting the one with a higher confidence level as the target lesion position from the current lesion position and the historical lesion position, and updating the mark of the target lesion position on the three-dimensional tooth model; if no, determining the current lesion position as a new lesion position, and adding a mark of the new lesion position on the three-dimensional tooth model; determining a next detection perspective and performing the above operation again until the operation under the last detection perspective is completed, thereby obtaining a plurality of lesion positions of the three-dimensional tooth model.

[0026] The technical solution of the present disclosure can quickly and efficiently detect the target lesion area that may have a lesion by using the oral cavity detection model, shorten the time of visual examination, and improve the efficiency of the visual examination; the target lesion area is back-projected to the three-dimensional tooth model, and the lesion position is presented to the user in the form of three-dimensional perspective, so that the display effect of the lesion position is more intuitive; the area overlap and the confidence level of the current lesion position and the historical lesion position are used to dynamically update the lesion position with better quality or add a new lesion position on the three-dimensional tooth model, thereby improving the marking accuracy of the lesion position. Therefore, the present disclosure can improve the oral cavity detection efficiency and the marking accuracy of the lesion position, and make the display of the lesion position more intuitive. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0029] Figure 1 The oral cavity detection method flowchart described in the embodiments of the present disclosure;

[0030] Figure 2 The schematic diagram of the three-dimensional tooth model described in the embodiments of the present disclosure;

[0031] Figure 3 The schematic diagram of the scanning image of the tooth described in the embodiments of the present disclosure;

[0032] Figure 4A schematic diagram of a projection result according to an embodiment of the present disclosure;

[0033] Figure 5 A schematic diagram of a three-dimensional tooth model and multiple target scan images according to an embodiment of the present disclosure;

[0034] Figure 6 A structural block diagram of an oral cavity detection device according to an embodiment of the present disclosure;

[0035] Figure 7 A structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0037] In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other different manners than those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present disclosure, and not all the embodiments.

[0038] In the oral cavity detection process, the doctor usually judges and identifies the possible lesion site by means of imaging pictures and according to his own experience, and manually marks the identified lesion site on the three-dimensional tooth model. This process requires the doctor to manually establish the correspondence between the local imaging image and the three-dimensional tooth model, which will consume a lot of time and effort of the doctor and is low in efficiency. Moreover, the scheme of relying on traditional digital images and manual identification to select the lesion site requires high clinical experience of the doctor, and it is difficult to find early lesions with concealment, and there is a risk of misjudgment and omission in the identification process.

[0039] In view of the above problems, the present disclosure provides an oral cavity detection method, device and equipment, which quickly and efficiently detects the target lesion area where the lesion may occur through the oral cavity detection model, improves the oral cavity detection efficiency, dynamically updates the lesion position with better quality or adds a new lesion position on the three-dimensional tooth model, and improves the marking accuracy of the lesion position. For ease of understanding, the embodiments of the present disclosure are described below.

[0040] Figure 1 A flowchart of an oral cavity detection method provided by an embodiment of the present disclosure, which can be applied to the scene of checking lesions such as dental caries in the oral cavity. The method can be executed by an oral cavity detection device, which can be implemented by software and / or hardware and can be generally integrated in an electronic device. As shown in Figure 1 The oral cavity detection method comprises the following steps.

[0041] In step S102, a three-dimensional tooth model is obtained by scanning. In this embodiment, the upper teeth or the lower teeth of a human mouth can be globally scanned in real time by using an oral scanning device to obtain a three-dimensional tooth model of the whole teeth. For example, the teeth of the human mouth can be scanned in different scanning angles by using the oral scanning device to obtain a plurality of scanning images in real time, and a three-dimensional tooth model is generated in real time based on the plurality of scanning images. The three-dimensional tooth model can be referred to as shown in FIG. 1. The oral scanning device can be an intraoral scanner or an extraoral scanner. Figure 2

[0042] In step S104, the target lesion area of the teeth and the confidence of the target lesion area are detected based on a pre-trained oral detection model under the current detection angle. The oral detection model is trained by using image samples labeled with lesion areas, and the specific training process can refer to the prior art, which will not be described in detail here.

[0043] In actual application, the detection of the lesion area can be performed on the obtained scanning images after the scanning is completed. In this detection mode, the scanning and the detection are performed separately, so that the requirement for the computing performance of the device is relatively low. Alternatively, the detection of the lesion area can be performed on the obtained scanning images during the scanning. In this detection mode, the scanning and the detection are performed simultaneously, so that the oral detection efficiency can be improved, and this detection mode is suitable for a scenario with relatively high computing performance.

[0044] In a possible embodiment, first, a current scanning image of each group of teeth is obtained by scanning under a current detection angle. Each group of teeth includes at least one tooth. In this embodiment, the teeth can be locally scanned under different detection angles by using an oral scanning device. In the scanning, the coverage range of each local scanning is usually pre-set and can include at least one tooth. Therefore, in this embodiment, at least one tooth covered by each local scanning is taken as a group of teeth, and the scanning images of the groups of teeth are obtained by scanning. In actual scanning, a group of teeth can include a small number of teeth (for example, one tooth per group) to improve the scanning accuracy, or a group of teeth can include a large number of teeth (for example, three teeth per group) to improve the scanning efficiency.

[0045] For the same group of teeth, the oral scanning device is controlled to change the pose to scan the teeth under a plurality of different detection angles. In this way, the scanning images containing more local tooth features under different detection angles can be obtained, the information of the whole tooth can be observed in the form of a local two-dimensional plane, and the problem that the lesion site is missed due to the occlusion of the angle can be effectively avoided.

[0046] ​In one mode, the above-mentioned oral scanning device can be an intraoral scanner or an extraoral scanner containing near-infrared imaging technology, so that not only texture images but also infrared images, near-infrared images or fluorescence images can be acquired using the oral scanning device. Correspondingly, the current scanning image acquired by the current detection view of the present embodiment includes infrared images and texture images; for example, Figure 3 Infrared images and texture images corresponding to multiple groups of teeth in a detection view are shown.

[0047] Compared with the prior art using X-rays, which is high in cost, causes cumulative radiation to the human body and cannot provide local detailed information, the oral scanning device containing near-infrared imaging technology used in the present embodiment can reduce the use cost, is convenient and fast to operate, does not cause adverse effects on the human body during scanning, and more importantly, can acquire texture images and infrared images containing rich local information.

[0048] Next, the current scanning image is detected based on the pre-trained oral detection model to obtain a target lesion area and a confidence of the target lesion area.

[0049] For lesions such as dental caries, which can occur on the tooth surface or in the interdental space, based on this, the present embodiment detects the infrared image based on the pre-trained oral detection model to obtain a target lesion area of the interdental space; and detects the texture image based on the pre-trained oral detection model to obtain a target lesion area of the tooth surface.

[0050] For lesions occurring on the tooth surface, such as external caries, the caries features of the external caries can be observed through the texture image; therefore, the present embodiment uses the oral detection model to detect the tooth surface of the texture image. For lesions occurring in the interdental space, such as interproximal caries, it is difficult to observe the features thereof through the naked eye and the texture image, and it is difficult to use the texture image to detect such lesions in the interdental space. Considering the imaging characteristics of the near-infrared image, the enamel region appears clear and transparent, the dentin appears white and solid, and the interproximal lesion occurring in the enamel changes the transparency of the original enamel. Based on this, the present embodiment uses the color difference characteristics of the above-mentioned infrared image to use the oral detection model to detect the lesions in the interdental space of the infrared image.

[0051] The present embodiment detects lesions on the tooth surface based on the texture image and detects lesions in the interdental space based on the infrared image, which can more comprehensively detect lesions of different depths and levels occurring in the teeth, avoid missing lesion sites, and improve the comprehensiveness and accuracy of lesion site detection.

[0052] In an implementation manner, the oral cavity detection model is a deep learning network using a two-stage process of detection and segmentation, which can include a target detection network and a semantic segmentation network. The embodiment detects whether there is a lesion area in the current scan image through the oral cavity detection model, and outputs the target lesion area and its confidence in the case that there is a lesion area in the current scan image. Accordingly, the process of detecting the current scan image based on the oral cavity detection model can include:

[0053] The semantic segmentation network in the oral cavity detection model is used to perform pixel recognition on the current scan image to obtain an original lesion area, and then the target detection network in the oral cavity detection model is used to perform target detection on the original lesion area to obtain a target lesion area and a confidence of the target lesion area. The confidence is used to represent the probability of the target lesion area having a lesion. The embodiment detects the texture image and the infrared image based on the oral cavity detection model in the same way, that is, the current scan image includes one or more of the texture image, the infrared image, the near-infrared image, and the fluorescence image.

[0054] The above detection process first performs semantic segmentation, which can limit the detection area of the current scan image to the tooth area, effectively reducing the problem of misidentification outside the tooth area caused by saliva reflection and mirror reflection in the scanning process. Then, the original lesion area limited to the tooth area is subjected to target detection, which can improve the recognition accuracy of the target lesion area.

[0055] The embodiment uses deep learning technology to replace manual recognition, absorbs the rich clinical experience of oral physicians, and can efficiently and accurately identify tooth lesion problems such as caries, shorten the time of visual examination by physicians, improve the efficiency of physicians, and relieve the pressure on social medical resources.

[0056] In step S106, the target lesion area is back-projected to the three-dimensional tooth model to obtain the current lesion position of the three-dimensional tooth model.

[0057] After obtaining the target lesion area in the current scan image according to the above embodiment, the correspondence between the local current scan image and the global three-dimensional tooth model can be established by using the current scan image and the pose information of the oral cavity scanning device corresponding thereto. According to the correspondence, the two-dimensional target lesion area in the current scan image is automatically back-projected to the three-dimensional tooth model to obtain the current lesion position of the three-dimensional tooth model. Taking the infrared image as an example, the target lesion area detected by using the current infrared image can be back-projected to the three-dimensional tooth model according to the camera extrinsic parameters and the camera intrinsic parameters corresponding to the infrared image.

[0058] Reference Figure 4The projection result shown is an example in which multiple lesion positions are marked on the three-dimensional tooth model. In actual operation, different three-dimensional tooth models can be generated according to the actual situation of the patient's oral cavity, and other colors different from the color of the teeth are used to mark the lesion positions on the three-dimensional tooth model, so that the current lesion positions are presented in three-dimensional form on the three-dimensional tooth model, making the presentation effect more intuitive.

[0059] Step S108: determining whether the area overlap degree of the current lesion position and the historical lesion position on the three-dimensional tooth model is higher than a preset threshold; the historical lesion position is a lesion position obtained at a detection view angle before the current detection view angle.

[0060] Since multiple scanning images under multiple detection view angles need to be obtained for the same set of teeth, each detection view angle corresponds to a lesion position on the three-dimensional tooth model. Therefore, in order to determine the same lesion position from multiple detection view angles, the embodiment can determine whether the area overlap degree of the current lesion position and the historical lesion position on the three-dimensional tooth model is higher than a preset threshold. Specifically, the area overlap degree between the three-dimensional mesh patches corresponding to the current lesion position and the historical lesion position on the three-dimensional tooth model can be calculated, and it is determined whether the area overlap degree is higher than the preset threshold; the preset threshold is, for example, 80% of the sum of the areas of the three-dimensional mesh patches. If it is higher, it means that the current lesion position and the historical lesion position are highly coincident, and they are the same lesion position; in this case, the following step S110 is performed. On the contrary, if it is not higher, it means that the current lesion position and the historical lesion position have low coincidence, and they are two different lesion positions; in this case, the following step S112 is performed.

[0061] Step S110: selecting the lesion position with high confidence from the current lesion position and the historical lesion position as the target lesion position, and updating the mark of the target lesion position on the three-dimensional tooth model.

[0062] In a case where the area overlap of the current lesion position and the historical lesion position on the three-dimensional tooth model is higher than a preset threshold, the current lesion position and the historical lesion position are determined as the same lesion position, and a target lesion position that is more accurate and easier to observe is selected from the current lesion position and the historical lesion position according to the confidence. The higher the confidence, the higher the probability of the lesion position having a lesion. Therefore, in the embodiment, the target lesion position is the one with higher confidence from the current lesion position and the historical lesion position, and the mark of the target lesion position is updated on the three-dimensional tooth model. For example, if the confidence corresponding to the current lesion position is higher than the confidence corresponding to the historical lesion position, the historical lesion position originally marked on the three-dimensional tooth model is updated to the current lesion position. It can be understood that the mark of the same lesion position on the three-dimensional tooth model changes with the change of the detection view. In the embodiment, the target lesion position with high confidence is selected from the current lesion position and the historical lesion position, so that the lesion position with a better view and higher accuracy can be updated constantly, and finally a high-quality lesion position is retained.

[0063] In step S112, the current lesion position is determined as a new lesion position, and a mark of the new lesion position is added on the three-dimensional tooth model.

[0064] In a case where the area overlap of the current lesion position and the historical lesion position on the three-dimensional tooth model is not higher than a preset threshold, the current lesion position is determined as a new lesion position different from the historical lesion position, and the new lesion position is marked on the three-dimensional tooth model.

[0065] In step S114, the next detection view is determined, and the above operations are performed again until the operation under the last detection view is completed, and a plurality of lesion positions of the three-dimensional tooth model are obtained.

[0066] In the current detection view, the target lesion position with better quality is updated according to the above step S110, or the new lesion position is added according to the above step S112, that is, the detection process under the current detection view is completed. Then, the next detection view can be determined, the next detection view is taken as a new current detection view, and the operations shown in the above steps S104 to S114 are performed again.

[0067] It should be noted that the detection view refers to the shooting view of the scanner for obtaining each frame of scanning image such as infrared image and / or texture image during the scanning process as shown in Figure 3 The embodiment can determine the sequence of the detection views according to the time sequence of the scanning images obtained during the entire scanning process.

[0068] In actual application, the computer can screen a part of the detection angles from the above-mentioned shooting angles of the scanning images according to preset rules or parameters; the preset rules or parameters can be a series of angles set by the user according to actual detection requirements or preset angle intervals. The detection angles screened according to the preset rules or parameters in this embodiment can remove repeated angles or angles with small intervals that are difficult to distinguish from a large number of shooting angles, thereby reducing the data processing amount.

[0069] When the above-mentioned operation under the last detection angle is completed, a plurality of lesion positions of the three-dimensional tooth model are obtained, each of which is the lesion position with the highest quality obtained through selection and update as shown in step S110. Each tooth can correspond to zero, one or more lesion positions, and the lesion position can be on the tooth surface or the tooth gap. As shown in FIG. 6, the lesion positions on a plurality of teeth are marked, in which there are both lesion positions on the tooth surface and lesion positions on the tooth gap. Figure 4

[0070] This embodiment utilizes the plurality of lesion positions on the three-dimensional tooth model to present the lesion positions in the form of the three-dimensional model; from a global perspective, this can facilitate the doctor to globally locate the lesion positions and clearly understand the absolute position of each lesion on the tooth.

[0071] After obtaining the plurality of lesion positions of the three-dimensional tooth model, the method provided in this embodiment can further include: determining target teeth corresponding to each lesion position on the three-dimensional tooth model; and binding each lesion position with the tooth number of the corresponding target tooth according to the tooth number of each tooth on the three-dimensional tooth model that is marked in advance, to feed back each lesion position in the form of the tooth number.

[0072] Specifically, one lesion position can involve at least one target tooth, for example, the lesion position of interproximal caries occurring in the gap between two teeth, which corresponds to two adjacent target teeth. In actual application, in order to facilitate the distinction between the teeth, the teeth are generally marked with tooth numbers. Therefore, this embodiment can bind the lesion position with the tooth number of the corresponding at least one target tooth, so as to feed back the lesion position in the form of the tooth number, which is more intuitive and more convenient for the doctor to locate and diagnose. The above-mentioned feedback mode can be, for example, voice broadcast, generation of a text-shaped or drawing-shaped oral detection report, or positioning display or marking on the three-dimensional tooth model.

[0073] ​In order to make the user more clearly view the tooth lesion condition and local features, the embodiment can further include: determining the scanning image of the detected target lesion area as a candidate scanning image; and determining a first target scanning image from the multiple candidate scanning images corresponding to the same target lesion area under multiple detection perspectives according to a preset voting strategy; wherein the voting strategy includes at least one of the following: confidence, area proportion of the target lesion area in the candidate scanning image, and definition of the candidate scanning image.

[0074] In the embodiment, the confidence, the area proportion of the target lesion area in the candidate scanning image, and the definition of the candidate scanning image can all be used alone as the voting strategy to determine the first target scanning image from the multiple candidate scanning images.

[0075] In an example, the voting strategy is the confidence. For the multiple candidate scanning images corresponding to the same target lesion area under multiple detection perspectives, the first target scanning image can be selected from the candidate scanning images with the highest confidence as the final candidate scanning image displayed to the user. It can be understood that the detection perspectives corresponding to the multiple first target scanning images selected by the confidence are all optimal perspectives suitable for the user to view.

[0076] In an example, the voting strategy is the area proportion of the target lesion area in the candidate scanning image. For any candidate scanning image, the area proportion of the target lesion area in the candidate scanning image can be calculated. Generally, the larger the area proportion is, the clearer and easier to view the target lesion area in the candidate scanning image is; thus, the first target scanning image can be selected from the candidate scanning images with the largest area proportion as the final candidate scanning image displayed to the user. It can be understood that the target lesion area in the multiple first target scanning images selected by the area proportion is larger, which is convenient for the user to view more comprehensively and in detail.

[0077] In an example, the voting strategy is the definition of the candidate scanning image. The example can select the first target scanning image from the multiple candidate scanning images corresponding to the same target lesion area under multiple detection perspectives as the final candidate scanning image displayed to the user. It can be understood that the multiple first target scanning images selected by the definition of the candidate scanning image can more clearly display the local details and boundaries of the target lesion area and other key information.

[0078] In the embodiment, at least two of the confidence, the area proportion of the target lesion area in the candidate scanning image, and the definition of the candidate scanning image can be combined to form the voting strategy to determine the first target scanning image from the multiple candidate scanning images.

[0079] In an example, the first target scan image is determined according to the confidence and the area proportion of the target lesion region in the candidate scan image, from the multiple candidate scan images corresponding to the same target lesion region under multiple detection perspectives.

[0080] A specific manner is, for example, first selecting the top K1 first scan images with the highest confidence from the multiple candidate scan images, and then selecting the top K2 first scan images with the largest area proportion from the K1 first scan images as the first target scan images. The first target scan images determined according to the confidence and the area proportion can not only ensure a better observation perspective, but also facilitate the user to view the target lesion region more comprehensively and in detail.

[0081] In an example, the first target scan image is determined according to the confidence and the definition of the candidate scan image, from the multiple candidate scan images corresponding to the same target lesion region under multiple detection perspectives. The first target scan image determined by this embodiment can take into account both a better perspective and image definition.

[0082] In an example, the first target scan image is determined according to the area proportion of the target lesion region in the candidate scan image and the definition of the candidate scan image, from the multiple candidate scan images corresponding to the same target lesion region under multiple detection perspectives. In the first target scan image determined by this embodiment, the target lesion region not only has a large area but also has a high definition.

[0083] In an example, the first target scan image is determined according to the confidence, the area proportion of the target lesion region in the candidate scan image, and the definition of the candidate scan image, from the multiple candidate scan images corresponding to the same target lesion region under multiple detection perspectives. This embodiment can determine the first target scan image that satisfies the user in terms of perspective, area size, and definition by considering the confidence, area proportion, and definition together.

[0084] The first target scan images in the above embodiments include target infrared images and target texture images. When displaying the images, the multiple first target scan images can be displayed in a preset order, such as an order from high to low confidence, an order from large to small area proportion, or an order from high to low definition. In addition, other remaining candidate scan images can be viewed by clicking, scrolling, and dragging, to facilitate the doctor to observe the lesion imaging effect from different dimensions and local granularity. It can be understood that the multiple first target scan images can display the lesion region from different perspectives, and the first target scan images in the front order are preferred images more suitable for the doctor to view.

[0085] According to the above embodiment, the first target scan image is determined by using the voting strategy, the local image feature information of the lesion position can be highlighted, and the target near-infrared image and the target texture image containing each lesion position are further supplemented on the basis of the three-dimensional tooth model, so that the user can conveniently observe the single lesion position.

[0086] The embodiment can determine a high-quality first target scan image, and simultaneously display the three-dimensional tooth model indicating each lesion position and the first target scan image corresponding to each lesion position according to the lesion position of the target lesion region in the first target scan image after back projection on the three-dimensional tooth model.

[0087] With reference to Figure 5 , the three-dimensional tooth model and multiple first target scan images are displayed, different first target scan images correspond to different teeth. The first target scan image includes a target infrared image and a target texture image, and a rectangular frame is used to indicate the lesion position in each target infrared image. Meanwhile, the lesion position on the three-dimensional tooth model and the corresponding first target scan image are directly indicated by a connecting line. According to the embodiment, the lesion position between the scan image and the three-dimensional tooth model can be associated by back projecting the two-dimensional first target scan image to the three-dimensional tooth model, so that a clearer and more intuitive display effect is formed.

[0088] Considering that the detected lesion region in the scan image under some detection angles may be noise, based on this, in order to reduce the influence of noise, the following embodiment can be provided, including the following steps (1) to (4).

[0089] (1) For the same set of teeth, the target image quantity of multiple scan images under different detection angles in which the target lesion region is detected is determined.

[0090] It can be understood that, for the same set of teeth, if the lesion region is detected only in the scan images under a few detection angles, it indicates that the credibility of the lesion region is not high, and the noise may be caused by reasons such as saliva reflection. On the contrary, if the lesion region is detected in the scan images under multiple different detection angles, it indicates that the credibility of the lesion region is high, and the lesion region can be regarded as an effective target lesion region.

[0091] In this case, the target image quantity of the scan images in which the target lesion region is detected, that is, the quantity of the detection angles in which the target lesion region is detected, is used to remove the scan images that may be noise and leave more accurate and reliable scan images.

[0092] (2) In the case where the target image quantity is greater than a preset quantity value, a target confidence reference value corresponding to the target image quantity is determined according to the corresponding relationship between the image quantity and the confidence reference value.

[0093] In this embodiment, in the case that the target image quantity is greater than the preset quantity value (such as 10), it can be generally considered that the lesion region detected from the 10 or more scanning images is an effective and reliable target lesion region. On this basis, the embodiment can further combine the confidence of the target lesion region to further improve the accuracy.

[0094] The embodiment pre-sets the corresponding relationship between the image quantity and the confidence reference value. Since in actual application, the more the image quantity, the more scattered the confidence, and the greater the distribution gap, therefore, the corresponding relationship can be set as: the more the image quantity, the lower the confidence reference value corresponding thereto. For example, when the image quantity ranges from 10 to 15, the confidence reference value is 0.9; when the image quantity ranges from 15 to 20, the confidence reference value is 0.8; when the image quantity ranges from 20 to 25, the confidence reference value is 0.75.

[0095] According to the above corresponding relationship, the target confidence reference value corresponding to the target image quantity is determined.

[0096] (3) From the plurality of scanning images under different detection perspectives, at least one scanning image reaching the target confidence reference value is selected as a second target scanning image.

[0097] In specific implementation, one or more second target scanning images can be selected from the scanning images reaching the target confidence reference value according to the confidence, the area proportion of the target lesion region in the scanning image, and / or the definition of the scanning image, according to the foregoing embodiments.

[0098] (4) The target lesion region in the second target scanning image is back-projected to the three-dimensional tooth model.

[0099] Similar to the first target scanning image in the foregoing embodiments, the embodiment can also simultaneously display the three-dimensional tooth model in which each lesion position is marked and the second target scanning image corresponding to each lesion position according to the lesion positions of the target lesion region back-projected on the three-dimensional tooth model in the second target scanning image, and the display effect can be referred to Figure 5 .

[0100] In addition, when the images are displayed, the plurality of second target scanning images can be displayed in a preset order; in order to preferentially display the second target scanning image with better quality, the preset order can be, for example, the order from high to low confidence, the order from large to small area proportion, or the order from high to low definition.

[0101] Compared with the first target scan image, the second target scan image can be a secondary selection or a supplement to the first target scan image, and is back projected to the three-dimensional tooth model and displayed to enrich the selection of the doctor, so that the doctor can select to view more scan images (such as two-dimensional infrared images and / or texture images) when observing the lesion area such as dental caries. In actual operation, the current display image can be switched between the first target scan image and the second target scan image through clicking, scrolling and the like.

[0102] It can be understood that, after the above embodiment associates the lesion positions between the scan images (such as two-dimensional infrared images and / or texture images) and the three-dimensional tooth model through back projection, the lesion positions between the current display image and the three-dimensional tooth model are kept consistent, which facilitates the doctor to dynamically view the lesion positions displayed by different scan images from the three-dimensional tooth model. It can be understood that the above current display image can be the first target scan image and / or the second target scan image.

[0103] The embodiment first effectively removes noise data by using the image quantity and the confidence level, and the target lesion area has high accuracy in the scan images meeting the image quantity and the confidence level, thereby improving the data accuracy. Then, the second target scan image is selected from the multiple scan images under different detection perspectives by using the target confidence level reference value, and the second target scan image is helpful to observe the lesion imaging effect of the lesion area.

[0104] Through the above embodiment, the lesion positions can be quickly positioned and counted according to the global three-dimensional tooth model and the first target scan image and the second target scan image highlighting the local features, and the omission can be effectively avoided.

[0105] In the embodiment of the disclosure, the oral cavity detection method can further include the following content.

[0106] In the embodiment, the lesion site table can be dynamically established according to the order of the detection perspectives, and the lesion site table is used to record the lesion positions detected under each detection perspective and the annotation information of the lesion positions on the three-dimensional tooth model as the detection perspectives move. In this case, the lesion site table used to record the lesion positions and the annotation information of the lesion positions on the three-dimensional tooth model can be obtained; wherein the annotation information such as includes: the three-dimensional mesh patches corresponding to the lesion positions on the three-dimensional tooth model, the order of the detection perspectives, the confidence level corresponding to the lesion positions and the current optimal perspective, the scan image corresponding to the lesion positions, and the like, and the above optimal perspective is, for example, the detection perspective corresponding to the highest confidence level.

[0107] The lesion site table can dynamically update the data in the table as the detection angle changes. An example of an update is shown as follows.

[0108] In a case where the current lesion position and the historical lesion position have a high area overlap on the three-dimensional tooth model, the annotation information of the historical lesion position is updated to the annotation information of the target lesion position in the lesion site table. The high area overlap of the current lesion position and the historical lesion position indicates that they are the same lesion position. In order to avoid repeated data in the lesion site table, only one target lesion position with higher quality can be retained for the same lesion position, that is, one with higher confidence is selected as the target lesion position from the current lesion position and the historical lesion position, and the annotation information of the historical lesion position is replaced by the annotation information of the target lesion position in the lesion site table. This can simplify the data in the table and make the data more intuitive and simple.

[0109] In a case where the current lesion position and the historical lesion position have a high area overlap on the three-dimensional tooth model, the annotation information of the historical lesion position is updated to the annotation information of the target lesion position in the lesion site table. The high area overlap of the current lesion position and the historical lesion position indicates that they are the same lesion position. In order to avoid repeated data in the lesion site table, only one target lesion position with higher quality can be retained for the same lesion position, that is, one with higher confidence is selected as the target lesion position from the current lesion position and the historical lesion position, and the annotation information of the historical lesion position is replaced by the annotation information of the target lesion position in the lesion site table. This can simplify the data in the table and make the data more intuitive and simple.

[0110] The above embodiments update or add the annotation information to dynamically update the optimal data in the lesion site table.

[0111] According to the lesion positions and the annotation information recorded in the lesion site table, the lesion position with higher quality can be updated and the new lesion position can be added on the three-dimensional tooth model.

[0112] After the target lesion area on the scanning image is determined according to the above embodiments, the lesion level of the target lesion area can be further determined.

[0113] When the target lesion area is located in the tooth gap, the lesion level of the target lesion area is determined as the first level. The target lesion area located in the tooth gap indicates that the lesion is in the tooth interior and has not affected the tooth surface. It is difficult to see by the naked eye. For such tooth interior lesions, it is usually in the early stage, such as interproximal caries. Generally, it can be defined as early caries. In this embodiment, the target lesion area located in the tooth gap is determined as the first level with the lowest lesion level.

[0114] When the target lesion area is located on the tooth surface, the color depth and / or area of the target lesion area in the current scanning image are detected; and the lesion level of the target lesion area is determined according to the color depth and / or area.

[0115] Generally, the target lesion area occurs on the surface of the tooth, such as dental caries. In this embodiment, the lesion level of the lesion site can be determined according to the color depth and / or area of the target lesion area, and the lesion level can include multiple levels higher than the first level.

[0116] In summary, the oral detection method provided by the embodiments of the present disclosure comprises: first, obtaining a three-dimensional tooth model by scanning; second, detecting a target lesion area of the tooth and a confidence of the target lesion area based on an oral detection model under a current detection view angle; and projecting the target lesion area to the three-dimensional tooth model to obtain a current lesion position of the three-dimensional tooth model; then, judging whether the area overlap of the current lesion position and a historical lesion position on the three-dimensional tooth model is higher than a preset threshold; and updating a mark of the target lesion position or adding a mark of a new lesion position on the three-dimensional tooth model according to the judgment result. The next detection view angle is determined and the above operations are performed again until the operation under the last detection view angle is completed, and multiple lesion positions of the three-dimensional tooth model are obtained.

[0117] In the technical solution, the target lesion area that may have a lesion is quickly and efficiently detected by using the oral detection model, the time for visual examination is shortened, and the efficiency of the visual examination is improved. The target lesion area is projected to the three-dimensional tooth model, the lesion position is presented to the user in the form of a three-dimensional perspective view, and the display effect of the lesion position is more intuitive. The area overlap of the current lesion position and the historical lesion position and the confidence are used to dynamically update the lesion position with better quality or add a new lesion position on the three-dimensional tooth model, and the marking accuracy of the lesion position is improved. Therefore, the present disclosure can improve the oral detection efficiency and the marking accuracy of the lesion position, and the display of the lesion position is more intuitive.

[0118] Figure 6 A structural block diagram of an oral detection device provided by the embodiments of the present disclosure is provided, and the device is used to implement the above-mentioned oral detection method. As shown in the structural block diagram, Figure 6 the oral detection device comprises:

[0119] A three-dimensional model acquisition module 602 is configured to acquire a three-dimensional tooth model by scanning;

[0120] A lesion area detection module 604 is configured to detect a target lesion area of a tooth and a confidence of the target lesion area based on a pre-trained oral detection model under a current detection view angle;

[0121] A projection module 606 is configured to project the target lesion area to the three-dimensional tooth model to obtain a current lesion position of the three-dimensional tooth model;

[0122] The judgment module 608 is configured to judge whether the area overlap degree of the current lesion position and a historical lesion position on the three-dimensional tooth model is higher than a preset threshold value, wherein the historical lesion position is a lesion position obtained at a detection view angle before the current detection view angle.

[0123] The label updating module 610 is configured to, in the case of yes, select a lesion position with high confidence as a target lesion position from among the current lesion position and the historical lesion position, and update a label of the target lesion position on the three-dimensional tooth model.

[0124] The label adding module 612 is configured to, in the case of no, determine that the current lesion position is a new lesion position, and add a label of the new lesion position on the three-dimensional tooth model.

[0125] The repeated execution module 614 is configured to determine a next detection view angle and execute the above operations again until the operation at the last detection view angle is completed, so as to obtain a plurality of lesion positions of the three-dimensional tooth model.

[0126] The oral cavity detection device provided in the embodiments of the present disclosure can execute the oral cavity detection method provided in any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method.

[0127] Figure 7 A structural schematic diagram of an electronic device provided in the embodiments of the present disclosure is shown in FIG. 7. As shown in FIG. 7, the electronic device 700 includes one or more processors 701 and a memory 702. Figure 7

[0128] The processor 701 can be a central processing unit (CPU) or other forms of processing units having data processing and / or instruction execution capabilities, and can control other components in the electronic device 700 to perform desired functions.

[0129] The memory 702 can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache memory, and / or the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer readable storage medium, and the processor 701 can run the program instructions to implement the oral cavity detection method of the embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, and the like can also be stored in the computer readable storage medium.

[0130] ​In one example, the electronic device 700 can further include an input device 703 and an output device 704, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0131] In addition, the input device 703 can further include, for example, a keyboard, a mouse, and the like.

[0132] The output device 704 can output various information, including the determined distance information, direction information, and the like, to the outside. The output device 704 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0133] Of course, in order to simplify, Figure 7 In the figure, only some of the components in the electronic device 700 related to the present disclosure are shown, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 700 can further include any other appropriate components according to specific application cases.

[0134] Further, the embodiment also provides a computer readable storage medium, which stores a computer program for executing the above-mentioned oral cavity detection method.

[0135] The computer program product of the oral cavity detection method, device, electronic device and medium provided by the embodiment of the present disclosure includes a computer readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the method described in the foregoing method embodiment. For specific implementation, please refer to the method embodiment, which will not be described here.

[0136] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0137] The foregoing is merely illustrative of the various implementations of the present disclosure and the general principles thereof. Numerous modifications can be made to these illustrations, and equivalents can be substituted therefor, without departing from the scope of the present disclosure. The specific embodiments commensurate with the specific application are intended to be illustrative only and not limiting of the scope of the application as set forth in the following claims.

Claims

1. An oral detection device, characterized in that, The method comprises the following steps: a three-dimensional model acquisition module is configured to acquire a three-dimensional tooth model obtained by scanning; a lesion area detection module is configured to acquire a current scan image obtained by scanning each set of teeth under a current detection view angle; each set of teeth comprises at least one tooth; the current scan image is detected based on a pre-trained oral cavity detection model to obtain a target lesion area and a confidence degree of the target lesion area; the oral cavity detection model is trained by using image samples labeled with lesion areas; a projection module is configured to project the target lesion area to the three-dimensional tooth model to obtain a current lesion position of the three-dimensional tooth model; a judgment module is configured to judge whether the current lesion position and a historical lesion position have a region overlap degree higher than a preset threshold on the three-dimensional tooth model; the historical lesion position is a lesion position obtained under a detection view angle before the current detection view angle; a label update module is configured to, in the case of yes, select a lesion position with a high confidence degree from among the current lesion position and the historical lesion position as a target lesion position, and update a label of the target lesion position on the three-dimensional tooth model; a label addition module is configured to, in the case of no, determine that the current lesion position is a new lesion position, and add a label of the new lesion position on the three-dimensional tooth model; a repeated execution module is configured to determine a next detection view angle and execute the above operations again until the operation under the last detection view angle is completed, thereby obtaining a plurality of lesion positions of the three-dimensional tooth model; the label update module is further configured to: acquire a lesion site table used to record the lesion positions and label information of the lesion positions on the three-dimensional tooth model; the label information comprises a three-dimensional mesh patch corresponding to the lesion position on the three-dimensional tooth model, an order of the detection view angle, a confidence degree corresponding to the lesion position, and a current optimal view angle; the optimal view angle is a detection view angle corresponding to the highest confidence degree; in the case where the current lesion position and the historical lesion position have a region overlap degree higher than the preset threshold on the three-dimensional tooth model, update the label information of the historical lesion position to the label information of the target lesion position in the lesion site table; in the case where the current lesion position and the historical lesion position have a region overlap degree not higher than the preset threshold on the three-dimensional tooth model, add the label information of the new lesion position in the lesion site table.

2. The apparatus of claim 1, wherein, The current scan image comprises an infrared image; the detection of the current scan image based on the pre-trained oral cavity detection model to obtain the target lesion area comprises: detecting the infrared image based on the pre-trained oral cavity detection model to obtain a target lesion area of a tooth gap.

3. The apparatus of claim 1, wherein, The current scan image comprises a texture image; the detection of the current scan image based on the pre-trained oral cavity detection model to obtain the target lesion area comprises: detecting the texture image based on the pre-trained oral cavity detection model to obtain a target lesion area of a tooth surface.

4. The apparatus of claim 1, wherein, The lesion area detection module is further configured to: determine a scan image detecting the target lesion region as a candidate scan image; determine a first target scan image from a plurality of candidate scan images corresponding to the target lesion region under a plurality of detection perspectives according to a preset voting strategy, wherein the voting strategy comprises at least one of the confidence, an area ratio of the target lesion region in the candidate scan image, and a definition of the candidate scan image.

5. The apparatus of claim 1, wherein, The lesion region detection module is further configured to: determine a target image number of a plurality of scan images under different detection perspectives detecting the target lesion region for the same set of teeth; when the target image number is greater than a preset number value, determine a target confidence reference value corresponding to the target image number according to a corresponding relationship between an image number and a confidence reference value; select at least one scan image reaching the target confidence reference value from the plurality of scan images under the different detection perspectives as a second target scan image; back-project the target lesion region in the second target scan image to the three-dimensional tooth model.

6. The apparatus of claim 1, wherein, The lesion region detection module is further configured to: when the target lesion region is located in a tooth gap, determine a lesion level of the target lesion region as a first level; when the target lesion region is located on a tooth surface, detect a color depth and / or an area of the target lesion region in a current scan image; determine a lesion level of the target lesion region according to the color depth and / or the area.

7. The apparatus of claim 1, wherein, The mark adding module is further configured to: determine a target tooth corresponding to each lesion position on the three-dimensional tooth model; bind each lesion position and a tooth number of the target tooth corresponding thereto according to a tooth number of each tooth on the three-dimensional tooth model, so as to feed back each lesion position in the form of a tooth number.

Images