Oral self-examination guiding method based on active learning

Abstract

This invention relates to the field of artificial intelligence technology, and more specifically, to a method for guiding oral self-examination based on active learning. The method includes the following steps: Step S1: The user terminal acquires a current oral color image; the oral self-examination processing module calls a tooth segmentation network to generate a tooth segmentation mask corresponding to the current oral color image, and extracts tooth image blocks; Step S2: Candidate target tooth areas with prediction uncertainty index values ​​greater than or equal to a first threshold are selected from the tooth area status table; the candidate shooting action with the highest information gain comprehensive score is selected to generate an oral self-examination guidance instruction; Step S3: The oral self-examination guidance instruction is displayed until the prediction uncertainty index values ​​of all tooth areas in the tooth area status table are less than a second threshold or the currently executed shooting count reaches the maximum shooting count. This invention can significantly reduce invalid shooting, improve information acquisition efficiency, and concentrate limited shooting opportunities on key tooth areas.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method for guiding oral self-examination based on active learning. Background Technology

[0002] In the field of oral health management for general users, image recognition technology has seen rapid application in recent years. Many mobile applications use cameras to capture color images of the oral cavity and automatically identify lesions such as cavities, cracks, and gingivitis using convolutional neural networks. These existing technologies typically employ a fixed shooting procedure, such as requiring users to take several images of the dental arch from a forward-facing perspective, after which the model performs classification. Some systems also utilize tooth segmentation networks to divide color images of the oral cavity into multiple independent dental regions, allowing for local analysis of each tooth. With these technologies, users can complete preliminary oral examinations at home without specialized equipment, helping to reduce the risks associated with delayed medical care.

[0003] Although existing technologies can effectively segment and classify oral images, the overall process remains relatively fixed and generally lacks the ability to dynamically adjust the shooting action based on image quality or analysis results. In traditional implementations, systems typically require users to take a fixed number of photos according to preset steps, such as taking a full-mouth image from a frontal view, followed by images of the left, right, and lingual sides. While this fixed approach ensures basic coverage, it is inefficient. Some dental areas may have already achieved high-confidence results in the first round of shooting, and requiring users to repeat the process would create unnecessary operational burden. Furthermore, some dental areas may have high uncertainty in the model output due to insufficient lighting, severe occlusion, or poor viewing angles during the initial shooting, and traditional methods cannot autonomously determine whether additional shots are needed based on these specific uncertainties.

[0004] Some systems attempt to use image quality assessment models to assist in judgment, such as detecting whether the image is overexposed, whether the focus is accurate, and whether the teeth are fully visible in the frame. However, these methods only focus on image quality itself and do not consider the differences in recognition confidence for different dental areas, thus failing to achieve truly personalized re-examination. In other words, existing technologies generally lack a method to dynamically select the next action based on the prediction uncertainty index value for each dental area. Furthermore, existing mobile port cavity self-examination applications generally adopt a single-model structure, i.e., based on a single convolutional neural network for lesion prediction. While this single model is structurally simple, its output often fails to fully reflect the uncertainty within the model. For example, when the model struggles to make a clear judgment among multiple lesion categories, its probability distribution may show similar probabilities for multiple categories, but there is a lack of further analysis to distinguish whether the uncertainty is due to feature ambiguity or judgment fluctuations caused by model bias. This leaves traditional systems without a basis for scheduling re-examinations or recommending further examinations. Summary of the Invention

[0005] The main objective of this invention is to provide a guided oral self-examination method based on active learning. This method involves segmenting tooth regions from color images of the oral cavity, constructing a multi-model network composed of multiple lesion classification convolutional neural network sub-models, and calculating the tooth region prediction uncertainty index by combining prediction entropy values ​​and the number of multi-model divergences. This dynamically identifies tooth regions requiring focused confirmation. Furthermore, it simulates the potential information gain of different shooting actions using virtual perspective tooth region image blocks, and filters executable candidate shooting actions based on the user's mouth opening level. The next shooting action that minimizes uncertainty is selected, enabling multi-round guided image acquisition. This continuously reduces uncertainty and automatically terminates the process after meeting stopping conditions. Through this approach, the system significantly reduces invalid shots, improves information acquisition efficiency, concentrates limited shooting opportunities on key tooth regions, and provides ordinary users with a more accurate and personalized oral self-examination experience in a non-professional environment.

[0006] To solve the above problems, the technical solution of the present invention is implemented as follows:

[0007] The active learning-based oral self-examination guidance method includes the following steps:

[0008] Step S1: The user terminal acquires the current oral cavity color image. The oral cavity self-check processing module calls the tooth region segmentation network to generate a tooth region segmentation mask corresponding to the current oral cavity color image and extracts tooth region image blocks. The tooth region image blocks are input into a multi-model network composed of three lesion classification convolutional neural network sub-models. Based on the prediction entropy value output by the three lesion classification convolutional neural network sub-models and the number of multi-model divergences, the prediction uncertainty index value is calculated, and a tooth region status table is constructed.

[0009] Step S2: Filter candidate target tooth regions in the tooth region status table whose prediction uncertainty index values ​​are greater than or equal to the first threshold; generate candidate shooting actions for the candidate target tooth regions; perform virtual view feature generation on the candidate shooting actions, obtain virtual view tooth region image patches and input them into a multi-model network to calculate the virtual view prediction uncertainty index values; calculate the comprehensive information gain score of the candidate shooting actions; combine the user's mouth opening level and select the candidate shooting action with the highest comprehensive information gain score to generate oral self-check guidance instructions;

[0010] Step S3: Display oral self-examination guidance instructions, guide the user to collect new oral color images and repeat the above steps until the prediction uncertainty index values ​​of all dental areas in the dental area status table are less than the second threshold or the current number of images has reached the maximum number of images.

[0011] Furthermore, in step 1, the tooth segmentation network consists of a convolutional layer, a downsampling layer, and a pixel-level classification layer in sequence. The input of the tooth segmentation network is the current oral color image, and the output is a tooth segmentation mask that corresponds one-to-one with the current oral color image at the pixel position. Each pixel in the tooth segmentation mask contains a tooth region identifier number. The specific process of cropping tooth region image blocks includes: the oral self-checking processing module determines the bounding rectangle of each tooth region based on the connected regions of the pixels corresponding to the same tooth region identifier number in the tooth segmentation mask, crops the image content in the current oral color image that is located within the bounding rectangle as a tooth region image block, and stores the tooth region image block in the tooth region image block list after interpolation and scaling according to a uniform size.

[0012] Furthermore, in the steps of oral image acquisition and dental region prediction uncertainty calculation, the three lesion classification convolutional neural network sub-models have the same network structure, all including an input layer, a feature extraction part consisting of four convolutional layers and four pooling layers alternately, a classification part consisting of two fully connected layers, and a normalized output layer that outputs the probability distribution of lesion categories. The three lesion classification convolutional neural network sub-models are trained in the following ways: the first lesion classification convolutional neural network sub-model uses the original training sample set containing dental region image blocks and their lesion category labels as training input; the second lesion classification convolutional neural network sub-model uses the enhanced training sample set after performing random rotation, translation, and brightness adjustment on each dental region image block in the original training sample set as training input; the third lesion classification convolutional neural network sub-model uses the original training sample set as training input, and after each forward calculation, randomly selects several convolutional feature map positions in the feature extraction part and sets their feature values ​​to fixed constants.

[0013] Furthermore, in step S1, the process of calculating the prediction entropy value and the number of multi-model divergences includes: for each lesion classification convolutional neural network sub-model, taking the product of the probability value of each output lesion category and the logarithm of the probability value of the lesion category as an item, summing the product results of each item and taking the negative number to obtain the prediction entropy value of the lesion classification convolutional neural network sub-model, and calculating the arithmetic mean of the prediction entropy values ​​of the three lesion classification convolutional neural network sub-models for the same dental region image block to form the average prediction entropy value of the dental region image block; in each lesion classification convolutional neural network sub-model, finding the lesion category with the largest probability value in the lesion category probability distribution as the main prediction category of the lesion classification convolutional neural network sub-model, counting the number of different lesion categories appearing in the main prediction category output by the three lesion classification convolutional neural network sub-models, and taking the number of different lesion categories as the number of multi-model divergences of the dental region image block.

[0014] Furthermore, in step S1, the specific process of calculating the prediction uncertainty index value includes: linearly scaling the average prediction entropy value to the interval of zero to one to form an entropy normalized value; dividing the number of multi-model divergences by three to form a divergence normalized value; adding the entropy normalized value and the divergence normalized value and then dividing by two to form the prediction uncertainty index value of the dental region image patch.

[0015] Furthermore, in step S2, the candidate imaging actions include the candidate target tooth area identification number, the imaging angle type, and the imaging distance level; the imaging angle type includes three categories: frontal view, buccal view, and lingual view; the imaging distance level includes two categories: mesial level and intermediate level; a total of six candidate imaging actions are formed for each candidate target tooth area.

[0016] Furthermore, in step S2, the process of generating virtual viewpoint features for the candidate shooting action includes: searching for the original dental region image block corresponding to the candidate target dental region identifier number of the candidate shooting action in the dental region image block list, and copying the original dental region image block as the dental region image block to be transformed; performing rotation transformation according to the shooting viewpoint type: if the shooting viewpoint type is a forward viewpoint, then performing a transformation with a horizontal rotation angle of zero; if the shooting viewpoint type is a buccal viewpoint, then performing a transformation with a horizontal clockwise rotation angle of a preset first angle; if the shooting viewpoint type is a lingual viewpoint, then performing a transformation with a horizontal counterclockwise rotation angle of a preset first angle; performing scaling transformation according to the shooting distance level: if the shooting distance level is near level, then performing a scaling transformation of the dental region image block to be transformed according to a preset magnification ratio and restoring it to a uniform size using bilinear interpolation; if the shooting distance level is mid-range level, then performing a scaling transformation of the dental region image block to be transformed according to a preset reduction ratio and restoring it to a uniform size using bilinear interpolation; after the transformation is completed, a virtual viewpoint dental region image block corresponding to the candidate shooting action is obtained.

[0017] Furthermore, in step S2, the specific process of calculating the comprehensive information gain score of the candidate imaging actions includes: inputting the virtual view dental region image block into three lesion classification convolutional neural network sub-models in sequence to obtain three new prediction entropy values ​​and calculating their arithmetic mean as the virtual view average prediction entropy value; simultaneously counting the number of different lesion categories appearing in the three new main prediction categories as the virtual view multi-model divergence number; calculating the uncertainty reduction value: subtracting the virtual view prediction uncertainty index value calculated based on the virtual view average prediction entropy value and the virtual view multi-model divergence number from the original dental region image block prediction uncertainty index value; if the calculation result is less than zero, the uncertainty reduction value is set to zero; calculating the multi-model divergence change value: the absolute value of the difference between the original multi-model divergence number and the virtual view multi-model divergence number; finding the maximum value of the uncertainty reduction value and the maximum value of the multi-model divergence change value in the calculation results of all candidate imaging actions, normalizing the values ​​of each candidate imaging action respectively, adding the normalized uncertainty reduction value and the normalized multi-model divergence change value of the same candidate imaging action and dividing by two to obtain the comprehensive information gain score of the candidate imaging action.

[0018] Furthermore, in step S2, the process of combining the user's mouth opening level includes: acquiring an oral cavity calibration image of the user in the maximum mouth opening state through the mouth opening calibration process of the user terminal, measuring the vertical distance between the corresponding pixel positions of the maxillary incisor occlusal edge and the mandibular incisor occlusal edge in the oral cavity calibration image; dividing the vertical distance into multiple mouth opening levels after comparing it with a preset vertical distance threshold, and preset the required mouth opening level for each shooting angle type and shooting distance level combination; for each candidate shooting action, reading the user's mouth opening level and comparing it with the required mouth opening level corresponding to the candidate shooting action, and adding the candidate shooting action to the list of executable candidate shooting actions when the user's mouth opening level is greater than or equal to the required mouth opening level; selecting the candidate shooting action with the highest information gain comprehensive score is selected from the list of executable candidate shooting actions.

[0019] Furthermore, in step S3, the specific process of the loop control includes: after generating a new tooth area status table in each round, the oral self-check processing module counts the prediction uncertainty index values ​​of all tooth areas in the tooth area status table. If the prediction uncertainty index value of at least one tooth area is greater than or equal to the second threshold, and the number of times the current shooting has been performed is less than the maximum number of shooting, then the count value of the number of times the current shooting has been performed is increased, and the target shooting action determination step based on active learning is called again to generate a new round of target shooting action and new oral self-check guidance instructions; otherwise, the oral self-check processing module ends the execution of the oral self-check guidance method based on active learning.

[0020] The oral self-examination guidance method based on active learning of this invention has the following beneficial effects: This invention uses the predicted uncertainty index value of dental regions as the core driving variable, transforming the oral self-examination process from a traditional fixed shooting procedure to an active guidance method based on dynamic decision-making, thereby significantly improving the effectiveness of each shooting action by the user. This invention utilizes the prediction results of multiple lesion classification convolutional neural network sub-models to construct multi-model divergence number and prediction entropy values, and fuses these two to form a predicted uncertainty index value that reflects the degree of local information loss, enabling the system to accurately identify which dental regions still have significant analytical ambiguity under the current information conditions. This invention further introduces a virtual perspective dental region image patch generation mechanism. By simulating the imaging features that may be obtained under different shooting angles and shooting distance levels, the potential information gain of each candidate shooting action is evaluated in advance, so that the next shooting no longer relies on experience selection, but directly points to the action that minimizes uncertainty. Simultaneously, this invention combines the user's mouth opening level to filter executable candidate shooting actions, making the shooting guidance not only algorithmically optimal but also practically feasible, avoiding shooting paths that the user cannot cooperate with. In multiple cycles, this invention continuously updates the dental region status table, enabling the system to track the changing trends of uncertainty in real time. The process automatically terminates when the uncertainty decreases to a set range or the maximum allowed number of shots is reached, effectively controlling user operating costs. Through this method, the invention can concentrate supplementary shots on key dental regions and achieve rapid convergence while minimizing unnecessary shots, making the oral self-examination process more efficient, intelligent, and personalized. It also allows ordinary users without professional experience to obtain a digitally assisted experience approaching professional examination results. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the change in the number of highly uncertain dental regions during multiple rounds of imaging iterations, provided by an embodiment of the present invention.

[0022] Figure 2 A schematic diagram illustrating the two-dimensional distribution of the dental region in terms of average prediction entropy and number of multi-model divergences, provided in an embodiment of the present invention.

[0023] Figure 3 A schematic diagram comparing the training loss curves of the three lesion classification convolutional neural network sub-models provided in this embodiment of the invention when using different training strategies;

[0024] Figure 4 This is a schematic diagram of the virtual viewpoint tooth region image blocks corresponding to six candidate shooting actions generated for a single candidate target tooth region, provided in an embodiment of the present invention. Detailed Implementation

[0025] The terms first, second, third, fourth, etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0026] The active learning-based oral self-examination guidance method includes the following steps:

[0027] Step S1: The user terminal acquires the current oral cavity color image. The oral cavity self-check processing module calls the tooth region segmentation network to generate a tooth region segmentation mask corresponding to the current oral cavity color image and extracts tooth region image blocks. The tooth region image blocks are input into a multi-model network composed of three lesion classification convolutional neural network sub-models. Based on the prediction entropy value output by the three lesion classification convolutional neural network sub-models and the number of multi-model divergences, the prediction uncertainty index value is calculated, and a tooth region status table is constructed.

[0028] Step S2: Filter candidate target tooth regions in the tooth region status table whose prediction uncertainty index values ​​are greater than or equal to the first threshold; generate candidate shooting actions for the candidate target tooth regions; perform virtual view feature generation on the candidate shooting actions, obtain virtual view tooth region image patches and input them into a multi-model network to calculate the virtual view prediction uncertainty index values; calculate the comprehensive information gain score of the candidate shooting actions; combine the user's mouth opening level and select the candidate shooting action with the highest comprehensive information gain score to generate oral self-check guidance instructions;

[0029] Step S3: Display oral self-examination guidance instructions, guide the user to collect new oral color images and repeat the above steps until the prediction uncertainty index values ​​of all dental areas in the dental area status table are less than the second threshold or the current number of images has reached the maximum number of images.

[0030] In one specific implementation, step S1 can be performed as follows to obtain a dental region status table from the current oral color image and lay the foundation for subsequent oral self-examination guidance based on active learning.

[0031] First, the user terminal captures a color image of the current oral cavity using a front-facing camera or a dedicated dental imaging accessory. To ensure the stability of subsequent processing, a preset viewfinder can be displayed on the interface, prompting the user to open their mouth to a specified range and center the dental arch approximately. The current oral cavity color image can be set to a width... Pixels, High A three-channel color image of pixels, where each pixel contains intensity values ​​for the red, green, and blue channels. After acquisition, the image is temporarily stored in the terminal memory as input for subsequent processing.

[0032] After acquiring the current oral cavity color image, a tooth segmentation network is invoked to perform fine-grained segmentation of the tooth regions in the image. In one optional implementation, the tooth segmentation network employs a typical encoder-decoder structure, consisting of several convolutional layers, downsampling layers, and pixel-level classification layers connected sequentially. Each convolutional operation in the encoding part uses... The convolution kernel has a stride of 1. After each convolutional layer, a non-linear activation, such as a linear rectified activation function, is added. A downsampling layer is inserted every two convolutional layers to halve the width and height of the feature map, extracting higher-level semantic features. The decoding part gradually restores spatial resolution through upsampling and convolution operations, finally outputting a tooth region segmentation mask at the pixel-level classification layer. The tooth region segmentation mask has the exact same dimensions in the width and height directions as the current oral color image, and each pixel location corresponds to an integer tooth region identifier. These identifiers can be... Reserved as a background category, with identifier numbering. to This corresponds sequentially to specific tooth positions, from the first molar in the upper right jaw to the second molar in the lower left jaw. In this way, each pixel is clearly assigned to a specific tooth region or background, providing precise boundaries for subsequent cropping of tooth region image blocks.

[0033] After generating the tooth segmentation mask, it is necessary to extract tooth region image blocks from the current oral color image. This can be accomplished through connected component analysis. For each non-zero tooth region identifier in the tooth segmentation mask, traverse the entire tooth segmentation mask image to find all pixels equal to that identifier, and treat these pixels as components of the same connected region. For each connected region, calculate its minimum row coordinate, maximum row coordinate, minimum column coordinate, and maximum column coordinate, denoted as _____. , , , ,in This indicates the index of the uppermost pixel row in the vertical direction of the tooth region. This indicates the index of the lowest pixel row. This represents the leftmost pixel column index in the horizontal direction. This represents the rightmost pixel column index. These four numbers can be used to define an outer rectangle, placing the row indices of the current oral cavity color image within this rectangle. to Between and column index in to All pixels between the specified points are extracted to form the original dental region image patch. Subsequently, each original dental region image patch is scaled to a uniform size using bilinear interpolation, for example, a width of... Pixels, High Pixels are normalized so that the subsequent lesion classification convolutional neural network sub-model can uniformly receive input. To reduce brightness differences caused by different shooting conditions, the pixels of each dental area image block can be normalized after scaling, reducing the original... to The grayscale value is linearly mapped to to This reduces the impact of lighting changes on the model output by limiting the range of illumination.

[0034] After the dental region image block is prepared, it is input into the system. A multi-model network, composed of several lesion classification convolutional neural network sub-models, performs lesion assessment and uncertainty measurement. The input to each lesion classification convolutional neural network sub-model is a... The output is a probability distribution of a tooth region image patch of a certain size across several preset lesion categories. Optionally, the lesion categories may include several categories such as "no obvious lesion," "early caries," "intermediate to late-stage caries," "suspected crack," and "enamel defect," with the total number of lesion categories denoted as [missing information]. For each lesion classification convolutional neural network sub-model, the probability distribution of the output can be represented as a set. ,in Indicates the first The convolutional neural network sub-model for lesion classification classifies the current dental region image block as the first... The probability value of each lesion category. The range of values ​​is arrive , The range of values ​​is arrive And for the same ,all The sum equals The design of the probability distribution ensures that each sub-model not only provides the most probable lesion category but also the confidence level for all categories, laying the foundation for subsequent numerical calculations of prediction entropy and the number of divergences among multiple models.

[0035] To quantitatively characterize the uncertainty of each lesion classification convolutional neural network sub-model in the current dental region, the prediction entropy value can be calculated for the above probability distribution. For the ... The prediction entropy value of a lesion classification convolutional neural network sub-model can be expressed as: ,in Indicates the first The information entropy value of the convolutional neural network sub-model for lesion classification on the current dental region image patch, denoted as _____. Indicates index from arrive Summation operation, For the aforementioned The convolutional neural network sub-model for lesion classification is used for the first... The probability value of each lesion category. This represents the logarithm operation with the natural constant as the base. For a specific sub-model of a convolutional neural network for lesion classification, when its output probability distribution is highly concentrated in a particular lesion category, the corresponding... There exists a close to The value is close to the other probability values. ,at this time A smaller value indicates that the model is more certain in its judgment of that dental region. When the output probabilities are relatively evenly distributed across multiple lesion categories, each... The differences between them are small. A large value indicates that the model fluctuates between multiple categories, reflecting significant uncertainty. By introducing information entropy as a quantitative indicator, the degree of ambiguity in a single model's judgment of dental lesions can be objectively reflected without the need for manually setting threshold intervals.

[0036] refer to Figure 3 This figure contains three parallel subplots, each corresponding to the training process of a lesion classification convolutional neural network sub-model. The horizontal axis represents the number of training epochs, ranging from 1 to 50, and the vertical axis represents the loss function value, ranging from 0 to 2.5. The first subplot shows the training curve of sub-model 1. This sub-model uses the original training sample set containing dental region image patches and their lesion category annotations as training input. The training loss curve is plotted as a solid line. The curve drops rapidly in the first 15 training epochs, from an initial value of about 2.0 to about 0.5, then enters the convergence phase, eventually stabilizing around 0.3. A hollow dot is marked on the curve every 5 training epochs to emphasize key positions. The text "rapid decline phase" is marked near the training epoch 10, and the text "convergence region" is marked near the training epoch 45.

[0037] The second subplot shows the training curve of submodel 2. This submodel uses an augmented training sample set, after random rotation, translation, and brightness adjustment of each tooth region image patch in the original training sample set, as its training input. The training loss curve is drawn as a dashed line, with alternating 5-pixel solid line segments and 3-pixel blank segments. Due to the increased diversity of the training samples caused by the data augmentation operations, the initial value of this curve is slightly higher than that of submodel 1, approximately 2.2, but the descent is smoother with smaller fluctuations, eventually converging to approximately 0.25. A gray square is marked on the curve every 5 training epochs. The subplot also includes a text box listing the specific data augmentation operations, including random rotation, random translation, and brightness adjustment.

[0038] The third sub-figure shows the training curve of sub-model 3. This sub-model uses the original training sample set as training input and, after each forward computation, randomly selects several convolutional feature map positions in the feature extraction part and sets their feature values ​​to fixed constants, which is equivalent to introducing the Dropout regularization strategy. The training loss curve is drawn as a dotted line with relatively close spacing. The fluctuation amplitude of this curve is significantly larger than that of the first two sub-models. The initial value is about 2.1, and it exhibits strong oscillation characteristics during training, eventually converging to about 0.35. A black triangle is marked on the curve every 5 training epochs. The text box in the sub-figure explains the specific implementation of the Dropout strategy, namely, randomly setting the feature map position values ​​to fixed constants. By comparing the three sub-figures, it can be seen that although the three lesion classification convolutional neural network sub-models use the same network structure, due to the differences in training strategies, their training loss curve shapes and convergence characteristics are significantly different. This difference is the basis for the multi-model network to evaluate the lesion status of the dental region from different perspectives, and it is also a prerequisite for calculating the number of divergences in the multi-model network.

[0039] After obtaining the prediction entropy values ​​of the three lesion classification convolutional neural network sub-models, the average prediction entropy value can be calculated to comprehensively reflect the overall uncertainty level of the multi-model network. The average prediction entropy value can be expressed as: ,in This represents the average prediction entropy value of the same dental region image patch under three lesion classification convolutional neural network sub-models. , , They represent the first , No. and the The predicted entropy value of a lesion classification convolutional neural network sub-model, with the denominator representing the number. This indicates the number of lesion classification convolutional neural network sub-models involved in the average calculation. By calculating the average prediction entropy value, bias caused by the occasional extreme predictions of a single sub-model can be avoided, making the overall uncertainty more stably reflect the combined opinions of the three.

[0040] Besides examining the inherent uncertainty of the probability distribution, whether divergences occur between different lesion classification convolutional neural network sub-models in a multi-model network is also an important indicator. Therefore, the number of divergences in the multi-model network can be calculated. For each lesion classification convolutional neural network sub-model, this can be determined from the set of probability distributions of its output. Find the lesion category with the highest probability value and denote the index of this lesion category as... ,in Indicates the first The main predicted class index of each of the three lesion classification convolutional neural network sub-models. The results were obtained from the three lesion classification convolutional neural network sub-models respectively. , and Then, we can count the number of distinct values ​​among these three indices and record this number as . ,in This represents the number of multi-model divergences. When... equal When the main predicted categories of the three convolutional neural network sub-models representing lesion classifications are completely consistent, the multi-model network has a high degree of consensus on this dental region; when equal or This indicates that at least one lesion classification convolutional neural network sub-model gives a different main predicted category than other models, or even that the three models each give three different main predicted categories, reflecting a significant divergence between the models. The number of multi-model divergences also characterizes uncertainty from another perspective: even if the prediction entropy of a single model is not particularly high, once the main predicted categories of different models change, it indicates that there are differences in the adaptation of the training data or model structure to the dental region, which should also be considered as a significant uncertainty that needs to be closely monitored in subsequent imaging.

[0041] To facilitate comparisons between different dental regions, the average predicted entropy value and the number of divergences between multiple models can be normalized separately. An optional approach is to pre-calculate the range of the average predicted entropy values ​​on the training or validation set, and denot the minimum value as... The maximum value is denoted as ,in This represents the lower bound of the average prediction entropy value. This represents the upper bound of the average prediction entropy value. It is the average prediction entropy value obtained for a specific dental region. It can calculate the entropy normalized value. ,in This represents the normalized entropy value corresponding to that dental region, in the numerator. This represents the difference between the current average predicted entropy value and the lower bound, with the denominator containing... This represents the length of the entire average prediction entropy value range. Through linear mapping, the original average prediction entropy values ​​are uniformly compressed to... to Within this range, it is easy to combine with other indicators. Simultaneously, the divergence normalization value can be calculated. ,in This represents the bifurcation normalized value corresponding to this dental region, numerator. This refers to the number of divergences in the aforementioned multi-model calculation, represented by the number in the denominator. This indicates the number of convolutional neural network sub-models for lesion classification in a multi-model network. Because... The range of values ​​is to Through such normalization, The range of values ​​is greater than or equal to and less than or equal to The larger the value, the more serious the discrepancy between the models.

[0042] After completing the above normalization, the entropy normalized value and the divergence normalized value can be combined to generate a single prediction uncertainty index value. The prediction uncertainty index value can be expressed as... ,in This represents the prediction uncertainty index value of the current dental region image patch. This is the normalized entropy value for this dental region. The normalized value of the bifurcation in this dental region, the number in the denominator This is used to calculate the arithmetic mean of the two. By averaging, the degree of ambiguity within the probability distribution and the degree of divergence between different lesion classification convolutional neural network sub-models can be considered simultaneously, avoiding bias caused by a single indicator. For example, when the average prediction entropy value of a certain dental region is high but the number of divergences among multiple models is small, it indicates that all three models are hesitant but tend to agree; when the average prediction entropy value is low but the number of divergences among multiple models is large, it indicates that each model gives a confident judgment, but the judgments are inconsistent. Both of these situations are areas that need to be focused on for subsequent active learning. Combining the two into a prediction uncertainty index value is beneficial for taking into account these two different types of uncertainty when selecting imaging targets.

[0043] In a specific numerical example, we can assume the average predicted entropy value of a certain dental region. for Assuming that the pre-statistics are... for , for Then the entropy normalized value Approximately Furthermore, assuming the number of multi-model bifurcations in this dental region... for Then the divergence normalized value for At this point, the value of the prediction uncertainty index... for Approximately ,exist to The uncertainty level falls within the range of moderate to high. Through this quantification method, the system can directly compare multiple dental areas, prioritizing areas with higher predicted uncertainty values.

[0044] After calculating the prediction uncertainty index for each dental region image patch, a dental region status table can be constructed. The dental region status table can be designed as a two-dimensional table or a set of structured records, with each record containing at least two fields: dental region identifier number and prediction uncertainty index value. Optionally, a current lesion category field can be added to the dental region status table to store the main predicted category obtained from the probability distribution statistics of the outputs of the three lesion classification convolutional neural network sub-models. For example, a voting method can be used to select the lesion category that appears most frequently among the main predicted categories of the three lesion classification convolutional neural network sub-models as the current main lesion category for that dental region. The dental region status table is usually sorted by dental region identifier number, for example, from... Arrived This facilitates the direct location of the corresponding tooth region and its uncertainty status during subsequent steps. Through this constructed tooth region status table, the system can quickly filter out candidate target tooth regions with prediction uncertainty index values ​​greater than or equal to a first threshold in subsequent steps, and generate candidate imaging actions accordingly, achieving highly targeted active learning guidance.

[0045] refer to Figure 2 This graph uses the average predicted entropy value as the horizontal axis and the number of multi-model divergences as the vertical axis. The position and color of the scatter points together represent the predicted uncertainty index values ​​for each dental region. The horizontal axis ranges from 0 to 1.6, and the vertical axis ranges from 0.8 to 3.2. The theoretical values ​​for the number of multi-model divergences are 1, 2, or 3. For easier visualization, a slight random jitter is applied to each scatter point along the vertical axis. The graph shows 32 scatter points, each corresponding to a dental region in the oral cavity. The color of the scatter points, from light to dark, represents the change in the predicted uncertainty index value from low to high, using a gradient from yellow to orange to red. Several contour lines are also superimposed on the graph. These contour lines represent the isopleths of the predicted uncertainty index values, labeled with values ​​of 0.20, 0.35, 0.50, 0.65, and 0.80. The contour lines are drawn as gray dashed lines, with the corresponding values ​​labeled on the lines. These contour lines allow for quick identification of the uncertainty level region of a particular dental region.

[0046] The figure specifically marks four representative regions: Region 7, Region 15, Region 22, and Region 30. Region 7 is located at an average prediction entropy of approximately 0.72 and a multi-model divergence count of 3. Its scatter plot is dark red, indicating a high uncertainty state, and is labeled "Region 7 (High Uncertainty)". Region 15 is located at an average prediction entropy of approximately 0.68 and a multi-model divergence count of 2. Its scatter plot is orange-red, indicating medium-high uncertainty, and is labeled "Region 15 (Medium-High Uncertainty)". Region 22 is located at an average prediction entropy of approximately 0.38 and a multi-model divergence count of 1. Its scatter plot is light orange, indicating medium-low uncertainty, and is labeled "Region 22 (Medium-Low Uncertainty)". Region 30 is located at an average prediction entropy of approximately 0.23 and a multi-model divergence count of 1. Its scatter plot is light yellow, indicating convergence to a low uncertainty state, and is labeled "Region 30 (Converged)". The figure also includes two threshold reference lines: a green dashed line perpendicular to the horizontal axis, located at a point where the average prediction entropy value equals 0.7, labeled "Average Prediction Entropy Threshold"; and a blue dashed line parallel to the horizontal axis, located at a point where the number of multi-model divergences equals 2, labeled "Multi-Model Divergence Threshold." These two threshold lines divide the two-dimensional space into different regions, facilitating the identification of which dental regions need to be prioritized in subsequent steps. The figure visually demonstrates how the prediction uncertainty index is calculated by combining the average prediction entropy value and the number of multi-model divergences, illustrating the significant differences in lesion prediction among different dental regions and providing a data foundation for candidate target dental region selection based on active learning.

[0047] In another alternative implementation, the tooth region segmentation network can employ different numbers of convolutional layers or different upsampling methods, such as a combination of transposed convolution or bilinear interpolation plus convolution, to adapt to different terminal processing capabilities. However, its core remains the same: assigning a tooth region identifier number to each pixel in the current oral color image. Alternatively, different logarithmic bases can be chosen when calculating the prediction uncertainty index, or a nonlinear mapping can be used for the entropy normalization value, for example, using... The form of square root mapping, where the symbol The square root operation is indicated to emphasize the differences in the medium-to-high uncertainty region. These optional modifications adjust the specific numerical mapping process while keeping the multi-model network structure and the tooth region state table structure unchanged, to adapt to different application scenarios and hardware environments.

[0048] After completing step S1, a dental region status table is obtained. Each record in the dental region status table corresponds to a dental region identifier number and includes the predicted uncertainty index value for that dental region and the optional current primary lesion category information. The task of step S2 is to determine, based on the dental region status table, the dental region most worthy of additional imaging and the corresponding imaging method, thereby minimizing overall uncertainty with a limited number of imaging attempts.

[0049] First, the system filters the data in the dental region status table. A first threshold is pre-set for the prediction uncertainty index value, denoted as [threshold value]. ,in For the middle and The constant between these two values ​​represents a lower limit; dental regions exceeding this lower limit are considered to have higher uncertainty. For each record in the dental region status table, the corresponding prediction uncertainty index value is read and denoted as... ,in The dental area identification number is The predictive uncertainty index value of the dental region, Number the dental region identifiers. Include all those that meet the criteria. The identified tooth region identifiers are collected to form a candidate target tooth region set. The purpose of this is to use a numerical condition to exclude tooth regions that are already relatively certain, concentrating computational and interactive resources on truly "uncertain" targets. If the first threshold is set high, for example... Therefore, only a very small number of highly uncertain dental regions will be selected into the candidate target dental region set, resulting in fewer additional imaging attempts; if the first threshold is set to... Moving the image to the left or right will result in more dental regions being included in the candidate target dental region set, allowing the system to perform more detailed re-images at more locations. The first threshold can be adjusted based on device performance and user acceptance.

[0050] After obtaining the set of candidate target tooth regions, candidate imaging actions need to be generated for each region. Each candidate imaging action consists of three fields: candidate target tooth region identifier number, imaging view type, and imaging distance level. In one implementation, the imaging view type includes three categories: frontal view, buccal view, and lingual view. The frontal view refers to the camera's optical axis being roughly perpendicular to the dental arch plane, suitable for observing the overall morphology of the tooth surface; the buccal view refers to shooting from a slightly tilted angle on the cheek side, allowing for clearer observation of the gingival margin and adjacent relationships on the outer side of the crown; the lingual view is shot from the lingual direction, which can be used to expose the inner surfaces that are not normally visible. The imaging distance level can be set to two categories: mesmerizing and intermediate. The mesmerizing level emphasizes local details, such as enamel cracks or microcaries, while the intermediate level strikes a balance between detail clarity and field of view, facilitating the identification of the relative positions between teeth. For each candidate target tooth region, the three imaging view types and two imaging distance levels are combined sequentially to obtain six different candidate imaging actions. The reason for this design is that by systematically enumerating the combination of viewing angles and distances, it is possible to cover a variety of visible features that may exist in the tooth area. For example, some caries may not be obvious in the forward view, but in the buccal view at the mesial level, due to the different angles of light spot reflection, clearer color differences or depressions may appear.

[0051] After generating candidate shooting actions, the user is not immediately required to execute them. Instead, the effects of these shooting actions are first simulated in the image space, and virtual viewpoint features are generated. For each candidate shooting action, its corresponding original dental region image block needs to be determined. The corresponding original dental region image block can be found in the list of dental region image blocks generated in step S1 based on the candidate target dental region identifier number in the candidate shooting action, and then copied as the dental region image block to be transformed. Next, a rotation operation is performed according to the shooting viewpoint type specified in the candidate shooting action. For example, when the shooting viewpoint type is a forward viewpoint, the horizontal rotation angle of the dental region image block to be transformed is [missing information]. The image is transformed to maintain its original orientation; when the shooting view type is cheek view, a horizontal clockwise rotation angle of a preset first angle can be performed, which can be set to... Degree to A value between degrees, for example The camera rotates horizontally by the same angle counterclockwise when the shooting angle is lingual, so that the buccal side of the tooth is more oriented towards the viewing direction. This rotation approximates the effect of a user slightly rotating the camera horizontally around the teeth during actual operation. Bilinear interpolation can be used during the rotation to reduce jagged edges and blocky artifacts.

[0052] After rotation, scaling is performed based on the shooting distance level. When the shooting distance level is close, the image block of the dental region to be transformed can be scaled according to a preset magnification ratio, for example, a magnification ratio of [missing value]. This means that the size in each direction is increased to the original size. The reason for zooming in is that when shooting at close range, the distance between the camera and the target dental area decreases, causing structural details in the image to occupy more pixels. Scaling can simulate this local magnification effect. When the shooting distance is at a medium range, a slightly smaller zoom level can be used. The scaling ratio, for example To maintain a relatively complete tooth surface structure while slightly enlarging local details, after scaling, bilinear interpolation is used to resize the scaled results to a uniform size, such as width, so that all tooth region image patches can be uniformly accepted as input by the multi-model network. Pixels, High Pixels. After two steps of rotation and scaling, the resulting image is the virtual viewpoint image patch of the dental region corresponding to the candidate shooting action. By generating virtual viewpoint features, the potential contribution of different shooting actions to lesion identification can be estimated without actually requiring the user to open their mouth multiple times for shooting, thus achieving something similar to "pre-interrogation assessment" in active learning with a small interaction cost.

[0053] Each virtual view dental region image patch is input into a multi-model network to calculate the virtual view prediction uncertainty index. Specifically, the virtual view dental region image patch is sequentially input into three lesion classification convolutional neural network sub-models, and each sub-model outputs a set of lesion category probability distributions. ,in This indicates that from a virtual perspective, the first The convolutional neural network sub-model for lesion classification classifies the current dental region as the first... The probability value of each lesion category. This represents the sub-model index, with a value range of [value range missing]. arrive , This represents the lesion category number, with a value range of [value missing]. arrive Based on these probability distributions, the three predicted entropy values ​​are calculated using the same method as in step S1. , and ,in Indicates the first The prediction entropy values ​​of the convolutional neural network sub-models for lesion classification under virtual perspective. The average prediction entropy value under virtual perspective can be expressed as: ,in This represents the arithmetic mean of the predicted entropy values ​​of the three sub-models from a virtual perspective. Meanwhile, in each sub-model... Select the lesion category index corresponding to the maximum value from the list, and denote it as... ,statistics , and The number of distinct values ​​in the data yields the number of virtual perspective multi-model divergences, denoted as . ,in This indicates the number of differences between the main predicted categories of the three sub-models in a virtual perspective. (This is achieved by...) and By normalizing and combining the results as described above, we can obtain the numerical value of the virtual perspective prediction uncertainty index. ,in This represents the prediction uncertainty index value of the virtual viewpoint tooth region image patch generated for the current candidate shooting action.

[0054] The next step is to calculate the comprehensive information gain score for each candidate shooting action, which measures its potential contribution to reducing uncertainty when the shooting action is actually executed. For a given candidate shooting action... The candidate target tooth region identifier number corresponding to this action can be set as follows: The prediction uncertainty index value of this dental region in the original dental region status table is denoted as... The corresponding number of multi-model divergences is denoted as The virtual perspective prediction uncertainty index is denoted as The number of divergences in the virtual perspective multi-model is denoted as First, calculate the reduction in uncertainty. Its form can be expressed as ,in Indicates the candidate shooting action Below, for the dental area The amount by which the value of the forecast uncertainty index is reduced. Indicates taking and The larger of the two values. If the virtual perspective predicts the uncertainty index value... If the difference is higher than the original forecast uncertainty index, then the difference is negative, and the maximum value operation is used to truncate it. This means that the candidate shooting action is not considered to contribute to reducing uncertainty under the current simulation. Subsequently, the numerical value of the multi-model divergence change is calculated. Its form can be expressed as ,in Indicates the candidate shooting action Below, the change in the number of divergences in the multi-model model, sign This represents the absolute value operation. This change reflects whether the main predicted categories of the three sub-models tend to be consistent or rearranged, thus measuring the convergence of opinions within the multi-model network.

[0055] To make the information gain composite score comparable across different candidate shooting actions, it is possible to... and Perform normalization. Assume there are a total of Given a number of candidate shooting actions, we can find the maximum value of the uncertainty reduction among all candidate shooting actions, denoted as [value]. ,in This represents the maximum contribution of all candidate shooting actions in the current round to reducing uncertainty; for each candidate shooting action Calculate uncertainty and reduce normalized values ,in Indicates candidate shooting action Relative score on the uncertainty reduction dimension. When equal At that time, all of them can be unified. Set as This indicates that the candidate shooting actions in this round do not differ significantly in terms of reducing prediction uncertainty. Similarly, the maximum value of the multi-model divergence change value is found among all candidate shooting actions and denoted as... ,in This represents the maximum value of the multi-model divergence change in the current round; for each candidate shooting action... Calculate the normalized numerical values ​​of the multi-model divergence change. ,in Indicates candidate shooting action Relative score on the dimension of multi-model divergence variation. When equal At that time, all candidate shooting actions perform the same in this dimension, which can make all Set uniformly .

[0056] The comprehensive information gain score can be expressed as: ,in Indicates candidate shooting action Information gain comprehensive score, This indicates that the normalized value reduces uncertainty. This represents the normalized value of the multi-model divergence change. By averaging these two items, the information gain comprehensive score simultaneously considers both "the ability to reduce the value of the expected prediction uncertainty index" and "the ability to make the opinions among models more consistent." This combination is equivalent to combining two different sampling strategies in active learning: emphasizing both the reduction of entropy values ​​in individual tooth regions and the coordination between multi-model predictions, which helps avoid "extreme optimization" in only one dimension while ignoring the risks of the other.

[0057] refer to Figure 4 This image consists of six sub-images arranged in a 2x3 matrix layout. Each sub-image shows the effect of a candidate shooting action on the dental image patch after virtual viewpoint feature generation. The six candidate shooting actions are: forward view with near distance level, buccal view with near distance level, lingual view with near distance level, forward view with mid-distance level, buccal view with mid-distance level, and lingual view with mid-distance level. The first sub-image, located in the first row and first column, corresponds to the candidate shooting action of forward view with near distance level. It depicts a tooth shape composed of a crown and a root. The crown has a trapezoidal structure, and the root has a conical structure. The tooth's rotation angle is 0 degrees, and the scaling ratio is 1.5. A dashed rectangular bounding box is drawn around the tooth, with a width and height of 128 pixels by 128 pixels. The upper right corner of the sub-image is labeled with the transformation parameters "θ=0°, S=1.5", and the bottom of the sub-image is labeled with the text "Forward-Near Distance". The second sub-image is located in the first row and second column, corresponding to the buccal perspective and the candidate shooting action at close range. The teeth are rotated at a rotation angle of +25 degrees, that is, rotated 25 degrees clockwise. The scaling ratio is 1.5, the bounding box color is orange, and the sub-image is labeled "buccal-close range" and "θ=25°, S=1.5".

[0058] The third sub-image, located in row 1, column 3, corresponds to a lingual view with a near-range candidate shooting action. The teeth are rotated at a rotation angle of -25 degrees (25 degrees counterclockwise), the scaling factor is 1.5, and the bounding box color is green. The sub-image is labeled "Lingual - Near" and "θ=-25°, S=1.5". The fourth sub-image, located in row 2, column 1, corresponds to a forward view with a mid-range candidate shooting action. The teeth are rotated at a rotation angle of 0 degrees, the scaling factor is 0.9, the bounding box color is blue, and the sub-image is labeled "Forward - Mid-range" and "θ=0°, S=0.9". The fifth sub-image, located in row 2, column 2, corresponds to a buccal view with a mid-range candidate shooting action. The teeth are rotated at a rotation angle of +25 degrees, the scaling factor is 0.9, the bounding box color is cyan, and the sub-image is labeled "Buccal - Mid-range" and "θ=25°, S=0.9". The 6th sub-image is located in the 2nd row and 3rd column. It corresponds to the lingual view with a mid-range candidate shooting action. The teeth are rotated at a rotation angle of -25 degrees, the scaling ratio is 0.9, the bounding box color is purple, and the sub-image is labeled "lingual-mid distance" and "θ=-25°, S=0.9".

[0059] Each sub-image includes detailed features such as crown surface texture lines and enamel margin lines to simulate the visual effect of real dental image patches. All six virtual viewpoint dental image patches, after rotation and scaling transformations, are restored to a uniform size of 128 pixels by 128 pixels using bilinear interpolation, thus meeting the requirements for subsequent input into the multi-model network for prediction. These six candidate shooting actions cover all combinations of three shooting viewpoint types and two shooting distance levels, demonstrating the systematic and complete nature of the virtual viewpoint feature generation steps. In practical applications, the system sequentially inputs these six virtual viewpoint dental image patches into three lesion classification convolutional neural network sub-models, calculates the virtual viewpoint prediction uncertainty index value corresponding to each candidate shooting action, and calculates a comprehensive information gain score based on the uncertainty reduction value and the multi-model divergence change value. Finally, the candidate shooting action with the highest score that meets the user's mouth opening level constraint is selected as the target shooting action, generating corresponding oral self-check guidance instructions to guide the user to complete the next round of oral color image acquisition.

[0060] After completing the comprehensive information gain scoring for all candidate shooting actions, the practical operational constraint of the user's mouth opening level also needs to be considered. The user's mouth opening level can be obtained in a separate mouth opening calibration process. Specifically, the user can be prompted to open their mouth as wide as possible, and a mouth calibration image can be taken. From this image, the pixel positions of the occlusal edges of the maxillary and mandibular incisors are detected, and the vertical pixel distance between them is calculated and denoted as . ,in This represents the pixel distance between the upper and lower teeth when the user's mouth is at its maximum open state. Several distance intervals can be predefined, for example... arrive Pixels represent the first aperture level. arrive Pixels represent the second aperture level. arrive The pixel is the third aperture level, which will The number corresponding to the interval into which the user falls is recorded as the user's opening level. ,in This is an integer representing the user's achievable mouth opening level. Different combinations of shooting angle types and shooting distance levels require different amounts of physical space. For example, when using a tongue-side view combined with a close-up level, the camera needs to penetrate the mouth from the tongue side, requiring a larger mouth opening. Therefore, the required mouth opening level can be preset for each combination of shooting angle type and shooting distance level. ,in This indicates the minimum mouth opening level required to perform the shooting action. For each candidate shooting action... Compare user speaking levels The required mouth opening level corresponding to this action If satisfied If the physical conditions for performing the shooting action are met, it means the user possesses the necessary physical capabilities, and the candidate shooting action can be added to the executable set. If not, even if the candidate shooting action has a high overall information gain score in the virtual view, it will not be actually recommended. This approach ensures the benefits of active learning while avoiding recommending complex poses that the user cannot perform.

[0061] Finally, from the set of executable candidate shooting actions, the action with the highest information gain score is selected. The highest-ranking candidate shooting action is selected as the target shooting action for this round. Based on this target shooting action, oral cavity self-examination guidance instructions are generated. These instructions may include the target tooth region identifier number, shooting angle type, shooting distance level, and a schematic diagram file identifier used to demonstrate the location and direction. For example, for a target tooth region identifier number... The camera is positioned in the lingual view of the tooth area, and the shooting distance is set to near range. An interface diagram can be displayed, marking the lower right molar region and indicating with an arrow the direction the camera should slowly approach from the lingual side. A text prompt reads, "Approach the lower right molar from the lingual side, maintaining a distance approximately equal to the width of the tooth crown." The user terminal displays corresponding images and text on the screen based on oral self-check guidance instructions, guiding the user to complete the acquisition of the next oral color image. Through the above steps, the entire process not only mathematically approximates the strategy of maximizing expected information gain in active learning, but also takes into account whether the user can actually complete the corresponding actions, thereby minimizing the prediction uncertainty index values ​​of key dental areas in the dental area status table within a limited number of user interactions.

[0062] In an alternative implementation, the calculation of the overall information gain score can be extended. For example, dental region weights can be introduced. ,in Indicates the dental area The importance weighting is used to reflect the differences in visual appearance and function between incisors and molars. The information gain comprehensive score can be rewritten as... ,in The value can be configured based on the function and historical risk of the dental region. Such a change does not alter the overall flow of step S2; it simply favors dental regions that are more prone to long-term problems when ranking candidate shooting actions. In another optional implementation, the rotation angle and scaling ratio during virtual viewpoint feature generation can be adaptively adjusted based on the position of the candidate target dental region in the current oral color image. For example, a slightly larger rotation angle and scaling ratio can be used for dental regions near the edge of the image to simulate the user's tendency to reshoot from the side. These variations can be flexibly selected without changing the overall structure of step S2.

[0063] At the end of step S2, the target shooting action for this round has been determined, and an oral self-check guidance instruction has been generated accordingly. Step S3 begins by displaying the oral self-check guidance instruction and continues until it is determined whether to continue executing the entire closed-loop process of steps S1 and S2. Upon receiving the oral self-check guidance instruction, the user terminal displays it on the screen in a combination of text and images. The oral self-check guidance instruction includes at least three elements: the target tooth region identifier number, the shooting angle type, and the shooting distance level. For example, when the target tooth region identifier number is the lower right first molar, the shooting angle type is a lingual view, and the shooting distance level is a near distance, the user terminal can display a top-down oral cavity diagram in the center of the interface, highlighting the corresponding tooth region with a bright color, and overlaying an arrow on the diagram indicating the path from the lingual side towards the tooth region; below the image, the text explains, "Please slowly approach the lower right first molar from the lingual side, filling the central area of ​​the screen with the tooth, maintaining a distance approximately one time the width of the tooth crown." In this way, users can intuitively understand how to move the camera instead of just trying things out based on experience, which can significantly reduce the number of invalid shots.

[0064] While displaying the oral cavity self-examination guidance instructions, the user terminal switches to a live view interface, overlaying a transparent reference outline on the screen to indicate the current alignment level. The reference outline can be pre-designed based on the target tooth region identification number and the shooting angle type, for example, a simple tooth shape outline. To quantify the deviation between the current image and the reference outline, an alignment error value can be defined, denoted as . ,in This represents the alignment error between the current preview image and the reference contour. One feasible approach is to extract the dental arch contour from the real-time preview image using an edge detection operator, perform distance matching between the extracted contour points and sampling points on the reference contour, calculate the Euclidean distance between all matching points, and take the average of these distances as the mean. The value. When Less than the preset alignment error threshold When the user has essentially aligned the image with the target tooth area according to the oral self-check guidance, a green indicator can be displayed on the interface, and a photo can be automatically taken if necessary. This design avoids users taking photos at positions significantly off-target, reducing the generation of unusable or discardable color images of the mouth, which significantly benefits both user experience and algorithm efficiency.

[0065] When alignment error value When the imaging conditions are met, the user terminal acquires a new oral color image. This newly acquired image is considered the new current oral color image, and the subsequent processing flow is completely consistent with the previous round, that is, step S1 is executed again. Specifically, the newly acquired current oral color image is input into the tooth region segmentation network to generate a new tooth region segmentation mask. Tooth region image blocks are cropped according to a predetermined method and scaled to a uniform size. Then, each tooth region image block is sequentially input into a multi-model network composed of three lesion classification convolutional neural network sub-models to calculate new prediction entropy values, multi-model divergence numbers, and prediction uncertainty index values, ultimately constructing a new tooth region state table. Through this closed-loop approach, step S3 is actually continuously refreshing the tooth region state table with new information, making the tooth regions with previously high prediction uncertainty index values ​​gradually more certain with the participation of supplementary images.

[0066] After the new dental region status table is generated, it is necessary to determine whether to continue with the next round of guided imaging. For this purpose, two control parameters need to be introduced: a second threshold and a maximum number of imaging attempts. The second threshold is denoted as... ,in For the middle and The real constants between these two values ​​are used to measure the "upper limit of acceptable uncertainty"; the maximum number of shots is denoted as... ,in Let be a positive integer representing the maximum number of oral color images that the system allows to be collected in a single self-check process; the number of images already collected is denoted as . ,in In the initial state, it is This indicates that the first current oral cavity color image has been acquired. After each new image acquisition, The value increases .

[0067] When determining whether to continue guidance, we can first calculate the maximum value of the prediction uncertainty index for all dental regions in the new dental region status table, denoted as . ,in This represents the maximum value of the prediction uncertainty index across all dental regions in the current round; for dental regions identified by number 1... The dental region, and its corresponding prediction uncertainty index value is denoted as . The mathematical form can be expressed as: , where the symbol This indicates the operation of finding the maximum value. This represents the set of all dental region identifier numbers. (By viewing...) The value can quickly determine whether there are still areas where the predictive uncertainty index is at a high level. If It has already fallen below the second threshold. This indicates that the prediction uncertainty index for each dental region has been controlled within an acceptable range. From an algorithmic perspective, there is little room for further reduction in uncertainty from additional shooting, and continuing to guide shooting will increase the user's burden while yielding limited benefits.

[0068] Therefore, one of the termination conditions can be set as follows: To avoid ambiguity caused by inequality symbols, it can be understood as: for each dental region identifier number in the dental region status table All satisfy In other words, the system requires that the prediction uncertainty index values ​​for all dental regions be below the second threshold before considering the uncertainty to have "converged." If only the index values ​​for a portion of the dental regions are considered, some areas that still have a relatively high risk may be overlooked. Therefore, using maximum value aggregation is a more prudent choice.

[0069] On the other hand, even if the prediction uncertainty index values ​​for some dental areas are still slightly higher than the second threshold, continuing to require user cooperation after multiple imaging sessions will significantly reduce their willingness to use the service. Therefore, a maximum number of imaging sessions is introduced. As another termination boundary. This refers to the number of shots already taken. Reaching or exceeding Even if the predictive uncertainty index values ​​for individual dental areas fail to drop below the second threshold, the self-examination process can be terminated, and the remaining uncertainty can be addressed by a dentist during a follow-up visit. A combined condition can be used to determine whether the self-examination process should be terminated, for example, by defining an termination flag variable. Its value can be expressed as In this expression, As the end marker, when equal When the termination condition is met, it means that the termination condition is satisfied. equal This indicates that the next steps will continue; This represents the maximum value of the uncertainty index for the current round of forecasting. The second threshold, This represents the number of shots that have been taken so far. This represents the maximum number of shots. Logically, the system terminates the bootstrapping process if either of the following conditions is met: "all uncertainty has been reduced below the target," or "the number of shots has reached the maximum." This design ensures strict control over overall uncertainty while limiting the time and number of operations per use.

[0070] In a specific numerical example, a second threshold can be set. for Maximum number of shots for Assuming in the initial round At that time, obtained from the first current oral color image for It is obviously greater than the second threshold. Therefore, it is necessary to proceed to step S2 to select the target imaging action and then execute step S3 to acquire new oral cavity color images. At the end of the second round, Calculations yielded For, still higher than Continue repeating steps S2 and S3. At the end of the third round, , Down to Slightly above the second threshold; at the end of the fourth round, , Down to It is now less than Therefore, even if the maximum number of shots has not been reached... This can also trigger the termination condition, causing the system to stop subsequent guidance and prompt the user that they have completed this oral self-examination. In another scenario, until the end of the fifth round... Still Then because It is already equal to ,satisfy If certain conditions are met, the system will terminate the process and provide a message stating, "There is still a high degree of uncertainty in some dental areas; it is recommended to visit a dentist for further examination." This dual-condition control ensures both safety and user experience.

[0071] refer to Figure 1This graph uses the horizontal axis to represent the number of shots and the vertical axis to represent the number of dental regions. It presents the changing trend of the number of dental regions with high uncertainty under two different threshold conditions using a side-by-side bar chart. Specifically, the horizontal axis marks five shooting rounds from the first to the fifth shot, and the vertical axis ranges from 0 to 22 dental regions, with increments in integer units. The graph contains two sets of bar data. The first set represents the number of dental regions with a prediction uncertainty index greater than or equal to 0.50: 12 dental regions in the first shot, decreasing to 8 in the second, 5 in the third, 3 in the fourth, and 2 in the fifth. The second set represents the number of dental regions with a prediction uncertainty index greater than or equal to 0.35: 18 dental regions in the first shot, decreasing to 14 in the second, 9 in the third, 6 in the fourth, and 3 in the fifth. Each bar chart has a corresponding value labeled at the top, making it easy to visually compare the differences between different shooting rounds.

[0072] The figure clearly shows that with the iterative execution of the active learning-based oral self-examination guidance method, the number of high-uncertainty dental areas exhibits a significant decreasing trend regardless of the threshold standard used, demonstrating the effectiveness of the method in reducing overall prediction uncertainty. The first threshold of 0.50 is used to screen candidate target dental areas in step S2, while the second threshold of 0.35 is used to determine whether to terminate the iteration in step S3. The difference in their values ​​leads to different statistical results, but both show the same decreasing trend. After the first image capture, the prediction uncertainty index values ​​for most dental areas are at a high level. At this point, the system uses an active learning strategy to select the image capture action with the highest comprehensive information gain score, guiding the user to perform supplementary image captures on key dental areas. After the second and third image captures, the number of high-uncertainty dental areas decreases significantly, indicating that active learning has significant benefits in the early stages. In the fourth and fifth image captures, the number of high-uncertainty dental areas continues to decrease, but the rate of decrease slows down, reflecting the characteristic of diminishing marginal returns, which is consistent with the prediction pattern of active learning theory. The data distribution in the figure also shows that when a more lenient threshold of 0.35 is used, the system is able to maintain attention on more dental areas, ensuring that a more comprehensive uncertainty control objective is achieved before the iteration is terminated.

[0073] When it's determined that further execution is unnecessary, the user terminal can also generate a visualization based on the final round of the dental region status table. For example, the predicted uncertainty index value for each dental region can be mapped to color depth: regions with uncertainty index values ​​close to zero are drawn in light green, those close to the second threshold are drawn in yellow, and those significantly above the second threshold are drawn in orange or red. These values ​​are then overlaid on a dental arch diagram to allow users to intuitively understand which areas are relatively stable and which areas still have significant uncertainty in this self-assessment. In this optional implementation, the primary lesion category and its probability for each dental region can also be displayed to facilitate communication between the user and the dentist.

[0074] In another alternative implementation, the second threshold It doesn't need to be fixed; it can be adjusted based on the number of shots already taken. Dynamic adjustment. For example, a decreasing function can be used, allowing for lower requirements in earlier rounds and permitting higher uncertainty in exchange for more lenient stopping conditions. As the threshold increases, the second threshold is gradually tightened, making the termination conditions for later rounds more stringent. This dynamic strategy can be represented by a simple linear relationship, such as... ,in Indicates the first The second threshold used in the round, This indicates the initial threshold used in the first round. This indicates the step size for each descent round. Adjustments can be made to... and The numerical value can implement different control strategies such as "lenient first, strict later" or "strict first, lenient later" to accommodate different levels of cooperation between child and adult users. These variations do not change the basic process of step S3, but only introduce more adaptive mechanisms in the calculation of specific termination conditions.

[0075] Overall, step S3 forms a closed-loop process that gradually converges from a "high uncertainty state" to a "low uncertainty state" by displaying oral self-check guidance instructions on the interface, providing real-time alignment assistance, acquiring new current oral color images, repeating steps S1 and S2, and using a combined condition based on a second threshold and the maximum number of shots. This closed-loop structure ensures that each user's shooting action is used to minimize the predictive uncertainty of key dental areas, while controlling the overall interaction cost through the maximum number of shots.

[0076] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A self-examination guidance method for oral cavity based on active learning, characterized in that, Includes the following steps: Step S1: The user terminal acquires the current oral cavity color image. The oral cavity self-check processing module calls the tooth region segmentation network to generate a tooth region segmentation mask corresponding to the current oral cavity color image and extracts tooth region image blocks. The tooth region image blocks are input into a multi-model network composed of three lesion classification convolutional neural network sub-models. Based on the prediction entropy value output by the three lesion classification convolutional neural network sub-models and the number of multi-model divergences, the prediction uncertainty index value is calculated, and a tooth region status table is constructed. The tooth segmentation network consists of convolutional layers, downsampling layers, and pixel-level classification layers. The input to the tooth segmentation network is the current oral color image, and the output is a tooth segmentation mask that corresponds one-to-one with the pixel position of the current oral color image. Each pixel in the tooth segmentation mask contains a tooth region identifier number. The specific process of cropping tooth region image blocks includes: the oral self-checking processing module determines the bounding rectangle of each tooth region based on the connected regions of the pixels corresponding to the same tooth region identifier number in the tooth segmentation mask, crops the image content within the bounding rectangle in the current oral color image as tooth region image blocks, and stores the tooth region image blocks in the tooth region image block list after interpolation and scaling according to a uniform size. The process of calculating the prediction entropy value and the number of multi-model divergences includes: For each lesion classification convolutional neural network sub-model, the product of the probability value of each output lesion category and the logarithm of the probability value of the lesion category is taken as a term. The sum of the product results of each term is summed and the negative number is taken to obtain the prediction entropy value of the lesion classification convolutional neural network sub-model. The arithmetic mean of the prediction entropy values ​​of the three lesion classification convolutional neural network sub-models for the same dental region image block is calculated to form the average prediction entropy value of the dental region image block. In each lesion classification convolutional neural network sub-model, the lesion category with the largest probability value in the lesion category probability distribution is selected as the main prediction category of the lesion classification convolutional neural network sub-model. The number of different lesion categories appearing in the main prediction category output by the three lesion classification convolutional neural network sub-models is counted. The number of different lesion categories is taken as the number of multi-model divergences of the dental region image block. The specific process of calculating the prediction uncertainty index includes: mapping the average prediction entropy value to the interval of zero to one through linear scaling to form the entropy normalization value; dividing the number of multi-model divergences by three to form the divergence normalization value; adding the entropy normalization value and the divergence normalization value and then dividing by two to form the prediction uncertainty index value of the dental region image patch. Step S2: Filter candidate target tooth regions in the tooth region status table whose prediction uncertainty index values ​​are greater than or equal to the first threshold; generate candidate shooting actions for the candidate target tooth regions; perform virtual view feature generation on the candidate shooting actions, obtain virtual view tooth region image patches and input them into a multi-model network to calculate the virtual view prediction uncertainty index values; calculate the comprehensive information gain score of the candidate shooting actions; combine the user's mouth opening level and select the candidate shooting action with the highest comprehensive information gain score to generate oral self-check guidance instructions; The candidate imaging actions include the candidate target tooth area identification number, the imaging angle type, and the imaging distance level; the imaging angle type includes three categories: frontal view, buccal view, and lingual view; the imaging distance level includes two categories: mesial level and intermediate level; a total of six candidate imaging actions are formed for each candidate target tooth area; The process of generating virtual viewpoint features for candidate shooting actions includes: searching for the original dental region image block corresponding to the candidate target dental region identifier number of the candidate shooting action in the dental region image block list, and copying the original dental region image block as the dental region image block to be transformed; performing rotation transformation according to the shooting viewpoint type: if the shooting viewpoint type is a forward viewpoint, a horizontal rotation angle of zero is performed; if the shooting viewpoint type is a buccal viewpoint, a horizontal clockwise rotation angle of a preset first angle is performed; if the shooting viewpoint type is a lingual viewpoint, a horizontal counterclockwise rotation angle of a preset first angle is performed; performing scaling transformation according to the shooting distance level: if the shooting distance level is near, the dental region image block to be transformed is scaled according to a preset magnification ratio and restored to a uniform size using bilinear interpolation; if the shooting distance level is medium, the dental region image block to be transformed is scaled according to a preset reduction ratio and restored to a uniform size using bilinear interpolation; after the transformation, a virtual viewpoint dental region image block corresponding to the candidate shooting action is obtained. The specific process for calculating the comprehensive information gain score of candidate imaging actions includes: inputting the virtual view dental region image patch into three lesion classification convolutional neural network sub-models in sequence to obtain three new prediction entropy values, and calculating their arithmetic mean as the virtual view average prediction entropy value. At the same time, the number of different lesion categories appearing in the three new main prediction categories is counted as the virtual view multi-model divergence number; calculating the uncertainty reduction value: subtracting the virtual view prediction uncertainty index value calculated based on the virtual view average prediction entropy value and the virtual view multi-model divergence number from the original dental region image patch prediction uncertainty index value. If the calculation result is less than zero, the uncertainty reduction value is set to zero; calculating the multi-model divergence change value: the absolute value of the difference between the original multi-model divergence number and the virtual view multi-model divergence number; finding the maximum value of the uncertainty reduction value and the maximum value of the multi-model divergence change value in the calculation results of all candidate imaging actions, normalizing the values ​​of each candidate imaging action, adding the normalized uncertainty reduction value and the normalized multi-model divergence change value of the same candidate imaging action, and dividing by two to obtain the comprehensive information gain score of the candidate imaging action. The process of combining user mouth opening levels includes: acquiring oral cavity calibration images of the user in their maximum mouth opening state through the mouth opening calibration process on the user terminal; measuring the vertical distance between the corresponding pixel positions of the maxillary incisor occlusal edges and mandibular incisor occlusal edges in the oral cavity calibration image; classifying the vertical distance into multiple mouth opening levels after comparing it with a preset vertical distance threshold, and presetting the required mouth opening level for each shooting angle type and shooting distance level combination; for each candidate shooting action, reading the user's mouth opening level and comparing it with the required mouth opening level corresponding to the candidate shooting action; adding the candidate shooting action to the list of executable candidate shooting actions when the user's mouth opening level is greater than or equal to the required mouth opening level; selecting the candidate shooting action with the highest comprehensive information gain score from the list of executable candidate shooting actions; Step S3: Display oral self-examination guidance instructions, guide the user to collect new oral color images and repeat the above steps until the prediction uncertainty index values ​​of all dental areas in the dental area status table are less than the second threshold or the current number of images has reached the maximum number of images.

2. The method according to claim 1, characterized in that, The three lesion classification convolutional neural network sub-models share the same network structure, each including an input layer, a feature extraction part consisting of alternating convolutional layers and pooling layers (4 layers each), a classification part consisting of 2 fully connected layers, and a normalized output layer that outputs the probability distribution of lesion categories. The three lesion classification convolutional neural network sub-models are trained in the following ways: the first sub-model uses the original training sample set containing dental region image patches and their lesion category labels as training input; the second sub-model uses an enhanced training sample set, after random rotation, translation, and brightness adjustment of each dental region image patch in the original training sample set, as training input; the third sub-model uses the original training sample set as training input, and after each forward computation, randomly selects several convolutional feature map positions in the feature extraction part and sets their feature values ​​to fixed constants.

3. The method according to claim 1, characterized in that, In step S3, the specific process of loop control includes: after generating a new tooth area status table in each round, the oral self-check processing module counts the prediction uncertainty index values ​​of all tooth areas in the tooth area status table. If the prediction uncertainty index value of at least one tooth area is greater than or equal to the second threshold, and the number of times the current shooting has been performed is less than the maximum number of shooting, then the count value of the number of times the current shooting has been performed is increased, and the target shooting action determination step based on active learning is called again to generate a new round of target shooting action and new oral self-check guidance instructions; otherwise, the oral self-check processing module ends the execution of the oral self-check guidance method based on active learning.

Images