Occlusion abnormal characterization automatic detection method and device based on multi-view oral cavity image, and medium

Abstract

The invention discloses an occlusion abnormal representation automatic detection method and device based on a multi-view oral cavity image and a medium. The method comprises the following steps: acquiring a multi-view oral image sequence of a sample object; aiming at each target anomaly representation, according to a minimum key detection view angle group required by diagnosis, view angle subset sampling is carried out on the image sequence to construct a training sample set, a subset covering the group is marked as a positive example, and otherwise, the subset covering the group is marked as a negative example; using the set to train a multi-view adaptive fusion neural network which can receive a variable number of view images, and dynamically fusing features through a multi-view interactive attention mechanism; and finally, inputting the view angle image subset of the target object into the trained network to obtain a detection result of each abnormal representation. According to the method, the detection visual angle is fully coded into a core rule of a training sample, so that the model can learn visual features and diagnostic logic at the same time, and the detection robustness and clinical practicability under the condition that the visual angle is incomplete are remarkably improved.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of medical image analysis and artificial intelligence, and in particular to an automatic detection method, device and medium for occlusal abnormalities based on multi-view oral images. Background Technology

[0002] With the deepening application of artificial intelligence in medical image analysis, automatic detection of dental abnormalities based on deep learning has become a research hotspot. The detection of oral diseases, especially occlusal disorders (such as deep overbite, open bite, reverse overbite, and malocclusion), heavily relies on observing the spatial relationships of teeth, jawbone, and soft tissues from multiple specific angles (such as frontal, lateral, and occlusal images). Current technical solutions mainly suffer from the following limitations: 1. Data Bottlenecks and Insufficient Model Robustness: High-quality, well-labeled multi-view oral imaging datasets are limited in size. Most studies use single-view images for classification or detection, ignoring the multi-view dependency in occlusal diagnosis. Even when using multi-view data, it is usually just a matter of inputting images from multiple views into the model separately and then fusing them for decision-making, failing to explicitly model the crucial clinical knowledge—that specific view combinations are necessary and sufficient for the diagnosis of specific conditions—at both the data and model levels. This results in poor generalization ability of the model in real-world scenarios with missing or incomplete viewpoints, making it prone to misjudgment due to local features.

[0003] 2. Limited Sample Construction and Supervision Signals: Existing supervised learning methods typically perform binary classification (symptom-positive / symptom-free) based on a single image or a fixed combination of viewpoints. This annotation method loses the dimension of whether the detection is based on sufficient observation viewpoints. The model may learn spurious features (such as specific lighting or background) unrelated to the disease from an image from a non-critical viewpoint, leading to overfitting and clinical unreliability.

[0004] 3. Poor support for multi-view variable inputs: Traditional convolutional neural networks (CNNs) typically require fixed-size and fixed-number inputs. When processing variable-number multi-view images, it is often necessary to pre-specify the order, perform padding, or use fixed pooling strategies, making it impossible to dynamically evaluate the relative importance and complementarity between different viewpoints.

[0005] 4. Disconnect between detection and localization: Current technologies often treat lesion detection and tooth localization as two separate tasks. Detection models can only determine the presence or absence of lesions, but cannot precisely locate the specific affected tooth (FDI number); while localization models are not concerned with lesion information. This separate approach does not align with the integrated diagnostic thinking of clinicians in discovering abnormalities and locating lesions, thus reducing the practical value of auxiliary detection.

[0006] Therefore, there is an urgent need for an intelligent dental disease auxiliary detection method and system that can effectively utilize limited multi-view data, explicitly encode view-disease association knowledge, robustly handle variable view input, and achieve coordinated disease detection and precise positioning. Summary of the Invention

[0007] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows: According to a first aspect of the present invention, an automatic detection method for occlusal abnormalities based on multi-view oral imaging is provided, the method comprising the following steps: S100: Acquire raw multi-view oral cavity image sequences of multiple sample objects.

[0008] S200: For each type of target occlusal abnormality, a training sample set is constructed by sampling the original multi-view oral image sequence of each sample object according to the minimum key detection view group required for detecting the target occlusal abnormality. If the view type contained in a training sample covers the corresponding minimum key detection view group, it is marked as a positive example sample of the abnormality. Otherwise, it is marked as a negative example sample.

[0009] S300, using the training sample set, a multi-view adaptive fusion neural network is trained. The multi-view adaptive fusion neural network is configured to receive a variable number of view images and dynamically integrate image features related to the target occlusal anomaly representation through an internal multi-view interactive attention mechanism.

[0010] S400, a subset of one-view images of the target object is input into the trained multi-view adaptive fusion neural network to obtain detection results on the occlusal anomalies of each target.

[0011] According to a second aspect of the present invention, an electronic device is provided, including a processor and a memory; the processor executes the steps of the method described in the first aspect of the present invention by invoking a program or instructions stored in the memory.

[0012] According to a third aspect of the present invention, a computer-readable storage medium is provided that stores a program or instructions that cause a computer to perform the steps of the method described in the first aspect of the present invention.

[0013] The present invention has at least the following beneficial effects: (1) By constructing a sample based on the minimum key detection perspective group, the clinical diagnosis logic (i.e., the need for a specific combination of perspectives to make a diagnosis) is transformed into a learnable supervision signal, which fundamentally solves the problem that the model is difficult to learn multi-perspective dependencies under small sample conditions and greatly enhances the generalization ability of the model.

[0014] (2) The multi-view adaptive fusion neural network used can flexibly process any number and combination of input views, and dynamically focus on key view information through the attention mechanism. Thus, when faced with view missing or incomplete data commonly found in clinical practice, it can still maintain high-precision detection capability and output reliable uncertainty assessment.

[0015] (3) This method realizes the leap from passive analysis to active interaction. When the detection uncertainty is high, it can automatically suggest the necessary perspective for re-shooting, forming a detection closed loop, effectively assisting clinical decision-making, and improving detection efficiency and automation level.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart illustrates an automatic detection method for occlusal abnormalities based on multi-view oral imaging, as provided in this embodiment of the invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0021] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0022] This invention provides an automatic detection method for occlusal abnormalities based on multi-view oral imaging, such as... Figure 1 As shown, the method includes the following steps: S100: Acquire raw multi-view oral cavity image sequences of multiple sample objects.

[0023] Specifically, the sample subjects refer to individuals undergoing oral examinations (such as patients or subjects). In this embodiment, to construct the training dataset, it is necessary to collect a sufficient number (e.g., hundreds to thousands) of oral imaging data of the sample subjects and ensure that the necessary ethical review consent and privacy anonymization processing have been obtained.

[0024] The acquisition of the original multi-view oral imaging sequence is achieved through standardized oral imaging equipment and procedures. Typically, professional physicians or technicians use equipment such as intraoral scanners, dental radiography systems, or cone-beam computed tomography (CBCT) machines, following a pre-defined, standardized imaging protocol covering the angles required for diagnosis. A complete sequence usually contains a set of multi-view renderings of two-dimensional color images or three-dimensional oral models taken from different fixed angles, such as: frontal smiling images, frontal occlusal images, left and right lateral occlusal images, and maxillary / mandibular arch occlusal images. Each image carries metadata identifying its viewpoint type (e.g., "frontal", "left_buccal"). Images in the sequence have uniform resolution, color space, and file format (e.g., .jpg, .png, or DICOM format) to ensure data consistency and facilitate subsequent processing.

[0025] S200: For each type of target occlusal abnormality, a training sample set is constructed by sampling the original multi-view oral image sequence of each sample object according to the minimum key detection view group required for detecting the target occlusal abnormality. If the view type contained in a training sample covers the corresponding minimum key detection view group, it is marked as a positive example sample of the abnormality. Otherwise, it is marked as a negative example sample.

[0026] In this embodiment of the invention, the target occlusal abnormalities specifically refer to certain types that require identification and evaluation through multi-view oral imaging, reflecting deviations from the normal physiological or ideal state of the relationship between the maxillary and mandibular dentitions and jawbones. These abnormalities are key diagnostic objects in orthodontics, prosthodontics, and other branches of dentistry, such as deep overbite, deep overjet, reverse overbite, open bite, locked bite, and malocclusion. Each abnormality has characteristic morphological features that can only be clearly observed and measured from a specific perspective.

[0027] In this embodiment of the invention, for each type of target occlusal abnormality, a minimum set of critical detection viewpoints required for its detection is predefined and stored. This set of viewpoints is determined by domain experts based on clinical diagnostic pathways and represents a minimum combination of viewpoint types necessary and sufficient for radiographic confirmation of the abnormality (e.g., diagnosing "deep overgrowth" may require a combination of "frontal" and "lateral" images). These predefined rules are structured and stored in the system's diagnostic rule database in the form of mapping tables or knowledge entries, providing the core logical basis for subsequent sample construction.

[0028] In S200, based on the above rules, a viewpoint subset is sampled from the complete image sequence of each sample object to automatically construct a training sample set. In specific implementation, any one or a combination of the following strategies can be used: Random sampling strategy: Randomly select one or more non-empty subsets from the complete view sequence of the object to simulate various view missing situations that may occur in clinical practice, thereby increasing the diversity of training data.

[0029] Traversal sampling strategy: To fully utilize limited data, systematically generate all non-empty proper subsets of the complete view sequence. The mathematical meaning of this operation is: assuming a complete view sequence of a sample object contains n different view types, then the number of all possible non-empty proper subsets is 2n. n -1. The algorithm automatically enumerates and generates image combinations corresponding to all these subsets. For example, if an object sequence contains three viewpoints {front, left, right}, then all its non-empty proper subsets are: {front}, {left}, {right}, {front, left}, {front, right}, {left, right}, {left, right}, {front, left, right}. This method ensures coverage of all possible viewpoint combinations, especially the boundary cases that either exactly satisfy or do not satisfy the minimum key detection viewpoint group, thus forcing the model to accurately learn the logical relationship between viewpoint sufficiency and anomaly representation.

[0030] Using the above strategy, each sampled subset of viewpoints (i.e., a training sample) is labeled based on whether the viewpoint types it contains completely cover the minimum key detection viewpoint group of the target anomaly representation: if completely covered, it is labeled as a positive example of the anomaly representation; otherwise, it is labeled as a negative example. This process directly encodes high-order clinical diagnostic logic (i.e., "which viewpoints are needed to make a judgment") into machine-learnable supervisory signals, which is the core of this method to achieve highly robust detection.

[0031] S300, using the training sample set, a multi-view adaptive fusion neural network is trained. The multi-view adaptive fusion neural network is configured to receive a variable number of view images and dynamically integrate image features related to the target occlusal anomaly representation through an internal multi-view interactive attention mechanism.

[0032] In this embodiment of the invention, S300 uses the constructed training sample set to train a multi-view adaptive fusion neural network. The core characteristics and structure of this network are as follows: 1. Basic Network Configuration and Adaptive Fusion Mechanism The network is designed to accept a variable number of viewpoint images as input. To achieve effective integration of cross-viewpoint information, its core fusion module employs a multi-viewpoint interactive attention mechanism based on the Transformer architecture. Specifically, the network first encodes each input viewpoint image independently into a corresponding feature map through a shared convolutional neural network (CNN) backbone such as ResNet. After flattening, a set of deep feature vector sequences X=[x1,x2,...,xN] is obtained, where N is the number of input viewpoints, and xi∈R. d , representing the d-dimensional feature vector of the i-th viewpoint. These feature vectors from different viewpoints are considered as a sequence of feature vectors.

[0033] Subsequently, the correlation between these feature vectors is calculated through the Multi-Head Cross-Attention layer in the Transformer. Its core operation can be summarized as follows: for each view's feature vector (as the query vector), it interacts with the feature vectors of all other views (as key and value vectors) through attention weights, thereby generating a new feature representation for each view that incorporates global contextual information.

[0034] Finally, the fused features from different perspectives are weighted and aggregated using learnable weights to generate a comprehensive, discriminative feature representation for final decision-making. This mechanism enables the network to dynamically evaluate and integrate image features most relevant to the target bite anomaly representation, without being limited by the fixed number or order of perspectives.

[0035] 2. Multi-task collaborative learning architecture and training process In a preferred embodiment, the multi-view adaptive fusion neural network is constructed as a multi-task learning network designed to simultaneously and collaboratively complete two closely related tasks: occlusal anomaly representation detection (primary task) and tooth instance localization and FDI numbering (auxiliary task). The two tasks share a feature extraction layer comprised of the aforementioned CNN backbone and multi-view attention fusion module to learn a general and robust representation of oral image features.

[0036] The network is trained by minimizing a multi-task joint loss function L. total To accomplish this, the function is a weighted sum of the losses from each task: L total =λ cls ×L cls +λ Loc ×L Loc +λ num ×L num , where L cls The classification loss (such as cross-entropy loss or Focal Loss) for anomaly detection corresponds to the network's output representing the presence or absence of predefined anomalies of various types. Loc For tooth localization, segmentation loss (such as a combination of Dice Loss and cross-entropy loss) or bounding box regression loss from object detection are typically used to optimize the generation of tooth instance segmentation maps. num The classification loss is λ, which predicts the FDI number for each detected tooth instance. cls , λ Loc , λ num This is a hyperparameter used to balance the learning speed and importance of different tasks.

[0037] The training process uses the standard backpropagation algorithm and a stochastic gradient descent optimizer (such as Adam). In each iteration, a batch of constructed samples (containing a subset of viewpoints and their positive / negative labels) is input into the network. The shared CNN backbone and multi-view attention module first extract and fuse global features. Subsequently, the features are fed into two task heads for computation, and L is jointly computed. total By optimizing the joint loss function, the network simultaneously learns to accurately detect abnormal representations, locate teeth, and identify their numbers. The two tasks complement each other through shared representations and subsequent feature interactions.

[0038] 3. Inter-task feature interaction module To achieve knowledge complementarity and mutual enhancement between the two tasks, the network introduces a dedicated feature interaction module. This feature interaction module is designed as follows: (1) Anomaly detection guides tooth localization: The anomaly characterization detection branch can generate a spatial attention heatmap A∈R that reflects the probability of anomalies in the intermediate or output layer. H×W H and W represent the spatial dimensions of the feature map, with highlighted areas indicating potential locations of anomalies. This heatmap modulates the intermediate feature map FM of the tooth localization branch through element-wise multiplication (or as weights in the SE module): FM C =A⊙FM. Where ⊙ represents element-wise multiplication, and FM... C This is the feature map after heatmap weighting. This operation is equivalent to injecting spatial attention priors from the anomaly detection task into the tooth localization branch, enabling it to focus on tooth regions more relevant to anomalies, thereby improving localization accuracy and computational efficiency.

[0039] (2) Tooth localization assists in anomaly detection: The tooth localization and numbering branch can output the instance segmentation mask and its standard FDI number (e.g., "36") for each tooth, thereby further abstracting the spatial topological information of the teeth (e.g., adjacency relationships, dental arch morphology). This structured information is organized into a graph structure, where nodes represent a tooth, and their node features are the depth features corresponding to the tooth region; edges represent the spatial adjacency relationships between teeth. This graph structure is then input into a graph convolutional network, which performs message passing and feature aggregation between nodes through multi-layer graph convolution operations, thereby modeling the structural dependencies between teeth. The graph features that the graph convolutional network finally outputs, containing the topological relationships of all teeth, will be fed into the anomaly representation detection branch, providing it with logical reasoning support based on anatomical structure, helping the model distinguish between local image artifacts and real anomaly patterns with structural correlation, and improving the robustness and interpretability of the detection.

[0040] Through the above architecture, the two tasks promote each other during the training process, realizing a closed-loop judgment of where abnormalities are seen and which tooth is abnormal, which significantly improves the overall diagnostic accuracy and clinical applicability of the system.

[0041] S400, a subset of one-view images of the target object is input into the trained multi-view adaptive fusion neural network to obtain detection results on the occlusal anomalies of each target.

[0042] Specifically, the target object refers to a new individual (such as a clinical patient) to be assessed for occlusion. The acquisition method, viewpoint type definition, and data format of their oral images are consistent with the sample object image sequence standard described in S100 to ensure that they match the data distribution learned by the training model.

[0043] During the inference phase, the image data of the target object does not need to cover all preset viewpoints. A subset of one-view images of the target object is received as input; this subset can be any combination of non-empty viewpoints obtained based on actual clinical imaging (e.g., containing only frontal and left-side images). The trained multi-view adaptive fusion neural network will automatically analyze this subset.

[0044] The trained multi-view adaptive fusion neural network first dynamically fuses cross-view information from the subset through its internal multi-view attention mechanism to evaluate the sufficiency of the current view combination in detecting various abnormal representations. Finally, the network outputs the detection result for each predefined target occlusal abnormal representation (such as deep overbite, underbite, etc.). This detection result is typically expressed as a binary judgment (yes / no) or a probability value indicating the possibility of existence (e.g., 0.85).

[0045] Furthermore, the multi-view adaptive fusion neural network also outputs a detection uncertainty metric U based on a subset of the current input viewpoint images, which is used to quantify the network's confidence in making a reliable detection under the current input conditions.

[0046] A typical way to calculate uncertainty measures is based on the probability distribution of the model output. For example, for a target bite anomaly representation, the network outputs a probability of its existence of p (0≤p≤1). The uncertainty measure U can be obtained by calculating entropy or confidence variance. A common definition is: U=1-|2p-1|, that is, when the probability p is close to 0.5 (the network is most uncertain), the U value is close to the maximum value of 1; when p is close to 0 or 1 (the model is very certain), the U value is close to the minimum value of 0.

[0047] Based on this, the method further includes the following steps: S500, Interactive diagnostic suggestion generation. When the calculated detection uncertainty metric U is higher than a preset threshold T, it is determined that the current perspective information is insufficient to make a high-confidence detection, that is, the anomaly representation is determined to be in a high-uncertainty state under the current input. The suggestion generation mechanism is then triggered. The preset threshold T is determined in advance through performance analysis on the validation set. For example, a critical value is selected that ensures that when U≤T, the network prediction accuracy is higher than a specific requirement (such as 95%).

[0048] The logic for generating suggestions is as follows: Step 1: Screening for high uncertainty characterization It iterates through each predefined target anomaly representation (e.g., deep overbite, underbite, open bite, etc.) in the detection rule base. For each representation, it determines whether its corresponding detection uncertainty metric is greater than a preset decision threshold. All anomaly representations that meet this condition are constructed into a high-uncertainty representation set.

[0049] Step Two: Analysis of Missing Necessary Perspectives For each anomalous representation in this set of high-uncertainty representations, perform the following independent operation: query its corresponding minimum key detection viewpoint group and compare it with the currently input viewpoint subset to calculate an independent list of missing viewpoints for that specific representation. At this point, multiple (potentially overlapping) lists of independent missing viewpoints will be obtained.

[0050] Step 3: Suggestions for Optimization and Merging Calculate the critical intersection: Obtain the intersection view of all independent missing lists. This type of intersection view is the "critical" view that can solve multiple uncertainties simultaneously. Re-enhancing these views is the most efficient and should be given the highest priority.

[0051] Merging and Redundancy Removal: The above-mentioned intersection viewpoints are merged with other necessary viewpoints that only appear in a single independent list of missing viewpoints to form a deduplicated global set of missing viewpoints.

[0052] Priority ranking: Each item in the global missing perspective set is ranked to provide clear guidance for clinical practice. The ranking logic is as follows: Determine the resolution power of each perspective: Based on the list of independent missing perspectives generated in step two, count the frequency of each perspective appearing in all lists. This frequency value is the resolution power of the set of uncertainties that the perspective can resolve. For example, if "right occlusal lateral view" appears in both the "deep overbite" and "jaw deviation" missing perspectives, its resolution power is 2.

[0053] Sorting can be performed using any one or a combination of the following strategies: a) Based on the effectiveness of problem-solving in descending order: the perspective that can solve the most uncertainties is given priority.

[0054] b) Based on the pre-set clinical acquisition cost in ascending order: priority is given to shooting from more convenient and common angles (such as "frontal view" usually takes precedence over "maxillofacial dynamic recording" at a specific angle).

[0055] c) Based on the weighted composite score: assign different weights to factors such as efficacy and clinical convenience, calculate the composite score and rank them.

[0056] Ultimately, the system generates a supplementary perspective recommendation sequence with clear priorities.

[0057] Step 4: Generation and Output of Structured Suggestions Based on the generated priority sequence of supplementary viewpoints, a structured suggestion for supplementary imaging is produced. This suggestion clearly lists the recommended viewpoints and their clinical purposes, and can be marked by priority.

[0058] Example 1 (Multiple features overlap): "The primary recommendation is to take a lateral view of the right occlusion, as this perspective helps to simultaneously assess the possibility of 'deep overbite' and 'facial deviation'." Example 2 (no clear overlap): "It is recommended to take additional photos from the following perspectives in order: 1. Frontal smiling image (for assessing midline deviation); 2. Maxillary shin profile image (for assessing dental arch morphology)." This step elevates the process from passive analysis to proactive interactive auxiliary detection. By quantifying uncertainty and linking it to a pre-defined rule base, it enables intelligent guidance in scenarios with incomplete information, effectively improving the efficiency of clinical workflows and the reliability of testing.

[0059] Furthermore, prior to S200, the following steps are also included: S110, using a self-supervised learning method, a feature encoding network is pre-trained on a number of unlabeled multi-view oral images greater than a first preset number; wherein, the feature extraction part of the multi-view adaptive fusion neural network inherits or is initialized from the pre-trained feature encoding network.

[0060] This step aims to utilize large-scale unlabeled data to provide an initial model with good general feature representation capabilities for subsequent supervised training. The core of self-supervised learning lies in designing a pre-training task that does not require manual annotation, enabling the model to discover patterns from the data's own structure and learn high-quality feature representations. In this embodiment, the inherent characteristics of multi-view oral imaging sequences are fully utilized, and cross-view consistency contrast learning is used as the pre-training task.

[0061] Specifically, the first preset quantity should be set such that the total amount of unlabeled multi-view oral images used is sufficient to allow the self-supervised pre-training task to fully converge, and the pre-trained feature encoding network to exhibit significantly better performance than the randomly initialized model on the downstream supervised task. In practice, the performance improvement curve of the downstream task as the amount of pre-training data increases can be monitored on a reserved validation set, and the data size corresponding to the performance plateau can be selected as a reference.

[0062] The specific implementation method of this step includes: (1) Constructing positive and negative sample pairs: From the unlabeled dataset, a complete multi-view image sequence of an object (e.g., a patient) is randomly selected. Two images from different views are randomly selected from this sequence to form a positive sample pair. This positive sample pair naturally contains information about the same oral anatomical structure under different viewing angles and has a high semantic relevance. At the same time, two images belonging to different objects (regardless of whether their views are the same) are randomly selected to form a negative sample pair. The construction of all sample pairs depends only on the object attribution relationship of the images and does not depend on any manually labeled diagnostic or segmentation labels.

[0063] (2) Define the pre-training objective and loss function: Based on the contrast sample pairs constructed above, the task objective of the self-supervised pre-training is to drive the feature encoding network to learn a feature representation such that the representations of images of the same object from different perspectives (positive sample pairs) in the feature space are as similar as possible, while the representations of images of different objects (negative sample pairs) in the feature space are as distinct as possible.

[0064] The feature encoding network can be any neural network architecture capable of extracting deep feature representations from a single image. In this embodiment, mature deep convolutional neural networks (such as ResNet and DenseNet) or Vision Transformers (ViT) and their variants can be optionally used as the backbone structure of the feature encoder, but are not limited to these. The specific choice depends on a comprehensive consideration of computational efficiency, feature representation capability, and adaptability to downstream tasks.

[0065] To achieve this goal, the network is trained by optimizing a contrastive loss function. In this embodiment, the InfoNCE (Noise-Contrastive Estimation) loss is used as an example. For a training batch, let the batch size be N, containing a positive sample pair (i.e., from the same object) consisting of image i and image j, then for anchor sample i, the loss L i The calculation is as follows: L i =-log[exp(sim(z)] i , z j ) / τ) / ∑ N k=1 I [k≠i] exp(sim(z) i , z k ) / τ)].

[0066] Among them, z a This represents the feature vector extracted and projected from the corresponding image using a feature encoding network, where 'a' is i, j, or k. 'sim()' is the similarity metric function (e.g., cosine similarity), and τ is an adjustable temperature hyperparameter (τ > 0) used to control the discrimination strength for difficult negative samples. [k≠i] It is an indicator function, with a value of 1 when k ≠ i (i.e., sample k is considered a negative sample) and a value of 0 when k = i (i.e., it excludes itself). In actual calculations, all other N-1 samples in the batch except j are usually considered as negative samples of sample i.

[0067] The loss for the entire batch is the average of the losses of all anchor samples within the batch. Minimizing this loss function is equivalent to driving the network to output high similarity scores for positive sample pairs and low similarity scores for negative sample pairs, thereby achieving the pre-training objective.

[0068] (3) Network training: Using the unlabeled dataset, the feature encoding network is trained fully through the above pre-training task until the model converges.

[0069] (4) After pre-training, the weight parameters of the obtained feature encoding network are saved. In step S300, when constructing the multi-view adaptive fusion neural network, the feature extraction part (i.e., the backbone network) used to extract features from single-view images will initialize the weights of the pre-trained feature encoding network, which can be fine-tuned in subsequent supervised training. Through this step, the model can grasp the basic anatomical structure and visual pattern commonality of oral images in advance, thus laying a solid foundation for efficient and robust learning of specific anomaly detection tasks on small-scale labeled data.

[0070] It should be noted that in the field of computer vision, pre-training feature encoding networks using self-supervised learning is an existing technology. Its general paradigm involves designing proxy tasks (such as image inpainting, rotation prediction, and instance discrimination) on large-scale unlabeled image sets to learn general feature representations. Among these, contrastive learning-based methods (such as SimCLR and MoCo) learn by constructing image augmentation views as counterparts, and have been widely verified to significantly improve the performance of downstream tasks. In medical image analysis, this paradigm has also been applied to the pre-training of modalities such as X-ray and CT scans.

[0071] In step S110, the present invention represents a creative adaptation and application of the aforementioned prior art paradigm, specifically embodied in: Specificity of data objects: This invention targets multi-view oral imaging sequences, whose data have a natural structured correspondence (multi-view images of the same object belong to the same semantic entity).

[0072] Targeted design of pre-training tasks: The “cross-view consistency contrast learning” task adopted in this invention directly utilizes the inherent correlation of multi-view sequences to construct positive sample pairs (different views of the same object). This is more in line with the multi-view and structured characteristics of oral images than the random image enhancement method used in general contrast learning, and can more effectively learn oral anatomical semantic features that are independent of view.

[0073] Synergistic effect in the overall scheme: This pre-training step serves as an optimization component of the overall method of this invention. Its purpose is to provide a better model initialization starting point for subsequent steps (i.e., sample construction based on the "minimum key detection perspective group" and training of multi-view adaptive fusion network), thereby synergistically improving the performance and data utilization efficiency of the final detection system.

[0074] Therefore, the self-supervised pre-training method of feature encoding networks is an application of existing technology, while the core innovation of this invention lies in its subsequent sample construction method based on diagnostic logic, multi-view adaptive fusion network architecture, and overall interactive detection process.

[0075] Furthermore, the method also includes the following steps: S120: For a newly added target occlusal anomaly representation, based on a number of labeled samples less than the second preset number, by analyzing the importance of different perspective features in distinguishing categories, automatically learn and recommend the initial candidate set of the minimum key detection perspective group corresponding to the newly added target occlusal anomaly representation.

[0076] This step aims to address how, when introducing a new target occlusal anomaly representation, with very little labeled data (the number of samples is less than a second preset number), to automatically and data-drivenly infer and recommend an initial candidate set of the minimum critical detection perspectives necessary for its diagnosis, thereby efficiently constructing or expanding the diagnostic rule base.

[0077] The second preset quantity represents the upper limit of labeled data for few-shot learning and initial viewpoint group recommendations. This quantity should be significantly smaller than the first preset quantity (i.e., the second preset quantity is much smaller than the first preset quantity), ensuring that the number of labeled samples used is within the typical range of few-shot learning. Typically, this order of magnitude should make it difficult to directly train a robust full-viewpoint supervised model; for example, the number of labeled samples per category should be in the tens to hundreds, but sufficient to support reliable statistical significance analysis or training a lightweight surrogate classifier to evaluate viewpoint importance. The specific value can be adjusted according to the difficulty of acquiring data for adding anomaly representations in practical applications and the system's requirements for initial recommendation accuracy.

[0078] The specific implementation method of this step includes the following process: (1) Data preparation and feature extraction: Collect a small number of labeled samples (positive examples) and corresponding normal samples (negative examples) of the newly added anomaly representation. All samples contain their multi-view image sequences. Using the feature encoding network pre-trained in step S110, extract the depth feature vector of each view image in each sample.

[0079] (2) Quantitative analysis of perspective importance: The importance of each perspective type is quantified by analyzing the contribution of features from different perspectives to distinguishing the newly added abnormal representation from the normal category. One or more of the following methods can be used: Analysis based on statistical tests: Calculate the distributional differences of each perspective feature across the positive and negative sample sets (e.g., calculate the p-value using a t-test, or calculate the distance between the feature means). The more significant the difference between perspectives, the stronger their discriminative power and the higher their importance.

[0080] Analysis based on surrogate classifiers: For each viewpoint type, a lightweight classifier (such as linear SVM or logistic regression) is trained separately using the features of that viewpoint, and its classification performance (such as accuracy and AUC) is evaluated on a small validation set. The better the performance of a viewpoint, the more important the information it provides.

[0081] Analysis of attention weights based on multi-view model: A simple multi-view fusion model (such as a shallow network with an attention mechanism) is trained using features from all viewpoints. The distribution of attention weights learned by the model for different viewpoints is analyzed. Viewpoints with consistently high attention weights are considered to be of higher importance.

[0082] (3) Generation and output of the initial candidate viewpoint group: Based on the results of the aforementioned perspective importance quantification analysis, one or more initial candidate minimal key detection perspective groups are generated for recommendation to domain experts. The generation of these initial candidate minimal key detection perspective groups is achieved by executing a specific automated algorithm strategy, which can be done in any of the following ways: Threshold screening method: Set an importance score threshold. Views with importance scores higher than this threshold across all view types are automatically grouped into an initial candidate minimum key detection view group.

[0083] Top-K selection method: Directly select the top K view types in importance ranking to form an initial candidate minimum key detection view group of size K. The value of K can be preset based on experience or determined by simulation through a small-scale validation set: that is, try different values ​​of K on a small amount of labeled data and select the value of K that enables a lightweight classifier to achieve the best performance.

[0084] Iterative Gain Method: Starting with an empty set of viewpoints, viewpoints are added to the set in descending order of importance. Each time a viewpoint is added, the classification performance of the current set in distinguishing the anomalous representation from normal samples is simulated and evaluated (e.g., using features from a subset of viewpoints to train and validate a classifier). The iteration stops when the performance improvement brought by adding a new viewpoint is less than a preset gain threshold, and the set of viewpoints added at this point is used as the initial candidate minimum key detection viewpoint group.

[0085] (4) Expert confirmation and application: The generated initial candidate minimal key detection perspective group (which may be one or multiple candidate groups generated by various algorithms for comparison) is output and submitted to domain experts for review and final confirmation. Experts, based on their clinical knowledge, judge the necessity, sufficiency, and clinical operability of the candidate groups, and may adopt, modify, or reject the system's recommendations. The final perspective group, confirmed by the experts, will be officially entered into the detection rule base as the minimal key detection perspective group for this newly added abnormal representation. This completes the diagnostic logic configuration for this new representation, enabling it to be immediately used in the sample construction and model training process of subsequent steps (S200).

[0086] The technical advantage of this step is that it frees domain experts from the arduous task of manually defining perspective rules, provides data-driven decision support through small-sample learning, greatly improves the efficiency and intelligence of expanding new diseases, and makes the construction of the rule base more objective.

[0087] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0088] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0089] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0090] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An automatic detection method for occlusal abnormalities based on multi-view oral imaging, characterized in that, The method includes the following steps: S100, acquire the original multi-view oral cavity image sequence of multiple sample objects; S200, for each type of target occlusal abnormality, according to the minimum key detection viewpoint group required for detecting the target occlusal abnormality, the original multi-view oral image sequence of each sample object is sampled for viewpoint subset to construct a training sample set; wherein, if the viewpoint type contained in a training sample covers the corresponding minimum key detection viewpoint group, it is marked as a positive example sample of the abnormality; otherwise, it is marked as a negative example sample. S300, using the training sample set, a multi-view adaptive fusion neural network is trained. The multi-view adaptive fusion neural network is configured to receive a variable number of view images and dynamically integrate image features related to the target occlusal anomaly representation through an internal multi-view interactive attention mechanism. S400, a subset of one-view images of the target object is input into the trained multi-view adaptive fusion neural network to obtain detection results on the occlusal anomalies of each target.

2. The method according to claim 1, characterized in that, In S200, the sampling of view subsets for the original multi-view oral image sequence of each sample object specifically includes: randomly selecting one or more non-empty view subsets from the original multi-view oral image sequence; or, traversally generating all non-empty true subsets of the original multi-view oral image sequence as view subsets.

3. The method according to claim 1, characterized in that, The multi-view interactive attention mechanism is a cross-attention mechanism based on the Transformer architecture; the multi-view adaptive fusion neural network treats image features from different perspectives as different feature vectors and calculates the attention weights between different feature vectors to achieve dynamic weighted fusion of cross-view features.

4. The method according to claim 3, characterized in that, The multi-view adaptive fusion neural network also outputs a detection uncertainty measure based on a subset of the view images currently input; The method further includes the following steps: S500, when the detection uncertainty metric is higher than a preset threshold, based on the minimum key detection viewpoint group corresponding to the target bite anomaly characterization, generating supplementary shooting viewpoint suggestion information for the target object.

5. The method according to claim 1, characterized in that, The multi-view adaptive fusion neural network is a multi-task learning network, configured to simultaneously perform the occlusal anomaly detection task and the tooth localization and numbering task; the anomaly detection task and the tooth localization and numbering task share a feature extraction layer and interact with each other through a feature interaction module.

6. The method according to claim 5, characterized in that, The feature interaction module is configured to use the feature heatmap generated by the anomaly characterization detection task as an attention weight to guide the tooth localization and numbering task to focus on the relevant tooth region. And / or, the tooth spatial topology information output by the tooth localization and numbering task is used as input to a graph convolutional network to assist the anomaly representation detection task in performing reasoning based on tooth adjacency relationships.

7. The method according to claim 1, characterized in that, Before S300, the following steps are also included: using a self-supervised learning method, a feature encoding network is pre-trained on a number of unlabeled multi-view oral images that is greater than a first preset number; wherein, the feature extraction part of the multi-view adaptive fusion neural network inherits or is initialized from the pre-trained feature encoding network.

8. The method according to claim 1, characterized in that, The method further includes the following steps: for a newly added target occlusal anomaly representation, based on a number of labeled samples less than a second preset number, by analyzing the importance of different perspective features in distinguishing categories, automatically learning and recommending an initial candidate set of the minimum key detection perspective group corresponding to the newly added target occlusal anomaly representation.

9. An electronic device, characterized in that, Including processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 8 by invoking programs or instructions stored in the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 8.

Images