Smart oral examination interaction guidance system and method based on augmented reality

Abstract

The application discloses an intelligent oral examination interactive guidance system and method based on augmented reality, collects oral images through a front camera of a smart terminal device and an oral examination application program, the application program comprises a device posture detection model, a display interface adjustment module, an augmented reality rendering module, an oral image analysis module and a virtual character interactive control module, a device inversion holding mode is innovatively adopted, a virtual dentist character is superimposed and displayed in real-time images in combination with an augmented reality technology, the virtual dentist character guides a user to adjust a shooting angle, a distance and light conditions through intuitive actions and expressions, and acquires an inside oral image in combination with a low-cost auxiliary dental mirror tool. The system also analyzes image quality in real time, the virtual character carries out targeted guidance, and finally, high-quality oral images are acquired. The feasibility and effect of oral self-examination of ordinary users under the lack of professional knowledge background are greatly improved, and effective technical support is provided for remote oral medical treatment and personal oral health management.

Description

Technical Field

[0001] This invention relates to the field of oral medical technology, and in particular to an intelligent oral examination interactive guidance system and method based on augmented reality. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] Oral health is an important component of overall human health, and regular dental checkups are crucial for the early detection of oral diseases and the prevention of oral health problems. Traditional dental checkups typically require patients to visit a dental clinic in person, where a professional dentist will use specialized tools such as dental endoscopes and probes. While this method is highly accurate, it suffers from high costs, significant time commitment, and low frequency, resulting in many oral diseases going undetected and untreated.

[0004] With the popularization of mobile internet technology and smart devices, telemedicine and self-health management are gradually becoming important development trends. In the field of oral health, some applications have emerged that use smartphone cameras for oral image acquisition and analysis. However, these applications currently suffer from the following technical problems: First, ordinary users lack professional oral medical knowledge and cannot accurately grasp the shooting angle and key areas, resulting in poor quality oral images that are not conducive to subsequent analysis; second, the internal structure of the oral cavity is complex, and lighting conditions are limited, making it difficult for ordinary users to operate the phone camera to obtain clear images of the oral cavity, especially in areas such as the posterior teeth and sides; third, children often have a fear of oral examinations and are difficult to cooperate in completing effective oral image acquisition; finally, existing applications lack real-time interactive guidance functions, and users are prone to making mistakes during operation without realizing it.

[0005] On the other hand, Augmented Reality (AR) technology, which overlays virtual information onto the real environment in real time, has been widely used in fields such as healthcare, education, and gaming. In the medical field, AR technology is used for surgical navigation and medical education, but its application in oral self-examination and remote oral healthcare is still in the exploratory stage. Existing technologies have not yet effectively solved the problem of how to provide intuitive and user-friendly oral examination guidance to ordinary users lacking professional knowledge using AR technology to ensure high-quality oral images. Furthermore, due to the conventional way smartphones are held (with the camera above the screen), users find it difficult to simultaneously view the screen and control the shooting angle when taking oral selfies, making operation inconvenient and prone to errors. Although some applications attempt to assist users through voice commands or text prompts, these methods lack intuitiveness and interactivity, resulting in a poor user experience, especially for special groups such as children and the elderly.

[0006] Existing technology discloses an intelligent scanning method and system based on a digital dental impression instrument. This system mainly analyzes user-uploaded oral photos using computer vision algorithms, but it does not address how to guide users to obtain high-quality oral images. Another existing technology proposes an intraoral imaging device. This device uses specialized hardware for oral imaging, which is relatively expensive, difficult to promote, and unsuitable for daily use by ordinary users. Summary of the Invention

[0007] To overcome the shortcomings of the existing technologies and address the technical problem that ordinary users lacking professional medical knowledge cannot obtain high-quality oral images, an intelligent oral examination interactive guidance system and method based on augmented reality is provided. This system utilizes intelligent mobile terminal devices combined with augmented reality technology to assist users in conducting oral self-examinations or remote professional examinations through virtual guidance.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] In a first aspect, the present invention provides an augmented reality-based intelligent oral examination interactive guidance system, including an intelligent terminal device and an installed oral examination application; the oral examination application includes:

[0010] The device posture detection module is used to detect the holding posture of the smart terminal device. When the smart terminal device is detected to be in a preset inverted holding state, it sends a posture signal to the display interface adjustment module to automatically adjust the orientation of the display interface.

[0011] The augmented reality rendering module is used to overlay a virtual dentist character onto real-time camera footage to guide users through oral examinations and capture standard oral images.

[0012] The oral image analysis module is used to evaluate oral image preview frames in real time during the oral examination guidance process, obtain real-time evaluation results based on dynamic indicators, and feed the real-time evaluation results back to the virtual character interaction control module; it is also used to stop the feedback when the real-time evaluation results meet the threshold conditions, and automatically capture static images and send them to the image quality evaluation module.

[0013] The virtual character interaction control module is used to control the actions and expressions of the virtual dentist character based on real-time evaluation results and preset oral examination procedures, and guide users to adjust the shooting angle, distance and lighting conditions;

[0014] The image quality assessment module is used to evaluate whether the received static images meet the standards. If not, it sends a reacquisition signal to the oral image analysis module. If yes, it saves the static images as standard oral images.

[0015] In a further technical solution, the device posture detection module detects the gripping state in real time through the sensors built into the smart terminal device. When it is identified that the gripping state is inverted, it sends a posture signal to the display interface adjustment module.

[0016] A further technical solution involves the augmented reality rendering module generating and displaying a virtual dentist character, specifically including:

[0017] A virtual dentist character model is pre-created using 3D modeling software, and corresponding skeletal animation and facial expression sequences are designed for each examination action;

[0018] A spatial perception algorithm is used to synchronously locate and map continuous oral image preview frames, extract environmental feature points and depth information in real time, establish a three-dimensional spatial coordinate system, and assign fixed anchor points to the virtual dentist character in this coordinate system.

[0019] During the rendering phase, the application sequentially reads the current device pose, updates the virtual dentist character's world coordinates, executes target skeletal animation, and overlays the virtual dentist character's image layer onto the camera view in each frame to complete real-time rendering.

[0020] A further technical solution involves an oral image analysis module employing a multi-stage cascaded convolutional neural network architecture, comprising a feature extraction network and a multi-task processing network connected sequentially. The feature extraction network receives oral image preview frames and extracts visual features through multi-layer convolution and pooling operations, then inputs these visual features into the multi-task processing network. The multi-task processing network uses a U-Net structure to segment the visual features and generate oral region masks, identifies and locates teeth through a region proposal network, and finally obtains the oral image analysis results through a regression network.

[0021] A further technical solution involves using a decision mapping engine to transform oral imaging analysis results into guidance strategies.

[0022] Further technical solutions also include auxiliary dental endoscope tools. The virtual character interaction control module uses computer vision algorithms to identify the position and orientation of the dental endoscope, combines the spatial relationship of the oral cavity, calculates the optimal posture for using the dental endoscope, and demonstrates it through a virtual dentist character.

[0023] A further technical solution is that the image quality assessment module performs quality assessment on static images by: calculating multi-dimensional assessment indicators for static images, identifying problem areas based on threshold analysis using the multi-dimensional assessment indicators, and generating suggestions using a suggestion generation algorithm.

[0024] Secondly, the present invention provides an intelligent oral examination interactive guidance method based on augmented reality, comprising:

[0025] Launch the oral examination application on the smart terminal device, activate the front camera and enter augmented reality interaction mode;

[0026] The system detects the holding posture of the smart terminal device and automatically adjusts the orientation of the display interface when the smart terminal device is detected to be in a preset inverted holding state.

[0027] A virtual dentist character is overlaid on the real-time camera footage captured by the camera to guide users through oral examinations and capture standard oral images;

[0028] In the oral examination guidance process, the oral image preview frame is evaluated in real time to obtain real-time evaluation results based on dynamic indicators, and the real-time evaluation results are fed back; when the real-time evaluation results meet the threshold conditions, the feedback stops and a static image is automatically captured.

[0029] Based on real-time evaluation results and preset oral examination procedures, the virtual dentist character is controlled to perform corresponding guiding actions and expressions, guiding users to adjust the shooting angle, distance, and lighting conditions;

[0030] The system assesses whether the received static images meet the standards. If not, it sends a reacquisition signal to the oral imaging analysis module. If so, it saves the static images as standard oral images.

[0031] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the augmented reality-based intelligent oral examination interactive guidance method described in the second aspect.

[0032] Fourthly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the augmented reality-based intelligent oral examination interactive guidance method described in the second aspect.

[0033] The above one or more technical solutions have the following beneficial effects:

[0034] This invention installs a dedicated oral examination application on a smart terminal device. This application includes a device posture detection module and a display interface adjustment module. It innovatively employs an inverted grip method for the smart terminal device, solving the technical problem of users simultaneously viewing the screen and controlling the camera in traditional grip methods, thus improving operational convenience. Through an augmented reality rendering module, an oral image analysis module, and a virtual character interaction control module, it implements augmented reality technology and a virtual dentist role to guide users, enabling ordinary users lacking professional medical knowledge to intuitively understand how to adjust the shooting angle and position, significantly improving the accuracy and quality of oral image acquisition. The image quality assessment module allows the system to promptly detect substandard oral images and, when necessary, prompts the user to re-acquire images of specific areas.

[0035] This invention introduces a virtual dentist role, combined with the inverted grip of a smart terminal device and a low-cost dental endoscope, to achieve intuitive and user-friendly oral examination guidance interaction, significantly improving the quality of oral image acquisition and user experience.

[0036] This invention combines a low-cost auxiliary dental endoscope tool with the guidance of a virtual dentist, enabling ordinary users to easily obtain clear images of areas that are difficult to directly photograph, such as the inner and posterior parts of the oral cavity.

[0037] This invention designs a human-like interactive method for virtual dentists, which is especially suitable for special user groups such as children and the elderly. It reduces the tension and resistance to oral examinations and improves user experience and cooperation.

[0038] This invention's system ensures that users can complete high-quality oral image acquisition in the shortest possible time by analyzing oral images in real time and providing immediate feedback, thus improving examination efficiency.

[0039] This invention provides technical support for remote oral healthcare and personal oral health management, helping to improve the accessibility and convenience of oral health services, and has significant social benefits and application value. It is applicable to scenarios such as personal oral health management, remote oral healthcare consultation, children's oral health education, and initial oral screening in primary healthcare institutions. With the development of mobile health technology, this invention has broad application prospects in areas such as intelligent healthcare, preventive health management, and the downward flow of medical resources.

[0040] The technical solution of this invention is simple to implement, has low hardware requirements, is suitable for widespread deployment on various smart terminal devices, and has good prospects for promotion and application. Attached Figure Description

[0041] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0042] Figure 1 This is a schematic diagram of the overall architecture of the intelligent oral examination interactive guidance system based on augmented reality according to an embodiment of the present invention;

[0043] Figure 2 This is a diagram illustrating the conventional gripping method of a smart terminal device according to an embodiment of the present invention;

[0044] Figure 3 This is a diagram illustrating the inverted holding method of a smart terminal device according to an embodiment of the present invention;

[0045] Figure 4 This is a flowchart of an intelligent oral examination interactive guidance method based on augmented reality, according to an embodiment of the present invention.

[0046] Among them, 1-the front-facing camera above the smart terminal device, 2-the area where the camera and the human eye's line of sight conflict with the screen, 3-the front-facing camera below the smart terminal device, and 4-auxiliary dental endoscope tool. Detailed Implementation

[0047] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0048] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0049] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0050] Example 1

[0051] like Figure 1 As shown, this embodiment discloses an augmented reality-based intelligent oral examination interactive guidance system, including an intelligent terminal device and an installed oral examination application; the oral examination application includes:

[0052] The device posture detection module is used to detect the holding posture of the smart terminal device. When the smart terminal device is detected to be in a preset inverted holding state, it sends a posture signal to the display interface adjustment module to automatically adjust the orientation of the display interface.

[0053] The augmented reality rendering module is used to overlay a virtual dentist character onto real-time camera footage to guide users through oral examinations and capture standard oral images.

[0054] The oral image analysis module is used to evaluate oral image preview frames in real time during the oral examination guidance process, obtain real-time evaluation results based on dynamic indicators, and feed the real-time evaluation results back to the virtual character interaction control module; it is also used to stop the feedback when the real-time evaluation results meet the threshold conditions, and automatically capture static images and send them to the image quality evaluation module.

[0055] The virtual character interaction control module is used to control the actions and expressions of the virtual dentist character based on real-time evaluation results and preset oral examination procedures, and guide users to adjust the shooting angle, distance and lighting conditions;

[0056] The image quality assessment module is used to evaluate whether the received static images meet the standards. If not, it sends a reacquisition signal to the oral image analysis module. If yes, it saves the static images as standard oral images.

[0057] Smart terminal devices can be portable devices such as smartphones and tablets equipped with a front-facing camera, display screen, processor, and memory. These smart terminal devices have a dedicated oral examination application installed, which includes a device posture detection module, a display interface adjustment module, an augmented reality rendering module, an oral image analysis module, a virtual character interaction control module, an image quality assessment module, and a data storage and transmission module.

[0058] The intelligent oral examination interactive guidance system also includes a low-cost auxiliary dental endoscope tool to help obtain images of the inside of the oral cavity.

[0059] In this embodiment, the device posture detection module uses sensors such as gravity sensors and gyroscopes built into the smart terminal device to detect the device's grip posture in real time. When it detects that the device is held in an inverted position (i.e., the screen is upside down and the front camera is at the bottom of the screen), it sends a posture signal to the display interface adjustment module. This module can accurately identify the device's spatial position and rotation state. By fusing data from multiple sensors, it effectively eliminates misjudgments caused by slight hand tremors, ensuring stable recognition of the grip posture by the system.

[0060] In this embodiment, after receiving the posture signal, the display interface adjustment module automatically adjusts the display orientation of the application interface to ensure that the interface content maintains a normal orientation relative to the user, allowing the user to view the screen content normally while holding the device upside down. This module not only adjusts the screen display orientation but also optimizes the mapping relationship of touch operations, ensuring that the user's touch operations in the upside-down holding state maintain the same interaction logic as when holding the device normally, reducing the learning curve.

[0061] like Figure 2 As shown, this illustrates the conventional way of holding a smart terminal device. In this case, the front-facing camera 1 on the top of the smart terminal device is located above the screen, creating a line-of-sight conflict area 2 between the camera and the human eye when viewing the screen, making it difficult for the user to look at the screen at the same time while taking a picture of the mouth. Figure 3 The invention demonstrates an inverted grip method for the smart terminal device. In this case, the front-facing camera 3 at the bottom of the smart terminal device is located below the screen, eliminating visual interference. Users can take pictures and view the screen at the same time. At the same time, combined with the handheld auxiliary dental endoscope tool 4, it is easier to observe the inside of the oral cavity, greatly improving the operational convenience and effectiveness of oral image acquisition.

[0062] In this embodiment, the augmented reality rendering module is responsible for overlaying and displaying a virtual dentist character onto the real-time images captured by the camera. This module first identifies the spatial location and depth information of the oral cavity region using the ORB-SLAM3 feature point extraction and matching algorithm. Then, based on this information, it renders the virtual dentist character at an appropriate location, making it appear as if it truly exists within the user's oral environment. This module employs advanced AR rendering technology, capable of handling complex lighting conditions and occlusion relationships, ensuring that the virtual character presents a natural and realistic visual effect in different environments. Furthermore, the augmented reality rendering module also achieves spatial anchoring between the virtual character and the real environment. Even when the user is using a mobile device, the virtual character can maintain a relatively stable spatial position, enhancing the continuity and immersion of the interaction.

[0063] The augmented reality rendering module uses a combination of 3D computer graphics and computer vision technologies to generate and display a virtual dentist character. The specific implementation includes the following steps:

[0064] First, a high-quality virtual dentist character model is pre-created using professional 3D modeling software, including skeletal structure, facial features, and clothing. Corresponding skeletal animations and expression sequences are designed for each examination action (such as opening the mouth, tilting the head, and raising a hand to indicate), forming a series of standard actions and expressions that meet the needs of guiding oral examinations, creating an action library and an expression library. These models and action data are stored in the system in an optimized lightweight format to ensure real-time rendering performance on mobile devices.

[0065] Secondly, spatial awareness algorithms are employed to simultaneously locate and map the image sequences input from the camera, extracting environmental feature points and depth information in real time to establish a three-dimensional spatial coordinate system. Fixed anchor points are then assigned to the virtual dentist character within this coordinate system, ensuring that the character's relative position to the real oral cavity scene remains consistent even as the device moves. Specifically, the system uses spatial awareness algorithms based on SLAM (Simultaneous Localization and Mapping) technology (such as ORB-SLAM3 and VINS-FUSION algorithms) to analyze the image sequences input from the camera in real time, extracting environmental feature points and depth information to construct a three-dimensional spatial coordinate system. Combined with data from the device's built-in gyroscope and accelerometer, the system can accurately perceive the device's position and attitude changes in three-dimensional space, providing stable spatial anchor points for the virtual character.

[0066] Finally, during the rendering phase, the application reads the current device pose for each frame. Update virtual dentist character world coordinates Perform target skeletal animation The process of "overlaying the image layer of the virtual dentist character onto the camera screen" completes real-time rendering, enabling seamless overlay and interactive guidance between the virtual dentist character and the user's oral images.

[0067] Furthermore, a feature-point-based spatial anchor point allocation mechanism is adopted to assign fixed anchor points to the virtual dentist character. Specific steps include: extracting FAST corner points and BRIEF descriptors from the oral environment as feature points using the ORB-SLAM3 algorithm; optimizing the 3D positions of the feature points using the Bundle Adjustment algorithm to construct an environmental map; selecting the maxillary midline and mandibular first molar as primary reference points based on oral anatomy features; calculating the relative distance and direction vectors between the virtual dentist character and these reference points during the initialization phase; continuously tracking reference point position changes through feature point matching during user mobile device movement; and dynamically updating the virtual character's world coordinates based on the real-time positions of the reference points, maintaining a constant relative positional relationship.

[0068] In this embodiment, the oral cavity image analysis module simultaneously processes preview frames and automatically captured static images. The system performs real-time evaluation of continuous preview frames from the camera to obtain dynamic indicators such as shooting angle, distance, and illumination, and immediately feeds the evaluation results back to the virtual character interaction control module. Specifically, the oral cavity region is located using a facial key point detection algorithm (based on the MobileNet-SSD model). Combined with device gyroscope data, the angle between the camera and the standard oral cavity plane is calculated to obtain the shooting angle. Combining stereo vision depth estimation and distance estimation based on target size, the actual distance between the device and the oral cavity is measured to obtain the shooting distance. Through HSV color space analysis and regional brightness histogram statistics, the mean, standard deviation, and illumination uniformity index of image brightness are calculated to obtain the illumination conditions. The Laplacian gradient operator is used to calculate the image sharpness score, and combined with contrast and texture feature analysis, the sharpness is obtained.

[0069] When the real-time evaluation results meet the threshold conditions (i.e., all dynamic indicators meet the threshold conditions), feedback stops, and the system automatically captures and stores a high-resolution still image. Subsequent image quality assessments and medical analyses are performed based on this still image. Through the "real-time-capture" dual-path mechanism, high-quality examination images can be obtained while ensuring smooth interaction.

[0070] Furthermore, in this embodiment, the threshold conditions for each dynamic indicator are set as follows: the shooting angle deviation is less than... The shooting distance should be between 10-30 cm; the illumination uniformity index should be greater than 0.7; and the sharpness score should be greater than the preset threshold (80 / 100). These threshold conditions are for illustrative purposes only and can be set according to actual image quality requirements. They are not specifically limited.

[0071] The oral image analysis module analyzes oral images acquired by the camera in real time, identifying which oral region is being captured, whether the shooting angle is appropriate, and whether the image clarity and lighting conditions meet standards. This module employs a deep learning algorithm, trained on a large amount of oral image data, enabling it to accurately identify oral tissues such as teeth, gums, and tongue, and evaluate image quality. The algorithm adapts to different oral morphologies and various lighting conditions, continuously improving its recognition accuracy through ongoing online learning. The real-time evaluation results are then transmitted to the virtual character interaction control module as a basis for guiding interactions.

[0072] The oral image analysis module achieves accurate oral region recognition and quality assessment based on deep learning technology. This module employs a multi-stage cascaded convolutional neural network architecture, comprising two main parts: a feature extraction network and a multi-task processing network. The feature extraction network uses an optimized MobileNet-V3 architecture, optimized for computational and memory usage on mobile devices, enabling real-time processing while maintaining high accuracy. This network receives raw images (preview frames) captured by a camera, extracts rich visual features through multi-layer convolution and pooling operations, and inputs these visual features into the multi-task processing network.

[0073] The multi-task processing network receives visual features and performs three key tasks, including oral cavity region segmentation, anatomical structure recognition, and image quality assessment.

[0074] (1) Oral region segmentation

[0075] The oral cavity region segmentation task employs the U-Net architecture to generate pixel-level oral cavity region masks, accurately distinguishing tissues such as teeth, gums, and tongue. The input to the U-Net architecture is visual features, specifically the feature maps extracted by the feature extraction network (with dimensions of [missing information - likely 1]). The spatial details are recovered through transposed convolution and skip connections, outputting a pixel-level oral cavity region mask R (with dimensions of ). The range is ).

[0076] (2) Anatomical structure identification

[0077] The anatomical structure recognition task uses a region proposal network (RPN) to identify and locate specific teeth and their numbers. The RPN takes visual features and an oral region mask R as input, extracts fixed-size features using oROI-Align, and outputs the bounding box coordinates, class label (e.g., variation 1-32), and confidence score for each tooth.

[0078] (3) Image quality assessment

[0079] The image quality assessment task calculates a comprehensive score for quality indicators such as sharpness, illumination uniformity, and angle appropriateness using a regression network. A regression network is constructed by global average pooling followed by a fully connected layer. Its input is the output of visual features and anatomical structure recognition, and its output is the shooting angle deviation angle. (Digits), Clarity (0-1) Illumination uniformity (0-1) and mask for suspected areas of dental / periodontal abnormalities It employs a multi-head attention mechanism to focus on key areas, improving clarity. and uniformity of illumination The overall quality score is generated by weighted average. .

[0080] The system employs knowledge distillation technology to transfer knowledge from a teacher network pre-trained on a large-scale oral medical image dataset to a lightweight student network, significantly reducing model size and inference time while maintaining high accuracy. Furthermore, the model undergoes quantization and pruning optimizations to address the heterogeneous computing characteristics of mobile devices, fully utilizing dedicated hardware acceleration units such as GPUs and NPUs to achieve real-time analysis performance exceeding 25 frames per second.

[0081] Based on real-time evaluation results, the system transforms these results into specific guidance strategies through a decision mapping engine. This engine, based on a predefined expert knowledge base and decision tree, considers factors such as the currently identified oral cavity region, shooting angle deviation, lighting conditions, and sharpness to generate the optimal sequence of guiding actions. For example, when the system detects that the current shot is of the right maxillary region, but the angle is too low and the lighting is insufficient, the decision engine will generate a guidance strategy of "adjusting upwards by 15 degrees and supplementing the lighting," and select the corresponding combination of actions and expressions from the virtual character action library.

[0082] Specifically, the process begins by constructing an expert knowledge base and a decision tree. This involves: The expert knowledge base is built by conducting structured interviews with several (e.g., 10) dentists, recording their guidance strategies for different oral examination scenarios; the C4.5 decision tree learning algorithm is used to extract rules from expert decisions and construct an initial decision tree; machine learning methods are then used to optimize the decision tree parameters from hundreds (e.g., 500) of real user interaction data; the final decision tree contains 72 decision nodes and 118 leaf nodes, covering common oral examination scenarios. Secondly, based on the results of oral image analysis (i.e., oral region mask R, shooting angle deviation angle...),... Clarity Illumination uniformity The decision tree first determines the current examination stage (e.g., anterior teeth area, left side teeth area, etc.); then it considers the deviation of the shooting angle. Determine the primary adjustment direction and generate an adjustment strategy (e.g., "adjust upwards by 15 degrees"); based on resolution... and uniformity of illumination Determine secondary adjustment strategies (such as "increase light source" or "decrease distance"); comprehensively consider user operation history and feedback to adjust strategy priority and execution order; map the final strategy to a predefined sequence of guiding actions (such as pointing or adjusting posture); dynamically generate guiding action parameters (such as pointing direction and adjustment range) and pass them to the virtual character interaction control module.

[0083] In this embodiment, the virtual character interaction control module controls the virtual dentist character to perform corresponding guiding actions and expressions based on the oral image analysis results (real-time evaluation results) and the preset oral examination process, guiding the user to adjust the shooting angle, distance, and lighting conditions. Specifically, the decision mapping engine maps the optimal sequence of guiding actions to behavior tree nodes, driving the motion skeleton and blend-shape expressions to execute synchronously. For example, when it detects that the user's shooting angle is inappropriate, the virtual dentist will make a gesture to guide to the left or right; when it detects insufficient lighting, it will make a prompt to add lighting; when it needs the user to open their mouth wide, the virtual dentist will demonstrate opening their mouth, etc. This module implements a complete virtual character motion control system, including multi-dimensional control of skeletal animation, facial expressions, and gestures, capable of generating smooth and natural sequences of guiding actions.

[0084] Meanwhile, this module also integrates context-aware capabilities, enabling it to adjust guidance strategies based on user reactions and interactions, achieving a more personalized and adaptive interactive experience. Specifically, it uses facial expression recognition algorithms to detect user emotional states (focus, confusion, lack of attention, tracking consistency between user actions and suggested actions, calculating compliance indices, patience, etc.); analyzes user response time and execution quality to assess comprehension; tracks consistency between user actions and suggested actions, and calculates compliance indices. A dynamic strategy adjustment mechanism is employed to adjust guidance strategies. When a user fails to adjust according to suggestions three consecutive times, the system reduces guidance complexity, providing more basic and intuitive guidance actions; when the user compliance index is below 0.5, the salience of visual cues and the frequency of voice prompts are increased.

[0085] To implement the aforementioned guidance strategy, the virtual character interaction control module employs parametric animation synthesis technology, blending basic actions in real-time using a Bézier curve interpolation algorithm to generate smooth and natural guidance animations. This module also implements action priority management, ensuring that when multiple guidance requests exist simultaneously, the system presents them one by one in a reasonable order, avoiding conflicting guidance information received by the user.

[0086] Furthermore, the steps to implement the action priority management function are as follows: Establish a hierarchical priority framework: The level is fully relevant guidance (such as avoiding accidental operation, preventing equipment from falling, etc.); Level 1 is for guiding critical quality impacts (such as severe angular deviation, extreme blur, etc.); The level is for routine quality optimization guidance (such as minor angle adjustments, lighting optimization, etc.). To enhance guidance (e.g., integrity checks, progress indicators, etc.); conflict resolution mechanism: a time-window-based guidance queue is used, where high-priority guidance can interrupt low-priority guidance; interrupted guidance tasks enter a delay queue and are rescheduled according to priority; for guidance of the same priority, a weighted round-robin scheduling algorithm is used to prevent certain types of guidance from continuously consuming resources. Context-aware scheduling: the system analyzes the current operation stage and user state, dynamically adjusting priority thresholds; during critical shooting moments (e.g., when aiming at a specific area), non-critical guidance is temporarily suppressed; for tasks that the user repeatedly fails to execute, the priority of their associated guidance is automatically increased.

[0087] When the system detects that a user is using an auxiliary dental endoscope, the virtual character interaction control module provides targeted guidance, demonstrating how to correctly hold the dental endoscope, how to adjust the angle of the endoscope to reflect the image inside the oral cavity, and how to maintain stability to obtain a clear image. This module uses computer vision algorithms to identify the position and orientation of the dental endoscope, combines this with the spatial relationships of the oral cavity, calculates the optimal posture for using the endoscope, and intuitively conveys this to the user through the virtual character's actions.

[0088] Furthermore, the steps for guided interaction using the dental endoscope tool are as follows:

[0089] (1) Dental endoscopy detection and identification

[0090] The YOLOv5-tiny model was used for real-time detection of dental endoscopes, achieving a high recognition rate. Reasoning time ;

[0091] The reflective surface of the dental endoscope is located using the circular Hough transform algorithm, achieving high accuracy. ;

[0092] The position and orientation of the dental mirror in 3D space were calculated using the PnP (Perspective-n-Point) algorithm.

[0093] (2) Constructing oral cavity spatial relationships

[0094] Based on the oral cavity segmentation results (oral cavity region mask R), a simplified 3D oral cavity model is constructed, including the tooth plane and the opening boundary;

[0095] Calculate the spatial relationship between the reflective surface of the dental mirror and the target area (such as the posterior teeth or lingual surface);

[0096] A ray tracing algorithm is used to simulate the reflection path of the dental endoscope and determine the optimal observation angle.

[0097] (3) Calculate the optimal posture for using the dental mirror

[0098] The optimal posture is calculated by taking into account the following factors through a geometric optimization algorithm: maximizing the visible area of ​​the target region in the dental mirror, minimizing the difficulty of holding the device and hand fatigue, and optimizing the shooting angle to reduce reflection and distortion.

[0099] The generated posture parameters include: dental endoscope insertion depth, angle, rotation direction, and wrist posture.

[0100] (4) Implement virtual boot

[0101] The virtual dentist character demonstrates the correct posture for holding and using a dental mirror by breaking down the movements.

[0102] Use a semi-transparent overlay to mark the ideal position and orientation of the dental mirror in a live view;

[0103] Provides real-time distance feedback and visual confirmation when the user's dental mirror approaches the ideal position.

[0104] Furthermore, the workflow for using a dental endoscope in conjunction with a smart terminal is as follows:

[0105] The user first inserts the dental mirror into the mouth and adjusts it to be near the target area;

[0106] The system identifies the position of the dental mirror and calculates the optimal adjustment direction;

[0107] The virtual dentist guides users to fine-tune the angle of the dental mirror until the optimal reflection position is achieved;

[0108] Once the optimal position is reached, the system locks the focus on the dental mirror's reflective surface;

[0109] The user keeps the dental endoscope stable, and the system automatically captures images at the optimal time.

[0110] In this embodiment, the image quality assessment module comprehensively evaluates the oral images (i.e., automatically captured static images) acquired by the user. It assesses whether the received static images meet the standards. If not, it sends a re-acquisition signal to the oral image analysis module; if so, it saves the static images as standard oral images. The module also provides quality scores and improvement suggestions for substandard static images. Based on professional oral image evaluation standards, this module quantitatively evaluates images from multiple dimensions, including sharpness, brightness, contrast, shooting angle, and coverage, generating a comprehensive quality score. Specifically, according to the MQE (Medical-image Quality Estimator) framework, it integrates BRISQUE, NIQE, and oral-specific illumination indices, and obtains the comprehensive quality score through XGBoost regression. For areas with substandard quality, the system marks specific problems and provides targeted improvement suggestions, such as "insufficient lighting in the right molar area" or "excessive shooting angle in the anterior teeth area."

[0111] The specific steps for assessing the quality of oral imaging are as follows:

[0112] (1) Calculate multi-dimensional evaluation indicators

[0113] Sharpness: The variance of the gradient magnitude is calculated using the Laplacian operator, combined with energy analysis of high-frequency components;

[0114] Brightness: The average brightness and standard deviation are calculated through brightness histogram analysis to assess the overall exposure level;

[0115] Contrast: The local and global contrast of the image is evaluated using the RMS contrast algorithm.

[0116] Shooting angle: Based on the geometric features of tooth arrangement, estimate the deviation between the shooting angle and the standard angle;

[0117] Coverage: Calculate the coverage ratio and completeness of the target oral cavity region in the image;

[0118] (2) Quality scoring based on multi-dimensional evaluation indicators (MQE framework quality scoring process)

[0119] Input the oral cavity image (raw image) and the extracted oral cavity region mask into the preprocessing module;

[0120] Parallel computation of evaluation index values ​​for each dimension (sharpness S_raw, brightness B_raw, etc.);

[0121] The BRISQUE (Blind / Referenceless Image Spatial Quality Evaluator) algorithm is applied to calculate the referenceless quality score;

[0122] Combined with the oral cavity-specific illumination index (considering the special reflective and shading characteristics in the oral cavity), all indicators were standardized to the [0,1] interval and input into the XGBoost model;

[0123] The XGBoost model outputs the final overall quality score. (0-100).

[0124] (3) Identify problem areas based on threshold analysis and generate suggestions using a suggestion generation algorithm.

[0125] Areas with a sharpness below the threshold S_th are marked as "blurry areas", areas with brightness below B_low or above B_high are marked as "too dark" or "too bright" respectively, and shooting angle deviations exceeding θ_th are marked as "inappropriate angles".

[0126] Calculate the severity and affected area ratio of each type of problem to assess the severity of the problem;

[0127] Detected problems are mapped to a predefined library of improvement suggestions using decision trees;

[0128] Based on the problem severity and user capability model, select the most suitable expression method;

[0129] Generate specific, actionable suggestions, such as "The right molar area is not well lit; please move closer to a window light source or turn on supplemental lighting."

[0130] For complex problems, a suggested sequence is proposed based on the priority of solving them.

[0131] In this embodiment, the data storage and transmission module is responsible for saving the acquired high-quality oral images (i.e., standard oral images) locally or transmitting them to a cloud server for subsequent software analysis or human physician diagnosis. This module achieves secure encrypted storage and efficient transmission of data, employing a hierarchical encryption strategy and differentiated compression algorithms. While ensuring user privacy and data security, it optimizes network transmission efficiency and adapts to data transmission needs in various network environments. Furthermore, this module provides multiple data sharing options, allowing users to securely share the acquired oral images with designated doctors or medical institutions for easy access to professional diagnostic opinions.

[0132] In summary, this system innovatively uses a front-facing camera on a smart terminal device to capture standard oral images, employs an inverted holding method, and integrates augmented reality technology to overlay a virtual dentist avatar onto the real-time images. This virtual avatar guides the user through intuitive movements and expressions, adjusting the shooting angle, distance, and lighting conditions, while utilizing low-cost auxiliary dental endoscopes to obtain images of the inner part of the mouth. The system analyzes image quality in real time, and the virtual avatar provides targeted guidance, ultimately resulting in high-quality oral images.

[0133] This invention provides intuitive and user-friendly oral examination guidance for ordinary users by combining augmented reality technology with appropriate human-computer interaction design, ensuring the acquisition of high-quality oral images, thereby improving the effectiveness of oral self-examination and the feasibility of remote oral healthcare.

[0134] Example 2

[0135] This embodiment discloses an intelligent oral examination interactive guidance method based on augmented reality, including:

[0136] Launch the oral examination application on the smart terminal device, activate the front camera and enter augmented reality interaction mode;

[0137] The system detects the holding posture of the smart terminal device and automatically adjusts the orientation of the display interface when the smart terminal device is detected to be in a preset inverted holding state.

[0138] A virtual dentist character is overlaid on the real-time camera footage captured by the camera to guide users through oral examinations and capture standard oral images;

[0139] In the oral examination guidance process, the oral image preview frame is evaluated in real time to obtain real-time evaluation results based on dynamic indicators, and the real-time evaluation results are fed back; when the real-time evaluation results meet the threshold conditions, the feedback stops and a static image is automatically captured.

[0140] Based on real-time evaluation results and preset oral examination procedures, the virtual dentist character is controlled to perform corresponding guiding actions and expressions, guiding users to adjust the shooting angle, distance, and lighting conditions;

[0141] The system assesses whether the received static images meet the standards. If not, it sends a reacquisition signal to the oral imaging analysis module. If so, it saves the static images as standard oral images.

[0142] In this embodiment, as Figure 4 As shown, the detailed steps of the augmented reality-based intelligent oral examination interactive guidance method are as follows:

[0143] S1: The user launches the oral examination application on their smart terminal device. The system automatically activates the front-facing camera and enters augmented reality interaction mode. During this process, the system loads a virtual dentist character model and related resources, preparing to begin the oral examination guidance. The system also checks device performance and ambient lighting conditions, providing optimization suggestions if necessary to ensure the best AR experience. During loading, the system displays a brief user tutorial and health tips to help users understand basic operating methods and oral health knowledge.

[0144] S2: The system detects the user's grip posture on the smart terminal device. When the system detects that the device is in a preset inverted grip position (i.e., the screen is upside down with the front-facing camera at the bottom), it automatically adjusts the display interface orientation, allowing the user to view the screen content normally while holding the device inverted. If the device is not in an inverted grip position, the virtual dentist character will guide the user to invert the device through gestures and text prompts for a better user experience. The system uses an intelligent detection algorithm that can distinguish between intentional user actions and unintentional shaking, avoiding the problem of frequent interface switching due to overly sensitive posture detection.

[0145] S3: The system overlays a virtual dentist avatar onto the real-time video captured by the camera. The virtual avatar initially appears with a friendly expression and welcoming gesture, and briefly explains the purpose and procedure of the oral examination through voice or text to alleviate user anxiety, especially building rapport with children. The system adjusts the virtual avatar's interaction based on user reactions (such as facial expressions and head movements) to ensure a natural and smooth interaction. The virtual avatar's appearance and voice can be personalized according to the user's age and preferences to enhance user acceptance.

[0146] S4: The system begins guiding the user to prepare the oral examination environment. The virtual dentist avatar guides the user through actions and expressions to adjust lighting conditions (such as facing natural light or turning on indoor lighting), check the camera's cleanliness, and prepare auxiliary dental endoscope tools (if needed). During this process, the system analyzes the ambient light and camera image quality in real time. When conditions meet the requirements, the virtual dentist gives a confirmation gesture to proceed to the next step. The system provides personalized optimization suggestions for different ambient light conditions, such as suggesting turning on the device's supplemental lighting function in low-light environments, or adjusting the angle to avoid glare in bright light environments.

[0147] S5: The system guides the user through oral image acquisition region by region according to the preset oral examination procedure. The typical examination order is: anterior teeth (incisors and canines), right lateral teeth (right premolars and molars), left lateral teeth (left premolars and molars), maxillary occlusal surface, mandibular occlusal surface, and soft tissues (tongue, oral mucosa, etc.). For each region, the system executes the following sub-steps:

[0148] S501: The virtual dentist avatar guides users to aim the camera at specific areas of the mouth through gesture demonstrations. For example, when examining the anterior teeth, the virtual dentist will make a gesture pointing to the front teeth and may display a semi-transparent target frame to indicate the ideal shooting area. The system dynamically adjusts the position and size of the target frame according to the user's actual oral structure to ensure accurate guidance. Simultaneously, the virtual dentist uses simple and clear voice prompts, such as "Please aim at the front teeth" and "Open your mouth slightly wider," to enhance user comprehension in conjunction with the gesture guidance.

[0149] S502: The system analyzes the oral cavity images captured by the camera in real time, evaluating the shooting angle, distance, and clarity. Based on the analysis results, the virtual dentist character performs corresponding guiding actions, such as gestures to the left, right, up, down, closer, and further away, helping the user adjust to the optimal shooting position. The system's analysis algorithm can identify minute angle deviations and focal length issues, providing fine-tuning guidance. To enhance the intuitiveness of the feedback, the system may also display directional arrows or distance indicators on the interface, complementing the virtual character's action guidance.

[0150] S503: When the system detects that the current image quality meets the preset standard, the virtual dentist character gives a gesture or expression of confirmation (such as a thumbs-up or a nod and smile), and the system automatically captures and saves a high-quality image of that area. The system uses an intelligent trigger mechanism to automatically capture the image the moment the quality meets the standard, avoiding device shaking and image blurring caused by the user manually clicking the screen. At the same time, the system will provide positive voice feedback, such as "Great! This photo is very clear," enhancing the user's sense of accomplishment and engagement.

[0151] S504: The system guides you to the next oral cavity area, repeating steps S501 to S503 until image acquisition for all areas is complete. During area switching, the system displays the current examination progress and prompts for the next area, helping users understand the overall examination progress. For areas that some users may find difficult (such as the posterior molar area), the system will provide more detailed operation prompts and multiple optional imaging methods in advance to increase the success rate.

[0152] S6: For oral regions requiring the use of auxiliary dental endoscopes (such as the inner surfaces of teeth and the posterior molar region), the system executes a special guided procedure:

[0153] S601: The virtual dentist avatar guides users on how to correctly hold a dental endoscope through demonstrations. The system identifies the dental endoscope in the user's hand and provides targeted guidance to ensure the endoscope is positioned correctly and safely. The system uses computer vision algorithms to detect the shape and position of the endoscope and identify whether the user's grip is correct. The virtual dentist demonstrates the correct grip method, such as "holding the lower part of the endoscope and keeping your fingers away from the mirror surface," and provides real-time feedback and adjustment suggestions based on the user's actual operation.

[0154] S602: The virtual dentist character demonstrates how to use a smart terminal device with a dental mirror to take pictures, including how to adjust the angle of the mirror to reflect the image inside the mouth, and how to maintain stability to obtain a clear image. The system provides detailed step-by-step guidance, such as first placing the dental mirror in the mouth, adjusting it to a suitable position, and then aiming at the reflected image in the mirror to take the picture. The virtual dentist's action demonstration takes into account the ergonomic characteristics of actual operation to ensure that the guided actions are easy to imitate and execute.

[0155] S603: The system analyzes oral images acquired through dental mirror reflection in real time, assesses quality, and provides adjustment suggestions. For dental mirror reflection images, the system employs a special image processing algorithm to handle image flipping and distortion caused by mirror reflection, accurately assessing image quality. The system provides solutions for common dental mirror usage problems, such as "There is fog on the dental mirror, please wipe it gently" and "The dental mirror angle is too large, please adjust it slightly." When the image quality meets the standards, the system automatically captures and saves the image.

[0156] S7: After completing image acquisition for all oral regions, the system performs a comprehensive quality assessment of the complete oral image set. The assessment includes the completeness of image coverage, the clarity of images in each region, and the uniformity of illumination. The system employs a comprehensive quality assessment model, taking into account professional oral examination standards and image technology indicators to objectively evaluate the acquisition results. The assessment process generates a detailed quality report, including the score for each region, existing problems, and an overall rating.

[0157] S8: Based on the quality assessment results, the system provides an overall score and improvement suggestions. If the image quality of certain areas is found to be substandard, the virtual dentist will guide the user to re-capture images of these areas until all images meet the requirements. The system will prioritize guiding the user to re-capture critical areas or the areas with the most serious problems, and provide more specific improvement suggestions, such as "The lighting was insufficient when the right molar area was last captured; please move closer to the window or turn on the lights and re-capture." For areas that are still difficult to meet the standards after multiple attempts, the system will provide alternative solutions or simplify the requirements to ensure that basic examination needs are met.

[0158] S9: The system saves or transmits the complete, high-quality set of oral images to a cloud server for subsequent software analysis or human physician diagnosis. During this process, the virtual dentist will provide confirmation feedback upon completion and may offer oral health suggestions based on preliminary AI analysis (if this feature is available). Data transmission employs segmented encryption and breakpoint resume technology to ensure secure and efficient transmission under various network conditions. After transmission is complete, the system will provide guidance for subsequent steps, such as how to view the analysis results and how to contact a professional physician for remote consultation.

[0159] Example 3

[0160] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 2.

[0161] Example 4

[0162] The purpose of this embodiment is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method of Embodiment 2.

[0163] The steps and methods involved in the apparatuses of Embodiments 3 and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 2. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0164] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0165] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0166] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An intelligent oral examination interactive guidance system based on augmented reality, characterized in that, This includes a smart terminal device and its installed oral examination application; the oral examination application includes: The device posture detection module is used to detect the holding posture of the smart terminal device. When the smart terminal device is detected to be in a preset inverted holding state, it sends a posture signal to the display interface adjustment module to automatically adjust the orientation of the display interface. The augmented reality rendering module is used to overlay a virtual dentist character onto real-time camera footage to guide users through oral examinations and capture standard oral images. The augmented reality rendering module generates and displays the virtual dentist character, specifically including: A virtual dentist character model is pre-created using 3D modeling software, and corresponding skeletal animation and facial expression sequences are designed for each examination action; A spatial perception algorithm is used to synchronously locate and map continuous oral image preview frames, extract environmental feature points and depth information in real time, establish a three-dimensional spatial coordinate system, and assign fixed anchor points to the virtual dentist character in this coordinate system. During the rendering phase, the application sequentially reads the current device pose, updates the virtual dentist character's world coordinates, executes target skeletal animation, and overlays the virtual dentist character's image layer onto the camera view in each frame to complete real-time rendering. The oral image analysis module is used to evaluate oral image preview frames in real time during the oral examination guidance process, obtain real-time evaluation results based on dynamic indicators, and feed the real-time evaluation results back to the virtual character interaction control module; it is also used to stop the feedback when the real-time evaluation results meet the threshold conditions, and automatically capture static images and send them to the image quality evaluation module. The virtual character interaction control module is used to control the actions and expressions of the virtual dentist character based on real-time evaluation results and preset oral examination procedures, and guide users to adjust the shooting angle, distance and lighting conditions; The image quality assessment module is used to evaluate whether the received static images meet the standards. If not, it sends a reacquisition signal to the oral image analysis module. If yes, it saves the static images as standard oral images.

2. The augmented reality-based intelligent oral examination interactive guidance system as described in claim 1, characterized in that, The device posture detection module detects the grip state in real time through the built-in sensors of the smart terminal device. When it detects that the grip is inverted, it sends a posture signal to the display interface adjustment module.

3. The augmented reality-based intelligent oral examination interactive guidance system as described in claim 1, characterized in that, The oral image analysis module adopts a multi-stage cascaded convolutional neural network architecture, including a feature extraction network and a multi-task processing network connected in sequence. The feature extraction network receives oral image preview frames and extracts visual features through multi-layer convolution and pooling operations, and inputs the visual features into the multi-task processing network. The multi-task processing network uses a U-Net structure to segment the visual features to generate oral region masks, identifies and locates teeth through a region proposal network, and finally obtains the oral image analysis results through a regression network.

4. The augmented reality-based intelligent oral examination interactive guidance system as described in claim 3, characterized in that, The decision mapping engine transforms oral imaging analysis results into guidance strategies.

5. The augmented reality-based intelligent oral examination interactive guidance system as described in claim 1, characterized in that, It also includes auxiliary dental endoscope tools. The virtual character interaction control module uses computer vision algorithms to identify the position and orientation of the dental endoscope, combines the spatial relationship of the oral cavity, calculates the best posture for using the dental endoscope, and demonstrates it through a virtual dentist character.

6. The augmented reality-based intelligent oral examination interactive guidance system as described in claim 1, characterized in that, The image quality assessment module performs quality assessment on static images by: calculating multi-dimensional assessment indicators for static images; using threshold analysis based on the multi-dimensional assessment indicators to identify problem areas; and using a suggestion generation algorithm to generate suggestions.

7. An intelligent oral examination interactive guidance method based on augmented reality, characterized in that, include: Launch the oral examination application on the smart terminal device, activate the front camera and enter augmented reality interaction mode; The system detects the holding posture of the smart terminal device and automatically adjusts the orientation of the display interface when the smart terminal device is detected to be in a preset inverted holding state. A virtual dentist character is overlaid on the real-time camera footage captured by the camera to guide users through oral examinations and capture standard oral images; A virtual dentist character model is pre-created using 3D modeling software, and corresponding skeletal animation and facial expression sequences are designed for each examination action; A spatial perception algorithm is used to synchronously locate and map continuous oral image preview frames, extract environmental feature points and depth information in real time, establish a three-dimensional spatial coordinate system, and assign fixed anchor points to the virtual dentist character in this coordinate system. During the rendering phase, the application sequentially reads the current device pose, updates the virtual dentist character's world coordinates, executes target skeletal animation, and overlays the virtual dentist character's image layer onto the camera view in each frame to complete real-time rendering. In the oral examination guidance process, the oral image preview frame is evaluated in real time to obtain real-time evaluation results based on dynamic indicators, and the real-time evaluation results are fed back; when the real-time evaluation results meet the threshold conditions, the feedback stops and a static image is automatically captured. Based on real-time evaluation results and preset oral examination procedures, the virtual dentist character is controlled to perform corresponding guiding actions and expressions, guiding users to adjust the shooting angle, distance, and lighting conditions; The system assesses whether the received static images meet the standards. If not, it sends a reacquisition signal to the oral imaging analysis module. If so, it saves the static images as standard oral images.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the augmented reality-based intelligent oral examination interactive guidance method as described in claim 7.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the augmented reality-based intelligent oral examination interactive guidance method as described in claim 7.

Images