Intelligent oral examination interaction guiding system and method based on augmented reality

Abstract

The invention discloses an intelligent oral examination interactive guidance system and method based on augmented reality, and the method comprises the steps: collecting an oral image through a front-facing camera and an oral examination application program of an intelligent terminal device; the application program comprises an equipment posture detection model, a display interface adjustment module, an augmented reality rendering module, an oral cavity image analysis module and a virtual role interaction control module, an equipment inverted holding mode is innovatively adopted, and a virtual dentist role is displayed in a real-time image in an overlapping manner in combination with an augmented reality technology; the virtual dentist role guides the user to adjust the shooting angle, distance and light conditions through intuitive actions and expressions, and an oral cavity inner side image is obtained in combination with a low-cost auxiliary dental mirror tool. And the system also analyzes the image quality in real time, and the virtual role performs targeted guidance, so that a high-quality oral image is finally obtained. The feasibility and effect of oral cavity self-inspection of common users under the background of lack of professional knowledge are greatly improved, and effective technical support is provided for remote oral cavity medical treatment and personal oral cavity health management.

Description

Technical Field

[0001] The present invention relates to the field of oral medical technology, and in particular, to an intelligent oral examination interactive guidance system and method based on augmented reality. Background Art

[0002] The statements in this part merely provide background technical information related to the present disclosure and do not necessarily constitute prior art.

[0003] Oral health is an important part of overall human health. Regular oral examinations are of great significance for early detection of oral diseases and prevention of oral health problems. Traditional oral examinations usually require patients to go to oral medical institutions in person, and professional doctors use professional tools such as oral mirrors and probes for examinations. Although this examination method has high accuracy, it has problems such as high medical costs, large time costs, and low examination frequencies, resulting in many oral diseases not being detected and intervened in a timely manner.

[0004] With the popularization of mobile Internet technology and intelligent devices, telemedicine and self-health management have gradually become important development trends. In the field of oral health, there have been some applications that use the camera of a smartphone for oral image acquisition and analysis. However, the current such applications mainly have the following technical problems: First, ordinary users lack professional oral medical knowledge and cannot accurately grasp the shooting angle and key areas, resulting in poor quality of the acquired oral images, which is not conducive to subsequent analysis; Second, the internal structure of the oral cavity is complex and the light conditions are limited. It is difficult for ordinary users to operate the smartphone camera to obtain clear images of the internal oral cavity, especially in areas such as the posterior teeth and the sides; Third, child users usually have a fear of oral examinations and it is difficult to cooperate to complete effective oral image acquisition; Finally, the existing applications lack a real-time interaction guidance function, and users are prone to make mistakes during the operation without realizing it.

[0005] On the other hand, augmented reality (AR) technology, as a technology that superimposes virtual information on the real environment in real time, has been widely used in fields such as medical treatment, education, and gaming. In the medical field, AR technology is used in scenarios such as surgical navigation and medical education, but its application in oral self-examination and remote oral medical fields is still in the exploratory stage. The existing technology has not effectively solved the problem of how to provide intuitive and friendly oral examination guidance for ordinary users lacking professional knowledge through AR technology to ensure the acquisition of high-quality oral images. In addition, due to the conventional holding method of a smartphone (the camera is located above the screen), it is difficult for users to watch the screen and control the shooting angle at the same time when taking an oral self-portrait, which is inconvenient to operate and prone to shooting mistakes. Although some applications attempt to assist users through voice commands or text prompts, these methods lack intuitiveness and interactivity, and the user experience is poor, especially for special groups such as children and the elderly.

[0006] The prior art discloses an intelligent scanning method and system based on an oral digital impression instrument. This system mainly analyzes oral photos uploaded by users through computer vision algorithms, but does not solve the problem of how to guide users to obtain high-quality oral images. Another prior art proposes an intraoral photographing device, which uses a special hardware device for oral photography. The cost is relatively high, and it is difficult to promote, and it is not suitable for ordinary users to use in daily life. Summary of the Invention

[0007] To overcome the deficiencies of the above prior art and aiming to solve the technical problem that it is difficult for ordinary users to obtain high-quality oral images without professional medical knowledge, an intelligent oral examination interactive guidance system and method based on augmented reality are provided. By using an intelligent mobile terminal device combined with augmented reality technology, it assists users in performing oral self-examinations or remote professional examinations through virtual guidance.

[0008] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention provides an intelligent oral examination interactive guidance system based on augmented reality, including an intelligent terminal device and an oral examination application program installed thereon; the oral examination application program includes: A device attitude detection module, configured to detect the holding attitude of the intelligent terminal device. When it detects that the intelligent terminal device is in a preset inverted holding state, it sends an attitude signal to the display interface adjustment module to automatically adjust the display interface direction; An augmented reality rendering module, configured to superimpose and display a virtual dentist character in the real-time camera image to guide the user to perform an oral examination and capture standard oral images; An oral image analysis module, configured to evaluate the preview frame of the oral image in real time during the oral examination guidance process, obtain a real-time evaluation result based on dynamic indicators, and feedback the real-time evaluation result to the virtual character interaction control module; it is also configured to stop feedback when the real-time evaluation result meets the threshold condition and automatically capture a static image and send it to the image quality evaluation module; A virtual character interaction control module, configured to control the actions and expressions of the virtual dentist character according to the real-time evaluation result and a preset oral examination process to guide the user to adjust the shooting angle, distance, and light conditions; An image quality evaluation module, configured to evaluate whether the received static image meets the standard. If not, it sends a re-acquisition signal to the oral image analysis module. If so, it saves the static image as a standard oral image.

[0009] In a further technical solution, the device attitude detection module detects the holding state in real time through a sensor built in the intelligent terminal device, and when it identifies that it is in an inverted holding state, it sends an attitude signal to the display interface adjustment module.

[0010] A further technical solution is that the augmented reality rendering module generates and displays a virtual dentist character, which specifically includes: Pre-create a virtual dentist character model through 3D modeling software, and design corresponding skeletal animations and expression sequences for each inspection action; Adopt a spatial perception algorithm to perform simultaneous localization and mapping construction on consecutive oral image preview frames, extract environmental feature points and depth information in real time, establish a three-dimensional space coordinate system, and assign a fixed anchor point to the virtual dentist character in this coordinate system; In the rendering stage, the application reads the current device pose in sequence for each frame, updates the world coordinates of the virtual dentist character, executes the target skeletal animation, and completes real-time rendering by overlaying the virtual dentist character image layer on the camera screen.

[0011] A further technical solution is that the oral image analysis module adopts a multi-stage cascaded convolutional neural network architecture, including a feature extraction network and a multi-task processing network connected in sequence; the feature extraction network receives oral image preview frames, extracts visual features through multi-layer convolution and pooling operations, and inputs the visual features into the multi-task processing network; the multi-task processing network uses a U-Net structure to segment the visual features to generate an oral region mask, identifies and locates teeth through a region proposal network, and finally obtains the oral image analysis result through a regression network.

[0012] A further technical solution is to convert the oral image analysis result into a guidance strategy through a decision mapping engine.

[0013] A further technical solution further includes an auxiliary dental mirror tool. The virtual character interaction control module identifies the position and orientation of the dental mirror through computer vision algorithms, calculates the optimal dental mirror usage posture in combination with the oral space relationship, and displays it through the virtual dentist character.

[0014] A further technical solution is that the image quality assessment module specifically assesses the quality of static images as follows: calculating multi-dimensional assessment indicators of the static image, identifying problem areas based on the multi-dimensional assessment indicators using threshold analysis, and generating suggestions using a suggestion generation algorithm.

[0015] In a second aspect, the present invention provides an intelligent oral examination interaction guidance method based on augmented reality, including: Start the oral examination application on the intelligent terminal device, activate the front camera and enter the augmented reality interaction mode; Detect the holding posture of the intelligent terminal device. When it is detected that the intelligent terminal device is in a preset inverted holding state, automatically adjust the display interface direction; Overlay and display a virtual dentist character in the real-time camera image captured by the camera to guide the user to perform an oral examination and take a standard oral image; During the oral examination guidance process, the oral image preview frame is evaluated in real time, a real-time evaluation result based on dynamic indicators is obtained, and the real-time evaluation result is fed back; when the real-time evaluation result meets the threshold condition, the feedback is stopped and a static image is automatically captured; According to the real-time evaluation result and the preset oral examination process, control the virtual dentist character to perform corresponding guiding actions and expressions to guide the user to adjust the shooting angle, distance, and light conditions; Evaluate whether the received static image meets the standard. If not, send a re-acquisition signal to the oral image analysis module. If so, save the static image as a standard oral image.

[0016] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps in the augmented reality-based intelligent oral examination interactive guidance method described in the second aspect are implemented.

[0017] In a fourth aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps in the augmented reality-based intelligent oral examination interactive guidance method described in the second aspect are implemented.

[0018] The above one or more technical solutions have the following beneficial effects: The present invention installs a dedicated oral examination application program on an intelligent terminal device. The program is provided with a device attitude detection module and a display interface adjustment module, and innovatively adopts an inverted holding method of the intelligent terminal device, solving the technical problem that it is difficult for users to view the screen and control shooting simultaneously in the traditional holding method, and improving the operation convenience; through the augmented reality rendering module, the oral image analysis module, and the virtual character interaction control module, the augmented reality technology and the guidance of the virtual dentist character are realized, enabling ordinary users lacking professional medical knowledge to intuitively understand how to adjust the shooting angle and position, and significantly improving the accuracy and quality of oral image acquisition; the image quality evaluation module enables the system to timely detect oral images that do not meet the standard, and will prompt the user to re-acquire images of specific areas if necessary.

[0019] The present invention realizes an intuitive and friendly oral examination guidance interaction by introducing a virtual dentist character, combining the inverted holding method of the intelligent terminal device and a low-cost dental mirror auxiliary tool, and significantly improves the quality of oral image acquisition and the user experience.

[0020] The present invention combines a low-cost auxiliary dental mirror tool, enabling ordinary users to easily obtain clear images of difficult-to-directly-photograph areas such as the inner and posterior parts of the oral cavity under the guidance of a virtual dentist.

[0021] The present invention designs an anthropomorphic interaction method for the virtual dentist role, which is particularly suitable for special user groups such as children and the elderly, reducing the tension and rejection psychology of oral examinations and enhancing the user experience and cooperation.

[0022] The system of the present invention ensures that users can complete high-quality oral image acquisition in the shortest time through real-time analysis of oral images and immediate feedback, improving the inspection efficiency.

[0023] The present invention provides technical support for remote oral healthcare and personal oral health management, helping to improve the accessibility and convenience of oral health services, and having significant social benefits and application values. It is applicable to application scenarios such as personal oral health management, remote oral medical consultation, children's oral health education, and oral preliminary screening in primary medical institutions. With the development of mobile health technology, the present invention has broad application prospects in intelligent healthcare, preventive health management, and the sinking of medical resources.

[0024] The technical solution of the present invention is simply implemented and has low hardware requirements, making it suitable for wide deployment on various intelligent terminal devices and having good promotion and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0026] Figure 1 is a schematic diagram of the overall architecture of the intelligent oral examination interactive guidance system based on augmented reality according to an embodiment of the present invention; Figure 2 is a diagram of the conventional holding method of the intelligent terminal device according to an embodiment of the present invention; Figure 3 is a diagram of the inverted holding method of the intelligent terminal device according to an embodiment of the present invention; Figure 4 is a flowchart of the intelligent oral examination interactive guidance method based on augmented reality according to an embodiment of the present invention.

[0027] Among them, 1 - the front camera above the intelligent terminal device, 2 - the line-of-sight conflict area between the camera and the human eye observing the screen, 3 - the front camera below the intelligent terminal device, 4 - the auxiliary dental mirror tool. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0029] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0030] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0031] Embodiment 1 As Figure 1 shown, this embodiment discloses an intelligent oral examination interaction guidance system based on augmented reality, including an intelligent terminal device and an oral examination application program installed thereon; the oral examination application program includes: A device attitude detection module, configured to detect the holding attitude of the intelligent terminal device, and when it detects that the intelligent terminal device is in a preset inverted holding state, send an attitude signal to the display interface adjustment module to automatically adjust the display interface direction; An augmented reality rendering module, configured to superimpose and display a virtual dentist character in the real-time camera image, guide the user to perform an oral examination, and take a standard oral image; An oral image analysis module, configured to evaluate the preview frame of the oral image in real time during the oral examination guidance process, obtain a real-time evaluation result based on dynamic indicators, and feedback the real-time evaluation result to the virtual character interaction control module; it is also configured to stop feedback when the real-time evaluation result meets the threshold condition, and automatically capture a static image and send it to the image quality evaluation module; A virtual character interaction control module, configured to control the actions and expressions of the virtual dentist character according to the real-time evaluation result and a preset oral examination process, and guide the user to adjust the shooting angle, distance, and lighting conditions; An image quality evaluation module, configured to evaluate whether the received static image meets the standard. If not, it sends a re-acquisition signal to the oral image analysis module. If so, it saves the static image as a standard oral image.

[0032] The intelligent terminal device can be a portable device such as a smart phone or a tablet computer, which has a front camera, a display screen, a processor, and a memory. A dedicated oral examination application program is installed on the intelligent terminal device, and the program includes a device attitude detection module, a display interface adjustment module, an augmented reality rendering module, an oral image analysis module, a virtual character interaction control module, an image quality evaluation module, and a data storage and transmission module.

[0033] The intelligent oral examination interaction guidance system further includes a low-cost auxiliary dental mirror tool for assisting in obtaining images of the inner side of the oral cavity.

[0034] In this embodiment, the device attitude detection module uses sensors such as a gravity sensor and a gyroscope built into the intelligent terminal device to detect the holding attitude of the device in real time. When it detects that the device is in an inverted holding state (i.e., the screen is upside down and the front camera is located below the screen), it sends an attitude signal to the display interface adjustment module. This module can accurately identify the spatial position and rotation state of the device, and effectively eliminate misjudgment caused by slight shaking of the user's hand by fusing multi-sensor data, ensuring stable recognition of the holding attitude by the system.

[0035] In this embodiment, after receiving the attitude signal, the display interface adjustment module automatically adjusts the display direction of the application interface so that the interface content remains in the normal orientation relative to the user, so that the user can view the screen content normally while holding the device inverted. This module not only adjusts the screen display direction, but also optimizes the mapping relationship of touch operations, ensuring that the touch operations of the user in the inverted holding state have the same interaction logic as when holding the device normally, and reducing the learning cost.

[0036] As Figure 2 shown, it shows the conventional holding method of the intelligent terminal device. At this time, the front camera 1 above the intelligent terminal device is located above the screen, generating a line-of-sight conflict area 2 between the camera and the user's eyes observing the screen, resulting in difficulty for the user to view the screen while taking pictures of the oral cavity; Figure 3 It shows the inverted holding method of the intelligent terminal device of the present invention. At this time, the front camera 3 below the intelligent terminal device is located below the screen, eliminating the line-of-sight conflict. The user can view the screen while taking pictures, and at the same time, combined with the handheld auxiliary dental mirror tool 4, it is more convenient to observe the inner side of the oral cavity, greatly improving the operation convenience and effectiveness of oral image acquisition.

[0037] In this embodiment, the augmented reality rendering module is responsible for superimposing and displaying a virtual dentist character on the real-time image captured by the camera. First, this module uses the ORB-SLAM3 feature point extraction and matching algorithm to identify the spatial position and depth information of the oral cavity area, and then renders the virtual dentist character at an appropriate position according to this information, making it seem as if it really exists in the user's oral cavity environment. This module adopts advanced AR rendering technology, which can handle complex lighting conditions and occlusion relationships, ensuring that the virtual character can present natural and realistic visual effects in different environments. In addition, the augmented reality rendering module also realizes the spatial anchoring between the virtual character and the real environment. Even when the user moves the device, the virtual character can maintain a relatively stable spatial position, enhancing the coherence and immersion of the interaction.

[0038] The augmented reality rendering module uses a method that combines three-dimensional computer graphics and computer vision technologies to generate and display the virtual dentist character. The specific implementation includes the following steps: First, a high-quality virtual dentist character model is pre-created through professional 3D modeling software, including skeletal structures, facial features, and appearance features such as clothing. And corresponding skeletal animations and expression sequences are designed for each inspection action (such as opening the mouth, turning the head sideways, raising a hand to indicate, etc.), that is, a series of standard actions and expressions that meet the requirements of oral examination guidance are designed to form an action library and an expression library. These model and action data are stored in the system in an optimized lightweight format to ensure real-time rendering performance on mobile devices.

[0039] Second, a spatial perception algorithm is used to perform simultaneous localization and mapping on the image sequence input by the camera, extract environmental feature points and depth information in real time, establish a three-dimensional space coordinate system, and assign a fixed anchor point to the virtual dentist character in this coordinate system so that the relative position of the character and the real oral cavity scene can still be maintained when the device moves. Specifically, the system uses a spatial perception algorithm based on SLAM (Simultaneous Localization and Mapping) technology (such as the ORB-SLAM3 algorithm, the VINS-FUSION algorithm) to analyze the image sequence input by the camera in real time, extract environmental feature points and depth information, and construct a three-dimensional space coordinate system. Combining the data of the gyroscope and accelerometer built into the device, the system can accurately perceive the position and attitude changes of the device in three-dimensional space and provide a stable spatial anchor point for the virtual character.

[0040] Finally, in the rendering stage, the application program performs real-time rendering according to the process of "reading the current device pose updating the world coordinates of the virtual dentist character executing the target skeletal animation superimposing the virtual dentist character image layer on the camera screen" to achieve seamless superimposition and interactive guidance between the virtual dentist character and the user's oral cavity image.

[0041] Furthermore, a spatial anchor point allocation mechanism based on feature points is adopted to allocate fixed anchor points for the virtual dentist role. The specific steps include: extracting FAST corner points and BRIEF descriptors from the oral environment as feature points through the ORB-SLAM3 algorithm; optimizing the 3D positions of the feature points using the Bundle Adjustment algorithm to construct an environmental map; selecting the maxillary midline and the first mandibular molar as the main reference points based on oral anatomical features; in the initialization stage, the system calculates the relative distances and direction vectors between the virtual dentist role and these reference points; during the process of the user moving the device, the system continuously tracks the position changes of the reference points through feature point matching; according to the real-time positions of the reference points, the world coordinates of the virtual role are dynamically updated to keep the relative position relationship unchanged.

[0042] In this embodiment, the oral image analysis module processes both the preview frames and the automatically captured static images. The system performs real-time evaluation on the continuous preview frames of the camera to obtain dynamic indicators such as shooting angle, distance, and illumination, and immediately feeds back the evaluation results to the virtual character interaction control module. Specifically, the oral region is located through a facial key point detection algorithm (based on the MobileNet-SSD model), and combined with the device gyroscope data, the angle between the camera and the standard oral plane is calculated to obtain the shooting angle; combined with stereo vision depth estimation and distance calculation based on the target size, the actual distance between the device and the oral cavity is measured to obtain the shooting distance; through HSV color space analysis and regional brightness histogram statistics, the average image brightness, standard deviation, and illumination uniformity index are calculated to obtain the illumination condition; the Laplacian gradient operator is used to calculate the image sharpness score, and combined with contrast and texture feature analysis, the sharpness is obtained. When the real-time evaluation results meet the threshold conditions (i.e., each dynamic indicator meets the threshold conditions), the feedback stops, and the system automatically captures a high-resolution static image and stores it. Subsequent image quality evaluation and medical analysis are both performed based on this static image. Through the "real-time - capture" dual-channel mechanism, high-quality examination images can be obtained while ensuring the smoothness of the interaction.

[0043] Furthermore, in this embodiment, the threshold conditions for each dynamic indicator are set as follows: the shooting angle deviation is less than ; the shooting distance is within the range of 10 - 30 cm; the illumination uniformity index is greater than 0.7; the sharpness score is greater than the preset threshold (80 / 100). These threshold conditions are only examples and can be set according to the actual image quality requirements without specific limitations.

[0044] The oral image analysis module analyzes the oral images obtained by the camera in real time, identifies which oral area is being photographed, whether the shooting angle is appropriate, whether the image clarity and lighting conditions meet the standards, etc. This module uses deep learning algorithms and is trained with a large amount of oral image data. It can accurately identify oral tissues such as teeth, gums, and tongue, and evaluate the image quality. This algorithm can adapt to different oral morphologies and various lighting conditions, and continuously improve the recognition accuracy through continuous online learning. The real-time evaluation results will be transmitted to the virtual character interaction control module as the basis for guiding the interaction.

[0045] The oral image analysis module realizes accurate oral area recognition and quality evaluation based on deep learning technology. This module adopts a multi-stage cascaded convolutional neural network architecture, which includes two main parts: a feature extraction network and a multi-task processing network. The feature extraction network adopts an optimized MobileNet-V3 architecture, which is optimized for computational complexity and memory occupancy for mobile devices, and can achieve real-time processing while maintaining high accuracy. This network receives the original image (preview frame) captured by the camera, extracts rich visual features through multiple layers of convolution and pooling operations, and inputs the visual features into the multi-task processing network.

[0046] The multi-task processing network receives the visual features and performs three key tasks, including oral area segmentation, anatomical structure recognition, and image quality evaluation.

[0047] (1)Oral area segmentation The oral area segmentation task adopts a U-Net architecture to generate a pixel-level oral area mask, accurately distinguishing tissues such as teeth, gums, and tongue. The input of the U-Net architecture is the visual features, that is, the feature map extracted by the feature extraction network (with dimensions of ), restores the spatial details through transposed convolution and skip connections, and outputs a pixel-level oral area mask R (with dimensions of , and the value range is ).

[0048] (2)Anatomical structure recognition The anatomical structure recognition task identifies and locates the specific tooth positions and numbers through a region proposal network. The input of the region proposal network (RPN) is the visual features and the oral area mask R. It extracts fixed-size features through oROI-Align and outputs the bounding box coordinates, class labels (such as change 1-32), and confidence levels of each tooth.

[0049] (3)Image quality evaluation The image quality evaluation task calculates the comprehensive scores of quality indicators such as clarity, lighting uniformity, and angle appropriateness through a regression network. The regression network is constructed by global average pooling followed by a fully connected layer. Its input is the visual features and the output results of the anatomical structure recognition, and it outputs the shooting angle deviation angle (Degree), clarity (0 - 1), illumination uniformity (0 - 1) and suspected area mask of tooth / periodontal abnormalities , using a multi - head attention mechanism to focus on key areas, clarity and illumination uniformity are weighted and averaged to generate a comprehensive quality score .

[0050] The system uses knowledge distillation technology to transfer the knowledge of a teacher network pre - trained on a large - scale oral medicine image dataset to a lightweight student network, significantly reducing the model size and inference time while maintaining high accuracy. In addition, for the heterogeneous computing characteristics of mobile devices, the model is optimized by quantization and pruning, making full use of dedicated hardware acceleration units such as GPUs and NPUs to achieve real - time analysis performance of more than 25 frames per second.

[0051] Based on the real - time evaluation results, the system converts the real - time evaluation results into specific guiding strategies through a decision mapping engine. This engine is based on a pre - defined expert knowledge base and decision tree, considering factors such as the currently identified oral area, shooting angle deviation, lighting conditions, and clarity, to generate an optimal sequence of guiding actions. For example, when the system detects that the currently captured area is the right maxillary area, but the angle is low and the light is insufficient, the decision engine will generate a guiding strategy of "adjust 15 degrees upward and supplement light", and select the corresponding action and expression combination from the virtual character action library.

[0052] Specifically, first, construct an expert knowledge base and a decision tree, including: the expert knowledge base records the guiding strategies for different oral examination scenarios by conducting structured interviews with several (such as 10) dentists; use the decision tree learning algorithm C4.5 to extract rules from expert decisions to construct an initial decision tree; optimize the decision tree parameters from hundreds of (such as 500) real - user interaction data through machine learning methods; the final decision tree contains 72 decision nodes and 118 leaf nodes, covering common oral examination scenarios. Second, based on the oral image analysis results (i.e., oral area mask R, shooting angle deviation angle , clarity , illumination uniformity ), the decision tree first determines the current examination stage (such as anterior tooth area, left tooth area, etc.); determines the main adjustment direction according to the shooting angle deviation to generate a direction adjustment strategy (such as "adjust 15 degrees upward"); based on clarity and illumination uniformity Determine secondary adjustment strategies (such as "increase light source", "decrease distance"); comprehensively consider the user operation history and feedback, adjust the strategy priority and execution order; map the final strategy to a predefined sequence of guiding actions (such as pointing, adjusting posture, etc.); dynamically generate guiding action parameters (such as pointing direction, adjustment amplitude, etc.) and pass them to the virtual character interaction control module.

[0053] In this embodiment, the virtual character interaction control module controls the virtual dentist character to perform corresponding guiding actions and expressions according to the oral image analysis results (real-time evaluation results) and the preset oral examination process, guiding the user to adjust the shooting angle, distance and light conditions. Specifically, the decision mapping engine maps the optimal sequence of guiding actions to the nodes of a Behavior Tree, driving the action skeleton and blend-shape expressions to execute synchronously. For example, when it is detected that the user's shooting angle is inappropriate, the virtual dentist will make a gesture to guide left or right; when it is detected that the light is insufficient, it will make an action to prompt filling light; when the user needs to open their mouth wide, the virtual dentist will make an action to demonstrate opening the mouth, etc. This module implements a complete virtual character action control system, including multi-dimensional control such as skeletal animation, facial expressions and gesture actions, and can generate smooth and natural sequences of guiding actions.

[0054] At the same time, this module also integrates situational awareness capabilities, which can adjust the guiding strategy according to the user's reactions and interaction process to achieve a more personalized and adaptable interaction experience. Specifically, it detects the user's emotional state (focused, confused, impatient, etc.) through a facial expression recognition algorithm; analyzes the user's response time and execution quality to the guidance to evaluate the degree of understanding; tracks the consistency between the user's operations and the recommended actions, and calculates the compliance index. A strategy dynamic adjustment mechanism is used to adjust the guiding strategy. When it is detected that the user fails to adjust according to the recommendation 3 times in a row, the system reduces the guiding complexity and provides more basic and intuitive guiding actions; when the user's compliance index is lower than 0.5, the significance of visual cues and the frequency of voice cues are increased.

[0055] To implement the above guiding strategy, the virtual character interaction control module uses parametric animation synthesis technology to blend the basic actions in real time through the Bezier curve interpolation algorithm to generate smooth and natural guiding actions. This module also implements action priority management to ensure that when multiple guiding requirements exist simultaneously, the system can present them one by one in a reasonable order to avoid the user receiving conflicting guiding information.

[0056] Furthermore, the steps to implement the action priority management function are: establish a hierarchical priority framework: The first level is fully relevant guidance (such as avoiding misoperations, preventing equipment from falling, etc.); Level is for critical quality impact guidance (such as severe angle deviation, extreme blurriness, etc.); Level is for conventional quality optimization guidance (such as slight angle adjustment, lighting optimization, etc.). Is for auxiliary enhancement guidance (such as detection integrity prompt, progress prompt, etc.); Conflict resolution mechanism: Use a guidance queue based on time window, and high-priority guidance can interrupt low-priority guidance; After a guidance task is interrupted, it enters the delay queue and the execution time is rearranged according to the priority; For guidance with the same priority, a weighted round-robin scheduling algorithm is adopted to avoid a certain type of guidance continuously occupying resources. Context-aware scheduling: The system analyzes the current operation stage and user status, and dynamically adjusts the priority threshold; At critical shooting moments (such as when aligning with a specific area), non-critical guidance is temporarily suppressed; For tasks that the user repeatedly fails to execute, the priority of their associated guidance is automatically increased.

[0057] When the system detects that the user uses the auxiliary dental mirror tool, the virtual character interaction control module will provide targeted guidance, demonstrating how to correctly hold the dental mirror, how to adjust the angle of the dental mirror to reflect the internal image of the oral cavity, and how to maintain stability to obtain a clear image. This module uses computer vision algorithms to identify the position and orientation of the dental mirror, combines the spatial relationship of the oral cavity area, calculates the optimal dental mirror usage posture, and intuitively conveys it to the user through the actions of the virtual character.

[0058] Furthermore, the steps of the guidance interaction for using the auxiliary dental mirror tool are as follows: (1) Dental mirror detection and recognition Use the YOLOv5-tiny model to detect the dental mirror in real time, and the recognition rate , and the inference time ; Locate the reflective surface of the dental mirror through the circular Hough transform algorithm, with an accuracy of ; Use the PnP (Perspective-n-Point) algorithm to calculate the position and pose of the dental mirror in 3D space.

[0059] (2) Construct the spatial relationship of the oral cavity Based on the oral cavity segmentation result (oral cavity area mask R), construct a simplified 3D oral cavity model, including the tooth plane and the opening boundary; Calculate the spatial relationship between the reflective surface of the dental mirror and the target area (such as the posterior teeth, the lateral surface of the tongue); Apply the ray tracing algorithm to simulate the reflection path of the dental mirror and determine the optimal viewing angle.

[0060] (3) Calculate the optimal dental mirror usage posture Through a geometric optimization algorithm, the following factors are comprehensively considered to calculate the optimal posture: maximizing the visible area of the target region in the dental mirror, minimizing the holding difficulty and hand fatigue, and optimizing the shooting angle to reduce reflection and distortion; The generated posture parameters include: the insertion depth of the dental mirror, the angle, the rotation direction, and the wrist posture.

[0061] (4) Implement virtual guidance The virtual dentist role demonstrates the correct holding and using postures of the dental mirror through decomposed actions; Use a semi-transparent overlay to mark the ideal position and orientation of the dental mirror in the real-time video; Provide real-time distance feedback and offer visual confirmation when the user's dental mirror approaches the ideal position.

[0062] Furthermore, the workflow of the dental mirror used in conjunction with the smart terminal is as follows: The user first inserts the dental mirror into the oral cavity and adjusts it near the target area; The system identifies the position of the dental mirror and calculates the optimal adjustment direction; The virtual dentist guides the user to fine-tune the angle of the dental mirror until the optimal reflection position; After reaching the optimal position, the system locks the focus on the reflection surface of the dental mirror; The user keeps the dental mirror stable, and the system automatically captures the image at the optimal time.

[0063] In this embodiment, the image quality evaluation module comprehensively evaluates the oral images collected by the user (i.e., the automatically captured static images), evaluates whether the received static images meet the standards. If not, it sends a signal to the oral image analysis module for re-acquisition. If so, it saves the static images as standard oral images. It also gives a quality score and provides improvement suggestions for the non-compliant static images. Based on professional oral image evaluation criteria, this module quantitatively evaluates the images from multiple dimensions such as clarity, brightness, contrast, shooting angle, and coverage range, and generates a comprehensive quality score. Specifically, according to the MQE (Medical-image Quality Estimator) framework, by fusing BRISQUE, NIQE, and the oral-specific illumination index, and through XGBoost regression, the comprehensive quality score is obtained. For the non-compliant regions, the system will mark the specific problems and provide targeted improvement suggestions, such as "insufficient light in the right molar region", "excessive shooting angle in the anterior tooth region", etc.

[0064] The specific steps of oral image quality evaluation are as follows: (1) Calculate multi-dimensional evaluation metrics Clarity: Calculate the variance of the gradient magnitude using the Laplace operator, combined with high-frequency component energy analysis; Brightness: Through brightness histogram analysis, calculate the average brightness and standard deviation to evaluate the overall exposure level; Contrast: Use the RMS contrast algorithm to evaluate the local and global contrast of the image; Shooting Angle: Based on the geometric features of tooth arrangement, estimate the deviation between the shooting perspective and the standard perspective; Coverage: Calculate the coverage ratio and integrity of the target oral area in the image; (2)Quality scoring based on multi-dimensional evaluation metrics (MQE framework quality scoring process) Input the oral image (original image) and the extracted oral area mask into the preprocessing module; Parallelly calculate the evaluation metric values of each dimension (sharpness S_raw, brightness B_raw, etc.); Apply the BRISQUE (Blind / Referenceless Image Spatial Quality Evaluator) algorithm to calculate the reference-free quality score; Combine the special oral lighting index (considering the special reflection and shadow characteristics in the oral cavity) to standardize all metrics to the [0,1] interval and input them into the XGBoost model; The XGBoost model outputs the final comprehensive quality score (0 - 100).

[0065] (3)Identify problem areas based on threshold analysis and generate suggestions using the suggestion generation algorithm Regions with sharpness lower than the threshold S_th are marked as "blurry regions", regions with brightness lower than B_low or higher than B_high are marked as "too dark" or "too bright" respectively, and regions with shooting angle deviation exceeding θ_th are marked as "improper angle"; Calculate the severity and the proportion of the affected area of each type of problem to evaluate the problem severity; Map the detected problems to the predefined improvement suggestion library through a decision tree; Based on the problem severity and the user ability model, select the most suitable expression; Generate specific and actionable suggestions, such as "The light in the right molar area is insufficient. Please get closer to the window light source or turn on the fill light"; For compound problems, sort the suggestion sequence according to the solution priority.

[0066] In this embodiment, the data storage and transmission module is responsible for saving the high-quality oral images (i.e., standard oral images) completed in acquisition to the local or transmitting them to the cloud server for subsequent software analysis or human doctor diagnosis. This module realizes the secure encrypted storage and efficient transmission of data. It adopts a hierarchical encryption strategy and a differential compression algorithm, optimizing the network transmission efficiency while ensuring user privacy and data security, and adapting to the data transmission requirements under various network environments. In addition, this module also provides multiple data sharing options, supporting users to securely share the acquired oral images with designated doctors or medical institutions for obtaining professional diagnostic opinions.

[0067] In summary, standard oral images are acquired through the front camera of the intelligent terminal device. The device-inverted holding method is innovatively adopted, and the virtual dentist role is superimposed and displayed in the real-time image in combination with the augmented reality technology. This virtual role guides the user to adjust the shooting angle, distance, and light conditions through intuitive actions and expressions, and combines a low-cost auxiliary dental mirror tool to obtain the inner oral cavity image. The system analyzes the image quality in real time, and the virtual role conducts targeted guidance to finally obtain high-quality oral images.

[0068] Through the combination of augmented reality technology and appropriate human-computer interaction design, the present invention provides intuitive and friendly oral examination guidance for ordinary users, ensures the acquisition of high-quality oral images, and thus improves the effect of oral self-examination and the feasibility of remote oral medicine.

[0069] Embodiment 2 This embodiment discloses an intelligent oral examination interactive guidance method based on augmented reality, including: Start the oral examination application program on the intelligent terminal device, activate the front camera, and enter the augmented reality interaction mode; Detect the holding posture of the intelligent terminal device. When it is detected that the intelligent terminal device is in the preset inverted holding state, automatically adjust the display interface direction; Superimpose and display the virtual dentist role in the real-time camera image obtained by the camera to guide the user to conduct an oral examination and take standard oral images; During the oral examination guidance process, evaluate the preview frame of the oral image in real time, obtain the real-time evaluation result based on dynamic indicators, and feedback the real-time evaluation result; when the real-time evaluation result meets the threshold condition, stop the feedback and automatically capture a static image; According to the real-time evaluation result and the preset oral examination process, control the virtual dentist role to perform corresponding guiding actions and expressions to guide the user to adjust the shooting angle, distance, and light conditions; Evaluate whether the received static image meets the standard. If not, send a re-acquisition signal to the oral image analysis module. If so, save the static image as a standard oral image.

[0070] In this embodiment, as Figure 4 shown, the detailed steps of the intelligent oral examination interactive guidance method based on augmented reality are as follows: S1: The user starts the oral examination application on the intelligent terminal device, and the system automatically activates the front camera and enters the augmented reality interaction mode. During this process, the system loads the virtual dentist role model and related resources to prepare for the oral examination guidance. The system also checks the device performance and ambient light conditions, and gives optimization suggestions if necessary to ensure the best AR experience. During the loading process, the system will display a short usage tutorial and health tips to help users understand the basic operation methods and oral health knowledge.

[0071] S2: The system detects the holding posture of the intelligent terminal device. When it detects that the device is in the preset inverted holding state (i.e., the screen is upside down, with the front camera located below the screen), the system automatically adjusts the display interface orientation so that the user can view the screen content normally while holding the device inverted. If the device is not in the inverted holding state, the virtual dentist role will guide the user to invert the device through actions and text prompts to obtain a better operation experience. The system uses an intelligent detection algorithm to distinguish between the user's intentional operations and unintentional shakes, avoiding the problem of frequent interface switching caused by overly sensitive posture detection.

[0072] S3: The system superimposes and displays the virtual dentist role in the real-time image captured by the camera. The virtual role initially appears with a friendly expression and a welcome gesture, and introduces the purpose and process of this oral examination through short voice or text to relieve the user's nervousness, especially to establish affinity with child users. The system will adjust the interaction method of the virtual role according to the user's reactions (such as facial expressions, head movements, etc.) to ensure the natural and smooth interaction. The appearance and voice of the virtual role can be personalized according to the user's age and preferences to enhance user acceptance.

[0073] S4: The system starts to guide the user to prepare the oral examination environment. The virtual dentist role guides the user to adjust the light conditions (such as facing the natural light source or turning on the indoor lighting) through actions and expressions, checks whether the camera is clean, and prepares the auxiliary dental mirror tool (if needed). During this process, the system analyzes the ambient light and the camera imaging quality in real time. When the conditions meet the requirements, the virtual dentist gives a confirmation gesture and proceeds to the next step. The system will provide personalized optimization suggestions for different ambient light conditions, such as suggesting turning on the device's fill light function in a low-light environment, or adjusting the angle to avoid glare in a strong light environment.

[0074] S5: The system guides the user to perform oral image acquisition area by area according to a preset oral examination process. The usual examination sequence is: anterior tooth area (incisors and canines), right tooth area (right premolars and molars), left tooth area (left premolars and molars), maxillary occlusal surface, mandibular occlusal surface, and soft tissues (tongue, oral mucosa, etc.). For each area, the system performs the following sub-steps: S501: The virtual dentist role guides the user to aim the camera at a specific oral area through action demonstrations. For example, for the anterior tooth area examination, the virtual dentist will make a gesture pointing to the anterior teeth and may show a semi-transparent target frame to indicate the ideal shooting range. The system dynamically adjusts the position and size of the target frame according to the user's actual oral structure to ensure the accuracy of the guidance. At the same time, the virtual dentist will use simple and clear voice prompts, such as "Please aim at the front teeth", "Open your mouth a little wider", etc., to cooperate with the action guidance and improve the user's understanding.

[0075] S502: The system analyzes the oral images obtained by the camera in real time, evaluates the shooting angle, distance, and clarity. Based on the analysis results, the virtual dentist role performs corresponding guiding actions, such as gestures of left, right, up, down, closer, farther away, etc., to help the user adjust to the best shooting position. The system's analysis algorithm can identify minor angle deviations and focal length problems and provide fine adjustment guidance. To enhance the intuitiveness of the feedback, the system may also display direction indicator arrows or distance indicator bars on the interface to complement the action guidance of the virtual role.

[0076] S503: When the system detects that the current image quality reaches the preset standard, the virtual dentist role gives a gesture or expression of shooting confirmation (such as giving a thumbs up or nodding and smiling), and the system automatically captures the high-quality image of this area and saves it. The system adopts an intelligent trigger mechanism to automatically capture at the moment when the image quality meets the standard, avoiding device shaking and image blurring caused by the user manually clicking the screen. At the same time, the system will give positive voice feedback, such as "Very good! This photo is very clear", etc., to enhance the user's sense of achievement and participation.

[0077] S504: The system enters the examination guidance for the next oral area and repeats steps S501 to S503 until the image acquisition of all areas is completed. During the area switching process, the system will display the current examination progress and the prompt for the next area to help the user understand the overall examination progress. For areas that may be difficult for some users (such as the posterior molar area), the system will give more detailed operation tips and multiple optional shooting methods in advance to increase the success rate.

[0078] S6: For oral areas that require the use of auxiliary dental mirror tools (such as the inner side of teeth and the posterior molar area), the system performs a special guiding process: S601: The virtual dentist role guides the user to hold the dental mirror tool correctly through action demonstrations. The system identifies the dental mirror in the user's hand and provides targeted guidance to ensure that the dental mirror is in the appropriate position and safe. The system will detect the shape and position of the dental mirror through computer vision algorithms and identify whether the user's holding posture is correct. The virtual dentist will demonstrate the correct holding method, such as "hold the lower part of the dental mirror and keep your fingers away from the mirror surface", etc., and give real-time feedback and adjustment suggestions based on the user's actual operation.

[0079] S602: The virtual dentist role demonstrates how to use the smart terminal device to take pictures in cooperation with the dental mirror, including how to adjust the angle of the dental mirror to reflect the internal oral image and how to keep stable to obtain a clear image. The system will provide detailed step-by-step guidance, such as first putting the dental mirror into the mouth, adjusting it to the appropriate position, and then taking pictures of the reflected image in the dental mirror, etc. The action demonstrations of the virtual dentist will consider the ergonomic characteristics of the actual operation to ensure that the guiding actions are easy to imitate and execute.

[0080] S603: The system analyzes the oral images obtained through the reflection of the dental mirror in real time, evaluates the quality and provides adjustment suggestions. For the dental mirror reflection images, the system uses special image processing algorithms that can handle problems such as image flipping and distortion caused by mirror reflection, and accurately evaluate the image quality. The system will provide solutions to common dental mirror usage problems, such as "there is fog on the dental mirror, please wipe it gently", "the angle of the dental mirror is too large, please adjust it slightly", etc. When the image quality meets the standard, the system automatically captures and saves it.

[0081] S7: After completing the image acquisition of all oral regions, the system conducts a comprehensive quality assessment of the complete set of oral images collected. The assessment content includes the integrity of the image coverage area, the clarity of the images in each region, the lighting uniformity, etc. The system uses a comprehensive quality assessment model that comprehensively considers professional oral examination standards and image technical indicators to objectively evaluate the acquisition results. The assessment process will generate a detailed quality report, including the scores of each region, existing problems, and the overall rating.

[0082] S8: Based on the quality assessment results, the system gives an overall score and improvement suggestions. If it is found that the image quality of some regions does not meet the standard, the virtual dentist role will guide the user to re-acquire the images of these regions until the image quality of all images meets the requirements. The system will give priority to guiding the user to re-acquire the key regions or the regions with the most serious problems and provide more specific improvement suggestions, such as "the lighting was insufficient when taking pictures of the right molar region last time, please get closer to the window or turn on the light and then re-take the pictures", etc. For regions that are still difficult to meet the standard after multiple attempts, the system will provide alternative solutions or simplify the requirements to ensure that the basic examination needs can be met.

[0083] S9: The system saves or transfers the complete high-quality oral image set to the cloud server for subsequent software analysis or human doctor diagnosis. During this process, the virtual dentist role will give a confirmation feedback on the completion of the operation and may provide oral health advice based on preliminary AI analysis (if this function is available). Data transmission uses segmented encryption and resume interrupted transfer technologies to ensure safe and efficient transmission under various network conditions. After the transmission is completed, the system will provide guidance on subsequent steps, such as how to view the analysis results and how to contact professional doctors for remote consultation, etc.

[0084] Embodiment III The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps of the method in Embodiment II.

[0085] Embodiment IV The purpose of this embodiment is to provide a computer-readable storage medium. A computer-readable storage medium has a computer program stored thereon. When the program is executed by a processor, it executes the steps of the method in Embodiment II.

[0086] The steps involved in the devices in the above Embodiments III and IV correspond to those in Method Embodiment I. For specific implementation manners, reference may be made to the relevant description part of Embodiment II. The term "computer-readable storage medium" should be understood to include a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0087] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. The present invention is not limited to any specific combination of hardware and software.

[0088] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0089] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, they are not limitations on the protection scope of the present invention. Those skilled in the art should understand that various modifications or deformations that can be made without creative efforts on the basis of the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. An intelligent oral examination interactive guidance system based on augmented reality, characterized in that, It includes a smart terminal device and an installed oral examination application program; the oral examination application program includes: A device attitude detection module, which is used to detect the holding attitude of the smart terminal device. When it detects that the smart terminal device is in a preset inverted holding state, it sends an attitude signal to the display interface adjustment module to automatically adjust the display interface direction; An augmented reality rendering module, which is used to superimpose and display a virtual dentist character in the real-time camera image, guide the user to conduct an oral examination, and take standard oral images; An oral image analysis module, which is used to evaluate the oral image preview frames in real time during the oral examination guidance process, obtain real-time evaluation results based on dynamic indicators, and feedback the real-time evaluation results to the virtual character interaction control module; it is also used to stop the feedback when the real-time evaluation results meet the threshold conditions, and automatically capture static images and send them to the image quality evaluation module; A virtual character interaction control module, which is used to control the actions and expressions of the virtual dentist character according to the real-time evaluation results and the preset oral examination process, and guide the user to adjust the shooting angle, distance, and lighting conditions; An image quality evaluation module, which is used to evaluate whether the received static image meets the standard. If not, it sends a re-acquisition signal to the oral image analysis module. If so, it saves the static image as a standard oral image.

2. The augmented reality-based intelligent oral examination interactive guidance system according to claim 1, characterized in that, The device attitude detection module detects the holding state in real time through the sensors built in the smart terminal device. When it recognizes that it is in an inverted holding state, it sends an attitude signal to the display interface adjustment module.

3. The intelligent oral examination interaction guidance system based on augmented reality according to claim 1, characterized in that The augmented reality rendering module generates and displays a virtual dentist character, specifically including: Pre-creating a virtual dentist character model through 3D modeling software, and designing corresponding skeletal animations and expression sequences for each inspection action; Using a spatial perception algorithm to perform simultaneous localization and mapping on consecutive oral image preview frames, extracting environmental feature points and depth information in real time, establishing a three-dimensional space coordinate system, and assigning fixed anchor points to the virtual dentist character in this coordinate system; In the rendering stage, the application program reads the current device attitude in sequence for each frame, updates the world coordinates of the virtual dentist character, executes the target skeletal animation, and superimposes the virtual dentist character image layer on the camera screen to complete real-time rendering.

4. The augmented reality-based intelligent oral examination interactive guidance system according to claim 1, wherein The oral image analysis module adopts a multi-stage cascaded convolutional neural network architecture, including a feature extraction network and a multi-task processing network connected in sequence; the feature extraction network receives the oral image preview frames, extracts visual features through multiple layers of convolution and pooling operations, and inputs the visual features into the multi-task processing network; the multi-task processing network uses a U-Net structure to segment the visual features to generate an oral region mask, identifies and locates the teeth through a region proposal network, and finally obtains the oral image analysis result through a regression network.

5. The augmented reality-based intelligent oral examination interactive guidance system according to claim 4, wherein Converting the oral image analysis result into a guidance strategy through a decision mapping engine.

6. The augmented reality-based intelligent oral examination interaction guidance system according to claim 1, characterized in that, It also includes an auxiliary dental mirror tool. The virtual character interaction control module identifies the position and orientation of the dental mirror through computer vision algorithms, combines the oral spatial relationship, calculates the optimal dental mirror usage posture, and displays it through the virtual dentist character.

7. The augmented reality-based intelligent oral examination interactive guidance system according to claim 1, wherein The quality assessment of the static image by the image quality assessment module is specifically as follows: calculating multi-dimensional assessment indicators of the static image, identifying problem areas using threshold analysis based on the multi-dimensional assessment indicators, and generating suggestions using a suggestion generation algorithm.

8. An intelligent oral examination interactive guidance method based on augmented reality, characterized in that, It includes: Starting the oral examination application program on the intelligent terminal device, activating the front camera and entering the augmented reality interaction mode; Detecting the holding posture of the intelligent terminal device, and automatically adjusting the display interface direction when it is detected that the intelligent terminal device is in a preset inverted holding state; Overlaying and displaying a virtual dentist character in the real-time camera image obtained by the camera to guide the user to perform an oral examination and take a standard oral image; Real-time evaluating the preview frame of the oral image during the oral examination guidance process, obtaining a real-time evaluation result based on dynamic indicators, and feeding back the real-time evaluation result; stopping the feedback and automatically capturing a static image when the real-time evaluation result meets the threshold condition; Controlling the virtual dentist character to perform corresponding guiding actions and expressions according to the real-time evaluation result and the preset oral examination process, and guiding the user to adjust the shooting angle, distance, and lighting conditions; Evaluating whether the received static image meets the standard, if not, sending a re-acquisition signal to the oral image analysis module, and if so, saving the static image as a standard oral image.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the augmented reality-based intelligent oral examination interactive guidance method described in claim 8.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the augmented reality-based intelligent oral examination interactive guidance method described in claim 8.

Images