Oral cavity operation automatic three-dimensional evaluation system based on Gaussian splashing technology

Abstract

The invention discloses an automatic three-dimensional evaluation system for oral operation based on a Gaussian splashing technology, and relates to the cross technical field of medical image processing, computer vision and oral medicine education, the system comprises six core modules, a student end acquisition module supports mobile terminal multi-view image and video acquisition and oral scanning data uploading; the scene preprocessing module carries out standardization processing on the collected data; the Gaussian splash three-dimensional reconstruction module generates a three-dimensional model with real textures based on the camera pose and the image data; the preparation quality evaluation module realizes automatic scoring and visual feedback through model registration and difference quantification; the clinical case teaching library module realizes structured storage and retrieval of case models and related information; and the interactive display module supports cross-terminal model operation, annotation and remote recheck. The system does not need high-end equipment, realizes automatic and standardized evaluation of oral operation, adapts to multi-scene teaching requirements, and improves evaluation efficiency and consistency.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of medical image processing, computer vision, and dental education, specifically to an automated three-dimensional evaluation system for dental procedures based on Gaussian splashing technology. Background Technology

[0002] In current oral clinical medicine teaching, after students complete operations such as tooth preparation, root canal access, and restoration adjustment in extracted model teeth, simulated head molds, or real patient mouths, the evaluation methods are generally divided into two categories: one is the traditional visual observation + simple probing, with the instructor giving a subjective score on the spot based on experience; the other is to rely on an intraoral scanner (oral scanner) or desktop scanner to obtain three-dimensional data of the tooth or restoration after the operation, and then compare and interpret it manually or semi-manually. Therefore, an automatic three-dimensional evaluation system for oral operations based on Gaussian splash technology has emerged.

[0003] Existing technology, such as the invention application patent with publication number CN106652710A, discloses a virtual dental training system based on virtual reality and pose sensing technology. It provides a set of hardware and a software system to achieve better, faster, and safer improvement of clinical dental skills. The hardware includes a spatial pose capture and sensing module, a bionic head module, an automatically recognizable replaceable maxillary module, an automatically recognizable replaceable mandibular module, and a pneumatic drive module. The software system includes a virtual 3D operation module, a digital human head model module, a fully automatic objective evaluation module, a knowledge learning and assessment module, a growth curve module, an open expansion module, and a user management module. The software system automatically presents different cases based on different maxillary and mandibular modules. Operational data is captured by the spatial pose capture and sensing module and simultaneously presented in the virtual 3D operation module and the digital human head model module. The system evaluates the data from each module in the fully automatic objective evaluation module to generate reference scores. Finally, the score curve is presented in the growth curve module.

[0004] Regarding the above-mentioned solutions, the inventors of this application have discovered that the above-mentioned technologies have at least the following technical problems: 1. The evaluation is highly dependent on manual labor and is difficult to automate and standardize. Whether it is traditional visual observation or manual measurement in software after obtaining three-dimensional data based on oral scanning, the evaluation process relies heavily on the teacher's subjective judgment and manual operation, making it difficult to achieve "automatic scoring by the system," which is not conducive to forming a unified and quantifiable evaluation standard. The workload of teachers is heavy, making it difficult to support large-scale online assessment and remote guidance.

[0005] 2. Existing oral scanners suffer from high costs, complex processes, and insufficient online integration. High-performance oral scanners are expensive and complex to maintain, making them difficult to popularize in ordinary colleges and universities. The scanning process usually needs to be completed in a specific environment by trained operators, and it is difficult to form a seamless online workflow for data uploading, processing, and evaluation, making it unsuitable for students to use frequently in various scenarios such as dormitories and internship sites.

[0006] 3. The disconnect between existing 3D reconstruction methods and teaching evaluation. Existing 3D reconstruction technologies, such as Gaussian splashing, mainly focus on imaging performance and have not yet been deeply integrated with the standard operating model library, scoring rules, and teaching management system for oral teaching. They lack a complete closed-loop technical solution from "image acquisition—3D reconstruction—automatic comparison and scoring—teacher review and feedback," and thus cannot fully realize their potential for online automatic scoring and unified standards.

[0007] 4. Lack of a unified evaluation platform supporting remote, multi-teacher evaluation. Existing oral scan data and results are mostly stored on local devices or offline software, which makes it inconvenient for multiple teachers in different locations and at different times to conduct unified review and verification based on the same 3D model. It is also difficult to provide unified online evaluation services for multiple colleges and clinical teaching sites. Summary of the Invention

[0008] To address the aforementioned technical shortcomings, the present invention aims to provide an automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides an automatic three-dimensional evaluation system for oral operation based on Gaussian splashing technology, including the following modules: Student-end acquisition module: used to guide students to acquire multi-view images and videos of extracted model teeth or patient oral cavity through mobile terminals, and at the same time, supports uploading three-dimensional scanning data obtained by intraoral scanner.

[0010] Scene preprocessing and camera pose estimation module: used to perform distortion correction, region of interest segmentation and multi-view camera pose estimation on the acquired image data, and to perform coordinate standardization and noise filtering on the 3D scan data.

[0011] Gaussian splash 3D reconstruction module: used to construct a 3D Gaussian representation of a dental scene based on camera pose and image data and complete optimization training, generate a Gaussian splash 3D model with realistic texture, adapt the Gaussian splash 3D model to a conventional server configuration, and convert the model to a unified format to adapt to the evaluation process.

[0012] The preparatory quality assessment and visualization feedback module is used to spatially register the student operation result model with the preset standard model, automatically calculate various morphological difference indicators, and generate sub-item and total scores according to the scoring rules, as well as generate difference heatmaps and three-dimensional visualization feedback content.

[0013] The Clinical Case 3D Teaching Database Management Module is used for the structured storage and retrieval of the Gaussian splash 3D models of clinical cases reconstructed by Gaussian splashing, along with their corresponding diagnostic and treatment information and automatic scoring results.

[0014] The interactive display and remote guidance module for teachers and students is used to load Gaussian splash 3D models and scoring results on mobile terminals or web pages, enabling model rotation, scaling, slicing, annotation functions, and voice and text feedback interaction. At the same time, it supports teachers to review and supplement the system's automatic scoring based on unified standards.

[0015] The beneficial effects of this invention are as follows: 1. It automates the evaluation process and significantly reduces manual operations. This invention uses Gaussian splash 3D reconstruction and automatic registration, along with difference analysis algorithms, to directly provide multi-dimensional quantitative indicators and total scores, achieving "automatic scoring." Teachers are freed from the heavy workload of measurement, screenshotting, and comparison, and only need to conduct necessary reviews based on unified standards, greatly improving evaluation efficiency.

[0016] 2. Achieving unified and replicable scoring standards. By pre-setting a standard operating model library and providing clear quantitative indicators and scoring rules, this invention can ensure that different teachers and teaching locations are evaluated under the same set of parameters and thresholds, significantly improving scoring consistency and repeatability, and providing a unified standard for large-scale, cross-institutional teaching and assessment.

[0017] 3. Lowering hardware barriers and improving ease of use. This invention uses ordinary smartphones or conventional cameras as the main acquisition terminals, with optional compatibility with port scanning data. This enables high-fidelity 3D recording and automatic evaluation even in teaching environments lacking high-end port scanning equipment or GPU resources, facilitating wider application in more schools, internship institutions, and individual students.

[0018] 4. Construct a closed-loop teaching platform that can be used online. This invention integrates image acquisition, 3D reconstruction, automatic scoring, report generation, and remote guidance into a unified online platform. It supports students in submitting operation results frequently inside and outside the classroom and receiving rapid feedback. Teachers can log in to the system remotely to view and provide guidance, which is conducive to building a digital skills portfolio that spans the entire learning process.

[0019] 5. Easily complements and upgrades existing oral scanning technologies. This invention can independently achieve low-cost 3D reconstruction based on images and Gaussian splashing, or directly access the mesh data generated by existing oral scanning as input to the automatic scoring pipeline. Thus, while retaining the geometric accuracy advantages of oral scanning, it upgrades the evaluation process from "manual interpretation" to "system automatic scoring + teacher review," achieving a smooth upgrade to the existing digital teaching model. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the system structure connection of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Examples of embodiments of the present invention Figure 1 As shown, the automatic 3D evaluation system for oral procedures based on Gaussian splashing technology includes the following modules: student-side data acquisition module, scene preprocessing and camera pose estimation module, Gaussian splashing 3D reconstruction module, preparatory quality evaluation and visualization feedback module, clinical case 3D teaching library management module, and teacher-student interactive display and remote guidance module.

[0024] The scene preprocessing and camera pose estimation module is connected to the student-end acquisition module and the Gaussian splash 3D reconstruction module, respectively. The pre-quality evaluation and visualization feedback module is connected to the Gaussian splash 3D reconstruction module and the clinical case 3D teaching library management module, respectively. The clinical case 3D teaching library management module is connected to the teacher-end and student-end interactive display and remote guidance module.

[0025] Student-side data acquisition module: This module guides students to acquire multi-view images and videos of extracted model teeth or patient oral cavity via mobile terminals. It also supports uploading 3D scan data obtained from an intraoral scanner.

[0026] In one specific embodiment, the method guides students to acquire multi-view images and videos of extracted model teeth or patient oral cavity via mobile terminals. The specific acquisition process is as follows: After starting the system, the student selects the corresponding tooth position and operation type. The system guides the student through the shooting trajectory unit of the acquisition module, prompting the student to shoot around the target area along three circumferential tracks (horizontal, top-down, and bottom-up) using screen arrows, virtual trajectories, or augmented reality markers. During the shooting process, the shooting quality detection unit evaluates the image clarity, exposure parameters, and viewing angle coverage in real time. When the key area has insufficient viewing angle, the image is blurry, or it is overexposed, a prompt to reshoot is issued. The acquisition parameters meet the following requirements: the number of shots is 20-80, or the continuous video duration is 5-20 seconds. After acquisition, the data can be directly uploaded to the system, and it is also compatible with the upload and adaptation of 3D data from intraoral scanners.

[0027] Scene preprocessing and camera pose estimation module: used to perform distortion correction, region of interest segmentation and multi-view camera pose estimation on the acquired image data, and to perform coordinate standardization and noise filtering on the 3D scan data.

[0028] In a specific embodiment, the distortion correction, region of interest segmentation, and multi-view camera pose solving of the acquired image data are performed as follows: First, the image preprocessing unit performs white balance correction, noise suppression, and background segmentation on the acquired images to highlight the dental arch, model teeth, or related soft tissue areas, and removes unqualified frames that are blurred or severely occluded.

[0029] Secondly, the camera intrinsic parameter calibration unit estimates the camera intrinsic parameters based on pre-calibrated mobile phone camera parameters or calibration boards placed in the scene, thereby unifying the imaging parameters of different devices.

[0030] Then, the multi-view pose solving unit calculates the camera extrinsic parameters corresponding to each frame image through feature matching and bundle adjustment algorithms, and performs global optimization to construct a complete camera pose set.

[0031] Finally, coordinate standardization is performed on the uploaded 3D oral scan data to unify the coordinate system. At the same time, a noise filtering algorithm is used to remove redundant and interfering information from the data to ensure data compatibility with subsequent modules.

[0032] It should be noted that white balance correction uses the gray-world algorithm to adjust the image gray-level mean to 128±5. Noise suppression is achieved through a combination of median filtering (3×3 window size) and Gaussian filtering (standard deviation 0.8). Background segmentation uses a semantic segmentation model to ensure target region extraction accuracy ≥95%. Blurred frame determination uses an image sharpness index ≥0.6 as the threshold, and frames with an occlusion area exceeding 30% of the target area are directly discarded. Camera intrinsic parameter calibration supports Zhang's calibration method, and a checkerboard template (5mm×5mm square size) is used for the calibration board. The intrinsic parameter estimation error is... ≤1%; Feature matching uses ORB algorithm to extract feature points, and high-quality matching pairs are selected through Lowe ratio test (ratio 0.75). The binding adjustment algorithm uses reprojection error ≤0.8 pixels as the convergence condition. After global optimization, the inter-frame pose consistency error is ≤0.5° and the translation deviation is ≤1mm. Oral scan data coordinate standardization establishes a unified coordinate system with the dental anatomy center as the origin. Noise filtering uses bilateral filtering algorithm to retain model details while controlling point cloud noise within 0.01mm, ensuring that the output data meets the accuracy requirements of subsequent 3D reconstruction and evaluation.

[0033] In a specific embodiment, the multi-view pose solving unit calculates the camera extrinsic parameters corresponding to each frame image through feature matching and bundle adjustment algorithms, and performs global optimization. The specific optimization process is as follows: Feature extraction and quantization screening: The ORB feature detection algorithm is adopted, with the maximum number of feature points set to 5000, the number of pyramid layers set to 8, and the scale factor set to 1.2. Stable feature points of tooth edges, pits and fissures, and cusps in each frame image are extracted; the BRIEF descriptors corresponding to the feature points are obtained, and feature matching is performed through Hamming distance. The matching distance threshold is set to ≤32. At the same time, the Lowe ratio test is performed with a ratio threshold of 0.75 to screen high-quality matching pairs.

[0034] Initial extrinsic parameter solution: Based on the selected high-quality matching pairs and the calibrated camera intrinsic parameters (including focal length, principal point coordinates, and distortion coefficients), the fundamental matrix is ​​solved using the eight-point method. The matrix is ​​then iteratively optimized using the RANSAC algorithm (1000 iterations, 1.5 pixel inlier threshold), outliers are removed, and the initial camera extrinsic parameters are obtained by decomposing the fundamental matrix. The initial camera extrinsic parameters include the rotation matrix and translation vector, and the reprojection error of the initial extrinsic parameters is ≤3 pixels.

[0035] Global Bundling Adjustment Optimization: An optimization function is constructed with the goal of minimizing reprojection error. All camera extrinsic parameters are denoted as R1,t1; R2,t2; ...Rn,tn, and 3D spatial coordinates are denoted as X1,Y1,Z1; X2,Y2,Z2; ...Xm,Ym,Zm as optimization variables. The Levenberg-Marquardt method is used for iterative solution. The initial damping factor is 1.0, the iteration convergence threshold is 1e-6, and the maximum number of iterations is 500. During the optimization process, the reprojection error of each feature point is acquired in real time. The iteration stops when the global average reprojection error is ≤0.8 pixels and the error change is ≤1e-4 over 10 consecutive iterations. The final output is the camera pose set.

[0036] It should be noted that after feature extraction, high-confidence feature points with response values ​​≥100 are additionally screened, and weak feature points with blurred edges or contrast below 30 are removed to ensure the basic reliability of the matching. During the basic matrix decomposition, rotation matrix orthogonality constraints and translation vector scale consistency constraints are added to avoid situations where the decomposition results have unreasonable physical meanings. In the global binding adjustment, the rotation angle range of the camera extrinsic parameters is limited to ±180°, the translation vector range is adapted to the shooting scene (≤50cm for off-body model teeth scenes), and the 3D spatial point coordinates are constrained within the outer sphere of the target area (radius ≤10cm) to prevent parameter divergence during the optimization process. After the iteration stops, the inter-frame continuity of the camera pose set needs to be verified. The rotation deviation between adjacent frames should be ≤2° and the translation deviation should be ≤3mm. At the same time, the standard deviation of the reprojection error of all feature points should be calculated to be ≤0.2 pixels. If the conditions are met, the pose set is output. If not, the feature matching and optimization process is re-executed to ensure that the final output camera pose set has high accuracy and stability, providing reliable viewpoint constraints for subsequent Gaussian splash 3D reconstruction.

[0037] Gaussian splash 3D reconstruction module: used to construct a 3D Gaussian representation of a dental scene based on camera pose and image data and complete optimization training, generate a Gaussian splash 3D model with realistic texture, adapt the Gaussian splash 3D model to a conventional server configuration, and convert the model to a unified format to adapt to the evaluation process.

[0038] In a specific embodiment, the generation process of the Gaussian splash 3D model with realistic texture is as follows: First, the dental scene is represented as several 3D Gaussian primitives containing position, covariance, color, transparency and orientation related terms, and a denser Gaussian distribution is initialized in the high detail areas of the cusps, marginal ridges and pits.

[0039] The second step involves projecting and differentiable rasterizing a 3D Gaussian image based on the given camera pose to generate a 2D rendered image, and then using GPU acceleration to achieve high frame rate rendering to support interactive browsing.

[0040] The third step is to obtain the reconstruction loss between the rendered image and the actual image. The reconstruction loss includes pixel intensity error and structural similarity index. Based on this loss, the Gaussian position, shape and color parameters are iteratively optimized.

[0041] The fourth step involves performing Gausky units splitting, merging, and pruning operations on the error distribution during training, so that Gausky units are densely distributed in areas of drastic morphological change and moderately sparse in flat areas.

[0042] The fifth step involves iterating until the error converges or the preset number of iterations is reached. Then, a Gaussian splash 3D model with realistic texture is output and converted into a point cloud or mesh in a unified format for subsequent accurate geometric comparison with the standard model.

[0043] It should be noted that the initial total number of 3D Gaussian primitives is set to 100k, the Gaussian distribution density in high-detail regions is three times that in flat regions, and the initial value of the covariance matrix is ​​adaptively set according to the target region size; differentiable rasterization adopts a soft rasterization algorithm, GPU acceleration uses CUDA cores for parallel computing, and the rendering frame rate is maintained above 30fps to ensure smooth interaction; the reconstruction loss calculation adopts a weighted summation method, with pixel intensity error accounting for 60% and structural similarity index accounting for 40%, and the iteration optimization step size is set to 0.001, with the loss value decreasing after each iteration. If the amplitude is less than 1e-5, it is considered to be error convergence; the Gaussian cell splitting threshold is set to reconstruction loss > 0.05, the merging threshold is set to distance between adjacent Gaussian cells < 0.1mm, the pruning threshold is set to transparency < 0.01, and the final total number of Gaussian cells is controlled between 80k and 150k; the number of iterations is preset to 30k times, the model conversion adopts the Poisson surface reconstruction algorithm, the point cloud density is set to 100 points / mm², and the number of triangular patches in the mesh model is controlled between 500,000 and 1,000,000 to ensure that the model retains the fine structure of the tooth while adapting to the computational efficiency requirements of subsequent geometric comparison.

[0044] In a specific embodiment, the adaptation of the Gaussian splash 3D model to a conventional server configuration is as follows: Ten sets of standard model teeth with different tooth positions and different operation types are selected. The different tooth positions include anterior teeth, premolars, and molars. The different operation types include Class I cavity preparation, full crown preparation, and root canal access. According to the preset acquisition rules, 40 images are taken per set, covering the horizontal, top, and bottom directions, to acquire multi-view images. A conventional server hardware configuration is set up: CPU: Intel Xeon E5-2680v4, memory: 32GB DDR4, GPU: NVIDIA Quadro P4000, storage: 1TB SSD. The Gaussian splash reconstruction algorithm is deployed on the conventional server hardware configuration. The algorithm running parameters are optimized for the ex vivo model tooth scene, with the initial number of Gaussian primitives set to 100k, the number of iterations set to 30k, and the batch processing size set to 8.

[0045] Gaussian splash 3D reconstruction was performed sequentially on 10 sets of model tooth images, and the reconstruction time for each set was recorded simultaneously. A high-precision desktop scanner with an accuracy level of ≤0.01mm was used to acquire standard 3D data of 10 sets of model teeth. The Gaussian splash 3D model reconstructed by the system was aligned with the standard data using the ICP registration algorithm. The Euclidean distance between corresponding points on the surfaces of the two models was calculated, and the maximum and average geometric deviations of each set of models were statistically analyzed. Adaptation results verification: The reconstruction time for each of the 10 sets of model teeth was ≤3 minutes, the maximum geometric accuracy error was ≤0.1mm and the average geometric accuracy error was ≤0.06mm, achieving efficient and high-precision reconstruction adaptation of ex vivo model teeth scenes under conventional server configuration.

[0046] The preparatory quality assessment and visualization feedback module is used to spatially register the student operation result model with the preset standard model, automatically calculate various morphological difference indicators, and generate sub-item and total scores according to the scoring rules, as well as generate difference heatmaps and three-dimensional visualization feedback content.

[0047] In a specific embodiment, the automatic calculation of multiple morphological difference indicators and the generation of sub-items and total scores according to the scoring rules are as follows: First, the standard Gaussian splash 3D model and the qualified range of key geometric parameters corresponding to the tooth position and operation type are retrieved from the standard model library.

[0048] Secondly, the spatial registration unit combines rigid body registration and refined registration to ensure that the student's operational result model and the standard model are precisely aligned in the same coordinate system.

[0049] Then, the difference quantization unit calculates the distance field between the two model surfaces to obtain the preparatory depth deviation, axial wall taper deviation, edge line position deviation, undercut area distribution, and volume removal difference index. At the same time, it supports weighted analysis of key areas near the occlusal surface, proximal surface, and gingival margin.

[0050] Next, the scoring rules and the automatic scoring unit map the above-mentioned difference indicators into sub-item scores and total scores according to preset rules, so as to achieve objective and unified automatic scoring.

[0051] Finally, the evaluation visualization unit generates a 3D difference heatmap, a partial cross-sectional view, and a chart-based scoring report, which are presented to students and teachers in a graphic or 3D overlay format. It also supports longitudinal comparison of results from multiple practice sessions.

[0052] It should be noted that spatial registration must meet the requirement of registration error ≤ 0.02mm. Rigid body registration adopts an iterative nearest-point algorithm based on feature points. Refined registration achieves sub-pixel level alignment by adjusting the model pose. The pre-deep deviation calculation is taken as the absolute difference between the actual depth and the standard depth at the corresponding anatomical position. The axial wall taper deviation is calculated based on the difference in the angle between the corresponding axial walls of the two models. The edge line position deviation is calculated using Hausdorff distance. The distribution of the concave area is obtained by judging the proportion of the actual concave volume of the model to the allowable concave volume of the standard model. The volume removal difference is the difference between the actual removal volume and the standard removal volume. Key areas are added. In the weighted analysis, the weighting coefficients for the occlusal surface, proximal surface, and near the gingival margin were set to 1.5, 1.3, and 1.2, respectively, while the coefficient for other areas was 1.0. The scoring mapping adopted a percentage system, with each sub-item having a full score of 20 points. Deviations within the acceptable range received full marks, while deviations outside the range were penalized according to a linear decreasing rule. The total score was the weighted sum of the scores for each sub-item. The three-dimensional difference heatmap used a color gradient range of 0-0.1 mm. The graphical scoring report needed to clearly state the measured values, acceptable ranges, and reasons for deductions for each indicator. The longitudinal comparison function supported displaying the changing trends of each indicator on a time axis, helping students accurately identify their weaknesses.

[0053] In a specific embodiment, the scoring rules and automatic scoring unit map the above-mentioned difference indicators into sub-item scores and total scores according to preset rules. The specific evaluation process is as follows: Standard model library update: Supports the addition of standard three-dimensional models with different tooth positions and different operation types based on the development of oral medicine teaching technology, revision of clinical diagnosis and treatment guidelines and the teaching characteristics of different institutions. At the same time, iteratively adjusts the qualified range of key geometric parameters of existing standard models, and the updated data is automatically synchronized to the system evaluation process.

[0054] Customizable scoring rules: Teachers can adjust the weighting of various indicators and the passing threshold range through the system backend, according to the teaching stage, course focus, or assessment requirements, to adapt to the specific assessment needs of different courses. Customized rules take effect immediately after being saved and are applied to the subsequent evaluation process.

[0055] The Clinical Case 3D Teaching Database Management Module is used for the structured storage and retrieval of the Gaussian splash 3D models of clinical cases reconstructed by Gaussian splashing, along with their corresponding diagnostic and treatment information and automatic scoring results.

[0056] In a specific embodiment, the structured storage and retrieval of the Gaussian splash 3D model of the clinical case reconstructed by Gaussian splashing, along with its corresponding diagnostic and treatment information and automatic scoring results, is implemented as follows: First, the Gaussian splash 3D model of the clinical case obtained through Gaussian splashing reconstruction is associated and bound with the patient's diagnostic information, treatment plan, follow-up time points, and the system's automatic scoring results. The system's automatic scoring results include the fit of the prosthesis margin, adjacency relationship, and occlusal contact.

[0057] Secondly, a multi-dimensional search index is established to support quick retrieval by tooth position, disease type, treatment method, time period, and scoring result, making it easier for teachers to retrieve typical cases when preparing lessons and discussing cases.

[0058] Finally, it supports structured storage of Gaussian splash 3D models of the same case at different time points, enabling side-by-side comparison and dynamic switching of models, and intuitively displaying the 3D evolution of lesion progression and treatment effects.

[0059] The interactive display and remote guidance module for teachers and students is used to load Gaussian splash 3D models and scoring results on mobile terminals or web pages, enabling model rotation, scaling, slicing, annotation functions, and voice and text feedback interaction. At the same time, it supports teachers to review and supplement the system's automatic scoring based on unified standards.

[0060] In a specific embodiment, the implementation of the model's rotation, scaling, slicing, annotation functions, and voice and text feedback interaction is as follows: First, it supports smooth loading of the Gaussian splash 3D model on mobile terminals and web pages. Students can perform rotation, scaling, translation, and local magnification operations on the model, and can observe cross-sections in different directions through the virtual slicing plane, making it easy to check the prepared internal shape, root canal path, or the suitability of the restoration edge.

[0061] Secondly, teachers can use the annotation function to select and annotate the model and draw its outline. At the same time, it supports adding text or voice annotations to explain the key points of evaluation.

[0062] Then, the system automatically sends the scoring results and teacher comments to the student's end. Students can view the automatic scores, teacher's modification opinions, and historical scoring records. It supports three-dimensional comparison of the effects of multiple practice sessions for the same tooth position and the same operation type.

[0063] Finally, the system supports multiple teachers to collaboratively review the same model based on unified scoring rules. Teachers can adjust the system's automatic scoring or supplement subjective evaluations to form the final scoring result.

[0064] It should be noted that the model rotation supports continuous 360° rotation around the X, Y, and Z axes, with a step size accurate to 0.5°. The scaling range is 0.5-5 times the original size, the translation accuracy is ≤1mm, and the maximum local magnification is 10x. The virtual slice plane supports custom angles (1° step size) and thickness (adjustable from 0.1-1mm). The response time when loading the model is ≤2 seconds, and the frame rate is ≥25fps to ensure smoothness. The line thickness for teacher annotations is selectable from 0.2-1mm, text annotations support custom font size and color, and voice annotations are ≤60 seconds in length. It automatically converts to text for backup, and all annotations and comments are bound to the model coordinates, so they do not shift after scaling and rotation. When students view feedback, they can jump to the corresponding model position. The history retains data from the last 10 practice sessions, and the system can switch the display of highlight effects for the difference areas when comparing 3D data. Collaborative review supports up to 3 teachers operating online at the same time, with permission priority settings. The lead evaluator has the final right to confirm the score. When adjusting the score, a reason for modification must be filled in. The system automatically records the score change log to ensure that the evaluation process is traceable and adapts to the standardized assessment needs in cross-institutional joint teaching.

[0065] The examples described in this invention are not limited to the specific embodiments listed above. The examples are merely illustrative to facilitate understanding of the invention and do not constitute a limitation on the scope of protection of this invention. Any modifications, equivalent substitutions, etc., made within the spirit and principles of this invention should be included within the scope of protection.

[0066] The above description is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined in this specification, they should all fall within the protection scope of the present invention.

Claims

1. An automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology, characterized in that, Includes the following modules: Student-side data acquisition module: This module guides students to acquire multi-view images and videos of extracted model teeth or patient oral cavity through mobile terminals. It also supports uploading 3D scan data obtained from an intraoral scanner. Scene preprocessing and camera pose estimation module: used to perform distortion correction, region of interest segmentation and multi-view camera pose estimation on the acquired image data, and to perform coordinate standardization and noise filtering on the 3D scan data. Gaussian splash 3D reconstruction module: used to construct a 3D Gaussian representation of a dental scene based on camera pose and image data and complete optimization training, generate a Gaussian splash 3D model with realistic texture, adapt the Gaussian splash 3D model to a regular server configuration, and convert the model to a unified format to adapt to the evaluation process. Preliminary Quality Assessment and Visual Feedback Module: This module is used to spatially register the student operation result model with the preset standard model, automatically calculate various morphological difference indicators, and generate sub-items and total scores according to the scoring rules. It also generates difference heatmaps and three-dimensional visual feedback content. Clinical Case 3D Teaching Database Management Module: Used for structured storage and retrieval of clinical case 3D models reconstructed by Gaussian splashing, along with their corresponding diagnostic and treatment information and automatic scoring results; The interactive display and remote guidance module for teachers and students is used to load Gaussian splash 3D models and scoring results on mobile terminals or web pages, enabling model rotation, scaling, slicing, annotation functions, and voice and text feedback interaction. At the same time, it supports teachers to review and supplement the system's automatic scoring based on unified standards.

2. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 1, characterized in that, The process involves guiding students to acquire multi-view images and videos of extracted model teeth or patient oral cavity using mobile devices. The specific acquisition process is as follows: After the student starts the system, they select the corresponding tooth position and operation type. The system guides the student through the shooting trajectory unit of the acquisition module, using screen arrows, virtual trajectories, or augmented reality markers to prompt the student to shoot around the target area along three circumferential tracks: horizontal, overhead, and upward. During the shooting process, the shooting quality detection unit evaluates the image clarity, exposure parameters, and viewing angle coverage in real time. If the viewing angle of key areas is insufficient, the image is blurry, or it is overexposed, a prompt to reshoot is issued. The acquisition parameters must meet the following requirements: the number of shots is 20 to 80, or the continuous video duration is 5 to 20 seconds. After the acquisition is completed, it supports direct uploading to the system and is also compatible with the uploading and adaptation of 3D data from intraoral scanners.

3. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 2, characterized in that, The process of performing distortion correction, region of interest segmentation, and multi-view camera pose determination on the acquired image data is as follows: First, the image preprocessing unit performs white balance correction, noise suppression and background segmentation on the acquired images to highlight the dental arch, model teeth or related soft tissue areas, and remove unqualified frames that are blurred or severely occluded. Secondly, the camera intrinsic parameter calibration unit estimates the camera intrinsic parameters based on the pre-calibrated mobile phone camera parameters or the calibration board placed in the scene, thereby achieving the unification of imaging parameters of different devices. Then, the multi-view pose solving unit calculates the camera extrinsic parameters corresponding to each frame image through feature matching and bundle adjustment algorithms, and performs global optimization to construct a complete camera pose set; Finally, coordinate standardization is performed on the uploaded 3D oral scan data to unify the coordinate system. At the same time, a noise filtering algorithm is used to remove redundant and interfering information from the data to ensure data compatibility with subsequent modules.

4. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 3, characterized in that, The multi-view pose solving unit calculates the camera extrinsic parameters corresponding to each frame image through feature matching and bundle adjustment algorithms, and performs global optimization. The specific optimization process is as follows: Feature extraction and quantization screening: The ORB feature detection algorithm was used, with a maximum number of feature points of 5000, a pyramid layer of 8, and a scale factor of 1.

2. Stable feature points of tooth edges, pits and fissures, and cusps in each frame of the image were extracted. The BRIEF descriptors corresponding to the feature points were obtained, and feature matching was performed by Hamming distance. The matching distance threshold was set to ≤32. At the same time, the Lowe ratio test was performed with a ratio threshold of 0.75 to screen high-quality matching pairs. Initial extrinsic parameter solution: Based on the selected high-quality matching pairs and the calibrated camera intrinsic parameters, including focal length, principal point coordinates, and distortion coefficients, the fundamental matrix is ​​solved using the eight-point method. Iterative optimization is performed using the RANSAC algorithm with 1000 iterations and an inlier threshold of 1.5 pixels. Outliers are removed, and the initial camera extrinsic parameters are obtained by decomposing the fundamental matrix. The initial camera extrinsic parameters include rotation matrix and translation vector. The reprojection error of the initial extrinsic parameters is ≤3 pixels. Global Bundling Adjustment Optimization: An optimization function is constructed with the goal of minimizing reprojection error. All camera extrinsic parameters are denoted as R1,t1; R2,t2; ...Rn,tn, and 3D spatial coordinates are denoted as X1,Y1,Z1; X2,Y2,Z2; ...Xm,Ym,Zm as optimization variables. The Levenberg-Marquardt method is used for iterative solution. The initial damping factor is 1.0, the iteration convergence threshold is 1e-6, and the maximum number of iterations is 500. During the optimization process, the reprojection error of each feature point is acquired in real time. The iteration stops when the global average reprojection error is ≤0.8 pixels and the error change is ≤1e-4 over 10 consecutive iterations. The final output is the camera pose set.

5. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 4, characterized in that, The specific process for generating the Gaussian splash 3D model with realistic texture is as follows: The first step is to represent the dental scene as several three-dimensional Gaussian elements containing terms related to position, covariance, color, transparency, and orientation, and initialize a denser Gaussian distribution in high-detail regions such as cusps, marginal ridges, and pits and fissures. The second step involves projecting and differentiable rasterizing the 3D Gaussian image based on the given camera pose to generate a 2D rendered image, and then using GPU acceleration to achieve high frame rate rendering to support interactive browsing. The third step is to obtain the reconstruction loss between the rendered image and the actual image. The reconstruction loss includes pixel intensity error and structural similarity index. Based on this loss, the Gaussian position, shape and color parameters are iteratively optimized. The fourth step involves performing Gausky units splitting, merging, and pruning operations on the error distribution during training, so that Gausky units are densely distributed in areas of drastic morphological changes and moderately sparse in flat areas. The fifth step involves iterating until the error converges or the preset number of iterations is reached. Then, a Gaussian splash 3D model with realistic texture is output and converted into a point cloud or mesh in a unified format for subsequent precise geometric comparison with the standard model.

6. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 5, characterized in that, The adaptation process for the Gaussian splash 3D model to a conventional server configuration is as follows: Ten sets of standard model teeth with different positions and operation types were selected. The different positions included anterior teeth, premolars, and molars. The different operation types included Class I cavity preparation, full crown preparation, and root canal access. According to the preset acquisition rules, 40 images were taken per set, covering horizontal, top, and bottom views, to acquire multi-view images. A standard server hardware configuration was set up with CPU: Intel Xeon E5-2680v4, memory: 32GB DDR4, GPU: NVIDIA Quadro P4000, and storage: 1TB SSD. The Gaussian splash reconstruction algorithm was deployed on the standard server hardware configuration. The algorithm's running parameters were optimized for the ex vivo model tooth scene, with the initial number of Gaussian elements set to 100k, the number of iterations set to 30k, and the batch processing size set to 8. Gaussian splash 3D reconstruction was performed sequentially on 10 sets of model tooth images, and the reconstruction time for each set was recorded simultaneously. A high-precision desktop scanner with an accuracy level of ≤0.01mm was used to acquire standard 3D data of 10 sets of model teeth. The Gaussian splash 3D model reconstructed by the system was aligned with the standard data using the ICP registration algorithm. The Euclidean distance between corresponding points on the surfaces of the two models was calculated, and the maximum and average geometric deviations of each set of models were statistically analyzed. Adaptation results verification: The reconstruction time for each of the 10 sets of model teeth was ≤3 minutes, the maximum geometric accuracy error was ≤0.1mm and the average geometric accuracy error was ≤0.06mm, achieving efficient and high-precision reconstruction adaptation of ex vivo model teeth scenes under conventional server configuration.

7. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 6, characterized in that, The automatic calculation of multiple morphological difference indicators and the generation of sub-item and total scores based on the scoring rules are as follows: First, retrieve the standard Gaussian splash 3D model and the qualified range of key geometric parameters for the corresponding tooth position and operation type from the standard model library; Secondly, the spatial registration unit uses a combination of rigid body registration and refined registration to ensure that the student's operation result model and the standard model are precisely aligned in the same coordinate system. Then, the difference quantization unit calculates the distance field between the two model surfaces to obtain the preparatory depth deviation, axial wall taper deviation, edge line position deviation, undercut area distribution and volume removal difference index. At the same time, it supports weighted analysis of key areas near the occlusal surface, proximal surface and gingival margin. Next, the scoring rules and the automatic scoring unit map the above-mentioned difference indicators into sub-item scores and total scores according to preset rules, so as to achieve objective and unified automatic scoring; Finally, the evaluation visualization unit generates a 3D difference heatmap, a partial cross-sectional view, and a chart-based scoring report, which are presented to students and teachers in a graphic or 3D overlay format. It also supports longitudinal comparison of results from multiple practice sessions.

8. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 7, characterized in that, The scoring rules and automatic scoring unit map the above-mentioned difference indicators into sub-item scores and total scores according to preset rules. The specific evaluation process is as follows: Standard model library update: Supports the addition of standard 3D models for different tooth positions and operation types based on the development of oral medicine teaching technology, revision of clinical diagnosis and treatment guidelines and the teaching characteristics of different institutions. At the same time, iteratively adjusts the qualified range of key geometric parameters of existing standard models, and the updated data is automatically synchronized to the system evaluation process. Customizable scoring rules: Teachers can adjust the weighting of various indicators and the passing threshold range through the system backend, according to the teaching stage, course focus, or assessment requirements, to adapt to the specific assessment needs of different courses. Customized rules take effect immediately after being saved and are applied to the subsequent evaluation process.

9. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 8, characterized in that, The specific implementation process for the structured storage and retrieval of the Gaussian splash 3D model of clinical cases reconstructed by Gaussian splashing, along with their corresponding diagnostic and treatment information and automatic scoring results, is as follows: First, the three-dimensional model of the clinical case obtained by Gaussian splashing reconstruction is associated and bound with the patient's diagnostic information, treatment plan, follow-up time point and the system's automatic scoring results. The system's automatic scoring results include the fit of the restoration margin, the adjacency relationship and the occlusal contact. Secondly Establish a multi-dimensional search index to support quick retrieval by tooth position, disease type, treatment method, time period, and scoring result, making it easier for teachers to retrieve typical cases when preparing lessons and discussing cases; Finally, it supports structured storage of Gaussian splash 3D models of the same case at different time points, enabling side-by-side comparison and dynamic switching of models, and intuitively displaying the 3D evolution of lesion progression and treatment effects.

10. The automated three-dimensional evaluation system for oral procedures based on Gaussian splashing technology according to claim 9, characterized in that, The implementation process for rotating, scaling, slicing, and labeling the model, along with voice and text feedback interaction, is as follows: First, it supports smooth loading of Gauss splash 3D models on mobile terminals and web pages. Students can rotate, scale, translate and zoom in on the models. They can also observe cross-sections in different directions through virtual slicing planes, which makes it easy to check the prepared internal shape, root canal path or the suitability of the restoration edge. Secondly, teachers can use the annotation function to select and annotate the model and draw its outline. At the same time, it supports adding text or voice annotations to explain the key points of evaluation. Then, the system automatically sends the scoring results and teacher comments to the student's end. Students can view the automatic scores, teacher's modification opinions and historical scoring records. It supports three-dimensional comparison of the effects of multiple practice sessions for the same tooth position and the same operation type. Finally, the system supports multiple teachers to collaboratively review the same model based on unified scoring rules. Teachers can adjust the system's automatic scoring or supplement subjective evaluations to form the final scoring result.

Images