Oral cavity three-dimensional model reconstruction method based on 3D Gaussian sputtering model and expression rate analysis system

Abstract

The invention relates to an oral cavity three-dimensional model reconstruction method based on a 3D Gaussian sputtering model and an expression rate analysis system, the oral cavity three-dimensional model reconstruction method comprises the following steps: acquiring intraoral image data of a patient in an orthodontic process, the intraoral image data comprising an intraoral two-dimensional image; according to the intraoral two-dimensional image, sparse point cloud information of the intraoral scene and a corresponding angle between the intraoral two-dimensional image and the intraoral scene are obtained; performing Gaussian parameter initialization to obtain a Gaussian initial sphere set based on an intraoral scene; and according to the intra-oral two-dimensional image and the corresponding angle, parameters of each Gaussian sphere are optimized until expectation is achieved, and an oral three-dimensional model based on the 3D Gaussian sputtering model is obtained through reconstruction. According to the method, a doctor can remotely and accurately judge the tooth movement condition of the patient and the occlusion relation before the upper dentition and the lower dentition, so that the doctor can conveniently carry out remote re-diagnosis on the patient, adjust the wearing of the tooth socket or go to hospital for re-diagnosis.

Description

Technical Field

[0001] The present application relates to the technical field of oral orthodontics, and in particular to a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model and an expression rate analysis system. Background Art

[0002] Orthodontic follow-up is essential to ensure the smooth progress of the tooth correction process. Through regular follow-up, the doctor can monitor the movement of the teeth, determine whether it is necessary to adjust the wearing method of the braces in time and whether it is necessary to add auxiliary devices to evaluate the possible tooth changes, ensure the smooth progress of the correction and finally obtain the ideal tooth arrangement and occlusion relationship.

[0003] Currently, doctors often take photos of the patient's mouth during orthodontic follow-up visits and compare them with the patient's initial photos to judge tooth movement. When the patient is unable to come to the outpatient clinic for follow-up visits, the doctor will ask the patient to use a mobile phone to take several independent two-dimensional photos and send them to the doctor remotely for evaluation. However, this method obtains relatively local two-dimensional images with a fixed perspective, which may not necessarily provide doctors with ideal information.

[0004] In addition, the patient's three-dimensional tooth model can be collected by intraoral scanner before and during treatment, and then the two can be compared to evaluate the effect of the patient's tooth movement. However, in actual application, intraoral scanners are expensive. On the other hand, as professional equipment, the scanning process of intraoral scanners requires high operation. Orthodontic patients must go to the doctor's clinic for regular follow-up visits, and it is impossible to accurately evaluate the efficiency of tooth movement remotely. Summary of the invention

[0005] In view of the cumbersome follow-up process for orthodontic patients, it is necessary to provide a method for reconstructing oral three-dimensional models based on a 3D Gaussian sputtering model, which can help doctors remotely obtain accurate occlusal information during tooth movement and accurately judge the efficiency of tooth movement (the efficiency between expected and actual movement).

[0006] The present application provides a method for reconstructing a three-dimensional model of an oral cavity based on a 3D Gaussian sputtering model, comprising:

[0007] Acquiring intraoral image data of a patient during orthodontic treatment, wherein the intraoral image data includes two-dimensional intraoral images at different angles;

[0008] Obtaining sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model and corresponding angles between the intraoral two-dimensional image and the intraoral scene according to the intraoral two-dimensional images at different angles;

[0009] Initializing Gaussian parameters based on the sparse point cloud information to obtain a Gaussian initial sphere set based on the intra-oral scene, wherein the Gaussian initial sphere set includes a plurality of Gaussian spheres;

[0010] According to the intraoral two-dimensional image and the corresponding angle, the parameters of each Gaussian sphere are optimized until the expected result is achieved, and a three-dimensional model of the oral cavity based on the 3D Gaussian sputtering model is reconstructed.

[0011] Optionally, the intraoral two-dimensional image is a mask image obtained through preprocessing extraction and contains tooth-related features, and the tooth-related features include tooth structure and periodontal area.

[0012] Optionally, the parameters of the Gaussian sphere include: a position parameter for indicating the position of the sphere, a shape parameter for indicating the shape of the sphere, a color parameter for indicating the color, and a covariance matrix for indicating the degree of scaling.

[0013] Optionally, obtaining sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model and corresponding angles between the intraoral two-dimensional image and the intraoral scene according to the intraoral two-dimensional image at different angles specifically includes:

[0014] A feature point dataset is extracted from the intraoral two-dimensional image, and the feature point dataset is matched and solved for any two intraoral two-dimensional images at different angles to obtain the 3D coordinates of the feature point dataset and the corresponding angles between the intraoral two-dimensional image and the intraoral scene, and sparse point cloud information of the intraoral scene is obtained based on the 3D coordinates.

[0015] Optionally, according to the intraoral two-dimensional image and the corresponding angle, the parameters of each Gaussian sphere are optimized until the desired result is achieved, and a three-dimensional model of the oral cavity based on a 3D Gaussian sputtering model is reconstructed, specifically including:

[0016] The intraoral two-dimensional image and the rendered image of the oral three-dimensional model at the corresponding angle are compared; the parameters of each Gaussian sphere are optimized using the comparison difference; until the expected result is achieved, a three-dimensional oral model based on the 3D Gaussian sputtering model is reconstructed.

[0017] Optionally, the two-dimensional intraoral image and the rendered image of the three-dimensional oral model at the corresponding angle are compared; the parameters of each Gaussian sphere are optimized using the comparison difference; until the desired result is achieved, a three-dimensional oral model based on a 3D Gaussian sputtering model is reconstructed, specifically including:

[0018] Constructing an image difference loss function, comparing the intraoral two-dimensional image with the rendered image of the oral three-dimensional model at the corresponding angle, and solving the image difference loss parameter;

[0019] Optimizing the parameters of each Gaussian sphere using the contrast difference obtained by solving, including: introducing regularization, setting density control conditions, optimizing the parameters of the Gaussian sphere using back propagation and gradient descent, and obtaining a 3D Gaussian rendering image through densification and pruning;

[0020] The above process is repeated until the desired result is achieved, and Poisson reconstruction is used to obtain a three-dimensional oral model based on a 3D Gaussian sputtering model.

[0021] Optionally, the method includes: obtaining an oral scan model obtained by intraoral scanning of the patient before orthodontic treatment, and performing rigid registration and semi-rigid registration on the oral scan model and the oral three-dimensional model to obtain a tooth correspondence relationship between the oral scan model and the oral three-dimensional model;

[0022] Based on the tooth correspondence, the displacement of teeth during the orthodontic process is compared and analyzed.

[0023] Optionally, providing a first display area for displaying the mouth scan model;

[0024] Providing a second display area for displaying the oral three-dimensional model;

[0025] Receive an angle display signal for human-computer interaction, render the oral scan model as a first rendered image corresponding to the angle display signal in the first display area, and render the oral three-dimensional model as a second rendered image corresponding to the angle display signal in the second display area.

[0026] The present application provides an expression rate analysis system, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the oral cavity three-dimensional model reconstruction method based on the 3D Gaussian sputtering model as described in the present application, and includes the following steps:

[0027] Planning an expected oral model of the patient after orthodontic treatment, and performing rigid registration and semi-rigid registration on the expected oral model and the oral three-dimensional model to obtain a tooth correspondence between the expected oral model and the oral three-dimensional model;

[0028] Based on the tooth correspondence, the degree to which the displacement of the teeth during the orthodontic process conforms to expectations is compared and analyzed, and an expression rate for representing the movement is output.

[0029] The present application provides an expression rate analysis system, comprising:

[0030] A data input processing module is used to acquire an oral scan model obtained by intraoral scanning of a patient before orthodontic treatment, and to acquire intraoral image data during the orthodontic treatment of the patient, wherein the intraoral image data includes intraoral two-dimensional images at different angles;

[0031] An oral three-dimensional model reconstruction module is used to obtain sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model and corresponding angles between the intraoral two-dimensional image and the intraoral scene according to the intraoral two-dimensional images at different angles;

[0032] Initializing Gaussian parameters based on the sparse point cloud information to obtain a Gaussian initial sphere set based on the intra-oral scene, wherein the Gaussian initial sphere set includes a plurality of Gaussian spheres;

[0033] According to the intraoral two-dimensional image and the corresponding angle, the parameters of each Gaussian sphere are optimized until the expected result is achieved, and a three-dimensional model of the oral cavity based on the 3D Gaussian sputtering model is reconstructed;

[0034] A comparison display module provides a first display area for displaying the oral scan model; provides a second display area for displaying the oral three-dimensional model; is used to receive an angle display signal for human-computer interaction, render the oral scan model in the first display area as a first rendered image corresponding to the angle display signal, and render the oral three-dimensional model in the second display area as a second rendered image corresponding to the angle display signal.

[0035] The present application provides a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model, comprising:

[0036] Acquire an oral scan model obtained by intraoral scanning of the patient before orthodontics, and establish an oral three-dimensional model before optimization based on a 3D Gaussian sputtering model based on the oral scan model;

[0037] Acquiring intraoral image data of a patient during orthodontic treatment, wherein the intraoral image data includes two-dimensional intraoral images at different angles;

[0038] Obtaining corresponding angles between the two-dimensional intraoral image and the three-dimensional oral model before optimization according to the two-dimensional intraoral image at different angles;

[0039] According to the intraoral two-dimensional image and the corresponding angle, parameters of the oral three-dimensional model before optimization are optimized until the expected results are achieved, and the optimized oral three-dimensional model is reconstructed.

[0040] The oral cavity three-dimensional model reconstruction method based on the 3D Gaussian sputtering model of the present application has at least the following effects:

[0041] The present application can reconstruct a three-dimensional oral model based on a 3D Gaussian sputtering model based on intraoral image data collected in a relatively simple manner, so as to facilitate doctors to remotely and accurately evaluate the position of tooth movement and the occlusal relationship between the upper and lower teeth, so as to facilitate doctors to conduct remote follow-up visits with patients, adjust the wearing of braces or order follow-up visits to the hospital. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the process of a method for reconstructing a three-dimensional model of an oral cavity based on a 3D Gaussian sputtering model in one embodiment of the present application;

[0043] Figure 2 Schematic diagram of the process of a method for reconstructing a three-dimensional model of an oral cavity based on a 3D Gaussian sputtering model in one embodiment of the present application;

[0044] Figure 3 This is an internal structure diagram of an expression rate analysis system in one embodiment of the present application;

[0045] Figure 4 This is a schematic diagram of the module structure of the expression rate analysis system in one embodiment of the present application;

[0046] Figure 5 A schematic diagram of intraoral image data of a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model in one embodiment of the present application (photographed based on an oral model);

[0047] Figure 6 In one embodiment of the present application, a method for reconstructing a three-dimensional model of an oral cavity based on a 3D Gaussian sputtering model is provided. Figure 5 A rendered image of a three-dimensional model of the oral cavity obtained by reconstructing the image data shown;

[0048] Figure 7 This is a rendered image of a patient's oral three-dimensional model reconstructed by a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model in one embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] See also Figure 1 In view of the cumbersome follow-up process for orthodontic patients, an embodiment of the present application provides a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model, including steps S100 to S400.

[0051] in:

[0052] Step S100 , acquiring intraoral image data of a patient during orthodontic treatment, wherein the intraoral image data includes intraoral two-dimensional images (intraoral two-dimensional image sequence) at different angles.

[0053] Step S200, obtaining sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model and corresponding angles between the intraoral two-dimensional image and the intraoral scene according to the intraoral two-dimensional images at different angles.

[0054] Step S300 , initializing Gaussian parameters based on the sparse point cloud information to obtain a Gaussian initial sphere set based on the intra-oral scene, wherein the Gaussian initial sphere set includes a plurality of Gaussian spheres.

[0055] Step S400, according to the intraoral two-dimensional image and the corresponding angle, the parameters of each Gaussian sphere are optimized until the desired result is achieved, and a three-dimensional model of the oral cavity based on the 3D Gaussian sputtering model is reconstructed.

[0056] The intraoral image data in this embodiment can be image data from different angles (e.g., more than 30 photos with different viewing angles), or video data composed of continuous frames of intraoral two-dimensional images (e.g., videos recorded with viewing angle changes for more than 10 seconds). Based on the 3D Gaussian-splatting model (3D gaussian-splatting, abbreviated as 3DGS or 3D Gaussian), the intraoral image data is used to reconstruct the oral cavity three-dimensional model based on the 3D Gaussian-splatting model, which can be used to simply and quickly determine the orthodontic progress of the patient.

[0057] This embodiment can reconstruct a three-dimensional oral model based on a 3D Gaussian sputtering model based on intraoral image data collected in a relatively simple manner, so as to facilitate doctors to remotely and accurately evaluate the progress of oral orthodontics and determine whether it is necessary to go to the hospital in time to adjust the angle and position of the orthodontic appliance.

[0058] See also Figure 1 and Figure 2 In step S100, the intraoral two-dimensional image is a mask image obtained through preprocessing extraction and containing tooth-related features, and the tooth-related features include tooth structure and periodontal area.

[0059] In step S300, the parameters of the Gaussian sphere include: a position parameter for indicating the position of the sphere, a shape parameter for indicating the shape of the sphere, a color parameter for indicating the color, and a covariance matrix for indicating the degree of scaling. Specifically, for opaque intraoral scenes, each Gaussian sphere generally contains the following sets of parameters, including: a position parameter p of its center, a covariance matrix ∑ indicating the degree of scaling, a color parameter c, and a shape parameter (ellipsoid shape and symmetry axis direction). The color parameter c is a spherical harmonic function representing the color, and different levels of the spherical harmonic function are used to represent different colors. All properties of the four parameters of each Gaussian sphere can be learned and optimized through stochastic gradient descent training.

[0060] Steps S100 to S300 can be generally summarized as follows: obtaining a patient's intraoral video sequence image dataset (i.e., intraoral image data), performing mask processing (creating a mask image) based on the dataset (intraoral image data), obtaining point cloud features and constructing an initial 3D Gaussian set (3D Gaussian initial scene).

[0061] Obtain a patient's intraoral video sequence image dataset. Use a camera or high-definition mobile phone to shoot various angles of the patient's mouth to obtain intraoral video images. Perform mask processing (i.e., preprocessing) based on the dataset. Use a pre-trained image neural network to obtain a mask image that deletes irrelevant areas such as the tongue and oral mucosa. Only the tooth structure and periodontal area are displayed in the mask image. Then, the above step S200 can be used to restore the sparse point cloud information of the scene, and step S300 can be implemented to initialize Gaussian parameters based on the sparse point cloud information to obtain the Gaussian initial sphere set G of the three-dimensional scene.

[0062] Step S200 specifically includes: extracting a feature point data set from the intraoral two-dimensional image, matching and solving the feature point data set for any two intraoral two-dimensional images at different angles, obtaining the 3D coordinates of the feature point data set, and the corresponding angles between the intraoral two-dimensional image and the intraoral scene, and obtaining sparse point cloud information of the intraoral scene based on the 3D coordinates.

[0063] In this process, the feature point data set is extracted based on the intraoral two-dimensional image, and then the image data is selected in pairs to solve the pose and matching feature points. The 3D coordinates of the feature points in the three-dimensional scene are solved by triangulation and added to the point cloud, and finally the sparse point cloud data of the scene is obtained. In this process, the corresponding angles of the intraoral two-dimensional image and the sparse point cloud data are obtained at the same time. After obtaining the scene point cloud data, the point cloud data can be converted into a Gaussian sphere set through initialization operations.

[0064] During point cloud processing, the scene can adaptively increase the density of Gaussian spheres to better capture the details of the scene. Gaussian sphere adaptation adjustment methods include splitting and copying operations. The splitting operation is used for over-reconstructed areas, and the copying operation is used for under-reconstructed areas.

[0065] Step S400 can be generally summarized as follows: training the data set, optimizing the 3D Gaussian set, and obtaining the intra-oral mesh model (oral three-dimensional model) through three-dimensional reconstruction.

[0066] Step S400 specifically includes: step S410, comparing the two-dimensional intraoral image and the rendered image of the three-dimensional oral model at the corresponding angle; step S420, optimizing the parameters of each Gaussian sphere using the comparison difference; step S430, until the expected result is achieved, reconstructing the three-dimensional oral model based on the 3D Gaussian sputtering model.

[0067] Step S410 specifically includes: constructing an image difference loss function, comparing the intraoral two-dimensional image and the rendered image of the oral three-dimensional model at a corresponding angle, and solving the image difference loss parameter.

[0068] Step S420 specifically includes: introducing regularization, setting density control conditions, optimizing the parameters of the Gaussian sphere by back propagation and gradient descent, and obtaining a 3D Gaussian rendering image by densification and pruning.

[0069] Step S430 specifically includes repeatedly executing the process of steps S410 to S420 above until the desired result is achieved, and using Poisson reconstruction to obtain a three-dimensional model of the oral cavity based on a 3D Gaussian sputtering model.

[0070] After step S400 is executed, the Gaussian image is fitted to the real intraoral scene and a mesh representation is extracted. The whole process specifically includes: (1) constructing a 3D gaussian-splatting differentiable rendering method, comparing the obtained rendered image pi with the intraoral two-dimensional image of the same angle in the intraoral video, and solving the loss parameter loss. (2) Introducing regularization to align the Gaussian sphere with the direction of the scene surface, and the Gaussian sphere is flattened on the surface of the scene. (3) Through data training, back propagation and gradient descent, the parameters of the Gaussian sphere in the scene are optimized, the density control conditions are set, and a reasonable 3DGS image is obtained through densification and pruning. (4) After the training is completed, Poisson reconstruction is used to generate a three-dimensional intraoral scene mesh model corresponding to the intraoral video.

[0071] Specifically, 3DGS is composed of multiple module layers, including a sampling layer, a rasterization layer and a loss layer. Step S400 is implemented based on each module layer of the 3DGS model. Among them, the sampling layer is used to obtain an image P0 of a camera perspective and a corresponding training data set through sampling, that is, the sampling layer obtains a two-dimensional intraoral image P0. The rasterization layer is used to project the Gaussian sphere onto the viewing plane through the splatting operation, and obtain the image P1 observed from any viewing angle by optimizing the sputtering parameters. Image P1 is the rendered image of the oral three-dimensional model at the corresponding angle. The loss layer is used to compare images P0 and P1, calculate the loss function, and optimize the parameters of the model through back propagation. In this process, the parameters of the Gaussian sphere are optimized and the 3D Gaussian model is optimized. During the training process, the parameters of the above-mentioned Gaussian sphere are optimized so that the Gaussian sphere set is unified with the real scene. After the training is completed as expected, the intraoral three-dimensional scene mesh model is obtained through Poisson reconstruction. For a new patient's intraoral video image, the trained 3DGS model can be used for transfer learning, thereby improving the speed of model convergence.

[0072] In some embodiments, a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model includes step S500, correcting the scale of the 3D Gaussian result through semi-rigid registration, and obtaining the current tooth movement status during the treatment process.

[0073] Step S500 specifically includes: step S510, obtaining the oral scan model (hereinafter referred to as the oral scan model) obtained by intraoral scanning of the patient before orthodontics, and performing rigid and semi-rigid registration of the oral scan model with the oral 3D model to obtain the tooth correspondence between the oral scan model and the oral 3D model; step S520, based on the tooth correspondence, comparing and analyzing the displacement of the teeth during the orthodontic process. It can be understood that the intraoral scene includes several rigid shapes (teeth) and the dynamic relative position relationship of the rigid shapes (tooth movement process in orthodontics).

[0074] In some embodiments, the implementation process of rigid registration includes: obtaining an overall correspondence between the three-dimensional oral model and the oral scan model by an iterative closest point pair method.

[0075] In some embodiments, the implementation process of semi-rigid registration includes: partitioning the oral three-dimensional model into a number of non-overlapping subsets, each subset is used to represent a rigid shape; establishing a correspondence between each subset and a local area in the oral scan model obtained by intraoral scanning, and then obtaining a tooth correspondence between the oral scan model and the oral three-dimensional model.

[0076] In other embodiments, the implementation process of semi-rigid registration includes: partitioning the oral three-dimensional model and the oral scan model obtained by intraoral scanning into a number of non-overlapping subsets, each subset is used to represent a rigid shape; matching the oral three-dimensional model and each subset of the oral scan model to obtain the correspondence between the oral three-dimensional model and the oral scan model, and then obtaining the tooth correspondence between the oral scan model and the oral three-dimensional model.

[0077] After obtaining the corresponding relationship, the displacement of each tooth during treatment can be compared. Step S500 aligns and scales the Gaussian image by comparing the oral scan model before orthodontic treatment, and compares it with the oral situation before treatment. Specifically, it includes semi-rigidly aligning the oral three-dimensional model (based on 3D Gaussian) obtained in step S400 with the oral scan model before orthodontic treatment, correcting the model scale of 3DGS, and obtaining the corresponding relationship between the teeth before treatment and during treatment. It can then be determined whether the current tooth arrangement and occlusion conditions meet orthodontic expectations.

[0078] In some embodiments, a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model includes step S600, where a viewing angle is interactively given and a scene image under the viewing angle is rendered.

[0079] Step S600 can be performed by the user interacting on the client, by giving a viewing angle through interaction, and rendering the scene corresponding to the viewing angle. Furthermore, the patient's initial or different stages of intraoral three-dimensional scenes are displayed in the same system. After the intraoral 3DGS model scene of a patient is obtained through training, a new camera angle can be given as input, and then the rasterized scene image can be obtained according to the model's trained parameters (including the parameters of each Gaussian sphere and the parameters of splatting). This method allows doctors and users to more accurately judge the differences before and after treatment and evaluate the degree of deviation from the treatment goal, thereby effectively improving the effect of orthodontic follow-up.

[0080] Step S600 specifically includes: step S610, providing a first display area for displaying the oral scan model; step S620, providing a second display area for displaying the oral three-dimensional model; step S630, receiving an angle display signal for human-computer interaction, rendering the oral scan model in the first display area as a first rendered image corresponding to the angle display signal, and rendering the oral three-dimensional model in the second display area as a second rendered image corresponding to the angle display signal.

[0081] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0082] See also Figure 3 In one embodiment of the present application, an expression rate analysis system is provided. The expression rate analysis system may be, for example, a computer device. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 3As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for reconstructing a three-dimensional model of an oral cavity based on a 3D Gaussian sputtering model is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0083] The expression rate analysis system includes a memory, a processor and a computer program stored in the memory. The processor executes the computer program to implement the steps of the oral three-dimensional model reconstruction method based on the 3D Gaussian sputtering model provided in each embodiment of the present application, and includes the following steps: planning the expected oral model of the patient after the end of orthodontic treatment (the planning method can be to use a computer to complete prediction and planning before orthodontic treatment, and the end of orthodontic treatment can be a staged end or an overall end of orthodontic treatment), rigidly aligning the expected oral model with the oral three-dimensional model and semi-rigidly aligning the tooth correspondence between the expected oral model and the oral three-dimensional model; based on the tooth correspondence, comparing and analyzing the degree to which the displacement of the teeth during the orthodontic process meets expectations, and outputting the expression rate used to represent the movement.

[0084] In one embodiment, a computer program product is provided, comprising computer instructions, which implement the following steps when executed by a processor: Step S100, collecting and obtaining intraoral image data of a patient during orthodontic treatment, wherein the intraoral image data includes intraoral two-dimensional images at different angles. Step S200, obtaining sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model, and the corresponding angles between the intraoral two-dimensional image and the intraoral scene based on the intraoral two-dimensional images at different angles. Step S300, initializing Gaussian parameters based on the sparse point cloud information to obtain a Gaussian initial sphere set based on the intraoral scene, wherein the Gaussian initial sphere set includes several Gaussian spheres. Step S400, optimizing the parameters of each Gaussian sphere based on the intraoral two-dimensional image and the corresponding angle until the expected result is achieved, and reconstructing a three-dimensional model of the oral cavity based on the 3D Gaussian sputtering model.

[0085] In the present embodiment, the computer program product includes a program code portion for executing the steps of the method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model in each embodiment of the present application when the computer program product is executed by one or more computing devices. The computer program product may be stored on a computer-readable recording medium. The computer program product may also be provided for download via a data network (e.g., via a RAN, via the Internet and / or via an RBS). Alternatively or additionally, the method may be encoded in a field programmable gate array (FPGA) and / or an application specific integrated circuit (ASIC), or functionality may be provided for download by means of a hardware description language.

[0086] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0087] See also Figure 4 In one embodiment of the present application, a gaussian-splatting-based orthodontic expression rate analysis system (referred to as the expression rate analysis system) is provided, including a data input processing module, an oral three-dimensional model reconstruction module, and a comparison display module.

[0088] Among them, the data input processing module is used to collect and obtain the oral scan model obtained by intraoral scanning of the patient before orthodontic treatment, and to collect and obtain the intraoral image data of the patient during the orthodontic treatment. The intraoral image data includes intraoral two-dimensional images at different angles.

[0089] The data input processing module can input the intraoral scan data of the patient before orthodontics, which is generally three-dimensional model data. "Before orthodontics" means that orthodontic treatment is required next. For example, orthodontics may include multiple cycles, and "before orthodontics" may be the cycle before orthodontic treatment is continued in the current cycle. The data input processing module is used to pre-process the collected intraoral video of the patient to obtain sparse point cloud information and an initial Gaussian sphere. The patient's intraoral image data needs to be input, and the intraoral image data generally includes teeth, gums, and surrounding tissues, such as captured images or videos. After pre-processing, the intraoral image data is used to extract key features, which will be used for subsequent 3DGS modeling. At the same time, it is used as a comparison.

[0090] The oral three-dimensional model reconstruction module is used to obtain sparse point cloud information of the intraoral scene based on the 3D Gaussian sputtering model and the corresponding angles between the intraoral two-dimensional images and the intraoral scene according to the intraoral two-dimensional images at different angles; initialize Gaussian parameters based on the sparse point cloud information to obtain a Gaussian initial sphere set based on the intraoral scene, and the Gaussian initial sphere set includes several Gaussian spheres; optimize the parameters of each Gaussian sphere according to the intraoral two-dimensional images and the corresponding angles until the expected results are achieved, and reconstruct a three-dimensional oral model based on the 3D Gaussian sputtering model.

[0091] The oral 3D model reconstruction module is used to construct a differentiable representation of the 3DGS set, optimize the parameters of the Gaussian sphere set G through data training and back propagation, and generate a 3D mesh model to obtain the 3D scene in the patient's mouth. The pre-processed data is modeled using 3DGS technology, including converting sparse point cloud data into a Gaussian sphere set, and by constructing 3DGS, the optimization algorithm adjusts the relevant parameters so that the obtained 3D scene can accurately present the morphology and structure of the teeth and surrounding tissues.

[0092] The comparison display module provides a first display area for displaying the patient's oral scan model; provides a second display area for displaying the oral three-dimensional model; is used to receive an angle display signal for human-computer interaction, render the oral scan model in the first display area as a first rendered image corresponding to the angle display signal, and render the oral three-dimensional model in the second display area as a second rendered image corresponding to the angle display signal.

[0093] The comparison display module displays the 3D scene (oral 3D model) obtained by 3DGS and the patient's initial intraoral model (oral scan model obtained by intraoral scanning before orthodontics) at the same time, and through the doctor's mouse or keyboard interaction, after receiving the angle display signal, the difference between the patient's initial intraoral model and the 3D model obtained by 3DGS can be compared from different angles. Through 3D rendering technology, the 3DGS model and intraoral scan data are displayed to the user in a visual form, allowing the user to intuitively see the differences between different models.

[0094] The expression rate analysis system may further include an expression rate calculation module, which is used to compare and analyze the degree to which the displacement of teeth during the orthodontic process conforms to expectations based on the tooth correspondence, and output an expression rate used to represent the movement.

[0095] The expression rate calculation module can perform quantitative and qualitative comparative analysis on the 3DGS model and the expected model of the patient's oral cavity after orthodontic treatment. Quantitative analysis can include calculating the distance, offset, projection distance and other indicators of the model, and qualitative analysis can include evaluating whether there are differences in the shape, position and angle of the teeth in the model.

[0096] Each embodiment of the present application can reconstruct the intraoral image data into an oral three-dimensional model through steps S100 to S400, and can compare the oral three-dimensional model with the oral scan model obtained by intraoral scanning before orthodontics through step S500 to evaluate the expression effect of tooth movement of the case. Step S600 parametrically displays the current oral three-dimensional model relative to the previous tooth movement. The expression rate calculation module can compare the oral three-dimensional model with the expected oral model, compare and analyze the degree to which the tooth displacement is in line with expectations, and then output the expression rate.

[0097] In some embodiments, a method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model is provided, comprising: step S1, acquiring an oral scan model obtained by intraoral scanning of a patient before orthodontics, and establishing an oral three-dimensional model before optimization based on the 3D Gaussian sputtering model based on the oral scan model. Step S2, acquiring intraoral image data of the patient during orthodontics, the intraoral image data including intraoral two-dimensional images at different angles. Step S3, obtaining corresponding angles between the intraoral two-dimensional images and the oral three-dimensional model before optimization based on the intraoral two-dimensional images at different angles. Step S4, optimizing parameters of the oral three-dimensional model before optimization based on the intraoral two-dimensional images and the corresponding angles until the expected results are achieved, and reconstructing the optimized oral three-dimensional model.

[0098] Based on the oral scan model, an optimized front oral cavity three-dimensional model based on the 3D Gaussian sputtering model is established. For example, the rendered images at different angles can be rendered based on the oral scan model, and then the optimized front oral cavity three-dimensional model is obtained in combination with the same principle process of step S200. Intraoral two-dimensional images at different angles can be matched to the optimized front oral cavity three-dimensional model by feature point matching to obtain corresponding angles.

[0099] It can be understood that since the amount of tooth movement during orthodontic correction is relatively small, the pre-optimization oral three-dimensional model established based on the oral scan model obtained through intraoral scanning can reduce the adverse effects of misalignment and scale differences caused by the mismatch of angles when directly using the patient's own shooting, improve the accuracy of the current optimized oral three-dimensional model, and help to increase the speed of model convergence.

[0100] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. When the technical features in different embodiments are embodied in the same figure, it can be regarded that the figure also discloses the combination examples of the various embodiments involved.

[0101] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model, characterized in that: include: Acquiring intraoral image data of a patient during orthodontic treatment, wherein the intraoral image data includes two-dimensional intraoral images at different angles; Obtaining sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model and corresponding angles between the intraoral two-dimensional image and the intraoral scene according to the intraoral two-dimensional images at different angles; Initializing Gaussian parameters based on the sparse point cloud information to obtain a Gaussian initial sphere set based on the intra-oral scene, wherein the Gaussian initial sphere set includes a plurality of Gaussian spheres; According to the intraoral two-dimensional image and the corresponding angle, the parameters of each Gaussian sphere are optimized until the expected result is achieved, and a three-dimensional model of the oral cavity based on the 3D Gaussian sputtering model is reconstructed.

2. The method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model according to claim 1, characterized in that: The intraoral two-dimensional image is a mask image obtained through preprocessing extraction and contains tooth-related features, and the tooth-related features include tooth structure and periodontal area.

3. The method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model according to claim 1, characterized in that: The parameters of the Gaussian sphere include: a position parameter for indicating the position of the sphere, a shape parameter for indicating the shape of the sphere, a color parameter for indicating the color, and a covariance matrix for indicating the degree of scaling.

4. The method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model according to claim 1, characterized in that: According to the intraoral two-dimensional images at different angles, sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model and corresponding angles between the intraoral two-dimensional images and the intraoral scene are obtained, specifically including: A feature point dataset is extracted from the intraoral two-dimensional image, and the feature point dataset is matched and solved for any two intraoral two-dimensional images at different angles to obtain the 3D coordinates of the feature point dataset and the corresponding angles between the intraoral two-dimensional image and the intraoral scene, and sparse point cloud information of the intraoral scene is obtained based on the 3D coordinates.

5. The method for reconstructing an oral cavity three-dimensional model based on a 3D Gaussian sputtering model according to claim 1, characterized in that: According to the intraoral two-dimensional image and the corresponding angle, the parameters of each Gaussian sphere are optimized until the desired result is achieved, and a three-dimensional model of the oral cavity based on the 3D Gaussian sputtering model is reconstructed, specifically including: The intraoral two-dimensional image and the rendered image of the oral three-dimensional model at the corresponding angle are compared; the parameters of each Gaussian sphere are optimized using the comparison difference; until the expected result is achieved, a three-dimensional oral model based on the 3D Gaussian sputtering model is reconstructed.

6. The method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model according to claim 5, characterized in that: Comparing the intraoral two-dimensional image with the rendered image of the oral three-dimensional model at the corresponding angle; Optimizing the parameters of each Gaussian sphere by using the contrast difference until the desired result is achieved, and reconstructing a three-dimensional model of the oral cavity based on a 3D Gaussian sputtering model, specifically including: Constructing an image difference loss function, comparing the intraoral two-dimensional image with the rendered image of the oral three-dimensional model at the corresponding angle, and solving the image difference loss parameter; Optimizing the parameters of each Gaussian sphere using the contrast difference obtained by solving, including: introducing regularization, setting density control conditions, optimizing the parameters of the Gaussian sphere using back propagation and gradient descent, and obtaining a 3D Gaussian rendering image through densification and pruning; The above process is repeated until the desired result is achieved, and Poisson reconstruction is used to obtain a three-dimensional oral model based on a 3D Gaussian sputtering model.

7. The method for reconstructing an oral cavity three-dimensional model based on a 3D Gaussian sputtering model according to claim 1, characterized in that: include: Obtain an oral scan model obtained by intraoral scanning of the patient before orthodontic treatment, perform rigid registration and semi-rigid registration on the oral scan model and the oral three-dimensional model, and obtain a tooth correspondence between the oral scan model and the oral three-dimensional model; Based on the tooth correspondence, compare and analyze the displacement of teeth during orthodontic process; Providing a first display area for displaying the oral scan model; Providing a second display area for displaying the oral cavity three-dimensional model; Receive an angle display signal for human-computer interaction, render the oral scan model as a first rendered image corresponding to the angle display signal in the first display area, and render the oral three-dimensional model as a second rendered image corresponding to the angle display signal in the second display area.

8. An expression rate analysis system, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model according to any one of claims 1 to 7, and includes the following steps: Planning an expected oral model of the patient after orthodontic treatment, and performing rigid registration and semi-rigid registration on the expected oral model and the oral three-dimensional model to obtain a tooth correspondence between the expected oral model and the oral three-dimensional model; Based on the tooth correspondence, the degree to which the displacement of the teeth during the orthodontic process conforms to expectations is compared and analyzed, and an expression rate for representing the movement is output.

9. An expression rate analysis system, characterized in that: include: A data input processing module is used to acquire an oral scan model obtained by intraoral scanning of a patient before orthodontic treatment, and to acquire intraoral image data during the orthodontic treatment of the patient, wherein the intraoral image data includes intraoral two-dimensional images at different angles; An oral three-dimensional model reconstruction module is used to obtain sparse point cloud information of the intraoral scene based on a 3D Gaussian sputtering model and corresponding angles between the intraoral two-dimensional image and the intraoral scene according to the intraoral two-dimensional images at different angles; Initializing Gaussian parameters based on the sparse point cloud information to obtain a Gaussian initial sphere set based on the intra-oral scene, wherein the Gaussian initial sphere set includes a plurality of Gaussian spheres; According to the intraoral two-dimensional image and the corresponding angle, the parameters of each Gaussian sphere are optimized until the expected result is achieved, and a three-dimensional model of the oral cavity based on the 3D Gaussian sputtering model is reconstructed; A comparison display module provides a first display area for displaying the oral scan model; provides a second display area for displaying the oral three-dimensional model; is used to receive an angle display signal for human-computer interaction, render the oral scan model in the first display area as a first rendered image corresponding to the angle display signal, and render the oral three-dimensional model in the second display area as a second rendered image corresponding to the angle display signal.

10. A method for reconstructing an oral three-dimensional model based on a 3D Gaussian sputtering model, characterized in that: include: Acquire an oral scan model obtained by intraoral scanning of the patient before orthodontics, and establish an oral three-dimensional model before optimization based on a 3D Gaussian sputtering model based on the oral scan model; Acquiring intraoral image data of a patient during orthodontic treatment, wherein the intraoral image data includes two-dimensional intraoral images at different angles; Obtaining corresponding angles between the two-dimensional intraoral image and the three-dimensional oral model before optimization according to the two-dimensional intraoral image at different angles; According to the intraoral two-dimensional image and the corresponding angle, the parameters of the oral three-dimensional model before optimization are optimized until the expected results are achieved, and the optimized oral three-dimensional model is reconstructed.

Images