Orthodontic treatment monitoring methods, devices, equipment and storage media

Abstract

This disclosure relates to a method, apparatus, device, and storage medium for monitoring orthodontic treatment. The orthodontic treatment monitoring method includes: acquiring multiple intraoral images of a user and a first dental model; determining initial transformation parameters for each pair of teeth in a second dental model corresponding to the first dental model and the multiple intraoral images; determining initial camera parameters and a loss function corresponding to the initial transformation parameters; and if the loss function corresponding to the initial camera parameters and the initial transformation parameters satisfies preset conditions, then determining the initial transformation parameters as target transformation parameters. According to embodiments of this disclosure, users can more conveniently monitor orthodontic treatment.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for monitoring orthodontic treatment. Background Technology

[0002] Orthodontics is the process of correcting teeth, removing misaligned teeth and deformities, in order to achieve aesthetically pleasing teeth.

[0003] Currently, orthodontic treatment monitoring mainly relies on patients visiting the clinic regularly for checkups. Doctors visually assess the orthodontic progress at each stage of treatment, or they use an intraoral scanner to scan the patient's teeth and then quantitatively compare the scanned teeth with the expected tooth shape to evaluate the overall effectiveness. However, some patients are unable to visit the clinic regularly due to busy work schedules, time constraints, or other reasons. This leads to delayed adjustments to braces, resulting in longer treatment periods, less effective results, and a poor patient experience. Therefore, a more convenient method for monitoring orthodontic treatment is urgently needed. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a method, apparatus, device, and storage medium for monitoring orthodontic treatment.

[0005] In a first aspect, this disclosure provides a method for monitoring orthodontic treatment, the method comprising: Acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model; Determine the initial transformation parameters for each pair of tooth models in the first dental model and the second dental model corresponding to the multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth; Determine the initial camera parameters and the loss function corresponding to the initial transformation parameters, wherein the initial camera parameters are the camera parameters corresponding to the multiple intraoral images, and the loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth model pairs after the similarity transformation is performed according to the corresponding initial transformation parameters; If the loss function corresponding to the initial camera parameters and the initial transformation parameters satisfies a preset condition, then the initial transformation parameters are determined as the target transformation parameters.

[0006] Secondly, this disclosure provides an orthodontic treatment monitoring device, the device comprising: The first acquisition module is used to acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model; The first determining module is used to determine the initial transformation parameters corresponding to each pair of tooth models in the first dental model and the second dental model corresponding to the multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth. The second determining module is used to determine the initial camera parameters and the loss function corresponding to the initial transformation parameters, wherein the initial camera parameters are the camera parameters corresponding to the multiple intraoral images, and the loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth model pairs after the similarity transformation is performed according to the corresponding initial transformation parameters. The third determining module is used to determine the initial transformation parameters as target transformation parameters if the loss function corresponding to the initial camera parameters and the initial transformation parameters meets a preset condition.

[0007] Thirdly, embodiments of this disclosure also provide an electronic device, the device comprising: One or more processors; Storage device for storing one or more programs. When one or more programs are executed by one or more processors, the one or more processors implement the orthodontic treatment monitoring method provided in the first aspect.

[0008] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the orthodontic treatment monitoring method provided in the first aspect.

[0009] The technical solution provided in this disclosure has the following advantages compared with the prior art: This disclosure discloses an orthodontic treatment monitoring method, apparatus, device, and storage medium capable of acquiring multiple intraoral images of a user and a first dental model, wherein the first dental model includes at least one first tooth model; determining initial transformation parameters for each pair of tooth models in the first dental model and the second dental model corresponding to the multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth; determining a loss function corresponding to initial camera parameters and initial transformation parameters, wherein the initial camera parameters are camera parameters corresponding to the multiple intraoral images, and the loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth models after similarity transformation according to the corresponding initial transformation parameters; if the loss function corresponding to the initial camera parameters and initial transformation parameters satisfies a preset condition, then the initial transformation parameters are determined as target transformation parameters. As can be seen, by adopting the above technical solution and using an intraoral scanner to scan the user's teeth, only multiple intraoral images are needed to obtain the target transformation parameters of each pair of teeth in the second and first dental models (i.e., the dental models corresponding to the desired orthodontic effect) of the multiple intraoral images. The doctor can then know the orthodontic treatment effect based on the target transformation parameters. In this way, users can monitor orthodontic treatment more conveniently. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0011] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating an orthodontic treatment monitoring method provided in an embodiment of this disclosure is shown; Figure 2 A flowchart illustrating another orthodontic treatment monitoring method provided in an embodiment of this disclosure is shown; Figure 3 A schematic diagram of the structure of an orthodontic treatment monitoring device provided in an embodiment of this disclosure is shown; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0013] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0014] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0015] To address the aforementioned issues, this disclosure provides an orthodontic treatment monitoring method, apparatus, device, and storage medium.

[0016] The following is combined with Figures 1 to 3 The orthodontic treatment monitoring method provided in this disclosure is described below. In this disclosure, the orthodontic treatment monitoring method can be executed by an electronic device. The electronic device may include devices with communication capabilities such as tablet computers, desktop computers, and laptop computers, or it may include devices simulated by virtual machines or simulators.

[0017] Figure 1 A flowchart illustrating an orthodontic treatment monitoring method provided in an embodiment of this disclosure is shown.

[0018] like Figure 1 As shown, this orthodontic treatment monitoring method may include the following steps.

[0019] S110. Acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model.

[0020] Specifically, an intraoral image is an image obtained by using an imaging device to capture images of the inside of the oral cavity. The image acquisition device may include, but is not limited to, mobile phones, computers, cameras, etc.

[0021] For example, acquiring multiple intraoral images of a user may include: receiving multiple intraoral images of the user sent by other electronic devices, or reading multiple intraoral images of the user from a storage device, but is not limited to these.

[0022] Specifically, the first dental model is the dental model corresponding to the desired orthodontic effect, and the first dental model includes a first tooth model corresponding to at least one tooth.

[0023] The first dental model may include the dental model corresponding to the effect of the first stage of orthodontic treatment, the dental model corresponding to the effect of the second stage of orthodontic treatment, or the dental model corresponding to the final orthodontic effect, but is not limited to these.

[0024] The specific representation of the first jaw model and the first tooth model can include a three-dimensional mesh model.

[0025] For example, obtaining the first dental model may include: performing mesh segmentation on the models corresponding to the teeth and gums, segmenting the tooth models corresponding to each tooth, and using DeepSDF technology to complete the segmented tooth models into a watertight mesh, thereby obtaining the first dental model, but is not limited to this.

[0026] S120. Determine the initial transformation parameters for each pair of tooth models in the first dental model and the second dental model corresponding to multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth.

[0027] Specifically, the second dental model is a dental model corresponding to (or behind) multiple intraoral images, and includes at least one second tooth model. Those skilled in the art should understand that the first dental model is the dental model corresponding to the desired orthodontic effect, and the second dental model is the dental model corresponding to multiple intraoral images. Since both the first and second dental models correspond to the user's teeth, by optimizing the pose of each first tooth model in the first dental model, the first dental model can be transformed into the second dental model, thereby revealing the differences between each first tooth model and its corresponding first tooth model. Based on this, embodiments of this disclosure need to determine the target transformation parameters for transforming the first dental model into the second dental model.

[0028] Specifically, for each pair of tooth models, the geometry of the first and second tooth models is consistent, differing only by a similarity transformation. In other words, by performing a similarity transformation on the first tooth model, the first and second tooth models can be aligned. The transformation parameters used in this similarity transformation include scaling, translation, and rotation. Since the scaling ratio between each pair of tooth models is the same as the scaling ratio between the world coordinate systems of the first and second jaw models, the scaling ratios of each pair are identical. Thus, each pair of tooth models in the first and second jaw models shares a common scaling ratio, as well as its corresponding translation and rotation matrices.

[0029] Specifically, the initial transformation parameters are the initial transformation parameters corresponding to the tooth model.

[0030] In some embodiments, S120 may include: randomly generating initial transformation parameters for each pair of tooth model pairs.

[0031] In other embodiments, S120 may include: performing three-dimensional reconstruction based on multiple intraoral images to obtain a third dental model; and performing coarse registration processing on the first and third dental models to obtain the initial transformation parameters corresponding to each pair of tooth models.

[0032] Specifically, the third dental model is a dental model corresponding to the user's teeth and jaws, obtained by three-dimensional reconstruction based on multiple intraoral images.

[0033] The specific representation of the third jaw model can include a three-dimensional point cloud model, a three-dimensional mesh model, etc., and is not limited here.

[0034] Specifically, the initial camera parameters are the approximate camera parameters corresponding to the acquisition of multiple intraoral images. In this embodiment of the disclosure, during the process of optimizing the pose of each first tooth model in the first dental model to transform the first dental model into the second dental model, the actual camera parameters (i.e., target camera parameters) corresponding to the acquisition of multiple intraoral images are gradually determined. For example, the initial camera parameters are optimized to gradually approach the target camera parameters.

[0035] The camera parameters can include camera intrinsic parameters and camera extrinsic parameters. The camera intrinsic parameters can include the camera focal length and the projected coordinates of the camera center on the pixel plane, etc. The camera extrinsic parameters can include the camera pose, which can include, for example, the camera rotation matrix and the camera translation matrix. The camera rotation matrix and the camera translation matrix together describe how to transform a point from the world coordinate system to the camera coordinate system.

[0036] For example, the Colmap algorithm is used to predict the camera parameters (i.e., initial camera parameters) and sparse point cloud (i.e., third jaw model) when acquiring multiple intraoral images. If the image acquisition device is a pinhole camera, the camera parameters include three intrinsic camera parameters, namely the camera focal length. and the projected coordinates of the camera center on the pixel plane. Each intraoral image corresponds to 6 camera extrinsic parameters (corresponding to 6 degrees of freedom).

[0037] Specifically, coarse registration is performed on the first and third dental models to obtain transformation parameters that align the first and third dental models as a whole, and these transformation parameters are used as the initial transformation parameters for each tooth model pair.

[0038] For example, feature-based coarse registration methods or coarse registration methods based on the random sample consensus (RANSAC) framework can be used to perform coarse registration processing on the first and third dental models, but are not limited to these.

[0039] Understandably, since both the second and third dental models correspond to multiple intraoral images, when the first and third dental models are coarsely aligned overall, the first and second dental models will also be coarsely aligned overall. That is, the first and second dental models in a tooth model pair are at least coarsely aligned. Therefore, the initial transformation parameters obtained through coarse registration are closer to the target transformation parameters, which is beneficial for quickly obtaining the target transformation parameters for each tooth model pair and improving the efficiency of orthodontic treatment monitoring. Furthermore, by performing coarse registration on the first and third dental models to obtain the initial transformation parameters, the initial transformation parameters for each tooth model pair can be obtained quickly.

[0040] In some other embodiments, S120 may include: performing three-dimensional reconstruction based on multiple intraoral images to obtain a third dental model, wherein the third dental model includes at least one third tooth model; performing coarse registration processing on each pair of tooth models in the first and third dental models to obtain initial transformation parameters corresponding to each pair of tooth models in the first and second dental models.

[0041] S130. Determine the loss function corresponding to the initial camera parameters and the initial transformation parameters. The initial camera parameters are the camera parameters corresponding to multiple intraoral images. The loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth models after the similarity transformation is performed according to the corresponding initial transformation parameters.

[0042] Specifically, the deviation between the first and second tooth models in a tooth model pair may include, but is not limited to, deviations in color values, mask values, etc.

[0043] S140. If the loss function corresponding to the initial camera parameters and the initial transformation parameters meets the preset conditions, then the initial transformation parameters are determined as the target transformation parameters.

[0044] Specifically, the initial camera parameters can also be determined as the target camera parameters.

[0045] Specifically, the preset conditions may include: the loss function is less than a preset threshold. The specific value of the preset threshold can be set by those skilled in the art according to the actual situation, but is not limited to this.

[0046] Specifically, if the loss function corresponding to the initial camera parameters and the initial transformation parameters meets the preset conditions, it indicates that after the first tooth model in each pair of tooth models is similarly transformed according to its corresponding initial transformation parameters, the alignment accuracy with the second tooth model is high (or the requirements are met), and the difference between the initial camera parameters and the actual camera parameters (i.e., the actual camera parameters corresponding to the acquisition of multiple intraoral images) is small (or the requirements are met). In this case, the initial transformation parameters can be determined as the target transformation parameters.

[0047] Those skilled in the art should understand that, since the first tooth model in the first dental model is aligned with the second dental model after undergoing a similarity transformation according to its corresponding target transformation parameters, in other words, the dental model obtained after the first tooth model in the first dental model undergoes a similarity transformation according to its corresponding target transformation parameters can be regarded as the second dental model. Therefore, the difference in tooth position between the first dental model and the second dental model can be determined according to the target transformation parameters, thereby realizing the monitoring of orthodontic treatment.

[0048] It is understood that the technical solution provided in this disclosure allows users to avoid the need to visit a clinic for visual inspection or intraoral scanning. Users only need to capture and upload multi-view images (i.e., multiple intraoral images) of their teeth. This eliminates the need for frequent clinic visits, saving users time and freeing them from time and space constraints. Furthermore, since it eliminates the need for doctors to visually assess or frequently scan the user's teeth with an intraoral scanner, it also saves doctors time. Moreover, compared to visual assessment, the technical solution provided in this disclosure can quantitatively obtain the target transformation parameters for each pair of tooth models, avoiding misjudgments caused by doctors' lack of experience and improving the accuracy of orthodontic treatment monitoring.

[0049] In this embodiment, there is no need to use an intraoral scanner to scan the user's teeth. Only multiple intraoral images are needed to obtain the target transformation parameters for each pair of teeth in the second and first dental models (i.e., the dental models corresponding to the desired orthodontic effect) corresponding to the multiple intraoral images. The doctor can then know the orthodontic treatment effect based on the target transformation parameters. In this way, users can monitor orthodontic treatment more conveniently.

[0050] In another embodiment of this disclosure, the method further includes: S150. If the loss function corresponding to the initial camera parameters and initial transformation parameters does not meet the preset conditions, then the initial camera parameters and initial transformation parameters are optimized until the corresponding loss function meets the preset conditions.

[0051] S160. Determine the optimized initial transformation parameters as the target transformation parameters.

[0052] Specifically, the optimized initial camera parameters can also be used as the target camera parameters.

[0053] Specifically, if the loss function corresponding to the initial camera parameters and initial transformation parameters does not meet the preset conditions, it indicates that the alignment accuracy between the first tooth model and the second tooth model in at least one pair of tooth models is low (or does not meet the requirements) after similarity transformation according to its corresponding initial transformation parameters, and the difference between the initial camera parameters and the actual camera parameters is large (or does not meet the requirements). In this case, the initial camera parameters and initial transformation parameters can be optimized, and then the loss function corresponding to the optimized initial camera parameters and initial transformation parameters can be determined. If the loss function corresponding to the optimized initial camera parameters and initial transformation parameters meets the preset conditions, then the optimized initial transformation parameters are determined as the target transformation parameters. If the loss function corresponding to the optimized initial camera parameters and initial transformation parameters does not meet the preset conditions, then the initial camera parameters and initial transformation parameters are optimized again until the corresponding loss function meets the preset conditions, and finally the optimized initial transformation parameters are determined as the target transformation parameters. Of course, the final optimized initial camera parameters can also be determined as the target camera parameters.

[0054] Understandably, by continuously optimizing the initial camera parameters and initial transformation parameters, the alignment accuracy of the first and second tooth models becomes increasingly higher after similarity transformations are performed according to the corresponding optimized initial transformation parameters. This allows for the acquisition of the target transformation parameters for each pair of tooth models. In this way, obtaining the target transformation parameters becomes simple and convenient.

[0055] Figure 2 This illustration shows a flowchart of another orthodontic treatment monitoring method provided by an embodiment of the present disclosure. This embodiment is an optimization based on the above embodiments, and can be combined with various optional solutions from one or more of the above embodiments.

[0056] like Figure 2 As shown, this orthodontic treatment monitoring method may include the following steps.

[0057] S210. Acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model.

[0058] Specifically, S210 and S110 are similar, and will not be described in detail here.

[0059] S220. Determine the initial transformation parameters for each pair of tooth models in the first dental model and the second dental model corresponding to multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth.

[0060] Specifically, S220 and S120 are similar, and will not be described in detail here.

[0061] S230. For each pixel in multiple intraoral images, based on the initial camera parameters and initial transformation parameters, detect whether the ray corresponding to the pixel in the world coordinate system intersects with the first dental model after similarity transformation according to the initial transformation parameters, obtain the detection result, and determine the first color value and first mask value of the pixel based on the detection result.

[0062] Specifically, the first color value is the color value of the pixel estimated based on the initial camera parameters and initial transformation parameters. The color value may include grayscale values ​​or RGB values, but is not limited to these.

[0063] Specifically, the first mask value is the mask value of the pixel estimated based on the initial camera parameters and initial transformation parameters. The mask value is used to characterize whether the pixel is a background pixel or the pixel corresponding to the teeth; for example, using... Characterizing background pixels, using The pixels that represent the teeth.

[0064] Specifically, the first dental model after similarity transformation according to the initial transformation parameters refers to the first dental model after similarity transformation of each first tooth model according to the corresponding initial transformation parameters.

[0065] Specifically, for each pixel, the ray corresponding to the pixel in the world coordinate system is determined based on the initial camera parameters corresponding to the intraoral image where the pixel is located. Then, it is detected whether the ray corresponding to the pixel intersects with the first dental model after similarity transformation according to the initial transformation parameters, and the detection result is obtained. Then, the first color value of the pixel is determined based on the detection result.

[0066] In some embodiments, determining the first color value of a pixel based on the detection result includes: if the detection result is intersecting, then querying the initial color field network based on the intersection point coordinates and the direction of the ray to obtain the first color value of the pixel; if the detection result is non-intersecting, then determining the first color value of the pixel to be 0.

[0067] Specifically, the network parameters in the initial color field network are obtained by random setting.

[0068] For example, the initial color field network is denoted as It is a three-dimensional coordinate system. With direction The function outputs the RGB values ​​rendered by the NDS. These are network parameters. If the intersection coordinates are... X 0, the direction of the ray isV 0, will X 0 and V 0 pending entries You can then obtain the first color value.

[0069] In some embodiments, determining the first mask value of a pixel based on the detection result includes: if the detection result is intersecting, determining the first mask value of the pixel as a first value; if the detection result is non-intersecting, determining the first mask value of the pixel as a second value.

[0070] The first value represents the pixel corresponding to a tooth. The second value represents the pixel as background pixels. For example, the first value could be... The second value can be .

[0071] S240. Determine the color loss based on the first and second color values ​​of each pixel in multiple intraoral images.

[0072] Specifically, the second color value is the actual color value of the pixel in its corresponding intraoral image.

[0073] In some embodiments, S240 includes: for each pixel in a plurality of intraoral images, determining a color difference between a first color value and a second color value of the pixel; and determining a color loss based on the color difference between each pixel in the plurality of intraoral images.

[0074] For example, determining the color loss based on the color difference of each pixel in multiple intraoral images may include summing the squares (or absolute values, etc.) of the color differences of each pixel to obtain the color loss, but is not limited to this.

[0075] S250. Determine the mask loss based on the first mask value and the second mask value of each pixel in multiple intraoral images, wherein the second mask value is determined by multiple mask images corresponding to the multiple intraoral images.

[0076] Specifically, for each intraoral image, semantic segmentation is performed to distinguish between the teeth and the background. The mask values ​​of the pixels in the teeth are set to the first value, and the mask values ​​of the pixels in the background are set to the second value, thus obtaining the corresponding mask image.

[0077] Specifically, the second mask value is the actual mask value of the pixel in its corresponding mask image.

[0078] In some embodiments, S250 includes: for each pixel in a plurality of intraoral images, determining a mask difference between a first mask value and a second mask value of the pixel; and determining a mask loss based on the mask difference of each pixel in the plurality of intraoral images.

[0079] For example, determining the mask loss based on the mask difference of each pixel in multiple intraoral images may include summing the squares (or absolute values, etc.) of the mask differences of each pixel to obtain the mask loss, but is not limited to this.

[0080] S260. Based on the color loss and mask loss, determine the loss functions corresponding to the initial camera parameters and initial transformation parameters.

[0081] In some embodiments, S260 includes: performing weighted summation on the color loss and mask loss to obtain the loss function corresponding to the initial camera parameters and the initial transformation parameters.

[0082] Understandably, by weighting and summing, the proportions of color loss and mask loss can be flexibly adjusted, allowing the loss function to focus on the loss type that is of greater concern. This, in turn, makes the target transformation parameters more accurately reflect the differences in the dental model regarding aspects that Chinese medicine practitioners pay more attention to.

[0083] Of course, you can also directly sum the color loss and mask loss to obtain the loss function.

[0084] To illustrate in detail the process of determining the loss function provided in the embodiments of this disclosure, a detailed explanation will be given below based on a specific example.

[0085] N semantically segmented intraoral images are denoted as ,in, I i For the i-th intraoral image after semantic segmentation, the pixel coordinates of a certain pixel in the semantically segmented intraoral image are... The corresponding RGB value (i.e., the second color value) is denoted as For each semantically segmented intraoral image, the mask value of pixels corresponding to the teeth is set to 1, and the mask value of pixels corresponding to the background is set to 0, generating a mask image. The mask images generated after semantic segmentation of N intraoral images are denoted as follows: ,in, m i Let be the mask image generated after semantic segmentation of the i-th intraoral image. Since all intraoral images are captured by the same image acquisition device, the camera intrinsic parameters corresponding to the N intraoral images are the same, denoted as . Each intraoral image has an independent camera pose, and the camera extrinsic parameters of the i-th intraoral image are denoted as... (6 degrees of freedom), then the overall camera parameters are denoted as: .

[0086] The first dental model includes The first tooth model Each first tooth model has a common scaling ratio. And their respective independent rigid body transformation parameters (i.e., translation and rotation matrices). (6 degrees of freedom), its overall number is denoted as .

[0087] The color field network of the first dental model is denoted as... It is a three-dimensional coordinate system. With direction The function outputs the RGB values ​​rendered by the NDS. These are network parameters.

[0088] The process of calculating the loss function corresponding to the initial camera parameters and initial transformation parameters is as follows: For each pixel in N intraoral images, the ray of the pixel in the world coordinate system can be calculated using the initial camera parameters corresponding to its intraoral image. This allows us to calculate the intersection point of the ray with the first dental model after similarity transformation according to the initial transformation parameters. If there is no intersection, the color rendered by NDS (i.e., the first color value) is 0; if there is an intersection, the intersection coordinates and ray direction are substituted into the... This yields the color rendered by the NDS (i.e., the first color value). Since the intersection point coordinates are related to both camera and transformation parameters, and the color rendered by the NDS is also related to the network parameters of the color field network, the color rendered by the NDS (i.e., the first color value) is denoted as... If they do not intersect, the first mask value of the pixel is... If they intersect, the first mask value of the pixel is... Since whether they intersect depends on both camera parameters and transformation parameters, the first mask value is denoted as... .

[0089] The first and second color values ​​of each pixel in N intraoral images are compared to obtain the color loss:

[0090] in, For color loss, For pixel coordinates The first color value of the pixel. For pixel coordinates The second color value of the pixel.

[0091] The first mask value and the second mask value of each pixel in N intraoral images are compared to obtain the mask loss:

[0092] in, For mask loss, For pixel coordinates The first mask value of the pixel. For pixel coordinates The second mask value of the pixel.

[0093] The color loss and mask loss are weighted and summed to obtain the loss function:

[0094] in, For loss function, For color loss The weight value, Loss of mask The weight value.

[0095] S270. If the loss function corresponding to the initial camera parameters and the initial transformation parameters meets the preset conditions, then the initial transformation parameters are determined as the target transformation parameters.

[0096] Specifically, if the initial camera parameters, initial transformation parameters, and the loss function corresponding to the initial color field network do not meet the preset conditions, then the initial transformation parameters are determined as the target transformation parameters. In this case, the initial camera parameters are the target camera parameters, and the initial color field network is the color field network of the second dental model.

[0097] Specifically, if the initial camera parameters, initial transformation parameters, and the loss function corresponding to the initial color field network do not meet the preset conditions, then the initial camera parameters, initial transformation parameters, and initial color field network are optimized until the corresponding loss function meets the preset conditions. At this point, the optimized initial camera parameters become the target camera parameters, and the optimized initial color field network becomes the color field network of the second dental model.

[0098] Those skilled in the art should understand that "determining the optimized initial camera parameters, optimized initial transformation parameters, and the loss function corresponding to the optimized color field network" is similar to S230-S260, except that the optimized initial camera parameters, optimized initial transformation parameters, and optimized color field network are used in the loss function calculation, which will not be elaborated here.

[0099] In this embodiment of the disclosure, by utilizing NDS (Neural Deferred Shading) to jointly optimize camera parameters and transformation parameters, the target transformation parameters can be extracted simply and quickly from multiple intraoral images taken by the user and the first dental model, thereby obtaining the tooth pose difference with accuracy that meets orthodontic requirements.

[0100] Figure 3 A schematic diagram of the structure of an orthodontic treatment monitoring device provided in an embodiment of this disclosure is shown.

[0101] like Figure 3 As shown, the orthodontic treatment monitoring device 300 may include: The first acquisition module 310 is used to acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model. The first determining module 320 is used to determine the initial transformation parameters corresponding to each pair of tooth models in the first dental model and the second dental model corresponding to multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth. The second determining module 330 is used to determine the loss function corresponding to the initial camera parameters and the initial transformation parameters. The initial camera parameters are the camera parameters corresponding to multiple intraoral images. The loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth models after the similarity transformation is performed according to the corresponding initial transformation parameters. The third determining module 340 is used to determine the initial transformation parameters as the target transformation parameters if the loss function corresponding to the initial camera parameters and the initial transformation parameters meets the preset conditions.

[0102] In another embodiment of this disclosure, the device further includes: The optimization module is used to optimize the initial camera parameters and initial transformation parameters until the corresponding loss function meets the preset conditions if the loss function corresponding to the initial camera parameters and initial transformation parameters does not meet the preset conditions. The fourth determination module is used to determine the optimized initial transformation parameters as the target transformation parameters.

[0103] In yet another embodiment of this disclosure, the first determining module 320 includes: The reconstruction submodule is used to perform three-dimensional reconstruction based on multiple intraoral images to obtain a third dentition model. The registration submodule is used to perform coarse registration processing on the first and third dental models to obtain the initial transformation parameters for each pair of tooth models.

[0104] In another embodiment of this disclosure, the second determining module 330 includes: The first determination submodule is used to detect whether the ray corresponding to the pixel in the world coordinate system intersects with the first dental model after similar transformation according to the initial transformation parameters for each pixel in multiple intraoral images, based on the initial camera parameters and initial transformation parameters, to obtain the detection result, and to determine the first color value and the first mask value of the pixel based on the detection result. The second determining submodule is used to determine the color loss based on the first color value and the second color value of each pixel in multiple intraoral images; The third determination submodule is used to determine the mask loss based on the first mask value and the second mask value of each pixel in multiple intraoral images, wherein the second mask value is determined by multiple mask images corresponding to the multiple intraoral images. The fourth determination submodule is used to determine the loss function corresponding to the initial camera parameters and initial transformation parameters based on the color loss and mask loss.

[0105] In another embodiment of this disclosure, the first sub-determining module includes: The query unit is used to query the initial color field network to obtain the first color value of the pixel if the detection result is an intersection. The first determining unit is used to determine the first color value of the pixel to be 0 if the detection result is non-intersecting; The second determining submodule includes: The second determining unit is used to determine the color difference between the first color value and the second color value of each pixel in multiple intraoral images. The third determining unit is used to determine the color loss based on the color difference between each pixel in multiple intraoral images.

[0106] In another embodiment of this disclosure, the first sub-determining module includes: The fourth determining unit is used to determine the first mask value of the pixel as the first value if the detection result is intersection; The fifth determining unit is used to determine the first mask value of the pixel as the second value if the detection result is non-intersecting; The third determining submodule includes: The sixth determining unit is used to determine the mask difference between the first mask value and the second mask value of each pixel in multiple intraoral images; The seventh determining unit is used to determine the mask loss based on the mask difference of each pixel in multiple intraoral images.

[0107] In another embodiment of this disclosure, the fourth determining submodule includes: The eighth determining unit is used to perform weighted summation of color loss and mask loss to obtain the loss function corresponding to the initial camera parameters and initial transformation parameters.

[0108] It should be noted that, Figure 3 The orthodontic treatment monitoring device 300 shown can perform Figures 1 to 2 The various steps in the method embodiment shown are implemented. Figures 1 to 2 The processes and effects in the method embodiments shown are not described in detail here.

[0109] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown.

[0110] like Figure 4 As shown, the electronic device may include a processor 401 and a memory 402 storing computer program instructions.

[0111] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0112] Memory 402 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway device. In a particular embodiment, memory 402 is a non-volatile solid-state memory. In a particular embodiment, memory 402 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0113] The processor 401 reads and executes computer program instructions stored in the memory 402 to perform the steps of the orthodontic treatment monitoring method provided in this embodiment of the present disclosure.

[0114] In one example, the electronic device may also include a transceiver 403 and a bus 404. Wherein, as... Figure 4 As shown, the processor 401, memory 402 and transceiver 403 are connected via bus 404 and communicate with each other.

[0115] Bus 404 includes hardware, software, or both. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 404 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0116] The following are embodiments of a computer-readable storage medium provided in this disclosure. This computer-readable storage medium belongs to the same inventive concept as the orthodontic treatment monitoring methods in the above embodiments. For details not described in detail in the embodiments of the computer-readable storage medium, please refer to the embodiments of the orthodontic treatment monitoring methods described above.

[0117] This embodiment provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an orthodontic treatment monitoring method, the method comprising: Acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model; Determine the initial transformation parameters for each pair of tooth models in the first dental model and the second dental model corresponding to multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth. Determine the loss function corresponding to the initial camera parameters and initial transformation parameters. The initial camera parameters are the camera parameters corresponding to multiple intraoral images. The loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth models after the similarity transformation is performed according to the corresponding initial transformation parameters. If the loss function corresponding to the initial camera parameters and the initial transformation parameters meets the preset conditions, then the initial transformation parameters are determined as the target transformation parameters.

[0118] Of course, the computer-executable instructions provided in the embodiments of this disclosure are not limited to the above-described method operations, but can also perform related operations in the orthodontic treatment monitoring method provided in any embodiment of this disclosure.

[0119] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this disclosure can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer cloud platform (which may be a personal computer, server, or network cloud platform, etc.) to execute the orthodontic treatment monitoring methods provided in the various embodiments of this disclosure.

[0120] Note that the above are merely preferred embodiments and technical principles of this disclosure. Those skilled in the art will understand that this disclosure is not limited to the specific embodiments described herein, and various obvious modifications, readjustments, and substitutions can be made without departing from the scope of protection of this disclosure. Therefore, although this disclosure has been described in detail through the above embodiments, this disclosure is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this disclosure, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for monitoring orthodontic treatment, characterized in that, include: Acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model; Determine the initial transformation parameters for each pair of tooth models in the first dental model and the second dental model corresponding to the multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth; Determine the initial camera parameters and the loss function corresponding to the initial transformation parameters, wherein the initial camera parameters are the camera parameters corresponding to the multiple intraoral images, and the loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth model pairs after the similarity transformation is performed according to the corresponding initial transformation parameters, and the total deviation includes color value deviation and mask value deviation; If the loss function corresponding to the initial camera parameters and the initial transformation parameters satisfies a preset condition, then the initial transformation parameters are determined as the target transformation parameters, wherein the preset condition includes the loss function being less than a preset threshold.

2. The method according to claim 1, characterized in that, Also includes: If the loss function corresponding to the initial camera parameters and the initial transformation parameters does not meet the preset condition, then the initial camera parameters and the initial transformation parameters are optimized until the corresponding loss function meets the preset condition. The optimized initial transformation parameters are determined as the target transformation parameters.

3. The method according to claim 1 or 2, characterized in that, The determination of the initial transformation parameters for each pair of teeth in the first dental model and the second dental model corresponding to the multiple intraoral images includes: A three-dimensional reconstruction was performed based on the multiple intraoral images to obtain a third dental model. Coarse registration is performed on the first and third dental models to obtain the initial transformation parameters corresponding to each pair of dental models.

4. The method according to claim 1 or 2, characterized in that, The determination of the loss function corresponding to the initial camera parameters and the initial transform parameters includes: For each pixel in the multiple intraoral images, based on the initial camera parameters and the initial transformation parameters, it is detected whether the ray corresponding to the pixel in the world coordinate system intersects with the first dental model after similarity transformation according to the initial transformation parameters, and the detection result is obtained. Based on the detection result, the first color value and the first mask value of the pixel are determined. The color loss is determined based on the first and second color values ​​of each pixel in the multiple intraoral images; Mask loss is determined based on the first mask value and the second mask value of each pixel in the multiple intraoral images, wherein the second mask value is determined by multiple mask images corresponding to the multiple intraoral images; Based on the color loss and the mask loss, determine the loss functions corresponding to the initial camera parameters and the initial transformation parameters.

5. The method according to claim 4, characterized in that, Determining the first color value of the pixel based on the detection result includes: If the detection result is an intersection, then the first color value of the pixel is obtained by querying the initial color field network based on the intersection point coordinates and the direction of the ray; If the detection result is non-intersecting, then the first color value of the pixel is determined to be 0; The step of determining the color loss based on the first and second color values ​​of each pixel in the plurality of intraoral images includes: For each pixel in the plurality of intraoral images, determine the color difference between the first color value and the second color value of the pixel; The color loss is determined based on the color difference between each pixel in the multiple intraoral images.

6. The method according to claim 4, characterized in that, Determining the first mask value of the pixel based on the detection result includes: If the detection result is an intersection, then the first mask value of the pixel is determined to be a first value; If the detection result is non-intersecting, then the first mask value of the pixel is determined to be the second value; The step of determining the mask loss based on the first mask value and the second mask value of each pixel in the plurality of intraoral images includes: For each pixel in the plurality of intraoral images, determine the mask difference between the first mask value and the second mask value of the pixel; The mask loss is determined based on the mask difference between each pixel in the multiple intraoral images.

7. The method according to claim 4, characterized in that, The step of determining the loss function corresponding to the initial camera parameters and the initial transform parameters based on the color loss and the mask loss includes: The color loss and the mask loss are weighted and summed to obtain the loss functions corresponding to the initial camera parameters and the initial transformation parameters.

8. An orthodontic treatment monitoring device, characterized in that, include: The first acquisition module is used to acquire multiple intraoral images of the user and a first dental model, wherein the first dental model includes at least one first tooth model; The first determining module is used to determine the initial transformation parameters corresponding to each pair of tooth models in the first dental model and the second dental model corresponding to the multiple intraoral images, wherein the second dental model includes at least one second tooth model, and the tooth model pair includes a first tooth model and a second tooth model corresponding to the same tooth. The second determining module is used to determine the initial camera parameters and the loss function corresponding to the initial transformation parameters, wherein the initial camera parameters are the camera parameters corresponding to the multiple intraoral images, and the loss function is used to characterize the total deviation between the first tooth model and the second tooth model in each pair of tooth model pairs after the similarity transformation is performed according to the corresponding initial transformation parameters, and the total deviation includes color value deviation and mask value deviation. The third determining module is used to determine the initial transformation parameters as target transformation parameters if the loss function corresponding to the initial camera parameters and the initial transformation parameters meets a preset condition, wherein the preset condition includes the loss function being less than a preset threshold.

9. An electronic device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the orthodontic treatment monitoring method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the orthodontic treatment monitoring method according to any one of claims 1-7.

Images