Method, device, and medium for face shape prediction and oral orthodontic plan generation

Abstract

Embodiments of the present application relate to the field of digital medical device design technology, and in particular, to a method, device, and medium for face shape prediction and oral orthodontic plan generation. The method for face shape prediction comprises: acquiring a composite facial model of a patient in a first state, the composite facial model comprising: a dental jaw part and a soft tissue part; identifying a pre-selected dental jaw feature point on the dental jaw part of the composite facial model, and acquiring a first position of the dental jaw feature point; acquiring, on the basis of an orthodontic plan, a second position of the dental jaw feature point of the patient in a second state in the composite facial model; calculating, according to the first positions and the second positions of the dental jaw feature points, a change relationship between the movement of the dental jaw feature point and mesh vertices in the composite facial model by adopting a deformation method; acquiring, on the basis of given target positions of the dental jaw part in the composite facial model and the change relationship, new positions of the mesh vertices of the soft tissue part in the composite facial model.

Description

Facial profile prediction and method, device, medium for generating an orthodontic treatment plan Cross-reference to Related Applications

[0001] The present application is based on the Chinese patent application with the application number "202410593264.6" and the filing date of May 13, 2024, the Chinese patent application with the application number "202410592017.4" and the filing date of May 13, 2024, and the Chinese patent application with the application number "202410593261.2" and the filing date of May 13, 2024, and claims priority to the above-mentioned Chinese patent applications, the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the technical field of digital design of medical devices, in particular to a facial profile prediction and a method, device, and medium for generating an orthodontic treatment plan. BACKGROUND

[0003] Orthodontics is a comprehensive treatment method for dental and oral and maxillofacial deformities, mainly aiming at the correction of dental and oral and maxillofacial deformities. Its main purpose is to treat irregular teeth, tooth gaps, tooth protrusion, and abnormal conditions such as jaw development. Through orthodontic treatment, teeth can become complete and beautiful, achieving the goal of balanced coordination of the dental and jaw system. In theory, the effect of orthodontics is significant, not only can it restore masticatory function, improve oral hygiene, restore facial beauty, but also can correct unclear pronunciation caused by developmental abnormalities, and improve learning and working efficiency. However, due to the long cycle and high cost of orthodontics, and the high requirement of medical professionalism, it is difficult for patients to judge whether the treatment plan is suitable or the treatment effect is as expected in the early stage based on the dental and jaw model alone, resulting in some users being unable to obtain satisfactory treatment effect after paying time and money, and having a large expectation difference.

[0004] Among the above factors affecting patient expectations, the appearance after orthodontics is a highly subjective evaluation parameter. Currently, the prediction of the dental and jaw part in this factor has been predicted through three-dimensional model reconstruction, digital design of treatment plans, and other technologies, and can be visualized through the model, providing reliable reference for patients, doctors, or professionals. However, appearance is often the main purpose of user treatment, and appearance is mainly based on soft tissue, so users cannot perceive changes in appearance through dental and jaw models. Therefore, we need related methods to provide more comprehensive reference for patients, doctors, or other professionals, so as to generate a more optimal orthodontic treatment plan. SUMMARY

[0005] The embodiments of the present application provide a face shape prediction method and a method for generating an orthodontic treatment plan, which can accurately predict face shape changes, and the prediction method is fast, reliable and highly generalizable; or, the face shape changes are accurately predicted, and a more optimal orthodontic treatment plan is generated according to the prediction result.

[0006] The face shape prediction method provided in the embodiments of the present application comprises: obtaining a face composite model of a patient in a first state, the face composite model comprising a dental arch part and a soft tissue part; identifying preselected dental arch feature points on the dental arch part of the face composite model, and obtaining first positions of the dental arch feature points; obtaining second positions of the dental arch feature points of the patient in a second state in the face composite model based on a treatment plan; calculating a movement of the dental arch feature points and a change relationship of grid vertices in the face composite model based on a deformation method according to the first positions and the second positions of the dental arch feature points; and obtaining new positions of the grid vertices of the soft tissue part in the face composite model based on the change relationship given a target position of the dental arch part in the face composite model.

[0007] The method for generating an orthodontic treatment plan provided in the embodiments of the present application comprises: obtaining a face composite model of a patient in an initial state, the face composite model comprising a dental arch part and a soft tissue part; obtaining P treatment plans of the patient respectively, each treatment plan corresponding to a target state of dental arch movement, and P being a natural number greater than 1; deforming the face composite model according to dental arch movements of the P treatment plans to obtain P face prediction models corresponding to the P target states respectively; and selecting an optimal treatment plan as a target treatment plan from the P treatment plans according to an evaluation of the face prediction models of the target states; wherein the P face prediction models of the target states are obtained by the face shape prediction method as described in any of the embodiments of the present application.

[0008] The embodiment of the present application also provides a method for generating an orthodontic treatment plan, comprising: obtaining a face composite model of a patient in an initial state, the face composite model comprising a dental arch part and a soft tissue part; obtaining a treatment plan of the patient, the treatment plan comprising N orthodontic steps executed in sequence, for realizing dental arch movement of the patient from the initial state to a target state, wherein each orthodontic step corresponds to a stage state; deforming the face composite model in the initial state to obtain a face prediction model of the orthodontic step according to the initial state and the stage state of the orthodontic step; obtaining M face prediction models according to the stage states of M orthodontic steps; wherein the M orthodontic steps are selected from the kth orthodontic step to the Nth orthodontic step, k, N and M are natural numbers greater than or equal to 1, and k and M are less than or equal to N; in the M stage states, selecting an optimal stage state as a treatment target according to evaluation of each face prediction model; updating the treatment plan according to the treatment target; wherein the M face prediction models are obtained by the face profile prediction method according to any embodiment of the present application.

[0009] The embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any of the above methods.

[0010] The embodiment of the present application also provides a computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement any of the above methods.

[0011] Therefore, the embodiment of the present application can achieve at least one of the following effects:

[0012] 1. By obtaining a face composite model comprising a dental arch part and a soft tissue part associated with a face, determining the correlation between the dental arch part and the soft tissue part, and then determining the change relationship of the face profile by the known change of the dental arch part in the treatment plan, so as to predict the position change of other parts by the change relationship, since the change relationship can be predicted only by the data of the patient itself without combining a large amount of historical data, the amount of data involved in the operation in the prediction of the face profile prediction method in the present application is small, and the accuracy is high;

[0013] 2. By predicting the changes of the face shape in the target state of the orthodontic plan, a reference is provided for the user in the selection of the orthodontic plan. Specifically, the changes of the face shape in the target state of each orthodontic plan are predicted, and then the obtained face prediction model is evaluated so as to select a better orthodontic plan from them, thereby avoiding the problem that only a single medical standard is evaluated in the design of the orthodontic plan without considering the personalized needs of the user, so that the generated orthodontic plan is more in line with the expectations of the user and the user experience is improved. Moreover, not only can the orthodontic plan be optimized among multiple orthodontic plans, but also the target state can be optimized in multiple stages of one orthodontic plan, so that the user can comprehensively consider the orthodontic period, the cost, the final orthodontic effect and the like, and the application scenarios are flexible, so as to promote the wide application of the scheme in the present application. BRIEF DESCRIPTION OF DRAWINGS

[0014] One or more embodiments are illustrated by way of example in the figures that form a part of this disclosure and which are shown by way of illustration in which like references indicate similar elements. The figures in the drawings are not necessarily drawn to scale, except as otherwise noted, and are presented for purposes of illustration only.

[0015] FIG. 1 is a flowchart of a method for establishing a face shape model according to an embodiment of the present application;

[0016] FIG. 2 is an effect diagram of a face lateral film according to an embodiment of the present application;

[0017] FIG. 3 is a flowchart of a method for obtaining a face mesh model according to an embodiment of the present application;

[0018] FIG. 4a is a schematic diagram of a dental arch model according to an embodiment of the present application;

[0019] FIG. 4b is a schematic diagram of a face mesh model according to an embodiment of the present application;

[0020] FIG. 5 is a schematic diagram of a face composite model according to an embodiment of the present application;

[0021] FIG. 6 is an effect diagram of the alignment of the lateral film and the center line of the face composite model according to an embodiment of the present application;

[0022] FIG. 7 is a schematic diagram of a face composite model with increased texture features after deformation according to an embodiment of the present application;

[0023] FIG. 8 is a flowchart of a face shape prediction method according to an embodiment of the present application;

[0024] FIG. 9 is a flowchart of a method for forming a face composite model according to an embodiment of the present application;

[0025] FIG. 10a is a schematic diagram of a face composite model before deformation according to an embodiment of the present application;

[0026] Fig. 10b is a schematic diagram of a deformed face composite model according to an embodiment of the present application;

[0027] Fig. 11 is a schematic diagram of a face composite model with added texture features after deformation in a face shape prediction method according to an embodiment of the present application;

[0028] Fig. 12 is a flowchart of a method for generating an orthodontic treatment plan according to an embodiment of the present application;

[0029] Fig. 13 is a flowchart of a method for generating an orthodontic treatment plan according to an embodiment of the present application;

[0030] Fig. 14 is a flowchart of another method for generating an orthodontic treatment plan according to an embodiment of the present application

[0031] Fig. 15 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are presented in order to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and based on various changes and modifications of the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the embodiments can be combined and referenced with each other on the premise of not contradicting.

[0033] The “anterior tooth region” and “posterior tooth region” mentioned in the embodiments of the present application are defined according to the classification of teeth in the 2nd edition of “Introduction to Oral Medicine” published by Peking University Medical Press, pages 36-38. The posterior tooth region includes premolars and molars, which are shown as teeth 4-8 in the FDI (Fédération Dentaire Internationale, World Dental Federation) marking method. The teeth in the anterior tooth region include incisors, lateral incisors and canines, which are shown as teeth 1-3 in the FDI marking method.

[0034] The “gnathic plane” mentioned in the embodiments of the present application is obtained according to the definition and confirmation method in the 6th edition of “Orthodontics”, page 83. One is the line connecting the occlusal midpoint of the first permanent molar and the midpoint between the upper and lower central incisors (1 / 2 of the occlusal plane or the opening of the jaw). The other is obtained by dividing the posterior tooth contact point, and the contact point of the first permanent molar and the first deciduous molar or the first premolar is often used.

[0035] Some embodiments of the present application relate to a method for establishing a face type model of a human face. In some embodiments, the method for establishing the face type model of the human face can be as shown in FIG. 1, which includes the following steps:

[0036] In step 101, a registration image is obtained. The registration image can be a composite image formed by fusing a patient's face lateral film and a profile photograph, or a patient's face lateral film.

[0037] In step 102, a face mesh model and a dental model are obtained respectively.

[0038] In step 103, the face mesh model and the dental model are fused based on the registration image to form a face composite model.

[0039] The method for establishing the face type model of the human face in the embodiments of the present application can fuse a face lateral film with both dental features and partial face type contour features and a profile photograph with face type contour features to form a composite image with rich dental features and face type contour features, and use the composite image as a registration reference for a three-dimensional model, so that the registration result is more accurate. In addition, the method can directly use a face lateral film for model registration, which is simple in data and fast in registration speed, and the face type model formed thereby is accurate. The implementation details of the method for establishing the face type model of the human face in the embodiments of the present application are described below. The following content is provided for the convenience of understanding and is not essential for implementing the present application.

[0040] It should be noted that the method for establishing the face type model of the human face in the embodiments of the present application can be realized by hardware or a combination of computer software and hardware. For hardware implementation, the method for establishing the face type model of the human face can be realized by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for realizing the function of face type prediction, or a combination of the above.

[0041] In step 101, the registration image is taken as an example of a composite image formed by fusing a patient's face lateral film and a profile photograph. In one embodiment, a patient's face lateral film and a profile photograph are obtained respectively, fused, and a composite image is formed.

[0042] In some embodiments, the facial profile and the profile photo can use the pre-treatment data collected by the doctor, such as the facial profile, which is a projection of the lateral surface of the skull taken at 90 degrees on the head side, and is the most commonly used X-ray film for orthodontic measurement. The original profile photo is a two-dimensional photo taken from the lateral view of the face (left view or right view), which can be color or black and white. According to the profile photo and the facial profile, the lateral profile and the profile photo can be registered and aligned by image fusion and feature recognition, and the marked feature points on the teeth (i.e., dental and jaw feature points) can be obtained, such as the feature points on the dentition (indicated by reference numeral A in FIG. 2) or the feature points on the jaw (indicated by reference numeral B in FIG. 2).

[0043] In some embodiments, when the registered image is a composite image formed by fusing the patient's facial profile and the profile photo, step 101 can be implemented by: identifying pre-selected facial contour feature points on the acquired patient's facial profile and profile photo; and forming the composite image based on the position registration fusion of the facial contour feature points. For example, the facial contour feature points on the facial profile can be indicated by reference numeral C in FIG. 2. By acquiring the facial contour feature points on each type of image for registration, since the facial profile has both dental and jaw features and part of the facial contour, the fusion is more operable, the feature information on the obtained composite image is rich, there are more feature points to choose from, and the subsequent three-dimensional model registration is facilitated. In some examples, the pre-selected facial contour feature points can include the tip of the nose and the convex points of the upper and lower lips. At this time, the tip of the nose and the convex points of the upper and lower lips are used as the facial contour feature points for registration, which are more accurate to identify during acquisition, so as to ensure the registration accuracy during subsequent registration.

[0044] In step 102, a facial mesh model and a dental and jaw model are respectively acquired.

[0045] In some embodiments, the acquired facial mesh model can be obtained by: splicing and fusing according to the patient's multi-angle facial photos. In some examples, the splicing and fusing method can be as shown in FIG. 3, which includes:

[0046] In step 301, the background of the multi-angle photographed facial photo is removed. In some embodiments, the K-means++ algorithm can be used to remove the background of the images taken at different angles. In addition to the K-means++ algorithm, other existing tools can also be used for background removal, which will not be listed one by one here.

[0047] At step 302, the multi-angle photographed face photo is processed in a chessboard format to realize calibration and stereoscopic correction, and to obtain camera intrinsic parameters. In the field of computer vision, camera calibration refers to calculating the intrinsic parameters of a camera through computer vision technology, so as to facilitate subsequent three-dimensional reconstruction, target tracking, image processing and other applications. The principle of chessboard calibration is to utilize the regular features of the chessboard in a three-dimensional coordinate system. This step can be divided into two parts: chessboard image extraction and camera parameter calculation.

[0048] Regarding the part of chessboard image extraction, some embodiments can be executed as follows: first, a picture of a chessboard is prepared, then the picture is photographed by a camera and saved. Then, the first step-chessboard image extraction can be entered. It needs to be explained that the purpose of chessboard image extraction is to obtain the corner point coordinates of the chessboard in the image. In this process, some image processing techniques can be used to extract the corner point coordinates, including gray scale transformation, image smoothing, edge detection, etc., which can be determined according to the actual form and quality of the image. In some embodiments, the image can be converted into a gray scale image, which can simplify the subsequent image processing operations. Then, the image needs to be smoothed to remove noise in the image. Common smoothing methods include Gaussian filtering, median filtering, etc. Then, the image needs to be edge detected to find the corner point coordinates. In this process, some classic edge detection algorithms can be used, such as Sobel operator, Canny operator, etc. When the corner point coordinates in the image are extracted, the second step-camera parameter calculation can be entered.

[0049] Regarding the part of camera parameter calculation, the intrinsic parameters of the camera can be calculated by the extracted corner point coordinates. The intrinsic parameters can include the focal length and optical center of the camera, etc. When calculating the intrinsic parameters of the camera, the coordinate information of the chessboard in the three-dimensional coordinate system needs to be utilized. By measuring the side length and the number of grids of the chessboard, the coordinates of the chessboard in the three-dimensional coordinate system can be calculated. Then, by using the picture taken by the camera and the extracted corner point coordinates, the correspondence between the pixel coordinates and the three-dimensional coordinates can be established, and then the intrinsic parameters of the camera can be solved. In some embodiments, when calculating the intrinsic parameters of the camera, the extrinsic parameters of the camera can also be calculated, including the rotation matrix and translation matrix of the camera. When calculating the extrinsic parameters of the camera, the coordinate information of the chessboard in the three-dimensional coordinate system and the corner point coordinates in the picture taken by the camera need to be utilized. By solving the correspondence between the pixel coordinates and the three-dimensional coordinates, the rotation matrix and the translation matrix of the camera, i.e. the extrinsic parameters of the camera, can be obtained. It can be seen that through the above calculation of camera parameters, basic support can be provided for subsequent computer vision applications, and the calculation of camera intrinsic and extrinsic parameters can improve the accuracy of subsequent analysis.

[0050] Step 303, using the parallax information of the multi-angle face photos and the camera intrinsic parameters, the depth and three-dimensional coordinates of the face in each face photo are calculated. Among them, stereo matching is performed on different angle images, and the parallax is calculated.

[0051] In some embodiments, after step 303, a point cloud denoising method can also be used to improve the quality of the point cloud data.

[0052] Step 304, select two face photos taken at different angles as a group for registration, and register at least two different face photo groups. Some embodiments use the ICP (Iterative Closest Point) algorithm to register two groups of point clouds (such as middle-left and middle-right) respectively.

[0053] Step 305, fuse and splice the registered face photo groups to form a three-dimensional point cloud model. Some embodiments use the ICP-merging algorithm to splice and fuse two groups of point cloud data to form a more comprehensive and consistent point cloud model. Since the data source used is a face photo, the pixel points in the photo can be used as point data to form a point cloud model.

[0054] Step 306, process the above three-dimensional point cloud model into a face mesh model.

[0055] As can be seen, the method of splicing and fusing multi-angle face photos to form a face mesh model in steps 301-306 above, such as using a chessboard format for positioning and correction, makes positioning and correction accurate and feasible to obtain an accurate three-dimensional mesh model.

[0056] Regarding the method of obtaining a face mesh model, in addition to the above-mentioned multi-angle face photo splicing and fusion method, in some embodiments, the patient's face can also be directly scanned to obtain a face mesh model. During scanning, existing third-party scanning equipment can be used, which will not be described here. As can be seen, the above embodiments clearly provide multiple methods for obtaining a face mesh model, so that different methods can be selected for different needs, expanding the application scenarios of the present application.

[0057] Regarding the acquisition of the dental model, the dental model only includes some embodiments which can be acquired by directly scanning the patient's dental arch, or by taking a dental model, manufacturing a plaster model, and then scanning the plaster model. The dental model also includes parts such as the jaw bone, which can be acquired by CBCT (Cone beam CT, Cone beam CT) scanning and reconstruction. As can be seen, the dental model can be acquired in different ways according to the actual application scenario, which will not be described here.

[0058] In some embodiments, the acquired dental model can only include the teeth of the upper and lower jaws, such as the teeth 20 of the upper and lower jaws shown in FIG. 4a, which can be used for cases of only tooth movement, and the data involved in subsequent analysis is simplified. In addition to the teeth, the dental model can also include parts of the jaw, skull, etc., which can be used for cases with tooth movement or jaw movement, and the data involved in subsequent analysis is complete. In some embodiments, the soft tissue part mainly includes the appearance part, i.e., the outer contour part of the soft tissue, such as the human face mesh model 10 shown in FIG. 4b, which can be used to represent the soft tissue part. Although the above FIG. 4a and FIG. 4b are presented in black and white form, in some embodiments, color patterns can also be used for expression, which will not be listed one by one here.

[0059] It is worth mentioning that the above-mentioned acquisition of the human face mesh model and the acquisition of the dental model can be acquired simultaneously or separately, and there is no limitation on the order of acquisition when acquired separately.

[0060] In step 103, the human face mesh model and the dental model are registered and fused based on the composite image to form the human face composite model.

[0061] Regarding the identification of various types of feature points, in some embodiments, step 103 can be implemented in the following way: identifying pre-selected human face contour feature points on the human face mesh model; identifying dental feature points on the dental model; and registering and fusing the human face mesh model and the dental model according to the relationship between the pre-selected dental feature points and the human face contour feature points and the composite image to form the human face composite model. The identification method of the dental feature points and the human face contour feature points can use existing identification methods, such as the method for acquiring landmarks in a cephalogram submitted by Zhengya Company on March 31, 2021 (application number: CN202110345400.6), or other feature point identification methods, which will not be listed one by one here. In one embodiment, when the landmarks are acquired, the type of the selected points can be pre-selected, such as setting pre-selected occlusion feature points, GO (gonion) points, etc., or some points can be selected as dental feature points from the identified landmarks.

[0062] Regarding the identification of pre-selected human face contour feature points on the human face mesh model, in one embodiment, the human face mesh model is an outer contour model, so the feature points representing contour changes can be selected from the human face mesh model, such as the lip protrusion point, the nose tip point, etc., which can be selected according to the needs of those skilled in the art, and will not be listed one by one here.

[0063] Regarding the registration fusion, the spatial positions of the same type of feature points recognized in different image data can be aligned, specifically by fixing one image data and moving and rotating another image data to realize the registration of the two, and then merging the two sets of image data to form one data file. One display example is shown in FIG. 5, including the fused dental model 20 and the face mesh model 10.

[0064] In one embodiment, when registering the lateral film and the face mesh model, the lateral film 30 can be aligned with the midline in the face composite model 10 (as shown in FIG. 6). The position relationship between the lateral film and the face composite model is relatively clear, so the midline alignment is directly performed, which is fast and effective. In one embodiment, after the midline alignment, the lateral film and the face mesh model can be quickly registered by registering various types of feature points.

[0065] It is worth mentioning that since the data obtained by different collection methods can be of different sizes, the image data can be calibrated before registration. In one embodiment, the following operation can be performed: Since the point cloud data is the pixel points in the above-mentioned photos, the distance between the pixel points can be calculated by setting the scale length, which refers to the actual length size measured by a straight line segment of a certain pixel in the registered and fused image, and is used to calibrate the actual distance between the pixel points. According to the actual length of the same object in the two types of image data, the scaling ratio of the two is calculated, and the distance between the same feature points in different image data is unified by scaling and other methods, so that the sizes of the calibrated images are consistent, avoiding registration errors and speeding up the registration.

[0066] It can be seen that in the embodiments of the present application, the face lateral film with both dental features and partial facial contour features and the profile photo with facial contour features are registered and fused to form a composite image including rich dental features and facial contour features, and the composite image is used as the registration reference of the three-dimensional model, so that the registration parameters are rich and the registration result is more accurate. In addition, the face lateral film can be directly used for model registration in the embodiments of the present application, the data is simple, the registration speed is fast, and the face shape model formed has high accuracy. At the same time, since the lateral film, profile photo and other data are routine collection data in orthodontic cases, they can be directly extracted and used when needed, simplifying data collection and facilitating the popularization of the present application.

[0067] It should be further explained that, although the composite image formed by registering and fusing the facial profile and the side view is used to register and fuse the face mesh model and the dental model in the above embodiment, in other embodiments, the facial profile (as shown in FIG. 2) can be directly used as the registration image to register and fuse the face mesh model and the dental model. Since the dental feature points and the facial contour feature points exist on the facial profile, the dental feature points and the facial contour feature points can be identified on the facial profile, and then the face mesh model and the dental model are registered and fused based on the two types of feature points. The registration and fusion process is substantially the same as that of the above embodiment, and will not be described again.

[0068] In addition, in some embodiments, before forming the face composite model, the texture feature of the face model can be scanned and acquired. Correspondingly, after forming the face composite model, the texture feature can be added to the face composite model. At this time, by adding the texture to the face model, the appearance of the model is closer to the real face effect. The face model after adding the texture is shown in FIG. 7. In addition, the added texture can be acquired when the face mesh model is acquired, reducing the newly added acquisition step. It should be noted that the part of the eye blocked by the white box in FIG. 7 is the privacy protection processing taken to protect the portrait right of the individual in the case, and is not a blank part in the established model.

[0069] The embodiments of the present application also relate to a face model display method. Specifically, the face model can be obtained according to any of the face model establishment methods provided in the above embodiments. Then, the face orthodontic prediction model is displayed on a display device. The display device can be an image output device, such as a display, a tablet computer (PAD), a mobile phone screen, and the like, which will not be listed one by one.

[0070] Some embodiments of the present application also relate to a face model prediction method. In some embodiments, the specific process of the face model prediction method can be as shown in FIG. 8, which includes the following steps.

[0071] In step 801, a face composite model of a patient in a first state is acquired, wherein the face composite model includes a dental part and a soft tissue part.

[0072] In step 802, preselected dental feature points are identified on the dental part of the face composite model, and the first positions of the dental feature points are acquired.

[0073] In step 803, based on a treatment plan, the second positions of the dental feature points of the patient in a second state are acquired.

[0074] At step 804, according to the first position and the second position of the dentognathic feature point, a deformation method is used to calculate the movement of the dentognathic feature point and the change relationship between the grid vertex in the human face composite model.

[0075] At step 805, given the target position of the dentognathic part in the human face composite model, based on the change relationship, the new position of the grid vertex of the soft tissue part in the human face composite model is obtained.

[0076] The implementation details of the human face profile prediction method of the embodiment shown in FIG. 8 will be specifically described below. The following content is only provided for the implementation details for easy understanding, and is not necessary for implementing the present solution.

[0077] It should be noted that the human face profile prediction method in the embodiments of the present application can be realized by hardware or a combination of computer software and hardware. For hardware implementation, the human face profile prediction method can be realized by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for realizing the human face profile prediction function, or a selected combination of the above devices.

[0078] In step 801, the human face composite model obtained in this step is a three-dimensional model including the associated dentognathic part and soft tissue part. The dentognathic model can only include the upper and lower jaw teeth, such as the upper and lower jaw teeth 20 shown in FIG. 4a, which can be used for cases of only tooth movement and can participate in subsequent analysis with simplified data. In addition to the teeth, the dentognathic model can also include the jaw bone, skull and other parts, which can be applied to cases with tooth movement or jaw movement and can participate in subsequent analysis with complete data. In some embodiments, the soft tissue part mainly includes the appearance part, i.e. the outer contour part of the soft tissue, such as the human face grid model 10 shown in FIG. 4b, which can be used to represent the soft tissue part. Although the above FIG. 4a and FIG. 4b are presented in black and white form, in some embodiments, color patterns can also be used for expression, which will not be listed one by one here.

[0079] In some embodiments, the human face composite model in the first state of the patient is obtained as shown in FIG. 9, and the formation method of the human face composite model includes:

[0080] At step 901, the patient's human face lateral film and profile are obtained respectively, registered and fused to form a composite image.

[0081] In step 901, the corresponding features have been described in the foregoing embodiments (which can be referred to the description in paragraphs

[0042] -

[0043] of the present specification), which will not be described one by one here.

[0082] At step 902, the face mesh model and the dental arch model are respectively acquired, and the face mesh model and the dental arch model are registered and fused based on the composite image to form the face composite model. The face mesh model and the dental arch model are registered and fused to form the face composite model in the embodiment of the present application, so that the face composite model obtained has the associated dental arch part and the soft tissue part representing the appearance. Since the face mesh model and the dental arch model are relatively simple to obtain, the generation of the face composite model in the embodiment of the present application is reliable and easy to promote, and the data acquisition difficulty of the terminal can be reduced. In the further embodiment of the present application, the face lateral film having the dental arch features and the partial facial contour features and the profile having the facial contour features are registered and fused to form the composite image including the rich dental arch features and the facial contour features, and the composite image is used as the registration reference of the three-dimensional model. The registration has rich optional parameters, so that the registration result is more accurate. As shown in FIG. 5, an effect diagram of the face composite model formed after the face mesh model 10 is fused with the dental arch part model 20 including the teeth is shown.

[0083] It can be seen that step 902 substantially corresponds to steps 102 and 103 in the foregoing embodiments, and therefore, the acquisition of the face mesh model and the dental arch model in step 902 can refer to the description of step 102 (specifically, the description in paragraphs

[0045] -

[0059] ), and the formation of the face composite model in step 902 can refer to the description of step 103 (specifically, the description in paragraphs

[0061] -

[0065] ). Therefore, step 902 will not be described here.

[0084] Although the above provides an example of registering and fusing the face lateral film and the profile to form the composite image to register and fuse the face mesh model and the dental arch model, this does not mean that only the scheme provided in the above example can be used. In other embodiments, the face lateral film (as shown in FIG. 2) can be directly used as the registration image to register and fuse the face mesh model and the dental arch model. Since the dental arch feature points and the face contour feature points exist on the face lateral film, the dental arch feature points and the face contour feature points can be identified on the face lateral film, and then the face mesh model and the dental arch model are registered and fused based on the two types of feature points. The registration and fusion process is substantially the same as that in the above embodiment, and will not be described here.

[0085] The above steps 901-902 are used to form the face composite model of the patient in the first state including the associated dental arch part and the soft tissue part. It can be understood that in other embodiments, the face composite model after fusion can be directly obtained by using the oral scanning and face scanning integration, which needs the cooperation of a third-party device, but the model can be obtained quickly. Therefore, the schemes for obtaining the face composite model in the first state provided in the related art will not be listed here.

[0086] It should be noted that the first state is not limited by the embodiments of the present application. The first state can be an initial state before orthodontic treatment of the patient, or a certain state during orthodontic treatment, etc. The specific corresponding stage can be selected according to the actual application scene, and will not be enumerated here.

[0087] In step 802, preselected dental arch feature points are identified on the dental arch part of the face composite model, and the first positions of the dental arch feature points are obtained. In some embodiments, the dental arch feature points can include points representing the shape, position, and other characteristics of the dental arch. A number of dental arch feature points are preselected from the dental arch feature points. In one example, the preselected dental arch feature points can include the occlusal contact points of the upper first molar and the lower first molar, i.e., U6 and L6 in FIG. 2, and the occlusal contact points of the upper central incisor and the lower central incisor, i.e., L1 in FIG. 2. In the embodiments of the present application, the occlusal contact points are used as the dental arch feature points for registration, and the occlusal contact points are characteristic and accurate for identification, so as to ensure the accuracy and stability of the subsequent registration results. The method of identifying the dental arch feature points on the face composite model can refer to existing various identification methods, such as the method for obtaining a landmark point in a cephalogram submitted by Zhengya Company on March 31, 2021 (application number: CN202110345400.6), or a recognition network based on machine learning method for recognizing the face composite model, which will not be enumerated here.

[0088] In step 803, based on the treatment plan, the second positions of the dental arch feature points of the patient in the second state of the face composite model are obtained. The treatment plan can include the positions of the teeth and the jaw in the digital orthodontic design, such as the target state of the dental arch, i.e., the position state of the dental arch in the digital design after the completion of orthodontic treatment, or a plurality of intermediate states successively passed, i.e., the intermediate state of the dental arch when reaching the target state of the dental arch. In the embodiments of the present application, the target state of the dental arch is taken as the second state, and the second positions of the dental arch feature points are the spatial positions of the dental arch feature points identified when the dental arch is in the target state. The specific expression can be represented in the form of position coordinates, or represented in the form of distance difference (equivalent to movement amount) of the corresponding feature points in the first state, etc., which will not be enumerated here.

[0089] In step 804, according to the first position and the second position of the dental class feature points, a deformation method is used to calculate the movement of the dental class feature points and the change relationship of the grid vertex in the human face composite model. In some embodiments, the deformation method can be calculated by using an interpolation deformation function. In some embodiments, the interpolation deformation function can be a thin-plate spline interpolation function. The interpolation deformation function is used for deformation calculation, which is more suitable for non-rigid deformation, so that the result of the predicted soft tissue deformation is more accurate. Moreover, the thin-plate spline interpolation function is used as the deformation function, and the Gaussian kernel function is introduced for calculation, so that the deformation is more in line with the deformation law of the soft tissue, the change is greater when it is closer to the moving tooth, and the change is smaller when it is farther away, so that the prediction result is more accurate. The basic principle of the thin-plate spline interpolation algorithm combined with the human face composite model is as follows:

[0090] Step S1, a mathematical function relationship between the control points and the prediction points is established.

[0091] Taking the positions of the control points and the prediction points in FIG. 10a as an example, the control points correspond to the first positions of the dental class feature points, and the prediction points correspond to the second positions of the dental class feature points. FIG. 10b is a model shape after deformation. The cusp point of the upper central incisor is selected as the control point, and the coordinates of the upper central incisor control point are (x0, y0, z0). It is assumed that the movement of the upper central incisor is buccal translation, and the translation amount is d x , then the tooth movement amount can be represented as (d x , 0, 0). It is assumed that the coordinates of the prediction point are (x1, y1, z1), so x1=x0+d x , y1=y0, and z1=z0. Based on the interpolation function, a function relationship between the control points and the prediction points can be established. The interpolation function can be selected from a polynomial interpolation, a spline function interpolation, etc. In this example, a thin-plate spline interpolation function is selected, and its mathematical expression includes (1)-(3), which are specifically represented as:

[0092]

[0093]

[0094]

[0095] In the equation, the left side of the equation is the position coordinates x1, y1, and z1 of the prediction point, and a1, a x , a y , a z , and w i on the right side of the equation are the coefficients of the interpolation function, N is the total number of all control points that need to be deformed, i is the serial number of all control points that need to be deformed, and U(||(x i , y i , z iThe smooth kernel function of the distance between the control points and the prediction points is constructed by the formula: f(r) = exp(-r2 / σ2), where r is the distance between the control points and the prediction points, and σ is the radius of the Gaussian kernel function.

[0096] Step S2: Based on the function relationship between the control points and the prediction points established in step S1, the linear equation set is solved to obtain the coefficients of the interpolation function.

[0097] In some examples, step S2 can be implemented by: substituting the coordinates of the control points and the prediction points into the interpolation mathematical relationship established in step S1, and substituting the Gaussian kernel function calculated according to the control points and the prediction points, and using the Gauss-Seidel method to solve the linear equation set. In addition to the equations determined by the interpolation mathematical relationship in step S1, four constraint equations of the interpolation coefficients are introduced, and the expression is as follows:

[0098]

[0099]

[0100]

[0101]

[0102] Thus, the linear equation set determined by the four equations is obtained, and the coefficients {a1, a2, a3, a4} of the interpolation function are finally calculated by a computer design program. x y z i N is the total number of all control points that need to be deformed, and i is the serial number of the control points that need to be deformed.

[0103] Those skilled in the art can understand that the above examples take the thin plate spline interpolation method as an example to illustrate deformation, and in other examples, the face composite model can also be deformed by using polynomial interpolation, radial basis function interpolation, Kriging interpolation, local interpolation method, machine learning-based method, etc., which will not be listed one by one here. The description of various methods is as follows:

[0104] Polynomial interpolation: Polynomial interpolation approximates given data points by fitting a polynomial function. Common polynomial interpolation methods include Lagrange interpolation and Newton interpolation, which are simple to calculate.

[0105] ​​​​Radial Basis Function Interpolation: This method uses radial basis functions to interpolate data. Common radial basis functions include Gaussian functions and multi-hole radial basis functions. RBF interpolation can flexibly handle irregularly distributed data and is effective for high-dimensional problems.

[0106] Kriging Interpolation: Kriging is a geostatistics-based interpolation method that estimates the value of unknown points by modeling spatial correlation. Kriging can effectively handle spatial correlation and variability.

[0107] Local Interpolation Methods: Local data is used to interpolate the grid, such as local weighted regression and nearest neighbor-based interpolation methods (such as K-Nearest Neighbor interpolation). These methods can provide good interpolation results in a local range.

[0108] Machine Learning-Based Methods: Machine learning techniques such as neural networks and support vector machines are applied to interpolation problems. These methods can interpolate by learning the complex nonlinear relationships between data, often requiring large amounts of training data and computational resources.

[0109] In addition to thin plate spline interpolation, other spline interpolation methods such as natural spline interpolation and Hermite spline interpolation can also be used. Spline interpolation can effectively approximate data points while maintaining smoothness.

[0110] In step 805, given the target position of the dental and jaw part in the face composite model, the new position of the grid vertex of the soft tissue part in the face composite model is obtained based on the variation relationship. In one embodiment, the given target position can be the second position obtained in step 803, or the target position of the dental and jaw feature point in state R when the face type variation in state R needs to be predicted. In one embodiment, the above target position can be used to represent the movement amount of the initial state position, such as marking the numerical value of the movement amount on the corresponding control point, or using the spatial coordinate point of the target position to represent, such as marking the corresponding position in the image.

[0111] In one embodiment, the coefficients obtained by solving step S2 can be substituted into the above expression (1-3) to perform thin plate spline (TPS) transformation on all pixel points of the profile image, and a new profile image is obtained. At this time, the prediction of the face profile after treatment is completed. The coefficients of the interpolation function calculated in step S2 are substituted into the interpolation function expression (1-3), and all pixel points on the profile are interpolated based on the target position of the state to be predicted. That is, all pixel points on the image are taken as control points, and finally the predicted point positions of all interpolated pixel points are calculated. The predicted points of all pixel points form a new profile image.

[0112] In some embodiments, the prediction method can further comprise: after obtaining the new positions of the mesh vertices of the dental arch part and the soft tissue part in the face composite model, establishing a face orthodontic prediction model. By establishing a three-dimensional face orthodontic prediction model through the new positions of the mesh vertices, it is convenient to provide for review, display, comparison, etc., so that users can get intuitive effects. In one example, when establishing the face orthodontic prediction model, the model can be constructed according to the topological structure in the original face composite model.

[0113] The above-mentioned embodiments are to predict the new positions of the model mesh vertices in a certain state, and in other embodiments, the changes of the model in multiple states can also be predicted. For example, the treatment plan includes multiple successive treatment steps, and each treatment step corresponds to a target position of the dental arch part; the target positions of the dental arch part in the given face composite model are repeatedly executed, and based on the change relationship, the new positions of the mesh vertices of the soft tissue part in the face composite model are obtained, and the steps of establishing a face orthodontic prediction model are established multiple face orthodontic prediction models; wherein each is established based on a corresponding one of the multiple successive treatment steps in the treatment plan. In the embodiments of the present application, the target positions of multiple treatment steps can be predicted respectively, so that multiple face orthodontic prediction models are established, so as to facilitate the user to compare the change process when providing the user subsequently.

[0114] In some embodiments, in establishing the face orthodontic prediction model, the texture of the face composite model can also be added to the established face orthodontic prediction model. The rendering of the face orthodontic prediction model after adding the texture is shown in FIG. 11. As can be seen from FIG. 11, the texture is added to the prediction model, so that the appearance of the model is closer to the real face effect, and in addition, the model can be updated based on the original texture, so that the model is more realistic. The original texture can be obtained from the face composite model in the first state obtained in step 801, or from the face mesh model in the first state obtained in step 802. It should be noted that the part of the eye blocked by the white box in FIG. 11 is a privacy protection treatment taken to protect the portrait right of the case, and is not a blank part in the generated model.

[0115] It can be seen that the face type prediction method in the embodiments of the present application obtains the composite model of the dental arch part and the soft tissue part of the associated face, determines the correlation of the dental arch part and the soft tissue part, and then determines the change relationship of the face type through the known change of the dental arch part in the treatment plan, so as to predict the position change of other parts through the change relationship. Since the position change can be predicted only through the data of the patient himself, without combining a large amount of historical data to speculate the change relationship, the amount of data involved in the operation in the prediction of the face type prediction method in the present application is small, and the accuracy is high.

[0116] The embodiments of the present application also relate to a face shape display method. Specifically, the face shape display method can be implemented according to any of the face shape prediction methods provided in the above embodiments. The face shape display method can obtain a face orthodontic prediction model, and then display the face orthodontic prediction model on a display device. The display device can be an image output device, such as a display, a tablet computer (PAD), a mobile phone screen, and the like.

[0117] The embodiments of the present application also relate to a method for generating an oral treatment plan. In some embodiments, the method for generating an oral treatment plan can include the following steps, as shown in FIG. 12.

[0118] In step 1201, a face composite model of a patient in an initial state is obtained. The face composite model includes a dental and jaw part and a soft tissue part.

[0119] In step 1202, P treatment plans of the patient are obtained respectively. Each treatment plan corresponds to a target state of dental and jaw movement.

[0120] In step 1203, the face composite model is deformed according to the dental and jaw movement of the P treatment plans, to obtain P face prediction models corresponding to the target states.

[0121] In step 1204, among the P treatment plans, an optimal treatment plan is selected as a target treatment plan according to an evaluation of the face prediction models of the target states.

[0122] The method for generating an oral treatment plan in the embodiments of the present application can provide a reference for a user in selecting a treatment plan, by predicting a face shape change in a target state of the treatment plan. Specifically, the face shape change in the target state of each of the treatment plans is predicted, and the obtained face prediction models are evaluated, so that a better treatment plan can be selected, and the generated treatment plan can meet the user's expectation, thereby improving the user experience.

[0123] It should be noted that the method for generating an oral treatment plan in the embodiments of the present application can be implemented by hardware or a combination of computer software and hardware. For hardware implementation, the method for generating an oral treatment plan can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic devices for implementing the functions of the method for generating an oral treatment plan, or a combination of the above.

[0124] In step 1201, the face composite model obtained in this step is a three-dimensional model including associated dental arch part and soft tissue part, wherein the dental arch part can only include the teeth of the upper and lower jaws, such as the dental arch model 20 shown in FIG. 4a, which is applied to cases of only tooth movement, and the data involved in subsequent analysis is simplified, or can include parts such as jaw bone and skull in addition to the teeth, which is applied to cases of tooth movement or jaw movement, and the data involved in subsequent analysis is complete. In some embodiments, the soft tissue part mainly includes the appearance part, i.e. the outer contour part of the soft tissue, such as the face mesh model 10 shown in FIG. 4b, which represents the soft tissue part. Although the above FIG. 4a and FIG. 4b are presented in black and white form, in some embodiments, color patterns can also be used for expression, which will not be listed one by one here.

[0125] As can be seen, step 1201 is substantially the same as step 801 in the foregoing embodiments, and the main difference is that the face composite model obtained in step 1201 corresponds to the initial state of the patient, while the face composite model obtained in step 801 corresponds to the first state of the patient, so that the patient states corresponding to the composite images, face mesh models, and dental arch models used in the face composite model are different, and the remaining parts are substantially the same. Therefore, the description of the corresponding features in step 1201 can refer to the related description of step 801 (i.e. the description in paragraphs

[0078] -

[0085] of the present description), which will not be repeated here.

[0126] It should be noted that the initial state is not limited in the embodiments of the present application. The initial state can be a certain state before orthodontic treatment of the patient, or a certain state during orthodontic treatment, and the specific corresponding stage can be selected according to the actual application scenario, which will not be listed one by one here.

[0127] In step 1202, P treatment plans of the patient are obtained, which can be obtained in advance or at the time of prediction. In one embodiment, P is a natural number greater than 1, such as 2, 5, 10, 40, etc., which will not be listed one by one here.

[0128] In one embodiment, the treatment plan can include a target state corresponding to dental arch movement, which can be presented in the form of text, table, two-dimensional image, three-dimensional model, etc. The dental arch movement can include tooth movement (orthodontic process) or jaw movement (orthognathic process), and the tooth movement can be presented in the form of single tooth movement or overall tooth movement. In one embodiment, the treatment plan can also include various plans during the treatment process, such as whether to extract teeth, whether to enamel, the block method of the jaw bone, etc., which will not be listed one by one here.

[0129] As for the acquisition of the treatment plan, some embodiments can be acquired by using the same tooth arrangement mode, and some embodiments can be acquired by using different tooth arrangement modes, wherein the tooth arrangement mode can select one of the following or any combination thereof: direct step-by-step method, space search method and intermediate interpolation method. The tooth arrangement mode has its own characteristics, and the different treatment plans obtained thereby are also different, so as to provide the user with more abundant treatment plans and improve the user's selection range. Further, even if the same tooth arrangement mode, when different tooth arrangement parameters are used, the final tooth arrangement result will still be different, thereby obtaining different treatment plans. In some embodiments, when orthodontic and orthognathic treatment is required in the treatment plan, the combination of the two can be different, such as orthodontic first, then orthognathic, orthognathic first, then orthodontic, orthodontic first, then orthognathic, orthodontic, then orthognathic, and so on, thereby obtaining different treatment plans.

[0130] In some embodiments, the P treatment plans are all acquired by using the same tooth arrangement mode, and in some embodiments, at least two of the P treatment plans are acquired by using different tooth arrangement modes, so as to further increase the richness of the plans.

[0131] As for the value of P, P can be a natural number greater than 1, such as 2, 3, 10, or even more. In existing orthodontic / orthognathic design, the treatment plan is rarely modified after being generated, although the patient may be communicated in the early stage of treatment plan generation, only some parameters are adjusted in the early stage, and after the parameters are determined, a treatment plan is designed accordingly, and the user has almost no choice. In the above embodiments, different tooth arrangement modes, different orthodontic methods, and different treatment parameters can be set to obtain different treatment plans. Especially for complex cases, there may be more treatment plans, which will not be listed one by one here.

[0132] In step 1203, the deformed face composite model is deformed according to the tooth and jaw movement of the P treatment plans, and the face prediction model corresponding to the P target states obtained is obtained.

[0133] As for the deformed face composite model, taking the face prediction model corresponding to the target state of a treatment plan as an example, the deformation process is shown in FIG. 13, which specifically includes:

[0134] Step 1301: Identify the preselected tooth and jaw feature points on the tooth and jaw part of the face composite model, denoted as the first position of the tooth and jaw feature points. As can be seen, step 1301 is roughly the same as step 802 of the foregoing embodiments, therefore, the description of the features of step 1301 can refer to the corresponding description of step 802 (i.e., the description in paragraph

[0087] of the present specification).

[0135] At step 1302, a second position of the dental arch feature point in the face composite model is obtained based on the dental arch movement from the initial state to the target state of the treatment plan. In some embodiments, the treatment plan can include the position of the teeth and jaw in the digital orthodontic design, such as the target state of the dental arch, i.e., the position of the dental arch in the digital design after the completion of orthodontics, or a plurality of intermediate states, i.e., the intermediate state of the dental arch when reaching the target state of the dental arch. In this embodiment, the target state of the dental arch is taken as the target state opposite to the initial state, and the second position of the dental arch feature point is the spatial position of each dental arch feature point identified when the dental arch is in the target state. The specific expression can be represented in the form of position coordinates or the distance difference (equivalent to the movement amount) between the corresponding feature points in the initial state. The representation form is various and will not be listed one by one here.

[0136] At step 1303, the relationship between the movement of the dental arch feature point and the change of the grid vertex in the face composite model is calculated by using the deformation method according to the first position and the second position of the dental arch feature point. It is not difficult to see that step 1303 is roughly the same as step 804 of the foregoing embodiments, and therefore, the description of the features related to step 1301 can refer to the corresponding description of step 804 (i.e., the description in paragraphs

[0089] -

[0109] of this specification).

[0137] At step 1304, the face prediction model corresponding to the treatment plan is obtained by deforming the face composite model based on the dental arch movement and the change relationship. The dental arch movement can be represented by the movement amount of the position in the initial state or by the spatial coordinate point of the target position. In one embodiment, the coefficients obtained by solving step S2 are substituted into the expression (1-3) provided in paragraphs

[0089] -

[0109] of this specification to perform interpolation transformation on all pixel points of the profile image, thereby obtaining a new profile image. At this time, the prediction of the profile of the face after the completion of the treatment is completed. The coefficients of the interpolation function calculated by step S2 in paragraphs

[0089] -

[0109] of this specification are substituted into the interpolation function expression (1-3) provided in paragraphs

[0089] -

[0109] of this specification, and all pixel points on the profile are subjected to interpolation transformation based on the target position of the state to be predicted, i.e., all pixel points on the image are taken as control points, and finally the predicted point position of all pixel points after interpolation is calculated, and the predicted points of all pixel points form the image of the new profile.

[0138] In some embodiments, a three-dimensional face prediction model is established by the new position of the grid vertex, for example, the new position of the grid vertex can be established as a three-dimensional face prediction model according to the topological relationship in the original face composite model, so as to provide for reference, display, comparison, etc., so that the user can obtain intuitive effects.

[0139] In some embodiments, the method further comprises: adding texture to the established face prediction model according to the texture feature of the face composite model. The texture added to the face prediction model is substantially the same as the texture added to the face composite model, and the details are described in paragraph

[0114] of the specification.

[0140] It can be seen that the steps 1301 to 1304 disclose the process of obtaining the face prediction model corresponding to a target state.

[0141] The above embodiments are for predicting the new position of the model mesh vertex in a certain state. In other embodiments, the treatment plan can include not only one treatment step, but also multiple treatment steps executed in sequence. Correspondingly, when obtaining the face prediction model, different face prediction models can be obtained for different states, so as to compare the differences between the face prediction models in different states. For example, if the treatment plan includes multiple treatment steps executed in sequence, each treatment step corresponds to a target position of the dental arch part; the steps of “given the target position of the dental arch part in the face composite model, obtaining the new position of the mesh vertex of the soft tissue part in the face composite model based on the change relationship, and establishing the face prediction model” are repeatedly executed, thereby establishing multiple face prediction models; each of which is established based on a corresponding one of the multiple treatment steps executed in sequence. It can be seen that predicting the target position of multiple treatment steps respectively can achieve the establishment of multiple face prediction models, so as to provide the user with the change process in the subsequent comparison.

[0142] In step 1204, among the P treatment plans, the optimal treatment plan is selected as the target treatment plan according to the evaluation of the face prediction model in the target state.

[0143] Regarding the evaluation of the face prediction model, some embodiments are based on the following parameters: face contour state. Since the face contour state is mainly determined by soft tissue, it is more in line with the user's demand for the appearance of the outer contour. In one embodiment, commonly used parameters for describing the face contour in medicine can be used, such as the smile curve. The smile curve is obtained through the frontal or lateral view of the face prediction model, and is composed of the incisal edge of the upper teeth from the teeth exposed by the upper and lower lips, combined with the lip line and the gum area exposed when smiling. After calculating the smile curve, the curvature of the smile curve, the distance from the upper and lower lip lines, and the like are determined. In one embodiment, the closer the various features of the smile curve to the ideal value, the higher the evaluation of the face prediction model, and vice versa. Since the calculation of the smile curve combines the position of the teeth with the shape of the soft tissue such as the lip line, the use of the composite model containing the associated dental and soft tissue parts in the present embodiment can realize the automatic collection and calculation of the smile curve. It can be understood that the smile curve is an objective evaluation parameter that takes into account the aesthetic factor, so it is convenient for the computer to automatically evaluate, and on the basis of providing a more beautiful treatment plan for the user, the degree of automation of the present embodiment is ensured.

[0144] In some embodiments, the evaluation of the face prediction model is determined according to the user's instruction. For example, the user can directly send an instruction including an evaluation score to determine the evaluation of each face prediction model by the score.

[0145] In some embodiments, the evaluation of the face prediction model for the target state is also combined with the following parameters: orthodontic step number parameter and / or tooth arrangement uniformity for reaching the target state. Since the orthodontic step number parameter and / or tooth arrangement uniformity will affect the treatment effect, the combination of the orthodontic step number and / or tooth arrangement uniformity can increase the objectivity of the evaluation, facilitate automatic evaluation, and provide auxiliary basis for the user. Regarding the orthodontic step number parameter, those skilled in the art know that the orthodontic step number will directly affect the orthodontic period, the economic expenditure of orthodontics, etc. In one embodiment, the user may expect a shorter orthodontic duration and less economic expenditure, so when the other effects of the face prediction model are similar, the orthodontic step number parameter can be combined, and it is set that the fewer the orthodontic step numbers, the higher the evaluation of the face prediction model. In one embodiment, the user may expect to pursue a more perfect tooth arrangement uniformity, so when the other effects of the face prediction model are similar, the tooth arrangement uniformity can be superimposed for evaluation, and it is set that the higher the tooth arrangement uniformity, the higher the evaluation of the face prediction model.

[0146] Regarding the tooth arrangement regularity, in some embodiments, the evaluation of the tooth arrangement regularity is based on a multi-objective comprehensive evaluation, wherein each objective is a medical element of orthodontics. Since there are multiple medical elements that can be evaluated for tooth arrangement regularity, multiple objectives can be selected as needed to participate in the evaluation of tooth arrangement regularity in order to obtain a comprehensive optimal tooth arrangement regularity treatment plan. In one embodiment, the medical elements of orthodontics are: arch curve, tooth crowding, overbite relationship, overjet relationship, arch protrusion, Spee curve, Bolton index, arch width, arch symmetry, tooth torsion, tooth axis inclination, or tooth torque. In one embodiment, a multi-objective optimization function can be used for calculation, each objective corresponds to a medical element, and the result of the comprehensive optimization of multiple medical parameters is used as the facial prediction model with the highest tooth arrangement regularity. In one embodiment, when evaluating tooth arrangement regularity, all of the above medical parameters can be selected, or some of the above medical parameters can be selected, such as selecting one parameter, such as arch curve, reducing the amount of data involved in the calculation based on accurate evaluation results, and improving calculation speed. It can be understood that tooth arrangement regularity, as a medical objective element involved in treatment plan design, can objectively evaluate the treatment plan. In one embodiment, the elements involved in evaluating tooth arrangement regularity can also be selected by the user. Since different patients may have different needs and feelings for regularity, selecting specific elements for evaluating tooth arrangement regularity by the user can improve the degree of individualization in treatment plan design, thereby providing a treatment plan that meets the user's needs.

[0147] In some embodiments, the above-mentioned parameters involved in the evaluation can be selected and combined for evaluation. Combining multiple parameters for evaluation can allow the user to consider multiple factors comprehensively, making the evaluation results more referential. In one embodiment, the user can determine the evaluation of the facial prediction model of the target state according to subjective needs using score instructions, selection instructions, etc. For example, the evaluation can be determined according to the score in the score instruction, or the facial prediction model selected in the selection instruction can be determined as the facial prediction model with the highest evaluation. In one embodiment, the user can also specify the parameters and medical elements involved in the evaluation, and the computer can calculate the evaluation score or evaluation level according to the predetermined formula or rule in order to determine the evaluation of the facial prediction model. In one embodiment, the user instruction and computer evaluation can also be combined, such as first giving a predetermined number of facial prediction models with higher evaluation according to computer evaluation, and then further selecting from the predetermined number of facial prediction models by the user in order to reduce the selection range of the user and facilitate the user to quickly select a facial orthodontic model with higher evaluation. It can be understood that different evaluation methods can adjust the influence of subjective judgment and objective evaluation of the user on the final evaluation result.

[0148] In one embodiment, if the number of parameters participating in the evaluation of the face prediction model is greater than 1, different weights are set for different parameters. Since the influence of multiple parameters on the treatment result is different, and different patients pay different attention to different parameters, the importance of the parameters participating in the evaluation can be adjusted by setting the weights to obtain an evaluation result that is more in line with the user's needs.

[0149] Regarding the selection of the face prediction model based on the evaluation result, in some embodiments, the number of parameters participating in the evaluation is one, and the optimal face prediction model is directly selected according to the evaluation result of the single parameter. In other embodiments, the number of parameters participating in the evaluation is multiple, and the optimal face prediction model is selected by comprehensively optimizing each parameter. The comprehensive optimization can be implemented in multiple ways. In one embodiment, the scores of different parameters can be evaluated, and the scores can be added at the end. It can be understood that the optimal face prediction model can be selected according to the way of adding the highest total score. The adding process can also set weight values for different parameters, and the calculation process after the weight values are added will not be described here. In another embodiment, the evaluation results of different parameters can be output, and the optimal face prediction model can be determined by the user in another way. In addition, other comprehensive ways can also be selected, which will not be listed one by one here.

[0150] After the evaluation method in any of the above embodiments is completed, the treatment scheme with the best evaluation result is selected as the target treatment scheme. In one embodiment, the target treatment scheme can be directly output to the user. Specifically, the face prediction model can be displayed to the user through a display device, or the target treatment scheme can be displayed in the form of text, images, tables, etc., which will not be listed one by one here.

[0151] It can be seen that the method for generating an oral treatment plan in the embodiments of the present application provides a reference basis for the user in the selection of a treatment plan by predicting the change in the facial form of the user in the target state of the treatment plan. Specifically, the change in the facial form is predicted through the target state of each treatment plan, and then the obtained facial prediction model is evaluated so as to select a better treatment plan from among them, so that the generated treatment plan is more in line with the expectations of the user, and the user experience is improved. In the method, the change relationship of the facial form is determined through the known change of the dental arch part in the treatment plan, and the position change of other parts is predicted based on the change relationship. Since the change relationship can be predicted only by using the data of the patient himself, without combining a large amount of historical data, the amount of data involved in the prediction in the facial form prediction method in the present application is small, and the accuracy is high. In addition, in the evaluation of the facial prediction model, multiple evaluation methods can be combined to avoid the problem that only a single medical standard is used to evaluate the treatment plan design without considering the personalized needs of the user, so as to provide the user with a more comprehensive evaluation method, improve the consideration of the aesthetic factor of the user in the treatment plan design, and better meet the actual needs of the user for oral treatment.

[0152] The embodiments of the present application also relate to another method for generating an oral treatment plan. The difference between the method for generating an oral treatment plan and the method for generating an oral treatment plan provided in the foregoing embodiments is that, in the foregoing embodiments, the facial prediction model in the treatment plan is evaluated only by obtaining the facial prediction model in the target state of the treatment plan, while in the method for generating an oral treatment plan, in addition to considering the target state, the change in the facial form of the user in the stage state of the treatment plan is also predicted, and the change in the facial form of the user in the stage state is predicted in addition to the change in the facial form of the user in the target state, and the target state and the intermediate stage state are combined in the evaluation to comprehensively evaluate the target treatment plan, so that the generated target treatment plan is more in line with the comprehensive and diverse expectations of the user.

[0153] In some embodiments, the treatment plan further includes a plurality of stage states through which the dental arch moves from the initial state to the target state. In particular, in the orthodontic stage, the treatment plan needs to go through a plurality of intermediate stage states, some of which may adjust the force of the orthodontic device, and some of which may change the type of the orthodontic device used. If the treatment plan includes an orthognathic stage, the orthognathic stage involves the segmentation and movement of bone blocks, which can be realized by orthognathic surgery, and can be expressed by a stage state in the treatment plan. However, the combination of the orthognathic stage and the orthodontic stage still has different combination sequences, such as orthodontic first, orthognathic first, or orthodontic first and orthognathic second and then orthodontic again. When the orthognathic stage is in the last stage of the entire treatment plan, the target of the orthognathic stage is the target state of the treatment plan. When the orthognathic stage is not in the last stage of the entire treatment plan, the target state of the orthognathic stage is an intermediate stage state.

[0154] The method for generating an oral correction scheme in some embodiments can further include obtaining different human face prediction models corresponding to different stage states. It can be understood that the obtaining of each human face prediction model is similar to the obtaining of the human face prediction model for the target state, and different human face prediction models can be obtained according to different dental arch movements corresponding to different stages. Therefore, the process of obtaining the human face prediction model for the stage state will not be described again. Correspondingly, in the process of selecting the optimal correction scheme as the target correction scheme according to the evaluation of the human face prediction model for the target state in the P correction schemes, the process can include: in the P correction schemes, selecting the optimal correction scheme as the target correction scheme according to the evaluation of the human face prediction model for the target state and the evaluation of the human face prediction model for each stage state.

[0155] In some embodiments, the evaluation of the human face prediction model for each stage state can use the same evaluation parameters as the evaluation of the human face prediction model for the target state in the related embodiments described above, or different parameters can be used for evaluation. In this case, the user can specify the parameters that are more concerned and remove the parameters that are not concerned, and the selection of the evaluation method is flexible and variable, and will not be listed one by one here.

[0156] In some embodiments, the selection of the optimal correction scheme as the target correction scheme according to the evaluation of the human face prediction model for the target state and the evaluation of the human face prediction model for each stage state includes: selecting the optimal correction scheme as the target correction scheme according to the evaluation of the human face prediction model for the target state and the evaluation of the human face prediction model for some stage states. Since the invisible orthodontic process is divided into multiple steps, there are many intermediate stage states, and only some stage states can be selected for evaluation to reduce the amount of data processing while obtaining accurate evaluation results. It can be understood that the number of human face prediction models participating in the evaluation can be limited, such as selecting all the human face prediction models of the intermediate stage states as the objects participating in the evaluation, or selecting only some human face prediction models of the stage states for evaluation. Further, when some stages are selected for evaluation, some continuous stages can be selected, such as selecting continuous stages close to the target state, or some stages can be selected at intervals, such as selecting some stages evenly spaced from the initial state to the target state, or selecting some stages close to the target state at intervals, so as to reduce the amount of data participating in the evaluation while maintaining sufficient accuracy of the evaluation. At the same time, the difference between the two adjacent stages participating in the evaluation is increased to improve the evaluation efficiency.

[0157] In some embodiments, when evaluating the face prediction model of the target state and the face prediction model of the multiple stage states comprehensively, a multi-objective optimization method can be used, and the evaluation results of the face prediction model of each state participating in the evaluation in each treatment plan are taken as one of the objectives, and the most optimal treatment plan is calculated. In one embodiment, each target can also set a weight value, and the influence of different targets participating in the evaluation can be adjusted as needed.

[0158] Regarding the acquisition of the treatment plan, in some embodiments, at least two treatment plans are acquired by using an intermediate interpolation method. For example, the intermediate interpolation method can refer to the digital tooth arrangement method disclosed in application No. CN202110746416.8, so as to accurately arrange the teeth on the target arch curve in the target position, and the result objectively meets the medical indicators and aesthetic requirements. In some examples, the initial state of the patient's teeth before treatment can be acquired, and the final target state can be determined by the doctor according to the initial tooth state. With the help of computer algorithms, interpolation is performed in the middle from the initial state to the target state. Among them, the target state can be the design target state of the end of orthodontics, or it can be a stage target state in the treatment process, that is, during the entire orthodontic stage, the teeth can first pass through a plurality of stage target states in sequence from the initial state, and finally reach the designed orthodontic target state, so that the teeth can be smoothly moved from the initial state to the target state. For the change of adjacent stage target states, the teeth are also moved from the previous stage target state to the next stage target state by interpolation calculation. Those skilled in the art can understand that although the design target state (such as the target position of the teeth) is the same, the number of intermediate interpolations and the interpolation positions can still be adjusted according to different needs, so that even if different treatment plans all use the intermediate interpolation method to arrange the teeth, the obtained tooth arrangement plans are still different. Different intermediate stage states can cause different tooth arrangement degrees and facial morphologies during the treatment process. For doctors, it is enough to reach the target state involved, but for patients, the entire treatment cycle needs to be considered for the appearance. Therefore, although oral treatment is mainly to pursue the aesthetics of the final target form for patients, the long treatment process cannot be ignored in the design of the treatment plan. If two treatment plans R1 and R2 have similar final design target states, but the intermediate stage states differ greatly, and the facial appearance of the intermediate state of the treatment plan R1 is obviously better than that of the treatment plan R2, in order to allow the patient to maintain an aesthetic appearance during the treatment process, the preferred treatment plan A will better meet the actual needs of the patient.

[0159] In addition, regarding the acquisition of the treatment scheme, in some embodiments, the direct step-by-step method can be used for acquisition, and details can be found in the disclosed patent with the application number CN201410831582.8 for producing a direct step-by-step method for generating a dental treatment state. After the dental model segmented into single teeth is acquired, the target parameters involved in the tooth arrangement are determined, such as using multiple medical factors as the target parameters, and the medical factors are, for example, dental arch curve, tooth arrangement crowding degree, interdental enamel reduction amount, overbite, overjet, dental arch protrusion, Spee curve degree, Bolton index, dental arch width, dental arch symmetry, tooth torsion degree, tooth axis inclination, tooth torque, and tooth center line. Then, the tooth arrangement scheme with fewer steps is obtained by calculation through the multi-objective optimization function. That is, the selection of the target parameters and the weight of the corresponding target parameters can be adjusted according to the needs, and the treatment scheme of different target states and different stage states can be obtained.

[0160] In addition, regarding the acquisition of the treatment scheme, in some embodiments, the spatial search method can be used for acquisition, and details can be found in the method for obtaining a target dental treatment state in the disclosed patent with the application number CN201410006532.6.

[0161] It is worth mentioning that the above-mentioned embodiments obtain different treatment schemes through different tooth arrangement methods, and in some embodiments, different treatment schemes can be obtained by combining the above-mentioned different tooth arrangement methods. For example, in one embodiment, the three treatment schemes to be selected can be obtained by the direct step-by-step method, the spatial search method, and the intermediate interpolation method, respectively. For another example, in one embodiment, the five treatment schemes to be selected can be obtained by the direct step-by-step method, and the other one can be obtained by the spatial search method. For another example, in one embodiment, the five treatment schemes to be selected can be obtained by the direct step-by-step method, the spatial search method, and the intermediate interpolation method, respectively. Those skilled in the art can understand that the selection of various tooth arrangement methods can be selected according to the needs, and one tooth arrangement method can be selected in one embodiment, or multiple tooth arrangement methods can be selected in one embodiment, and the selection method is flexible and variable, which will not be listed one by one here.

[0162] It can be seen that when selecting multiple treatment schemes in the embodiments of the present application, in addition to the face prediction model evaluating the target state of each treatment scheme, the face prediction model evaluating the stage state in these treatment schemes is also combined, and the intermediate process other than the target state is combined to comprehensively consider the whole orthodontic process during evaluation. Since the oral treatment lasts for a long time, it is considered that the patient not only needs the optimal final target state, but also needs the optimal intermediate stage state, so the target state and the stage state are evaluated respectively, and the most optimal treatment scheme is selected, which can improve the user's satisfaction with the intermediate stage state, better meet the actual needs of the user, and increase the user's willingness to wear.

[0163] The embodiment of the present application also relates to a method for generating an oral treatment plan, which is basically the same as the method for generating an oral treatment plan provided by the foregoing embodiment, with the main difference being that the method for generating an oral treatment plan provided by the foregoing embodiment selects from a plurality of treatment plans, while the method for generating an oral treatment plan is based on a treatment plan, and different stages are evaluated to select a more suitable target state, so that the generated treatment plan is more in line with the needs of the user.

[0164] In some embodiments, the method for generating an oral treatment plan is as shown in FIG. 14, and specifically as follows:

[0165] Step 1401: Obtain a face composite model of the initial state of the patient, and the face composite model includes a dental arch part and a soft tissue part. This step is similar to step 1201 in the foregoing embodiment, and thus is not described again here to avoid repetition.

[0166] Step 1402: Obtain a treatment plan for the patient, and the treatment plan includes N sequentially executed treatment steps for realizing the dental arch movement of the patient from the initial state to the target state, wherein each treatment step corresponds to a stage state.

[0167] Step 1403: Deform the face composite model of the initial state according to the initial state and the stage state of a treatment step to obtain a face prediction model of the treatment step. Regarding the face prediction model of the stage state of a treatment step as an example in this step, the deformation process is as shown in FIG. 13 and includes:

[0168] Step 1301: Identify preselected dental arch feature points on the dental arch part of the face composite model to obtain the first positions of the dental arch feature points.

[0169] Step 1302: Obtain the second positions of the dental arch feature points in the face composite model based on the dental arch movement between the initial state and the stage state.

[0170] Step 1303: According to the first positions and the second positions of the dental arch feature points, calculate the movement of the dental arch feature points and the change relationship of the grid vertices in the face composite model by using a deformation method. In some embodiments, the calculation is obtained by using an interpolation deformation function. In some embodiments, the interpolation deformation function is a thin-plate spline interpolation function. In this embodiment, the thin-plate spline interpolation function is limited, and a Gaussian kernel function is introduced to participate in the calculation, so that the deformation is more in line with the deformation law of the soft tissue, the change is greater for the teeth closer to the movement, and the change is smaller for the teeth farther from the movement, so that the prediction result is more accurate. The deformation process in this step has been described in the foregoing embodiment, and thus is not described again here to avoid repetition.

[0171] At step 1304, the face composite model is deformed based on the dental arch movement and the change relationship to obtain a face prediction model corresponding to the treatment step. That is, based on the dental arch movement between the initial state and the stage state, the face composite model obtained in step 603 is combined to deform the face composite model obtained in step 1401 to obtain the face prediction model corresponding to the stage state.

[0172] It can be understood that the face prediction model for one state is obtained through the above steps 1301 to 1304. In specific applications, the state is the final target state or an intermediate stage state, and the deformation calculation process is substantially the same. The relevant description has been described in the foregoing embodiments, and thus will not be described here again to avoid repetition.

[0173] At step 1404, M face prediction models are obtained according to the stage states of the M treatment steps. That is, the M treatment steps are selected from the N sequentially executed treatment steps. In one embodiment, the M treatment steps are selected from the kth treatment step to the Nth treatment step, and k, N, and M are natural numbers greater than or equal to 1, and k and M are less than or equal to N. The process of selecting M from the kth treatment step to the Nth treatment step is described as follows: In some embodiments, k can be set as N-M: In one embodiment, N is 40, and k is 15. M is calculated as 40-15=25, that is, when selecting the treatment steps, 25 treatment steps after the 15th treatment step are selected from the 40 treatment steps. In one embodiment, k can be greater than or equal to N / 2. In one embodiment, N is 40, and k is 25. M is calculated as 40-25=15, that is, when selecting the treatment steps, 15 treatment steps after the 25th treatment step are selected from the 40 treatment steps. In another embodiment, k can be set as N-M: In one embodiment, N is 40, and k is 15. When selecting the treatment steps, 10 or 15 treatment steps can be selected from the 15th treatment step to the 40th treatment step in the 40 treatment steps. It can be understood that in this way, all treatment steps in the stage can be selected to reduce the amount of data participating in the evaluation, so as to speed up the subsequent evaluation speed.

[0174] It can be understood that step 1404 repeatedly executes step 1403 multiple times (i.e., the execution process of step 1404 is equivalent to repeatedly executing steps 1301-1304 multiple times), and the number of repetitions is determined according to the M face prediction models to be obtained.

[0175] Step 1405, in the M stage states, according to the evaluation of each face prediction model, the optimal stage state is selected as the treatment target.

[0176] Regarding the evaluation of the face prediction model in the M stage states, similar to the evaluation process of the face prediction model in the target state in step 1204 in the foregoing embodiments, different parameters can be selected to evaluate the face shape state in the current state, and in some embodiments, the evaluation can also be combined with the tooth arrangement regularity and / or the orthodontic step number parameter to reach the current state. The selection method and evaluation method of the parameters are similar, and to avoid repetition, they will not be described one by one here.

[0177] Step 1406, updating the treatment plan according to the treatment target.

[0178] In some embodiments, the updating method can directly delete the treatment steps after the optimal stage state selected in step 1405, which is equivalent to taking the optimal stage state as the design target state of the treatment plan. In some embodiments, the optimal stage state can be taken as the design target state, and the intermediate interpolation method or other methods can be used to design the teeth again in order to obtain a more suitable treatment plan to replace the original treatment plan. Those skilled in the art can understand that in addition to the above updating methods, other updating methods can also be used, which will not be listed one by one here.

[0179] It can be seen that in the embodiments of the present application, the face shape of each stage state is predicted to obtain different face prediction models, and then each model is evaluated to select the stage state with the optimal evaluation as the new treatment target. In addition, since different users have different actual needs for oral treatment, the corresponding time and money costs are also different, so whether to use the final target in the original design as the target of treatment completion can be adjusted according to the actual needs of the user. For example, in combination with the treatment steps, the face shape state is similar at 20 steps before the final design target, and the alignment degree of the dentition may also be similar. Therefore, if the user wants to save economic and time costs as much as possible, the treatment plan can be ended in advance, but if the user wants the face shape to present a more perfect state, the treatment can be continued to achieve the most perfect face. It can be seen that in the embodiments of the present application, the design target of the treatment is comprehensively evaluated by evaluation, so that the user can comprehensively consider the time, money cost and final orthodontic effect, and improve the flexibility of the user to select the scheme. Further, in the embodiments of the present application, the face composite model including the related jaw part and soft tissue part of the face is obtained, the relationship between the jaw part and the soft tissue part is determined, and then the change relationship of the face shape is determined through the known change of the jaw part in the treatment plan, so as to predict the position change of other parts through the change relationship. Since the change relationship can be predicted only through the data of the patient himself, it is not necessary to combine a large amount of historical data to predict the change relationship, so the amount of data involved in the operation in the prediction in the face shape prediction method in the embodiments of the present application is small, and the accuracy is high. In addition, the target positions of multiple treatment steps can be predicted respectively, so that multiple face prediction models are established, which is convenient for the user to compare the change process when provided to the user subsequently. In addition, in the embodiments of the present application, when predicting a face prediction model, the position change of the jaw feature point is known, the movement of the feature point and the position change relationship of the network vertex in the model are calculated through deformation, so as to obtain a face prediction model more consistent with the actual state after treatment.

[0180] It should be noted that the above method for generating an oral treatment plan is to solve the related technical problems, and the design process of the oral treatment plan is studied in depth. It is proposed to evaluate the generated treatment plan, and the specific evaluation elements are not limited to the commonly used medical evaluation elements, but are combined with the actual influence on the user's appearance, so that the generated oral treatment plan can better improve the user's appearance after oral treatment, and better meet the actual needs of the user, and improve the user's satisfaction with the result of oral treatment. In this process, different treatment plans can be combined and optimized by using multi-dimensional evaluation, so as to more flexibly obtain a treatment plan that meets the needs of the user.

[0181] Some of the above embodiments can predict the appearance of the face corresponding to the target state of different orthodontic schemes according to multiple orthodontic schemes, and obtain a face prediction model through computer algorithm, to evaluate different face prediction models in multiple dimensions. The evaluation process can not only automatically evaluate multiple parameters (such as the alignment degree of the teeth, the number of orthodontic steps, etc.), but also can combine the actual needs of the user (smile aesthetics, appearance aesthetics, etc.), increase the influence of user decision in evaluation, so as to achieve a treatment scheme that is more in line with the individual needs of the user. Some other embodiments also consider the appearance state of the user during the treatment process, in addition to considering the target state, combined with multiple stage states of each orthodontic scheme, so as to have a more comprehensive evaluation of each orthodontic scheme. Some embodiments can also use the appearance prediction method to evaluate a single orthodontic scheme in multiple stages, combine the target state and the stage state of the treatment, and optimize the treatment scheme in a way that is more in line with the needs of the user. When a state that is more in line with the actual needs of the user is found in the middle stage, the treatment process can be ended in advance, saving time, money and effort for the user, and improving the user's satisfaction with the treatment scheme.

[0182] In some embodiments, the evaluation of the face prediction model can include the evaluation of the face shape state, such as the smile curve, and in some embodiments, the evaluation can also be combined with the degree of alignment of the teeth and / or the number of orthodontic steps to achieve the target state, to comprehensively consider the outer contour of the face and the dental arrangement in the mouth.

[0183] In some embodiments, when the optimal stage state of a treatment scheme is selected as the treatment target, the evaluation of the face prediction model of the stage state can be based on the evaluation of the face prediction model of the target state. The evaluation of the face prediction model of each stage state is similar to the evaluation of the face prediction model of the target state, and different parameters can be selected to evaluate the face shape state at that time, and / or the degree of alignment of the teeth and / or the number of orthodontic steps to achieve the state at that time.

[0184] In some embodiments, when the optimal treatment scheme is selected, the above two methods can be combined, and in addition to selecting the face prediction model of the final target state for evaluation, the face prediction model of the stage state of each treatment scheme can also be selected for combined evaluation.

[0185] In some embodiments, the above evaluation of the face prediction model can select one parameter for evaluation, or multiple parameters for evaluation at the same time, and the evaluation can be performed automatically by the computer, or by the user through instructions, or by the combination of the computer and the user, with flexible forms.

[0186] The above embodiments specifically describe implementation details of the method for generating an oral correction scheme, which are provided for the convenience of understanding and are not mandatory for the implementation.

[0187] It should be noted that the above examples in the embodiments of the present application are provided for illustration and do not limit the technical solutions of the present application.

[0188] The division of steps of the above methods is only for the purpose of clear description, and in actual implementation, one step can be combined or some steps can be split and decomposed into multiple steps, as long as the same logical relationship is included, which is within the protection scope of the present patent; adding insignificant modifications or introducing insignificant designs in the algorithm or flow, but not changing the core design of the algorithm and flow, are within the protection scope of the present patent.

[0189] The embodiments of the present application also relate to an electronic device, which, in some embodiments, includes at least one processor 1501, and a memory 1502 connected with the at least one processor 1501, as shown in FIG. 15; the memory 1502 stores instructions executable by the at least one processor 1501, and the instructions are executed by the at least one processor 1501 to enable the at least one processor 1501 to perform the above method embodiments.

[0190] The memory and the processor are connected in a bus mode, the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers and power management circuits, which are well known in the art, and therefore, they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on the transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0191] The processor is responsible for managing the bus and general processing, and can also provide various functions including timing, peripheral interface, voltage regulation, power management and other control functions. The memory can be used to store data used by the processor in performing operations.

[0192] The embodiments of the present application also relate to a computer readable storage medium storing a computer program. The computer program is executed by the processor to implement the above method embodiments.

[0193] That is, a person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a program stored in a storage medium, including a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0194] A person of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for implementing the present application, and in actual applications, various changes can be made in form and details without departing from the spirit and scope of the present application.

Claims

1. A method for predicting a face type, comprising: obtaining a face composite model of a patient in a first state, the face composite model comprising a dental and maxillofacial part and a soft tissue part; identifying preselected dental and maxillofacial feature points on the dental and maxillofacial part of the face composite model, and obtaining first positions of the dental and maxillofacial feature points; obtaining second positions of the dental and maxillofacial feature points of the patient in a second state in the face composite model based on a treatment plan; calculating a relationship between movement of the dental and maxillofacial feature points and changes of grid vertices in the face composite model by using a morphing method according to the first positions and the second positions of the dental and maxillofacial feature points; and obtaining new positions of the grid vertices of the soft tissue part in the face composite model based on the relationship and a target position of the dental and maxillofacial part in the face composite model.

2. The face shape prediction method of claim 1, wherein, The obtaining of the face composite model of the patient in the first state comprises: obtaining a registration image, wherein the registration image is a composite image formed by fusing a patient's lateral cephalogram and a profile photo, or is a patient's lateral cephalogram; obtaining a face grid model and a dental and maxillofacial model respectively, and fusing the face grid model and the dental and maxillofacial model based on the registration image to form the face composite model.

3. The face shape prediction method of claim 2, wherein, The fusing of the lateral cephalogram and the profile photo of the patient to form the composite image comprises: identifying preselected face contour feature points on the obtained lateral cephalogram and profile photo of the patient; fusing and registering based on positions of the face contour feature points to form the composite image.

4. The face shape prediction method of claim 3, wherein, The preselected face contour feature points comprise a tip of a nose and convex points of upper and lower lips.

5. The face shape prediction method of claim 3, wherein, The fusing and registering of the face grid model and the dental and maxillofacial model based on the registration image to form the face composite model comprises: identifying the preselected face contour feature points on the face grid model; identifying dental and maxillofacial feature points on the dental and maxillofacial model; fusing and registering the face grid model and the dental and maxillofacial model based on a relationship between the preselected dental and maxillofacial feature points and the preselected face contour feature points and the registration image to form the face composite model.

6. The face shape prediction method according to any one of claims 2-5, wherein, The face grid model is obtained by: splicing and fusing based on multi-angle face photos of the patient; or scanning the face of the patient.

7. The face shape prediction method of claim 6, wherein, The splicing and fusing based on multi-angle face photos of the patient comprises: removing backgrounds of the multi-angle face photos; performing a chessboard format processing on the multi-angle face photos to realize calibration and stereoscopic correction, and obtaining camera intrinsic parameters; calculating depths and three-dimensional coordinates of the face in each face photo by using parallax information of the multi-angle face photos and the camera intrinsic parameters; selecting two face photos taken at different angles as a group for registration, and registering at least two different face photo groups; splicing and fusing the registered face photo groups to form a three-dimensional point cloud model; processing the three-dimensional point cloud model into the face grid model.

8. The face shape prediction method according to any one of claims 2-7, wherein, Before the formation of the face composite model, texture features of a face type model are scanned and obtained. After the formation of the face composite model, the texture features are added to the face composite model.

9. The face shape prediction method according to any one of claims 2-8, wherein, The method further comprises, after the forming of the face composite model: displaying the face composite model on a display device.

10. The face shape prediction method according to any one of claims 1-9, wherein, The deformation method comprises: using an interpolation deformation function, polynomial interpolation, radial basis function interpolation, Kriging interpolation, local interpolation method, and machine learning method to obtain the deformation.

11. The face shape prediction method of claim 10, wherein, The interpolation deformation function is a thin-plate spline interpolation function.

12. The face shape prediction method according to any one of claims 1-11, wherein, The dental arch features include occlusal contact points of the upper first molar and the lower first molar, and occlusal contact points of the upper central incisor and the lower central incisor.

13. The face shape prediction method according to any one of claims 1-12, wherein, The prediction method further comprises, after obtaining the new positions of the mesh vertices of the dental arch part and the soft tissue part in the face composite model: establishing a face orthodontic prediction model.

14. The face shape prediction method of claim 13, wherein, The treatment scheme comprises a plurality of sequentially executed treatment steps, and each treatment step corresponds to a target position of the dental arch part. The method further comprises: repeatedly executing the steps of giving the target position of the dental arch part in the face composite model, obtaining the new positions of the mesh vertices of the soft tissue part in the face composite model based on the change relationship, and establishing a face orthodontic prediction model, to establish a plurality of face orthodontic prediction models, wherein each face orthodontic prediction model is established based on a corresponding one of the plurality of sequentially executed treatment steps in the treatment scheme.

15. The face shape prediction method according to claim 13 or 14, wherein The establishment of the face orthodontic prediction model comprises: adding texture to the established face orthodontic prediction model according to the texture features of the face composite model.

16. The face shape prediction method of any one of claims 13-15, wherein, The method further comprises, after the establishment of the plurality of face orthodontic prediction models: displaying the face orthodontic prediction models on a display device.

17. A method for generating an oral treatment scheme, comprising: obtaining a face composite model of a patient in an initial state, the face composite model comprising: a dental arch part and a soft tissue part; obtaining P treatment schemes of the patient respectively, each treatment scheme corresponding to a target state of dental arch movement, P being a natural number greater than 1; deforming the face composite model according to the dental arch movement of the P treatment schemes respectively, to obtain P face prediction models corresponding to the target states; in the P treatment schemes, selecting an optimal treatment scheme as a target treatment scheme according to an evaluation of the face prediction models of the target states; wherein the P face prediction models of the target states are obtained by the face profile prediction method of any one of claims 1-16.

18. The method for generating an oral treatment plan of claim 17, wherein, The treatment scheme further comprises a plurality of stage states through which the dental arch moves from the initial state to the target state sequentially; the method further comprises: obtaining different face prediction models corresponding to different stage states; the selecting of the optimal treatment scheme as the target treatment scheme according to the evaluation of the face prediction models of the target states in the P treatment schemes comprises: in the P treatment schemes, selecting an optimal treatment scheme as a target treatment scheme according to an evaluation of the face prediction models of the target states and an evaluation of the face prediction models of the stage states.

19. The method for generating an oral treatment plan of claim 17 or 18, wherein, The evaluation of the face prediction models of the target states is based on the following parameters: face appearance state.

20. The method for generating an oral remediation regimen of any of claims 17-19, wherein, The evaluation of the face prediction models is determined according to a user's instruction.

21. The method for generating an oral remediation regimen of any of claims 17-20, wherein, The evaluation of the face prediction model of the target state is also combined with the following parameters: orthodontic step number parameters and / or tooth arrangement uniformity parameters for reaching the target state.

22. The method for generating an oral remediation regimen of any of claims 17-21, wherein, If the number of parameters participating in the evaluation of the face prediction model is greater than 1, different weights are set for different parameters.

23. The method for generating an oral treatment plan of claim 22, wherein, If the face shape state is included in the multiple parameters participating in the evaluation of the face prediction model, the weight of the face shape state is higher.

24. The method for generating an oral treatment plan of claim 21, wherein, The evaluation of the tooth arrangement uniformity is based on the comprehensive evaluation of multiple targets, wherein each target is a medical element of orthodontics.

25. The method for generating an oral treatment plan of claim 24, wherein, The medical element of orthodontics is: dental arch curve, tooth arrangement crowding, overbite relationship, overjet relationship, dental arch protrusion, Spee curve, Bolton index, dental arch width, dental arch symmetry, tooth torsion, tooth axis inclination or tooth torque.

26. The method for generating an oral remediation regimen of any of claims 17-25, wherein, In the P orthodontic schemes, any one of the orthodontic schemes is obtained according to one of the following tooth arrangement methods or any combination thereof: direct step-by-step method, space search method and intermediate interpolation method.

27. The method for generating an oral remediation regimen of any of claims 17-26, wherein, In the P orthodontic schemes, any two orthodontic schemes are obtained using different tooth arrangement methods.

28. The method for generating an oral remediation regimen of any of claims 17-26, wherein, At least two of the P orthodontic schemes are arranged according to the intermediate interpolation method.

29. The method for generating an oral remediation regimen of any of claims 17-28, wherein, In the step of deforming the face composite model according to the tooth movement of the orthodontic scheme, the corresponding face prediction model of the target state is obtained, comprising: Identifying preselected dental arch feature points on the dental arch part of the face composite model, denoted as the first position of the dental arch feature points; Based on the tooth movement from the initial state to the target state of the orthodontic scheme, the second position of the dental arch feature points in the face composite model is obtained; According to the first position and the second position of the dental arch feature points, a deformation method is used to calculate the movement of the dental arch feature points and the change relationship of the grid vertices in the face composite model; Based on the tooth movement and the change relationship, the face composite model is deformed to obtain the face prediction model corresponding to the orthodontic scheme.

30. A method for generating an oral treatment plan, wherein, Comprising: Obtaining the face composite model of the patient in the initial state, the face composite model comprising: dental arch part and soft tissue part; Obtaining the orthodontic scheme of the patient, the orthodontic scheme comprising N orthodontic steps executed in sequence for realizing the tooth movement of the patient from the initial state to the target state, wherein each orthodontic step corresponds to a stage state; Deforming the face composite model in the initial state according to the initial state and the stage state of a orthodontic step to obtain the face prediction model of the orthodontic step; According to the stage state of M orthodontic steps, M face prediction models are obtained; wherein the M orthodontic steps are selected from the kth orthodontic step to the Nth orthodontic step, k, N and M are natural numbers greater than or equal to 1, and k and M are less than or equal to N; In the M stage states, the optimal stage state is selected as the orthodontic target according to the evaluation of each face prediction model; Updating the orthodontic scheme according to the orthodontic target; The M face prediction models are obtained by the face shape prediction method according to any one of claims 1-16.

31. The method for generating an oral treatment plan of claim 30, wherein, The face composite model is deformed according to the initial state and a stage state of a treatment step to obtain a face prediction model corresponding to the treatment step, and the method comprises: identifying preselected dental arch feature points on a dental arch part of the face composite model, denoted as first positions of the dental arch feature points; obtaining second positions of the dental arch feature points in the face composite model based on dental arch movement between the initial state and the stage state; calculating a change relationship between movement of the dental arch feature points and changes of grid vertices in the face composite model according to the first positions and the second positions of the dental arch feature points by using a deformation method; deforming the face composite model based on dental arch movement of the stage state and the change relationship to obtain a face prediction model corresponding to the treatment step.

32. An electronic device, comprising: comprise: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the face type prediction method according to any one of claims 1-16, or perform the method for generating an orthodontic treatment plan according to any one of claims 17-29, or perform the method for generating an orthodontic treatment plan according to claim 30 or 31.

33. A computer readable storage medium storing a computer program, wherein the computer program comprises instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1-32. The computer program is executed by the processor to implement the face type prediction method according to any one of claims 1-16, or implement the method for generating an orthodontic treatment plan according to any one of claims 17-29, or implement the method for generating an orthodontic treatment plan according to claim 30 or 31.

Images