Mandibular movement modeling method, electronic equipment and storable medium

Abstract

The embodiment of the invention relates to the technical field of medical instruments, and discloses a mandibular movement modeling method, electronic equipment and a storable storage.The mandibular movement modeling method comprises the steps that image data pictures of the mandible of a patient at different positions are received, and a three-dimensional maxillary model and a mandibular model are digitized; the image data picture at least can display a part of maxillary dentition and a part of mandibular dentition; according to the relative position of the maxillary dentition and the mandibular dentition in the image data picture, determining the spatial position and posture of the digital three-dimensional mandibular model relative to the maxillary model; wherein the image data pictures of different positions of the mandible correspondingly determine the spatial position and posture of one mandible model; and analyzing each spatial position and posture of the determined mandibular model to obtain a movement track of the mandibular model relative to the maxillary model, thereby solving the problem of automatically establishing the mandibular dynamic movement model conforming to the actual condition of the patient.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of medical device technology, and in particular to a mandibular movement modeling method, electronic device, and storage medium. Background Art

[0002] Digital orthodontics is a type of technology that combines modern stomatology, computer-aided 3D diagnosis, personalized design, and digital prototyping technology. It collects 3D image data of soft and hard tissues on the patient's jaws, constructs a digital jaw model, and simulates the movement of teeth and gums based on the digital jaw model to design orthodontic appliances. In order to better design orthodontic appliances, it is usually necessary to accurately distinguish the crowns and gums of each tooth on the digital jaw model in order to more accurately simulate the movement of teeth and gums.

[0003] In digital orthodontic design, the position of the mandible is very important. The accuracy of its position is directly related to the accuracy of the design results. The temporomandibular joint connects the mandible and the temporal bone of the skull. The mandibular motion model is very complex and the motion trajectory is nonlinear. For example, mandibular motion includes opening the mouth, lateral jaw movement, and forward and backward telescopic movement. These movements are not simple rotational or translational movements around the temporomandibular joint point. It is a complex nonlinear movement. How to establish a mandibular motion model and extract simple parameters to represent these movements is a difficult point.

[0004] At present, the mandible is static data during data collection. In actual use, it is often necessary to match the mandible with the maxilla for analysis. At this time, the existing technology simply moves the two models together by changing the spatial position. However, when the mandible actually moves, it is restricted by bones, muscles, etc., and the movement is not the same, so the matching result of the upper and lower jaws may also be inaccurate. Another existing method is to use dedicated collection equipment to collect movement conditions, which is costly and professional.

[0005] In view of the above problems, it is necessary to find a simple, efficient and low-cost method to obtain a model that shows the dynamic movement of the mandible. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a mandibular motion modeling method, electronic device and storage medium, which can solve the problem of automatically establishing a mandibular motion model that conforms to the patient's actual dynamics.

[0007] To solve the above technical problems, an embodiment of the present application provides a method for modeling mandibular movement, comprising: receiving image data pictures of a patient's mandible in different positions, a digitized three-dimensional maxillary model and a mandibular model, wherein the image data pictures can at least display part of the maxillary dentition and part of the mandibular dentition; determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative positions of the maxillary dentition and the mandibular dentition in the image data pictures; wherein, for each image data picture in which the mandible is located in a different position, a spatial position and posture of a mandibular model are determined correspondingly; and the determined spatial positions and postures of the mandibular models are analyzed to obtain the movement trajectory of the mandibular model relative to the maxillary model.

[0008] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned mandibular movement modeling method.

[0009] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned mandibular movement modeling method when executed by a processor.

[0010] The mandibular motion modeling method of the embodiment of the present application collects the movement of the patient's mandible through image data pictures, matches the pre-acquired upper and lower mandibular models according to the image data pictures, confirms the position of the mandible relative to the maxilla in three-dimensional space during the movement of the mandible, and analyzes the position to obtain the motion trajectory of the mandibular model relative to the maxillary model. It can be seen that the mandibular motion modeling method in the present application can automatically generate a motion trajectory, which is simple and efficient, reduces the patient's participation process, and can also provide more realistic patient data for the design and manufacturing stages of the orthodontic appliance, so that the obtained motion trajectory is more consistent with the actual motion situation.

[0011] In addition, the analysis of the determined spatial positions and postures of the mandibular model to obtain the motion trajectory of the mandibular model relative to the maxillary model includes: analyzing the determined spatial positions and postures of the mandibular model to obtain the basic motion direction of the mandibular model relative to the maxillary model, the basic motion direction characterizing the mouth opening direction, lateral jaw motion direction, and sagittal forward and backward motion direction of the mandibular bone; and determining the motion trajectory based on the basic motion direction. Since the mandibular model is a compound motion, the basic motion direction is extracted for the mandibular motion, and then the motion trajectory is determined, so as to accurately obtain the trajectory of the compound motion of the mandible.

[0012] In addition, the analysis of the determined spatial positions and postures of the mandibular model to obtain the basic movement direction of the mandibular model relative to the maxillary model includes: using a kernel principal component analysis method to obtain the basic movement direction. The kernel principal component analysis method is limited to be used to analyze and obtain the basic movement direction. The principal component analysis can obtain the eigenvalues ​​and eigenvectors of a set of data. The eigenvector is also the basis vector. The result obtained by the analysis also represents the basic movement direction of the composite movement.

[0013] In addition, the analysis using the kernel principal component analysis method includes: obtaining a covariance matrix using a preset kernel function; performing eigenvalue decomposition on the covariance matrix to obtain N eigenvalues ​​and corresponding eigenvectors; wherein N is a natural number greater than 3; selecting the first M eigenvectors with the largest eigenvalues ​​from the N eigenvalues ​​as the M basic motion directions; wherein M is a natural number less than or equal to N and greater than 3. Further limiting the analysis process of the kernel principal component analysis method, selecting an eigenvector with a larger eigenvalue can select a motion direction that is more representative of the composite motion.

[0014] In addition, the analysis of the determined spatial positions and postures of the mandibular model to obtain the basic movement direction of the mandibular model relative to the maxillary model includes: analyzing the spatial positions and postures through a neural network to obtain the basic movement direction. The neural network analysis method is limited to analyzing complex movements through the neural network, and the movement rules can be more accurately confirmed.

[0015] In addition, the neural network adopts a pre-trained neural network based on a multi-layer perceptron.

[0016] In addition, determining the motion trajectory based on the basic motion directions includes: linearly combining the basic motion directions of the mandible obtained through analysis to obtain the motion trajectory of the compound motion of the mandible.

[0017] In addition, the method of determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative position of the maxillary dentition and the mandibular dentition in the image data picture includes: extracting the first visual feature data of the dentition part in the image data picture; combining the maxillary model and the mandibular model, projecting the dentition part of the combined maxillary model and the mandibular model on the imaging plane corresponding to the preset camera parameters, adjusting the relative posture of the maxillary model and / or the mandibular model, projecting each relative posture respectively, and obtaining the second visual feature data after each projection respectively; finding the second visual feature data matching the first visual feature data, and determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative posture corresponding to the second visual feature data matching the first visual feature data. It is limited to aligning the projection graphics of the three-dimensional model with the visual feature data in the image data picture, so as to confirm the posture of the three-dimensional model, and the result is accurate and the degree of automation is high.

[0018] In addition, the method of determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model based on the relative positions of the maxillary dentition and the mandibular dentition in the image data picture includes: marking first-type tooth feature points on the dentition part in the image data picture to obtain a plurality of first feature points, marking the first-type tooth feature points in the maxillary model and the mandibular model to obtain a plurality of second feature points, wherein the number of the first feature points is greater than or equal to 3 and they are not on the same straight line, and the number of the second feature points is greater than or equal to 3 and they are not on the same straight line; combining the maxillary model and the mandibular model, adjusting the relative posture of the maxillary model and the mandibular model, respectively obtaining combined models corresponding to the respective relative postures, changing the observation angle of the combined model, respectively obtaining the projection position of each second feature point of each combined model at each observation angle relative to the camera plane, and the camera plane is perpendicular to the observation angle; finding the projection position of the second feature point that matches the position of each first feature point, and determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative posture of the upper and lower jaws in the combined model corresponding to the projection position. The method of marking feature points is limited to confirming the position and posture of the 3D model by aligning the feature points. The result is accurate and the amount of data involved in the calculation is small.

[0019] In addition, before obtaining the motion trajectory of the mandibular model relative to the maxillary model, it includes: obtaining the orthodontic plan of the patient; after obtaining the motion trajectory of the mandibular model relative to the maxillary model, it includes: performing collision detection between the upper and lower mandibular models at each stage in the orthodontic plan according to the maxillary model, the mandibular model and the motion trajectory. The obtained motion trajectory is limited to detecting bite collisions. Since the motion trajectory is more consistent with the actual bite action of the user, the detection of the user's bite collision can be more real and comprehensive.

[0020] In addition, the orthodontic plan includes: a tooth arrangement plan and a jaw position adjustment plan.

[0021] In addition, after obtaining the motion trajectory of the mandibular model relative to the maxillary model, the method further comprises: when an adjustment position of the mandibular model needs to be obtained, adjusting the mandibular model according to the motion trajectory to obtain the position and posture of the adjusted mandibular model. The mandibular model can be adjusted according to the motion trajectory determined in the preceding steps, so that the adjusted mandibular position is more realistic and meets medical needs.

[0022] In addition, the image data pictures include: photos or X-rays. The types of different images that can be selected are limited, and different types can be selected under different requirements, making the implementation more flexible and facilitating the promotion of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0024] Figure 1 is a flowchart of a method for modeling mandibular movement provided according to one embodiment of the present application;

[0025] Figure 2 is a schematic diagram of a two-dimensional image used in one embodiment of the present application;

[0026] Figure 3 is a schematic diagram of a digital three-dimensional maxillary model and a mandibular model used in one embodiment of the present application;

[0027] Figure 4 is Figure 2 Schematic diagram after marking FA point in the middle;

[0028] Figure 5 is Figure 3 Schematic diagram after marking FA point in the middle;

[0029] Figure 6is a flow chart of a process of determining the spatial position and posture of a digitized three-dimensional mandibular model relative to a maxillary model according to one embodiment of the present application;

[0030] Figure 7 It is a flowchart of a process of aligning the maxillary model and the mandibular model into a preset tooth-jaw coordinate system according to one embodiment of the present application;

[0031] Figure 8 is a schematic diagram of a tooth-jaw coordinate system established according to an embodiment of the present application;

[0032] Fig. 9 and Fig.10 They are three-dimensional models of the mandibular model in different relative postures determined in one embodiment of the present application;

[0033] Fig.11 and Fig.12 They are three-dimensional models of the mandibular model in different relative positions under different shooting directions determined in one embodiment of the present application;

[0034] Fig.13 is a flow chart of a process of determining a basic motion direction according to an embodiment of the present application;

[0035] Fig.14 is a flow chart of another process of determining the spatial position and posture of a digitized three-dimensional mandibular model relative to a maxillary model according to another embodiment of the present application;

[0036] Fig.15 A patient's dentition photograph provided according to an embodiment of the present application;

[0037] FIG. 16( a ) is a schematic diagram of a peripheral contour line of a dentition according to an embodiment of the present application;

[0038] FIG16( b ) is a schematic diagram of a set of contour lines of a single tooth provided according to an embodiment of the present application;

[0039] Fig.17 is a flow chart for extracting a first contour line according to an embodiment of the present application;

[0040] Fig.18 is a flow chart of extracting a second contour line according to an embodiment of the present application;

[0041] Fig.19 It is a structural schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0043] An embodiment of the present application relates to a mandibular movement modeling method. The specific process of the mandibular movement modeling method of this embodiment can be as follows: Figure 1 As shown, including:

[0044] Step 101, receiving image data pictures of the patient's mandible in different positions, a digitized three-dimensional maxillary model and a mandibular model, wherein the image data pictures can at least show a portion of the maxillary dentition and a portion of the mandibular dentition.

[0045] Step 102, based on the relative positions of the maxillary dentition and the mandibular dentition in the image data picture, determine the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model; wherein, for each image data picture in which the mandible is located at a different position, a spatial position and posture of a mandibular model is correspondingly determined.

[0046] Step 103: Analyze the determined spatial positions and postures of the mandibular model to obtain a motion trajectory of the mandibular model relative to the maxillary model.

[0047] In this embodiment, the movement of the patient's mandible is collected through image data pictures, and the pre-obtained upper and lower jaw models are matched according to the image data pictures to confirm the position of the mandible relative to the maxilla in three-dimensional space during the movement of the mandible, and the position is analyzed at the same time to obtain the movement trajectory of the mandibular model relative to the maxillary model. It can be seen that the mandibular movement modeling method in this application can automatically generate a movement trajectory, which is simple and efficient, reduces the patient's participation process, and can also provide more realistic patient data for the design and manufacturing stages of the orthodontic appliance, so that the obtained movement trajectory is more consistent with the actual movement situation.

[0048] The implementation details of the mandibular movement modeling method of this embodiment are specifically described below. The following content is only provided for easy understanding of the implementation details and is not necessary for implementing this solution.

[0049] In step 101, the received image data picture may be a picture of the patient's clinically acceptable dentition taken in advance by a camera. Specifically, clinically acceptable can be represented by the patient's dentition status required by the doctor. The clinically acceptable standard of this embodiment can at least show part of the maxillary dentition and part of the mandibular dentition, that is, the image data picture needs to be able to show part of the maxillary dentition 10 and part of the mandibular dentition 20 of the patient, and the maxillary dentition and the mandibular dentition are the upper and lower dentitions of the same patient taken at the same time. At this time, the upper and lower dentitions on the image data picture are the relative positions of the upper and lower jaws that meet the upper and lower jaw occlusion relationship of the corresponding patient, such as Figure 2 As shown. In other words, there are multiple received image data pictures, including image data of the mandible in different positions. At this time, the user can be guided to open the mouth, move the jaw sideways, move the jaw forward and backward, and collect photos at different positions. It is understandable that the image data pictures in this embodiment can also be X-rays collected by CT / X-ray, etc., and the picture color can be black and white / color, etc., which will not be listed here one by one.

[0050] In this embodiment, the image data picture carries information about the shooting direction, wherein the shooting direction information includes: front shooting, left shooting or right shooting, then the image data picture can be a two-dimensional image of the dentition obtained from the front of the patient, or a two-dimensional image of the dentition obtained from the left side of the patient, or a two-dimensional image of the dentition obtained from the right side of the patient, or a two-dimensional image of the dentition obtained from the front of the patient, a two-dimensional image of the dentition obtained from the left side of the patient, and a two-dimensional image of the dentition obtained from the right side of the patient, and Figure 2 What is shown is a dental photograph taken from the front of the patient, and other shooting directions are not listed one by one. It is worth mentioning that the two-dimensional image used in this embodiment can be a two-dimensional image collected when the patient is in a certain occlusal position.

[0051] It should be further explained that the digital three-dimensional maxillary model 30 and mandibular model 40 received in this step can be obtained by taking molds of the patient to obtain the upper and lower mandibular digital models, such as Figure 3 As shown, it is worth mentioning that although the above figure shows the maxillary model 30 and the mandibular model 40 in one figure, in fact, the maxillary model 30 and the mandibular model 40 are two discrete independent models and have no mutual positional relationship.

[0052] In one example, when taking image data pictures, multiple image data pictures can be taken in each shooting direction, for example, multiple two-dimensional images of the dentition can be obtained by taking pictures from the front of the patient, and multiple two-dimensional images of the dentition can be obtained by taking pictures from the left side of the patient, or multiple two-dimensional images of the dentition can be obtained from the right side of the patient, so as to select a two-dimensional image that better meets the preset conditions from the multiple image data pictures taken in each direction, for example, select an image data picture with higher clarity and easier contour extraction.

[0053] In step 102, the position and posture of the digitized three-dimensional upper and lower jaw models in three-dimensional space are determined based on the relative positions of the maxillary dentition and the mandibular dentition in the two-dimensional image data picture.

[0054] In some embodiments, Figure 6 As shown, the following steps are used to determine, specifically, the alignment is performed based on the characteristic points marked on the dentition, so as to use the relative positions of the upper and lower dentitions in the two-dimensional image to determine the relative posture of the mandibular dentition with respect to the maxillary dentition in the three-dimensional space.

[0055] Step 201, marking first type tooth feature points on the maxillary dentition and the mandibular dentition in the image data picture respectively to obtain a plurality of first feature points.

[0056] Specifically, the first type of tooth feature points are parameters that reflect tooth features, such as the most convex FA point of the long axis of the clinical crown, the end point of the alveolar ridge on the long axis of the tooth, or the labial junction of the adjacent gum line. In addition, the center point of a single tooth area in the figure can also be used. Figure 4 Indicated in Figure 2 In the form after the FA point is marked, 11, 12, 13, and 14 are the first feature points of the example.

[0057] More specifically, the present embodiment can be manually marked by the user on a computer device based on experience. Since doctors and other professionals have rich experience in tooth feature recognition, manual marking is more accurate after determining the type of tooth feature points to be marked.

[0058] In practical applications, in addition to the above-mentioned manual marking method, machine learning can also be used for automatic identification and marking, such as using the YOLO v3 model to automatically mark feature points from photos. In practical applications, network models such as Faster R-CNN, Mask R-CNN, SSD, etc. can also be used. One of the different network models or a combination of them can be selected according to the conditions of the photo and the limitations of the usage scenario. I will not go into details here.

[0059] It should also be noted that the number of marked first feature points is greater than or equal to 3, and they are not on the same straight line. In practical applications, as many first feature points as possible can be marked to ensure the accuracy of subsequent registration results, and the first feature points are not on the same straight line to avoid problems such as reversal during registration. It is worth mentioning that the marking points can be marked in color (such as red) or in grayscale, and the marking form can be dots, crosses, etc., which are not limited here.

[0060] Step 202: Mark the first type of tooth feature points in the maxillary model and the mandibular model respectively to obtain a plurality of second feature points.

[0061] Specifically, the type of tooth feature points marked in this step is the same as that selected in step 102. For example, if the FA point is selected in step 102, the FA point is also marked in this step. If the labial junction point of the adjacent gum line is selected in step 102, the labial junction point of the adjacent gum line is also marked in this step. The type of tooth feature points selected is the same to ensure the accuracy of the registration. Figure 5 Indicated in Figure 3 The form after marking the FA point, 31, 32, 33, 34 in the figure are the second characteristic points of the example. It is worth mentioning that although the upper jaw model and the lower jaw model are shown in one figure, in fact, the upper jaw model and the lower jaw model are two discrete independent models at this time, and they do not have a mutual position relationship.

[0062] More specifically, this embodiment can be manually marked by the user on a computer device based on experience. For example, the intersection of the center point of the tooth on the labial tooth surface along the labial and lingual directions is the FA point. Doctors and other professionals have rich experience in tooth feature recognition. Therefore, manual marking is more accurate after determining the type of tooth feature point to be identified.

[0063] In some embodiments, the marking method in this step can also adopt the ray intersection solution method. Taking the first type of tooth feature point as the FA point as an example, the identification of the FA point of a certain tooth is explained, and the specific solution process includes: taking the center of gravity of the tooth as the starting point of the ray, the long axis as the ray direction, and intersecting with the tooth grid at two points; taking the midpoint of the two intersections as the temporary midpoint, calculating the angle between the intersection of the above-mentioned ray orthogonal direction and the temporary midpoint connection line, and the bottom control point and the temporary midpoint connection line, and dividing them into 49 parts; rotating the long axis around the mesiodistal axis by the above angle to obtain 50 directions; taking the temporary midpoint as the starting point of the ray, the above direction as the ray direction, and intersecting the 50 intersections of the tooth grid; taking the point where the above 50 intersections are projected on the long axis, selecting the control point corresponding to the point in the middle of the position, and using it as the FA point.

[0064] In practical applications, in addition to the above-mentioned manual marking and ray intersection solution methods, machine learning can also be used for automatic identification and marking. The selected network can be PointNet network, PointNet++ network, or DGCNN network, which are not listed here one by one.

[0065] It should also be noted that the number of the second feature points is greater than or equal to 3 and they are not on the same straight line. In practical applications, as many first feature points as possible can be marked to ensure the accuracy of subsequent registration results, and the first feature points are no longer on the same straight line to avoid problems such as reversal during registration.

[0066] It is worth mentioning that in actual applications, the order of the above steps 201 and 202 can be changed. The feature points of the two-dimensional image can be marked first, and then the feature points of the three-dimensional model can be marked. Alternatively, the feature points of the three-dimensional model can be marked first, and then the feature points of the two-dimensional image can be marked. The change in order does not affect the execution of the overall solution.

[0067] Step 203, combining the maxillary model and the mandibular model, that is, aligning the maxillary model and the mandibular model to the preset dental coordinate system, and then projecting the dentition part of the maxillary model and the mandibular model aligned to the dental coordinate system on the imaging plane corresponding to the preset camera parameters, and adjusting the relative posture of the maxillary model and / or the mandibular model at the same time, so as to obtain various relative postures of the maxillary model and / or the mandibular model, and projecting each relative posture of the maxillary model and / or the mandibular model relative to the camera plane, and finally obtaining each projection position of the second feature point after each projection. It should be noted that the above-mentioned camera plane is not an actual camera, but a plane corresponding to the observation angle. In order to illustrate the reference plane during projection, the plane can also be represented by numerical values ​​in the coordinate system, which is more concise and convenient.

[0068] In some embodiments, the image data picture may carry information of the shooting direction. Correspondingly, when changing the observation angle of the combined model, the observation angle of the combined model may be changed according to the shooting direction of the image data picture. Since the projection positions corresponding to different shooting directions differ greatly, adjusting the observation angle of the combined model in the shooting direction is conducive to reducing the number of direction adjustments and speeding up the registration speed. Further, changing the observation angle of the combined model according to the shooting direction of the image data picture includes: presetting the initial observation angle of the combined model according to the shooting direction of the image data picture; adjusting the initial observation angle within a preset range to obtain multiple observation angles of the combined model. Further limiting the determination of the initial observation angle according to the shooting direction so that the initial observation angle is closer to the shooting direction of the photo, further reducing the number of observation angle adjustments during the registration process, and speeding up the registration speed. If the shooting direction is frontal shooting, the preset initial observation angle may be the opposite direction of the Z axis, and the shooting direction may not be a certain positive viewing direction, but its angle deviation is within a certain range, so that the range of subsequent adjustments can be narrowed so as to find an observation angle consistent with the shooting direction as soon as possible.

[0069] In some embodiments, step 203 may be performed by Figure 7 The process shown aligns the maxillary model and the mandibular model to the preset tooth-jaw coordinate system, that is, realizes the combination of the maxillary model and the mandibular model. In this part of the embodiment, the tooth-jaw coordinate system is obtained by the standard tooth-jaw model, so that the coordinate system is pre-stored and can be directly called and used when the object is changed, which specifically includes:

[0070] Step 701, obtaining the center of gravity of each tooth in the maxillary model and the mandibular model.

[0071] Step 702: align the center of gravity of each tooth in the maxillary model and the mandibular model with the center of gravity of each tooth in the standard maxillary model in the maxillary coordinate system.

[0072] Specifically, by aligning the center of gravity of each tooth in the maxillary model and the mandibular model with the center of gravity of each tooth in the standard mandibular model in the mandibular coordinate system, the posture data required by the maxillary model and the mandibular model can be obtained when each tooth in the maxillary model and the mandibular model has the best alignment relationship with each tooth in the mandibular coordinate system, and the posture data specifically includes the translation and rotation of the maxillary model and the mandibular model. Among them, the alignment algorithm includes: iterative closest point algorithm or consistent point drift algorithm.

[0073] Step 703: Align each tooth in the maxillary model and the mandibular model to the tooth-jaw coordinate system according to the registration result.

[0074] Specifically, by adjusting the posture of the maxillary model and the mandibular model according to the posture data required for each tooth in the maxillary model and the mandibular model to have the best alignment relationship with each tooth in the dental coordinate system, each tooth in the maxillary model and the mandibular model can be aligned to the dental coordinate system respectively.

[0075] In one example, since the patient may have missing teeth, in order to avoid the situation where the maxillary model and the mandibular model cannot be accurately aligned to the dental coordinate system, before aligning the maxillary model and the mandibular model to the dental coordinate system respectively, it is first necessary to use the same mask to process the standard dental model and the maxillary model and / or the mandibular model respectively, so as to generate a maxillary model and / or a mandibular model with the same number of teeth as the standard dental model; wherein the mask is used to hide the corresponding teeth in the standard dental model and the maxillary model and / or the mandibular model. It can be understood that the standard dental and jaw model is specifically any digital model of the dental and jaw without missing teeth and with 14 complete teeth in both the upper and lower jaws, which is used to establish a dental and jaw coordinate system, and specifically includes a standard maxillary model and a standard mandibular model. When the standard dental and jaw model, the maxillary model and the mandibular model are processed respectively using the same mask, specifically the standard maxillary model and the maxillary model are processed using the same mask, and the standard mandibular model and the mandibular model are processed using the same mask to generate a maxillary model with the same number of teeth as the standard maxillary model, and a mandibular model with the same number of teeth as the standard mandibular model.

[0076] In a specific implementation, in addition to the above-mentioned use of a standard dental model to make a preset dental coordinate system, the preset dental coordinate system can also be obtained through the principal component analysis method, so that the dental model calculation for each patient can be established, and the position data of the characteristic points on the teeth can be directly matched without the need for alignment and other steps. Specifically, the principal component analysis of the center of gravity of all individual tooth models can be performed to obtain the principal component analysis results of each individual tooth model, and the principal component analysis results of each individual tooth model can reflect the center of gravity characteristics of all individual tooth models, so that the center of gravity distribution of all individual tooth models in the standard dental model can be obtained, and then the X-axis, Y-axis and Z-axis of the three-dimensional dental coordinate system can be determined according to the center of gravity distribution of each individual tooth model in the standard dental model.

[0077] Among them, the principal component analysis method is a statistical method of dimensionality reduction. It uses an orthogonal transformation to transform the original random vector whose components are correlated into a new random vector whose components are uncorrelated. Geometrically, it transforms the original coordinate system into a new orthogonal coordinate system to point to the p orthogonal directions with the largest sample point distribution. Then, the multidimensional variable system is subjected to dimensionality reduction processing so that it can be converted into a low-dimensional variable system with a higher accuracy. Then, by constructing an appropriate value function, the low-dimensional system is further converted into a one-dimensional system.

[0078] Therefore, the principal component analysis results of the center of gravity of all single tooth models in this embodiment obtained by principal component analysis can include three orthogonal eigenvectors, namely the first eigenvector, the second eigenvector and the third eigenvector. Therefore, the first eigenvector in the principal component analysis result is used as the X-axis of the three-dimensional tooth-jaw coordinate system, the second eigenvector is used as the Y-axis of the three-dimensional tooth-jaw coordinate system, and the third eigenvector is used as the Z-axis of the three-dimensional tooth-jaw coordinate system, that is, the directions of the coordinate axes of the three-dimensional tooth-jaw coordinate system are obtained. It can be understood that the distribution of the center of gravity of each single tooth model in the coronal, sagittal and vertical directions in the standard dentition model has specific characteristics. After principal component analysis, the center of gravity of each single tooth model will be distributed in the three orthogonal directions of the standard dentition model. The looseness of the distribution of each center of gravity in the first direction of the standard dentition model is the largest, that is, the coronal direction, the looseness of the distribution of each center of gravity in the second direction is the second largest, that is, the sagittal direction, and the looseness of the distribution of each center of gravity in the third direction is the third largest, that is, the vertical direction. In practical applications, the degree of looseness can be determined based on the distance variance of each set of centroid points. For example, the center point position of each centroid point is first calculated, and then the distance between each centroid point and the center point in the direction is calculated respectively to obtain multiple distance values. The variance of each distance value is then calculated, and the degree of looseness can be confirmed by the variance.

[0079] That is, in this embodiment, the location of the average value of the center of gravity can be used as the origin of the three-dimensional dental coordinate system, and the direction in which the looseness of the center of gravity distribution of each individual tooth model in the standard dental arch model is the largest as the X-axis of the three-dimensional dental coordinate system, and the direction in which the looseness of the center of gravity distribution of each individual tooth model in the standard dental arch model is the second largest as the Y-axis of the three-dimensional dental coordinate system, and the direction in which the looseness of the center of gravity distribution of each individual tooth model in the standard dental arch model is the third largest as the Z-axis of the three-dimensional dental coordinate system. Finally, a three-dimensional dental coordinate system is established based on the determined origin, X-axis, Y-axis and Z-axis of the three-dimensional dental coordinate system. See the schematic diagram of the established three-dimensional dental coordinate system. Figure 8 .

[0080] In practical applications, when adjusting the relative posture of the maxillary model and the mandibular model, the present embodiment: preset the first posture of the maxillary model, keep the maxillary model in the first posture unchanged, and adjust the posture of the mandibular model. The relative posture of the upper and lower jaws is changed by keeping one jaw fixed and changing the posture of the opposing jaw, so as to simplify the calculation. In addition, during implementation, the second posture of the mandibular model can also be preset, the mandibular model can be kept in the second posture unchanged, and the posture of the maxillary model can be adjusted.

[0081] It should also be noted that, in adjusting the relative posture of the maxillary model and / or the mandibular model, the adjustment methods include: translation and / or rotation, so as to obtain the second visualization feature data of the maxillary model and / or the mandibular model in various relative postures, thereby improving the matching degree between the selected second visualization feature data and the first visualization feature data.

[0082] In step 204, the position of each first feature point is compared with the projection position of each second feature point to find the projection position of the second feature point aligned with the position of each first feature point. Specifically, in this step, the goal is to find the projection position of the second feature point aligned with the position of each first feature point in the image data picture, with the sum of the distance differences between the position of each first feature point in the image data picture and the projection position of the position in the maxillary model and the mandibular model under a camera plane meeting the preset conditions. The alignment is determined by the sum of the display position differences of each point, making the present application more feasible. In some embodiments, the preset conditions may include: the sum of the distance differences is minimum.

[0083] Take a single comparison as an example to illustrate, take the positions of all the first feature points in the two-dimensional image as a whole, and the projection positions of all the second feature points of a certain relative posture of the upper and lower jaw models under a certain projection as a whole, compare the two sets of feature points by translation, rotation, etc., find the state with the smallest sum of the position differences of all feature points, and record the sum of the position differences. Repeat the above process, perform multiple comparisons, and find the one that best meets the preset conditions, that is, the one with the smallest sum of distance differences.

[0084] In practical applications, a threshold can also be set, and the sum of distance differences less than or equal to the threshold can be used as the target for the end of the comparison, so as to reduce the amount of calculation. The above two preset conditions can also be used in combination, and the end of the comparison is confirmed when one of them is met, which can further reduce the amount of calculation and speed up the overall alignment process.

[0085] It is further explained that after finding the matching projection position by comparison, the relative position and posture of the upper and lower jaws corresponding to the projection position can be confirmed, that is, the relative position and posture of the mandibular model with respect to the maxillary model can be obtained.

[0086] It can be seen that the above steps 201 to 204 can determine the spatial position and posture of the digital three-dimensional mandibular model relative to the maxillary model. By receiving the clinically acceptable imaging data of the patient's dentition, the digital three-dimensional maxillary model and the mandibular model, and the clinically acceptable standard can at least display part of the maxillary dentition and part of the mandibular dentition, the upper and lower teeth of the same patient under a certain occlusal relationship are displayed. The tooth feature points of the maxillary dentition and the mandibular dentition in the two-dimensional image are marked respectively to obtain the first feature points, thereby obtaining the positional relationship of each tooth feature point in the two-dimensional display, and then the tooth feature points of the digital three-dimensional model are marked to obtain the second feature points, thereby obtaining the positional relationship of each tooth feature point in the three-dimensional display. Since the two-dimensional display of the upper and lower jaws of the same patient is the projection display of the upper and lower jaw models in a certain specific posture and a certain observation angle in the three-dimensional space relative to the camera plane, the projection position of the second feature point that matches can be found by comparing the first feature point and the second feature point, and the relative posture of the three-dimensional upper and lower jaw models can be obtained. It can be seen that the whole process has a high degree of automation, and is more efficient and accurate than manually adjusting the upper and lower jaws through plaster models. In addition, there is no need to perform a separate oral scan specifically to obtain a digital model of the occlusal state, which further reduces the cost of orthodontics. In addition, the types of defined feature points are diverse, which is conducive to improving the flexibility of selecting tooth feature points in this application. At the same time, points inside the tooth area are selected to reduce the possibility of feature points being blocked in different images or models. Furthermore, different methods of defining and marking feature points can select different marking methods under different needs, which is convenient for the promotion of this application.

[0087] It is understandable that if Fig. 9 and Fig.10 As shown in FIG. 1 , three-dimensional models with different relative positions can be determined according to the above steps. In addition, although the above pictures and models are all taken from the front shooting direction as an example, in actual applications, Fig.11 and Fig.12 As shown, the above method can be used to confirm the corresponding three-dimensional model based on the image data pictures taken in different directions.

[0088] After the relative position and posture of the mandibular model relative to the maxillary model is determined, the following description is continued: in step 103, each of the determined spatial positions and postures of the mandibular model is analyzed to obtain a motion trajectory of the mandibular model relative to the maxillary model. Specifically, the following steps include: analyzing each of the determined spatial positions and postures of the mandibular model to obtain a basic motion direction of the mandibular model relative to the maxillary model; and determining the motion trajectory based on the basic motion direction.

[0089] Specifically, since the movement of the mandible belongs to a compound movement mode, the inventors of the present application conducted an in-depth analysis of the movement of the mandible and found that the movement of the mandible is based on a combination of three basic movement directions, wherein the basic movement directions include the mouth opening direction, the lateral jaw movement direction, and the sagittal forward and backward movement direction that characterize the mandible. Therefore, the direct analysis of compound movements can be simplified by analyzing the basic movement directions.

[0090] In this embodiment, the basic motion direction is analyzed by using the kernel principal component analysis method to obtain the above-mentioned basic motion direction. Fig.13 As shown, specifically including:

[0091] Step 301, use the preset kernel function to obtain the covariance matrix. The specific matrix is ​​as follows:

[0092]

[0093] Among them, n represents the amount of data, x (i) For each data (Ri, ti), Φ is the kernel function. In practical applications, Gaussian kernel function or polynomial kernel function can be selected. i represents the i-th mandibular posture, Ri is the relative rotation of the mandibular model relative to the maxillary model, and ti is the relative translation of the mandibular model relative to the maxillary model.

[0094] Step 302, perform eigenvalue decomposition on the covariance matrix to obtain six eigenvalues ​​and corresponding eigenvectors. Specifically, the number of eigenvalues ​​N obtained by decomposition in this embodiment is 6, and the number of eigenvalues ​​N obtained by decomposition in other embodiments may also be other natural numbers greater than 3.

[0095] Step 303: Select the first three eigenvectors with the largest eigenvalues ​​from the six eigenvalues ​​as three basic movement directions.

[0096] Specifically, the number of eigenvectors M selected above can be a natural number less than or equal to N and greater than 3. To further illustrate, in this embodiment, each data has 6 dimensional vectors after the translation and rotation are combined, and according to the previous analysis, there are three basic movement directions of the mandible, so after the eigenvalue decomposition of the above covariance matrix, 6 eigenvalues ​​and corresponding eigenvectors can be obtained, and the first 3 eigenvectors with the largest eigenvalues ​​are selected, which are recorded as P1, P2, and P3.

[0097] After the basic movement direction is obtained through analysis, the movement trajectory is also determined based on the basic movement direction. Specifically, in this embodiment, the basic movement directions are recorded as P1, P2, and P3, and the movement trajectory of the mandible in this embodiment can be expressed as:

[0098] (R, t) = P1*d1+P2*d2+P3*d3;

[0099] Specifically, the three basic movements of the mandible obtained by analysis are linearly combined to obtain the composite movement trajectory of the mandible. The above d1, d2 and d3 are the changes in the three basic movement directions. In addition, when the values ​​of d1, d2 and d3 are changed, (R, t) respectively represents the rotation and translation of the mandibular model relative to the maxillary model when opening the mouth, moving left and right, and moving forward and backward.

[0100] It can be seen that the mandibular motion modeling method in this embodiment collects the movement of the patient's mandible through image data pictures, matches the pre-acquired upper and lower mandibular models according to the image data pictures, confirms the position of the mandible relative to the maxilla in three-dimensional space during the mandibular movement, and analyzes the position at the same time to obtain the motion trajectory of the mandibular model relative to the maxillary model. It can be seen that the mandibular motion modeling method in this application can automatically generate the motion trajectory, which is simple and efficient, reduces the patient's participation process, and can also provide more realistic patient data for the appliance design and manufacturing stage, so that the obtained motion trajectory is more in line with the actual motion situation. In addition, the mandibular motion is further analyzed by the kernel principal component analysis. Since the principal component analysis can obtain the eigenvalues ​​and eigenvectors of a set of data, the eigenvectors are also basis vectors, and the results obtained by the analysis can also represent the basic motion direction of the mandibular motion. Therefore, when analyzing the mandibular motion, the kernel principal component analysis method is used to make the analysis process simple and direct.

[0101] Another embodiment of the present application also relates to a method for modeling mandibular movement. This embodiment is substantially the same as the above embodiment, with the main difference being that in the above embodiment, the basic movement direction is obtained by kernel principal component analysis, while in this embodiment, the basic movement direction is obtained by neural network analysis.

[0102] Specifically, the neural network in this embodiment adopts a pre-trained neural network based on a multi-layer perceptron, that is, an encoder-decoder network structure. More specifically, the encoder-decoder network structure can be used to analyze a series of mandibular positions and postures to determine the movement trajectory of the mandibular bone; and then the decoder network structure is used to analyze the position and posture of the mandibular bone when different trajectory parameters are given.

[0103] It can be understood that in addition to the neural network analysis method in this embodiment and the kernel principal component analysis method used in the above embodiment, other analysis methods can also be used in actual applications, which are not listed here one by one.

[0104] Another embodiment of the present application also relates to a method for modeling mandibular movement. This embodiment is substantially the same as the above embodiment, with the main difference being that in the above embodiment, alignment is performed based on feature points marked on the dentition to obtain the relative position and posture of the mandibular model relative to the maxillary model, while in the present embodiment, comparison is performed based on the visual feature data of the dentition to obtain the relative position and posture of the mandibular model relative to the maxillary model.

[0105] The process of confirming the relative position of the mandibular model with respect to the maxillary model, such as Fig.14 As shown, specifically including:

[0106] Step 401, extracting first visual feature data of the dentition part in the image data picture.

[0107] Specifically, the first visualization feature data can be represented by the contour line of the dentition part. In other embodiments, the first visualization feature data can be represented by one or a combination of the following: the contour line of the dentition, the projection graph of the dentition, the shape of the dentition, the aspect ratio of the dentition, and the gum line.

[0108] In some examples, the outline of the dentition includes the outer contour of the entire dentition, the outline of all individual teeth, or the outline of a preset number of individual teeth. Fig.15 The peripheral contour lines of the maxillary dentition and the mandibular dentition are extracted from the image data picture shown in FIG. 16( b ); the contour line set of a single tooth is shown in FIG. 16( b ), which is specifically based on Fig.15 The set of contour lines of all single teeth extracted from the image data picture shown; and the set of contour lines of a preset number of single teeth can be a set of contour lines of a preset number of teeth in the maxillary dentition and a set of contour lines of a preset number of teeth in the mandibular dentition, for example, the set of contour lines of teeth No. 1, 2, 3 and 4 in the maxillary dentition and the set of contour lines of teeth No. 1, 2, 3 and 4 in the mandibular dentition as shown in Figure 16(b).

[0109] In some embodiments, if the outline of the dentition is the outer contour of the dentition, then only the set of the boundary lines of the maxillary dentition and the mandibular dentition is needed as the outline of the dentition. In other embodiments, the outline of the dentition is the set of the contours of all individual teeth. In this embodiment, the boundary lines of all individual teeth in the maxillary dentition and the boundary lines of all individual teeth in the mandibular dentition can be obtained respectively, and then a set is formed as the outline of the dentition.

[0110] In one example, the contour set of all single teeth (i.e., the first visual feature data) is represented as a first contour line, which can be specifically represented by Fig.17 The process shown implements the extraction of the first contour line, including:

[0111] Step 501, performing image recognition on the image data picture to obtain each single tooth.

[0112] Specifically, a pre-trained dentition segmentation model is used to segment the dentition in the image data image to obtain each individual complete tooth. In order to improve the segmentation accuracy, the dentition segmentation model can be obtained based on the FCIS network or the MaskRCNN network training. In one example, the output of the preset neural network can also be modified to image segmentation, for example, the output of the UNet series network is modified to multi-classification image segmentation to output the segmented tooth image.

[0113] Step 502: extract the boundary line of each single tooth as the first contour line.

[0114] Specifically, the schematic diagram of the first contour line extracted in this embodiment can be seen in Figure 16(b). The line width of each boundary line of a single tooth can be set to 2-4 pixels. In some embodiments, the specific value is 3 pixels to improve the clarity of the extracted first contour line.

[0115] After the first contour line is obtained according to steps 501 and 502, the first visual feature data of the dentition part in the image data picture is obtained, and the subsequent steps are continued:

[0116] Step 402, combine the maxillary model and the mandibular model, project the dentition parts of the combined maxillary model and the mandibular model onto an imaging plane corresponding to preset camera parameters, and adjust the relative posture of the maxillary model and / or the mandibular model at the same time, project each relative posture separately, and obtain second visualization feature data after each projection.

[0117] In a specific implementation, the second visualization feature data is the same as the first visualization feature data and can also be represented by a contour line; similarly, the second visualization feature data can also be represented by one of the following or a combination thereof: the contour line of the dentition, the projection figure of the dentition, the shape of the dentition, the aspect ratio of the dentition, the gum line, wherein the contour line of the dentition can also be the outer contour line of the entire dentition, the contour line set of all individual teeth, or the contour line set of a preset number of individual teeth.

[0118] In some embodiments, the contour set of all single teeth (i.e., the second visual feature data) is represented as a second contour line, which can be specifically represented by Fig.18 The process shown implements the extraction of the second contour line, including:

[0119] Step 601, segment all single teeth of the maxillary model and the mandibular model to obtain a plurality of crowns and gums corresponding to each crown.

[0120] Specifically, the dentition of the maxillary model and the dentition of the mandibular model are first segmented to obtain all the single teeth of the maxillary model and the mandibular model, and then all the single teeth of the maxillary model and the mandibular model are segmented again to obtain a number of crowns and gums corresponding to each crown. It should be noted that after segmentation, each crown and corresponding gum will be assigned a medical number.

[0121] Step 602: Obtain the projection of the target crown among the crowns that is not blocked by the corresponding gums on the imaging plane.

[0122] Specifically, first, the segmented crowns and gums are projected onto the imaging plane, and the intersecting projection areas of the projections of each crown and each gum, as well as the first crown projection area that does not intersect with the projections of each gum, are extracted. It can be seen that the crown corresponding to the first crown projection area must be the projection area of ​​each crown that is completely unobstructed by the gums. Since in the camera coordinate system of the image data picture, the center of the imaging plane is the origin, the X-axis and the Y-axis are on the imaging plane, and the Z-axis points from the camera to the object (i.e., the patient's dentition), the closer the gingival projection area with the smallest depth (i.e., the smallest Z value) is to the camera, the more visible the gingiva corresponding to this part of the gingival projection area is to the camera under the camera's viewing angle. Therefore, the gingival projection area with the smallest depth in this part of the gingival projection area can be used as the target gingival projection area, and the second crown projection area intersecting with the target gingival projection area can be extracted from the projection area that intersects the projection of each crown and the projection of each gingiva. Then, the crown corresponding to this part of the crown projection area is also visible to the camera, that is, the crown corresponding to the second crown projection area is not blocked by the gingiva. Therefore, the target crown projection area composed of the first crown projection area and the second crown projection area corresponds to the projection area of ​​the target crown that is not blocked by the corresponding gingiva among all crowns.

[0123] Step 603: Use the boundary line of the projection of the target tooth crown on the imaging plane as the second contour line.

[0124] Specifically, the second contour line extracted in this embodiment is specifically the projection of the boundary line of the crown that is not blocked by the gums, and it can be seen from step 602 that the projection of the target crown on the imaging plane is the target crown projection area, then the boundary line of the target crown projection area is the second contour line, that is, the boundary line of the target crown projection area composed of the first crown projection area and the second crown projection area is used as the second contour line.

[0125] In one example, since the projection of the gums whose distance from the crown is greater than the first preset threshold is not helpful for obtaining the contour of the gum line, in order to reduce the amount of data involved in the calculation, before projecting each crown and each gum onto the imaging plane in step 602, for each tooth, the gums whose distance from the crown is less than the first preset threshold are extracted from the gums, and then the gums whose distance from the crown is less than the first preset threshold are projected onto the imaging plane, that is, part of the gums can be selected for projection, and the projection efficiency and subsequent processing efficiency are higher. Among them, the value of the first preset threshold is less than or equal to 1.5 cm and greater than or equal to 0.5 cm, which can improve the extraction efficiency of the second contour line while ensuring the extraction accuracy of the second contour line.

[0126] It can be seen that steps 601 to 603 can realize the extraction of the second contour line (i.e., the second visualization feature data obtained by projecting on the imaging plane), and this extraction method is relatively general and easy to implement. In addition, in other embodiments, the second visualization feature data can be obtained specifically by texture map rendering method, and can also be obtained by other methods in actual applications. The method for obtaining the second visualization feature data is not limited here.

[0127] After the second contour line is extracted, the second visual feature data in this embodiment is obtained, and then the following steps are performed:

[0128] Step 403, finding the second visualization feature data matching the first visualization feature data, and determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative posture corresponding to the second visualization feature data matching the first visualization feature data.

[0129] In some embodiments, when adjusting the relative posture of the maxillary model and / or the mandibular model in step 403, the maxillary model can be fixed, and the posture of the mandibular model relative to the maxillary model can be adjusted to obtain the target relative posture of the maxillary model and the mandibular model. The mandibular model can also be fixed, and the posture of the maxillary model relative to the mandibular model can be adjusted to obtain the target relative posture of the maxillary model and the mandibular model. The posture of the maxillary model and the mandibular model can also be adjusted at the same time to obtain the target relative posture of the maxillary model and the mandibular model. After adjusting the relative posture of the maxillary model and / or the mandibular model, the maxillary model and the mandibular model in each relative posture are projected onto the imaging plane to obtain the corresponding second visualization feature data.

[0130] Among them, the adjustment method used when adjusting the relative posture of the maxillary model and / or the mandibular model specifically includes: translation and / or rotation, so as to adjust the second visualization feature data of the maxillary model and / or the mandibular model in various relative postures, thereby improving the matching degree between the selected second visualization feature data and the first visualization feature data.

[0131] In one example, since adjusting the relative posture of the maxillary model and / or the mandibular model is essentially adjusting the position of the center of gravity of each tooth in the maxillary model and / or the mandibular model, and the rotation adjustment method will cause the center of gravity of each tooth in the maxillary model and / or the mandibular model to deviate too much, which is not conducive to the subsequent rapid acquisition of the second visualization feature data and the first visualization feature data matching result, the translation method is preferably used to adjust the center of gravity of each tooth to adjust the relative posture of the maxillary model and / or the mandibular model, which can quickly and easily adjust the required target relative posture of the maxillary model and the mandibular model, and then project the second visualization feature data under the target relative posture. In specific implementation, first, multiple translation values ​​are set for the maxillary model and / or the mandibular model in each direction, and in each direction, the maxillary model and / or the mandibular model are translated by each translation value, so that a variety of different relative postures of the maxillary model and the mandibular model can be obtained. In other examples, the relative position of the maxillary model and / or the mandibular model may be adjusted by rotation to accurately adjust the relative position corresponding to the second visualization feature data that matches the first visualization feature data.

[0132] Furthermore, in step 403, the first visualization feature data is also registered with the second visualization feature data obtained from each projection, and the second visualization feature data with the smallest registration error is used as the second visualization feature data matching the first visualization feature data. Finally, the maxillary model and the mandibular model are set according to the relative posture of the second visualization feature data matching the first visualization feature data, that is, the relative posture of the maxillary model and the mandibular model is adjusted, so that the occlusal relationship between the maxillary model and the mandibular model corresponds to the occlusal relationship between the upper and lower jaws in the image data picture, and the relative posture of the mandibular model with respect to the maxillary model can be obtained.

[0133] In a specific implementation, the types of the first visualization feature data and the second visualization feature data used in the registration can be the same, for example, the first visualization feature data and the second visualization feature data are both represented by the outline of the dentition, or are both represented by the projection graph of the dentition, etc.; the types of the first visualization feature data and the second visualization feature data can also be different, for example, the first visualization feature data is the outline of the dentition, and the second visualization feature data is represented by the projection graph of the dentition, etc., and the specific types are not limited.

[0134] It can be seen that in this embodiment, the comparison is performed based on the visual feature data of the dentition part to obtain the relative posture of the mandibular model with respect to the maxillary model, which is also easy to implement and has a high degree of automation.

[0135] Another embodiment of the present application also relates to a mandibular motion modeling method. This embodiment is further improved compared to the above-mentioned embodiment. The main improvement is that collision detection is performed after the motion trajectory is obtained.

[0136] Specifically, before obtaining the motion trajectory of the mandibular model relative to the maxillary model, the orthodontic plan of the patient is obtained. The orthodontic plan obtained can be specified by a doctor, or determined by the upper and lower jaw positions that meet the normal occlusal relationship in the malocclusion classification, which will not be listed here one by one.

[0137] Accordingly, after obtaining the motion trajectory of the mandibular model relative to the maxillary model, the method includes: performing collision detection between the upper and lower mandibular models at each stage in the orthodontic scheme according to the maxillary model, the mandibular model and the motion trajectory.

[0138] To continue explaining, in some embodiments, the orthodontic plan may include a tooth arrangement plan, which may include several dentition positions, such as a middle position, a target position, etc. During collision detection, according to the mandibular movement trajectory, it is checked whether the mandibular dentition and the maxillary dentition will collide in the middle positions when the mandibular bone moves to the above-mentioned dentition positions.

[0139] It should also be noted that in some other embodiments, the orthodontic plan may include a jaw position adjustment plan. For example, in cases of adjusting the inter-jaw relationship, the mandible will correspond to a target position that needs to be adjusted. Therefore, during collision detection, it is also possible to verify based on the mandibular movement trajectory obtained in the previous steps whether a collision between the mandibular and maxillary dentition will occur in the intermediate positions during the process of the mandible moving to the above-mentioned dentition position.

[0140] It should be noted that in the design of orthodontic schemes, it is often necessary to determine whether each dentition or tooth will collide. At present, the designed tooth target position is usually used for collision detection, and the tooth target position is only a static position. The absence of collision in this state does not mean that there will be no collision during the intermediate movement process. Although the mandible generally moves in the direction of opening the mouth during the bite action, the bite action of some patients may be combined with lateral jaw movement. At this time, even if the target position does not collide, collision may still occur during lateral jaw movement. Therefore, the mandibular dynamic model in this application can solve this problem well, and can comprehensively and accurately verify whether the mandible will collide during the movement process, so as to optimize the orthodontic scheme based on the verification results, so that the optimized orthodontic scheme can be more in line with the patient's actual movement conditions, thereby improving the achievable rate of the orthodontic scheme.

[0141] Another embodiment of the present application also relates to a method for mandibular motion modeling. This embodiment is further improved compared to the above-mentioned embodiment. The main improvement is that after obtaining the motion trajectory, the position and posture of the mandibular model can be obtained as needed.

[0142] Specifically, after obtaining the motion trajectory of the mandibular model relative to the maxillary model, the method includes: when it is necessary to obtain an adjustment position of the mandibular model, the mandibular model can be adjusted according to the motion trajectory to obtain the position and posture of the adjusted mandibular model. The adjustment position can currently be obtained manually by a doctor with low accuracy, but in this embodiment, it can be obtained by calculation from the mandibular motion trajectory obtained above. When predicting the jaw position in orthognathic surgery, the use of this motion model will greatly improve the accuracy of orthognathic surgery prediction.

[0143] It should be noted that the above examples in this embodiment are all illustrative descriptions for easy understanding and do not constitute a limitation on the technical solutions of the present invention.

[0144] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0145] Another embodiment of the present application relates to an electronic device, such as Fig.19 As shown, it includes: at least one processor 901; and a memory 902 that is communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions that can be executed by the at least one processor 901, and the instructions are executed by the at least one processor 901 so that the at least one processor 901 can execute the mandibular movement modeling method in the above-mentioned embodiments.

[0146] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0147] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0148] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0149] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0150] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for modeling mandibular motion, characterized in that: include: Receiving a patient's digitized three-dimensional maxillary model and mandibular model, and imaging data pictures of the mandible in different positions, wherein the imaging data pictures can at least show a portion of the maxillary dentition and a portion of the mandibular dentition; According to the relative positions of the maxillary dentition and the mandibular dentition in the image data picture, the spatial position and posture of the digital three-dimensional mandibular model relative to the maxillary model are determined; wherein, for each image data picture in which the mandible is located at a different position, the spatial position and posture of a mandibular model is correspondingly determined; The determined spatial positions and postures of the mandibular model are analyzed to obtain a motion trajectory of the mandibular model relative to the maxillary model.

2. The mandibular movement modeling method according to claim 1, characterized in that: The analyzing the determined spatial positions and postures of the mandibular model to obtain the motion trajectory of the mandibular model relative to the maxillary model includes: Analyze each determined spatial position and posture of the mandibular model to obtain a basic movement direction of the mandibular model relative to the maxillary model, wherein the basic movement direction includes a mouth opening direction, a lateral jaw movement direction, and a sagittal forward and backward movement direction representing the mandibular bone; The motion trajectory is determined based on the basic motion direction.

3. The mandibular movement modeling method according to claim 2, characterized in that: The step of analyzing the determined spatial positions and postures of the mandibular model to obtain a basic movement direction of the mandibular model relative to the maxillary model includes: using a kernel principal component analysis method to analyze and obtain the basic movement direction.

4. The method for modeling mandibular movement according to claim 3, characterized in that: The analysis using the kernel principal component analysis method includes: Use the preset kernel function to obtain the covariance matrix; Performing eigenvalue decomposition on the covariance matrix to obtain N eigenvalues ​​and corresponding eigenvectors; wherein N is a natural number greater than 3; The first M eigenvectors with the largest eigenvalues ​​are selected from the N eigenvalues ​​as the M basic motion directions; wherein M is a natural number less than or equal to N and greater than 3.

5. The method for modeling mandibular movement according to claim 2, characterized in that: The step of analyzing the determined spatial positions and postures of the mandibular model to obtain a basic movement direction of the mandibular model relative to the maxillary model includes: analyzing the spatial positions and postures through a neural network to obtain the basic movement direction.

6. The method for modeling mandibular movement according to claim 5, characterized in that: The neural network adopts a pre-trained neural network based on a multi-layer perceptron.

7. The method for modeling mandibular movement according to claim 2, characterized in that: The determining the motion trajectory based on the basic motion directions includes: linearly combining the basic motion directions of the mandible obtained through analysis to obtain the motion trajectory of the compound motion of the mandible.

8. The method for modeling mandibular movement according to claim 1, characterized in that: Determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative positions of the maxillary dentition and the mandibular dentition in the image data picture includes: Extracting first visual feature data of the dentition part in the image data picture; Combine the maxillary model and the mandibular model, project the dentition parts of the combined maxillary model and the mandibular model on an imaging plane corresponding to preset camera parameters, adjust the relative posture of the maxillary model and / or the mandibular model, project each relative posture respectively, and obtain second visualization feature data after each projection respectively; Find the second visualization feature data that matches the first visualization feature data, and determine the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative posture corresponding to the second visualization feature data that matches the first visualization feature data.

9. The method for modeling mandibular movement according to claim 1, characterized in that: Determining the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model according to the relative positions of the maxillary dentition and the mandibular dentition in the image data picture includes: Marking first-class tooth feature points on the dentition part in the image data picture respectively to obtain a plurality of first feature points, and marking the first-class tooth feature points in the maxillary model and the mandibular model respectively to obtain a plurality of second feature points, wherein the number of the first feature points is greater than or equal to 3 and are not on the same straight line, and the number of the second feature points is greater than or equal to 3 and are not on the same straight line; Combine the maxillary model and the mandibular model, adjust the relative positions of the maxillary model and the mandibular model, respectively obtain the combined models corresponding to the relative positions, change the observation angle of the combined model, and respectively obtain the projection position of each second feature point of each combined model relative to the camera plane at each observation angle, wherein the camera plane is perpendicular to the observation angle; The projection position of the second feature point aligned with the position of each first feature point is found, and the spatial position and posture of the digitized three-dimensional mandibular model relative to the maxillary model is determined according to the relative posture of the upper and lower jaws in the combined model corresponding to the projection position.

10. The method for modeling mandibular movement according to claim 1, characterized in that: Before obtaining the motion trajectory of the mandibular model relative to the maxillary model, the method includes: obtaining an orthodontic plan for the patient; After obtaining the motion trajectory of the mandibular model relative to the maxillary model, the method includes: According to the maxillary model, the mandibular model and the motion trajectory, collision detection of the upper and lower mandibular models is performed at each stage in the orthodontic plan.

11. The method for modeling mandibular movement according to claim 10, characterized in that: The orthodontic plan includes: a tooth arrangement plan and a jaw position adjustment plan.

12. The method for modeling mandibular movement according to claim 1, characterized in that: After obtaining the motion trajectory of the mandibular model relative to the maxillary model, the method includes: When it is necessary to obtain an adjusted position of the mandibular model, the mandibular model is adjusted according to the motion trajectory to obtain the adjusted position and posture of the mandibular model.

13. The method for modeling mandibular movement according to claim 1, characterized in that: The image data pictures include: photos or X-rays.

14. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the mandibular movement modeling method according to any one of claims 1 to 13.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for modeling mandibular movement according to any one of claims 1 to 13 is implemented.

Images