Model alignment method, device and equipment

Abstract

The invention provides a model alignment method, device and equipment. The method comprises the following steps: acquiring a tooth model and a face model of a user, wherein the face model comprises a first face model and a second face model; segmenting a first rigid body region in the first face model and a second rigid body region in the second face model; performing point cloud data matching on the second rigid body region and the first rigid body region, and determining a first transformation matrix for aligning the second face model to the first face model; performing point cloud data matching on tooth structure areas of the tooth model and a second face model, and determining a second transformation matrix for aligning the tooth model to the second face model; and aligning the tooth model and the first face model according to the first transformation matrix and the second transformation matrix. By means of the method, the tooth model can be accurately aligned to the face model without tooth structure areas or with few tooth structure areas.

Description

Technical Field

[0001] The present application belongs to the field of three-dimensional scanning and relates to a model alignment method, device and equipment. Background Art

[0002] In the field of 3D digital medicine, the design of treatment plans and the evaluation of treatment effects are increasingly dependent on the alignment / registration technology of 3D models. Taking the field of oral medicine as an example, in the treatment process of digital smile design and bite reconstruction, it is crucial to ensure the precise alignment of the user's 3D tooth model with the 3D facial model.

[0003] However, when the aligned 3D facial model does not have a tooth structure area, related technologies often rely on the user's facial computed tomography (CT) data to align the 3D tooth model with the 3D facial model. When the user's facial CT scan data cannot be obtained, it will be difficult to accurately align the tooth model to the 3D facial model without a tooth structure area. Summary of the invention

[0004] In view of the above, it is necessary to provide a model alignment method, device and equipment that can solve the technical problem that it is difficult to accurately align a dental model to a three-dimensional facial model without a dental structure area due to reliance on CT scan data.

[0005] On the one hand, the present application provides a model alignment method, which includes: obtaining a user's dental model and facial model, the facial model including a first facial model and a second facial model, segmenting a first rigid body area in the first facial model and a second rigid body area in the second facial model, determining a first transformation matrix for aligning the second facial model to the first facial model based on matching point cloud data of the second rigid body area with the first rigid body area, determining a second transformation matrix for aligning the dental model to the second facial model based on matching point cloud data of the dental model and the dental structure area of ​​the second facial model, and aligning the dental model with the first facial model according to the first transformation matrix and the second transformation matrix.

[0006] In some embodiments of the present application, the dental structure between the first facial model and the second facial model includes the following situations: the first facial model does not include a dental structure area and the second facial model includes a dental structure area, or, both the first facial model and the second facial model have a dental structure area, and the dental structure area in the second facial model corresponds to more dental structures than the dental structure area in the first facial model.

[0007] In some embodiments of the present application, the first rigid body region includes multiple first target vertices, and segmenting the first rigid body region in the first facial model includes: identifying a first feature point in the facial vertices of the first facial model, transforming the first facial model from a face coordinate system to a standard coordinate system based on the first feature point, selecting the multiple first target vertices from the facial vertices based on the first feature point in the standard coordinate system, and determining the region formed by the multiple first target vertices as the first rigid body region.

[0008] In some embodiments of the present application, the multiple first target vertices include facial vertices to which a preset area of ​​the user belongs, and the preset area includes a forehead area and / or a nose bridge area.

[0009] In some embodiments of the present application, the second rigid body region includes multiple second target vertices, and determining the first transformation matrix includes: performing point matching on the first feature point with the second feature point of the second facial model to obtain multiple groups of point pairs, constructing a covariance matrix according to the coordinates of the multiple groups of point pairs, and obtaining an initial transformation matrix based on the covariance matrix, determining first point cloud data according to the multiple first target vertices in the first rigid body region, and determining second point cloud data according to the multiple second target vertices in the second rigid body region, and aligning the first point cloud data and the second point cloud data based on the initial transformation matrix to obtain the first transformation matrix.

[0010] In some embodiments of the present application, constructing a covariance matrix based on the coordinates of the multiple groups of point pairs includes: calculating the first centroid coordinates corresponding to each first feature point in the multiple groups of point pairs based on the coordinates of the first feature point in the multiple groups of point pairs, and calculating the second centroid coordinates corresponding to each second feature point in the multiple groups of point pairs based on the coordinates of the second feature point in the multiple groups of point pairs, calculating the updated coordinates of the first feature point based on the coordinates of each first feature point in the multiple groups of point pairs and the first centroid coordinates, and calculating the updated coordinates of the second feature point based on the coordinates of each second feature point in the multiple groups of point pairs and the second centroid coordinates, and calculating the covariance matrix based on the updated coordinates of the first feature point and the second feature point in the multiple groups of point pairs.

[0011] In some embodiments of the present application, aligning the dental model with the first facial model according to the first transformation matrix and the second transformation matrix includes: calculating the initial coordinates of each tooth vertex according to the second transformation matrix and the three-dimensional coordinates of each tooth vertex in the dental model, calculating the target coordinates of each tooth vertex according to the first transformation matrix and the initial coordinates of each tooth vertex, and completing the alignment of the dental model with the first facial model.

[0012] In some embodiments of the present application, the first facial model and the second facial model are acquired at different time points or at different time points of the scanned objects represented.

[0013] On the other hand, the present application provides a model alignment device, which includes: an acquisition module for acquiring a user's dental model and facial model, the facial model including a first facial model and a second facial model, a segmentation module for segmenting a first rigid body area in the first facial model and a second rigid body area in the second facial model, a determination module for determining a first transformation matrix for aligning the second facial model to the first facial model based on matching point cloud data of the second rigid body area with the first rigid body area, the determination module is also used to determine a second transformation matrix for aligning the dental model to the second facial model based on matching point cloud data of the dental model and the tooth structure area of ​​the second facial model, and an alignment module for aligning the dental model with the first facial model according to the first transformation matrix and the second transformation matrix.

[0014] On the other hand, the present application provides an electronic device, comprising: a memory storing at least one instruction; and a processor executing the at least one instruction to implement the model alignment method.

[0015] In the model alignment scheme provided in the embodiment of the present application, the first facial model may not include the tooth structure area, the second facial model includes the tooth structure area, the first rigid body area and the second rigid body area are areas in the first facial model and the second facial model that are not easily affected by expression changes or orthodontic effects, and the shapes of the first rigid body area and the second rigid body area are approximately the same, so even if the user's expression changes, so that the shapes of the second facial model and the partial area of ​​the first facial model are different, the first transformation matrix for aligning the second facial model to the first facial model can be accurately determined by matching the point cloud data of the second rigid body area with the first rigid body area. Since the second facial model includes the tooth structure area, the second transformation matrix for aligning the tooth model to the second facial model can be accurately determined by matching the point cloud data of the tooth model with the tooth structure area in the second facial model. According to the first transformation matrix and the second transformation matrix, the tooth model can be accurately aligned to the first facial model without the tooth structure area or with less tooth structure area without relying on CT scan data, so as to achieve matching of facial models with tooth models at different expressions or at different time points. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a model alignment method provided in one embodiment of the present application.

[0017] Figure 2 is a schematic diagram of a first facial model provided in an embodiment of the present application.

[0018] Figure 3 is a schematic diagram of a second facial model provided in an embodiment of the present application.

[0019] Figure 4 It is a schematic diagram of a tooth model provided in an embodiment of the present application.

[0020] Figure 5 It is a front view of the standard coordinate system provided by an embodiment of the present application.

[0021] Figure 6 It is a side view of a standard coordinate system provided in one embodiment of the present application.

[0022] Figure 7 This is a flowchart of a method for determining a first transformation matrix provided in one embodiment of the present application.

[0023] Figure 8 It is a flowchart of a method for determining a first transformation matrix provided in another embodiment of the present application.

[0024] Fig. 9 It is a functional module diagram of a model alignment device provided in one embodiment of the present application.

[0025] Fig.10 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0026] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0027] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. The following embodiments and features in the embodiments may be combined with each other without conflict.

[0028] In the field of 3D digital medicine, the design of treatment plans and the evaluation of treatment effects are increasingly dependent on the alignment / registration technology of 3D models. Taking the field of oral medicine as an example, in the treatment process of digital smile design and bite reconstruction, it is crucial to ensure the precise alignment of the user's 3D tooth model with the 3D facial model.

[0029] However, some alignment schemes in the related art often require that the aligned 3D facial model have a tooth structure area in order to accurately align the tooth model with the 3D facial model. When the aligned 3D facial model does not have a tooth structure area, other alignment schemes in the related art often rely on the user's facial computed tomography (CT) data to align the 3D tooth model with the 3D facial model. In this way, when the user's facial CT scan data cannot be obtained, it will be difficult to accurately align the tooth model to the 3D facial model that does not have a tooth structure area.

[0030] To solve this technical problem, the present application provides a model alignment method that can accurately align a tooth model to a facial model that has no tooth structure area or has less tooth structure area. The model alignment method provided in the present application can be applied to one or more electronic devices, and the electronic device can be a laptop, a mobile phone, a computer, a server, etc. The present application does not limit the type of electronic device.

[0031] like Figure 1 FIG. 1 is a flowchart of a model alignment method provided by an embodiment of the present application. According to different requirements, the order of each step in the flowchart can be adjusted according to actual requirements, and some steps can be omitted. The method is applied to electronic devices, such as Fig.10 An electronic device 10 is shown.

[0032] S11, obtaining a tooth model and a facial model of the user, where the facial model includes a first facial model and a second facial model.

[0033] In some embodiments of the present application, the dental model and the facial model may be a three-dimensional mesh model (mesh) of the same user or a three-dimensional mesh model of different users, and the present application does not impose any restrictions on this. The dental model and the facial model may be obtained by reconstructing the corresponding point cloud data or a series of image frames, respectively, or the dental model and / or the facial model may be obtained by repairing the reconstructed corresponding models. The scanning devices corresponding to the dental model and the facial model may be different. For example, the dental model may be obtained by scanning the user's oral cavity with an oral scanning device (intraoral scanner, extraoral scanner, tablet computer or mobile phone, etc.); it may also be obtained by a computer through a simulation algorithm, or it may be obtained by modifying and adjusting an existing dental model by a computer.

[0034] The first face model and the second face model may be obtained in the same or different ways. For example, they may be obtained by scanning the user's face with a face scanning device (face scanner, tablet computer, mobile phone or AR glasses, etc.); they may also be obtained by a computer through a simulation algorithm, or they may be obtained by modifying and adjusting an existing face model.

[0035] The acquisition time points between the first facial model and the second facial model or the time points of the scanned objects represented may be the same or different. Exemplarily, taking different acquisition time points as an example, the first facial model and the second facial model may be obtained by three-dimensional scanning and reconstruction of the user's face at different stages of the orthodontic process (dental correction), and the first facial model and the second facial model can reflect the facial features of multiple different time points in the orthodontic process, and can show the changes in the user's facial morphology as the orthodontic process progresses. Alternatively, taking the different time points of the scanned objects represented as an example, the first facial model may be obtained by three-dimensional scanning and reconstruction of the user's face before orthodontic treatment, and the second facial model may be reconstructed from the expected orthodontic effect diagram, wherein the expected orthodontic effect diagram may be predicted and generated based on the user's face before orthodontic treatment or the first facial model through computer simulation, prediction algorithm and other technologies, the first facial model can reflect the user's facial morphology before orthodontic treatment, and the second facial model can reflect the expected changes in the user's face after orthodontic treatment.

[0036] Among them, the method of reconstructing the tooth model, the first facial model and the second facial model through the corresponding point cloud data can refer to the point cloud fusion method in the relevant technology, and this application will not describe it in detail.

[0037] In some embodiments of the present application, the tooth structures between the first facial model and the second facial model may include the following situations: Situation 1: the first facial model does not include a tooth structure area, and the second facial model includes a tooth structure area; Situation 2: both the first facial model and the second facial model have a tooth structure area, wherein the tooth structure corresponding to the tooth structure area in the second facial model may be more than the tooth structure corresponding to the tooth structure area in the first facial model. Figure 2 , which is a schematic diagram of a first facial model provided by an embodiment of the present application. Figure 3 FIG. 1 is a schematic diagram of a second facial model provided by an embodiment of the present application. Figure 4 , which is a schematic diagram of a tooth model provided in an embodiment of the present application. Figure 1 The first facial model shown does not include a tooth structure region, and when the first facial model is acquired, the scanned object may keep its mouth closed, that is, in a state of no expression. Figure 2The second facial model shown includes a tooth structure area. When acquiring the second facial model, the scanned object can keep its mouth open, that is, have an expression, to expose as much tooth structure as possible. The scanned object can wear a mouth expander or not.

[0038] To improve matching accuracy, the dental and jaw states of the scanned object when the first facial model is acquired and the dental and jaw states of the scanned object when the second facial model is acquired can be the same, for example, both the upper and lower jaws are in the resting jaw position.

[0039] Among them, if the first facial model includes a tooth structure area, the alignment of the tooth model with the tooth structure area of ​​the first facial model can be completed more accurately. If the second facial model includes a larger tooth structure area, the alignment of the tooth model with the tooth structure area of ​​the second facial model can be completed faster and more accurately. Therefore, in order to improve the accuracy of the alignment, a three-dimensional face model corresponding to a larger tooth structure area can be obtained as the second facial model.

[0040] In some embodiments of the present application, the electronic device can obtain the tooth model and the facial model through various methods. For example, the electronic device can read the model data in the model file under the preset path to obtain the tooth model and the facial model, wherein the preset path can be customized, and the present application does not limit this; or, the electronic device can integrate the mouth scanning device and the face scanning device, the electronic device can directly perform the scanning of the user's mouth and face, and reconstruct the tooth model and the facial model respectively through the point cloud data obtained by the scanning; or, the electronic device can communicate with the mouth scanning device and the face scanning device, receive the point cloud data sent by the mouth scanning device and the point cloud data sent by the face scanning device, reconstruct the point cloud data sent by the mouth scanning device to obtain the tooth model, and reconstruct the point cloud data sent by the face scanning device to obtain the facial model. The above-mentioned method for obtaining the tooth model and the facial model is only an example, and the present application does not limit the method for obtaining the tooth model and the facial model.

[0041] In this embodiment, the three-dimensional face model corresponding to the larger tooth structure area is obtained as the second face model, which can improve the accuracy of aligning the tooth model with the second face model.

[0042] S12, segmenting a first rigid body region in the first facial model and a second rigid body region in the second facial model.

[0043] In some embodiments of the present application, the first rigid body area and the second rigid body area may be areas in the first facial model and the second facial model that are not easily affected by changes in expression or orthodontic effects, respectively, and the shapes of the first rigid body area and the second rigid body area may be the same or approximately the same.

[0044] In some embodiments of the present application, the electronic device may convert the first facial model and the second facial model to the same coordinate system, and segment the first rigid body region from the first facial model and the second rigid body region from the second facial model in the unified coordinate system. For example, the electronic device may convert the first facial model and the second facial model to a standard coordinate system, and segment the first rigid body region from the first facial model and the second rigid body region from the second facial model in the standard coordinate system. The standard coordinate system may be customized, and the present application does not limit this. In order to clearly illustrate the segmentation process of the first rigid body region and the second rigid body region, the standard coordinate system will be used as an example for explanation below.

[0045] The first rigid body area includes multiple first target vertices, and the electronic device segments the first rigid body area in the first facial model including: identifying a first feature point in the facial vertices of the first facial model, transforming the first facial model from a face coordinate system to a standard coordinate system based on the first feature point, selecting the multiple first target vertices from the facial vertices in the standard coordinate system based on the first feature point, and determining the area formed by the selected multiple first target vertices as the first rigid body area.

[0046] Among them, the facial vertices can be coordinate points with corresponding coordinates (for example, three-dimensional coordinates). The first feature points include but are not limited to: facial feature points such as the nose tip point, the eye corner point, the eye socket point and the chin point. The multiple first target vertices include facial vertices belonging to the user's preset area, and the preset area can be one or more. For example, the preset area can be a combination of one or more of the forehead area, the nose bridge area, the chin area and the like. For example, Figure 5 , which is a front view of a standard coordinate system provided by an embodiment of the present application. Figure 6 , which is a side view of a standard coordinate system provided by an embodiment of the present application. Figure 5-Figure 6 As shown, the X-axis of the standard coordinate system can be the direction from the right face to the left face, the Y-axis can be the direction from the chin to the head, and the Z-axis can be the direction from the back of the head to the front face.

[0047] The electronic device can use various methods to identify the first feature point in the facial vertex, and the present application does not limit the identification method. For example, the electronic device can use a feature point detection algorithm and a convolutional neural network to identify the first feature point in the facial vertex. Among them, the feature point detection algorithm can be a Harris corner detection algorithm and a Shi-Tomasi corner detection algorithm, etc., and the convolutional neural network can be a FaceNet network and an MTCNN network, etc.

[0048] The method by which the electronic device selects multiple first target vertices from facial vertices based on the first feature point can be customized, and this application does not impose any restrictions on this. For example, if the preset area includes the bridge of the nose area, the nose tip point and the eye socket point can be selected as the first feature point, and the electronic device can determine a circular area with the nose tip point as the center and a preset value as the radius in the first facial model, and use all facial vertices above the nose tip point and below the eye socket point in the circular area as the first target vertices corresponding to the bridge of the nose area, and determine the area formed by all the first target vertices corresponding to the bridge of the nose area as the first rigid body area.

[0049] In some embodiments of the present application, the electronic device can determine the horizontal plane and the longitudinal plane of the first facial model according to the first feature point, and align the first facial model with the standard coordinate system by adjusting the directions of the horizontal plane and the longitudinal plane, thereby transforming the first facial model from the face coordinate system to the standard coordinate system. The method for determining the horizontal plane and the longitudinal plane of the first facial model according to the first feature point can refer to the relevant technology.

[0050] The electronic device can convert the second facial model to a standard coordinate system according to the second feature point in the second facial model, and segment the second rigid body area under the standard coordinate system. Among them, the description of the second feature point can refer to the introduction of the first feature point, and the method of converting the second facial model to the standard coordinate system according to the second feature point in the second facial model can refer to the method of transforming the first facial model from the human face coordinate system to the standard coordinate system, and the present application will not repeat the description. The second rigid body area corresponds to the first rigid body area. For example, if the first rigid body area is the area corresponding to the nose bridge area of ​​the user in the first facial model, then the second rigid body area is the area corresponding to the nose bridge area of ​​the user in the second facial model, or if the first rigid body area is the area corresponding to the nose bridge area and forehead area of ​​the user in the first facial model, then the second rigid body area is the area corresponding to the nose bridge area and forehead area of ​​the user in the second facial model. The segmentation method of the second rigid body area can refer to the segmentation method of the first rigid body area, and the present application will not repeat the description. For example, the electronic device can determine multiple second target vertices from the facial vertices in the second facial model, and determine the area formed by the multiple second target vertices on the second facial model as the second rigid body area. The segmentation method for determining a plurality of second target vertices from the facial vertices in the second facial model may refer to the segmentation process of the first target vertex.

[0051] In this embodiment, areas of the first facial model and the second facial model that are not easily affected by expression changes or orthodontic effects are respectively used as the first rigid body area and the second rigid body area, thereby providing a basis for matching point cloud data of the second rigid body area with the first rigid body area in the following text.

[0052] S13, determining a first transformation matrix for aligning the second facial model to the first facial model based on matching the point cloud data of the second rigid body region and the first rigid body region.

[0053] In some embodiments of the present application, the first rigid body area corresponds to the first point cloud data, and the second rigid body area corresponds to the second point cloud data. The electronic device can obtain multiple groups of point pairs by performing point matching on the first feature point and the second feature point, generate an initial transformation matrix according to the coordinates of the multiple groups of point pairs, and align the second point cloud data to the first point cloud data according to the initial transformation matrix to obtain the first transformation matrix.

[0054] Each point pair includes a first feature point and a second feature point that correspond to each other, and the coordinates of the point pair may be the three-dimensional coordinates of the first feature point and the three-dimensional coordinates of the second feature point in the point pair.

[0055] In this embodiment, since the first rigid body region and the second rigid body region are regions in the first facial model and the second facial model that are not easily affected by expression changes or orthodontic effects and are not deformed, the shapes of the first rigid body region and the second rigid body region are the same or approximately the same. Even if the user's expression changes, making the shapes of the second facial model different from other parts of the first facial model, the first transformation matrix of aligning the second facial model to the first facial model can be accurately determined by matching the point cloud data of the second rigid body region and the first rigid body region.

[0056] S14, determining a second transformation matrix for aligning the tooth model to the second facial model based on matching point cloud data of the tooth model and the tooth structure region of the second facial model.

[0057] In some embodiments of the present application, the electronic device can convert the tooth model and the second facial model to the same coordinate system. For example, the second facial model has been converted to the standard coordinate system above, and the electronic device can convert the tooth model to the standard coordinate system so that the tooth model and the second facial model are in the same coordinate system. In order to clearly illustrate the segmentation process of the tooth structure area, the standard coordinate system will be used as an example for explanation below.

[0058] The electronic device can select an area of ​​preset width from the tooth structure of the second facial model as the tooth structure area in the standard coordinate system. The preset width can be customized, or the preset width can be determined by the tooth model, which is not limited in this application. The tooth structure area corresponds to the tooth model. For example, if the tooth model is the front tooth area, the tooth structure area is also the front tooth area.

[0059] In some embodiments of the present application, the electronic device can project the tooth model onto a two-dimensional plane to obtain two-dimensional coordinate points of the tooth, process the two-dimensional coordinate points of the tooth according to a preset processing method to obtain a two-dimensional curve, fit the dental arch line of the tooth model based on the parameters of the two-dimensional curve, and convert the tooth model to a standard coordinate system by adjusting the direction of the dental arch line, wherein the fitting method of the dental arch line and the adjustment method of the direction of the dental arch line can refer to the relevant technology. The tooth model corresponds to the third point cloud data, and the tooth structure area corresponds to the fourth point cloud data. For example, the electronic device can select a first preset number of first tooth feature points (vertices) from the tooth model in the standard coordinate system as the third point cloud data corresponding to the tooth model, or the electronic device can directly determine all the first tooth feature points in the tooth model as the third point cloud data corresponding to the tooth model. The electronic device can select a first preset number of second tooth feature points (vertices) from the tooth structure area in the standard coordinate system as the fourth point cloud data corresponding to the tooth structure area, or the electronic device can directly determine all the second tooth feature points in the tooth structure area as the third point cloud data corresponding to the tooth structure area. Among them, the first preset number can be customized, and this application does not limit this. For example, the first preset number may be 20. The first tooth feature point and the second tooth feature point correspond to each other and have the same meaning. For example, if the first tooth feature point includes a tooth cusp point, the second tooth feature point also includes a tooth cusp point.

[0060] In some embodiments of the present application, the electronic device can calculate an initial point cloud transformation matrix through the first tooth feature points and the second tooth feature points, and align the fourth point cloud data and the third point cloud data based on the initial point cloud transformation matrix, so as to determine the second transformation matrix for aligning the tooth model to the second facial model. The method for determining the second transformation matrix can refer to the method for generating the first transformation matrix in step S13, and the embodiments of the present application will not be repeated.

[0061] In this embodiment, by matching the third point cloud data with the fourth point cloud data, the second transformation matrix for aligning the tooth model to the second facial model can be accurately determined.

[0062] S15, aligning the tooth model and the first facial model according to the first transformation matrix and the second transformation matrix.

[0063] In some embodiments of the present application, the electronic device can calculate the initial coordinates of each tooth vertex in the tooth model based on the second transformation matrix and the three-dimensional coordinates of each tooth vertex, and calculate the target coordinates of each tooth vertex based on the first transformation matrix and the initial coordinates of each tooth vertex to complete the alignment of the tooth model with the first facial model.

[0064] Among them, the first transformation matrix includes a first rotation matrix and a first translation matrix, and the second transformation matrix includes a second rotation matrix and a second translation matrix. The electronic device can determine the first operation result between the three-dimensional coordinates of each tooth vertex and the second rotation matrix and the second translation matrix as the initial coordinates of the tooth vertex, and determine the second operation result between the initial coordinates of the tooth vertex and the first rotation matrix and the first translation matrix as the target coordinates of the tooth vertex.

[0065] For example, the calculation method of the target coordinates of each tooth vertex can refer to formula (1):

[0066] Y=R 1 (R 2 X+T 2 )+Y 1 ; (1)

[0067] Among them, Y represents the target coordinate of the tooth vertex, R 1 represents the first rotation matrix, R 2 represents the second rotation matrix, X represents the three-dimensional coordinates of the tooth vertices, T 2 represents the second translation matrix, T 1 represents the first translation matrix, R 2 X+T 2 Represents the initial coordinates of the tooth vertices.

[0068] In this embodiment, if the first facial model does not include a tooth structure region, the tooth model can be accurately aligned to the first facial model without a tooth structure region or with a small tooth structure region without relying on CT scan data according to the first transformation matrix and the second transformation matrix, so as to achieve matching between the face model and the tooth model at different expressions or at different time points. If the first facial model includes a tooth structure region, the alignment accuracy between the tooth model and the first facial model can be further improved.

[0069] In other embodiments of the present application, the electronic device may also splice the tooth model to the first facial model according to the target coordinates of each tooth vertex in the tooth model.

[0070] In this embodiment, since the target coordinates of each tooth vertex in the tooth model are the coordinates obtained after aligning the tooth model to the first facial model, the tooth model is spliced ​​to the first facial model according to the target coordinates, so that the spliced ​​first facial model can be made more realistic and natural. The spliced ​​first facial model can be applied to various scenarios, such as facial animation generation, virtual reality, medical simulation, etc.

[0071] In the model alignment scheme provided in the embodiment of the present application, the first facial model may not include the tooth structure area, the second facial model includes the tooth structure area, the first rigid body area and the second rigid body area are areas in the first facial model and the second facial model that are not easily affected by expression changes or orthodontic effects, and the shapes of the first rigid body area and the second rigid body area are approximately the same, so even if the user's expression changes, so that the shapes of the second facial model and the partial area of ​​the first facial model are different, the first transformation matrix for aligning the second facial model to the first facial model can be accurately determined by matching the point cloud data of the second rigid body area with the first rigid body area. Since the second facial model includes the tooth structure area, the second transformation matrix for aligning the tooth model to the second facial model can be accurately determined by matching the point cloud data of the tooth model with the tooth structure area in the second facial model. According to the first transformation matrix and the second transformation matrix, the tooth model can be accurately aligned to the first facial model without the tooth structure area or with less tooth structure area without relying on CT scan data, so as to achieve matching of the face model and the tooth model at different expressions or at different time points.

[0072] In some embodiments of the present application, Figure 7 , which is a flowchart of a method for determining a first transformation matrix provided in one embodiment of the present application.

[0073] S141, performing point matching on the first feature point and the second feature point of the second facial model to obtain multiple groups of point pairs.

[0074] In some embodiments of the present application, the electronic device determines the feature points corresponding to each other in the first feature point and the second feature point by performing point matching between the first feature point and the second feature point, thereby obtaining multiple sets of point pairs. The two feature points in each set of point pairs may be feature points with the smallest distance or a distance within a preset range, wherein the distance may be a Euclidean distance or a geodesic distance, which is not limited in the present application. The electronic device may use various methods to determine the feature points matching the second feature point from the first feature point, thereby obtaining multiple sets of point pairs.

[0075] For example, the electronic device can determine the feature points in the first feature point that match the second feature point by constructing a K-dimensional (KD) tree or an octree tree, thereby obtaining multiple sets of point pairs. The K-dimensional tree can assign a hierarchical structure to the first feature point and the second feature point. Each node in the K-dimensional tree represents a space, and each space point can be used to divide the data into two parts. This process will recursively continue to divide the remaining subspace until each feature point in the first feature point and the second feature point is covered. Octree is a space partitioning data structure that can recursively divide the first feature point and the second feature point into eight equal-sized cubic sub-regions. Each cubic sub-region is called an octree node. For each cubic sub-region, if the number of feature points contained therein exceeds a preset threshold, the sub-region is further recursively divided into eight sub-regions until the number of feature points in each sub-region is less than the preset threshold.

[0076] S142, constructing a covariance matrix according to the coordinates of the multiple groups of point pairs, and obtaining an initial transformation matrix based on the covariance matrix.

[0077] In some embodiments of the present application, the covariance matrix may be a 3*3 matrix. The electronic device may calculate the first centroid coordinates corresponding to each first feature point in the multiple point pairs according to the coordinates of the first feature point in the multiple point pairs, and calculate the second centroid coordinates corresponding to each second feature point in the multiple point pairs according to the coordinates of the second feature point in the multiple point pairs, calculate the updated coordinates of the first feature point according to the coordinates of each first feature point in the multiple point pairs and the first centroid coordinates, and calculate the updated coordinates of the second feature point according to the coordinates of each second feature point in the multiple point pairs and the second centroid coordinates, and calculate the covariance matrix according to the updated coordinates of the first feature point and the second feature point in the multiple point pairs.

[0078] For example, the electronic device may determine the first centroid coordinates as the average or weighted average of the coordinates of all the first feature points in the multiple groups of point pairs. If the coordinates of each first feature point are three-dimensional coordinates, the first centroid coordinates are also three-dimensional coordinates, and each coordinate component (X, Y, Z) in the three-dimensional first centroid coordinates can be calculated by the coordinate components corresponding to all the first feature points in the multiple groups of point pairs. The calculation method of the second centroid coordinates can refer to the calculation method of the first centroid coordinates, and this application will not repeat the description. The electronic device may determine the difference between the coordinates of each feature point in the multiple groups of point pairs and the corresponding centroid coordinates as the updated coordinates corresponding to the feature point. The method for calculating the covariance matrix based on the updated coordinates of the multiple groups of point pairs can refer to the method for calculating the covariance matrix based on the mean in the related art.

[0079] In some embodiments of the present application, the electronic device may decompose the covariance matrix using various methods to obtain an initial transformation matrix. For example, the electronic device may decompose the covariance matrix using methods such as singular value decomposition (SVD) and stochastic gradient descent (SGD) to obtain an initial transformation matrix.

[0080] In this embodiment, by solving the initial transformation matrix, a basis is provided for aligning the first point cloud data and the second point cloud data below.

[0081] S143, determining first point cloud data according to a plurality of first target vertices in the first rigid body region, and determining second point cloud data according to a plurality of second target vertices in the second rigid body region.

[0082] In some embodiments of the present application, the electronic device may determine a second preset number of first target vertices or a set of all first target vertices in the first rigid body region as the first point cloud data corresponding to the first rigid body region, and determine a second preset number of second target vertices or a set of all second target vertices in the second rigid body region as the second point cloud data corresponding to the second rigid body region. The second preset number may be custom set, and the present application does not limit this. For example, the second preset number may be 30.

[0083] In some embodiments of the present application, in order to ensure the accuracy of the data, the first point cloud data and the second point cloud data may be first subjected to processes such as noise filtering and down-sampling.

[0084] S144, aligning the first point cloud data and the second point cloud data based on the initial transformation matrix to obtain a first transformation matrix.

[0085] In some embodiments of the present application, aligning the first point cloud data with the second point cloud data based on the initial transformation matrix can be an iterative optimization process, through multiple iterations of alignment, calculating the updated initial transformation matrix until the condition for stopping iteration is met, and determining the updated initial transformation matrix as the first transformation matrix. Among them, the condition for stopping iteration can be customized, and the present application does not limit this. For example, the condition for stopping iteration can refer to the description of step S1444.

[0086] In this embodiment, since the first rigid body region and the second rigid body region are regions in the first facial model and the second facial model that are not easily affected by changes in expression or orthodontic effects, and the shapes of the first rigid body region and the second rigid body region are approximately the same, even if the user's expression changes, making the shapes of some regions of the second facial model different from those of the first facial model, the first transformation matrix of the second facial model aligned to the first facial model can be accurately determined by matching the point cloud data of the second rigid body region and the first rigid body region.

[0087] In some embodiments of the present application, Figure 8 FIG. 2 is a flowchart of a method for determining a first transformation matrix provided by another embodiment of the present application, comprising the following steps:

[0088] S1441: align the second point cloud data to the first point cloud data using the initial transformation matrix to obtain aligned second point cloud data.

[0089] In some embodiments of the present application, the electronic device may operate the three-dimensional coordinates of each second target vertex in the second point cloud data with the initial transformation matrix to obtain updated three-dimensional coordinates of each second target vertex, thereby completing the alignment of the second point cloud data with the first point cloud data.

[0090] Among them, the updated three-dimensional coordinates of each second target vertex can refer to the calculation method of the target coordinates in step S15, and this application will not repeat the description.

[0091] S1442, performing point matching on the aligned second point cloud data and the first point cloud data to obtain updated multiple groups of point pairs.

[0092] In some embodiments of the present application, the method of performing point matching on the aligned second point cloud data and the first point cloud data to obtain updated multiple groups of point pairs can refer to the generation process of multiple groups of point pairs in step S142.

[0093] S1443, constructing an updated covariance matrix using the updated coordinates of the multiple groups of point pairs, and decomposing the updated covariance matrix to obtain an updated initial transformation matrix.

[0094] In some embodiments of the present application, the method for constructing the updated covariance matrix and the method for decomposing the updated initial transformation matrix may refer to the description of the method for constructing the covariance matrix and the method for decomposing the initial transformation matrix in step S143.

[0095] S1444: Determine whether the difference between the multiple updated initial transformation matrices obtained by continuous solution is less than or equal to a preset difference threshold.

[0096] In some embodiments of the present application, the difference between the multiple updated initial transformation matrices may be the distance between the multiple updated initial transformation matrices, such as Euclidean distance and Manhattan distance. The preset difference threshold may be customized, and the present application does not limit this. For example, the preset difference threshold may be 0.01 (1%).

[0097] In this embodiment, if the difference between the multiple updated initial transformation matrices obtained by continuous solution is less than or equal to the preset difference threshold, step S1445 is executed; if the difference between the multiple updated initial transformation matrices obtained by continuous solution is greater than the preset difference threshold, the process returns to step S1441.

[0098] In other embodiments of the present application, the above-mentioned conditions for stopping iteration are only examples and are not limited to this in practical applications. For example, when the similarity between multiple updated initial transformation matrices obtained by continuous solution is greater than a preset similarity threshold, the iteration is stopped and the updated initial transformation matrix is ​​determined as the first transformation matrix; or, when the number of iterations reaches a preset number, the iteration is stopped and the updated initial transformation matrix is ​​determined as the first transformation matrix; or, when the solution time reaches a preset time threshold, the iteration is stopped and the updated initial transformation matrix is ​​determined as the first transformation matrix. The preset similarity threshold, preset number of times, and preset time threshold can be customized, and this application does not limit this. For example, the preset similarity threshold can be 0.98 (98%), the preset number of times can be 50 times, and the preset time threshold can be 0.5 hours.

[0099] S1445, stop iteration, and determine the updated initial transformation matrix as the first transformation matrix.

[0100] In this embodiment, if the difference between multiple updated initial transformation matrices obtained by continuous solution is less than or equal to a preset difference threshold, it is determined that the updated initial transformation matrix has converged, and thus the updated initial transformation matrix is ​​determined as the first transformation matrix, which can ensure the accuracy of the first transformation matrix.

[0101] like Fig. 9 , which is a functional module diagram of a model alignment device provided in an embodiment of the present application. The model alignment device 11 includes an acquisition module 110, a segmentation module 111, a determination module 112 and an alignment unit 113. The module / unit referred to in the present application refers to a Fig.10 A series of computer readable instruction segments acquired by the processor 103 in the embodiment and capable of completing fixed functions, which are stored in Fig.10 In the memory 102 in the present embodiment, the functions of each module / unit will be described in detail in the subsequent embodiments.

[0102] The acquisition module 110 is used to acquire the tooth model and the facial model of the user, where the facial model includes a first facial model and a second facial model.

[0103] In some embodiments of the present application, the tooth structure between the first facial model and the second facial model includes the following situations: the first facial model does not include a tooth structure area and the second facial model includes a tooth structure area, or the first facial model and the second facial model both have a tooth structure area, and the tooth structure area in the second facial model corresponds to more tooth structures than the tooth structure area in the first facial model. The acquisition time point or the time point of the scanned object represented by the first facial model and the second facial model are different.

[0104] The segmentation module 111 is used to segment a first rigid body region in the first facial model and a second rigid body region in the second facial model.

[0105] In some embodiments of the present application, the first rigid body area includes multiple first target vertices, and the segmentation module 111 is also used to identify a first feature point in the facial vertices of the first facial model, and according to the first feature point, transform the first facial model from the face coordinate system to the standard coordinate system. In the standard coordinate system, the multiple first target vertices are selected from the facial vertices based on the first feature point, and the area formed by the multiple first target vertices is determined as the first rigid body area.

[0106] In some embodiments of the present application, the multiple first target vertices include facial vertices belonging to a preset area of ​​the user, and the preset area includes a forehead area and / or a nose bridge area.

[0107] The determination module 112 is configured to determine a first transformation matrix for aligning the second facial model to the first facial model based on matching point cloud data of the second rigid body region and the first rigid body region.

[0108] In some embodiments of the present application, the second rigid body region includes multiple second target vertices, and the determination module 112 is also used to perform point matching on the first feature point with the second feature point of the second facial model to obtain multiple groups of point pairs, construct a covariance matrix according to the coordinates of the multiple groups of point pairs, and obtain an initial transformation matrix based on the covariance matrix, determine the first point cloud data according to the multiple first target vertices in the first rigid body region, and determine the second point cloud data according to the multiple second target vertices in the second rigid body region, and align the first point cloud data and the second point cloud data based on the initial transformation matrix to obtain a first transformation matrix.

[0109] In some embodiments of the present application, the determination module 112 is also used to calculate the first centroid coordinates corresponding to each first feature point in the multiple point pairs based on the coordinates of the first feature point in the multiple point pairs, and calculate the second centroid coordinates corresponding to each second feature point in the multiple point pairs based on the coordinates of the second feature point in the multiple point pairs, calculate the updated coordinates of the first feature point based on the coordinates of each first feature point in the multiple point pairs and the first centroid coordinates, and calculate the updated coordinates of the second feature point based on the coordinates of each second feature point in the multiple point pairs and the second centroid coordinates, and calculate the covariance matrix based on the updated coordinates of the first feature point and the second feature point in the multiple point pairs.

[0110] The determination module 112 is further configured to determine a second transformation matrix for aligning the tooth model to the second facial model based on matching point cloud data of the tooth model and the tooth structure region of the second facial model.

[0111] The alignment module 113 is used to align the tooth model and the first facial model according to the first transformation matrix and the second transformation matrix.

[0112] In some embodiments of the present application, the alignment module 113 is also used to calculate the initial coordinates of each tooth vertex according to the second transformation matrix and the three-dimensional coordinates of each tooth vertex in the tooth model, and calculate the target coordinates of each tooth vertex according to the first transformation matrix and the initial coordinates of each tooth vertex, so as to complete the alignment of the tooth model with the first facial model.

[0113] like Fig.10 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 10 can be a mobile phone, a tablet computer, a laptop computer, or other electronic device. The present embodiment of the present application does not impose any restrictions on the specific type of the electronic device.

[0114] like Fig.10 As shown, the electronic device 10 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104 and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102 and the input / output interface 104 through the bus 105.

[0115] The communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as universal serial bus (USB), controller area network bus (CAN, Controller Area Network). The wireless communication module may provide one or more wireless communication solutions such as wireless fidelity (wireless fidelity, Wi-Fi), Bluetooth (Bluetooth, BT), mobile communication network, frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR), etc.

[0116] The memory 102 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 103, and can be used to store executable programs (such as machine instructions) of other running programs, and can also be used to store user and application data. The random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0117] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 103. The non-volatile memory may include a disk storage device and a flash memory. For example, the flash memory may be a flash memory (Nand Flash).

[0118] The memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include multiple instructions, and when the multiple instructions are executed by the processor 103, the model alignment method executed on the electronic device 10 can be implemented.

[0119] In other embodiments, Fig.10 The electronic device 10 shown also includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10 .

[0120] The processor 103 may include one or more processing units, for example, the processor 103 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0121] The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute a computer program stored in the memory 102 to implement the above-mentioned model alignment method.

[0122] The input / output interface 104 is used to provide a channel for user input or output. For example, the input / output interface 104 can be used to connect various input and output devices, such as a mouse, keyboard, touch device, display screen, etc., so that the user can enter information or visualize information.

[0123] The bus 105 is at least used to provide a channel for mutual communication among the communication module 101 , the memory 102 , the processor 103 , and the input / output interface 104 in the electronic device 10 .

[0124] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0125] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the methods in the above-mentioned embodiments of the present application.

[0126] The computer-readable storage medium may be an internal memory of the electronic device described in the above embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device.

[0127] In some embodiments, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0129] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0131] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present application is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0132] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in this application can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A model alignment method, characterized in that: The method comprises: Acquire a tooth model and a facial model of the user, wherein the facial model includes a first facial model and a second facial model; Segmenting a first rigid body region in the first facial model and a second rigid body region in the second facial model; Determining a first transformation matrix for aligning the second facial model to the first facial model based on matching point cloud data of the second rigid body region and the first rigid body region; Determining a second transformation matrix for aligning the tooth model to the second facial model based on matching point cloud data of the tooth structure region of the tooth model and the second facial model; The tooth model and the first facial model are aligned according to the first transformation matrix and the second transformation matrix.

2. The model alignment method according to claim 1, characterized in that: The tooth structure between the first facial model and the second facial model includes the following situations: The first facial model does not include a tooth structure region and the second facial model includes a tooth structure region; or The first facial model and the second facial model both have tooth structure regions, and the tooth structure regions in the second facial model correspond to more tooth structures than the tooth structure regions in the first facial model.

3. The model alignment method according to claim 1, characterized in that: The first rigid body region includes a plurality of first target vertices, and the first rigid body region in the segmented first facial model includes: identifying first feature points among facial vertices of the first facial model; According to the first feature point, transforming the first facial model from a face coordinate system to a standard coordinate system; In the standard coordinate system, the plurality of first target vertices are selected from the facial vertices based on the first feature point, and an area formed by the plurality of first target vertices is determined as the first rigid body area.

4. The model alignment method according to claim 3, characterized in that: The multiple first target vertices include facial vertices to which a preset area of ​​the user belongs, and the preset area includes a forehead area and / or a nose bridge area.

5. The model alignment method according to claim 3, characterized in that: The second rigid body region includes a plurality of second target vertices, and the determination of the first transformation matrix includes: Performing point matching on the first feature point and the second feature point of the second facial model to obtain multiple groups of point pairs; Constructing a covariance matrix according to the coordinates of the plurality of groups of point pairs, and obtaining an initial transformation matrix based on the covariance matrix; Determine first point cloud data according to a plurality of first target vertices in the first rigid body region, and determine second point cloud data according to a plurality of second target vertices in the second rigid body region; Based on the initial transformation matrix, the first point cloud data and the second point cloud data are aligned to obtain the first transformation matrix.

6. The model alignment method according to claim 5, characterized in that: The step of constructing a covariance matrix according to the coordinates of the plurality of point pairs comprises: Calculating first centroid coordinates corresponding to each first feature point in the multiple point pairs according to the coordinates of the first feature points in the multiple point pairs, and calculating second centroid coordinates corresponding to each second feature point in the multiple point pairs according to the coordinates of the second feature points in the multiple point pairs; Calculating updated coordinates of the first feature points according to the coordinates of each first feature point in the plurality of point pairs and the first centroid coordinates, and calculating updated coordinates of the second feature points according to the coordinates of each second feature point in the plurality of point pairs and the second centroid coordinates; The covariance matrix is ​​calculated according to the updated coordinates of the first feature point and the second feature point in the plurality of point pairs.

7. The model alignment method according to claim 1, characterized in that: The aligning the tooth model with the first facial model according to the first transformation matrix and the second transformation matrix comprises: Calculating the initial coordinates of each tooth vertex according to the second transformation matrix and the three-dimensional coordinates of each tooth vertex in the tooth model; According to the first transformation matrix and the initial coordinates of each tooth vertex, the target coordinates of each tooth vertex are calculated to complete the alignment of the tooth model with the first facial model.

8. The model alignment method according to claim 1, characterized in that: The first facial model and the second facial model are acquired at different time points or at different time points of the scanned objects represented.

9. A model alignment device, characterized in that: The device comprises: An acquisition module, used to acquire a tooth model and a facial model of a user, wherein the facial model includes a first facial model and a second facial model; a segmentation module, configured to segment a first rigid body region in the first facial model and a second rigid body region in the second facial model; a determination module, configured to determine a first transformation matrix for aligning the second facial model to the first facial model based on matching point cloud data of the second rigid body region and the first rigid body region; The determination module is further configured to determine a second transformation matrix for aligning the tooth model to the second facial model based on matching point cloud data of the tooth model and the tooth structure region of the second facial model; An alignment module is used to align the tooth model and the first facial model according to the first transformation matrix and the second transformation matrix.

10. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one instruction; and A processor executes the at least one instruction to implement the model alignment method as described in any one of claims 1 to 8.

Images