Model alignment method and apparatus, and device

Abstract

The present application provides a model alignment method and apparatus, and a device. The method comprises: acquiring a dental model and a facial model of a user, the facial model comprising a first facial model and a second facial model; segmenting a first rigid region in the first facial model and a second rigid region in the second facial model; on the basis of the matching of point cloud data between the second rigid region and the first rigid region, determining a first transformation matrix for aligning the second facial model to the first facial model; on the basis of the matching of point cloud data between the dental model and a dental structure region of the second facial model, determining a second transformation matrix for aligning the dental model to the second facial model; and on the basis of the first transformation matrix and the second transformation matrix, aligning the dental model and the first facial model. By means of the described method, a dental model can be accurately aligned to a facial model that lacks dental structure regions or has relatively few dental structure regions.

Description

Model alignment method, device and equipment

[0001] The present disclosure claims priority to the Chinese patent application No. 202411147183.X, filed on August 20, 2024, and entitled "Model alignment method, device and equipment", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application belongs to the field of three-dimensional scanning, and relates to a model alignment method, device and equipment. BACKGROUND

[0003] In the field of three-dimensional digital medical treatment, the design of treatment plans and the evaluation of treatment effects are increasingly dependent on three-dimensional model alignment / registration technology. For example, in the field of oral medical treatment, it is crucial to ensure the accurate alignment of a user's three-dimensional tooth model and a three-dimensional face model in the treatment processes of digital smile design and occlusion reconstruction.

[0004] However, when the aligned three-dimensional face model does not have a tooth structure region, the related technology often relies on the user's face computed tomography (CT) data to align the three-dimensional tooth model with the three-dimensional face model. When the user's face CT scan data cannot be obtained, it is difficult to accurately align the tooth model to the three-dimensional face model without a tooth structure region. SUMMARY

[0005] In view of the above, it is necessary to provide a model alignment method, device and equipment that can solve the technical problem of difficulty in accurately aligning a tooth model to a three-dimensional face model without a tooth structure region due to reliance on CT scan data.

[0006] In one aspect, the present application provides a model alignment method, which comprises: obtaining a tooth model and a face model of a user, the face model comprising a first face model and a second face model, segmenting a first rigid body region in the first face model and a second rigid body region in the second face model, determining a first transformation matrix for aligning the second face model to the first face model based on matching point cloud data of the second rigid body region and the first rigid body region, determining a second transformation matrix for aligning the tooth model to the second face model based on matching point cloud data of the tooth model and a tooth structure region of the second face model, and aligning the tooth model and the first face model according to the first transformation matrix and the second transformation matrix.

[0007] In some embodiments of the present application, the tooth structure between the first face model and the second face model includes a case that the first face model does not include a tooth structure region and the second face model includes the tooth structure region, or both the first face model and the second face model have the tooth structure region, and the tooth structure corresponding to the tooth structure region in the second face model is more than the tooth structure corresponding to the tooth structure region in the first face model.

[0008] In some embodiments of the present application, the first rigid region includes a plurality of first target vertices, and the segmentation of the first rigid region in the first face model includes: identifying a first feature point in a face vertex of the first face model, transforming the first face model from a face coordinate system to a standard coordinate system according to the first feature point, selecting a plurality of first target vertices from the face vertex based on the first feature point in the standard coordinate system, and determining a region formed by the plurality of first target vertices as the first rigid region.

[0009] In some embodiments of the present application, the plurality of first target vertices includes a face vertex to which a preset region of the user belongs, and the preset region includes a forehead region and / or a nose bridge region.

[0010] In some embodiments of the present application, the second rigid region includes a plurality of second target vertices, and the determination of the first transformation matrix includes: performing point matching between the first feature point and a second feature point of the second face model to obtain a plurality of groups of point pairs, constructing a covariance matrix according to coordinates of the plurality of groups of point pairs, and obtaining an initial transformation matrix based on the covariance matrix, determining first point cloud data according to the plurality of first target vertices in the first rigid region, and determining second point cloud data according to the plurality of second target vertices in the second rigid region, aligning the first point cloud data and the second point cloud data based on the initial transformation matrix to obtain the first transformation matrix.

[0011] In some embodiments of the present application, the construction of the covariance matrix according to the coordinates of the plurality of groups of point pairs includes: calculating a first centroid coordinate corresponding to each first feature point in the plurality of groups of point pairs according to coordinates of the first feature point, and calculating a second centroid coordinate corresponding to each second feature point in the plurality of groups of point pairs according to coordinates of the second feature point, calculating an updated coordinate of the first feature point according to the coordinate of the first feature point and the first centroid coordinate, and calculating an updated coordinate of the second feature point according to the coordinate of the second feature point and the second centroid coordinate, and calculating the covariance matrix according to the updated coordinates of the first feature point and the second feature point in the plurality of groups of point pairs.

[0012] In some embodiments of the present application, aligning the tooth model and the first face model according to the first transformation matrix and the second transformation matrix comprises: calculating initial coordinates of each tooth vertex according to the second transformation matrix and three-dimensional coordinates of each tooth vertex in the tooth model, calculating target coordinates of each tooth vertex according to the first transformation matrix and the initial coordinates of each tooth vertex, and completing the alignment of the tooth model and the first face model.

[0013] In some embodiments of the present application, the time points of acquisition or the time points of representing the scanned object between the first face model and the second face model are different.

[0014] In another aspect, the present application provides a model alignment device, the device comprising: an acquisition module configured to acquire a tooth model and a face model of a user, the face model comprising a first face model and a second face model, a segmentation module configured to segment a first rigid region in the first face model and a second rigid region in the second face model, a determination module configured to determine a first transformation matrix for aligning the second face model to the first face model based on matching point cloud data of the second rigid region and the first rigid region, and determine a second transformation matrix for aligning the tooth model to the second face model based on matching point cloud data of a tooth structure region of the tooth model and the second face model, and an alignment module configured to align the tooth model and the first face model according to the first transformation matrix and the second transformation matrix.

[0015] In another aspect, the present application provides an electronic device, the electronic device comprising: a memory storing at least one instruction; and a processor executing the at least one instruction to implement a model alignment method.

[0016] In another aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium storing a computer program, the computer program being executed by a processor in an electronic device to implement a model alignment method.

[0017] In the model alignment solution provided in the embodiments of the present application, the first face model can not include a tooth structure region, the second face model includes the tooth structure region, the first rigid body region and the second rigid body region are regions in the first face model and the second face model that are not easily affected by expression changes or not easily affected by orthodontic effects, the shapes of the first rigid body region and the second rigid body region are approximately the same, and therefore even if the expression of the user changes, so that the shape of the part of the second face model and the first face model is not the same, the first transformation matrix of the alignment of the second face model to the first face model can be accurately determined by matching the point cloud data of the second rigid body region and the first rigid body region. Since the second face model includes the tooth structure region, the second transformation matrix of the alignment of the tooth model to the second face model can be accurately determined by matching the point cloud data of the tooth model and the tooth structure region in the second face model. According to the first transformation matrix and the second transformation matrix, the tooth model can be accurately aligned to the first face model without a tooth structure region or with fewer tooth structure regions, and the matching of the face model and the tooth model at different expressions or different time points can be achieved without relying on CT scan data. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0020] FIG. 1 is a flowchart of a model alignment method provided by an embodiment of the present application.

[0021] FIG. 2 is a schematic diagram of a first face model provided by an embodiment of the present application.

[0022] FIG. 3 is a schematic diagram of a second face model provided by an embodiment of the present application.

[0023] FIG. 4 is a schematic diagram of a tooth model provided by an embodiment of the present application.

[0024] FIG. 5 is a front view of a standard coordinate system provided by an embodiment of the present application.

[0025] FIG. 6 is a side view of a standard coordinate system provided by an embodiment of the present application.

[0026] FIG. 7 is a flowchart of a determination method of a first transformation matrix provided by an embodiment of the present application.

[0027] FIG. 8 is a flowchart of a method for determining a first transformation matrix according to another embodiment of the present application.

[0028] FIG. 9 is a functional block diagram of a model alignment apparatus according to an embodiment of the present application.

[0029] FIG. 10 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] It should be noted that the term "at least one" in the present application means one or more, and the term "multiple" means two or more than two. The term "and / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application are used to distinguish similar objects, and are not intended to describe a specific order or sequence.

[0031] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of the words "exemplary" or "for example" is intended to present concepts in a concrete manner. The embodiments described below and the features in the embodiments can be combined with each other without conflict, if possible.

[0032] In the field of three-dimensional digital medical treatment, the design of treatment plans and the evaluation of treatment effects are increasingly dependent on three-dimensional model alignment / registration technology. For example, in the field of oral medical treatment, it is crucial to ensure the accurate alignment of a three-dimensional tooth model and a three-dimensional face model in the treatment process of digital smile design and occlusal reconstruction.

[0033] However, in some alignment solutions in the related art, it is often required that the three-dimensional face model to be aligned has a tooth structure region to accurately align the tooth model with the three-dimensional face model. When the three-dimensional face model to be aligned does not have a tooth structure region, some other alignment solutions in the related art often need to rely on the user's face computed tomography (CT) data to align the three-dimensional tooth model with the three-dimensional face model. Thus, when the user's face CT scan data cannot be obtained, it is difficult to accurately align the tooth model to the three-dimensional face model without a tooth structure region.

[0034] To solve this technical problem, the embodiments of the present application provide a model alignment method, which can accurately align a tooth model to a face model without or with less tooth structure region. The model alignment method provided by the embodiments of the present application can be applied to one or more electronic devices, which can be notebook computers, mobile phones, computers, servers and the like, and the type of electronic device is not limited in the present application.

[0035] As shown in FIG. 1, it is a flowchart of the model alignment method provided by an embodiment of the present application. According to different requirements, the order of each step in the flowchart can be adjusted according to actual requirements, and some steps can be omitted. The model alignment method is applied to an electronic device or a cloud, for example, the electronic device 10 shown in FIG. 10.

[0036] S11, obtaining a tooth model and a face model of a user, the face model including a first face model and a second face model.

[0037] In some embodiments of the present application, the tooth model and the face model can be three-dimensional mesh models of the same user, or three-dimensional mesh models of different users, and the tooth model and the face model can be obtained by the same device or different devices, which is not limited in the present application. The tooth model and the face model can be obtained by reconstructing corresponding point cloud data or a series of image frames, or the tooth model and / or the face model can be obtained by repairing the corresponding model after reconstruction. The scanning device corresponding to the tooth model and the face model can be the same or different. For example, the scanning device corresponding to the tooth model is an oral scanning device, and the scanning device corresponding to the face model can be a face scanning device. The face scanning device and the oral scanning device can be integrated or independently set.

[0038] The tooth model can be obtained by oral scanning of the user by an oral scanning device (intraoral scanner, extraoral scanner, CBCT scanner, CT scanner, X-ray scanner, tablet computer or mobile phone, etc.); it can also be obtained by a computer through a simulation algorithm, or it can be obtained by modifying and adjusting an existing tooth model by a computer.

[0039] The first face model and the second face model can be obtained by the same or different methods. For example, the face model can be obtained by face scanning of the user by a face scanning device (for example, a face scanning device, a tablet computer, a mobile phone or AR glasses, etc.); it can also be obtained by a computer through a simulation algorithm, or it can be obtained by modifying and adjusting an existing face model by a computer.

[0040] The acquisition time points between the first facial model and the second facial model or the time points of the scanned objects represented thereby can be the same or different. For example, the first facial model and the second facial model can be obtained by three-dimensional scanning and reconstruction of the user's face at different stages of the orthodontic process (also referred to as "tooth correction"). The first facial model and the second facial model can reflect the facial features at different time points during the orthodontic process and can show the changes in the user's facial morphology as the orthodontic process progresses. Alternatively, the first facial model can be obtained by three-dimensional scanning and reconstruction of the user's face before orthodontic treatment, and the second facial model can be obtained by reconstruction of the expected orthodontic effect diagram. The expected orthodontic effect diagram can be generated by computer simulation, prediction algorithm, etc. based on the user's face before orthodontic treatment or the first facial model. The first facial model can reflect the user's facial morphology before orthodontic treatment, and the second facial model can reflect the expected change in the user's facial morphology after orthodontic treatment.

[0041] The method of reconstructing the tooth model, the first facial model and the second facial model from the corresponding point cloud data can refer to the point cloud fusion method in the related art, which will not be described in detail herein.

[0042] In some embodiments of the present application, the tooth structure between the first facial model and the second facial model can include the following cases: case one, the first facial model does not include a tooth structure region, and the second facial model includes a tooth structure region; case two, the first facial model and the second facial model both have a tooth structure region, and the tooth structure corresponding to the tooth structure region in the second facial model can be more than the tooth structure corresponding to the tooth structure region in the first facial model. For example, as shown in FIG. 2, it is a schematic diagram of the first facial model according to an embodiment of the present application. As shown in FIG. 3, it is a schematic diagram of the second facial model according to an embodiment of the present application. As shown in FIG. 4, it is a schematic diagram of the tooth model according to an embodiment of the present application. The first facial model shown in FIG. 1 does not include a tooth structure region, and the scanned object can maintain a closed mouth state, i.e. an expressionless state, when the first facial model is acquired. The second facial model shown in FIG. 2 includes a tooth structure region, and the scanned object can maintain an open mouth state, i.e. an expressive state, when the second facial model is acquired, so as to expose the tooth structure as much as possible. The scanned object can wear a mouth expander or not.

[0043] To improve the matching accuracy, the jaw state of the scanned object when the first facial model is acquired and the jaw state of the scanned object when the second facial model is acquired can be the same, for example, the upper and lower jaws are in the rest position.

[0044] If the first face model includes a tooth structure region, the alignment of the tooth model and the tooth structure region of the first face model can be more accurate. If the second face model includes a larger tooth structure region, the alignment of the tooth model and the tooth structure region of the second face model can be faster and more accurate. Therefore, to improve the accuracy of alignment, a three-dimensional face model corresponding to a larger tooth structure region can be obtained as the second face model.

[0045] In some embodiments of the present application, the electronic device can obtain the tooth model and the face model by various methods. For example, the electronic device can read model data in a model file in a preset path to obtain the tooth model and the face model, where the preset path can be customized, and the present application does not limit this; or the electronic device can integrate an oral scanning device and a facial scanning device, and the electronic device can directly perform scanning on the user's oral cavity and face, and reconstruct the tooth model and the face model from the obtained point cloud data; or the electronic device can communicate with the oral scanning device and the facial scanning device, receive point cloud data sent by the oral scanning device and point cloud data sent by the facial scanning device, reconstruct the tooth model from the point cloud data sent by the oral scanning device, and reconstruct the face model from the point cloud data sent by the facial scanning device. The above-mentioned methods for obtaining the tooth model and the face model are only examples, and the present application does not limit the method for obtaining the tooth model and the face model.

[0046] In the present embodiment, obtaining a three-dimensional face model corresponding to a larger tooth structure region as the second face model can improve the accuracy of aligning the tooth model and the second face model.

[0047] S12, segmenting a first rigid region in the first face model and a second rigid region in the second face model.

[0048] In some embodiments of the present application, the first rigid region and the second rigid region can be regions in the first face model and the second face model that are not easily affected by expression changes or not easily affected by orthodontic effects, respectively, and the shapes of the first rigid region and the second rigid region can be the same or approximately the same.

[0049] In some embodiments of the present application, the electronic device can convert the first face model and the second face model to the same coordinate system, and segment the first rigid body region from the first face model and the second rigid body region from the second face model in the unified coordinate system. For example, the electronic device can convert the first face model and the second face model to a standard coordinate system, and segment the first rigid body region from the first face model and the second rigid body region from the second face model in the standard coordinate system. The standard coordinate system can be self-defined, and the present application does not limit this. In order to clearly illustrate the segmentation process of the first rigid body region and the second rigid body region, the standard coordinate system will be taken as an example in the following.

[0050] The first rigid body region includes a plurality of first target vertices. The electronic device segments the first rigid body region in the first face model, including: identifying a first feature point in a face vertex of the first face model, converting the first face model from a face coordinate system to a standard coordinate system according to the first feature point, selecting a plurality of first target vertices from the face vertex based on the first feature point in the standard coordinate system, and determining a region composed of the selected plurality of first target vertices as the first rigid body region.

[0051] The face vertex can be a coordinate point with corresponding coordinates (for example, three-dimensional coordinates). The first feature point includes, but is not limited to, a nose tip point, a corner of the eye point, an eye socket point, and a chin point, and the like. The plurality of first target vertices include face vertices belonging to a preset region of the user. The preset region can be one or more, for example, the preset region can be a combination of one or more of the forehead region, the nose bridge region, the chin region, and the like. For example, as shown in FIG. 5, it is a front view of the standard coordinate system provided by an embodiment of the present application. As shown in FIG. 6, it is a side view of the standard coordinate system provided by an embodiment of the present application. As shown in FIGS. 5-6, the X-axis of the standard coordinate system can be a direction from the right face to the left face, the Y-axis can be a direction from the chin to the head, and the Z-axis can be a direction from the back of the head to the front face.

[0052] The electronic device can identify the first feature point in the face vertex by using various methods, and the present application does not limit the identification method. For example, the electronic device can identify the first feature point in the face vertex by using a feature point detection algorithm and a convolutional neural network, and the like. The feature point detection algorithm can be a Harris corner detection algorithm and a Shi-Tomasi corner detection algorithm, and the like. The convolutional neural network can be a FaceNet network and an MTCNN network, and the like.

[0053] The method for the electronic device to select a plurality of first target vertices from the face vertices based on the first feature points can be customized, and the present application does not limit this. For example, if the preset region includes the nose bridge region, the tip of the nose and the eye socket points can be selected as the first feature points. The electronic device can determine, in the first face model, a circular region with the tip of the nose as the center and a preset value as the radius, select all face vertices above the tip of the nose and below the eye socket points in the circular region as the first target vertices corresponding to the nose bridge region, and determine the region formed by all the first target vertices corresponding to the nose bridge region as the first rigid region.

[0054] In some embodiments of the present application, the electronic device can determine a horizontal plane and a longitudinal plane of the first face model according to the first feature points, and align the first face model with the standard coordinate system by adjusting the directions of the horizontal plane and the longitudinal plane, so as to transform the first face model from the face coordinate system to the standard coordinate system. The method for determining the horizontal plane and the longitudinal plane of the first face model according to the first feature points can refer to related technologies.

[0055] The electronic device can convert the second face model to the standard coordinate system according to the second feature points in the second face model, and segment the second rigid region in the standard coordinate system. The description of the second feature points can refer to the introduction of the first feature points, and the method for converting the second face model to the standard coordinate system according to the second feature points in the second face model can refer to the method for transforming the first face model from the face coordinate system to the standard coordinate system, which will not be described again. The second rigid region corresponds to the first rigid region. For example, if the first rigid region is the region corresponding to the nose bridge region of the user in the first face model, the second rigid region is the region corresponding to the nose bridge region of the user in the second face model, or if the first rigid region is the region corresponding to the nose bridge region and the forehead region of the user in the first face model, the second rigid region is the region corresponding to the nose bridge region and the forehead region of the user in the second face model. The segmentation method of the second rigid region can refer to the segmentation method of the first rigid region, which will not be described again. For example, the electronic device can determine a plurality of second target vertices from the face vertices in the second face model, and determine the region formed by the plurality of second target vertices in the second face model as the second rigid region. The segmentation method of the plurality of second target vertices from the face vertices in the second face model can refer to the segmentation process of the first target vertices.

[0056] In the present embodiment, the regions in the first face model and the second face model that are not easily affected by expression changes or not easily affected by orthodontic effects are respectively selected as the first rigid region and the second rigid region, thereby providing a basis for matching the point cloud data of the second rigid region and the first rigid region in the following.

[0057] S13, determine a first transformation matrix for aligning the second face model to the first face model based on matching the point cloud data of the second rigid region and the first rigid region.

[0058] In some embodiments of the present application, the first rigid region corresponds to the first point cloud data, and the second rigid region corresponds to the second point cloud data. The electronic device can obtain a plurality of groups of point pairs by matching the first feature points and the second feature points, generate an initial transformation matrix according to the coordinates of the plurality of groups of point pairs, align the second point cloud data to the first point cloud data according to the initial transformation matrix, and obtain the first transformation matrix.

[0059] Each group of point pairs includes corresponding first feature points and second feature points, and the coordinates of the point pairs can be the three-dimensional coordinates of the first feature points and the three-dimensional coordinates of the second feature points in the point pairs.

[0060] In the present embodiment, since the first rigid region and the second rigid region are regions in the first face model and the second face model that are not easily affected by expression changes or orthodontic effects, the shapes of the first rigid region and the second rigid region are the same or approximately the same. Even if the user's expression changes, causing the shapes of other parts of the second face model and the first face model to be different, the first transformation matrix for aligning the second face model to the first face model can be accurately determined by matching the point cloud data of the second rigid region and the first rigid region.

[0061] S14, determine a second transformation matrix for aligning the tooth model to the second face model based on matching the point cloud data of the tooth model and the tooth structure region of the second face model.

[0062] In some embodiments of the present application, the electronic device can convert the tooth model and the second face model to the same coordinate system. For example, the second face model has been converted to the standard coordinate system as described above, and the electronic device can convert the tooth model to the standard coordinate system so that the tooth model and the second face model are in the same coordinate system. In order to clearly illustrate the segmentation process of the tooth structure region, the standard coordinate system will be taken as an example in the following description.

[0063] The electronic device can select a region of a preset width in the tooth structure of the second face model as the tooth structure region in the standard coordinate system. The preset width can be customized, or the preset width can be determined by the tooth model, which is not limited in the present application. The tooth structure region corresponds to the tooth model, for example, if the tooth model is the anterior tooth region, the tooth structure region is also the anterior tooth region.

[0064] In some embodiments of the present application, the electronic device can project the tooth model to a two-dimensional plane to obtain tooth two-dimensional coordinate points, process the tooth two-dimensional coordinate points according to a preset processing manner to obtain a two-dimensional curve, fit an arch line of the tooth model based on parameters of the two-dimensional curve, and convert the tooth model to a standard coordinate system by adjusting a direction of the arch line, wherein the fitting method of the arch line and the adjustment method of the direction of the arch line can refer to related technologies. The tooth model corresponds to third point cloud data, and the tooth structure region corresponds to fourth point cloud data. For example, the electronic device can select a first preset number of first tooth feature points (vertices) from the tooth model as the third point cloud data corresponding to the tooth model in the standard coordinate system, or the electronic device can directly determine all the first tooth feature points in the tooth model as the third point cloud data corresponding to the tooth model. The electronic device can select a first preset number of second tooth feature points (vertices) from the tooth structure region as the fourth point cloud data corresponding to the tooth structure region in the standard coordinate system, or the electronic device can directly determine all the second tooth feature points in the tooth structure region as the third point cloud data corresponding to the tooth structure region. The first preset number can be customized, and the present application does not limit this. For example, the first preset number can be 20. The first tooth feature points and the second tooth feature points correspond to and have the same meaning. For example, if the first tooth feature points include cusp points, the second tooth feature points also include cusp points.

[0065] In some embodiments of the present application, the electronic device can calculate an initial point cloud transformation matrix through the first tooth feature points and the second tooth feature points, align the fourth point cloud data and the third point cloud data based on the initial point cloud transformation matrix, and thereby determine a second transformation matrix for aligning the tooth model to the second face model. The determination method of the second transformation matrix can refer to the generation method of the first transformation matrix in step S13, and the embodiments of the present application will not be described again.

[0066] In the present embodiment, by matching the third point cloud data and the fourth point cloud data, the second transformation matrix for aligning the tooth model to the second face model can be accurately determined.

[0067] S15, aligning the tooth model and the first face model according to the first transformation matrix and the second transformation matrix.

[0068] In some embodiments of the present application, the electronic device can calculate an initial coordinate of each tooth vertex according to the second transformation matrix and the three-dimensional coordinates of each tooth vertex in the tooth model, calculate a target coordinate of each tooth vertex according to the first transformation matrix and the initial coordinate of each tooth vertex, and complete the alignment of the tooth model and the first face model.

[0069] The first transformation matrix includes a first rotation matrix and a first translation matrix, and the second transformation matrix includes a second rotation matrix and a second translation matrix. The electronic device can determine, as an initial coordinate of each tooth vertex, a first operation result between the three-dimensional coordinate of the tooth vertex and the second rotation matrix and the second translation matrix, and determine, as a target coordinate of the tooth vertex, a second operation result between the initial coordinate of the tooth vertex and the first rotation matrix and the first translation matrix.

[0070] For example, a calculation method of the target coordinate of each tooth vertex can refer to formula (1): Y=R1(R2X+T2)+T1; (1)

[0071] In the formula, Y represents the target coordinate of the tooth vertex, R1 represents the first rotation matrix, R2 represents the second rotation matrix, X represents the three-dimensional coordinate of the tooth vertex, T2 represents the second translation matrix, T1 represents the first translation matrix, and R2X+T2 represents the initial coordinate of the tooth vertex.

[0072] In this embodiment, if the first face model does not include the tooth structure region, the tooth model can be accurately aligned to the first face model without the tooth structure region or with less tooth structure region according to the first transformation matrix and the second transformation matrix, so as to realize the matching of the face model with different expressions or at different time points and the tooth model. If the first face model includes the tooth structure region, the alignment accuracy between the tooth model and the first face model can be further improved.

[0073] In other embodiments of the present application, the electronic device can further splice the tooth model to the first face model according to the target coordinate of each tooth vertex in the tooth model.

[0074] In this embodiment, since the target coordinate of each tooth vertex in the tooth model is a coordinate obtained after the tooth model is aligned to the first face model, the first face model after being spliced according to the target coordinate can be more realistic and natural. The spliced first face model can be applied to various scenes, such as expression animation generation, virtual reality, medical simulation, etc.

[0075] In the model alignment scheme provided in the embodiments of the present application, the first face model can not include a tooth structure region, the second face model includes a tooth structure region, the first rigid region and the second rigid region are regions in the first face model and the second face model that are not easily affected by expression changes or not easily affected by orthodontic effects, the shapes of the first rigid region and the second rigid region are approximately the same, and therefore, even if the expression of the user changes, causing the shape of the part of the second face model and the first face model to be different, the first transformation matrix of the second face model aligned to the first face model can be accurately determined by matching the point cloud data of the second rigid region and the first rigid region. Since the second face model includes a tooth structure region, the second transformation matrix of the tooth model aligned to the second face model can be accurately determined by matching the point cloud data of the tooth model and the tooth structure region in the second face model. According to the first transformation matrix and the second transformation matrix, the tooth model can be accurately aligned to the first face model without the tooth structure region or with fewer tooth structure regions, and the matching of the face model and the tooth model of different expressions or different time points can be realized.

[0076] In some embodiments of the present application, as shown in FIG. 7, it is a flowchart of the method for determining the first transformation matrix provided in an embodiment of the present application.

[0077] S141, performing point matching on the first feature points and the second feature points of the second face model to obtain a plurality of groups of point pairs.

[0078] In some embodiments of the present application, the electronic device determines the feature points corresponding to each other in the first feature points and the second feature points by performing point matching on the first feature points and the second feature points, and obtains a plurality of groups of point pairs. The two feature points in each group of point pairs can be the feature points with the minimum distance or the distance within a preset range, where the distance can be the Euclidean distance or the geodesic distance, which is not limited in the present application. The electronic device can determine the feature points matched with the second feature points from the first feature points by using various methods, thereby obtaining a plurality of groups of point pairs.

[0079] For example, the electronic device can determine the feature points in the first set of feature points that match the feature points in the second set of feature points by constructing a K-Dimensional (KD) tree or an Octree, and thus obtain a plurality of point pairs. The KD tree can assign a hierarchical structure to the feature points in the first set of feature points and the feature points in the second set of feature points, each node in the KD tree represents a block of space, and each spatial point can be used to split the data into two parts, and the process will continue to recursively split the remaining subspaces until each feature point in the first set of feature points and the second set of feature points is covered. The Octree is a spatial partitioning data structure that can recursively divide the feature points in the first set of feature points and the feature points in the second set of feature points into eight equal-sized cubic sub-regions. Each cubic sub-region is referred to as an Octree node. For each cubic sub-region, if the number of feature points contained therein exceeds a predetermined threshold, the sub-region is further recursively divided into eight sub-regions until the number of feature points in each sub-region is less than the predetermined threshold.

[0080] In S142, a covariance matrix is constructed according to the coordinates of the plurality of point pairs, and an initial transformation matrix is obtained based on the covariance matrix.

[0081] In some embodiments of the present application, the covariance matrix can be a 3*3 matrix. The electronic device can calculate a first centroid coordinate corresponding to each first feature point in the plurality of point pairs according to the coordinates of the first feature point, calculate a second centroid coordinate corresponding to each second feature point in the plurality of point pairs according to the coordinates of the second feature point, calculate an updated coordinate of each first feature point in the plurality of point pairs according to the coordinate of the first feature point and the first centroid coordinate, calculate an updated coordinate of each second feature point in the plurality of point pairs according to the coordinate of the second feature point and the second centroid coordinate, and calculate the covariance matrix according to the updated coordinates of the first feature points and the second feature points in the plurality of point pairs.

[0082] For example, the electronic device can determine the average or weighted average of the coordinates of all first feature points in the plurality of point pairs as the first centroid coordinate. If the coordinate of each first feature point is a three-dimensional coordinate, the first centroid coordinate is also a three-dimensional coordinate, and each coordinate component (X, Y, Z) in the three-dimensional first centroid coordinate can be calculated from the coordinate components corresponding to all first feature points in the plurality of point pairs. The calculation method of the second centroid coordinate can refer to the calculation method of the first centroid coordinate, which will not be repeated here. The electronic device can determine the difference between the coordinate of each feature point in the plurality of point pairs and the corresponding centroid coordinate as the updated coordinate of the feature point. The method for calculating the covariance matrix according to the updated coordinates of the plurality of point pairs can refer to the method for calculating the covariance matrix based on the mean in the related art.

[0083] In some embodiments of the present application, the electronic device can decompose the covariance matrix to obtain the initial transformation matrix by using various methods. For example, the electronic device can decompose the covariance matrix to obtain the initial transformation matrix by using a singular value decomposition (SVD), a stochastic gradient descent (SGD), or the like.

[0084] In the present embodiment, the initial transformation matrix is solved to provide a basis for aligning the first point cloud data and the second point cloud data hereinafter.

[0085] S143, determine the first point cloud data according to the plurality of first target vertices in the first rigid region, and determine the second point cloud data according to the plurality of second target vertices in the second rigid region.

[0086] In some embodiments of the present application, the electronic device can determine the set of the second preset number of first target vertices or all first target vertices in the first rigid region as the first point cloud data corresponding to the first rigid region, and determine the set of the second preset number of second target vertices or all second target vertices in the second rigid region as the second point cloud data corresponding to the second rigid region. The second preset number can be set by the user, which is not limited in the present application. For example, the second preset number can be 30.

[0087] In some embodiments of the present application, in order to ensure the accuracy of the data, the first point cloud data and the second point cloud data can be processed by denoising and down-sampling, etc.

[0088] S144, align the first point cloud data and the second point cloud data based on the initial transformation matrix to obtain the first transformation matrix.

[0089] In some embodiments of the present application, aligning the first point cloud data and the second point cloud data based on the initial transformation matrix can be an iterative optimization process. Through multiple iterations of alignment, an updated initial transformation matrix is calculated until the condition for stopping iteration is met, and the updated initial transformation matrix is determined as the first transformation matrix. The condition for stopping iteration can be set by the user, which is not limited in the present application. For example, the condition for stopping iteration can refer to the description of step S1444.

[0090] In the embodiment, since the first rigid body region and the second rigid body region are regions in the first face model and the second face model that are not easily affected by expression changes or orthodontic effects, and the shapes of the first rigid body region and the second rigid body region are approximately the same, even if the expression of the user changes such that the shape of the second face model is not the same as that of the part of the first face model, the first transformation matrix of the second face model aligned to the first face model can be accurately determined by matching the point cloud data of the second rigid body region and the first rigid body region.

[0091] In some embodiments of the present application, as shown in FIG. 8, it is a flowchart of a method for determining the first transformation matrix provided by another embodiment of the present application, including the following steps:

[0092] S1441, aligning the second point cloud data to the first point cloud data by using the initial transformation matrix to obtain the aligned second point cloud data.

[0093] In some embodiments of the present application, the electronic device can perform operations on the three-dimensional coordinates of each second target vertex in the second point cloud data and the initial transformation matrix to obtain the updated three-dimensional coordinates of each second target vertex, thereby completing the alignment of the second point cloud data and the first point cloud data.

[0094] The updated three-dimensional coordinates of each second target vertex can refer to the calculation method of the target coordinates in step S15, which will not be described again in the present application.

[0095] S1442, performing point matching on the aligned second point cloud data and the first point cloud data to obtain an updated plurality of point pairs.

[0096] In some embodiments of the present application, the method for performing point matching on the aligned second point cloud data and the first point cloud data to obtain an updated plurality of point pairs can refer to the generation process of the plurality of point pairs in step S142.

[0097] S1443, constructing an updated covariance matrix by using the coordinates of the updated plurality of point pairs, decomposing the updated covariance matrix to obtain an updated initial transformation matrix.

[0098] In some embodiments of the present application, the construction method of the updated covariance matrix and the decomposition method of the updated initial transformation matrix can refer to the description of the construction method of the covariance matrix and the decomposition method of the initial transformation matrix in step S143.

[0099] S1444, determining whether the difference between a plurality of updated initial transformation matrices obtained by continuous solving is less than or equal to a preset difference threshold.

[0100] In some embodiments of the present application, the difference between the plurality of updated initial transformation matrices can be a distance between the plurality of updated initial transformation matrices, such as an Euclidean distance, a Manhattan distance, or the like. The preset difference threshold can be customized, and the present application is not limited thereto. For example, the preset difference threshold can be 0.01 (1%).

[0101] In the present embodiment, if the difference between the plurality of updated initial transformation matrices obtained by continuous solving is less than or equal to the preset difference threshold, step S1445 is executed, and if the difference between the plurality of updated initial transformation matrices obtained by continuous solving is greater than the preset difference threshold, the flow returns to step S1441.

[0102] In other embodiments of the present application, the above-mentioned stopping iteration condition is only an example, and the present application is not limited thereto in actual application. For example, if the similarity between the plurality of updated initial transformation matrices obtained by continuous solving is greater than a preset similarity threshold, the iteration is stopped, and the updated initial transformation matrix is determined as the first transformation matrix; or, if the iteration number reaches a preset number of times, the iteration is stopped, and the updated initial transformation matrix is determined as the first transformation matrix; or, if the solving time reaches a preset time threshold, the iteration is stopped, and the updated initial transformation matrix is determined as the first transformation matrix. The preset similarity threshold, the preset number of times, and the preset time threshold can be customized, and the present application is not limited thereto. For example, the preset similarity threshold can be 0.98 (98%), the preset number of times can be 50 times, and the preset time threshold can be 0.5 hours.

[0103] S1445, stop iteration, and determine the updated initial transformation matrix as the first transformation matrix.

[0104] In the present embodiment, if the difference between the plurality of updated initial transformation matrices obtained by continuous solving is less than or equal to the preset difference threshold, it can be determined that the updated initial transformation matrix has converged, and thus the updated initial transformation matrix is determined as the first transformation matrix, which can ensure the accuracy of the first transformation matrix.

[0105] As shown in FIG. 9, it is a functional module diagram of a model alignment device provided by an embodiment of the present application. The model alignment device 11 includes an acquisition module 110, a segmentation module 111, a determination module 112, and an alignment unit 113. The module / unit referred to by the present application refers to a series of computer readable instruction segments capable of being acquired by the processor 103 in FIG. 10 and capable of completing a fixed function, which is stored in the memory 102 in FIG. 10. In the present embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0106] The acquisition module 110 is configured to acquire a tooth model and a face model of a user, the face model including a first face model and a second face model.

[0107] In some embodiments of the present application, the tooth structure between the first face model and the second face model includes the following cases: the first face model does not include the tooth structure region and the second face model includes the tooth structure region, or both the first face model and the second face model have the tooth structure region, and the tooth structure corresponding to the tooth structure region in the second face model is more than the tooth structure corresponding to the tooth structure region in the first face model. The acquisition time point or the time point of the scanned object represented by the first face model and the second face model is not the same.

[0108] The segmentation module 111 is configured to segment the first rigid body region in the first face model and the second rigid body region in the second face model.

[0109] In some embodiments of the present application, the first rigid body region includes a plurality of first target vertices, and the segmentation module 111 is further configured to identify a first feature point in the face vertex of the first face model, transform the first face model from the face coordinate system to the standard coordinate system according to the first feature point, select a plurality of first target vertices from the face vertex based on the first feature point in the standard coordinate system, and determine the region composed of the plurality of first target vertices as the first rigid body region.

[0110] In some embodiments of the present application, the plurality of first target vertices includes the face vertex to which the preset region of the user belongs, and the preset region includes the forehead region and / or the nose bridge region.

[0111] The determination module 112 is configured to determine the first transformation matrix for aligning the second face model to the first face model based on the point cloud data matching of the second rigid body region and the first rigid body region.

[0112] In some embodiments of the present application, the second rigid body region includes a plurality of second target vertices, and the determination module 112 is further configured to perform point matching of the first feature point and the second feature point of the second face model to obtain a plurality of groups of point pairs, construct a covariance matrix according to the coordinates of the plurality of groups of point pairs, and obtain an initial transformation matrix based on the covariance matrix, determine first point cloud data according to the plurality of first target vertices in the first rigid body region, and determine second point cloud data according to the plurality of second target vertices in the second rigid body region, align the first point cloud data and the second point cloud data based on the initial transformation matrix to obtain the first transformation matrix.

[0113] In some embodiments of the present application, the determining module 112 is further configured to calculate a first centroid coordinate corresponding to each first feature point in the plurality of sets of point pairs according to the coordinates of the first feature points in the plurality of sets of point pairs, and calculate a second centroid coordinate corresponding to each second feature point in the plurality of sets of point pairs according to the coordinates of the second feature points in the plurality of sets of point pairs, calculate an updated coordinate of each first feature point according to the coordinate of the first feature point and the first centroid coordinate, and calculate an updated coordinate of each second feature point according to the coordinate of the second feature point and the second centroid coordinate, and calculate a covariance matrix according to the updated coordinates of the first feature points and the second feature points in the plurality of sets of point pairs.

[0114] The determining module 112 is further configured to determine a second transformation matrix for aligning the tooth model to the second face model based on matching the point cloud data of the tooth structure region of the tooth model and the second face model.

[0115] The aligning module 113 is configured to align the tooth model to the first face model according to the first transformation matrix and the second transformation matrix.

[0116] In some embodiments of the present application, the aligning module 113 is further configured to calculate an initial coordinate of each tooth vertex according to the second transformation matrix and the three-dimensional coordinate of each tooth vertex in the tooth model, calculate a target coordinate of each tooth vertex according to the first transformation matrix and the initial coordinate of each tooth vertex, and complete the alignment of the tooth model to the first face model.

[0117] As shown in FIG. 10, it is a structural schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device 10 can be a mobile phone, a tablet computer, a notebook computer, a computer, a three-dimensional scanner (an oral scanning device and / or a face scanning device), a desktop computer, a smart television, a cloud server, a VR / AR device, etc. The specific type of the electronic device is not limited in the embodiments of the present application.

[0118] In some embodiments, the electronic device 10 can be integrated with the three-dimensional scanner (the oral scanning device and / or the face scanning device), or can be connected with the three-dimensional scanner (the oral scanning device and / or the face scanning device) through a wired or wireless manner.

[0119] As shown in FIG. 10, the electronic device 10 can include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104 and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102 and the input / output interface 104 through the bus 105.

[0120] The communication module 101 can include a wired communication module and / or a wireless communication module. The wired communication module can provide one or more of the following wired communication solutions: universal serial bus (USB), Controller Area Network (CAN) bus, etc. The wireless communication module can provide one or more of the following wireless communication solutions: wireless fidelity (Wi-Fi), Bluetooth (BT), mobile communication network, frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0121] The memory 102 can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). The random access memory can be directly readable and writable by the processor 103, and can be used to store executable programs (e.g., machine instructions) of programs that are currently running or other programs, and can also be used to store data of users and applications, etc. The random access memory can include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0122] The non-volatile memory can also store executable programs and store data of users and applications, etc., and can be loaded in advance into the random access memory for direct reading and writing by the processor 103. The non-volatile memory can include a magnetic disk storage device, a flash memory. For example, the flash memory can be a flash memory (Nand Flash).

[0123] The memory 102 is configured to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include a plurality of instructions which, when executed by the processor 103, can implement the model alignment method executed on the electronic device 10.

[0124] In other embodiments, the electronic device 10 as shown in FIG. 10 further comprises an external memory interface for connecting an external memory, so as to expand the storage capacity of the electronic device 10.

[0125] The processor 103 can include one or more processing units, for example: the processor 103 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0126] The processor 103 provides computing and control capabilities, for example, the processor 103 is used to execute the computer program stored in the memory 102, so as to implement the model alignment method described above.

[0127] The input / output interface 104 is used to provide a channel for user input or output, for example, the input / output interface 104 can be used to connect various input / output devices, such as a mouse, a keyboard, a touch device, a display screen, etc., so that the user can enter information, or make the information visualized.

[0128] The bus 105 is used to provide a communication channel between the communication module 101, the memory 102, the processor 103, and the input / output interface 104 in the electronic device 10.

[0129] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0130] The embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program includes program instructions, and the method implemented when the program instructions are executed can refer to the method in each of the embodiments of the present application.

[0131] The computer readable storage medium can be an internal storage of the electronic device, such as a hard disk or a memory of the electronic device, or an external storage of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like.

[0132] In some embodiments, the computer readable storage medium can include a program storage area and a data storage area, where the program storage area can store an operating system, an application required by at least one function, and the like, and the data storage area can store data created according to use of the electronic device, and the like.

[0133] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the above-described device embodiments are merely illustrative, for example, the division of the modules is merely a logical function division, and another division manner can be used in actual implementation.

[0134] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical units, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0135] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0136] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be regarded as limiting the claims involved.

[0137] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the present application can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

[0138] It should be finally pointed out that the above embodiments are only used to illustrate the technical solutions of the present application but not to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application. Industrial applicability

[0139] In the model alignment method provided by the present disclosure, the first face model can not include a tooth structure region, the second face model includes a tooth structure region, the first rigid region and the second rigid region are regions in the first face model and the second face model that are not easily affected by expression changes or not easily affected by orthodontic effects, the shapes of the first rigid region and the second rigid region are approximately the same, so even if the user's expression changes, causing the shape of the part of the second face model and the first face model to be different, the second rigid region and the first rigid region can be matched by point cloud data to accurately determine the first transformation matrix of the second face model aligned to the first face model. Since the second face model includes a tooth structure region, the tooth model and the tooth structure region in the second face model can be matched by point cloud data to accurately determine the second transformation matrix of the tooth model aligned to the second face model. According to the first transformation matrix and the second transformation matrix, the tooth model can be accurately aligned to the first face model without relying on CT scan data, the first face model does not have a tooth structure region or has fewer tooth structure regions, the matching of face models at different expressions or different time points and tooth models is realized, and it has strong industrial applicability.

Claims

1. A model alignment method, wherein, The method comprises: obtaining a tooth model and a face model of a user, the face model comprising a first face model and a second face model; segmenting a first rigid region in the first face model and a second rigid region in the second face model; determining a first transformation matrix for aligning the second face model to the first face model based on matching of point cloud data of the second rigid region and the first rigid region; determining a second transformation matrix for aligning the tooth model to the second face model based on matching of point cloud data of the tooth model and a tooth structure region of the second face model; aligning the tooth model and the first face model according to the first transformation matrix and the second transformation matrix.

2. The model alignment method of claim 1, wherein, The tooth structure between the first face model and the second face model comprises the following cases: the first face model does not comprise a tooth structure region and the second face model comprises a tooth structure region; or the first face model and the second face model both have tooth structure regions, and the tooth structure region in the second face model corresponds to more tooth structures than the tooth structure region in the first face model.

3. The model alignment method of claim 1, wherein, The first rigid region comprises a plurality of first target vertices, and the segmentation of the first rigid region in the first face model comprises: identifying a first feature point in a face vertex of the first face model; transforming the first face model from a face coordinate system to a standard coordinate system according to the first feature point; in the standard coordinate system, selecting the plurality of first target vertices from the face vertex based on the first feature point, and determining a region formed by the plurality of first target vertices as the first rigid region.

4. The model alignment method of claim 3, wherein, The plurality of first target vertices comprise face vertices belonging to a preset region of the user, and the preset region comprises a forehead region and / or a nose bridge region.

5. The model alignment method of claim 3, wherein, The second rigid region comprises a plurality of second target vertices, and the determination of the first transformation matrix comprises: performing point matching of the first feature point and a second feature point of the second face model to obtain a plurality of point pairs; constructing a covariance matrix according to coordinates of the plurality of point pairs, and obtaining an initial transformation matrix based on the covariance matrix; determining first point cloud data according to the plurality of first target vertices in the first rigid region, and determining second point cloud data according to the plurality of second target vertices in the second rigid region; aligning the first point cloud data and the second point cloud data based on the initial transformation matrix to obtain the first transformation matrix.

6. The model alignment method of claim 5, wherein, The construction of the covariance matrix according to the coordinates of the plurality of point pairs comprises: calculating a first centroid coordinate corresponding to each first feature point in the plurality of point pairs according to coordinates of the first feature point in the plurality of point pairs, and calculating a second centroid coordinate corresponding to each second feature point in the plurality of point pairs according to coordinates of the second feature point in the plurality of point pairs; According to the coordinates of each first feature point in the multiple sets of point pairs and the first centroid coordinates, updated coordinates of the first feature point are calculated, and according to the coordinates of each second feature point in the multiple sets of point pairs and the second centroid coordinates, updated coordinates of the second feature point are calculated; The covariance matrix is calculated according to the updated coordinates of the first feature points and the second feature points in the multiple sets of point pairs.

7. The model alignment method of claim 1, wherein, The aligning the dental model and the first face model according to the first transformation matrix and the second transformation matrix comprises: According to the second transformation matrix and the three-dimensional coordinates of each tooth vertex in the dental model, initial coordinates of the each tooth vertex are calculated; According to the first transformation matrix and the initial coordinates of the each tooth vertex, target coordinates of the each tooth vertex are calculated, and the aligning of the dental model and the first face model is completed.

8. The model alignment method of claim 1, wherein, The time points of acquisition or the time points of representing the scanned object between the first face model and the second face model are different.

9. A model alignment apparatus wherein, The apparatus comprises: An acquisition module configured to acquire a dental model and face models of a user, the face models comprising a first face model and a second face model; A segmentation module configured to segment a first rigid region in the first face model and a second rigid region in the second face model; A determination module configured to determine a first transformation matrix for aligning the second face model to the first face model based on matching of point cloud data of the second rigid region and the first rigid region; The determination module is further configured to determine a second transformation matrix for aligning the dental model to the second face model based on matching of point cloud data of the dental model and a tooth structure region of the second face model; An aligning module configured to align the dental model and the first face model according to the first transformation matrix and the second transformation matrix.

10. An electronic device, comprising: The electronic device comprises: A memory storing at least one instruction; and A processor executing the at least one instruction to implement the model aligning method according to any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer program stored on the computer readable storage medium is executed by the processor in the electronic device to implement the model aligning method according to any one of claims 1 to 8.

Images