Method and device for establishing three-dimensional human face model

Abstract

The present application provides a method and device for establishing a three-dimensional face model. The method includes obtaining the frontal face information of a user in a first time interval to establish a three-dimensional frontal face model and generate a frontal face plane image corresponding to the three-dimensional frontal face model, wherein the frontal face plane image includes a plurality of first feature points; obtaining the full face information of the user in a second time interval to establish a three-dimensional full face model and generate a full face plane image corresponding to the three-dimensional full face model, wherein the three-dimensional full face model has a specific part and the full face plane image includes a plurality of second feature points; according to the correspondence between the first feature point and the second feature point, superimposing a first block corresponding to the specific part of the three-dimensional full face model in the frontal face plane image to a second block corresponding to the specific part of the three-dimensional full face model in the full face plane image to obtain a complete full face plane image; and generating a three-dimensional face model according to the complete full face plane image.

Description

Technical Field

[0001] The present application relates to a method and device for establishing a three-dimensional face model. Background Art

[0002] The 3D face model is constructed based on the 2D face image. In the process of capturing the 2D face image, a positioning bracket is used to fix the user's head to increase the accuracy of the captured image.

[0003] However, if a positioning bracket is used to capture facial images, the positioning bracket will block the user's face, resulting in blocked areas in the scanned two-dimensional facial image, making the established three-dimensional facial model incomplete, reducing the quality of the acquired image, and also affecting the application of the three-dimensional facial model in dental correction or chin beauty. Summary of the invention

[0004] In view of this, the present application provides a method for establishing a three-dimensional face model, comprising: obtaining a user's frontal face information within a first time interval to establish a three-dimensional frontal face model, and generating a frontal face plane image corresponding to the three-dimensional frontal face model, wherein the frontal face plane image includes a plurality of first feature points. Obtaining the user's full face information within a second time interval to establish a three-dimensional full face model, and generating a full face plane image corresponding to the three-dimensional full face model, wherein the three-dimensional full face model has a specific part and the full face plane image includes a plurality of second feature points. Based on the correspondence between the first feature point and the second feature point, a first block corresponding to the specific part of the three-dimensional full face model in the frontal face plane image is superimposed on a second block corresponding to the specific part of the three-dimensional full face model in the full face plane image, so as to obtain a complete full face plane image. A three-dimensional face model is generated based on the complete full face plane image.

[0005] The present application further provides a device for establishing a three-dimensional face model, comprising a three-dimensional image capture device and a computing device. The three-dimensional image capture device is used to scan a user's front face in a first time interval to obtain front face information, and is used to scan the user's full face in a second time interval to obtain full face information. The computing device is electrically connected to the three-dimensional image capture device, and the computing device establishes a three-dimensional front face model according to the front face information, and generates a front face plane image corresponding to the three-dimensional front face model, and establishes a three-dimensional full face model according to the full face information, and generates a full face plane image corresponding to the three-dimensional full face model, wherein the three-dimensional full face model has a specific part; the computing device overlaps a first block corresponding to the specific part of the three-dimensional full face model in the front face plane image with a second block corresponding to the specific part of the three-dimensional full face model in the full face plane image according to the correspondence between a plurality of first feature points of the front face plane image and a plurality of second feature points of the full face plane image, so as to obtain a complete full face plane image, and then generates a three-dimensional face model according to the complete full face plane image.

[0006] In summary, the present application utilizes the method of feature point superposition to repair image defects in specific parts (such as the chin or forehead) and obtain a clear and complete three-dimensional face model, so as to solve the problem in the prior art that when the face is subjected to three-dimensional scanning, the contact parts will be obscured or deformed by the positioning bracket.

[0007] The details of other effects and embodiments of the present application are described below with reference to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the figures required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the figures described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other figures can be obtained based on these figures without paying creative work.

[0009] Figure 1 is a block diagram of an embodiment of a device according to the present application;

[0010] Figure 2 A schematic diagram of user positioning according to an embodiment of the present application;

[0011] Figure 3 is a flow chart of an embodiment of a method according to the present application;

[0012] Figure 4 is a schematic diagram of a three-dimensional frontal face model according to an embodiment of the present application;

[0013] Figure 5is a schematic diagram of a frontal plan view image according to an embodiment of the present application;

[0014] Figure 6 It is a schematic diagram of supporting a user's face with a positioning bracket according to an embodiment of the present application;

[0015] Figure 7 A schematic diagram of a 3D image capturing device moving left and right according to an embodiment of the present application;

[0016] Figure 8 is a schematic diagram of a three-dimensional full face model according to an embodiment of the present application;

[0017] Fig. 9 is a schematic diagram of a full-face plan view image according to an embodiment of the present application;

[0018] Fig.10 It is a schematic diagram of a plane image during superposition processing according to an embodiment of the present application;

[0019] Fig.11 is a schematic diagram of a plan view image of a human face according to an embodiment of the present application;

[0020] Fig.12 Schematic diagram of a three-dimensional face model according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, features and effects of the present application easier to understand, the following provides embodiments and drawings for describing the present application in detail.

[0022] Figure 1 See the block diagram of an embodiment of the device according to the present application. Figure 1 As shown, the 3D face model building device 10 includes a 3D image capturing device 12 and a computing device 14. The 3D image capturing device 12 corresponds to the face of the subject to capture and obtain at least one corresponding face image. The computing device 14 is electrically connected to the 3D image capturing device 12 to receive the face image and perform subsequent computing and application according to the face image. In one embodiment, the 3D image capturing device 12 and the computing device 14 can be independent devices, for example, the 3D image capturing device 12 is a 3D infrared camera, and the computing device 14 can be a mobile phone, a notebook computer, a tablet computer, a desktop computer, etc., but not limited thereto. In another embodiment, the 3D image capturing device 12 and the computing device 14 can be further integrated into an electronic device (not shown in the figure), the 3D image capturing device 12 is a 3D infrared camera built into the electronic device, and the computing device 14 can be a central processing unit, a microcontroller, etc. in the electronic device, and the electronic device is a mobile phone, a notebook computer, a tablet computer or a desktop computer, etc., but not limited thereto.

[0023] Figure 2 This is a schematic diagram of a user positioning according to an embodiment of the present application. Please also refer to Figure 1 and Figure 2 As shown, before starting to capture the facial image, the user 16 is positioned in front of the 3D image capturing device 12. In one embodiment, the 3D image capturing device 12 is disposed on a rotating bracket 18, and the motor device 20 is connected to the rotating bracket 18, so that the motor device 20 drives the rotating bracket 18 to move, and the rotating bracket 18 drives the 3D image capturing device 12 to rotate left and right to capture facial images at different angles, and the 3D facial model building device 10 further includes a positioning bracket 22, which is disposed on the motor device 20 and connected to the motor device 20, so that the motor device 20 drives the positioning bracket 22 to rise or fall.

[0024] In one embodiment, the positioning bracket 22 can be made of any material. In another embodiment, in order to facilitate the infrared light of the three-dimensional infrared camera used as the three-dimensional image capture device 12 to pass through, the positioning bracket 22 can also be made of a highly transparent material, for example, an acrylic material or glass material with a visible light transmittance of 92% and allowing infrared light with a wavelength less than 2800nm ​​to pass through. In one embodiment, the positioning bracket 22 can also be designed as a Y-shaped bracket to support the contact part 162 of the user 16 in a manner with a minimum contact area.

[0025] Figure 3 This is a flow chart of an embodiment of the method according to the present application. Please also refer to Figures 1 to 3 As shown, after the user 16 is positioned, the 3D image capturing device 12 scans the front face of the user 16 at a fixed point within a first time interval to capture and obtain front face information and transmit it to the computing device 14. The computing device 14 receives the front face information and performs computing to obtain the front face information. Figure 4 A three-dimensional frontal face model 24 is shown, and the three-dimensional frontal face model 24 includes a plurality of feature points 26 evenly distributed on the face, and a frontal face plane image 28 corresponding to the three-dimensional frontal face model 24 is generated (such as step S10), such as Figure 5 shown.

[0026] Then, the user's full face information is obtained in a second time interval to establish a three-dimensional full face model, and a full face plane image corresponding to the three-dimensional full face model is generated (such as step S12). Figures 1 to 7 As shown, the motor device 20 drives the positioning bracket 22 to rise to support the contact part 162 of the user 16, as shown in FIG. Figure 6As shown, the contact part 162 can be a part of the face such as the chin and forehead. In the present embodiment, the contact part 162 takes the chin as an example, but the present application is not limited thereto. Any part that can support the head without shaking or displacement is acceptable. In step S12, when the positioning bracket 22 has been used to support the contact part 162 of the face of the user 16, the three-dimensional image capture device 12 begins to scan and photograph the full face of the user 16 left and right within a second time interval, wherein the second time interval is greater than the first time interval. At this time, the motor device 20 drives the rotating bracket 18 to move left and right by a preset angle. In one embodiment, the preset angle is 99 degrees, so that the three-dimensional image capture device 12 on the rotating bracket 18 also moves left and right by 99 degrees each, so as to sequentially scan the full face of the user 16 at different angles, and transmit the full face information obtained by the shooting to the computing device 14. The computing device 14 receives the full face information and performs calculations to obtain the following. Figure 8 The three-dimensional full face model 30 shown in FIG. 1 has a specific part 36 and includes a plurality of feature points 32 evenly distributed on the face. The computing device 14 generates a full face plane image 34 (eg, Fig. 9 As shown in FIG. 1 ). Since the 3D image capturing device 12 has a larger viewing angle and a larger imaging range (including the left and right ears and the chin) in the second time interval, the established 3D full face model 30 and the full face plane image 34 also have a wider range. Among them, since the contact part 162 of the human face is supported by the positioning bracket 22, the contact position of the human face contacting the positioning bracket 22 corresponds to the specific part 36 of the 3D full face model. Therefore, the specific part 36 corresponding to the contact part 162 (chin) will have a corresponding imprint mark 361 and produce image defects.

[0027] In one embodiment, the first time interval is one second so as to keep the face of the user 16 still without using any positioning bracket 22. The second time interval is 12 seconds so as to use the positioning bracket 22 to stably support the face of the user 16, so that the 3D image capture device 12 has enough time to completely capture images of the face from all angles.

[0028] Please also refer to Figures 1 to 12 As shown, the frontal face plan view image 28 includes a plurality of first feature points 261-264 (the first feature points 261-264 are selected from the feature points 26), and the full face plan view image 34 includes a plurality of second feature points 321-324 (the second feature points 321-324 are selected from the feature points 32). The computing device 14 overlaps the first block 281 of the frontal face plan view image 28 with the second block 341 of the full face plan view image 34 (as shown in FIG. 1 ). Fig.10As shown in FIG. 1 , the area of ​​the first block 281 is equal to the area of ​​the second block 341, wherein the first block 281 and the second block 341 correspond to the specific part 36 of the three-dimensional full face model 30, and the second block 341 is the area where the imprint mark 361 is generated. The computing device 14 uses the skin color detection technology to identify the position of the imprint mark 361 of the specific part 36, and determines the range size of the second block 341 and its corresponding first block 281, so as to overlap the first block 281 that is clear and has no imprint mark on the second block 341 with the imprint mark 361, thereby obtaining a complete full face plane image 38 (as shown in step S14). Fig.11 Finally, the computing device 14 generates a complete and flawless three-dimensional face model 40 according to the complete full-face planar image 38 (as shown in step S16). Fig.12 shown.

[0029] The method disclosed in this application includes multiple steps or actions for implementing the described method. The steps in the above method can be interchangeable with each other without departing from the claims. For example, in Figure 3 In the flowchart shown, step S10 and step S12 can be interchanged with each other. In other words, first obtain Figure 5 The front plan image 28 or Fig. 9 The full face planar image 34 will not affect the subsequent calculation processing, and the superposition processing of the subsequent step S14 can still be performed without being affected by the interchange step.

[0030] In summary, the present application utilizes feature point superposition to repair image defects at specific locations (such as the chin or forehead) and obtain a clear and complete three-dimensional face model to improve the quality of images obtained during three-dimensional scanning.

[0031] The embodiments and / or implementation methods described above are only used to illustrate the preferred embodiments and / or implementation methods for realizing the technology of the present application, and are not intended to limit the implementation methods of the technology of the present application in any form. Any technical personnel in this field may make slight changes or modifications to other equivalent embodiments without departing from the scope of the technical means disclosed in the content of the present application, but they should still be regarded as technologies or embodiments that are essentially the same as the present application.

Claims

1. A method for establishing a three-dimensional face model, characterized in that: Include: Acquire the frontal face information of the user within a first time interval to establish a three-dimensional frontal face model, and generate a frontal face plane image corresponding to the three-dimensional frontal face model, wherein the frontal face plane image includes a plurality of first feature points; Fixing the face of the user by using a positioning bracket, obtaining the full face information of the user within a second time interval to establish a three-dimensional full face model, and generating a full face plane image corresponding to the three-dimensional full face model, wherein the three-dimensional full face model has a specific part and the full face plane image includes a plurality of second feature points; According to the correspondence between the first feature points and the second feature points, superimposing the first block of the frontal face plan view image corresponding to the specific part of the three-dimensional full face model onto the second block of the full face plan view image corresponding to the specific part of the three-dimensional full face model to obtain a complete full face plan view image; and Generate a three-dimensional face model based on the complete full-face planar image, The contact position of the human face with the positioning bracket corresponds to the specific part of the three-dimensional full face model, and the positioning bracket is not used in the first time interval.

2. The method for establishing a three-dimensional face model according to claim 1, characterized in that: The second time interval is greater than the first time interval.

3. The method for establishing a three-dimensional face model according to claim 1, characterized in that: The user's face is scanned using a three-dimensional image capture device.

4. The method for establishing a three-dimensional face model according to claim 3, characterized in that: In the step of obtaining the frontal face information of the user within the first time interval, the three-dimensional image capturing device performs fixed-point scanning to photograph the face of the user.

5. The method for establishing a three-dimensional face model according to claim 3, characterized in that: In the step of obtaining the full face information of the user within the second time interval, the three-dimensional image capturing device moves left and right at a preset angle to scan and capture the face of the user.

6. The method for establishing a three-dimensional face model according to claim 3, characterized in that: The three-dimensional image capturing device is a three-dimensional infrared camera.

7. A device for building a three-dimensional face model, characterized in that: Include: A three-dimensional image capturing device, which is used to scan the front face of the user in a first time interval to obtain front face information, and is used to scan the full face of the user in a second time interval to obtain full face information; and a computing device electrically connected to the three-dimensional image capturing device, wherein the computing device establishes a three-dimensional frontal face model according to the frontal face information and generates a frontal face plan view image corresponding to the three-dimensional frontal face model, and establishes a three-dimensional full face model according to the full face information and generates a full face plan view image corresponding to the three-dimensional full face model, wherein the three-dimensional full face model has a specific part, and the computing device overlaps a first block in the frontal face plan view image corresponding to the specific part of the three-dimensional full face model with a second block in the full face plan view image corresponding to the specific part of the three-dimensional full face model according to a correspondence between a first feature point of the frontal face plan view image and a second feature point of the full face plan view image, so as to obtain a complete full face plan view image, and generates a three-dimensional face model according to the complete full face plan view image; as well as a positioning bracket, used for fixing the face of the user in the second time interval, wherein the contact position of the face of the user in contact with the positioning bracket corresponds to the specific part of the three-dimensional full face model, The positioning bracket is not used during the first time interval.

8. The device for establishing a three-dimensional face model according to claim 7, characterized in that: The positioning bracket is made of a highly light-transmitting material.

9. The device for establishing a three-dimensional face model according to claim 7, characterized in that: The positioning bracket is a Y-shaped bracket.

10. The device for establishing a three-dimensional face model according to claim 7, characterized in that: The second time interval is greater than the first time interval.

11. The device for establishing a three-dimensional face model according to claim 7, characterized in that: The three-dimensional image capturing device scans and photographs the front face of the user at a fixed point within the first time interval.

12. The device for establishing a three-dimensional face model according to claim 7, characterized in that: The three-dimensional image capturing device moves left and right at a preset angle within the second time interval to scan and capture the full face of the user.

13. The device for establishing a three-dimensional face model according to claim 7, characterized in that: The three-dimensional image capturing device is a three-dimensional infrared camera.

Images