Method and system for generating 3D model of face and hair on basis of artificial intelligence

Abstract

A method and a system for generating a 3D model of a face and hair on the basis of artificial intelligence are provided. The method for generating a 3D model, according to an embodiment of the present invention, acquires a captured image of a person, reconstructs a 3D model of a face by using the captured image, reconstructs a 3D model of hair by using the captured image, and merges the reconstructed 3D models into one 3D model. Therefore, 3D models for a facial part and hair can be separated and generated through methods appropriate for each 3D model by using structured light pattern images, and thus more realistic 3D human models can be generated and utilized.

Description

Method and system for generating 3D models of face and hair based on artificial intelligence

[0001] The present invention relates to an artificial intelligence-based 3D model generation technology, and more particularly, to a method and system for generating a 3D model of a person from captured images of the person.

[0002] Figure 1 illustrates the results of a conventional 3D facial model restoration. This 3D model is restored from a facial image. Technological advancements have made it possible to model even pores and fine wrinkles with such precision.

[0003] However, it can be confirmed that the hair part is not properly reflected in the 3D model. This is analyzed to be because the shape of hair is irregular and has detailed characteristics that are difficult to observe with capture equipment.

[0004] In particular, since the deviation depending on the color is severe, the capture performance for black hair tends to be significantly lower. This is analyzed to be because the general capture equipment uses a camera or lidar, so it is not able to properly determine the depth of black space.

[0005] The present invention has been devised to solve the above problems, and an object of the present invention is to provide a method and system for creating a more realistic 3D human model by distinguishing 3D models for the face and hair using structured light pattern images and creating them in an appropriate manner for each.

[0006] A method for generating a 3D model according to one embodiment of the present invention for achieving the above object includes the steps of: acquiring an image captured of a person; reconstructing a 3D model of a face using the captured image; reconstructing a 3D model of hair using the captured image; and merging the reconstructed 3D models into a single 3D model.

[0007] The acquisition step may be to acquire structured light pattern images using a structured light 3D scanner.

[0008] Structured light 3D scanners can be installed in photo booths.

[0009] The step of reconstructing a 3D model for a face may include a step of recognizing a structured light pattern from acquired structured light pattern images; and a step of restoring a 3D model for the face by combining the structured light pattern images based on the recognition result.

[0010] The step of reconstructing a 3D model for a face may further include a step of removing noise from the reconstructed 3D model; a step of filling holes in the 3D model from which noise has been removed; and a step of smoothing the 3D model from which holes have been filled.

[0011] The method for generating a 3D model according to the present invention may further include a step of calibrating the camera and the projector when the position of at least one of the projector and the camera of the structured light 3D scanner has changed.

[0012] The step of reconstructing a 3D model of hair may be to restore a 3D model of hair using the NeRF (Neural Radiance Fields) technique.

[0013] 3D models of the face and 3D models of the hair can be represented as point cloud data.

[0014] The merging step may be to merge the 3D model for the face and the 3D model for the hair by matching their positions and scales through feature point matching.

[0015] According to another aspect of the present invention, a 3D model generation system is provided, comprising: an image capture device for acquiring an image captured of a person; a 3D face model generation unit for reconstructing a 3D model of a face using the captured image; a 3D hair model generation unit for reconstructing a 3D model of hair using the captured image; and a 3D model merging unit for merging the reconstructed 3D models into a single 3D model.

[0016] According to another aspect of the present invention, a method for generating a 3D model is provided, comprising: a step of reconstructing a 3D model of a face using a captured image of a person; a step of reconstructing a 3D model of hair using the captured image of the person; and a step of merging the reconstructed 3D models into a single 3D model.

[0017] According to another aspect of the present invention, a 3D model generation system is provided, comprising: a 3D face model generation unit that reconstructs a 3D model of a face using a captured image of a person; a 3D hair model generation unit that reconstructs a 3D model of hair using the captured image of the person; and a 3D model merging unit that merges the reconstructed 3D models into a single 3D model.

[0018] As described above, according to embodiments of the present invention, by distinguishing 3D models for the face and hair using structured light pattern images and generating them in an appropriate manner for each, a more realistic 3D human model can be generated and utilized.

[0019] In addition, according to embodiments of the present invention, after creating a 3D model for a facial portion, additional work is not required to supplement insufficiently expressed hair portions, thereby reducing additional costs (time, cost, manpower, etc.) incurred thereby, facilitating the use of digital characters.

[0020] Figure 1 is an example of a conventional 3D face model restoration result.

[0021] Figure 2 is a 3D model generation system according to one embodiment of the present invention;

[0022] Figure 3 shows image capture equipment installed in a lightweight photo booth.

[0023] Figure 4 is a method for reconstructing a 3D face model;

[0024] Figure 5 is a method for creating a 3D character model according to another embodiment of the present invention.

[0025] Hereinafter, the present invention will be described in more detail with reference to the drawings.

[0026] In an embodiment of the present invention, a method and system for generating 3D models of faces and hair based on artificial intelligence are presented. This technology utilizes structured light pattern images to distinguish 3D models of the face and hair, generates them in an appropriate manner for each, and then merges them to create a more realistic 3D human model.

[0027] FIG. 2 is a diagram illustrating a configuration of a 3D model generation system according to an embodiment of the present invention. The 3D model generation system according to an embodiment of the present invention is a system for generating a 3D model for a facial portion including hair, and as illustrated, is configured to include an image capture device (110), a 3D hair model generation unit (120), a 3D facial model generation unit (130), and a 3D model merging unit (140).

[0028] The image capture device (110) is a structured light 3D scanner that acquires structured light pattern images from various angles of a person from whom a 3D model is to be created. The image capture device (110) can be installed in a lightweight photo booth, as illustrated in FIG. 3.

[0029] The 3D hair model generation unit (120) reconstructs a 3D model of hair using structured light pattern images generated by the image capture device (110). At this time, the 3D hair model generation unit (120) can restore the 3D model of hair using the NeRF (Neural Radiance Fields) technique.

[0030] The 3D face model generation unit (130) reconstructs a 3D model of the face using structured light pattern images generated by the image capture device (110). The method of reconstructing a 3D face model by the 3D face model generation unit (130) will be described in detail later with reference to FIG. 4.

[0031] The 3D model for hair generated by the 3D hair model generation unit (120) and the 3D model for face generated by the 3D face model generation unit (130) are both expressed as point cloud data.

[0032] The 3D model merging unit (140) merges the 3D model for hair generated by the 3D hair model generating unit (120), which is a point cloud data format, and the 3D model for face generated by the 3D face model generating unit (130) into one 3D model.

[0033] For merging, the 3D model merging unit (140) matches the positions and scales of the 3D model for the face and the 3D model for the hair through feature point matching.

[0034] The method for reconstructing a 3D face model by the 3D face model generation unit (130) will be described in detail below with reference to FIG. 4. FIG. 4 is a diagram showing the flow of the method for reconstructing a 3D face model.

[0035] To reconstruct a 3D face model, the camera and projector of the structured light 3D scanner are first calibrated as shown in the figure (S210). Calibration is performed if the position of at least one of the projector and camera of the structured light 3D scanner has changed, and is omitted otherwise.

[0036] In the following step S210, structured light pattern images are acquired using the image capture device (110) on which calibration has been performed (S220). Then, the 3D face model generation unit (130) recognizes the structured light pattern in the structured light pattern images acquired in step S220 (S230), and based on the recognition result, combines the structured light pattern images to restore a 3D model of the face (S240).

[0037] Thereafter, the 3D face model generation unit (130) performs post-processing on the 3D model of the face generated in step S240. To this end, the 3D face model generation unit (130) first removes noise from the 3D model of the face generated in step S240 (S250).

[0038] The next 3D face model generation unit (130) performs hole filling to fill empty holes in the 3D model of the face from which noise has been removed in step S250 (S260). The empty holes are caused by the acquired images not being able to cover the entire field of view, and step S260 corrects this.

[0039] Finally, the 3D face model generation unit (130) smoothes the 3D model with the holes filled in step S260 (S270).

[0040] FIG. 5 is a diagram illustrating a flow of a 3D character model creation method according to another embodiment of the present invention.

[0041] To create a 3D human model, first, the image capture device (110) acquires structured light pattern images from various different angles of the human for which a 3D model is to be created (S310).

[0042] Then, the 3D hair model generation unit (120) reconstructs a 3D hair model using the NeRF (Neural Radiance Fields) technique for the structured light pattern images generated in step S310 (S320).

[0043] The next 3D face model generation unit (130) reconstructs a 3D face model using the structured light pattern images generated in step S310, and performs post-processing such as noise removal, hole filling, and smoothing (S330).

[0044] Finally, the 3D model merging unit (140) merges the 3D hair model generated in step S320 and the 3D face model generated in step S330 by matching their positions and scales through feature point matching (S340).

[0045] The generated 3D model can be used to create virtual 3D characters or avatars.

[0046] So far, a preferred embodiment of a method and system for generating 3D models of faces and hair based on artificial intelligence has been described in detail.

[0047] In the above example, a method is presented to distinguish 3D models for the face and hair using structured light pattern images, create them in different appropriate ways for each, and then merge them.

[0048] By this, the 3D modeling process for a person is simplified by enabling the 3D model of hair, which is difficult to obtain using conventional methods, to be obtained simultaneously with the process of obtaining a 3D face model, thereby enabling the acquisition of a more realistic digital character.

[0049] Additionally, since there is no need for additional work to supplement insufficiently expressed hair after creating a 3D model of the face, it can reduce additional costs (time, money, manpower, etc.) resulting from this, making it easier to utilize digital characters.

[0050] In addition, hair implemented through additional work is difficult to create in a sophisticated shape, so the more precisely the 3D capture of a person is made, the greater the sense of incongruity becomes. However, in the embodiment of the present invention, it is possible to create a more realistic digital character while simultaneously acquiring hair.

[0051] Meanwhile, it goes without saying that the technical idea of ​​the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

[0052] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. Step of obtaining an image capturing a person; A step of reconstructing a 3D model of a face using a captured image; A step of reconstructing a 3D model of hair using a captured image; A method for creating a 3D model, characterized by including a step of merging reconstructed 3D models into a single 3D model.

2. In claim 1, The acquisition phase is, A method for creating a 3D model, characterized by obtaining structured light pattern images using a structured light 3D scanner.

3. In claim 2, Structured light 3D scanners are, A method for creating a 3D model, characterized in that it is installed in a photo booth.

4. In claim 2, The steps to reconstruct a 3D model of the face are: A step of recognizing a structured light pattern from acquired structured light pattern images; A method for generating a 3D model, characterized by including a step of restoring a 3D model of a face by combining structured light pattern images based on the recognition result.

5. In claim 4, The steps to reconstruct a 3D model of the face are: Step 1: Remove noise from the restored 3D model; Step 1: Filling holes in a 3D model with noise removed; A method for generating a 3D model, characterized in that it further comprises a step of smoothing a 3D model with holes filled.

6. In claim 4, A method for creating a 3D model, characterized in that it further comprises a step of calibrating the camera and the projector when the position of at least one of the projector and the camera of the structured light 3D scanner has changed.

7. In claim 1, The steps to reconstruct the 3D model of the hair are: A 3D model creation method characterized by restoring a 3D model of hair using the NeRF (Neural Radiance Fields) technique.

8. In claim 1, 3D models for the face and 3D models for the hair, A method for creating a 3D model, characterized in that it is expressed as point cloud data.

9. In claim 1, The merge step is, A method for generating a 3D model, characterized by merging a 3D model for a face and a 3D model for hair by matching the positions and scales thereof through feature point matching.

10. Image capture equipment for acquiring images of people; A 3D face model generation unit that reconstructs a 3D model of a face using a captured image; A 3D hair model generation unit that reconstructs a 3D model of hair using a captured image; A 3D model generation system, characterized by including a 3D model merging unit that merges reconstructed 3D models into a single 3D model.

11. A step of reconstructing a 3D model of a face using a character capture image; A step of reconstructing a 3D model of hair using a character capture image; A method for creating a 3D model, characterized by including a step of merging reconstructed 3D models into a single 3D model.

12. A 3D face model generation unit that reconstructs a 3D model of a face using a character capture image; A 3D hair model generation unit that reconstructs a 3D model of hair using a character capture image; A 3D model generation system, characterized by including a 3D model merging unit that merges reconstructed 3D models into a single 3D model.

Images