3D Microgeometry and Reflectance Modeling

Abstract

A system and method for three-dimensional (3D) microgeometry and reflectance modeling are provided. The system receives images including a first set of images of faces and a second set of images of faces. The faces in the first set of images and the second set of images are exposed to omnidirectional illumination and directional illumination, respectively. The system generates a 3D face mesh based on the received images and executes a set of skin-reflectance modeling operations to estimate a set of texture maps for the faces using the generated 3D face mesh and the second set of images. Based on the estimated set of texture maps, the system textures the generated 3D face mesh. Textures include an operation in which texture information, including microgeometry skin details and skin reflectance details of the estimated set of texture maps, is mapped onto the generated 3D face mesh.

Description

Technology Field

[0001] <Cross-reference / Integration by reference of related applications>

[0002] This application claims the benefit of priority of U.S. Patent Application No. 17 / 342,058, filed with the U.S. Patent and Trademark Office on June 8, 2021. Each of the applications referenced above is hereby incorporated herein by reference in its entirety.

[0003] Technology Field

[0004] Various embodiments of the present disclosure relate to three-dimensional (3D) modeling. More specifically, various embodiments of the present disclosure relate to systems and methods for 3D microgeometry and reflectance modeling. Background Technology

[0005] Advancements in the field of computer graphics have led to the development of various techniques for estimating the 3D shape and texture of human faces for photorealistic 3D face modeling. High-fidelity 3D models may be required in numerous industries, such as the entertainment, gaming, design, and healthcare industries. For example, in the entertainment and gaming industries, 3D face modeling can be utilized to generate photorealistic face animations or to develop 3D faces for game characters. Conventional imaging setups used for 3D modeling may have inconsistent lighting conditions. Consequently, 3D models constructed based on images acquired by conventional imaging setups may contain inaccuracies associated with the 3D shape of the model. Additionally, 3D models may contain poor surface-level details in terms of surface geometry and surface reflections.

[0006] The limitations and disadvantages of conventional and ordinary approaches will become apparent to those skilled in the art through a comparison of the described systems and some aspects of the disclosure, as presented with reference to the remainder of this application and the drawings.

[0007] A system and method for three-dimensional (3D) microgeometry and reflectance modeling is substantially provided as illustrated in at least one of the drawings and / or described in connection therewith, as more fully presented in the claims.

[0008] These and other features and advantages of the present disclosure can be understood from a review of the following detailed description of the present disclosure, together with the accompanying drawings in which similar reference numerals refer to similar parts throughout. Brief explanation of the drawing

[0009] FIG. 1 is a block diagram illustrating an exemplary network environment for three-dimensional (3D) microgeometry and reflectance modeling according to an embodiment of the present disclosure. FIG. 2 is a block diagram illustrating an exemplary system for 3D microgeometry and reflectance modeling according to an embodiment of the present disclosure. FIG. 3 illustrates an exemplary photogrammetry setup for 3D microgeometry and reflectance modeling according to an embodiment of the present disclosure. FIG. 4a illustrates exemplary images captured under omni-directional lighting conditions according to an embodiment of the present disclosure. FIG. 4b illustrates exemplary images captured under directional lighting conditions according to an embodiment of the present disclosure. FIGS. 5a, 5b, 5c, and 5d collectively illustrate exemplary operations for 3D microgeometry and reflectance modeling according to embodiments of the present disclosure. FIG. 6 is a flowchart illustrating an exemplary method for 3D microgeometry and reflectance modeling according to an embodiment of the present disclosure. Specific details for implementing the invention

[0010] The following described embodiments may be found in the disclosed system and method for three-dimensional (3D) microgeometry and reflectance modeling. Exemplary aspects of the present disclosure provide a system and method that may be configured to receive a plurality of images. The plurality of images may include a first set of images of a face and a second set of images of a face. According to an embodiment, the system may control a plurality of imaging devices to capture a plurality of images of a human subject's face from a plurality of viewpoints. In some embodiments, the system may simultaneously activate a set of flash units while the plurality of imaging devices are capturing the first set of images. The face in the first set of images may be exposed to omni-directional lighting. The system may further activate a set of flash units in a sequential pattern while the plurality of imaging devices are capturing the second set of images. The face in the second set of images may be exposed to directional lighting.

[0011] Based on a plurality of received images, the system may be configured to generate a 3D face mesh. The system may execute a set of skin-reflectance modeling operations to estimate a set of texture maps for the face by using the generated 3D face mesh and a second set of images. According to an embodiment, the set of skin-reflectance modeling operations may include a diffused reflection modeling operation, a specular separation operation, and a specular reflection modeling operation. In some embodiments, the system may execute the diffuse reflection modeling operation (e.g., based on a Lambertian light model) to generate a diffuse normal map and a diffuse albedo map of a human subject's face. The diffuse albedo map may be a first texture map of the estimated set of texture maps. In one or more embodiments, the system may perform a specular separation operation to separate specular reflection information from a second set of images based on a generated diffuse normal map and a generated diffuse albedo map. In some embodiments, the system may perform a specular reflection modeling operation (e.g., based on a Blinn-Phong light model) to generate a specular albedo map of a face, a specular normal map of a face, and a roughness map of a face. The specular albedo map, the specular normal map, and the roughness map may be referred to as second texture maps of an estimated set of texture maps.The system can texture a 3D face mesh generated based on an estimated set of texture maps (e.g., a first texture map and second texture maps). Texture creation may include mapping texture information, including microgeometric skin details and skin reflectance details of the estimated set of texture maps, onto the generated 3D face mesh.

[0012] In some conventional methods, images of a face can only be acquired under omnidirectional illumination. Consequently, these images may lack the appropriate information required to generate accurate texture maps for creating high-fidelity 3D models. However, the system of the present disclosure can control multiple imaging devices and activate a set of flash units to capture images under both omnidirectional and directional illumination. Images under omnidirectional illumination can be used to estimate the accurate 3D shape of the face, and images under directional illumination can be used to estimate texture maps that include both microgeometric skin details (e.g., pore-level details, ridges, and furrows) and skin reflectance details (e.g., specular albedo and roughness). The system can generate a high-fidelity and photorealistic 3D model of the face using both the accurate 3D shape and texture maps.

[0013] FIG. 1 is a block diagram illustrating an exemplary network environment for three-dimensional (3D) microgeometry and reflectance modeling according to an embodiment of the present disclosure. Referring to FIG. 1, a network environment (100) is illustrated. The network environment (100) may include a system (102), a plurality of imaging devices (104), a set of flash units (106), and a communication network (108). The system (102), the plurality of imaging devices (104), and the set of flash units (106) may be configured to communicate with each other through the communication network (108). In the network environment (100), for example, a human face (110) is illustrated. The plurality of imaging devices (104) may acquire a plurality of images (112) including a first set of images (114) and a second set of images (116) of the face (110).

[0014] The system (102) may include appropriate logic, circuitry, and interfaces that can be configured to receive a plurality of images (112) associated with a human face (110) (e.g., a first set of images (114) and a second set of images (116)). The system (102) may be further configured to generate a 3D face mesh (118) based on the received plurality of images (112) and to execute a set of skin-reflectivity modeling operations to generate texture maps for texturing the generated 3D face mesh (118). Texturing can generate microgeometric skin details and skin reflectivity details on the 3D face mesh (118). Examples of systems (102) may include, but are not limited to, mainframe machines, servers, computer workstations, game devices (e.g., game consoles), head-mounted displays (e.g., XR (eXtended Reality) headsets), wearable display devices, consumer electronic (CE) devices, or mobile computers.

[0015] A plurality of imaging devices (104) may include appropriate logic, circuitry, and interfaces that can be configured to capture a plurality of images (112) of a person's face (110) from corresponding viewpoints. A plurality of imaging devices (104) may be further configured to transmit the captured images (112) to a system (102). Examples of imaging devices may include, but are not limited to, image sensors, wide-angle cameras, action cameras, camcorders, digital cameras (e.g., DSLR (digital single reflex camera) or DSLM (digital single lens mirrorless)), camera phones, and / or any image capture device capable of capturing images in a number of formats and different frame rates.

[0016] A flash unit set (106) may be configured to generate a flash of light based on trigger signals generated by the system (102). The flash of light may be generated to illuminate a person's face (110). Examples of a flash unit set (106) may include, but are not limited to, built-in and pop-up camera flash units, dedicated camera flash units, macro ring light camera flash units, and hammerhead camera flash units.

[0017] It should be noted that the present disclosure is not limited to the implementation of a plurality of imaging devices (104) and a set of flash units as devices separate from the system (102). Accordingly, in some embodiments, a plurality of imaging devices (104) and a set of flash units may be included in the system (102) without departing from the scope of the present disclosure.

[0018] A communication network (108) may include a communication medium through which a system (102), a plurality of imaging devices (104), and a set of flash units (106) can communicate with each other. The communication network (108) may be either a wired connection or a wireless connection. Examples of the communication network (108) may include, but are not limited to, the Internet, a cloud network, a cellular or wireless mobile network (e.g., Long-Term Evolution and 5G New Radio), a Wi-Fi (Wireless Fidelity) network, a PAN (Personal Area Network), a LAN (Local Area Network), or a MAN (Metropolitan Area Network). Various devices in the network environment (100) may be configured to connect to the communication network (108) according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of TCP / IP (Transmission Control Protocol and Internet Protocol), UDP (User Datagram Protocol), HTTP (Hypertext Transfer Protocol), FTP (File Transfer Protocol), Zig Bee, EDGE, IEEE 802.11, Li-Fi (light fidelity), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless AP (access point), device-to-device communication, cellular communication protocols, and BT (Bluetooth) communication protocols.

[0019] In operation, the system (102) may be configured to control a plurality of imaging devices (104) to capture a plurality of images (112) from a plurality of corresponding viewpoints. The plurality of imaging devices (104) may be arranged at a corresponding first plurality of locations on a 3D structure. For example, the 3D structure may be a dome-shaped cage structure capable of providing sufficient space to accommodate at least one person. A set of flash units (106) may be arranged at a corresponding set of locations on the 3D structure. Each imaging device and each flash unit may be arranged on the 3D structure to surround a person inside the space within the 3D structure from a plurality of viewpoints. An example of such an arrangement is provided in FIG. 3.

[0020] A flash unit set (106) may be activated within the duration during which multiple imaging devices (104) capture multiple images (112) of the face (110). As an example, not limited to, the flash unit set (106) may be activated for two camera shots within a defined duration (~1.5 seconds). In the first shot, while multiple imaging devices (104) can capture the first set of images (114), the flash unit set (106) may be activated simultaneously to expose the face (110) to omnidirectional lighting. In the second shot, while multiple imaging devices (104) can capture the second set of images (116), the flash unit set (106) may be activated in a sequential pattern to expose the face (110) to directional lighting. Details regarding the capture of multiple images (112) may be further provided, for example, in FIG. 3.

[0021] At any time-instant, the system (102) may be configured to receive a plurality of images (112) (which may include a first set of images (114) and a second set of images (116)) from a plurality of imaging devices (104). In an embodiment, the system (102) may receive a plurality of images (112) from a server that maintains a repository of images from various sources. The faces (110) of the first set of images (114) may be exposed to omnidirectional illumination, and the faces (110) of the second set of images (116) may be exposed to directional illumination.

[0022] According to an embodiment, the system (102) may be further configured to generate a first 3D face mesh based on a received first image set (114). The first 3D face mesh may be a raw 3D scan of a face (110) and may include artifacts, such as spikes or pointed edges, large and small holes, and other shape irregularities. The system (102) may be configured to apply a set of model clean-up operations to the generated first 3D face mesh to obtain a refined first 3D face mesh. As an example, but not a limitation, these model clean-up operations may be applied to remove artifacts from the first 3D face mesh. The system (102) may further generate a second 3D face mesh based on a received second image set (116). For the 3D reconstruction of a 3D face mesh from 2D images, there are many techniques known to those skilled in the art. For example, both the first 3D face mesh and the second 3D face mesh may be generated using a photogrammetry-based method (such as SfM (structure from motion)), a method requiring stereoscopic images, or a method requiring monocular cues (such as SfS (shape from shading), photometric stereo, or SfT (shape from texture)). Details of these techniques have been omitted from this disclosure for brevity.

[0023] The system (102) can estimate an affine transformation between a refined first 3D face mesh and a generated second 3D face mesh. Then, the system (102) can apply the estimated affine transformation to the refined first 3D face mesh to generate a 3D face mesh (118). The generated 3D face mesh can be rigidly aligned with the generated second 3D face mesh and may be untextured. Details regarding the generation of the 3D face mesh (118) are further provided, for example, in FIGS. 5a and 5b.

[0024] By using the generated 3D face mesh (118) and the second image set (116), the system (102) can execute a set of skin-reflectivity modeling operations to estimate a set of texture maps for the face (110). According to an embodiment, the set of skin-reflectivity modeling operations may include diffuse reflection modeling operations, specular separation operations, and specular reflection modeling operations. Details regarding the execution of the set of skin-reflectivity modeling operations are provided, for example, in FIGS. 5c and 5d.

[0025] The system (102) can texture a 3D face mesh (118) generated based on an estimated texture map set. Texture mapping may include mapping texture information, including microgeometric skin details and skin reflectance details of the estimated texture map set, onto the generated 3D face mesh (118). Details regarding the texture mapping of the generated 3D face mesh (118) are further provided, for example, in FIG. 5d.

[0026] FIG. 2 is a block diagram illustrating an exemplary system for 3D microgeometry and reflectance modeling according to an embodiment of the present disclosure. FIG. 2 is described in relation to elements from FIG. 1. Referring to FIG. 2, a block diagram (200) of a system (102) is shown. The system (102) may include a circuit section (202), a memory (204), an input / output (I / O) device (206), and a network interface (208). The circuit section (202) may be coupled to communicate with the memory (204), the I / O devices (206), and the network interface (208).

[0027] The circuit section (202) may include appropriate logic, circuit sections, and interfaces that can be configured to execute program instructions associated with different operations to be executed by the system (102). The circuit section (202) may include one or more special processing units, each of which may be implemented as a separate processor. In an embodiment, one or more special processing units may be implemented as an integrated processor or a cluster of processors that collectively perform the functions of one or more special processing units. The circuit section (202) may be implemented based on a number of processor technologies known in the art. Exemplary implementations of the circuit section (202) may include, but are not limited to, x86-based processors, x64-based processors, GPUs (Graphics Processing Unit), RISC (Reduced Instruction Set Computing) processors, ASIC (Application-Specific Integrated Circuit) processors, co-processors (e.g., VPU (Vision Processing Unit)), CISC (Complex Instruction Set Computing) processors, microcontrollers, central processing units (CPUs), and / or combinations thereof.

[0028] The memory (204) may include appropriate logic, circuits, and interfaces that can be configured to store program instructions to be executed by the circuit (202). The memory (204) may be configured to store a plurality of images (112) (including a first set of images (114) and a second set of images (116)). The memory (204) may also be configured to store a generated 3D face mesh (118) and an estimated set of texture maps. Exemplary implementations of the memory (204) may include, but are not limited to, RAM (Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), HDD (Hard Disk Drive), SSD (Solid-State Drive), CPU cache, and / or SD (Secure Digital) card.

[0029] The I / O device (206) may include appropriate logic, circuitry, and interfaces that can be configured to receive input from a user and provide output based on the received input. The I / O device (206), which may include various input and output devices, may be configured to communicate with the circuitry (202). For example, the system (102) may receive user input through the I / O device (206) and control a plurality of imaging devices (104) to capture a plurality of images (112). The I / O device (206), such as a display, may render inputs and / or outputs, such as a generated 3D face mesh (118), an estimated texture map set, or a textured 3D face mesh. Examples of the I / O device (206) may include, but are not limited to, a touch screen, a display device, a keyboard, a mouse, a joystick, a microphone, and a speaker.

[0030] The network interface (208) may include suitable logic, circuits, and interfaces that can be configured to facilitate communication between the circuit (202), a plurality of imaging devices (104), and a set of flash units (106) through the communication network (108). The network interface (208) may be implemented using various known technologies to support wired or wireless communication between the communication network (108) and the system (102). The network interface (208) may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC (coder-decoder) chipset, a SIM (subscriber identity module) card, or a local buffer circuit.

[0031] The network interface (208) can be configured to enable wired or wireless communication with networks, such as the Internet, intranet or wireless networks, such as cellular telephone networks, wireless LAN (local area network), and MAN (metropolitan area network). Wireless communication may be configured to use one or more of a number of communication standards, protocols and technologies, such as GSM (Global System for Mobile Communications), EDGE (Enhanced Data GSM Environment), W-CDMA (wideband code division multiple access), LTE (Long Term Evolution), 5G NR, CDMA (code division multiple access), TDMA (time division multiple access), Bluetooth, Wi-Fi (Wireless Fidelity) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g or IEEE 802.11n), VoIP (voice over Internet Protocol), Li-Fi (light fidelity), Wi-MAX (Worldwide Interoperability for Microwave Access), protocols for email, instant messaging, and SMS (Short Message Service).

[0032] Functions or operations executed by the system (102) as described in FIG. 1 may be performed by the circuit unit (202). Operations executed by the circuit unit (202) are described in detail, for example, in FIG. 3, FIG. 4a, FIG. 4b, FIG. 5a, FIG. 5b, FIG. 5c, and FIG. 5d.

[0033] FIG. 3 is a diagram illustrating an exemplary photogrammetry setup for 3D microgeometry and reflectance modeling according to an embodiment of the present disclosure. FIG. 3 is described in relation to elements from FIG. 1 and FIG. 2. Referring to FIG. 3, a diagram (300) is shown comprising a 3D structure (302), a plurality of imaging devices (104), a set of flash units (306), and a set of diffusers (308). The diagram (300) shows a person (310) in a seated position inside the 3D structure (302).

[0034] A plurality of imaging devices (104) may be arranged (or mounted) at corresponding first plurality locations on the 3D structure (302). As illustrated, for example, the 3D structure (302) may be a dome-shaped lighting rig having sufficient space to include a person (310) or at least the face of the person (310). A plurality of flash units (306) and a diffuser set (308) may be arranged or mounted at corresponding second plurality locations and corresponding third plurality locations, respectively, on the 3D structure (302). The arrangement of the plurality of imaging devices (104), the flash unit set (106), and the diffuser set (308) on the 3D structure (302) may be such that each imaging device, flash unit, and diffuser can be directed toward the face (312) of the person (310) from a specific point in time. The imaging device can acquire an image of the face (312) from a first time point, but the flash unit can illuminate the face (312) from a second time point (which may be the same as or different from the first time point).

[0035] In the diagram (300), a set of coded targets (314) placed on the face (312) (e.g., forehead) of a person (310) is illustrated. In some instances, these coded targets (314) may include unique codes or identifiers, each of which may help to uniquely identify the location of a part of the face (312) in each of the captured multiple images (112). For example, when the set of coded targets (314) is placed on the face (312), it may appear at different angles for multiple imaging devices. If the code value of a specific coded target is identified in multiple images from different points in time, the location of the coded target in each image may refer to a common part of the face (312).

[0036] The circuit unit (202) may be configured to control a plurality of imaging devices (104) to capture a plurality of images (112) from a plurality of corresponding times. As an example, not limited to, the circuit unit (202) may control a plurality of imaging devices (104) at a first time-instance to capture a first set of images (114), and the circuit unit (202) may further control a plurality of imaging devices (104) at a second time-instance to capture a second set of images (116). There may be a time difference of about 1.5 seconds between the first time-instance and the second time-instance.

[0037] In an embodiment, the circuit portion (202) may be configured to simultaneously activate a flash unit set (106) while a plurality of imaging devices (104) capture a first image set (114) in a first time instance. The number of flash units in the flash unit set (106) may be less than or equal to the plurality of imaging devices (104). Simultaneous activation of the flash unit set (106) may cause the illumination of the 3D structure (302) to be omnidirectional. Omnidirectional illumination may cause the face (312) of the person (310) to be evenly illuminated. The first image set (114) may include images of the face (312) of the person (310) exposed to omnidirectional illumination. The first image set (114) may be utilized to generate an accurate 3D face mesh (e.g., 3D face mesh (118)) of the face (312) of the person (310).

[0038] In an embodiment, the circuit section (202) may further activate a flash unit set (106) in a sequential pattern while a plurality of imaging devices (104) capture a second set of images (404) in a second time-instance. Sequential activation of the flash unit set (106) may cause the illumination of the 3D structure (302) to be directional. The directional illumination may partially illuminate the face (312) of the person (310) in each image. According to an embodiment, the light intensity of the directional illumination may be greater than the light intensity of the omnidirectional illumination. The light intensity of the omnidirectional illumination may be reduced to decrease the amount of illumination on the face (312) of the person (310). While the plurality of imaging devices (104) capture images of the face (312), a diffuser set (308) may be utilized to soften the effect of illumination on the face (312).

[0039] In some scenarios, each imaging device may have an associated delay due to the difference between the control time of the imaging device (such as the activation of the shutter) and the capture time of the image (of the plurality of images (112)) by the imaging device. This time delay may be due to hardware limitations of the plurality of imaging devices (104). The circuit (202) may activate each flash unit in a sequential pattern at intervals set to match the delays caused by each imaging device. In an embodiment, the circuit (202) may activate a first flash unit subset of a flash unit set (106) while a first group of imaging devices (among the plurality of imaging devices (104)) captures one or more first images of a second image set (116). The circuit section (202) can activate a second flash unit subset of a flash unit set (106) while a second imaging device group (among a plurality of imaging devices (104)) captures one or more second images of a second image set (116). The second image set (116) can be used to capture microgeometric skin details and skin reflectance details of a face (312) of a person (310).

[0040] FIG. 4a illustrates exemplary images captured under omnidirectional lighting conditions according to an embodiment of the present disclosure. FIG. 4a is described in relation to elements from FIG. 1, FIG. 2, and FIG. 3. Referring to FIG. 4a, a diagram (400A) including a first set of images (402) is shown. The first set of images (402) may be generated by a plurality of imaging devices (304) at a first time-instance. The first set of images (402) may include faces (312) of a person (310) from a plurality of corresponding time points. The faces (312) of the first set of images (402) may be exposed to omnidirectional lighting. For example, the first image may include a left-side view of the face (312), the second image may include a right-side view of the face (312), and the third image may include a front view of the face (312). The number of images in the first image set (402) may depend on the number of imaging devices that can be controlled in the first time-instance to capture the first image set (402). As an example, not a limitation, the number of imaging devices may be 24, and the number of images in the first image set (402) may be 24.

[0041] FIG. 4b illustrates exemplary images captured under directional lighting conditions according to an embodiment of the present disclosure. FIG. 4b is described in relation to elements from FIG. 1, FIG. 2, FIG. 3, and FIG. 4a. Referring to FIG. 4b, a diagram (400B) including a second set of images (404) is shown. The second set of images (404) may be output by a plurality of imaging devices (304) at a second time-instance. The second set of images (404) may include the face (312) of a person (310) from a plurality of corresponding time points. The face (312) of the second set of images (404) may be exposed to directional lighting. For example, the first image may include a left-side view of the face (312), the second image may include a right-side view of the face (312), and the third image may include a front-side view of the face (312). The number of images in the second image set (404) may depend on the number of imaging devices. For example, the number of imaging devices may be 24, and the number of images in the second image set (404) may be 24.

[0042] FIGS. 5A, 5B, 5C, and 5D collectively illustrate exemplary operations for 3D microgeometry and reflectance modeling according to embodiments of the present disclosure. With reference to FIGS. 5A, 5B, 5C, and 5D, a block diagram (500) illustrating exemplary operations from 502 to 526 as described herein is illustrated. The exemplary operations illustrated in the block diagram (500) may start from 502 and may be performed by any computing system, device, or device, such as the system (102) of FIG. 1 or the circuit part (202) of FIG. 2. Although illustrated as individual blocks, exemplary operations associated with one or more blocks of the block diagram (500) may be divided into additional blocks, combined into a smaller number of blocks, or removed depending on the implementation of the exemplary operations.

[0043] In 502, a plurality of images (112) may be received. According to an embodiment, the circuit portion (202) may be configured to receive a plurality of images (112) from a plurality of imaging devices (304). The plurality of images (112) may include a first image set (402) and a second image set (404). As illustrated, for example, the first image set (402) may include a first image (402A), a second image (402B), a third image (402C), and a Nth image (402N). The second image set (404) may include a first image (404A), a second image (404B), a third image (404C), and a Nth image (404N). The first image set (402) and the second image set (404) may include the face (312) of a person (310). The face (312) of the first image set (402) can be exposed to omnidirectional lighting, whereas the face (312) of the second image set (404) can be exposed to directional lighting.

[0044] In 504, a first 3D face mesh (504A) may be generated based on the received first image set (402). According to an embodiment, the circuit (202) may be configured to generate the first 3D face mesh (504A) based on the received first image set (402). For the 3D reconstruction of a 3D face mesh from 2D images, there are many techniques known to those skilled in the art. For example, the first 3D face mesh (504A) may be generated using a photogrammetry-based method (such as SfM (structure from motion)), a method requiring stereoscopic images, or a method requiring monocular cues (such as SfS (shape from shading), photometric stereo, or SfT (shape from texture)). Details of these techniques have been omitted from this disclosure for brevity.

[0045] In an embodiment, the first 3D face mesh (504A) may be a raw 3D scan of the face (312) of a person (310) and may include artifacts, such as sharp edges (i.e., edges having large dihedral angles), spikes, or holes (large and small sizes). To refine the first 3D face mesh (504A), a set of model cleanup operations may be performed as described herein.

[0046] In 506, a set of model cleanup actions may be applied to the first 3D face mesh (504A) generated to obtain the refined first 3D face mesh (506A). According to an embodiment, the circuit portion (202) may be configured to apply a set of model cleanup actions to the first 3D face mesh (504A) generated to obtain the refined first 3D face mesh (506A). In an embodiment, the set of model cleanup actions may include removing unwanted regions from the first 3D face mesh (504A), filling small holes (e.g., empty spaces) in the first 3D face mesh (504A), and removing spikes from the first 3D face mesh (504A).

[0047] Unwanted areas, such as inaccurately estimated polygons on the first 3D face mesh (504A) (which may not be part of the face (312) of the person (310)), can be removed. In the first 3D face mesh (504A), there may be some empty spaces or holes (i.e., sufficiently large spaces without polygons). These holes may affect the fidelity of the first 3D face mesh (504A). Therefore, these unwanted holes may be removed, for example, using an appropriate prediction method that may rely on an array of nodes or geometry adjacent to the holes in the first 3D face mesh (504A). In some instances, inaccurate estimation of depth at some locations may lead to the creation of unwanted spikes in the first 3D face mesh (504A). The circuit section (202) can remove or smooth these unwanted spikes on the first 3D face mesh (504A) to obtain a refined first 3D face mesh (506A).

[0048] In 508, a second 3D face mesh (508A) may be generated based on the received second image set (404). According to an embodiment, the circuit (202) may be configured to generate the second 3D face mesh (508A) based on the received second image set (404). Similar to the first 3D face mesh (504A), the second 3D face mesh (508A) may be generated using a photogrammetry-based method (such as SfM (structure from motion)), a method requiring stereoscopic images, or a method requiring monocular cues (such as SfS (shape from shading), photometric stereo, or SfT (shape from texture)). Details of these techniques have been omitted from the present disclosure for brevity.

[0049] The second 3D face mesh (508A) may be a raw 3D scan of the face (312) of a person (310) and may include one or more artifacts. According to an embodiment, the second 3D face mesh (508A) may be further refined. The circuit unit (202) may refine the second 3D face mesh (508A) generated based on the application of a set of model cleanup actions (as described in 506) to the second 3D face mesh (508A).

[0050] In 510, an affine transformation between a refined first 3D face mesh (506A) and a generated second 3D face mesh (508A) can be estimated. The circuit (202) can be configured to estimate an affine transformation between a refined first 3D face mesh (506A) and a generated second 3D face mesh (508A). According to an embodiment, the affine transformation can be estimated based on a coded target set (314). As an example, not limited to, the circuit (202) can be configured to determine first locations of the coded target set (314) in a received first image set (402). The circuit (202) can further determine second locations of the coded target set (314) on a face (312) in a received second image set (404). An affine transformation can be estimated based on a comparison of the determined first locations and the determined second locations.

[0051] Since both the refined first 3D face mesh (506A) and the second 3D face mesh may not be rigidly aligned initially, the difference between corresponding nodes of the refined first 3D face mesh (506A) and the second 3D face mesh (508A) may be non-zero. The difference may be calculated, for example, using an L1 or L2 norm. As illustrated, for example, a heatmap (510A) represents the node-wise difference between the refined first 3D face mesh (506A) and the generated second 3D face mesh (508A). Points in the left half of the heatmap (510A) show a higher difference between corresponding nodes of the two meshes compared to points in the right half of the heatmap (510A).

[0052] In 512, the estimated affine transformation can be applied to the refined first 3D face mesh (506A). According to an embodiment, the circuit (202) may be configured to apply the estimated affine transformation to the refined first 3D face mesh (506A) to generate a 3D face mesh (512A). The generated 3D face mesh (512A) may be rigidly aligned with the generated second 3D face mesh (508A).

[0053] The affine transformation may include a matrix (or matrices) of rotation and translation values. The relative position and orientation of the refined first 3D face mesh (506A) may be updated based on the matrix to match those of the second 3D face mesh (508A). As illustrated, for example, a heatmap (510B) indicates the node-wise difference between the 3D face mesh (512A) and the generated second 3D face mesh (508A). All points on the face region of the heatmap (510B) indicate that the difference between the corresponding nodes of the 3D face mesh (512A) and the generated second 3D face mesh (508A) is nearly zero. Thus, the heatmap (510B) indicates that both the 3D face mesh (512A) and the generated second 3D face mesh (508A) can be rigidly aligned.

[0054] In 514, a white balancing operation may be applied to a second image set (404) to generate a white-balanced image set. Due to lighting variations, the color of the face (312) may differ slightly in different images of the second image set (404). A white balancing operation may be applied to correct the skin color of the face (312) in all or some images of the second image set (404). According to an embodiment, the circuit (202) may be configured to apply a white balancing operation to the second image set (404) to generate a white-balanced image set. Then, the circuit (202) may obtain a specular-less image set (e.g., a first specular-less image set (514A) and a second specular-less image (514B)) by removing specular information from the white-balanced image set. The removal of specular information can lead to the removal of highlights from the face (312) of a person (310) in a white-balanced set of images. Specular information can be removed within each image of the white-balanced set of images based on the conversion of color information from red-green-blue (RGB) space to SUV color space. Color information from RGB space can be converted to SUV color space by rotating the RGB coordinate vectors of RGB space.

[0055] In 516, a UV coordinate map of the face (312) can be determined based on the generated 3D face mesh (512A). According to an embodiment, the circuit (202) may be configured to determine the UV coordinate map of the face (312) based on the generated 3D face mesh (512A). The UV coordinate map of the face (312) may be a representation of the 3D face mesh (512A) in a 2D UV coordinate space. Then, the circuit (202) may generate an initial texture map of the face (312) by texture-mapping a set of specularless images (e.g., a first specularless image (514A) and a second specularless image (514B)) to the determined UV coordinates. The initial texture map of the face (312) may include texture information and color information from the skin and / or other visible parts of the face (312) of the set of specularless images. The initial texture map can be represented in UV coordinate space, where "U" and "V" are 2D coordinates of texture values.

[0056] In 518, a set of skin-reflectivity modeling operations may be executed. By using the generated 3D face mesh (512A) and the second set of images (404), the circuit (202) may execute a set of skin-reflectivity modeling operations as described herein. These operations may be executed to estimate a set of texture maps for the face (312) of a person (310). According to an embodiment, the set of skin-reflectivity modeling operations may include diffuse reflection modeling operations, specular separation operations, and specular reflection modeling operations. The diffuse reflection modeling operations may be executed to generate a diffuse normal map and a diffuse albedo map. The specular separation operations may be executed to generate a specular reflection information separation map. The specular reflection modeling operations may be executed to generate a specular albedo map, a specular normal map, and a roughness map of the face (312) of a person (310).

[0057] In 520, a diffuse reflection modeling operation may be performed. In an embodiment, the circuit portion (202) may be configured to perform a diffuse reflection modeling operation on a second set of images (404). The diffuse reflection modeling operation may be performed to generate a diffuse normal map (520A) of the face (312) based on an initial texture map of the face (312) (obtained in 516). The diffuse reflection modeling operation may be further performed to generate a diffuse albedo map (520B) of the face (312) based on the initial texture map and the generated diffuse normal map (520A). The diffuse albedo map (520B) may be referred to as the first texture map of the estimated set of texture maps (e.g., in 518).

[0058] In the embodiment, the generation of the diffuse normal map (520A) and the diffuse albedo map may be based on a Lambertian light model. The Lambertian light model can be expressed by Equation (1) as follows:

[0059]

[0060] Here, n is the diffusion normal, ρ is the diffusion albedo, and L I is the direction of light. The direction of light can be determined from the predefined position and orientation of each of the plurality of imaging devices (304).

[0061] In 522, a specular separation operation may be performed. According to an embodiment, the circuit portion (202) may be configured to perform a specular separation operation to separate specular reflection information from a second image set (404). The specular reflection information may be separated based on a generated diffuse normal map (520A) and a generated diffuse albedo map (520B). A map (522A) containing the separated specular reflection information is illustrated as an example.

[0062] In 524, a specular reflection modeling operation may be performed. According to an embodiment, the circuit portion (202) may be configured to perform a specular reflection modeling operation on a second image set (404). The specular reflection modeling operation may be performed to generate a specular normal map (524A) of the face (312), a specular albedo map (524B) of the face (312), and a roughness map (524C) of the face (312). The specular normal map (524A), the specular albedo map (524B), and the roughness map (524C) may be generated based on separate specular reflection information (and the second image set (404)). The specular normal map (524A) may include shine and highlight information of the face (312) in the second image set (404). The specular albedo map (524B) may include color information of the face (312) and may exclude highlight information and shadow information of the face (312) of the person (310). The roughness map (524C) may represent the roughness of the skin of the face (312) of the person (310). The roughness map (524C) may be represented as a black and white color texture image. The specular normal map (524A), specular albedo map (524B), and roughness map (524C) may be referred to as the second texture maps of the texture map set (estimated in 516). The first texture map and the second texture map may include microgeometric skin details and skin reflectance details of the face (312) of the person (310).

[0063] In the embodiment, the generation of the specular normal map (524A), specular albedo map (524B), and roughness map (524C) may be based on the Blinn-Phong light model. The Blinn-Phong light model can be expressed by Equation 2 as follows.

[0064]

[0065] Here, n is the specular normal, ρ is the specular albedo, and α is related to surface roughness.

[0066] In 526, the generated 3D face mesh (512A) may be textured. According to an embodiment, the circuit unit (202) may be configured to texture the generated 3D face mesh (512A) based on an estimated texture map set. Textured may include an operation in which texture information, including microgeometric skin details and skin reflectance details of the estimated texture map set, is mapped to the generated 3D face mesh (512A). According to an embodiment, the estimated texture map set may include a diffuse albedo map (520B) of the face (312), a specular normal map (524A) of the face (312), a specular albedo map (524B) of the face (312), and a roughness map (524C) of the face (312). The microgeometric skin details may include texture information for various skin components such as pores, ridges, freckles, and wrinkles. Similarly, skin reflectance details may include information on diffuse reflection components, specular reflection components, albedo components, and roughness components. The textured 3D face model (526A) may include both microgeometric skin details and skin reflectance details. Thus, the textured 3D face model (526A) can be treated as a high-fidelity 3D model of the face (312) of a person (310).

[0067] Block diagram (500) is illustrated with individual operations such as 502, 504, 506, 508, 510, 512, 514, 516, 518, 520, 522, 524, and 526, but the present disclosure is not limited thereto. Accordingly, in certain embodiments, these individual operations may be further subdivided into additional operations, combined into fewer operations, or eliminated according to a particular implementation without compromising the essence of the disclosed embodiments.

[0068] FIG. 6 is a flowchart illustrating an exemplary method for 3D microgeometry and reflectance modeling according to an embodiment of the present disclosure. FIG. 6 is described in relation to elements from FIG. 1, FIG. 2, FIG. 3, FIG. 4a, FIG. 4b, FIG. 5a, FIG. 5b, FIG. 5c, and FIG. 5d. Referring to FIG. 6, a flowchart (600) is illustrated. The method illustrated in the flowchart (600) may be executed by any computer system, such as by a system (102) or a circuit part (202). The method may start at 602 and proceed to 604.

[0069] In 604, a plurality of images (112) may be received, which may include a first image set (402) of a face (312) and a second image set (404) of a face (312). According to an embodiment, the circuit portion (202) may be configured to receive a plurality of images (112) which may include a first image set (402) of a face (312) of a person (310) and a second image set (404) of a face (312). The face (312) of the first image set (402) may be exposed to omnidirectional illumination, and the face (312) of the second image set (404) may be exposed to directional illumination. Details regarding the reception of the plurality of images (112) are further provided, for example, in FIG. 3.

[0070] In 606, a 3D face mesh (512A) can be generated based on a plurality of received images (112). According to an embodiment, the circuit portion (202) may be configured to generate a 3D face mesh (512A) based on a plurality of received images (112). Details regarding the generation of the 3D face mesh (512A) are further provided, for example, in FIG. 5a.

[0071] In 608, a skin-reflectivity modeling action set may be executed to estimate a texture map set for the face (312) by using the generated 3D face mesh (512A) and the second image set (404). According to an embodiment, the circuit (202) may be configured to execute a skin-reflectivity modeling action set to estimate a texture map set for the face (312) by using the generated 3D face mesh (512A) and the second image set (404). Details regarding the execution of the skin-reflectivity modeling action set are provided, for example, in FIGS. 5c and 5d.

[0072] In 610, the generated 3D face mesh (512A) may be textured based on an estimated texture map set. According to an embodiment, the circuit (202) may be configured to texture the generated 3D face mesh (512A) based on an estimated texture map set. Textured textured textured texture information, including microgeometric skin details and skin reflectance details of the estimated texture map set, may be mapped to the generated 3D face mesh (512A). Details regarding the textured textured texture of the 3D face mesh (512A) are further provided, for example, in FIG. 5d. Control may be terminated.

[0073] Although the flowchart (600) is illustrated with individual operations such as 602, 604, 606, 608, and 610, the present disclosure is not limited thereto. Accordingly, in certain embodiments, these individual operations may be further subdivided into additional operations, combined into fewer operations, or eliminated according to a particular implementation without compromising the essence of the disclosed embodiments.

[0074] Various embodiments of the present disclosure may provide a non-transient computer-readable medium and / or storage medium in which instructions executable by a machine and / or computer to operate a system (e.g., system (102)) are stored. The instructions may cause the machine and / or computer to perform operations that may include receiving a plurality of images (e.g., a plurality of images (112)) that may include a first set of images (e.g., a first set of images (114)) of a face (e.g., a face (110)) and a second set of images (e.g., a second set of images (116)) of a face (110). The face (110) of the first set of images (114) may be exposed to omnidirectional illumination, and the face (110) of the second set of images (116) may be exposed to directional illumination. The operations may further include generating a three-dimensional (3D) face mesh (e.g., a 3D face mesh (118)) based on the received plurality of images (112). The operations may, by using the generated 3D face mesh (118) and the second set of images (116), on the face (110) It may further include executing a set of skin-reflectivity modeling actions to estimate a set of texture maps. The actions may further include textureizing a 3D face mesh (118) generated based on the estimated set of texture maps. Textureizing may include an action in which texture information, including microgeometric skin details and skin reflectivity details of the estimated set of texture maps, is mapped onto the generated 3D face mesh (118).

[0075] Exemplary embodiments of the present disclosure may provide a system (e.g., system (102) of FIG. 1) comprising a circuit portion (e.g., circuit portion (202)). The circuit portion (202) may be configured to receive a plurality of images (e.g., a plurality of images (112)) which may include a first set of images (e.g., a first set of images (114)) of a face (e.g., a face (110)) and a second set of images (e.g., a second set of images (116)) of a face (110). The face (110) of the first set of images (114) may be exposed to omnidirectional illumination, and the face (110) of the second set of images (116) may be exposed to directional illumination. The circuit unit (202) may be further configured to generate a three-dimensional (3D) face mesh (e.g., 3D face mesh (118)) based on a plurality of received images (112). The circuit unit (202) may be further configured to execute a set of skin-reflectivity modeling operations to estimate a set of texture maps for a face (110) by using the generated 3D face mesh (118) and a second set of images (116). The circuit unit (202) may be further configured to texture the generated 3D face mesh (118) based on the estimated set of texture maps. Texture mapping may include an operation in which texture information, including microgeometric skin details and skin reflectivity details of the estimated set of texture maps, is mapped onto the generated 3D face mesh (118).

[0076] According to an embodiment, the system (102) may further include a plurality of imaging devices (e.g., a plurality of imaging devices (304)) arranged at corresponding first plurality of locations on a 3D structure (e.g., a 3D structure (302)). The circuit portion (202) may be further configured to control the plurality of imaging devices (304) to capture a plurality of images (112) from a plurality of corresponding viewpoints.

[0077] According to an embodiment, the system (102) may further include a set of flash units (e.g., a set of flash units (306)) arranged at corresponding second plurality of locations on a 3D structure (302). The circuit portion (202) may be further configured to activate the set of flash units (306) simultaneously while a plurality of imaging devices (304) capture a first set of images (402). The circuit portion (202) may activate the set of flash units (306) in a sequential pattern while a plurality of imaging devices (304) capture a second set of images (404).

[0078] According to the embodiment, the light intensity of the directional illumination may be greater than the light intensity of the omnidirectional illumination.

[0079] According to an embodiment, the circuit unit (202) may be further configured to generate a first 3D face mesh (e.g., a first 3D face mesh (504A)) based on a received first image set (402). The circuit unit (202) may apply a set of model cleanup operations to the generated first 3D face mesh (504A) to obtain a refined first 3D face mesh (e.g., a refined first 3D face mesh (506A)). The circuit unit (202) may further generate a second 3D face mesh (e.g., a second 3D face mesh (508A)) based on a received second image set (404). The circuit unit (202) may estimate an affine transformation between the refined first 3D face mesh (506A) and the generated second 3D face mesh (508A). The circuit section (202) may additionally apply an estimated affine transformation to the refined first 3D face mesh (506A) to generate a 3D face mesh (512A). The generated 3D face mesh (512A) can be rigidly aligned with the generated second 3D face mesh (508A).

[0080] According to an embodiment, the circuit unit (202) may be further configured to determine first locations of a coded target set (e.g., a coded target set (314)) on a face (312) in a received first image set (402). The circuit unit (202) may determine second locations of a coded target set (314) on a face (312) in a received second image set (404). The circuit unit (202) may further estimate an affine transformation based on a comparison of the determined first locations and the determined second locations.

[0081] According to an embodiment, the circuit unit (202) may be further configured to apply a white balancing operation to a second image set (404) to generate a white-balanced image set. The circuit unit (202) may obtain a specular-less image set (e.g., a first specular-less image (514A) and a second specular-less image (514B)) by removing specular information from the white-balanced image set. Specular information may be removed based on the conversion of color information from the Red-Green-Blue (RGB) space to the SUV color space within each image of the received second image set (404).

[0082] According to an embodiment, the circuit (202) may be further configured to determine a UV coordinate map of the face (312) based on a generated 3D face mesh (512A). The circuit (202) may further generate an initial texture map of the face (312) by texture-mapping a set of specularless images onto the determined UV coordinate map.

[0083] According to an embodiment, a set of skin-reflectance modeling actions may include a diffuse reflection modeling action, a specular separation action, and a specular reflection modeling action.

[0084] According to an embodiment, the circuit section (202) may be further configured to perform a diffuse reflection modeling operation to generate a diffuse normal map of the face (312) (e.g., a diffuse normal map (520A)) based on an initial texture map. The circuit section (202) may further generate a diffuse albedo map of the face (312) (e.g., a diffuse albedo map (520B)) based on the initial texture map and the generated diffuse normal map (520A). The diffuse albedo map (520B) may be a first texture map of the estimated texture map set.

[0085] According to an embodiment, the circuit portion (202) may be further configured to perform a specular separation operation to separate specular reflection information from a second image set (404) based on a generated diffuse normal map (520A) and a generated diffuse albedo map (520B).

[0086] According to an embodiment, the circuit portion (202) may be further configured to perform a specular reflection modeling operation to generate a specular albedo map of the face (312) (e.g., specular albedo map (524B)), a specular normal map of the face (312) (e.g., specular normal map (524A)), and a roughness map of the face (312) (e.g., roughness map (524C)) based on separated specular reflection information. The specular albedo map (524B), specular normal map (524A), and roughness map (524C) may be second texture maps of an estimated texture map set.

[0087] According to an embodiment, the estimated texture map set may include a diffuse albedo map (520B) of the face (312), a specular albedo map (524B) of the face (312), a specular normal map (524A) of the face (312), and a roughness map (524C) of the face (312).

[0088] The present disclosure may be realized in hardware or a combination of hardware and software. The present disclosure may be realized in a centralized manner in at least one computer system or in a distributed manner, wherein different elements may be distributed across multiple interconnected computer systems. A computer system or other device adapted to perform the methods described herein may be suitable. A combination of hardware and software may be a general-purpose computer system having a computer program capable of controlling the computer system to perform the methods described herein when loaded and executed. The present disclosure may be realized in hardware comprising a part of an integrated circuit that also performs other functions.

[0089] The present disclosure may also be implemented as a computer program product that includes all features enabling the implementation of the methods described herein and is capable of performing these methods when loaded into a computer system. In this context, a computer program means any representation of any language, code, or notation of a set of instructions intended to cause a system having information processing capabilities to perform a specific function directly, or after either or both of the following: a) conversion to another language, code, or notation; b) reproduction in a different material form.

[0090] Although the present disclosure is described with reference to specific embodiments, it will be understood by those skilled in the art that various modifications may be made and equivalents substituted without departing from the scope of the present disclosure. Furthermore, many modifications may be made to adapt specific situations or materials to the teachings of the present disclosure without departing from the scope thereof. Accordingly, the present disclosure is not limited to the specific embodiments disclosed, and the present disclosure is intended to include all embodiments within the scope of the appended claims.

Claims

Claim 1 As a system, the circuit comprises receiving a plurality of images including a first set of images of a face exposed to omni-directional lighting and a second set of images of a face exposed to directional lighting, generating a three-dimensional (3D) face mesh based on the received plurality of images, and by using the generated 3D face mesh and the second set of images, executing a set of skin-reflectance modeling operations to estimate a set of texture maps for the face, and texturing the generated 3D face mesh based on the estimated set of texture maps, wherein the texturing includes an operation in which texture information, including microgeometry skin details and skin reflectance details of the estimated set of texture maps, is mapped to the generated 3D face mesh, further comprising a plurality of imaging devices disposed at corresponding first plurality of positions on a 3D structure, and the circuit further comprises controlling the plurality of imaging devices to capture the plurality of images from corresponding plurality of viewpoints, and the 3D A system further comprising a set of flash units arranged at corresponding second plurality of locations on a structure, wherein the circuit is further configured to activate the set of flash units simultaneously while the plurality of imaging devices capture the first set of images, and to activate the set of flash units in a sequential pattern while the plurality of imaging devices capture the second set of images. Claim 2 delete Claim 3 delete Claim 4 A system according to claim 1, wherein the light intensity of the directional illumination is greater than the light intensity of the omnidirectional illumination. Claim 5 In claim 1, the circuit is further configured to: generate a first 3D face mesh based on the received first image set; apply a set of model clean-up operations to the generated first 3D face mesh to obtain a refined first 3D face mesh; generate a second 3D face mesh based on the received second image set; estimate an affine transformation between the refined first 3D face mesh and the generated second 3D face mesh; and apply the estimated affine transformation to the refined first 3D face mesh to generate the 3D face mesh, wherein the generated 3D face mesh is rigidly aligned with the generated second 3D face mesh, a system. Claim 6 In claim 5, the circuit is further configured to: determine first locations of a coded target set on the face in the received first image set; determine second locations of a coded target set on the face in the received second image set; and estimate an affine transformation based on a comparison of the determined first locations and the determined second locations. Claim 7 In claim 1, the circuit is further configured to: apply a white balancing operation to the second image set to generate a white-balanced image set; and to obtain a specular-less image set by removing specular information from the white-balanced image set, wherein the specular information is removed based on the conversion of color information from a red-green-blue (RGB) space to an SUV color space within each image of the received second image set. Claim 8 In claim 7, the circuit is further configured to: determine a UV coordinate map of the face based on the generated 3D face mesh; and generate an initial texture map of the face by texture-mapping a set of specularless images onto the determined UV coordinate map, the system. Claim 9 In claim 8, the skin-reflectance modeling operation set comprises a diffused reflection modeling operation, a specular separation operation, and a specular reflection modeling operation, a system. Claim 10 In claim 9, the circuit is configured to: generate a diffuse normal map of the face based on the initial texture map; and perform the diffuse reflection modeling operation to generate a diffuse albedo map of the face based on the initial texture map and the generated diffuse normal map, wherein the diffuse albedo map is the first texture map of the estimated texture map set, the system. Claim 11 A system according to claim 10, wherein the circuit portion is configured to perform the specular separation operation to separate specular reflection information from the second image set based on the generated diffuse normal map and the generated diffuse albedo map. Claim 12 In claim 11, the circuit is configured to perform the specular reflection modeling operation to generate the specular albedo map of the face, the specular normal map of the face, and the roughness map of the face based on the separated specular reflection information, wherein the specular albedo map, the specular normal map, and the roughness map are second texture maps of the estimated texture map set, a system. Claim 13 A system according to claim 1, wherein the estimated texture map set comprises a diffuse albedo map of the face, a specular albedo map of the face, a specular normal map of the face, and a roughness map of the face. Claim 14 A method comprising: receiving a plurality of images including a first set of images of a face exposed to omnidirectional illumination and a second set of images of a face exposed to directional illumination; generating a three-dimensional (3D) face mesh based on the received plurality of images; estimating a set of texture maps for the face by executing a set of skin-reflectivity modeling operations using the generated 3D face mesh and the second set of images; and textureizing the generated 3D face mesh based on the estimated set of texture maps, wherein the textureizing includes an operation in which texture information, including microgeometric skin details and skin reflectivity details of the estimated set of texture maps, is mapped to the generated 3D face mesh; further comprising the step of controlling a plurality of imaging devices arranged at corresponding first plurality of locations on a 3D structure to capture the plurality of images from corresponding plurality of viewpoints, and simultaneously activating a set of flash units arranged at corresponding second plurality of locations on the 3D structure while the plurality of imaging devices capture the first set of images. A method further comprising the step of activating the set of flash units in a sequential pattern while the plurality of imaging devices capture the second set of images. Claim 15 delete Claim 16 delete Claim 17 A method according to claim 14, further comprising: generating a first 3D face mesh based on the received first image set; applying a set of model cleanup actions to the generated first 3D face mesh to obtain a refined first 3D face mesh; generating a second 3D face mesh based on the received second image set; estimating an affine transformation between the refined first 3D face mesh and the generated second 3D face mesh; and applying the estimated affine transformation to the refined first 3D face mesh to generate the 3D face mesh, wherein the generated 3D face mesh is rigidly aligned with the generated second 3D face mesh. Claim 18 A method according to claim 17, further comprising: determining first locations of a coded target set on the face in the first received image set; determining second locations of a coded target set on the face in the second received image set; and estimating an affine transformation based on a comparison of the determined first locations and the determined second locations. Claim 19 In claim 14, the estimated texture map set comprises a diffuse albedo map of the face, a specular albedo map of the face, a specular normal map of the face, and a roughness map of the face. Claim 20 A non-transient computer-readable medium storing computer-executable instructions, wherein, when executed by a system, the computer-executable instructions cause the system to execute operations, said operations include: receiving a plurality of images including a first set of images of a face exposed to omnidirectional illumination and a second set of images of a face exposed to directional illumination; generating a three-dimensional (3D) face mesh based on said received plurality of images; executing a set of skin-reflectivity modeling operations to estimate a set of texture maps for said face by using said generated 3D face mesh and said second set of images; and textureizing said generated 3D face mesh based on said estimated set of texture maps, said texturing including texture information including microgeometric skin details and skin reflectivity details of said estimated set of texture maps, said texturing includes mapping texture information to said generated 3D face mesh, said texturing includes controlling a plurality of imaging devices arranged at corresponding first plurality of locations on a 3D structure to capture said plurality of images from corresponding plurality of viewpoints, while said plurality of imaging devices capture said first set of images A non-transient computer-readable medium further comprising: an operation of activating a set of flash units arranged at corresponding second plurality of positions on the 3D structure at the same time; and an operation of activating the set of flash units in a sequential pattern while the plurality of imaging devices capture the second set of images.

Images