A method for simulating 3D lighting for image videos containing portraits

Abstract

The present application discloses a method for simulating 3D lighting for an image or video containing a portrait, comprising the following specific steps: S1, using a universal head model and a preset information production module; S2, using a face key point detection module; S3, using a face region segmentation module; S4, using a 3D head simulation module; S5, using a simulated lighting rendering module, and the obtained texture result is used as the final output; the technical key points are as follows: by performing a three-dimensional perspective fitting calculation on the face key point information of an input two-dimensional image, the position and rotation matrix of the universal head model in the camera coordinate system are obtained, and then the normal vector information of each pixel point in the face region of the original two-dimensional image is obtained, and after mixing and smoothing with the assumed default background normal vector, conventional lighting is performed, thereby producing a realistic, high-performance, highly reliable, and naturally integrated simulated lighting effect, which can be used for image and video editing to meet real user needs.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular is a method for simulating 3D lighting for an image video containing a portrait. Background Art

[0002] At present, image processing, also known as image processing, is a technology that uses computers to process images to achieve the desired results. Originated in the 1920s, it is generally digital image processing; the main contents of image processing technology include image compression, enhancement and restoration, and matching description and recognition. Common processing includes image digitization, image coding, image enhancement, image restoration, image segmentation and image analysis. Image processing is the use of computers to process image information to meet human visual psychology or application needs. It is widely used and is mostly used in surveying and mapping, atmospheric science, astronomy, image beautification, and improving image recognition.

[0003] With the rapid development of mobile Internet technology and the large-scale rise of Vlog and short videos, the demand for portrait editing among ordinary users is also increasing. In the field of image and video post-processing, in addition to the conventional basic functions such as whitening, skin smoothing, and facial feature adjustment, many users also need to modify the light and shadow effects of the face in the image and video or adjust the overall atmosphere and lighting in the image and video, so as to improve the final film effect.

[0004] However, the inventors of this application found that the above technology has at least the following technical problems in the process of implementing the technical solutions in the embodiments of this application:

[0005] When users post-process images or videos with portraits, they are often limited by the lighting conditions during shooting, and the imaging effect is not ideal. In order to improve the atmosphere and film effect, it is necessary to re-light the picture or video, or change the overall lighting atmosphere.

[0006] Most of the image and video editing applications currently on the market can only be achieved through filters or by adjusting brightness, saturation, highlights, shadows, etc., and cannot simulate real light and shadow effects; the few image editing tools that support simulated face lighting also have many problems such as poor performance, unrealistic normal vector generation, and unnatural fusion of face and background areas, resulting in the final lighting and shadow effects being unnatural, and cannot be applied to the video field, nor can they meet real user needs; especially in the field of mobile video processing, support for simulated lighting effects is even more lacking. Summary of the invention

[0007] Technical issues solved

[0008] In view of the deficiencies in the prior art, the present invention provides a method for simulating 3D lighting for an image video containing a portrait, thereby solving the problems mentioned in the prior art.

[0009] Technical Solution

[0010] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0011] A method for simulating 3D lighting for an image video containing a portrait includes the following specific steps:

[0012] S1. Use the universal head model and preset information to create a module.

[0013] Make a general 3D head model with normal vectors for subsequent 3D fitting of the face in the 2D image. Select a set of key points on the face surface, mark the 3D coordinates of all key points in the head model coordinate system, and use them as the output of this module together with the head model.

[0014] S2, using the face key point detection module,

[0015] This module receives image texture or video frame texture as input, detects facial key points, and outputs facial key point data required for subsequent processing;

[0016] S3, using the face region segmentation module,

[0017] Since the effective fitting of the head model is concentrated in the face area, it is necessary to segment the face area of ​​the input image texture or video frame texture and generate a face segmentation mask as an output;

[0018] S4, use 3D head to simulate the module,

[0019] According to the preset camera intrinsic parameters, the face key point detection module detects the face key point results of the image texture or video frame texture, and the universal head model and the three-dimensional coordinate data of the corresponding key points obtained by the preset information production module, the PnP problem is solved and the corresponding camera extrinsic parameters tvec and rvec are calculated;

[0020] The tvec: here indicates how a point in the three-dimensional world coordinate system is moved parallel to the camera coordinate system;

[0021] rvec: represents how a point in the three-dimensional world coordinate system is rotated and moved to the camera coordinate system; through tvec and rvec, an affine matrix can be calculated, which can transform the point in the world coordinate system into the camera coordinate system.

[0022] S5. Use the simulated lighting rendering module.

[0023] The data generated in steps S1, S3 and S4 are processed, and based on the processing results, the original input image texture or video frame texture is illuminated using the Phong lighting model, and the texture result is used as the final output.

[0024] Preferably, in S1, the center of mass of the 3D head model should be located at the origin of the model, and the left-hand or right-hand coordinate system can be freely selected according to the needs of subsequent fitting calculations and 3D rendering.

[0025] Preferably, a right-handed coordinate system is selected, and the face of the head model faces the positive direction of the Z axis, the left ear points to the positive direction of the X axis, and the six points of the center of the two eyes, the tip of the nose, the sides of the mouth and the chin are used as key points.

[0026] Preferably, in S2, the key points output by the facial key point detection module should at least include all the key points preset and marked in the universal head model and the preset information making module.

[0027] Preferably, in S2, the specific detection model used can be flexibly set according to the actual situation, and the video-related face key point detection also needs to be smoothed and filtered using a dedicated filter. The specific solution is:

[0028] (1) For the current frame to be detected, the detection results of all frames with a time interval not exceeding T are taken as a set of reference data;

[0029] (2) Traverse the data obtained in (1), and calculate the time coefficient timeContribute according to the time difference t between the time of the group of data and the time of the current frame. Preferably, the time coefficient calculation formula in the present invention is:

[0030] (3) Traverse the data obtained in (1) and calculate the distance coefficient distanceContribute according to the Euclidean distance d of the key points corresponding to the data set. The distance coefficient calculation formula is: The value of sigma should be flexibly set according to the input image size and the error distribution of the face detection model;

[0031] (4) According to timeContribute*distanceContribute, obtain the weight of each reference data;

[0032] (5) According to the weights obtained in (4), the weighted sum is calculated for each key point to obtain the filtering result as the final face key point detection output.

[0033] Preferably, in S2, when there are multiple faces in one image texture or video frame texture, multiple groups of key point results should be output.

[0034] Preferably, in S3, when there are multiple faces in an image texture or video frame texture, multiple face segmentation mask images should be output as results, and correspond one-to-one with the face key point detection results in the aforementioned face key point detection module.

[0035] Preferably, in S4, the universal 3D head model can be transformed from the world coordinate system to the camera coordinate system through the camera extrinsic parameters, and the model coordinate system can be directly regarded as the world coordinate system to save the conversion from the local coordinate system to the world coordinate system, and a standard pinhole camera model is adopted, and it is assumed that there is no camera distortion.

[0036] Preferably, in the S4, the camera lens is a telephoto lens.

[0037] Preferably, in S5, the specific solution of the 3D head simulation module is:

[0038] (1) Generate a default normal vector texture of the same size as the input image texture or video frame texture. Assuming that the normalized coordinates of a point on the texture are (x, y), the general formula for the default normal vector of the point is:

[0039] normalize(vector3(x*2–1,y*2-1,zDefault),

[0040] The value of zDefault can be set flexibly as needed, and 1.031 can be used as the default value;

[0041] (2) Based on the tvec and rvec calculated by the 3D head simulation module, render the 3D head model into a texture of matching specifications, and use the normal vector after GPU interpolation as output to obtain a blurred normal vector texture of the face area; if there are multiple faces, render each face in turn and mix it with the previous result normally;

[0042] (3) According to the default normal vector map and the parallel light position information, the default lighting diffuse reflection intensity map DiffuseNormal is obtained. The diffuse reflection intensity is calculated using the formula: max(dot(lightDir,normal),0.0).

[0043] (4) According to the normal vector texture of the face area and the parallel light position information, the diffuse reflection intensity map DiffuseFace of the face area is obtained. The diffuse reflection intensity is calculated using the formula: max(dot(lightDir,normal),0.0).

[0044] (5) According to the face segmentation mask obtained by the face area segmentation module, DiffuseNormal and DiffuseFace are mixed to obtain the comprehensive diffuse reflection result DiffuseCompose. The mixing formula is:

[0045] DiffuseNormal*(1-faceMaskValue)+DiffuseFace*faceMaskValue

[0046] Perform box blur on DiffuseCompose. The blur degree can be flexibly set as needed to obtain the blurred diffuse intensity map BlurryDiffuseCompose texture.

[0047] (6) Based on the obtained BlurryDiffuseCompose texture, the original input image texture or video frame texture is illuminated using the Phong lighting model, and the texture result is used as the final output.

[0048] Note:

[0049] The vector3 mentioned above: Set it to a function that wraps the three values ​​passed in into a three-dimensional vector (x, y, z);

[0050] normalize: Set it to another function that normalizes a three-dimensional vector, specifically: convert a three-dimensional vector into a unit vector;

[0051] zDefault: Set it to a predefined constant that is used to improve the curvature of the default normal vector;

[0052] In the above S5, in the specific scheme of the 3D head simulation module, the formula in step (1) means: convert the three values ​​2x-1, 2y-1, and zDefault into a three-dimensional vector v through the vector3 function, and then normalize the vector v;

[0053] dot: Set it as the vector dot product function, and perform dot product on the two unit vectors lightDir and normal;

[0054] max: Set it as the maximum value function, pass in one or more values, and return the largest value.

[0055] In summary, the present invention includes at least one of the following beneficial technical effects:

[0056] The present invention implements a method, which performs a three-dimensional perspective fitting calculation on the key point information of the face of an input two-dimensional image, obtains the position and rotation matrix of a universal head model in a camera coordinate system, and then obtains the normal vector information of each pixel point in the face area of ​​the original two-dimensional image, mixes and smoothes it with the assumed default background normal vector, and then performs conventional lighting, thereby producing a realistic, high-performance, high-reliability, and naturally integrated simulated lighting effect, which can be used for image and video editing to meet real user needs;

[0057] Among them, the present invention solves the PnP problem and uses a universal head model to quickly fit the normal vector of the face area in the original image, thereby completing real-time lighting, and has the advantages of good performance, good expression, and strong universality;

[0058] The present invention assumes that the background area is located in a convex surface to obtain the normal vector of the background area, and merges the normal vector of the face area and the normal vector of the background area through smoothing means, thereby reducing the sense of separation and unnaturalness at the face boundary. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a specific step flow chart of the present invention. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, for the convenience of description below, the referenced "upper", "lower", "left", and "right" are the same as the upper, lower, left, and right directions of the drawings themselves. The "first", "second", etc. below are distinguished for the description and have no other special meanings.

[0061] A method for simulating 3D lighting for an image video containing a portrait, wherein the specific steps or schemes of each module are as follows:

[0062] 1-Universal human head model and preset information production module

[0063] (1) Create a general 3D head model with normal vectors for subsequent 3D fitting of the face in the 2D image. For performance and fitting effect considerations, this 3D head model should meet the following requirements:

[0064] 1) The number of model faces should not be too high;

[0065] 2) Since the model cannot completely fit the real face in the input image, the closer the position and proportion of the facial features of the model are to the average value of the general public, the more realistic the average lighting effect after the final fitting will be;

[0066] 3) The outer contour of the model should be as large as possible to eliminate the problem of incomplete fitting coverage caused by camera distortion or the user's head shape being too large in a few input images;

[0067] 4) The center of mass of the model should be located at the origin of the model. According to the needs of subsequent fitting calculations and 3D rendering, the left-hand or right-hand coordinate system can be freely selected. Preferably, the right-hand coordinate system will be selected in the present invention, and the face of the head model faces the positive direction of the Z axis, and the left ear points to the positive direction of the X axis.

[0068] (2) Select a set of key points on the face surface. The key points should satisfy the following requirements:

[0069] 1) Not on the same straight line in three-dimensional space;

[0070] 2) The selected key points are not easily deformed or deform slightly when the face makes various expressions and movements;

[0071] 3) The number of points should be greater than or equal to 6 and less than or equal to 18;

[0072] The selection of key point sequence is not unique and can be freely selected according to the above conditions and actual needs.

[0073] Preferably, the present invention uses the centers of the two eyes, the tip of the nose, the sides of the mouth, and the chin as six key points.

[0074] According to the key points selected in the previous step, the three-dimensional coordinates (x, y, z) of all key points are marked in the human head model coordinate system and used together with the human head model as the final output of this module.

[0075] 2-Face key point detection module (filtering face key points to reduce face key point model jitter)

[0076] This module receives image texture or video frame texture as input, performs face key point detection, and outputs face key point data required for subsequent processing; the key points output by this module should at least include all key points preset and marked in the general head model and preset information production module; the higher the fit between the key points detected by the face and the real face in the input image texture or video frame texture, the better the final fitting effect; the specific detection model used can be flexibly set according to the actual situation;

[0077] In particular, for video-related facial key point detection, the existing deep learning detection models will have more or less key point jitter due to the existence of disturbances and detection errors. To address this problem, smoothing is usually required. Common solutions include one euro filter or kalman filter.

[0078] Preferably, the present invention adopts a filter designed by itself for filtering.

[0079] The specific plan is:

[0080] (1) For the current frame to be detected, the detection results of all frames with a time interval not exceeding T are taken as a set of reference data;

[0081] (2) Traverse the data obtained in 1), and calculate the time coefficient timeContribute according to the time difference t between the time of the group of data and the time of the current frame. Preferably, the time coefficient calculation formula in the present invention is:

[0082] (3) Traverse the data obtained in 1), and calculate the distance coefficient distanceContribute according to the Euclidean distance d of the key points corresponding to the group of data. Preferably, the distance coefficient calculation formula in the present invention is: Among them, the value of sigma should be flexibly set according to the input image size and the error distribution of the face detection model;

[0083] (4) According to timeContribute*distanceContribute, obtain the weight of each reference data;

[0084] (5) According to the weights obtained in the previous step, the weighted sum is calculated for each key point to obtain the filtering result as the final face key point detection output.

[0085] In particular, when there are multiple faces in an image texture or video frame texture, multiple groups of key point results should be output; preferably, the present invention adopts a self-developed deep learning-based face key point detection model for detection.

[0086] The present invention assumes that the background area is located in a convex surface to obtain the normal vector of the background area, and merges the normal vector of the face area and the normal vector of the background area through smoothing means, thereby reducing the sense of separation and unnaturalness at the face boundary.

[0087] 3-Face region segmentation module

[0088] Since the effective fitting of the head model is concentrated in the face area, it is necessary to segment the face area of ​​the input image texture or video frame texture to generate a face segmentation mask map as output; the specific segmentation model used can be flexibly set according to the actual situation.

[0089] In particular, when there are multiple faces in an image texture or video frame texture, multiple face segmentation mask images should be output as results, and correspond one-to-one with the face key point detection results in the aforementioned face key point detection module.

[0090] 4-3D head simulation module (by fitting the universal 3D head model to solve the normal vector and diffuse reflection intensity of the face area in the input image, to achieve real face lighting)

[0091] According to the preset camera intrinsic parameters, the facial key point results detected by the facial key point detection module on the image texture or video frame texture, and the three-dimensional coordinate data of the corresponding key points obtained by the universal head model and the preset information production module, the PnP problem (Perspective-n-Point) is solved and the corresponding camera extrinsic parameters tvec and rvec are calculated; through the camera extrinsic parameters, the universal 3D head model can be transformed from the world coordinate system to the camera coordinate system; at the same time, for the convenience of conversion, the model coordinate system can be directly regarded as the world coordinate system to save the conversion from the local coordinate system to the world coordinate system; the specific solution to the PnP problem can be flexibly selected according to actual needs.

[0092] Preferably, the present invention will adopt a standard pinhole camera model and assume that there is no camera distortion to improve the robustness of fitting and simplify the complexity of calculation; the focal length of the camera should not be too small, and the present invention will adopt a telephoto lens to reduce the distortion problem caused when the fitted 3D head model is too close to the camera.

[0093] The present invention solves the PnP problem and uses a universal head model to quickly fit the normal vector of the face area in the original image, thereby completing real-time lighting. It has the advantages of good performance, excellent expression, strong universality, etc.

[0094] 5-Simulation lighting rendering module

[0095] (1) Generate a default normal vector texture of the same size as the input image texture or video frame texture. Assuming that the normalized coordinates of a point on the texture are (x, y), the general formula for the default normal vector of the point is:

[0096] normalize(vector3(x*2–1,y*2-1,zDefault),

[0097] The value of zDefault can be flexibly set as needed. Preferably, the present invention adopts 1.031 as the default value;

[0098] (2) Based on the tvec and rvec calculated by the 3D head simulation module, render the 3D head model into a smaller texture, use the normal vector after GPU interpolation as the output, and obtain a blurred normal vector texture of the face area. In particular, if there are multiple faces, render each face in turn and blend it with the previous result normally;

[0099] (3) According to the default normal vector map and the parallel light position information, the default illumination diffuse reflection intensity map DiffuseNormal is obtained. The calculation formula of the diffuse reflection intensity is: max(dot(lightDir,normal),0.0);

[0100] (4) According to the normal vector texture of the face area and the parallel light position information, the diffuse reflection intensity map DiffuseFace of the face area is obtained. The calculation formula of the diffuse reflection intensity is: max(dot(lightDir,normal),0.0);

[0101] (5) According to the face segmentation mask obtained by the face area segmentation module, DiffuseNormal and DiffuseFace are mixed to obtain the comprehensive diffuse reflection result DiffuseCompose. The mixing formula is:

[0102] DiffuseNormal*(1-faceMaskValue)+DiffuseFace*faceMaskValue;

[0103] (6) Perform box blur on DiffuseCompose. The blur degree can be flexibly set according to needs to obtain the blurred diffuse intensity map BlurryDiffuseCompose texture;

[0104] (7) Based on the obtained BlurryDiffuseCompose texture, the original input image texture or video frame texture is illuminated using the Phong lighting model, and the texture result is used as the final output.

[0105] Note: The following algorithms are required when image processing is involved in the method of the present invention:

[0106] The EBCOT optimized interception embedded block coding algorithm is the core coding algorithm of the new generation of still image compression standard JPEG2000. This algorithm makes the code stream have many excellent characteristics such as progressive encoding and decoding, quality and resolution progression; the MQ algorithm is a context-based adaptive binary algorithm. It is based on context to facilitate the removal of source correlation, uses the Bayesian learning process in the conditional exchange and probability estimation state machine to realize the symbol probability model adaptation process, and uses bit filling technology to solve the carry problem in coding. It is a high-efficiency physically implementable compression coding algorithm.

[0107] Finally, it should be noted that: Obviously, the above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from this are still within the scope of protection of the present invention.

Claims

1. A method for simulating 3D lighting for an image video containing a portrait, characterized in that: The specific steps include: S1. Use the universal head model and preset information to create a module. Make a general 3D head model with normal vectors for subsequent 3D fitting of the face in the 2D image. Select a set of key points on the face surface, mark the 3D coordinates of all key points in the head model coordinate system, and use them as the output of this module together with the head model. S2, using the face key point detection module, The image texture or video frame texture received by this module is used as input, and after detecting the key points of the face, the key point data of the face required for the subsequent processing is output; S3, using the face region segmentation module, Since the effective fitting of the head model is concentrated in the face area, it is necessary to segment the face area of ​​the input image texture or video frame texture and generate a face segmentation mask as output; S4, use 3D head to simulate the module, According to the preset camera intrinsic parameters, the face key point results detected by the face key point detection module on the image texture or video frame texture, and the three-dimensional coordinate data of the corresponding key points obtained by the universal head model and the preset information production module, the PnP problem is solved and the corresponding camera extrinsic parameters tvec and rvec are calculated; among which tvec represents how a point in the three-dimensional world coordinate system is parallelly moved to the camera coordinate system; rvec represents how a point in the three-dimensional world coordinate system is rotated and moved to the camera coordinate system; S5. Use the simulated lighting rendering module. Processing the data generated in steps S1, S3 and S4, and according to the processing results, lighting the original input image texture or video frame texture using the Phong lighting model, and the texture result is used as the final output; In S5, the specific scheme of the simulated lighting rendering module is: (1) Generate a default normal vector texture of the same size as the input image texture or video frame texture. Assuming that the normalized coordinates of a point on the texture are (x, y), the general formula for the default normal vector of the point is: normalize(vector3(x*2–1, y*2-1, zDefault)), The value of zDefault uses 1.031 as the default value; the function vector3 is used to package the three input values ​​into a three-dimensional vector, and the function normalize is used to convert a three-dimensional vector into a unit vector; (2) Based on the tvec and rvec calculated by the 3D head simulation module, render the 3D head model into a texture of matching specifications, and use the normal vector after GPU interpolation as the output to obtain a blurred normal vector texture of the face area; if there are multiple faces, render each face in turn and mix it normally with the previous result; (3) According to the default normal vector map and the parallel light position information, the default illumination diffuse reflection intensity map DiffuseNormal is obtained. The calculation formula of diffuse reflection intensity is: max(dot(lightDir, normal), 0.0); where the function dot is a vector dot multiplication function, which is used to perform dot multiplication of the unit vector lightDir and the unit vector normal; the function max is a maximum value function; (4) According to the normal vector texture of the face area and the parallel light position information, the diffuse reflection intensity map DiffuseFace of the face area is obtained. The calculation formula of the diffuse reflection intensity is: max(dot(lightDir, normal), 0.0); (5) According to the face segmentation mask obtained by the face area segmentation module, DiffuseNormal and DiffuseFace are mixed to obtain the comprehensive diffuse reflection result DiffuseCompose. The mixing formula is: DiffuseNormal*(1-faceMaskValue)+DiffuseFace*faceMaskValue Perform box blur on DiffuseCompose to get the blurred diffuse intensity map BlurryDiffuseCompose texture; (6) Based on the obtained BlurryDiffuseCompose texture, the original input image texture or video frame texture is illuminated using the Phong lighting model, and the texture result is used as the final output.

2. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 1, characterized in that: In S1, the center of mass of the 3D head model is located at the origin of the model, and the left-hand or right-hand coordinate system can be freely selected according to the needs of subsequent fitting calculations and 3D rendering.

3. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 2, characterized in that: The right-hand coordinate system is selected, and the face of the head model faces the positive direction of the Z axis, the left ear points to the positive direction of the X axis, and the six key points are the center of the two eyes, the tip of the nose, the sides of the mouth, and the chin.

4. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 1, characterized in that: In S2, the key points output by the facial key point detection module at least include the universal head model and all the key points preset and marked in the preset information making module.

5. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 1, characterized in that: In S2, the face key points detection related to the video is performed, and smoothing processing is also required, and filtering is performed using a special filter. The specific scheme is as follows: (1) For the current frame to be detected, the detection results of all frames with a time interval not exceeding T are taken as a set of reference data; (2) Traverse the reference data obtained in (1) and calculate the time coefficient timeContribute based on the time difference t between the time of the reference data and the current frame time. The calculation formula of the time coefficient is: ; (3) Traverse the reference data obtained in (1), and calculate the distance coefficient distanceContribute according to the Euclidean distance d of the key points corresponding to the reference data. The distance coefficient calculation formula is: ,The value of sigma is set according to the input image size and the error distribution of the face detection model; (4) According to timeContribute * distanceContribute, get the weight of each reference data; (5) According to the weights obtained in (4), the weighted sum is calculated for each key point to obtain the filtering result as the final face key point detection output.

6. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 5, characterized in that: In S2, when there are multiple faces in an image texture or a video frame texture, multiple groups of key point results are output.

7. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 1, characterized in that: In the S3, when there are multiple faces in an image texture or a video frame texture, multiple face segmentation mask images are output as results, and correspond one-to-one with the face key point detection results in the aforementioned face key point detection module.

8. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 1, characterized in that: In S4, the universal 3D head model is transformed from the world coordinate system to the camera coordinate system through the camera extrinsic parameters, and the model coordinate system is directly regarded as the world coordinate system to save the conversion from the local coordinate system to the world coordinate system, and a standard pinhole camera model is adopted, and it is assumed that there is no camera distortion.

9. The method for simulating 3D lighting for an image video containing a portrait as claimed in claim 8, characterized in that: In the S4, the camera lens is a telephoto lens.

Images