Method for transforming face image in virtual scene

The three-dimensional face model is obtained through micro laser 3D scanning and UVC cameras, and the light source and angle are adjusted in Unity, which solves the problem of difficulty in authenticating face image identity in virtualized scenes by traditional video equipment, and improves image recognition and Authenticity.

CN119091033APending Publication Date: 2024-12-06SUZHOU XIAOYOU INTELLIGENT TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202310996513.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-08-09
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The limitations of traditional video equipment lead to difficulty in authenticating the identity of face images in virtualized scenes, and changes in shooting conditions affect image clarity and authenticity.

Method used

A micro laser 3D scanning module is used to obtain a three-dimensional face point cloud, and a UVC camera is used to acquire RGB images, perform feature point registration and grid fitting, generate a complete three-dimensional model, and adjust the light source and angle in Unity to simulate different environments and Face rotation effect.

Benefits of technology

It improves the recognition and authenticity of face images, and obtains high-quality multi-angle face information through three-dimensional scanning and light adjustment, thereby enhancing the recognition and authenticity of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091033A_ABST
    Figure CN119091033A_ABST
Patent Text Reader

Abstract

The invention discloses a face image transformation method in a virtual scene. The method comprises the following steps: reconstructing a face three-dimensional point cloud by using a 3D acquisition reconstruction technology; a 3D splicing technology is utilized, a face area, needing fitting transformation, of the model is obtained according to a distance threshold value, grid fitting iteration transformation is carried out on the face model until convergence is achieved, texture remapping is carried out on the fitted complete face model, and the faces collected by the multiple cameras are spliced into a complete face; the spliced human face is loaded to a Unity scene; and adjusting the deflection angle to obtain face information with high recognition degree. According to the face image transformation method in the virtual scene, a novel structural design is adopted, the face image change adjustment identification degree and authenticity are improved, reconstruction is performed after a plurality of 3D structured light devices are used for scanning a face, obtained scanning models are spliced to obtain a complete face, the light angle and the like of the environment where the face is located are further simulated, and the face image transformation efficiency is improved. The best matching effect is achieved, and the recognition accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of portrait inspection and identification in the field of image analysis, and in particular to a method for transforming a face image in a virtualized scene. Background Art

[0002] The inspection and identification of facial images of people involved in cases is increasingly urgent as evidence for analysis and solving cases, especially as the only evidence or key evidence in court proceedings. Facial images of people involved in cases are collected through surveillance and other shooting equipment. The angle and light of surveillance shooting are affected by the environment, and the clarity of the facial images shot is limited;

[0003] A facial image fusion method based on facial key points with application number CN113052783A includes the following steps: (a) obtaining facial key points; (b) drawing a contour map of non-empty pixels through the eye, nose and lip coordinate points given by the obtained facial key points, and then performing an OR operation with the original image to obtain a mask map. The mask map refers to a face frame without any other background; (c) after the input reference face has obtained the mask map, the target face needs to be aligned to the original reference face; (d) when there is a large color difference between the two pictures input by the user, the color of the target face needs to be calibrated to be close to the color of the reference face, (e) after the color correction of the character is weighted averaged, the face of the reference picture can be fused. It can solve the problems of determining facial key points, aligning facial orientation, eliminating color difference between the reference face and the target face, and feathering of the face fitting edge; this facial image fusion method is not convenient for adjusting facial image information in a virtual scene, and the authenticity of the image after face adjustment and repair is affected;

[0004] Since shooting is affected by changes in scene, light, angle and other factors, traditional two-dimensional face recognition is limited by conditions, and the matching method has a high error rate or is difficult to identify. The limitations of traditional video equipment lead to the inclusion of many influencing factors in the facial images of the specimens, which makes it difficult to authenticate the identity of the facial images.

[0005] Therefore, it is necessary to design a method for transforming facial images in virtual scenes to address the above problems. Summary of the invention

[0006] The purpose of the present invention is to provide a method for transforming a facial image in a virtualized scene, so as to solve the problems raised in the above background technology that the limitations of traditional video equipment lead to the inclusion of many influencing factors in the facial images of the specimens, which brings difficulties to the identity authentication of the facial images.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for transforming a face image in a virtual scene, comprising the following steps:

[0008] S1: Use a micro laser 3D scanning module to scan the left and right faces respectively. After loading the image sequence of structured light modulation, perform phase unwrapping, perform distortion correction on the unwrapped image, calculate the matching container for stereo matching, and then calculate the 3D point cloud based on the disparity map. Then perform noise reduction processing to obtain the final point cloud scanned by a single module.

[0009] S2: Use the calibrated UVC camera to collect RGB images and obtain the RGB information of the 3D face;

[0010] S3: Obtain feature points from the scanned face, perform initial registration and alignment with the complete face model with known feature points, obtain the face area that needs to be fitted and transformed according to the distance threshold, perform grid fitting and iterative deformation on the face model until convergence, perform texture remapping on the fitted complete face model, obtain a three-dimensional model with complete face texture information, and write it into a file in Obj format for storage;

[0011] S4: Load the model saved in obj format through file reading in Unity and adjust the model position;

[0012] S5: Add two light sources to the left and right front of the model, and make the brightness of the light sources adjustable. By adjusting them separately, you can achieve the effect of lighting the left and right sides of the face. By adjusting them simultaneously, you can achieve the effect of lighting the front face. The brightness of the light sources can be adjusted as needed.

[0013] S6: Obtain multi-angle face images by rotating the face angle instead of rotating the camera position. Gradually rotate with a small angle difference within a certain range to obtain images with approximately continuous angles, simulating the effect of face rotation to increase the accuracy of later matching.

[0014] Preferably, the Unity can simulate the brightness of human faces in various environments by adjusting the angle distance and intensity of the two light sources to assist in adjusting the brightness of the light source;

[0015] When it is outdoors during the day, move the light source farther away and increase the brightness to make the lighting even;

[0016] When in a bright indoor environment, the distance between the two light sources is increased, the distance to the face is moved closer, and the brightness is increased. When in a low-brightness environment, the light source is adjusted according to the angle and light direction and the brightness is reduced.

[0017] By adjusting the light source, the real ambient light can be simulated to improve the authenticity of the face image adjustment.

[0018] Preferably, the Unity projects the face, decomposes the face rotation angle into lateral rotation and longitudinal rotation, sets the starting position of the lateral rotation, i.e., angle α, the ending position of the lateral rotation, i.e., angle β, the step length of the lateral rotation, i.e., the angle γ of each rotation, sets the starting position of the longitudinal rotation, i.e., angle δ, the ending position of the longitudinal rotation, i.e., angle φ, and the step length of the longitudinal rotation, i.e., the angle ε of each rotation;

[0019] According to the acquired α, β, γ, δ, φ, ε, traverse, by controlling the start and end positions and the step angle, determine the number of images collected and the face angle range, and obtain the corresponding face image;

[0020] Adjust the angle of the face to enhance the recognition and authenticity of the image after face adjustment.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: the method for transforming a face image in a virtual scene, after three-dimensional scanning and splicing to obtain a real 3D face, adjusts the light and obtains face information at any angle, and processes the three-dimensional model with the help of Unity, which can meet the requirements. The specific contents are as follows:

[0022] Use a micro laser 3D scanning module to collect facial information and obtain a 3D facial point cloud image; use a calibrated UVC camera to collect RGB images to obtain RGB information of the 3D face; stitch the facial point cloud images collected by multiple micro laser 3D scanning modules to obtain a complete facial information, and map the corresponding RGB information of the stitched face; load the complete facial data into the Unity scene; adjust the light to an appropriate brightness environment according to the scene; adjust the facial angle to obtain a specific angle or full angle facial image according to needs; the facial information spliced ​​by this variation method has a high degree of recognition and authenticity. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the face image change operation flow of the present invention;

[0024] Figure 2 A schematic diagram of a portion of a human face acquired by a single 3D scanning module of the present invention;

[0025] Figure 3 A schematic diagram of a facial image collected by the present invention;

[0026] Figure 4 A schematic diagram of the effective face area ultimately used in the present invention;

[0027] Figure 5 This is a schematic diagram of a 3D face of a person after the splicing is completed in the present invention;

[0028] Figure 6This is a schematic diagram of a facial image obtained by adding a simulated light source according to the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] See also Figure 1 The present invention provides a technical solution: a method for transforming a face image in a virtual scene, which mainly includes the following steps:

[0031] Example

[0032] Step 1: Use a micro laser 3D scanning module to scan the left and right faces respectively, load the image sequence of structured light modulation, perform phase unwrapping, perform distortion correction on the unwrapped image, calculate the matching container for stereo matching, and then calculate the 3D point cloud based on the disparity map, and then perform noise reduction to obtain the final point cloud scanned by a single module;

[0033] Step 2: Use a calibrated UVC camera to capture RGB images and obtain the RGB information of the 3D face;

[0034] Step 3: Obtain feature points from the scanned face, perform initial registration and alignment with the complete face model with known feature points, obtain the face area that needs to be fitted and transformed according to the distance threshold, perform grid fitting and iterative deformation on the face model until convergence, perform texture remapping on the fitted complete face model, obtain a three-dimensional model with complete face texture information, and save it in a file in Obj format;

[0035] Step 4: Load the model saved in obj format by reading the file in Unity and adjust the model position;

[0036] Step 5. Add two light sources to the left and right front of the model. Make the light source brightness adjustable. Adjusting them separately can achieve the effect of lighting the left and right sides of the face. Adjusting them together can achieve the effect of lighting the front face. The light source brightness can be adjusted as needed.

[0037] By adjusting the angle, distance and intensity of the two light sources, it is basically possible to simulate the brightness of the face in various environments. When in an outdoor environment during the day, the light source is farther away and the brightness is increased to make the lighting uniform; when in a bright indoor environment, the two light sources are farther away and closer to the face, and the brightness is increased. When in an environment with lower brightness, the light source is adjusted according to the angle and light direction and the brightness is reduced.

[0038] Step 6: Obtain multi-angle face images by rotating the face angle instead of rotating the camera position. Gradually rotate with a small angle difference within a certain range to obtain images with approximately continuous angles, simulating the effect of face rotation to increase the accuracy of later matching.

[0039] Decompose the face rotation angle into horizontal rotation and vertical rotation, set the starting position of the horizontal rotation, that is, angle α, the ending position of the horizontal rotation, that is, angle β, and the step length of the horizontal rotation, that is, the angle γ of each rotation; set the starting position of the vertical rotation, that is, angle δ, the ending position of the vertical rotation, that is, angle φ, and the step length of the vertical rotation, that is, the angle ε of each rotation, and traverse according to the obtained α, β, γ, δ, φ, ε, and determine the number of collected images and the range of face angles by controlling the starting and ending positions and the step length angles to obtain the corresponding face image;

[0040] The acquired facial images have high recognition and authenticity.

[0041] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for transforming a face image in a virtual scene, comprising the following steps, characterized in that: S1: Use a micro laser 3D scanning module to scan the left and right faces respectively. After loading the image sequence of structured light modulation, perform phase unwrapping, perform distortion correction on the unwrapped image, calculate the matching container for stereo matching, and then calculate the 3D point cloud based on the disparity map. Then perform noise reduction processing to obtain the final point cloud scanned by a single module. S2: Use the calibrated UVC camera to collect RGB images and obtain the RGB information of the 3D face; S3: Obtain feature points from the scanned face, perform initial registration and alignment with the complete face model with known feature points, obtain the face area that needs to be fitted and transformed according to the distance threshold, perform grid fitting and iterative deformation on the face model until convergence, perform texture remapping on the fitted complete face model, obtain a three-dimensional model with complete face texture information, and write it into a file in Obj format for storage; S4: Load the model saved in obj format through file reading in Unity and adjust the model position; S5: Add two light sources to the left and right front of the model, and make the brightness of the light sources adjustable. By adjusting them separately, you can achieve the effect of lighting the left and right sides of the face. By adjusting them simultaneously, you can achieve the effect of lighting the front face. The brightness of the light sources can be adjusted as needed. S6: Obtain multi-angle face images by rotating the face angle instead of rotating the camera position. Gradually rotate with a small angle difference within a certain range to obtain images with approximately continuous angles, simulating the effect of face rotation to increase the accuracy of later matching.

2. The method for transforming a facial image in a virtual scene according to claim 1, characterized in that: The Unity can simulate the brightness of human faces in various environments by adjusting the angle distance and intensity of the two light sources to assist in adjusting the brightness of the light source; When it is outdoors during the day, move the light source farther away and increase the brightness to make the lighting even; When in a bright indoor environment, the distance between the two light sources is increased, the distance to the face is moved closer, and the brightness is increased. When in a lower-brightness environment, the light source is adjusted according to the angle and light direction and the brightness is lowered.

3. The method for transforming a facial image in a virtual scene according to claim 1, characterized in that: The Unity projects the face, decomposes the face rotation angle into horizontal rotation and vertical rotation, sets the starting position of the horizontal rotation, i.e., angle α, the ending position of the horizontal rotation, i.e., angle β, the step length of the horizontal rotation, i.e., the angle γ of each rotation, sets the starting position of the vertical rotation, i.e., angle δ, the ending position of the vertical rotation, i.e., angle φ, and the step length of the vertical rotation, i.e., the angle ε of each rotation; According to the obtained α, β, γ, δ, φ, ε, traverse, by controlling the start and end positions and the step angle, determine the number of collected photos and the face angle range, and obtain the corresponding face image.

Citation Information

Patent Citations

  • Face image fusion method based on face key points

    CN113052783A