Face three-dimensional image processing method and system
By processing and optimizing multiple face two-dimensional images, extracting and giving three-dimensional face model features, the problem of large differences between the three-dimensional model and the real three-dimensional image is solved, and a more accurate and real three-dimensional face model is achieved.
Patent Information
- Application Number
- CN202510010549.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-03
- Publication Date
- 2025-04-18
AI Technical Summary
In the existing three-dimensional image processing technology, the three-dimensional model fails to effectively process three-dimensional images from different perspectives when it is established, resulting in a large difference between the three-dimensional model and the real three-dimensional image.
By obtaining multiple 2D images of human faces, extracting three-dimensional features of human faces, and giving them to general human face grid models for optimization. The specific steps include image differentiation, Laplace transformation, sharpening processing, overlapping and contrasting extraction of distinctive features, and substituting them into a three-dimensional face model for optimization.
The three-dimensional face model is consistently displayed with the sharpened image, the distinctive features are extracted and optimized, and the accuracy and authenticity of the three-dimensional face model are improved.
Smart Images
Figure CN119850469A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and system for processing three-dimensional facial images. Background Art
[0002] With the development and progress of software and hardware technology, the conversion of two-dimensional images into three-dimensional models no longer relies on traditional manual modeling, and the conversion between image models is gradually becoming automated.
[0003] The improved three-dimensional image processing method and system invention disclosed in the prior art generally include an acquisition module, a determination module, a processing module and an electronic device structure, wherein the acquisition module is connected to the position of the determination module, the determination module is connected to the position of the processing module, the electronic device structure includes a memory, a processor and a communication interface, the memory, the processor and the communication interface are all connected to a communication bus, the output end of the acquisition module is electrically connected to the input end of the determination module, the output end of the determination module is electrically connected to the input end of the processing module, and the memory, processor and communication interface are directly or indirectly electrically connected to each other to realize data transmission or interaction.
[0004] The existing technology mainly deals with the call of device memory during the three-dimensional image processing process to avoid system crash when the three-dimensional model image is established. However, the basis for establishing the three-dimensional model in this technology is three-dimensional images from different perspectives, and the three-dimensional image is not processed again, resulting in many differences between the three-dimensional model and the real three-dimensional image. Summary of the invention
[0005] To solve the above problems, the purpose of the present invention is to provide a method and system for processing three-dimensional facial images, which processes the initial two-dimensional facial image through an algorithm, sharpens and highlights the facial features, adjusts the three-dimensional facial model to be consistent with the displayed image of the sharpened image, overlaps and compares the sharpened image with the two-dimensional image presented by the three-dimensional facial model, extracts distinguishing features, and substitutes the distinguishing features into the three-dimensional facial model for optimization.
[0006] The technical solution provided by the present invention is: a method for processing a three-dimensional face image, comprising the following steps: Acquire multiple two-dimensional face images and extract three-dimensional features of the face, wherein the multiple two-dimensional face images include at least a front two-dimensional image, a left side two-dimensional image, and a right side two-dimensional image; Assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features; Performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image; The two-dimensional face image corresponding to the three-dimensional face model at the target angle is obtained, the two-dimensional face image is overlapped and compared with the sharpened image to reveal distinguishing features, and only the distinguishing features are extracted and substituted into the three-dimensional face model to optimize the three-dimensional face model.
[0007] Preferably, the three-dimensional features of the face include but are not limited to face similarity calculation, face contour extraction, face positioning, face internal contour extraction, eye positioning, nose positioning, mouth positioning, eyebrow positioning, ear positioning and chin positioning, to extract the three-dimensional features of the face.
[0008] Preferably, after acquiring the plurality of two-dimensional face images, the method further comprises: The plurality of two-dimensional face images are subjected to normalization processing such as image scaling, image cropping, standardization and normalization.
[0009] Preferably, the general face mesh model is pre-built based on Blender, Tinkercad, or Smoothie-3D.
[0010] Preferably, extracting the three-dimensional features of the face further includes: Using color space mapping, a skin color model is established based on the face image accurately identified by the skin color method, and then a similarity matrix is obtained, which is finally normalized.
[0011] Preferably, assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features further comprises: Sparse point cloud generation, point cloud to model reconstruction, texture image creation and editing and texture model, to obtain the 3D face model input into the face model feature library.
[0012] Preferably, performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image further includes: The image differential adopts the second-order differential. The second-order digital differential requires that the differential value in the constant grayscale area is zero, the differential value at the starting point of the grayscale step or slope is non-zero, and the differential value along the slope is zero. The formula of the second-order digital differential is: Defined as the grayscale value of the pixel on the right + the grayscale value of the pixel on the left - 2*the grayscale value of the current pixel; The formula for second-order differential sharpening is: The basic method of Laplace transform for image enhancement is: The multiple two-dimensional face images and the image after Laplace transformation are added to form a sharpened image, in which the ordinate is intensity and the abscissa is data index.
[0013] Based on the same concept, the present invention also provides a human face three-dimensional image processing system, comprising: Acquire multiple two-dimensional face images and extract three-dimensional features of the face, wherein the multiple two-dimensional face images include at least a front two-dimensional image, a left side two-dimensional image, and a right side two-dimensional image; Assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features; Performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image; The two-dimensional face image corresponding to the three-dimensional face model at the target angle is obtained, the two-dimensional face image is overlapped and compared with the sharpened image to reveal distinguishing features, and only the distinguishing features are extracted and substituted into the three-dimensional face model to optimize the three-dimensional face model.
[0014] Based on the same concept, the present invention also provides an electronic device, comprising: A memory, the memory being used to store a processing program; A processor, wherein when executing the processing program, the processor implements any one of the above-mentioned methods for processing a three-dimensional face image.
[0015] Based on the same concept, the present invention also provides a readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, any one of the above-mentioned methods for processing a three-dimensional face image is implemented.
[0016] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art: The present invention obtains two-dimensional face images from two or more angles, first uses three-dimensional model software to build a general face model, then grids the three-dimensional face model, attaches the built three-dimensional face model features to the general face model, builds a specific three-dimensional face model, and then identifies the skin color through color space mapping to preliminarily build a face model. Then, the initial two-dimensional face image is processed by an algorithm to sharpen and highlight the face features, adjust the three-dimensional face model to be consistent with the sharpened image display image, overlap and compare the sharpened image and the two-dimensional image presented by the three-dimensional face model, extract distinguishing features, and substitute the distinguishing features into the three-dimensional face model for optimization. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flow chart of the method for processing three-dimensional facial images of the present invention. DETAILED DESCRIPTION
[0018] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.
[0019] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0020] First embodiment See also Figure 1 , showing a schematic flow chart of the three-dimensional face image processing of the present invention.
[0021] The present embodiment provides a method for processing a three-dimensional face image, comprising the following steps: Acquire multiple two-dimensional face images and extract three-dimensional features of the face, wherein the multiple two-dimensional face images at least include a front two-dimensional image, a left side two-dimensional image, and a right side two-dimensional image; Assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features; Performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image; The two-dimensional face image corresponding to the three-dimensional face model at the target angle is obtained, the two-dimensional face image is overlapped and compared with the sharpened image to reveal distinguishing features, and only the distinguishing features are extracted and substituted into the three-dimensional face model to optimize the three-dimensional face model.
[0022] The present invention obtains two-dimensional face images at two angles or more, first uses three-dimensional model software to build a general face model, then uses a gridded three-dimensional face model to attach the established face three-dimensional model features to the general face model, builds a specific three-dimensional face model, and then identifies the skin color through color space mapping to preliminarily build a face model, and then processes the initial two-dimensional face image through an algorithm, sharpens and highlights the face features, adjusts the three-dimensional face model to be consistent with the sharpened image display image, overlaps and compares the sharpened image with the two-dimensional image presented by the three-dimensional face model, extracts distinguishing features, and substitutes the distinguishing features into the face three-dimensional model for optimization. The means for acquiring facial features of the person include but are not limited to infrared scanning acquisition and CCD shooting acquisition. When the user needs to obtain high-definition facial features of the person and has corresponding equipment, the infrared scanner and CCD can be used to extract facial features of the person. When the user does not have corresponding equipment support, the camera provided by the mobile terminal can also be used for shooting.
[0023] Preferably, the three-dimensional features of the face include but are not limited to face similarity calculation, face contour extraction, face positioning, face internal contour extraction, eye positioning, nose positioning, mouth positioning, eyebrow positioning, ear positioning and chin positioning, to extract the three-dimensional features of the face.
[0024] By taking multiple two-dimensional images from different angles (front, left side, right side), we can capture facial feature information more comprehensively, which helps to build a more accurate and realistic three-dimensional face model.
[0025] Preferably, after acquiring the plurality of two-dimensional face images, the method further comprises: The plurality of two-dimensional face images are subjected to normalization processing such as image scaling, image cropping, standardization and normalization.
[0026] By scaling and cropping, we can ensure that all images have the same size and proportion, which helps the subsequent image processing steps. At the same time, unified standards and normalization can reduce the differences between different images, allowing the model to better learn and recognize facial features.
[0027] Preferably, the general face mesh model is pre-built based on Blender, Tinkercad, or Smoothie-3D.
[0028] Using these professional 3D modeling software, you can quickly and accurately build a 3D model of the human face. By importing facial feature point data or using the modeling tools that come with the software, you can accurately capture the shape and details of the face to generate a high-quality 3D model.
[0029] Preferably, extracting the three-dimensional features of the face further includes: Using color space mapping, a skin color model is established based on the face image accurately identified by the skin color method, and then a similarity matrix is obtained, which is finally normalized.
[0030] Through color space mapping (such as conversion from RGB to YCbCr), skin color information can be better separated and the impact of lighting changes on detection results can be reduced. The skin color model established based on the skin color method can effectively distinguish skin color areas from non-skin color areas, thereby improving the accuracy of face detection. The calculation of the similarity matrix further quantifies the probability that a pixel belongs to a skin color area, making the detection more precise.
[0031] Preferably, assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features further comprises: Sparse point cloud generation, point cloud to model reconstruction, texture image creation and editing and texture model, to obtain the 3D face model input into the face model feature library.
[0032] By generating sparse point clouds and reconstructing point clouds into models, the detailed features of the face can be accurately captured, thereby improving the accuracy of the 3D face model.
[0033] Preferably, performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image further includes: The image differential adopts the second-order differential. The second-order digital differential requires that the differential value in the constant grayscale area is zero, the differential value at the starting point of the grayscale step or slope is non-zero, and the differential value along the slope is zero. The formula of the second-order digital differential is: Defined as the grayscale value of the pixel on the right + the grayscale value of the pixel on the left - 2*the grayscale value of the current pixel; The formula for second-order differential sharpening is: The basic method of Laplace transform for image enhancement is: The multiple two-dimensional face images and the image after Laplace transformation are added to form a sharpened image, in which the ordinate is intensity and the abscissa is data index.
[0034] Through the second-order differential, the high-frequency components in the image, such as edges and textures, can be effectively highlighted, making the details in the image clearer, which is helpful for subsequent three-dimensional modeling and feature extraction. Second-order differential sharpening can enhance the contrast of the image and make the differences between different areas in the image more obvious. This is crucial for application scenarios such as face recognition and expression analysis, because these applications usually rely on subtle differences in the image to make judgments. During the image processing process, the image quality may be degraded due to various reasons (such as uneven lighting, camera noise, etc.). Through sharpening processing such as second-order differential and Laplace transform, the image quality can be improved to a certain extent, and the image usability and accuracy can be improved.
[0035] Based on the same concept, the present invention also provides a human face three-dimensional image processing system, comprising: Acquire multiple two-dimensional face images and extract three-dimensional features of the face, wherein the multiple two-dimensional face images include at least a front two-dimensional image, a left side two-dimensional image, and a right side two-dimensional image; Assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features; Performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image; The two-dimensional face image corresponding to the three-dimensional face model at the target angle is obtained, the two-dimensional face image is overlapped and compared with the sharpened image to reveal distinguishing features, and only the distinguishing features are extracted and substituted into the three-dimensional face model to optimize the three-dimensional face model.
[0036] Based on the same concept, the present invention also provides an electronic device, comprising: A memory, the memory being used to store a processing program; A processor, wherein when executing the processing program, the processor implements any one of the above-mentioned methods for processing a three-dimensional face image.
[0037] Based on the same concept, the present invention also provides a readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, any one of the above-mentioned methods for processing a three-dimensional face image is implemented.
[0038] The present invention obtains two-dimensional face images from two or more angles, first uses three-dimensional model software to build a general face model, then grids the three-dimensional face model, attaches the built three-dimensional face model features to the general face model, builds a specific three-dimensional face model, and then identifies the skin color through color space mapping to preliminarily build a face model. Then, the initial two-dimensional face image is processed by an algorithm to sharpen and highlight the face features, adjust the three-dimensional face model to be consistent with the sharpened image display image, overlap and compare the sharpened image and the two-dimensional image presented by the three-dimensional face model, extract distinguishing features, and substitute the distinguishing features into the three-dimensional face model for optimization.
[0039] If the human face three-dimensional image processing method is implemented in the form of program instructions and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of software. The computer software is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0040] Those skilled in the art can clearly understand that, for the sake of convenience and brevity in description, the identification content specifically executed by the above-described system and device can refer to the corresponding process in the aforementioned method embodiment.
[0041] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the protection scope of the present invention.
Claims
1. A method for processing a three-dimensional face image, characterized in that: The following steps are involved: Acquire multiple two-dimensional face images and extract three-dimensional features of the face, wherein the multiple two-dimensional face images include at least a front two-dimensional image, a left side two-dimensional image, and a right side two-dimensional image; Assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features; Performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image; The two-dimensional face image corresponding to the three-dimensional face model at the target angle is obtained, the two-dimensional face image is overlapped and compared with the sharpened image to reveal distinguishing features, and only the distinguishing features are extracted and substituted into the three-dimensional face model to optimize the three-dimensional face model.
2. The method for processing a three-dimensional face image according to claim 1, characterized in that: The three-dimensional features of the face include but are not limited to face similarity calculation, face contour extraction, face positioning, face internal contour extraction, eye positioning, nose positioning, mouth positioning, eyebrow positioning, ear positioning and chin positioning, and extracting the three-dimensional features of the face.
3. The method for processing a three-dimensional face image according to claim 1, characterized in that: After obtaining multiple two-dimensional face images, the method further includes: The plurality of two-dimensional face images are subjected to normalization processing such as image scaling, image cropping, standard unification and normalization.
4. The method for processing a three-dimensional face image according to claim 1, characterized in that: The general face mesh model is pre-built based on Blender, Tinkercad, and Smoothie-3D.
5. The method for processing a three-dimensional face image according to claim 1, characterized in that: Extracting three-dimensional features of the face further includes: Using color space mapping, a skin color model is established based on the face image accurately identified by the skin color method, and then a similarity matrix is obtained, which is finally normalized.
6. The method for processing a three-dimensional face image according to claim 1, characterized in that: Assigning the three-dimensional face features to a general face mesh model to form a three-dimensional face model integrating the three-dimensional face features further comprises: Sparse point cloud generation, point cloud to model reconstruction, texture image creation and editing and texture model, to obtain the 3D face model input into the face model feature library.
7. The method for processing a three-dimensional face image according to claim 1, characterized in that: Performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image further includes: The image differential adopts the second-order differential. The second-order digital differential requires that the differential value in the constant grayscale area is zero, the differential value at the starting point of the grayscale step or slope is non-zero, and the differential value along the slope is zero. The formula of the second-order digital differential is: Defined as the grayscale value of the pixel on the right + the grayscale value of the pixel on the left - 2*the grayscale value of the current pixel; The formula for second-order differential sharpening is: The basic method of Laplace transform for image enhancement is: The multiple two-dimensional face images and the image after Laplace transformation are added to form a sharpened image, in which the ordinate is intensity and the abscissa is data index.
8. A three-dimensional face image processing system, characterized in that: include: Acquire multiple two-dimensional face images and extract three-dimensional features of the face, wherein the multiple two-dimensional face images at least include a front two-dimensional image, a left side two-dimensional image, and a right side two-dimensional image; Assigning the three-dimensional facial features to a general facial mesh model to form a three-dimensional facial model integrating the three-dimensional facial features; Performing image differentiation, digital differentiation, and Laplace transform of the image function on the acquired multiple two-dimensional face images to achieve second-order differential sharpening to obtain a sharpened image; The two-dimensional face image corresponding to the three-dimensional face model at the target angle is obtained, the two-dimensional face image is overlapped and compared with the sharpened image to reveal distinguishing features, and only the distinguishing features are extracted and substituted into the three-dimensional face model to optimize the three-dimensional face model.
9. An electronic device, characterized in that: include: A memory, the memory being used to store a processing program; A processor, wherein when executing the processing program, the processor implements the three-dimensional face image processing method as described in any one of claims 1 to 7.
10. A readable storage medium, characterized in that: The readable storage medium stores a processing program, and when the processing program is executed by a processor, the three-dimensional face image processing method according to any one of claims 1 to 7 is implemented.