Image enhancement method and system for simulating camera shooting at any viewing angle

Abstract

The invention provides an image enhancement method and system for simulating camera shooting at any viewing angle, and relates to the field of computer technology. First, facial expression images at different viewing angles are obtained, and a three-dimensional face model is generated according to the facial expression images. Then, a closed three-dimensional face model is generated by closing the mouth and face edge of the three-dimensional face model. Then, the facial expression image is expanded according to the closed three-dimensional face model, so as to remove the viewing angle information of the facial expression image at each viewing angle. Then, the expanded facial expression image is synthesized to generate a closed three-dimensional face map. Finally, the facial expression image at any viewing angle is generated according to the closed three-dimensional face model and the closed three-dimensional face map, so as to achieve obtaining a facial expression image at any viewing angle and with good consistency when only facial expression images at several viewing angles are collected.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image enhancement method for simulating camera shooting at any viewing angle. Background Art

[0002] In the field of AI, the quality of model training results depends largely on the quality of data. To ensure that the model can complete recognition tasks in multiple environments, the most common approach is to continuously collect data that the model has not learned. The main drawback is that continuously collecting data that the model has not learned requires a lot of manpower, and it is necessary to consider that errors in the process of manually producing data will affect the effect of high-precision regression tasks.

[0003] Of course, preprocessing the data is a more labor-saving method. Common image preprocessing methods include: changing the contrast and resolution of the image, rotating the image in the plane, adding random noise to the pixels of the image, etc. In the facial expression application scenario, if all the above data preprocessing methods are used, the data of the image is enhanced on the two-dimensional plane or at the pixel value level. If the expression effect needs to be consistent at any viewing angle, the above image enhancement methods cannot solve the problem. Summary of the invention

[0004] The purpose of the present invention is to solve the problems mentioned in the background technology and to provide an image enhancement method and system for simulating camera shooting at any viewing angle.

[0005] To achieve the above object, the present invention first proposes an image enhancement method for simulating camera shooting at any viewing angle, comprising the following steps:

[0006] Obtain facial expression images from different perspectives;

[0007] Generate a three-dimensional face model based on the facial expression image;

[0008] Generate a closed three-dimensional face model by closing the mouth and face edge of the three-dimensional face model;

[0009] Expanding the facial expression image according to the closed three-dimensional face model, thereby removing the perspective information of the facial expression image at each perspective;

[0010] By synthesizing the unfolded facial expression image, a closed three-dimensional face map is generated;

[0011] Generate facial expression images at any viewing angle based on a closed three-dimensional face model and a closed three-dimensional face map.

[0012] Optionally, obtaining a three-dimensional face model based on a facial expression image includes the following steps: annotating the facial expression image to obtain key point annotations; obtaining a transfer matrix, wherein the transfer matrix is ​​a relative displacement parameter between a camera and a basic three-dimensional face model when the facial expression image coincides with a basic face image, and the basic face image is a two-dimensional image obtained by rendering a basic three-dimensional face model; establishing a corresponding relationship between the basic three-dimensional face model and its key point annotations, such that S=map(P(f*T*M)+pix_bias)), wherein S represents the key point annotations, map represents the index model vertices, P(x, y, z)=[x / z, y / z, z / z], x, y, z represent spatial coordinates, f represents the camera focal length, T represents the transfer matrix, M represents the basic three-dimensional face model, and pix_bias represents the pixel center error; and optimizing the basic three-dimensional face model according to the corresponding relationship to generate a three-dimensional face model.

[0013] Optionally, performing a closing operation on the oral cavity of the three-dimensional face model specifically comprises the following steps: selecting N vertices Vl at the inner lips of the three-dimensional face model, wherein the vertices Vl are arranged in a circle along the inner lips in sequence; specifying a triangular face F according to the vertices Vl; i =(Vl i ,Vl i+1 ,Vl i+2 ), where i represents the i-th vertex.

[0014] Optionally, N is 10.

[0015] Optionally, performing a closing operation on the face edge of the three-dimensional face model specifically includes the following steps: selecting M vertices Vb at the face edge of the three-dimensional face model, wherein the vertices Vb are arranged in a circle along the face edge in sequence; calculating a unit normal vector n of each vertex Vb; selecting a vertex Vbo at a distance l from the normal vector direction of each vertex Vb, wherein Vbo=l*n+Vb; specifying a triangular face F according to the vertices Vb and Vbo; i1 =(Vb i ,Vbo i ,Vb i+1 ), triangle F i2 =(Vbo i ,Vb i+1 ,Vbo i+1 ), i represents the i-th vertex.

[0016] Optionally, M is 15.

[0017] Optionally, the unfolding of the facial expression image according to the closed three-dimensional face model includes the following steps: obtaining all visible vertices on the closed three-dimensional face model and visible triangular faces formed by the visible vertices; converting the closed three-dimensional face model to a pixel coordinate system; obtaining the centroid coordinates corresponding to each pixel in the pixel coordinate system of the closed three-dimensional face model, thereby obtaining the centroid coordinates corresponding to each pixel of the facial expression image; obtaining the pixel coordinates PIwarp of the unfolded facial expression image, PIwarp=u*Pvis Vd1 +v*Pvis Vd2 +w*Pvis Vd3 , where Pvis Vd1 、Pvis Vd2 、Pvis Vd3 The pixel coordinates corresponding to the three vertices of each visible triangle, u, v, w are respectively the components of the centroid coordinates corresponding to each pixel of the facial expression image; the facial expression image is sampled by the pixel coordinates PIwarp to obtain the unfolded facial expression image Iwarp.

[0018] Optionally, facial expression images from upper and lower perspectives are collected by two cameras placed side by side in the upper and lower parts of the face capture helmet.

[0019] Optionally, the two expanded images are synthesized using the formula Icom=w1*Iwarp1+(1-w1)*Iwarp2, where Icom represents the generated closed three-dimensional face map, Iwarp1 and Iwarp2 represent the expanded images of the facial expression images of the upper and lower perspectives, respectively, and w1 represents an adjustable coefficient.

[0020] The embodiment of the present invention also provides an image enhancement system for simulating camera shooting at any viewing angle, comprising: a facial expression image acquisition module, the facial expression image acquisition module is configured to acquire facial expression images at different viewing angles; a three-dimensional face model generation module, the three-dimensional face model generation module is configured to generate a three-dimensional face model according to the facial expression image; a closed three-dimensional face model generation module, the closed three-dimensional face model generation module is configured to generate a closed three-dimensional face model by closing the mouth and face edge of the three-dimensional face model; a facial expression image expansion module, the facial expression image expansion module is configured to expand the facial expression image according to the closed three-dimensional face model, thereby removing the viewing angle information of the facial expression image at each viewing angle; a facial map generation module, the facial map generation module is configured to generate a closed three-dimensional face map by synthesizing the expanded facial expression image; and a facial expression image generation module, the facial expression image generation module is configured to generate a facial expression image at any viewing angle according to the closed three-dimensional face model and the closed three-dimensional face map.

[0021] Beneficial effects of the present invention:

[0022] The embodiment of the present invention provides an image enhancement method and system for simulating camera shooting at any viewing angle. The method first obtains facial expression images at different viewing angles, generates a three-dimensional face model based on the facial expression images, then generates a closed three-dimensional face model by closing the mouth and face edges of the three-dimensional face model, and then expands the facial expression image based on the closed three-dimensional face model to remove the viewing angle information of the facial expression image at each viewing angle. The expanded facial expression image is then synthesized to generate a closed three-dimensional face map. Finally, the facial expression image at any viewing angle is generated based on the closed three-dimensional face model and the closed three-dimensional face map, thereby achieving the acquisition of facial expression images at any viewing angle and with good consistency when only facial expression images at a few viewing angles are collected.

[0023] The features and advantages of the present invention will be described in detail through embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is one of the flowcharts of an image enhancement method for simulating camera shooting at any viewing angle according to an embodiment of the present invention;

[0025] Figure 2 This is a second flowchart of an image enhancement method for simulating camera shooting at any viewing angle according to an embodiment of the present invention;

[0026] Figure 3 This is a flowchart of a method for enhancing an image by simulating shooting at any viewing angle of a camera according to an embodiment of the present invention;

[0027] Figure 4 This is a fourth flowchart of an image enhancement method for simulating camera shooting at any viewing angle according to an embodiment of the present invention;

[0028] Figure 5 This is a flowchart of a method for enhancing an image by simulating shooting at any viewing angle of a camera according to an embodiment of the present invention;

[0029] Figure 6 This is one of the structural block diagrams of an image enhancement system for simulating camera shooting at any viewing angle according to an embodiment of the present invention;

[0030] Figure 7 This is a second structural block diagram of an image enhancement system for simulating camera shooting at any viewing angle according to an embodiment of the present invention;

[0031] Figure 8 This is a third structural block diagram of an image enhancement system for simulating camera shooting at any viewing angle according to an embodiment of the present invention;

[0032] Fig. 9This is the fourth structural block diagram of an image enhancement system simulating camera shooting at any viewing angle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to facilitate understanding by those skilled in the art, the present invention will be further described in detail below in conjunction with specific embodiments.

[0034] See also Figure 1 The embodiment of the present invention provides an image enhancement method for simulating camera shooting at any viewing angle, comprising the following steps:

[0035] Step S10, acquiring facial expression images at different viewing angles.

[0036] In this step, facial expression images from any several viewing angles can be simultaneously captured by multiple image capture devices disposed at different positions on the target face, and there is no limit to the number of image capture devices.

[0037] Preferably, facial expression images from the upper and lower perspectives can be collected by placing two cameras side by side on the face capture helmet. It should be noted that the face capture helmet is a device specifically for collecting facial images, and has the advantages of fixed lighting environment and camera position. Compared with the use of ordinary cameras for face collection, the face capture helmet can provide cleaner image data, thereby ensuring that the three-dimensional model reconstructed using the facial expression image is more like the target face and more realistic. In addition, the purpose of collecting facial expression images from the upper and lower perspectives is to obtain the facial expression movement information of the jaw and the front of the face.

[0038] Step S20, generating a three-dimensional face model according to the facial expression image.

[0039] In this step, the principle of generating a three-dimensional face model based on a facial expression image is first introduced: first, the facial expression image is annotated to obtain key point annotations S, and then the camera focal length f is kept unchanged and the camera is moved so that the two-dimensional image obtained by the basic three-dimensional face model M coincides with the facial expression image. At this time, the displacement of the camera relative to the basic three-dimensional face model M is the transfer matrix T. In practical applications, the camera focal length f and the transfer matrix T at each viewing angle can be obtained through calibration.

[0040] Then, the correspondence between the basic three-dimensional face model M and the key point annotation S is established by indexing the vertices on the basic three-dimensional face model M. Because in the existing process of generating three-dimensional face models, any three-dimensional face model can be represented by adding an offset ds to the vertices of the basic three-dimensional face model, the optimization problem of the three-dimensional face model and the basic three-dimensional face model M can be constructed based on the above correspondence, and then the value of the offset ds is calculated through the existing nonlinear optimization framework, thereby realizing the generation of a three-dimensional face model according to the facial expression image.

[0041] Preferably, according to Figure 2 The process shown in the figure is used to generate a three-dimensional face model, including the following steps:

[0042] Step S210, annotating the facial expression image to obtain key point annotations;

[0043] Step S220, obtaining a transfer matrix, wherein the transfer matrix is ​​a relative displacement parameter between the camera and the basic three-dimensional face model when the facial expression image and the basic face image overlap, wherein the basic face image is a two-dimensional image obtained by rendering the basic three-dimensional face model;

[0044] Step S230, establish a correspondence between the basic three-dimensional face model and its key point annotations, so that S = map(P(f*T*M)+pix_bias)), where S represents the key point annotations, map represents the index model vertices, P(x, y, z) = [x / z, y / z, z / z], x, y, z represent spatial coordinates, f represents the camera focal length, T represents the transfer matrix, M represents the basic three-dimensional face model, and pix_bias represents the pixel center error.

[0045] Step S240: optimizing the basic three-dimensional face model according to the corresponding relationship to generate a three-dimensional face model.

[0046] In this embodiment, considering that the camera is very close to the photographed face, a perspective operation P(x, y, z) = [x / z, y / z, z / z] is introduced to eliminate the error caused by the perspective phenomenon, so that the subsequent facial expression image expansion and synthesis can be better and more accurate. The invention of this embodiment is to create a simplified camera model P(f*T*M)+pix_bias), and establish a corresponding relationship between the basic three-dimensional face model M and the key point annotation S. Compared with the existing three-dimensional face model generation method, it has the advantages of lower computational complexity and better effect.

[0047] Step S30, generating a closed three-dimensional face model by closing the mouth and face edges of the three-dimensional face model.

[0048] Please refer to Figure 3 , wherein the closing operation on the oral cavity of the three-dimensional face model specifically includes the following steps:

[0049] Step S310, select N vertices Vl at the inner lips of the three-dimensional face model, and the vertices Vl are arranged in a circle along the inner lips in sequence. When N is 10, it can reduce the required calculation amount without affecting the effect of the closed three-dimensional face model finally generated, so it is preferred to select 10 vertices Vl at the inner lips of the three-dimensional face model.

[0050] Step S320, specify the triangle F according to the vertex V1 i =(Vl i ,Vl i+1 ,Vl i+2 ), where i represents the i-th vertex.

[0051] In this embodiment, according to the N vertices Vl at the inner lip specified above, three vertices located at the upper and lower lips are selected in turn to create triangular faces corresponding to the vertices. When N is 10, i is less than 9.

[0052] Please refer to Figure 4 The closing operation on the face edge of the three-dimensional face model specifically includes the following steps:

[0053] Step S330, select M vertices Vb at the edge of the face of the 3D face model, and the vertices Vb are arranged in a circle along the edge of the face in order. Since when M is 15, it can reduce the required calculation amount without affecting the effect of the closed 3D face model finally generated, so it is preferred to select 15 vertices Vb at the edge of the face of the 3D face model.

[0054] Step S340, calculating the unit normal vector n of each vertex Vb.

[0055] Step S350, select vertex Vbo at a distance l from the normal vector direction of each vertex Vb, Vbo = l*n + Vb. Since the vertices of the upper lip and the lower lip of the mouth can form a surface, and the vertices on the edge contour of the face need to form a plane with the environment, the face closure has an additional step of reselecting vertices compared to the mouth closure.

[0056] Step S360, specify the triangle F according to the vertex Vb and the vertex Vbo i1 =(Vb i ,Vbo i ,Vb i+1 ), triangle F i2 =(Vbo i ,Vb i+1 ,Vbo i+1 ), i represents the i-th vertex.

[0057] Step S40, expanding the facial expression image according to the closed three-dimensional face model, thereby removing the perspective information of the facial expression image at each perspective.

[0058] Please refer to Figure 5 ,Expanding the facial expression image according to the closed three-dimensional face model includes the following steps;

[0059] Step S410, obtaining all visible vertices on the closed three-dimensional face model and visible triangular surfaces formed by the visible vertices.

[0060] Step S420: convert the closed three-dimensional face model into a pixel coordinate system.

[0061] Step S430, obtaining the centroid coordinates corresponding to each pixel in the closed three-dimensional face model pixel coordinate system, thereby obtaining the centroid coordinates corresponding to each pixel in the facial expression image.

[0062] Specifically, by determining the visible triangle to which each pixel coordinate of the closed three-dimensional face model belongs, the centroid coordinates [u, v, w] corresponding to each pixel of the closed three-dimensional face model are obtained. Because the pixel coordinates of the facial expression image and the closed three-dimensional face model completely coincide with each other, the centroid coordinates [u, v, w] corresponding to each pixel of the facial expression image can be obtained.

[0063] Step S440, obtaining the pixel coordinates PIwarp of the expanded facial expression image, PIwarp = u*Pvis Vd1 +v*Pvis Vd2 +w*Pvis Vd3 , where Pvis Vd1 、Pvis Vd2 、Pvis Vd3 are the pixel coordinates corresponding to the three vertices of each visible triangle, and u, v, and w are the components of the centroid coordinates corresponding to each pixel of the facial expression image.

[0064] Step S450, sampling the facial expression image according to the pixel coordinates PIwarp to obtain an expanded facial expression image Iwarp.

[0065] Step S50, generating a closed three-dimensional face map by synthesizing the unfolded facial expression image.

[0066] When there are two viewing angles, the two expanded images are synthesized by the formula Icom=w1*Iwarp1+(1-w1)*Iwarp2, where Icom represents the generated closed three-dimensional face map, Iwarp1 and Iwarp2 represent the expanded images of the facial expression images of the upper and lower viewing angles, respectively, and w1 represents an adjustable coefficient. In the case of multiple viewing angles, those skilled in the art can add items representing additional viewing angles to the above formula, which will not be repeated here.

[0067] Step S60, generating a facial expression image at any viewing angle according to the closed three-dimensional face model and the closed three-dimensional face map.

[0068] In summary, an embodiment of the present invention provides an image enhancement method for simulating camera shooting at any viewing angle. Facial expression images at different viewing angles are first obtained, and a three-dimensional face model is generated according to the facial expression images. Then, a closed three-dimensional face model is generated by closing the mouth and face edges of the three-dimensional face model. Then, the facial expression image is expanded according to the closed three-dimensional face model, thereby removing the viewing angle information of the facial expression image at each viewing angle. Then, the expanded facial expression image is synthesized to generate a closed three-dimensional face map. Finally, facial expression images at any viewing angle are generated according to the closed three-dimensional face model and the closed three-dimensional face map. Thus, facial expression images at any viewing angle and with good consistency can be obtained when only facial expression images at a few viewing angles are collected.

[0069] Based on the above-mentioned image enhancement method for simulating camera shooting at any angle, the embodiment of the present invention also provides an image enhancement system for simulating camera shooting at any angle, such as Figure 6 As shown, the system includes the following modules:

[0070] The facial expression image acquisition module 11 is configured to acquire facial expression images at different viewing angles.

[0071] The three-dimensional face model generation module 21 is configured to generate a three-dimensional face model according to a facial expression image.

[0072] The closed 3D face model generation module 31 is configured to generate a closed 3D face model by performing a closing operation on the mouth and face edge of the 3D face model.

[0073] The facial expression image expansion module 41 is configured to expand the facial expression image according to the closed three-dimensional face model, so as to remove the perspective information of the facial expression image at each perspective.

[0074] The face map generation module 51 is configured to generate a closed three-dimensional face map by synthesizing the unfolded facial expression image.

[0075] The facial expression image generation module 61 is configured to generate a facial expression image at any viewing angle according to a closed three-dimensional face model and a closed three-dimensional face map.

[0076] Please refer to Figure 7 The three-dimensional face model generation module 21 also includes the following submodules:

[0077] The labeling module 211 is configured to label the facial expression image to obtain key point annotations.

[0078] A transfer matrix acquisition module 212 is configured to acquire a transfer matrix, wherein the transfer matrix is ​​a relative displacement parameter between a camera and a basic three-dimensional face model when a facial expression image and a basic face image overlap, wherein the basic face image is a two-dimensional image obtained by rendering a basic three-dimensional face model.

[0079] A mapping module 213 is configured to establish a correspondence between a basic three-dimensional face model and its key point annotations, so that S=map(P(f*T*M)+pix_bias)), wherein S represents the key point annotations, map represents the index model vertices, P(x, y, z)=[x / z, y / z, z / z], x, y, z represent spatial coordinates, f represents the camera focal length, T represents the transfer matrix, M represents the basic three-dimensional face model, and pix_bias represents the pixel center error.

[0080] The optimization module 214 is configured to optimize the basic 3D face model according to the corresponding relationship to generate a 3D face model.

[0081] Please refer to Figure 8 The closed 3D face model generation module 31 also includes the following submodules:

[0082] The inner lip vertex module 311 is configured to select N vertices Vl at the inner lips of the three-dimensional face model, and the vertices Vl are arranged in a circle along the inner lips in sequence.

[0083] Inner lip triangulation module 312, the inner lip triangulation module is configured to specify triangulation F according to vertex V1 i =(Vl i ,Vl i+1 ,Vl i+2 ), where i represents the i-th vertex.

[0084] The first face edge vertex module 313 is configured to select M vertices Vb at the face edge of the three-dimensional face model, and the vertices Vb are arranged in a circle along the face edge in sequence.

[0085] The normal vector module 314 is configured to calculate a unit normal vector n of each vertex Vb.

[0086] The second face edge vertex module 315 is configured to select a vertex Vbo at a distance l from the normal vector direction of each vertex Vb, where Vbo=l*n+Vb.

[0087] The face edge triangle module 316 is configured to specify the triangle F according to the vertex Vb and the vertex Vbo. i1 =(Vb i ,Vbo i ,Vb i+1 ), triangle F i2 =(Vbo i ,Vb i+1 ,Vbo i+1 ), i represents the i-th vertex.

[0088] Please refer to Fig. 9 The facial expression image expansion module 41 also includes the following submodules:

[0089] The vertex triangle module 411 is configured to obtain all visible vertices on the closed three-dimensional face model and the visible triangles formed by the visible vertices.

[0090] The conversion module 412 converts the closed 3D face model into a pixel coordinate system.

[0091] The centroid coordinate module 413 obtains the centroid coordinate corresponding to each pixel in the closed three-dimensional face model pixel coordinate system, thereby obtaining the centroid coordinate corresponding to each pixel in the facial expression image.

[0092] Pixel expansion module 414, obtains pixel coordinates PIwarp of the expanded facial expression image, PIwarp = u*Pvis Vd1 +v*Pvis Vd2 +w*Pvis Vd3 , where Pvis Vd1 、Pvis Vd2 、Pvis Vd3 are the pixel coordinates corresponding to the three vertices of each visible triangle, and u, v, and w are the components of the centroid coordinates corresponding to each pixel of the facial expression image;

[0093] The sampling module 415 samples the facial expression image according to the pixel coordinates PIwarp to obtain an expanded facial expression image Iwarp.

[0094] In summary, the image enhancement system for simulating camera shooting at any angle according to the embodiment of the present invention can be implemented in the form of a program and run on a computer device. The memory of the computer device can store various program modules constituting the image enhancement system for simulating camera shooting at any angle, such as: Figure 5The facial expression image acquisition module 11, the 3D facial model generation module 21, the closed 3D facial model generation module 31, the facial expression image expansion module 41, the facial map generation module 51, and the facial expression image generation module 61 are shown. The program composed of each program module enables the processor to execute the steps of the image enhancement method of simulating camera arbitrary viewing angle shooting in each embodiment of the present application described in this specification.

[0095] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] The above embodiments are explanations of the present invention, not limitations of the present invention. Any solution that is simply modified from the present invention belongs to the protection scope of the present invention. The above is only a preferred embodiment of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for image enhancement that simulates the image captured by a camera at any viewing angle. It is characterized in that The following steps are involved: Obtain facial expression images from different perspectives; Generate a three-dimensional face model based on the facial expression image; Generate a closed three-dimensional face model by closing the mouth and face edge of the three-dimensional face model; Expanding the facial expression image according to the closed three-dimensional face model, thereby removing the perspective information of the facial expression image at each perspective; By synthesizing the unfolded facial expression image, a closed three-dimensional face map is generated; Generate facial expression images at any viewing angle based on a closed three-dimensional face model and a closed three-dimensional face map.

2. The image enhancement method according to claim 1, It is characterized in that The method of obtaining a three-dimensional face model according to the facial expression image comprises the following steps: Annotate facial expression images to obtain key point annotations; Obtaining a transfer matrix, wherein the transfer matrix is ​​a relative displacement parameter between a camera and a basic three-dimensional face model when the facial expression image and the basic face image overlap, wherein the basic face image is a two-dimensional image obtained by rendering the basic three-dimensional face model; Establish a correspondence between the basic 3D face model and its key point annotations, so that S = map (P (f * T * M) + pix_bias), where S represents the key point annotation, map represents the index model vertex, P (x, y, z) = [x / z, y / z, z / z], x, y, z represent spatial coordinates, f represents the camera focal length, T represents the transfer matrix, M represents the basic 3D face model, and pix_bias represents the pixel center error; The basic three-dimensional face model is optimized according to the corresponding relationship to generate a three-dimensional face model.

3. The image enhancement method for simulating camera shooting at any viewing angle according to claim 1, It is characterized in that The closing operation of the oral cavity of the three-dimensional face model specifically includes the following steps: Select N vertices Vl at the inner lips of the three-dimensional face model, and the vertices Vl are arranged in a circle along the inner lips in sequence; Specify the triangle F according to the vertex Vl i =(Vl i , Vl i+1 , Vl i+2 ), where i represents the i-th vertex.

4. The image enhancement method according to claim 3, It is characterized in that The N is 10.

5. The image enhancement method according to claim 1, It is characterized in that The closing operation on the face edge of the three-dimensional face model specifically includes the following steps: Select M vertices Vb at the edge of the face of the 3D face model, wherein the vertices Vb are arranged in a circle along the edge of the face in sequence; Calculate the unit normal vector n of each vertex Vb; Select the vertex Vbo at a distance l from the normal vector direction of each vertex Vb, Vbo = l*n + Vb; Specify the triangle F based on vertices Vb and Vbo i1 =(Vb i , Vbo i , Vb i+1 ), triangle F i2 =(Vbo i , Vb i+1 , Vbo i+1 ), i represents the i-th vertex.

6. The image enhancement method according to claim 5, It is characterized in that The M is 15.

7. The image enhancement method according to claim 1, It is characterized in that The unfolding of the facial expression image according to the closed three-dimensional face model comprises the following steps: Obtain all visible vertices on a closed three-dimensional face model and visible triangular faces formed by the visible vertices; Convert the closed 3D face model to a pixel coordinate system; Obtain the centroid coordinates corresponding to each pixel in the pixel coordinate system of the closed three-dimensional face model, thereby obtaining the centroid coordinates corresponding to each pixel in the facial expression image; Get the pixel coordinates PIwarp of the expanded facial expression image, PIwarp = u*Pvis Vd1 +v*Pvis Vd2 +w*Pvis Vd3 , where Pvis Vd1 、Pvis Vd2 、Pvis Vd3 are the pixel coordinates corresponding to the three vertices of each visible triangle, and u, v, and w are the components of the centroid coordinates corresponding to each pixel of the facial expression image; The facial expression image is sampled by the pixel coordinates PIwarp to obtain an expanded facial expression image Iwarp.

8. The image enhancement method according to claim 1, It is characterized in that The face capture helmet uses two cameras placed side by side up and down to collect facial expression images from two perspectives.

9. The image enhancement method according to claim 8, It is characterized in that The two expanded images are synthesized using the formula Icom=w1*Iwarpl+(1-w1)*Iwarp2, where Icom represents the generated closed three-dimensional face map, Iwarp1 and Iwarp2 represent the expanded images of the facial expression images of the upper and lower perspectives respectively, and w1 represents the adjustable coefficient.

10. An image enhancement system that simulates camera shooting at any angle, It is characterized in that include: A facial expression image acquisition module, wherein the facial expression image acquisition module is configured to acquire facial expression images at different viewing angles; A 3D face model generation module, wherein the 3D face model generation module is configured to generate a 3D face model according to a facial expression image; A closed 3D face model generation module, wherein the closed 3D face model generation module is configured to generate a closed 3D face model by performing a closing operation on the mouth and face edge of the 3D face model; A facial expression image expansion module, wherein the facial expression image expansion module is configured to expand the facial expression image according to the closed three-dimensional face model, thereby removing the perspective information of the facial expression image at each perspective; A face map generation module, wherein the face map generation module is configured to generate a closed three-dimensional face map by synthesizing the unfolded facial expression image; A facial expression image generation module is configured to generate a facial expression image at any viewing angle based on a closed three-dimensional face model and a closed three-dimensional face map.

Images