Image processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image processing method, device, electronic device and storage medium, the method comprising: obtaining a facial image to be processed and a target processing item, the target processing item being used to characterize the processing method for the facial image to be processed; obtaining a target reference offset map corresponding to the target processing item, the color value in the target reference offset map being used to characterize the offset vector of each pixel after the reference image is processed by the target processing item; transforming the target reference offset map according to the posture angle of the facial image to be processed to obtain a target offset map corresponding to the facial image to be processed; processing the facial image to be processed according to the target offset map to obtain a target image. The present disclosure improves processing efficiency while retaining the liquefaction deformation processing effect, and the processing method is simple, reduces performance occupancy, and reduces costs.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, and storage medium. Background Art

[0002] With the development of video communication technology and mobile devices, client-side facial image processing, such as face beautification, has become a basic user requirement. Face beautification technology can be applied in camera applications or video applications.

[0003] Among related technologies, the core of facial enhancement technology is to adjust facial features and external contours through image deformation based on facial key points. These image deformation methods are mainly divided into two categories: liquefaction deformation and triangulation deformation. Liquefaction deformation requires controlling a large number of parameters, making the adjustment process more cumbersome, requiring high performance and low processing efficiency. Triangulation deformation relies on the detailed design of the triangular mesh, and the development of each function requires significant development costs. Summary of the Invention

[0004] The present disclosure provides an image processing method, apparatus, electronic device, and storage medium to at least address the problems of low processing efficiency and high cost in related technologies. The technical solutions of the present disclosure are as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0006] Acquire a facial image to be processed and a target processing item, wherein the target processing item is used to represent a processing method for the facial image to be processed;

[0007] Obtaining a target reference offset map corresponding to the target processing item, where the color values ​​in the target reference offset map are used to represent the offset vector of each pixel after the reference image is processed by the target processing item;

[0008] transforming the target reference offset map according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed;

[0009] The face image to be processed is processed according to the target offset map to obtain a target image.

[0010] Optionally, transforming the target reference offset map according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed includes:

[0011] Dividing the face image to be processed into a triangular mesh according to the facial key points of the face image to be processed to obtain a target face mesh;

[0012] Mapping the target reference offset map onto the target face mesh according to the reference face mesh corresponding to the reference image and the target face mesh to obtain an initial offset map corresponding to the face image to be processed, wherein the reference face mesh is obtained by triangulating the reference image based on facial key points of the reference image;

[0013] According to the posture angle of the face image to be processed, the color values ​​in the initial offset map are adjusted to obtain a target offset map corresponding to the face image to be processed.

[0014] Optionally, adjusting the color values ​​in the initial offset map according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed includes:

[0015] determining a transformation matrix based on a roll angle in the posture angle and a ratio of an image size of the facial image to be processed to an image size of the reference image, wherein the transformation matrix is ​​used to represent a position transformation relationship of the facial image to be processed relative to the reference image;

[0016] Determine the difference between the integer 1 and the yaw angle in the attitude angle as an adjustment coefficient of the color value in the initial offset map;

[0017] determining an average area of ​​a triangle mesh of a target mapping point in the target face mesh as a first average area, determining a reference mapping point in the reference face mesh corresponding to the target mapping point, determining an average area of ​​the triangle mesh of the reference mapping point as a second average area, and determining a ratio of the first average area to the second average area as a vertex scale of the target mapping point in the target face mesh;

[0018] The color values ​​in the initial offset map are adjusted according to the transformation matrix, the adjustment coefficient and the vertex scale to obtain a target offset map.

[0019] Optionally, adjusting the color values ​​in the initial offset map according to the transformation matrix, the adjustment coefficient, and the vertex scale to obtain a target offset map includes:

[0020] According to the transformation matrix, the adjustment coefficient, and the vertex scale, the r channel color value and the g channel color value in the initial offset map are adjusted according to the following formula to obtain a target offset map:

[0021]

[0022] in, Is a binary vector, representing the adjusted r channel color value and the adjusted g channel color value, It is a binary vector, representing the color value of the r channel in the initial offset map and the color value of the g channel in the initial offset map, rotMat represents the transformation matrix, degree represents the adjustment coefficient, and scale represents the vertex scale.

[0023] Optionally, dividing the face image to be processed into a triangular mesh according to the facial key points of the face image to be processed to obtain a target face mesh includes:

[0024] Determining target expansion points corresponding to facial key points of the facial image to be processed, where a distance between the target expansion points and the facial key points in the facial image to be processed is greater than a distance between a reference expansion point in the reference image and the facial key points in the reference image, the reference expansion points being expansion points corresponding to facial key points in the reference image, and the reference face mesh being obtained by triangulating the reference image based on the facial key points in the reference image and the reference expansion points;

[0025] According to the facial key points of the facial image to be processed and the target expansion points, the facial image to be processed is divided into a triangular mesh to obtain a target facial mesh.

[0026] Optionally, processing the face image to be processed according to the target offset map to obtain a target image includes:

[0027] Determining, according to the target offset map, an offset vector corresponding to performing the target processing item on the face image to be processed;

[0028] Performing offset processing of the offset vector on pixel points in the face image to be processed to obtain the target image.

[0029] Optionally, determining, based on the target offset map, an offset vector corresponding to performing the target processing item on the facial image to be processed includes:

[0030] Determining a horizontal offset vector in the offset vector according to an r channel color value in the target offset map;

[0031] A longitudinal offset vector in the offset vector is determined according to a g channel color value in the target offset map.

[0032] Optionally, before obtaining the target reference offset map corresponding to the target processing item, the method further includes:

[0033] Performing the liquefaction deformation processing of the target processing item on the reference image to obtain an offset vector of a pixel point in the reference image;

[0034] The target reference offset map is generated according to the offset vectors of the pixels in the reference image.

[0035] Optionally, generating the target reference offset map according to the offset vectors of the pixels in the reference image includes:

[0036] According to the offset vector of the pixel in the reference image, the color value of the pixel in the target reference offset image is determined according to the following formula:

[0037]

[0038] Among them, rgbaColor is the color value of the pixel in the target reference offset image, and xyOffset is the offset vector of the pixel in the reference image. is a binary vector representing the preset color adjustment value of the r channel and the preset color adjustment value of the g channel of the pixel point in the target reference offset map, m is the preset color value of the b channel of the pixel point in the target reference offset map, and n is the preset color value of the a channel of the pixel point in the target reference offset map.

[0039] Optionally, also include:

[0040] splicing the target reference offset map and reference offset maps corresponding to other processing items other than the target processing item into a texture image;

[0041] The acquiring of the target reference offset map corresponding to the target processing item includes:

[0042] A target reference offset map corresponding to the target processing item is obtained from the texture image.

[0043] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0044] An image acquisition module is configured to acquire a facial image to be processed and a target processing item, wherein the target processing item is used to represent a processing method for the facial image to be processed;

[0045] a reference offset map acquisition module configured to acquire a target reference offset map corresponding to the target processing item, wherein the color values ​​in the target reference offset map are used to represent the offset vector of each pixel after the reference image is processed by the target processing item;

[0046] an offset map transformation module configured to transform the target reference offset map according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed;

[0047] The image processing module is configured to process the face image to be processed according to the target offset map to obtain a target image.

[0048] Optionally, the offset map transformation module includes:

[0049] a mesh division unit configured to perform triangular mesh division on the face image to be processed according to the facial key points of the face image to be processed to obtain a target face mesh;

[0050] an offset map mapping unit configured to map the target reference offset map onto the target face mesh based on a reference face mesh corresponding to the reference image and the target face mesh, thereby obtaining an initial offset map corresponding to the face image to be processed, wherein the reference face mesh is obtained by triangulating the reference image based on facial key points of the reference image;

[0051] The offset map adjustment unit is configured to adjust the color values ​​in the initial offset map according to the posture angle of the face image to be processed, so as to obtain a target offset map corresponding to the face image to be processed.

[0052] Optionally, the offset map adjustment unit includes:

[0053] a transformation matrix determination subunit, configured to determine a transformation matrix based on a roll angle in the posture angle and a ratio of an image size of the face image to be processed to an image size of the reference image, wherein the transformation matrix is ​​used to represent a position transformation relationship of the face image to be processed relative to the reference image;

[0054] an adjustment coefficient determination subunit, configured to determine a difference between an integer 1 and the yaw angle in the attitude angle as an adjustment coefficient for the color value in the initial offset map;

[0055] a vertex scale determination subunit configured to determine an average area of ​​a triangle mesh of a target mapping point in the target face mesh as a first average area, determine a reference mapping point in the reference face mesh corresponding to the target mapping point, determine an average area of ​​the triangle mesh of the reference mapping point as a second average area, and determine a ratio of the first average area to the second average area as a vertex scale of the target mapping point in the target face mesh;

[0056] The offset map adjustment subunit is configured to adjust the color values ​​in the initial offset map according to the transformation matrix, the adjustment coefficient and the vertex scale to obtain a target offset map.

[0057] Optionally, the offset map adjustment subunit is configured to perform:

[0058] According to the transformation matrix, the adjustment coefficient, and the vertex scale, the r channel color value and the g channel color value in the initial offset map are adjusted according to the following formula to obtain a target offset map:

[0059]

[0060] in, Is a binary vector, representing the adjusted r channel color value and the adjusted g channel color value, It is a binary vector, representing the color value of the r channel in the initial offset map and the color value of the g channel in the initial offset map, rotMat represents the transformation matrix, degree represents the adjustment coefficient, and scale represents the vertex scale.

[0061] Optionally, the grid division unit includes:

[0062] a target expansion point determination subunit configured to determine a target expansion point corresponding to a facial key point of the facial image to be processed, wherein a distance between the target expansion point and a facial key point in the facial image to be processed is greater than a distance between a reference expansion point in the reference image and a facial key point in the reference image, the reference expansion point being an expansion point corresponding to a facial key point in the reference image, and the reference face mesh being obtained by triangulating the reference image based on the facial key points in the reference image and the reference expansion point;

[0063] The mesh division subunit is configured to perform triangular mesh division on the face image to be processed according to the face key points of the face image to be processed and the target expansion points to obtain a target face mesh.

[0064] Optionally, the image processing module includes:

[0065] an offset vector determining unit configured to determine, based on the target offset map, an offset vector corresponding to performing the target processing item on the face image to be processed;

[0066] The offset processing unit is configured to perform offset processing of the offset vector on the pixel points in the face image to be processed to obtain the target image.

[0067] Optionally, the offset vector determining unit is configured to execute:

[0068] Determining a horizontal offset vector in the offset vector according to an r channel color value in the target offset map;

[0069] A longitudinal offset vector in the offset vector is determined according to a g channel color value in the target offset map.

[0070] Optionally, the device further includes:

[0071] a reference image liquefaction deformation module, configured to perform liquefaction deformation processing of the target processing item on the reference image to obtain an offset vector of a pixel point in the reference image;

[0072] The reference offset map generation module is configured to generate the target reference offset map according to the offset vector of the pixel point in the reference image.

[0073] Optionally, the reference offset map generation module is configured to execute:

[0074] According to the offset vector of the pixel in the reference image, the color value of the pixel in the target reference offset image is determined according to the following formula:

[0075]

[0076] Among them, rgbaColor is the color value of the pixel in the target reference offset image, and xyOffset is the offset vector of the pixel in the reference image. is a binary vector representing the preset color adjustment value of the r channel and the preset color adjustment value of the g channel of the pixel point in the target reference offset map, m is the preset color value of the b channel of the pixel point in the target reference offset map, and n is the preset color value of the a channel of the pixel point in the target reference offset map.

[0077] Optionally, the device further includes:

[0078] an offset map stitching module configured to stitch the target reference offset map and reference offset maps corresponding to other processing items other than the target processing item into a texture image;

[0079] The reference offset map acquisition module is configured to execute:

[0080] A target reference offset map corresponding to the target processing item is obtained from the texture image.

[0081] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0082] processor;

[0083] a memory for storing instructions executable by the processor;

[0084] The processor is configured to execute the instructions to implement the image processing method as described in the first aspect.

[0085] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method as described in the first aspect.

[0086] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method according to the first aspect is implemented.

[0087] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0088] The disclosed embodiment obtains a target reference offset map corresponding to the target processing item after obtaining the face image to be processed and the target processing item, transforms the target reference offset map according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed, processes the face image to be processed according to the target offset map to obtain the target image. Since the target reference offset map is a pre-generated offset vector for representing each pixel point after the reference image is processed by the target processing item, when processing the image, it is only necessary to map the target reference offset map to the face image to be processed, and use the obtained target offset map to process the face image to be processed. There is no need to control a large number of parameters. On the basis of retaining the liquefaction deformation processing effect, the amount of calculation is reduced, the performance occupancy is reduced, the processing efficiency is improved, and the processing method is simple, which reduces the cost.

[0089] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0091] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0092] Figure 2 is a flow chart of determining a target offset map corresponding to a face image to be processed in an exemplary embodiment;

[0093] Figure 3 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0094] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0095] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0096] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0097] The liquefaction deformation method uses pixels as processing units, deforming only points within the liquefied area (a circle centered at a control point). Positions closer to the center of the circle experience greater deformation, while those closer to the edge experience less deformation, ultimately reducing the deformation to zero at the edges. This principle ensures the relative positions of the original vertices remain fixed, while the gradual shift in movement ensures smooth deformation. However, the radiating gradient from the center point results in limited control flexibility. In practice, various aesthetic effects are achieved by superimposing basic deformation units, including push, rotate, and scale. Each pixel is traversed, and the deformation vectors of each point under the influence of each deformation unit are calculated and superimposed to obtain the deformation vector for the final rendering. The liquefaction deformation algorithm can achieve a variety of complex deformation effects by adjusting parameters such as the liquefaction radius and liquefaction strength at each control point. However, due to the large number of parameters required, the adjustment process is cumbersome and difficult to control in the overlapping areas of liquefied regions with multiple control points.

[0098] The triangular deformation method divides the face area to be deformed into multiple triangles. Custom deformation rules are then applied to each processing item to calculate the coordinates of the deformed vertices. An affine transformation is then applied to each triangle, mapping the original texture coordinates to the transformed triangle area and rendering the resulting deformation. Specifically, for each triangle mesh, the affine transformation matrix is ​​calculated based on the three vertex coordinates before and after deformation. This transformation is then applied to all pixels within the mesh to obtain the final deformation result. This method is simple in principle, fast, and easily allows for fine-grained local control. Because adjacent triangles share a common edge, the texture on that edge is fixed, ensuring smoothness while maintaining the relative positions of the triangle vertices (up, down, left, and right). However, this method relies on meticulous design of the triangle mesh, resulting in significant development costs for each feature, and the designer's control over the results is limited. Furthermore, poor triangle mesh design and deviations in the relative positions of midpoints for different face shapes can lead to broken lines, requiring extensive control logic during development, further reducing efficiency. To ensure that the relative positions of vertices remain fixed and to avoid problems like creases, this method relies heavily on the design of the motion vectors.

[0099] Table 1 shows the advantages and disadvantages of liquefaction deformation and triangulation deformation:

[0100] Table 1 Advantages and disadvantages of liquefaction deformation and triangulation deformation

[0101]

[0102]

[0103] After analyzing the problems and shortcomings of related technical solutions, the image processing method of the disclosed embodiment can simultaneously meet the following characteristics: 1) controllable deformation range; 2) smooth and natural deformation; 3) low development cost; 4) controllable performance. At the same time, due to the movement and deformation of the face, the background is significantly pulled, which greatly affects the user experience. This disclosed embodiment also solves this problem. The specific solution of the disclosed embodiment is as follows:

[0104] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1 As shown, the image processing method can be used in electronic devices such as mobile phones and tablet computers, and includes the following steps.

[0105] In step S11, a facial image to be processed and a target processing item are obtained, where the target processing item is used to represent a processing method for the facial image to be processed.

[0106] The target processing item may be, for example, a processing method such as a thin face or a small face.

[0107] The face image to be processed can be obtained by taking a photo, or the photo album can be displayed and a face image selected by the user can be obtained as the face image to be processed. At the same time, a processing item selected by the user can be obtained from multiple processing items displayed as the target processing item.

[0108] In step S12, a target reference offset map corresponding to the target processing item is obtained, where the color values ​​in the target reference offset map are used to represent the offset vector of each pixel after the reference image is processed by the target processing item.

[0109] The reference image is an image including a model's face. In the reference image, the model's face is generally a frontal face.

[0110] The reference image is processed in advance for various processing items to obtain the offset vector of each pixel point of the reference image under the various processing items, thereby generating reference offset maps corresponding to the various processing items based on the offset vector of each pixel point under the various processing items, and saving the reference offset maps corresponding to the various processing items.

[0111] After obtaining the face image to be processed and the target processing item, the target reference offset map corresponding to the target processing item is obtained from the storage location of the reference offset map.

[0112] In step S13, the target reference offset map is transformed according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed.

[0113] The target reference offset map is the offset vector corresponding to the target processing item performed on the reference image including the model's frontal face. When processing the facial image to be processed, the target reference offset map needs to be mapped onto the facial image to be processed. Based on the pose angle of the facial image to be processed, the target reference offset map can be transformed to map to the target offset map required for performing the target processing item on the facial image to be processed. The color values ​​in the target offset map represent the offset vector for each pixel required for performing the target processing item on the facial image to be processed.

[0114] Figure 2 FIG. 1 is a flow chart of determining a target offset map corresponding to a face image to be processed in an exemplary embodiment. Figure 2 As shown, according to the posture angle of the face image to be processed, the target reference offset map is transformed to obtain the target offset map corresponding to the face image to be processed, including steps S131 to S133:

[0115] In step S131, the face image to be processed is divided into a triangular mesh according to the facial key points of the face image to be processed to obtain a target face mesh.

[0116] The key point detection model is used to process the face image to be processed to obtain the face key points and posture angles of the face image to be processed, wherein the face key points include the key points of the face contour.

[0117] Based on the facial key points of the face image to be processed, the face image to be processed is divided into a triangular mesh, such that the triangular mesh vertices include the facial key points, such as the key points of the facial contour, to obtain a target face mesh corresponding to the face image to be processed. Each point in the target face mesh corresponds to a pixel point in the face image to be processed, and the size of the target face mesh is the same as that of the face image to be processed.

[0118] In step S132, the target reference offset map is mapped onto the target face mesh according to the reference face mesh corresponding to the reference image and the target face mesh, so as to obtain an initial offset map corresponding to the face image to be processed, wherein the reference face mesh is obtained by triangulating the reference image based on the facial key points of the reference image.

[0119] Based on the facial key points of the reference facial image, the reference facial image is pre-divided into a triangular mesh, such that the triangular mesh vertices include facial key points, such as key points of the facial contour, to obtain a reference facial mesh corresponding to the reference facial image. The reference facial image is meshed in the same manner as the processed facial image, so that the triangular meshes in the reference facial mesh correspond to the triangular meshes in the target facial mesh. When performing triangular meshing, the finer the meshing, the more accurate the deformation corresponding to each area; conversely, the deformation may be misaligned. For example, if the entire reference facial image is divided into two triangles for deformation, the deformation may not be applied to the face. The reference facial mesh covers the entire non-gray area in the target reference offset map, that is, the mesh covers the areas with deformation in the target reference offset map. Each point in the reference facial mesh corresponds to a pixel in the reference facial image and a pixel in the target reference offset map. The size of the reference facial mesh is the same as that of the reference facial image and the target reference offset map.

[0120] According to the reference face mesh and the target face mesh, the target reference offset map is mapped. The reference face mesh can be overlaid on the target reference offset map. Based on the triangular mesh in the reference face mesh and the triangular mesh in the target face mesh, the target reference offset map is transformed so that the color values ​​in the triangular mesh of the reference face mesh in the target reference offset map are mapped to the corresponding triangular mesh in the target face mesh, thereby obtaining the initial offset map corresponding to the face image to be processed.

[0121] In an exemplary embodiment, the face image to be processed is triangulated according to the facial key points of the face image to be processed to obtain a target face mesh, including: determining a target expansion point corresponding to the facial key point of the face image to be processed, the distance between the target expansion point and the facial key point in the face image to be processed being greater than the distance between the reference expansion point in the reference image and the facial key point in the reference image, the reference expansion point is the expansion point corresponding to the facial key point in the reference image, and the reference face mesh is obtained by triangulating the reference image based on the facial key points in the reference image and the reference expansion point; according to the facial key points of the face image to be processed and the target expansion point, the face image to be processed is triangulated to obtain a target face mesh.

[0122] Among them, when the face area in the face image to be processed is the same size as the face area in the reference image, the distance between the target expansion point and the face key point in the face image to be processed is greater than the distance between the reference expansion point and the face key point in the reference image. If the face area in the face image to be processed is different in size from the face area in the reference image, they can be transformed into the same size before processing.

[0123] Based on the distance between the target expansion point and the facial key point in the face image to be processed, the target expansion point corresponding to the facial key point in the face image to be processed is determined, and then based on the facial key points and the target expansion point of the face image to be processed, the face image to be processed is triangulated, that is, the facial key points and the target expansion point in the face image to be processed are respectively formed into triangular meshes, and multiple triangular meshes are obtained, that is, the target face mesh corresponding to the face image to be processed is obtained.

[0124] After determining the target face mesh using the above method, the coverage area of ​​the target face mesh is larger than the coverage area of ​​the reference face mesh. The coverage area of ​​the target face mesh includes the face area in the face image to be processed, as well as the portion of the area outside the face area in the face image to be processed; the coverage area of ​​the reference face mesh includes the face area in the reference image, as well as the portion of the area outside the face area in the reference image. The area outside the face area within the coverage area of ​​the target face mesh is larger than the area outside the face area within the coverage area of ​​the reference face mesh. This can address the dynamic effects of background distortion and distribute deformation outside the face area to a larger area, modifying deformation outside the face area while ensuring that deformation within the face remains essentially consistent. After mesh mapping, the color of the target offset map fades and diffuses outward, reducing deformation outside the face area. This separation of the interior and exterior regions of the face is achieved through mesh division. The corresponding positions of the reference face mesh and the target face mesh in the interior region remain consistent, while the exterior region achieves deformation distribution by mapping the mesh containing the offset vector in the target reference offset map to a larger area on the target face mesh map.

[0125] It should be noted that in addition to using the coverage area of ​​the target face grid as described above to be larger than the coverage area of ​​the reference face grid to solve the problem of background distortion, other purposes can also be achieved through other mapping relationships, such as protecting certain areas by controlling the position of the deformed grid in the target offset map.

[0126] In step S133, the color values ​​in the initial offset map are adjusted according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed.

[0127] The color values ​​in the initial offset map are the offset vectors corresponding to the target processing items when the face in the face image to be processed is a frontal face. However, the face in the face image to be processed is not necessarily a frontal face. In this case, it is necessary to adjust the color values ​​in the initial offset map based on the posture angle of the face image to be processed to obtain the target offset map corresponding to the face image to be processed, so that the color values ​​in the target offset map correspond to the posture angle in the face image to be processed, which is convenient for processing the face image to be processed.

[0128] By dividing the face image to be processed into a triangular mesh, and mapping the target reference offset map to the target face mesh based on the reference face mesh and the target face mesh, an initial offset map is obtained. Based on the posture angle of the face image to be processed, the color value in the initial offset map is adjusted to obtain a target offset map. In this way, the offset map is mapped by triangular mesh division, and the deformation in the target reference offset map is mapped to each pixel point of the face image to be processed through texture mapping. The texture coordinate mapping relationship and mapping area can be flexibly controlled, and the deformation influence range can be accurately controlled, thereby improving the effect of face image processing.

[0129] In an exemplary embodiment, adjusting the color values ​​in the initial offset map according to the posture angle of the facial image to be processed to obtain a target offset map corresponding to the facial image to be processed includes: determining a transformation matrix according to the roll angle in the posture angle and the ratio of the image size of the facial image to be processed to the image size of the reference image, the transformation matrix being used to characterize the position transformation relationship of the facial image to be processed relative to the reference image; determining the difference between the integer 1 and the yaw angle in the posture angle as an adjustment coefficient for the color values ​​in the initial offset map; determining an average area of ​​a triangular mesh of a target mapping point in the target facial mesh as a first average area, determining a reference mapping point in the reference facial mesh corresponding to the target mapping point, determining an average area of ​​the triangular mesh of the reference mapping point as a second average area, and determining a ratio of the first average area to the second average area as the vertex scale of the target mapping point in the target facial mesh; and adjusting the color values ​​in the initial offset map according to the transformation matrix, the adjustment coefficient, and the vertex scale to obtain a target offset map.

[0130] The transformation matrix represents the effect of the roll angle and image size on the deformation, and the coefficients are adjusted based on the proportion of the face in the face image to be processed. For example, if the face accounts for a large proportion, the deformation range will be larger, and the coefficients in the transformation matrix are greater than 1. If the face accounts for a small proportion, the deformation range will be smaller, and the coefficients in the transformation matrix are less than 1.

[0131] The deformation range is determined based on the roll angle in the pose angle and the ratio of the image size of the target face image to the image size of the reference image. This results in a positional transformation of the target face image relative to the reference image, i.e., a transformation matrix. The yaw angle in the pose angle represents the angle by which the face is turned left or right, indicating whether it is in profile. The colors of the initial offset map are adjusted based on the yaw angle in the pose angle, so that the colors on the inward side of the face are lighter. The adjustment coefficient for the color values ​​in the initial offset map is determined as: degree = 1 - yaw, where degree is the adjustment coefficient and yaw is the yaw angle. The average area of ​​the triangle meshes corresponding to the target mapping point in the target face mesh is determined to obtain a first average area value. A reference mapping point corresponding to the target mapping point in the reference face mesh is determined to obtain a second average area value. The ratio of the first average area value to the second average area value is determined as the vertex scale of the target mapping point in the target face mesh. The color values ​​in the initial offset map are adjusted using the transformation matrix, adjustment coefficient, and vertex scale to obtain a target offset map. The color values ​​in the initial offset map are adjusted based on the posture angle of the face image to be processed, and a more accurate target offset map is obtained, thereby improving the image processing effect.

[0132] In an exemplary embodiment, adjusting the color values ​​in the initial offset map according to the transformation matrix, the adjustment coefficient, and the vertex scale to obtain a target offset map includes:

[0133] According to the transformation matrix, the adjustment coefficient, and the vertex scale, the r channel color value and the g channel color value in the initial offset map are adjusted according to the following formula to obtain a target offset map:

[0134]

[0135] in, Is a binary vector, representing the adjusted r channel color value and the adjusted g channel color value, It is a binary vector, representing the color value of the r channel in the initial offset map and the color value of the g channel in the initial offset map, rotMat represents the transformation matrix, degree represents the adjustment coefficient, and scale represents the vertex scale.

[0136] Adjusting the r and g channel color values ​​in the initial offset map based on the above formula yields the r and g channel color values ​​in the target offset map. By adjusting the color values ​​in the initial offset map according to the above formula, a more accurate target offset map can be obtained.

[0137] In step S14, the face image to be processed is processed according to the target offset map to obtain a target image.

[0138] An offset vector for each pixel point is obtained from the target offset map, and based on the offset vector of each pixel point, an offset processing of the offset vector is performed on the corresponding pixel point in the face image to be processed, so as to obtain a target image corresponding to the face image to be processed and the target processing item, that is, a target image after the face image to be processed is processed by the target processing item.

[0139] In an exemplary embodiment, the facial image to be processed is processed according to the target offset map to obtain a target image, including: determining, according to the target offset map, an offset vector corresponding to the target processing item performed on the facial image to be processed; and performing offset processing of the offset vector on pixel points in the facial image to be processed to obtain the target image.

[0140] The color values ​​in the target offset map represent the offset vector corresponding to the target processing item when the facial image to be processed is processed. At this time, the color values ​​in the target offset map are analyzed to determine the offset vector corresponding to the target processing item when the facial image to be processed is processed. Each pixel in the facial image to be processed is offset by the offset vector corresponding to the pixel to obtain a target image corresponding to the facial image to be processed and the target processing item, that is, a target image after the facial image to be processed is processed by the target processing item. By determining the offset vector corresponding to the target processing item when the facial image to be processed is processed according to the target offset map, and performing offset processing on each pixel in the facial image to be processed based on the offset vector, the processing effect of the obtained target image can be improved.

[0141] In an exemplary embodiment, based on the target offset map, the offset vector corresponding to the target processing item of the facial image to be processed is determined, including: determining the horizontal offset vector in the offset vector based on the r channel color value in the target offset map; and determining the vertical offset vector in the offset vector based on the g channel color value in the target offset map.

[0142] Using the inverse operation of the target reference offset map used to calculate the target reference facial image, the offset vector corresponding to the target processing item for the facial image to be processed is determined based on the color values ​​in the target offset map. Using the inverse operation of the target reference offset map used to calculate the target reference facial image, the r channel color values ​​in the target offset map are converted to the horizontal offset vector in the offset vector, and the g channel color values ​​in the target offset map are converted to the vertical offset vector in the offset vector. That is, the horizontal offset vector and the vertical offset vector are determined according to the following formula:

[0143] xyOffset = rgbaColor.xy - 0.5

[0144] Among them, xyOffset is the offset vector of the current pixel point in the plane coordinate system. It is a binary vector, including the horizontal offset vector and the vertical offset vector, and the range is the normalized range [-0.5, 0.5]. rgbaColor.xy is a binary vector that represents the r channel color value and the g channel color value in the target offset map.

[0145] The r channel color value in the target offset map represents the horizontal offset vector, and the g channel color value represents the vertical offset vector. This can obtain a more accurate offset vector and improve the image processing effect.

[0146] The image processing method provided by this exemplary embodiment obtains a target reference offset map corresponding to the target processing item after obtaining a face image to be processed and a target processing item, transforms the target reference offset map according to the posture angle of the face image to be processed to obtain a target offset map corresponding to the face image to be processed, processes the face image to be processed according to the target offset map to obtain a target image, and since the target reference offset map is a pre-generated offset vector for representing each pixel point after the reference image is processed by the target processing item, when processing the image, it is only necessary to map the target reference offset map to the face image to be processed, and use the obtained target offset map to process the face image to be processed, without the need to control a large number of parameters. On the basis of retaining the liquefaction deformation processing effect, the amount of calculation is reduced, the performance occupancy is reduced, the processing efficiency is improved, and the processing method is simple, which reduces the cost.

[0147] On the basis of the above technical solution, before obtaining the target reference offset map corresponding to the target processing item, it also includes: performing liquefaction deformation processing of the target processing item on the reference image to obtain the offset vector of the pixel point in the reference image; generating the target reference offset map according to the offset vector of the pixel point in the reference image.

[0148] Use the liquefaction formula corresponding to the target processing item to perform liquefaction deformation processing on each pixel in the reference image to obtain the offset vector corresponding to each pixel in the reference image. Use color values ​​to represent the offset vector corresponding to each pixel to obtain the target reference offset map.

[0149] When performing liquefaction deformation on a reference image as a target processing item, pixels are used as processing units, and only points within the liquefied area (a circle centered at the control point, i.e., the current pixel) are deformed. Position coordinates closer to the center of the circle are deformed more, while those closer to the edge of the circle are deformed less, reaching zero at the edges. This principle ensures the relative positions of the original vertices remain fixed, while the gradual change in the movement ensures smooth deformation. However, the radiating gradient from the center point results in inflexible control of the effect. In practical applications, various aesthetic effects can be achieved by superimposing basic deformation units, including push, rotate, zoom, etc. The algorithm traverses each pixel, calculates the deformation vector of each point under the action of each deformation unit, and superimposes them to obtain the deformation vector of each point for final rendering. The liquefaction deformation algorithm can achieve various complex deformation effects by adjusting parameters such as the liquefaction radius and liquefaction strength at each control point.

[0150] The disclosed embodiment predetermines the target reference offset map of the target processing item corresponding to the reference image, does not occupy the performance of real-time processing, reduces the amount of calculation in the image processing process, and uses liquefaction deformation processing to ensure smooth and natural deformation.

[0151] On the basis of the above technical solution, generating the target reference offset map according to the offset vectors of the pixels in the reference image includes:

[0152] According to the offset vector of the pixel in the reference image, the color value of the pixel in the target reference offset image is determined according to the following formula:

[0153]

[0154] Among them, rgbaColor is the color value of the pixel in the target reference offset image, xyOffset is the offset vector of the pixel in the reference image, which is a binary vector. is a binary vector representing the preset color adjustment values ​​for the r and g channels of the target reference offset image. m is the preset color value for the b channel of the target reference offset image, and n is the preset color value for the a channel of the target reference offset image. For example, the preset color value for the b channel can be 0.5, or other values. Here, 0.5 ensures that the non-deformed area is gray. The preset color value for the a channel can be 1.0.

[0155] For any pixel in the reference image, rgbaColor is the color value of the pixel in the target reference offset map, and xyOffset is the offset vector of the pixel in the plane coordinate system. The range of xyOffset is normalized to [-0.5, 0.5].

[0156] The color value of the pixel in the offset map is determined by the above formula to ensure that the deformed area has color and the non-deformed area is gray (ie, gray represents 0 offset), which facilitates deformation processing.

[0157] Based on the above technical solution, the method further includes: splicing the target reference offset map and reference offset maps corresponding to other processing items other than the target processing item into a texture image;

[0158] The acquiring the target reference offset map corresponding to the target processing item includes: acquiring the target reference offset map corresponding to the target processing item from the texture image.

[0159] In addition to pre-liquifying and deforming the reference image for the target processing item, the reference image also needs to be pre-liquified and deformed for other processing items, that is, the reference image is pre-liquified and deformed for various processing items, and reference offset maps for the reference image under various processing items are generated. The target reference offset map and the reference offset maps corresponding to other processing items other than the target processing item are spliced ​​into a texture image. This can avoid the limitation of no more than 8 textures on low-end models. Here, the reference offset maps are simply spliced ​​onto a texture image without superposition. This is because the strength of each target processing item selected by the user is different. Therefore, different target processing items can be superimposed when processing the face image to be processed, and different parameters can be adjusted for each target processing item.

[0160] Based on the splicing position of the target reference offset map corresponding to the target processing item in the texture image, the texture image at that splicing position is extracted from the texture image to obtain the target reference offset map corresponding to the target processing item. By including reference offset maps corresponding to multiple processing items in a texture image and extracting the target reference offset map based on the target processing item, the limit of eight textures on low-end models can be avoided.

[0161] A key step in liquefaction deformation is calculating the deformation offset vector. This process requires traversing each pixel point and each deformation configuration unit in the configuration list and calculating the deformation offset vector for each pixel point using the liquefaction formula. In the disclosed embodiment, the process of calculating the deformation offset vector is placed in the production of the reference offset map. The reference offset map is directly built into the project as a material and mapped to each pixel of the facial image to be processed via texture mapping. This eliminates the need to use the liquefaction formula to calculate the offset vector during the image processing process, greatly reducing the amount of calculation. At the same time, the texture coordinate mapping relationship and mapping area can be flexibly controlled in the texture map to control the scope of influence.

[0162] This disclosed embodiment combines the advantages of liquefaction and triangulation deformation techniques, offering the flexibility of liquefaction with the controllability and efficiency of triangulation. To address the issue of background distortion caused by the movement of faces in real-time image processing, this disclosed embodiment controls the deformation area, allowing the background to share some of the distortion, significantly reducing the background distortion in real-time image processing.

[0163] Figure 3 FIG. 1 is a block diagram of an image processing apparatus according to an exemplary embodiment. Figure 3 The device includes an image acquisition module 31, a reference offset map acquisition module 32, an offset map transformation module 33 and an image processing module 34.

[0164] The image acquisition module 31 is configured to acquire a face image to be processed and a target processing item, wherein the target processing item is used to represent a processing method for the face image to be processed;

[0165] The reference offset map acquisition module 32 is configured to acquire a target reference offset map corresponding to the target processing item, wherein the color value in the target reference offset map is used to represent the offset vector of each pixel after the reference image is processed by the target processing item;

[0166] The offset map transformation module 33 is configured to transform the target reference offset map according to the posture angle of the face image to be processed, so as to obtain a target offset map corresponding to the face image to be processed;

[0167] The image processing module 34 is configured to process the face image to be processed according to the target offset map to obtain a target image.

[0168] Optionally, the offset map transformation module includes:

[0169] a mesh division unit configured to perform triangular mesh division on the face image to be processed according to the facial key points of the face image to be processed to obtain a target face mesh;

[0170] an offset map mapping unit configured to map the target reference offset map onto the target face mesh based on a reference face mesh corresponding to the reference image and the target face mesh, thereby obtaining an initial offset map corresponding to the face image to be processed, wherein the reference face mesh is obtained by triangulating the reference image based on facial key points of the reference image;

[0171] The offset map adjustment unit is configured to adjust the color values ​​in the initial offset map according to the posture angle of the face image to be processed, so as to obtain a target offset map corresponding to the face image to be processed.

[0172] Optionally, the offset map adjustment unit includes:

[0173] a transformation matrix determination subunit, configured to determine a transformation matrix based on a roll angle in the posture angle and a ratio of an image size of the face image to be processed to an image size of the reference image, wherein the transformation matrix is ​​used to represent a position transformation relationship of the face image to be processed relative to the reference image;

[0174] an adjustment coefficient determination subunit, configured to determine an integer 1 and a yaw angle in the attitude angle as an adjustment coefficient of a color value in the initial offset map;

[0175] a vertex scale determination subunit configured to determine an average area of ​​a triangle mesh of a target mapping point in the target face mesh as a first average area, determine a reference mapping point in the reference face mesh corresponding to the target mapping point, determine an average area of ​​the triangle mesh of the reference mapping point as a second average area, and determine a ratio of the first average area to the second average area as a vertex scale of the target mapping point in the target face mesh;

[0176] The offset map adjustment subunit is configured to adjust the color values ​​in the initial offset map according to the transformation matrix, the adjustment coefficient and the vertex scale to obtain a target offset map.

[0177] Optionally, the offset map adjustment subunit is configured to perform:

[0178] According to the transformation matrix, the adjustment coefficient, and the vertex scale, the r channel color value and the g channel color value in the initial offset map are adjusted according to the following formula to obtain a target offset map:

[0179]

[0180] in, Is a binary vector, representing the adjusted r channel color value and the adjusted g channel color value, It is a binary vector, representing the color value of the r channel in the initial offset map and the color value of the g channel in the initial offset map, rotMat represents the transformation matrix, degree represents the adjustment coefficient, and scale represents the vertex scale.

[0181] Optionally, the grid division unit includes:

[0182] a target expansion point determination subunit configured to determine a target expansion point corresponding to a facial key point of the facial image to be processed, wherein a distance between the target expansion point and a facial key point in the facial image to be processed is greater than a distance between a reference expansion point in the reference image and a facial key point in the reference image, the reference expansion point being an expansion point corresponding to a facial key point in the reference image, and the reference face mesh being obtained by triangulating the reference image based on the facial key points in the reference image and the reference expansion point;

[0183] The mesh division subunit is configured to perform triangular mesh division on the face image to be processed according to the face key points of the face image to be processed and the target expansion points to obtain a target face mesh.

[0184] Optionally, the image processing module includes:

[0185] an offset vector determining unit configured to determine, based on the target offset map, an offset vector corresponding to performing the target processing item on the face image to be processed;

[0186] The offset processing unit is configured to perform offset processing of the offset vector on the pixel points in the face image to be processed to obtain the target image.

[0187] Optionally, the offset vector determining unit is configured to execute:

[0188] Determining a horizontal offset vector in the offset vector according to an r channel color value in the target offset map;

[0189] A longitudinal offset vector in the offset vector is determined according to a g channel color value in the target offset map.

[0190] Optionally, the device further includes:

[0191] a reference image liquefaction deformation module, configured to perform liquefaction deformation processing of the target processing item on the reference image to obtain an offset vector of a pixel point in the reference image;

[0192] The reference offset map generation module is configured to generate the target reference offset map according to the offset vector of the pixel point in the reference image.

[0193] Optionally, the reference offset map generation module is configured to execute:

[0194] According to the offset vector of the pixel in the reference image, the color value of the pixel in the target reference offset image is determined according to the following formula:

[0195]

[0196] Among them, rgbaColor is the color value of the pixel in the target reference offset image, and xyOffset is the offset vector of the pixel in the reference image. is a binary vector representing the preset color adjustment value of the r channel and the preset color adjustment value of the g channel of the pixel point in the target reference offset map, m is the preset color value of the b channel of the pixel point in the target reference offset map, and n is the preset color value of the a channel of the pixel point in the target reference offset map.

[0197] Optionally, the device further includes:

[0198] an offset map stitching module configured to stitch the target reference offset map and reference offset maps corresponding to other processing items other than the target processing item into a texture image;

[0199] The reference offset map acquisition module is configured to execute:

[0200] A target reference offset map corresponding to the target processing item is obtained from the texture image.

[0201] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0202] Figure 4 4 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0203] Reference Figure 4 , electronic device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0204] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 402 may include one or more modules to facilitate interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate interaction between the multimedia component 408 and the processing component 402.

[0205] The memory 404 is configured to store various types of data to support operations on the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0206] The power supply assembly 406 provides power to the various components of the electronic device 400. The power supply assembly 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 400.

[0207] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0208] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0209] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0210] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the electronic device 400. For example, the sensor assembly 414 can detect the open / closed state of the electronic device 400, the relative positioning of components, such as the display and keypad of the electronic device 400. The sensor assembly 414 can also detect changes in the position of the electronic device 400 or a component of the electronic device 400, the presence or absence of user contact with the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and temperature changes of the electronic device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0211] The communication component 416 is configured to facilitate wired or wireless communication between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0212] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned image processing method.

[0213] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions. The instructions can be executed by the processor 420 of the electronic device 400 to perform the above-mentioned image processing method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0214] In an exemplary embodiment, a computer program product is further provided, including a computer program, wherein the computer program implements the above-mentioned image processing method when executed by a processor.

[0215] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0216] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Acquire a face image to be processed and a target processing item, wherein the target processing item is used to represent a processing method for the face image to be processed; Acquire a target reference offset map corresponding to the target processing item, wherein the color value in the target reference offset map is used to represent the offset vector of each pixel after the reference image is processed by the target processing item; According to the facial key points of the facial image to be processed, the facial image to be processed is divided into triangular meshes to obtain a target facial mesh; According to the reference face mesh corresponding to the reference image and the target face mesh, the target reference offset map is mapped onto the target face mesh to obtain an initial offset map corresponding to the face image to be processed, wherein the reference face mesh is obtained by dividing the reference image into triangular meshes based on the face key points of the reference image; Determine a transformation matrix according to the roll angle in the attitude angle and the ratio of the image size of the face image to be processed to the image size of the reference image, wherein the transformation matrix is ​​used to represent the position transformation relationship of the face image to be processed relative to the reference image; Determine the difference between the integer 1 and the yaw angle in the attitude angle as an adjustment coefficient of the color value in the initial offset map; Determine an average area of ​​a triangular mesh of a target mapping point in the target face mesh as a first average area, determine a reference mapping point in the reference face mesh corresponding to the target mapping point, determine an average area of ​​the triangular mesh of the reference mapping point as a second average area, and determine a ratio of the first average area to the second average area as a vertex scale of the target mapping point in the target face mesh; According to the transformation matrix, the adjustment coefficient and the vertex scale, the r channel color value and the g channel color value in the initial offset map are adjusted according to the following formula to obtain a target offset map: in, Is a binary vector, representing the adjusted r channel color value and the adjusted g channel color value, is a binary vector, representing the color value of the r channel in the initial offset map and the color value of the g channel in the initial offset map, rotMat represents the transformation matrix, degree represents the adjustment coefficient, and scale represents the vertex scale; The face image to be processed is processed according to the target offset map to obtain a target image.

2. The method according to claim 1, characterized in that According to the facial key points of the facial image to be processed, the facial image to be processed is divided into triangular meshes to obtain a target facial mesh, including: Determine a target expansion point corresponding to a facial key point of the face image to be processed, wherein a distance between the target expansion point and a facial key point in the face image to be processed is greater than a distance between a reference expansion point in the reference image and a facial key point in the reference image, the reference expansion point is an expansion point corresponding to a facial key point in the reference image, and the reference face mesh is obtained by triangulating the reference image based on the facial key points in the reference image and the reference expansion point; According to the facial key points of the facial image to be processed and the target expansion points, the facial image to be processed is divided into triangular meshes to obtain a target facial mesh.

3. The method according to claim 1, characterized in that Processing the face image to be processed according to the target offset map to obtain a target image includes: Determining, according to the target offset map, an offset vector corresponding to performing the processing of the target processing item on the face image to be processed; The pixel points in the face image to be processed are subjected to the offset processing of the offset vector to obtain the target image.

4. The method according to claim 3, characterized in that Determining, according to the target offset map, an offset vector corresponding to processing the target processing item on the face image to be processed, comprising: Determining a lateral offset vector in the offset vector according to an r channel color value in the target offset map; A longitudinal offset vector in the offset vector is determined according to a g channel color value in the target offset map.

5. The method according to claim 1, characterized in that Before acquiring the target reference offset map corresponding to the target processing item, the method further includes: Performing liquefaction deformation processing of the target processing item on the reference image to obtain an offset vector of a pixel point in the reference image; The target reference offset map is generated according to the offset vectors of the pixels in the reference image.

6. The method according to claim 5, characterized in that Generating the target reference offset map according to the offset vector of the pixel point in the reference image includes: According to the offset vector of the pixel in the reference image, the color value of the pixel in the target reference offset image is determined according to the following formula: Among them, rgbaColor is the color value of the pixel in the target reference offset image, xyOffset is the offset vector of the pixel in the reference image, is a binary vector representing the preset color adjustment value of the r channel and the preset color adjustment value of the g channel of the pixel point in the target reference offset map, m is the preset color value of the b channel of the pixel point in the target reference offset map, and n is the preset color value of the a channel of the pixel point in the target reference offset map.

7. The method according to claim 5, characterized in that Also includes: splicing the target reference offset map and reference offset maps corresponding to other processing items other than the target processing item into a texture image; The obtaining of the target reference offset map corresponding to the target processing item includes: A target reference offset map corresponding to the target processing item is obtained from the texture image.

8. An image processing device, characterized in that: include: An image acquisition module is configured to acquire a face image to be processed and a target processing item, wherein the target processing item is used to represent a processing method for the face image to be processed; A reference offset map acquisition module is configured to acquire a target reference offset map corresponding to the target processing item, wherein the color value in the target reference offset map is used to represent the offset vector of each pixel after the reference image is processed by the target processing item; A mesh division unit is configured to perform triangular mesh division on the face image to be processed according to the face key points of the face image to be processed to obtain a target face mesh; an offset map mapping unit, configured to map the target reference offset map onto the target face mesh according to the reference face mesh corresponding to the reference image and the target face mesh, to obtain an initial offset map corresponding to the face image to be processed, wherein the reference face mesh is obtained by triangulating the reference image based on the face key points of the reference image; a transformation matrix determination subunit, configured to determine a transformation matrix according to a roll angle in a posture angle and a ratio of an image size of the face image to be processed to an image size of the reference image, wherein the transformation matrix is ​​used to represent a position transformation relationship of the face image to be processed relative to the reference image; An adjustment coefficient determination subunit is configured to determine the difference between the integer 1 and the yaw angle in the attitude angle as the adjustment coefficient of the color value in the initial offset map; a vertex scale determination subunit, configured to determine an average area of ​​a triangle mesh of a target mapping point in the target face mesh as a first average area, determine a reference mapping point in the reference face mesh corresponding to the target mapping point, determine an average area of ​​the triangle mesh of the reference mapping point as a second average area, and determine a ratio of the first average area to the second average area as a vertex scale of the target mapping point in the target face mesh; The offset map adjustment subunit is configured to adjust the r channel color value and the g channel color value in the initial offset map according to the transformation matrix, the adjustment coefficient and the vertex scale according to the following formula to obtain a target offset map: in, Is a binary vector, representing the adjusted r channel color value and the adjusted g channel color value, is a binary vector, representing the color value of the r channel in the initial offset map and the color value of the g channel in the initial offset map, rotMat represents the transformation matrix, degree represents the adjustment coefficient, and scale represents the vertex scale; The image processing module is configured to process the face image to be processed according to the target offset map to obtain a target image.

9. The device according to claim 8, characterized in that The grid division unit comprises: a target expansion point determination subunit, configured to determine a target expansion point corresponding to a facial key point of the to-be-processed facial image, wherein a distance between the target expansion point and a facial key point in the to-be-processed facial image is greater than a distance between a reference expansion point in the reference image and a facial key point in the reference image, the reference expansion point is an expansion point corresponding to a facial key point in the reference image, and the reference facial mesh is obtained by triangulating the reference image based on the facial key points in the reference image and the reference expansion point; The mesh division subunit is configured to perform triangular mesh division on the face image to be processed according to the face key points of the face image to be processed and the target expansion points to obtain a target face mesh.

10. The device according to claim 8, characterized in that The image processing module comprises: an offset vector determining unit, configured to determine, according to the target offset map, an offset vector corresponding to the processing of the target processing item on the face image to be processed; The offset processing unit is configured to perform offset processing of the offset vector on pixel points in the face image to be processed to obtain the target image.

11. The device according to claim 10, characterized in that The offset vector determination unit is configured to perform: Determining a lateral offset vector in the offset vector according to an r channel color value in the target offset map; A longitudinal offset vector in the offset vector is determined according to a g channel color value in the target offset map.

12. The device according to claim 8, characterized in that The device also includes: A reference image liquefaction deformation module is configured to perform liquefaction deformation processing of the target processing item on the reference image to obtain an offset vector of a pixel point in the reference image; The reference offset map generation module is configured to generate the target reference offset map according to the offset vector of the pixel point in the reference image.

13. The device according to claim 12, characterized in that The reference offset map generation module is configured to perform: According to the offset vector of the pixel in the reference image, the color value of the pixel in the target reference offset image is determined according to the following formula: Among them, rgbaColor is the color value of the pixel in the target reference offset image, xyOffset is the offset vector of the pixel in the reference image, is a binary vector representing the preset color adjustment value of the r channel and the preset color adjustment value of the g channel of the pixel point in the target reference offset map, m is the preset color value of the b channel of the pixel point in the target reference offset map, and n is the preset color value of the a channel of the pixel point in the target reference offset map.

14. The device according to claim 12, characterized in that The device also includes: An offset map stitching module is configured to stitch the target reference offset map and reference offset maps corresponding to other processing items other than the target processing item into a texture image; The reference offset map acquisition module is configured to execute: A target reference offset map corresponding to the target processing item is obtained from the texture image.

15. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 7. 16 . A computer-readable storage medium, when instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method according to claim 1 .

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Images