Image deformation method and device, equipment, storage medium and product
By converting the target deformation grid of the image to the map coordinate space and performing coordinate blur, the problem of unnatural transition between image deformation and background is solved, and a more natural deformation effect is achieved.
Patent Information
- Application Number
- CN202410984067.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-26
AI Technical Summary
In the existing image deformation technology, the transition between the deformed image and the background is unnatural, resulting in poor deformation effect.
The target deformation grid of the image to be processed is converted to the map coordinate space, coordinate blur processing is performed, a second map image is generated, and the original image is sampled according to the image to achieve a smoother deformation effect.
Through coordinate blur processing, the image deformation effect is improved, making the deformed image and background transition more natural, and the quality of the deformation effect is significantly improved.
Smart Images

Figure CN119027302A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image deformation method, device, equipment, storage medium and product. Background Art
[0002] With the development of computer technology and image processing technology, the types of image processing are becoming more and more diverse, and people's requirements for image processing effects are also getting higher and higher. For example, people can achieve richer image effects through image deformation processing.
[0003] Image deformation processing can be understood as image processing that deforms a set target in an image so that the target in the image changes shape, such as deforming a face in an image to achieve deformation effects such as thin face, enlarged eyes, and adding tentacles. At present, image deformation generally directly converts the target deformation grid in the original image into the mapping coordinate space to obtain a mapping image, and samples the original image according to the mapping image to obtain the deformed target image. However, in this deformation processing method, after the deformed image is overlaid on the original image, the position of the deformed image and the background position change greatly, the change between the deformation effect and the background is more abrupt, and the image deformation effect is poor. Summary of the invention
[0004] The embodiments of the present application provide an image deformation method, device, equipment, storage medium and product to solve the technical problem of poor image deformation effect in related technologies, and can effectively improve the image deformation effect.
[0005] In a first aspect, an embodiment of the present application provides an image deformation method, comprising:
[0006] Converting a target deformable mesh corresponding to the image to be processed into a mapping coordinate space to obtain a first mapping image, wherein the first mapping image records mapping coordinate values and target mask values corresponding to the target deformable mesh;
[0007] Performing coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image;
[0008] The image to be processed is sampled according to the second map image to obtain a target image.
[0009] In a second aspect, an embodiment of the present application provides an image deformation device, including a texture generation module, a blur processing module and an image sampling module, wherein:
[0010] The texture generation module is configured to convert the target deformed mesh corresponding to the image to be processed into a texture coordinate space to obtain a first texture image, wherein the first texture image records the texture coordinate value and the target mask value corresponding to the target deformed mesh;
[0011] The blur processing module is configured to perform coordinate blur processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image;
[0012] The image sampling module is configured to perform sampling processing on the image to be processed according to the second map image to obtain a target image.
[0013] In a third aspect, an embodiment of the present application provides an image deformation device, including: a memory and one or more processors;
[0014] The memory is used to store one or more programs;
[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the image deformation method as described in the first aspect.
[0016] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the image deformation method as described in the first aspect when executed by a computer processor.
[0017] In the fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium, and at least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the image deformation method described in the first aspect.
[0018] The embodiment of the present application obtains a first map image by converting a target deformation mesh corresponding to an image to be processed into a map coordinate space, obtains a second map image by performing coordinate blur processing on the first map image according to map coordinate values corresponding to the first map image and target mask values, and obtains a target image by sampling the image to be processed according to the second map image, wherein the map coordinate values of the second map image have been subjected to coordinate blur processing, thereby making the transition between the deformation effect and the background smoother and effectively improving the image deformation effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is an effect diagram of face deformation based on the existing image deformation scheme;
[0020] Figure 2is a flow chart of an image deformation method provided by an embodiment of the present application;
[0021] Figure 3 This is a schematic diagram of the deformation effect of the first target image;
[0022] Figure 4 is a flow chart of another image deformation method provided by an embodiment of the present application;
[0023] Figure 5 is a schematic diagram of a target template grid provided in an embodiment of the present application;
[0024] Figure 6 is a schematic diagram of a deformable template grid provided in an embodiment of the present application;
[0025] Figure 7 is a schematic diagram of a target original grid provided in an embodiment of the present application;
[0026] Figure 8 is a schematic diagram of a target deformable mesh provided in an embodiment of the present application;
[0027] Fig. 9 is a schematic diagram of the deformation effect of the second target image provided in an embodiment of the present application;
[0028] Fig.10 is a schematic diagram of a mapping image provided in an embodiment of the present application;
[0029] Fig.11 is a structural schematic diagram of an image deformation device provided in an embodiment of the present application;
[0030] Fig.12 It is a structural schematic diagram of an image deformation device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.
[0032] The image deformation method provided in the present application can be applied to scenarios such as image processing applications, image shooting applications, and video live broadcast applications to meet users' needs for image deformation of videos, pictures, etc. It aims to perform coordinate blur processing on the map image corresponding to the target deformation grid, and sample the processed image based on the map image after coordinate blur processing to obtain the target image, so that the transition between the deformation effect and the background is smoother, thereby improving the image deformation effect.
[0033] In existing image deformation schemes, the original image is generally sampled directly according to the texture image corresponding to the target deformation grid to obtain the deformed target image. After the deformed image is overlaid on the original image, the change between the deformation effect and the background is abrupt, and the image deformation effect is poor. Figure 1 The effect diagram of face deformation based on the existing image deformation scheme is shown in the figure. Assuming that the antenna effect is added to the face, the corresponding deformation effect is to deform the forehead into the shape of antennae. Figure 1 The face deformation effect is directly covered on the background image, and the edge of the face deformation effect changes directly to the background. The deformation effect has a serious sense of disharmony and the deformation effect is poor. Based on this, an image deformation method of an embodiment of the present application is provided to solve the technical problem that the image deformation effect of the existing image deformation solution is poor.
[0034] Figure 2 A flow chart of an image deformation method provided in an embodiment of the present application is given. The image deformation method provided in an embodiment of the present application can be executed by an image deformation device, which can be implemented by hardware and / or software and integrated in the image deformation device.
[0035] The following description is made by taking the image deformation device executing the image deformation method as an example. Figure 1 , the image deformation method includes:
[0036] S110: Convert the target deformable mesh corresponding to the image to be processed into a mapping coordinate space to obtain a first mapping image, wherein the first mapping image records mapping coordinate values corresponding to the target deformable mesh and target mask values.
[0037] The image to be processed provided by this solution can be understood as an image in which the deformable target is to be deformed. The image to be processed can be a picture or a video frame in a video. In one embodiment, the target deformable mesh can be a mesh generated after deforming the deformable target in the image to be processed (for example, deforming the mesh of the deformable target to the target deformable mesh), or it can be a mesh added to the image to be processed. Optionally,
[0038] In one embodiment, the target deformable mesh includes a plurality of mesh vertices, and the plurality of mesh vertices form a plurality of mesh patches. For example, the target deformable mesh may be a triangle mesh, and the triangle mesh has a plurality of mesh vertices, and every three mesh vertices form a triangle patch.
[0039] Exemplarily, a target deformed mesh in the image to be processed is determined, and the target deformed mesh is converted into a mapping coordinate space (UV coordinate space) to obtain a first mapping image (UV image), wherein the first mapping image records mapping coordinate values (UV coordinate values) corresponding to the target deformed mesh and a target mask value (mask value). The target mask value can be used to indicate whether the corresponding pixel point is a pixel point corresponding to the target (foreground pixel point) or a pixel point corresponding to the background.
[0040] In one embodiment, U and V in the texture coordinate space represent the coordinates of the image in the horizontal and vertical directions of the screen, respectively, and these coordinates are usually between (0, 0) and (1, 1). The texture coordinates can be used as a basis for mapping the texture to the target pattern. Through the correspondence between the coordinates of the texture image and the corresponding image, the image can be sampled to obtain a transformed or deformed image.
[0041] S120: performing coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image.
[0042] Exemplarily, after obtaining the first map image, coordinate blurring is performed on the first map image according to the map coordinate values corresponding to each pixel point recorded in the first map image and the target mask value to obtain the second map image. The second map image at least records the map coordinate values of each pixel point after coordinate blurring, and may also include the corresponding target mask value. Optionally, the coordinate blurring of the first map image may be performed by Gaussian blurring of the first map image.
[0043] S130: Sampling the image to be processed according to the second mapping image to obtain a target image.
[0044] Exemplarily, after obtaining the second texture image, the image to be processed is sampled according to the second texture image to obtain the target image. For example, according to the mapping relationship between the second texture image and the image to be processed, color sampling is performed in the image to be processed and the target image is drawn.
[0045] Among them, the mapping coordinate values (UV coordinate values) corresponding to the second mapping image can be converted into pixel coordinates corresponding to the image to be processed, so as to determine the pixels corresponding to each pixel point of the second mapping image in the image to be processed and the color values of the pixels (such as RGB values or YUV values), and draw the deformed image according to the color value, and overlay the deformed image to the corresponding position of the image to be processed to obtain the target image.
[0046] Among them, in the related art, the first map image is generally directly used for sampling and processing to obtain an image with a deformation effect. However, this deformation effect will jump with the background at the edge of the deformation target, the deformation effect is too abrupt, and the image deformation effect is poor. This solution converts the target deformation grid into the map coordinate space, draws a first map image that records the map coordinate value and the target mask value, and effectively achieves the transition effect of the deformation position by performing coordinate fuzzy processing on the first map image. The image to be processed is sampled according to the second map image to obtain the following Figure 3 The schematic diagram of the deformation effect of the first target image is shown. The transition between the target image and the background at the edge of the deformed target is more natural, thereby improving the image deformation effect.
[0047] As described above, a first texture image is obtained by converting a target deformation mesh corresponding to an image to be processed into a texture coordinate space, and a second texture image is obtained by performing coordinate blurring on the first texture image according to texture coordinate values corresponding to the first texture image and target mask values, and a target image is obtained by sampling the image to be processed according to the second texture image, wherein the texture coordinate values of the second texture image have undergone coordinate blurring, thereby making the transition between the deformation effect and the background smoother and effectively improving the image deformation effect.
[0048] Based on the above embodiments, Figure 4 A flowchart of another image deformation method provided in an embodiment of the present application is given, which is a specific implementation of the above-mentioned image deformation method. Figure 4 , the image deformation method includes:
[0049] S210: Convert the target deformable mesh corresponding to the image to be processed into a mapping coordinate space to obtain mapping coordinate values corresponding to each mesh vertex in the target deformable mesh.
[0050] S220: Draw a first mapping image according to the mapping coordinate values corresponding to the target deformable mesh and the target mask corresponding to the target deformable mesh.
[0051] Exemplarily, the target deformable mesh corresponding to the image to be processed is converted into a mapping coordinate space to obtain mapping coordinate values (UV coordinate values) corresponding to each mesh vertex in the target deformable mesh. After obtaining the mapping coordinate values corresponding to each mesh vertex in the target deformable mesh, a first mapping image is drawn according to the mapping coordinate values corresponding to the target deformable mesh and a target mask corresponding to the target deformable mesh.
[0052] Optionally, the target mask corresponding to the target deformable mesh can be determined based on a partial range of the target deformable mesh on the image to be processed. The target mask records the target mask values corresponding to each pixel in a set range in the image to be processed. The pixel point can be determined as a background corresponding pixel point or a target deformable mesh corresponding pixel point based on the target mask value. This solution converts the target deformable mesh corresponding to the image to be processed into a mapping coordinate space to obtain the mapping coordinate values corresponding to each mesh vertex in the target deformable mesh.
[0053] In one embodiment, the image deformation method provided by the present scheme draws a first mapping image according to the mapping coordinate values corresponding to the target deformed mesh and the target mask corresponding to the target deformed mesh. It can be that the first color channel value and the second color channel value are drawn according to the mapping coordinate values corresponding to the target deformed mesh, and the third color channel value is drawn according to the target mask corresponding to the target deformed mesh to obtain the first mapping image.
[0054] Among them, the first color channel value, the second color channel value and the third color channel value can be RGB or YUV three color channels. This scheme is described using RGB three channels as an example, that is, the first color channel value, the second color channel value and the third color channel value have no R channel, G channel and B channel respectively.
[0055] Exemplarily, the texture coordinate values corresponding to each pixel point of the target deformed mesh are drawn to the first color channel value and the second color channel value, and the target mask corresponding to each pixel point of the target deformed mesh is drawn to the third color channel value, to obtain a first texture image. Optionally, drawing the texture coordinate values and the target mask to the corresponding color channels can be performed in a vertex shader.
[0056] Among them, the first color channel in the first mapping image can represent the horizontal coordinate (U coordinate) corresponding to the mapping coordinates, the second color channel can represent the vertical coordinate (V coordinate) corresponding to the mapping coordinates, and the third color channel can represent whether the corresponding pixel point is a background corresponding pixel point or a target deformation grid corresponding pixel point.
[0057] For example, when the value corresponding to the target mask is the first set value (for example, 1), it can indicate that the corresponding pixel point is the pixel point corresponding to the target deformed mesh. When the value corresponding to the target mask is the second set value (for example, 01), it can indicate that the corresponding pixel point is the pixel point corresponding to the background. When the value corresponding to the target mask is the first set value (for example, 1), it can indicate that the corresponding pixel point is the pixel point corresponding to the background.
[0058] This solution accurately obtains the mapping coordinate values corresponding to the target deformation grid and the first mapping image of the target mask value by drawing the mapping coordinate values and the target mask to the first color channel value, the second color channel value and the third color channel value respectively, thereby effectively improving the accuracy of image deformation.
[0059] In a possible embodiment, the image deformation method provided by the present scheme can also deform the target original mesh corresponding to the image to be processed according to the mesh deformation information to obtain the target deformed mesh before converting the target deformed mesh corresponding to the image to be processed to the mapping coordinate space to obtain the first mapping image, and the mesh deformation information is obtained based on the target template mesh and the deformation template mesh.
[0060] In one embodiment, the corresponding target template mesh and deformed template mesh can be determined according to the target to be deformed and the deformation method, and mesh deformation information can be obtained according to the target template mesh and the deformed template mesh. The mesh deformation information records the coordinate deformation information of each mesh vertex between the deformed template mesh and the target template mesh.
[0061] Exemplarily, after obtaining the image to be processed, target recognition is performed on the image to be processed according to the target type that needs to be deformed, the target original grid corresponding to the deformed target in the image to be processed is determined, and the target original grid is deformed according to pre-set grid deformation information to obtain a target deformed grid.
[0062] In one embodiment, the deformation effect is to deform the shape of tentacles at the forehead of a human face as an example. Figure 5 A target template grid diagram and Figure 6 As shown in the provided schematic diagram of a deformable template mesh, a target template mesh is built according to a standard face or a built target template mesh is obtained, and a deformable template mesh with the same topology is made based on the target template mesh. The deformation value corresponding to each mesh vertex can be determined according to the difference between the coordinates of each mesh vertex of the deformable template mesh and the coordinates of the corresponding mesh vertex of the target template mesh, and the mesh deformation information is determined according to the deformation value of each mesh vertex.
[0063] like Figure 7 A schematic diagram of the target original grid and Figure 8 As shown in a schematic diagram of a target deformed mesh provided, after obtaining the target original mesh, the deformation values corresponding to each mesh vertex recorded in the mesh deformation information can be superimposed on the corresponding mesh vertex coordinate values to obtain the target deformed mesh.
[0064] This solution deforms the target original mesh corresponding to the processed image according to the mesh deformation information to obtain the target deformed mesh. Subsequently, the mapping image can be generated and the coordinate fuzzy processing can be performed based on the target deformed mesh to accurately realize the image deformation processing and ensure the image deformation effect.
[0065] S230: performing coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image.
[0066] In one embodiment, the coordinate blurring process of the first texture image in this solution can be performed in a graphics processing unit (GPU) or in a central processing unit (CPU). For example, the process can be performed using a patch accelerator or an acceleration library of an image processor, and the parallel computing advantage of the graphics processor can be utilized to achieve a real-time pixel-level image deformation effect.
[0067] In a possible embodiment, the image deformation method provided by the present scheme performs coordinate blur processing on the first map image according to the map coordinate value and the target mask value to obtain the second map image, which can be: based on the set blur range control value and blur transition control value, coordinate blur processing is performed on the first map image according to the map coordinate value and the target mask value to obtain the second map image.
[0068] Exemplarily, for each pixel in the first map image, coordinate blurring is performed based on the map coordinate value corresponding to the pixel and the target mask value to obtain the blurred map coordinate value, and the second map image is obtained based on the blurred map coordinate value corresponding to each pixel in the first map image.
[0069] The fuzzy range control value and fuzzy transition control value provided in this solution are used to configure the fuzzy range and fuzzy smoothness corresponding to the coordinate fuzzy processing. The larger the fuzzy range control value, the larger the fuzzy range corresponding to the coordinate fuzzy processing; the larger the fuzzy transition control value, the smoother the coordinate fuzzy processing. Fig. 9 The second target image deformation effect diagram is shown in FIG. Figure 3 The difference between the first target image deformation effect diagram provided is that, Fig. 9 The larger the corresponding blur range control value, the larger the corresponding blur range, and the larger the transition area between the deformation effect and the background.
[0070] This solution flexibly controls the blur range and blur smoothness corresponding to the coordinate blur processing through the blur range control value and the blur transition control value, and more flexibly performs coordinate blur processing on the first map image, thereby improving the flexibility of generating the second map image, and can more accurately configure the smooth transition of the deformation effect, thereby improving the image deformation effect.
[0071] In a possible embodiment, the present solution provides an image deformation method, characterized in that coordinate blurring is performed on a first texture image according to texture coordinate values and a target mask value to obtain a second texture image, which may be: coordinate blurring is performed on the first texture image in a first direction according to the texture coordinate values and the target mask value to obtain an intermediate texture image; coordinate blurring is performed on the intermediate texture image in a second direction according to the texture coordinate values and the target mask value to obtain a second texture image.
[0072] Exemplarily, the first map image may be subjected to coordinate blur processing in the first direction and the second direction respectively, so as to reduce the amount of calculation for coordinate blur processing and improve the efficiency of coordinate blur processing. For example, according to the map coordinate value and the target mask value, the first map image is first subjected to coordinate blur processing in the first direction to obtain an intermediate map image, and then according to the map coordinate value and the target mask value, the intermediate map image is subjected to coordinate blur processing in the second direction to obtain a second map image. Optionally, the first direction and the second direction may be a horizontal direction and a vertical direction, for example, the first direction and the second direction may be an x direction and a y direction.
[0073] Optionally, based on the set blur range control value and blur transition control value, coordinate blur processing can be performed on the first map image in a first direction according to the map coordinate values and the target mask value to obtain an intermediate map image; based on the set blur range control value and blur transition control value, coordinate blur processing can be performed on the intermediate map image in a second direction according to the map coordinate values and the target mask value to obtain a second map image.
[0074] Optionally, the first map image may be subjected to coordinate blurring processing based on the following coordinate blurring processing formula in the first direction (x direction):
[0075]
[0076] Among them, uv x is the coordinate value of the intermediate texture image at the (x+k,y) position, uv(x+k,y) is the coordinate value of the first texture image at the (x+k,y) position, ksize is the fuzzy range control value, and σ is the fuzzy transition control value.
[0077] After the intermediate texture image is obtained, the coordinate blur processing can be performed on the intermediate texture image based on the following coordinate blur processing formula in the second direction (x direction):
[0078]
[0079] Among them, uv y is the coordinate value of the second texture image at the position (x+k,y), uv x (x, y+k) is the coordinate value of the intermediate map image at the position (x+k, y).
[0080] Optionally, a mapping image whose position and size correspond to the positioning frame can be generated at the positioning frame corresponding to the deformed target in the image to be processed. The positioning frame setting corresponding to the deformed target can be determined based on the first target positioning frame (bounding box) determined by target recognition of the image to be processed, for example, the first target positioning frame or the first target positioning frame of a set multiple is used as the positioning frame corresponding to the deformed target. It can also be determined based on the second target positioning frame determined by the target deformed grid added to the image to be processed (for example, the second target positioning frame that surrounds or inscribes the target deformed grid is determined based on the position of the target deformed grid on the image to be processed), for example, the second target positioning frame or the second target positioning frame of a set multiple is used as the positioning frame corresponding to the deformed target. This solution reduces the amount of data processing and improves the efficiency of image deformation by generating a mapping image whose position and size correspond to the positioning frame at the positioning frame corresponding to the deformed target in the image to be processed.
[0081] like Fig.10 A kind of map image schematic diagram is shown in the provided Fig.10 In which m11 is the first texture image in the image to be processed, m21 is the middle texture image in the image to be processed, and m31 is the second texture image in the image to be processed. After the target deformation grid corresponding to the image to be processed is converted into the texture coordinate space to obtain the first texture image m11, the first texture image m1 is subjected to coordinate fuzzy processing in the horizontal direction to obtain the middle texture image m21, and then the middle texture image m31 is subjected to coordinate fuzzy processing in the vertical direction to obtain the second texture image m31. This solution effectively reduces the amount of calculation of coordinate fuzzy processing by performing coordinate fuzzy processing on the first texture image in the first direction and the second direction respectively, and coordinate fuzzy processing can be processed in parallel in the graphics processor to improve the efficiency of coordinate fuzzy processing and achieve real-time pixel-level image deformation effect.
[0082] In one embodiment, the image deformation method provided by the present scheme performs coordinate fuzzy processing on the first map image according to the map coordinate value and the target mask value to obtain the second map image, which can be: for each pixel point in the first map image, when the target mask value is a first set value, coordinate fuzzy processing is performed based on the map coordinate value corresponding to the target deformation grid; for each pixel point in the first map image, when the target mask value is a second set value, coordinate fuzzy processing is performed based on the original coordinate value corresponding to the image to be processed.
[0083] Exemplarily, for each pixel point in the first map image, determine whether the target mask value is the first set value or the second set value, wherein when the target mask value is the first set value, it can be understood that the corresponding pixel point is the pixel point corresponding to the target deformation grid, and when the target mask value is the second set value, it can be understood that the corresponding pixel point is the background pixel point.
[0084] When the target mask value is a first set value, coordinate fuzzy processing is performed based on the mapping coordinate values corresponding to the target deformed mesh, and when the target mask value is a second set value, coordinate fuzzy processing is performed based on the original coordinate values corresponding to the image to be processed.
[0085] For example, in the patch shader, the pixel is determined to be a foreground pixel (the pixel corresponding to the target deformation mesh) or a background pixel according to the third channel value of the first texture image. When the pixel is a foreground pixel, the texture coordinate value (U value and V value) of the first texture image is used as the first color channel value and the second color channel. When the pixel is a background pixel, the original coordinate value (horizontal coordinate value and vertical coordinate value) corresponding to the image to be processed is used as the first color channel value and the second color channel. At this time, uv(x+k,y) is the texture coordinate value of the first texture image or the original coordinate value corresponding to the image to be processed, and uv x (x, y+k) is the texture coordinate value of the intermediate texture image or the original coordinate value corresponding to the image to be processed.
[0086] The coordinate value after Gaussian blurring of the coordinates is obtained by accumulating the pixel coordinate values (UV values) within the range corresponding to the pixel blur range control value. For example, for the first direction and the second direction, the mapping coordinate values corresponding to the target deformed grid or the original coordinate values corresponding to the image to be processed are substituted into the coordinate blurring processing formulas for the first direction and the second direction according to the target mask values of each pixel point, and the coordinate blurring processing for the first direction and the second direction is performed to obtain the second mapping image. This solution determines to perform coordinate blurring processing according to the mapping coordinate values or the original coordinate values according to the target mask value, and accurately performs coordinate blurring processing on the deformation target range of the mapping image, thereby improving the image deformation effect.
[0087] S240: Sampling the image to be processed according to the second map image to obtain a target image.
[0088] In the above, the first texture image is obtained by converting the target deformed grid corresponding to the image to be processed into the texture coordinate space, the first texture image is subjected to coordinate fuzzy processing according to the texture coordinate value and the target mask value corresponding to the first texture image to obtain the second texture image, and the image to be processed is sampled according to the second texture image to obtain the target image, wherein the texture coordinate value of the second texture image has been subjected to coordinate fuzzy processing, so that the transition between the deformation effect and the background is smoother, and the image deformation effect is effectively improved. At the same time, by drawing the texture coordinate value and the target mask to the first color channel value, the second color channel value and the third color channel value respectively, the first texture image reflecting the texture coordinate value and the target mask value corresponding to the target deformed grid is accurately obtained, and the image deformation accuracy is effectively improved. And by performing coordinate fuzzy processing on the first texture image in the first direction and the second direction respectively, the calculation amount of the coordinate fuzzy processing is effectively reduced, and the coordinate fuzzy processing can be processed in parallel in the graphics processor, the efficiency of the coordinate fuzzy processing is improved, and the real-time pixel-level image deformation effect is achieved.
[0089] Fig.11 is a schematic diagram of the structure of an image deformation device provided in an embodiment of the present application. Fig.11 The image deformation device includes a map generation module 31, a blur processing module 32 and an image sampling module 33.
[0090] Among them, the map generation module 31 is configured to convert the target deformation mesh corresponding to the image to be processed into the map coordinate space to obtain a first map image, and the first map image records the map coordinate values and target mask values corresponding to the target deformation mesh; the blur processing module 32 is configured to perform coordinate blur processing on the first map image according to the map coordinate values and the target mask value to obtain a second map image; the image sampling module 33 is configured to sample the image to be processed according to the second map image to obtain the target image.
[0091] As described above, a first texture image is obtained by converting a target deformation mesh corresponding to an image to be processed into a texture coordinate space, and a second texture image is obtained by performing coordinate blurring on the first texture image according to texture coordinate values corresponding to the first texture image and target mask values, and a target image is obtained by sampling the image to be processed according to the second texture image, wherein the texture coordinate values of the second texture image have undergone coordinate blurring, thereby making the transition between the deformation effect and the background smoother and effectively improving the image deformation effect.
[0092] In a possible embodiment, the texture generation module 31 converts the target deformed mesh corresponding to the image to be processed into a texture coordinate space to obtain a first texture image, which is configured as follows:
[0093] The target deformable mesh corresponding to the image to be processed is converted into a mapping coordinate space to obtain the mapping coordinate value corresponding to each mesh vertex in the target deformable mesh;
[0094] A first mapping image is drawn according to the mapping coordinate values corresponding to the target deformable mesh and the target mask corresponding to the target deformable mesh.
[0095] In a possible embodiment, the texture generation module 31 draws the first texture image according to the texture coordinate values corresponding to the target deformed mesh and the target mask corresponding to the target deformed mesh, and is configured as follows:
[0096] The first color channel value and the second color channel value are drawn according to the mapping coordinate value corresponding to the target deformable mesh, and the third color channel value is drawn according to the target mask corresponding to the target deformable mesh, so as to obtain a first mapping image.
[0097] In a possible embodiment, the fuzzy processing module 32 performs coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain the second texture image, which is configured as follows:
[0098] For each pixel point in the first texture image, when the target mask value is the first set value, coordinate fuzzy processing is performed based on the texture coordinate value corresponding to the target deformed mesh;
[0099] For each pixel point in the first mapping image, when the target mask value is the second set value, coordinate fuzzy processing is performed based on the original coordinate value corresponding to the image to be processed.
[0100] In a possible embodiment, the fuzzy processing module 32 performs coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain the second texture image, which is configured as follows:
[0101] Based on the set fuzzy range control value and fuzzy transition control value, coordinate fuzzy processing is performed on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image.
[0102] In a possible embodiment, the fuzzy processing module 32 performs coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain the second texture image, which is configured as follows:
[0103] According to the mapping coordinate value and the target mask value, coordinate blurring is performed on the first mapping image in a first direction to obtain an intermediate mapping image;
[0104] According to the mapping coordinate values and the target mask value, coordinate blurring is performed on the intermediate mapping image in the second direction to obtain a second mapping image.
[0105] In a possible embodiment, the image deformation device also includes a mesh deformation module, which is configured to deform the target original mesh corresponding to the processed image according to the mesh deformation information to obtain a target deformed mesh, and the mesh deformation information is obtained based on the target template mesh and the deformed template mesh.
[0106] It is worth noting that in the embodiment of the above-mentioned image deformation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.
[0107] The embodiment of the present application further provides an image deformation device, which can integrate the image deformation apparatus provided in the embodiment of the present application. Fig.12 Schematic diagram of the structure of an image deformation device provided by an embodiment of the present application. Fig.12 The image deformation device includes: an input device 43, an output device 44, a memory 42 and one or more processors 41; the memory 42 is used to store one or more programs; when the one or more programs are executed by the one or more processors 41, the one or more processors 41 implement the image deformation method provided in the above embodiment. The image deformation device, equipment and computer provided above can be used to execute the image deformation method provided in any of the above embodiments, and have corresponding functions and beneficial effects.
[0108] The embodiments of the present application also provide a non-volatile storage medium storing computer executable instructions, which are used to execute the image deformation method provided in the above embodiments when executed by a computer processor. Of course, the non-volatile storage medium storing computer executable instructions provided in the embodiments of the present application, whose computer executable instructions are not limited to the image deformation method provided above, can also execute the related operations in the image deformation method provided in any embodiment of the present application. The image deformation device, equipment and storage medium provided in the above embodiments can execute the image deformation method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the image deformation method provided in any embodiment of the present application.
[0109] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the image deformation method provided in each embodiment of the present application.
Claims
1. An image deformation method, characterized in that: include: Converting a target deformable mesh corresponding to the image to be processed into a mapping coordinate space to obtain a first mapping image, wherein the first mapping image records mapping coordinate values and target mask values corresponding to the target deformable mesh; Performing coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image; The image to be processed is sampled according to the second map image to obtain a target image.
2. The image deformation method according to claim 1, characterized in that: The step of converting the target deformable mesh corresponding to the image to be processed into a mapping coordinate space to obtain a first mapping image includes: Converting the target deformable mesh corresponding to the image to be processed into a mapping coordinate space to obtain mapping coordinate values corresponding to each mesh vertex in the target deformable mesh; A first mapping image is drawn according to the mapping coordinate values corresponding to the target deformable mesh and the target mask corresponding to the target deformable mesh.
3. The image deformation method according to claim 2, characterized in that: The step of drawing a first map image according to the map coordinate values corresponding to the target deformed mesh and the target mask corresponding to the target deformed mesh comprises: A first color channel value and a second color channel value are drawn according to the mapping coordinate values corresponding to the target deformed mesh, and a third color channel value is drawn according to the target mask corresponding to the target deformed mesh, so as to obtain a first mapping image.
4. The image deformation method according to claim 1, characterized in that: The step of performing coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image includes: For each pixel point in the first map image, when the target mask value is a first set value, coordinate fuzzy processing is performed based on the map coordinate value corresponding to the target deformed mesh; For each pixel point in the first mapping image, when the target mask value is the second set value, coordinate fuzzy processing is performed based on the original coordinate value corresponding to the image to be processed.
5. The image deformation method according to claim 1, characterized in that: The step of performing coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image includes: Based on the set fuzzy range control value and fuzzy transition control value, coordinate fuzzy processing is performed on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image.
6. The image deformation method according to claim 1, characterized in that: The step of performing coordinate fuzzy processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image includes: According to the mapping coordinate value and the target mask value, coordinate blurring is performed on the first mapping image in a first direction to obtain an intermediate mapping image; According to the mapping coordinate value and the target mask value, coordinate blur processing is performed on the intermediate mapping image in a second direction to obtain a second mapping image.
7. The image deformation method according to claim 1, characterized in that: Before converting the target deformable mesh corresponding to the image to be processed into the mapping coordinate space to obtain the first mapping image, the method further includes: The target original mesh corresponding to the processed image is deformed according to the mesh deformation information to obtain a target deformed mesh, wherein the mesh deformation information is obtained based on the target template mesh and the deformed template mesh.
8. An image deformation device, characterized in that: It includes a map generation module, a blur processing module and an image sampling module, among which: The texture generation module is configured to convert the target deformed mesh corresponding to the image to be processed into a texture coordinate space to obtain a first texture image, wherein the first texture image records the texture coordinate value and the target mask value corresponding to the target deformed mesh; The blur processing module is configured to perform coordinate blur processing on the first texture image according to the texture coordinate value and the target mask value to obtain a second texture image; The image sampling module is configured to perform sampling processing on the image to be processed according to the second map image to obtain a target image.
9. An image deformation device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image deformation method as described in any one of claims 1 to 7.
10. A non-volatile storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to perform the image deformation method according to any one of claims 1 to 7 when executed by a computer processor.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image deformation method according to any one of claims 1 to 7 is implemented.
