Image processing method and device and electronic equipment

Abstract

The present invention discloses an image processing method, device and electronic device. The method comprises: obtaining a first group of positions of a first group of feature points in a first image to be deformed, and moving the first group of feature points to obtain a second group of positions; determining radial basis fitting parameters according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions and the second group of position values, and determining a target deformation function according to the radial basis parameters; according to the target deformation function, moving each pixel point in the first image to a corresponding pixel point in the second image to obtain a deformed second image, thereby achieving a deformation field of an image based on a radial basis function, and generating a deformed image by transmitting the deformation field to a graphics processor, thereby solving the technical problems in the prior art that the spatial structure of points is determined due to the grid, arbitrary large-scale deformation cannot be performed, and the smoothness is low in image deformation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method and device and electronic equipment. Background Art

[0002] The research on face deformation technology has always been one of the important topics in image processing research. It is widely used in face effects in short videos, such as face fusion and face swapping, as well as some beautification effects in virtual images, such as face thinning and enlarging eyes. Real-time drag deformation is difficult in face deformation technology because of the influence of large-scale deformation and the need for real-time speed while ensuring the smoothness and naturalness of the deformation edges. Most beautification algorithms such as face thinning and enlarging eyes mainly focus on small-scale deformation in small local areas.

[0003] There are two main types of existing 2D face deformation schemes. The first type is interpolation algorithms, such as thin plate spline and IDW interpolation, which perform deformation in the entire pixel domain. The speed of this type of deformation is proportional to the number of pixels, especially for high-resolution images. The speed is still very slow when performing global deformation, which also limits this type of algorithm from being able to perform very complex global face deformation. The second type is mainly algorithms such as triangular patch subdivision, such as MLS (moving least squares) deformation, ARAP (as rigid as possible) deformation, BBW (Bounded biharmonic weights) deformation, etc., which mainly use grid subdivision technology to subdivide the entire 2D image, calculate the grid point position and texture coordinates on the image, and use OpenGL to render to obtain the final deformation effect. This type of deformation calculation can also perform real-time deformation on high-resolution images, but is generally limited to local small-scale deformation. Since the spatial structure of the points is determined by the grid, arbitrary large-scale deformation cannot be performed, and there is also a problem of smooth deformation.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide an image processing method and device and an electronic device to at least solve the technical problems in the prior art that the spatial structure of points is determined due to the grid, arbitrary large-scale deformation cannot be performed, and the smoothness is low in image deformation.

[0006] According to one aspect of an embodiment of the present invention, there is provided an image processing method, comprising: obtaining positions of a first group of feature points in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions; moving the first group of feature points to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions; determining radial basis fitting parameters according to an offset between each position in the first group of positions and each position in a third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, wherein the third group of positions are positions selected from the first group of positions; determining the target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine a movement amount corresponding to an input position according to an input position, and the target deformation function is used to determine a movement amount corresponding to an input position according to an input position; according to the target deformation function, moving each pixel point in the first image to a corresponding pixel point in the second image to obtain the deformed second image.

[0007] According to another aspect of an embodiment of the present invention, there is further provided an image processing device, comprising: a first acquisition unit, configured to acquire positions of a first group of feature points in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions; a first obtaining unit, configured to move the first group of feature points to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions; a first determining unit, configured to determine radial basis fitting parameters according to an offset between each position in the first group of positions and each position in a third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; a second determining unit, configured to determine the target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine a movement amount corresponding to an input position according to an input position; and a second obtaining unit, configured to move each pixel point in the first image to a corresponding pixel point in the second image according to the target deformation function, to obtain the deformed second image.

[0008] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned image processing method when running.

[0009] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the image processing method through the computer program.

[0010] In an embodiment of the present invention, positions of a first group of feature points are obtained in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions; the first group of feature points are moved to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions; radial basis fitting parameters are determined according to an offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; a target deformation function is determined according to the radial basis fitting parameters, wherein the target deformation function is used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions. The invention is to determine the movement amount corresponding to the input position according to the input position; according to the target deformation function, each pixel point in the first image is moved to the corresponding pixel point in the second image to obtain the deformed second image, so as to achieve the goal of constructing the deformation field of the image based on the radial basis function according to the first group of positions and the second group of positions of a group of feature points, and to generate a deformed image by transmitting the deformation field to the graphics processor to obtain a smooth deformation result. Since it is not constrained by the point space structure, it produces a smooth deformation effect in a large range, thereby solving the technical problems in the prior art that the spatial structure of the points is determined by the grid, arbitrary large-scale deformation cannot be performed, and the smoothness of the image deformation is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0012] Figure 1 is a schematic diagram of an application environment of an optional image processing method according to an embodiment of the present invention;

[0013] Figure 2 is a flow chart of an optional image processing method according to an embodiment of the present invention;

[0014] Figure 3 is a schematic diagram of an optional correspondence relationship between a small-scale deformation field and a large-scale deformation field according to an embodiment of the present invention;

[0015] Figure 4 is a flowchart of an optional large-scale face deformation method based on deformation field according to an embodiment of the present invention;

[0016] Figure 5is a flow chart of an optional large-scale face deformation based on deformation field according to an embodiment of the present invention;

[0017] Figure 6 is a schematic structural diagram of an optional image processing device according to an embodiment of the present invention;

[0018] Figure 7 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] According to one aspect of an embodiment of the present invention, an image processing method is provided. Optionally, as an optional implementation, the image processing method may be, but is not limited to, applied to: Figure 1 In the environment shown, there are a terminal device 102, a network 104 and a server 106. The terminal device 102 runs an image client for completing image processing.

[0022] The terminal device 102 may include but is not limited to: a human-computer interaction screen 104, a processor 106 and a memory 108. The human-computer interaction screen 104 is used to obtain a human-computer interaction instruction through a human-computer interaction interface, and is also used to present a first image to be deformed and a second image after the first image is deformed; the processor 106 is used to respond to the human-computer interaction instruction, drag the first image to obtain the second image to complete the image processing. The memory 108 is used to store the attribute information of the first image to be deformed, the first group of positions, the second group of positions, the deformation function and the attribute information of the second image. Here, the server may include but is not limited to: a database 114 and a processing engine 116, wherein the processing engine 116 is used to call the first image to be deformed stored in the database 114, obtain the positions of the first group of feature points in the first image to be deformed, and the positions of the first group of feature points constitute the first group of positions; move the first group of feature points to obtain the second group of feature points, wherein the positions of the second group of feature points constitute the second group of positions; determine radial basis fitting parameters according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions and the second group of position values, wherein the radial basis fitting parameters are used to fit the target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; determine the target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine the movement amount corresponding to the input position according to the input position; according to the target deformation function, move each pixel point in the first image to the corresponding pixel point in the second image to obtain the deformed second image. The deformation field of the image is constructed based on the radial basis function according to the first group of positions and the second group of positions of a group of feature points. By transmitting the deformation field to the graphics processor, a deformed image is generated, and a smooth deformation result is obtained. Since it is not constrained by the point space structure, a smooth deformation effect is produced in a large range, thereby solving the technical problems in the prior art that the spatial structure of the points is determined by the grid, arbitrary large-scale deformation cannot be performed, and the smoothness of the image deformation is low.

[0023] The specific process is as follows: the human-computer interaction screen 104 in the terminal device 102 displays the first image to be deformed (such as Figure 1The screenshot of the game is shown). As in steps S102-S112, the positions of the first group of feature points are obtained in the first image to be deformed, wherein the positions of the first group of feature points constitute the first group of positions; the first group of feature points are moved to obtain the second group of feature points, wherein the positions of the second group of feature points constitute the second group of positions, and the first group of positions and the second group of positions are sent to the server 112 through the network 110. The server 112 determines the radial basis fitting parameters according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions and the second group of position values, wherein the radial basis fitting parameters are used to fit the target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; according to the radial basis fitting parameters, the target deformation function is determined, wherein the target deformation function is used to determine the movement amount corresponding to the input position according to the input position; according to the target deformation function, each pixel point in the first image is moved to the corresponding pixel point in the second image to obtain the deformed second image. Then the second image determined as above is returned to the terminal device 102.

[0024] Then, as in steps S114-S116, the terminal device 102 obtains the positions of the first group of feature points in the first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions; the first group of feature points are moved to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions; radial basis fitting parameters are determined according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit the target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; the target deformation function is determined according to the radial basis fitting parameters, wherein the target The target deformation function is used to determine the movement amount corresponding to the input position according to the input position; according to the target deformation function, each pixel point in the first image is moved to the corresponding pixel point in the second image to obtain the deformed second image, thereby achieving the goal of constructing a deformation field of the image based on a radial basis function according to a first group of positions and a second group of positions of a set of feature points, and generating a deformed image by transmitting the deformation field to a graphics processor to obtain a smooth deformation result. Since it is not constrained by the point spatial structure, a smooth deformation effect in a large range is produced, thereby solving the technical problems in the prior art that the spatial structure of the points is determined by the grid, arbitrary large-scale deformation cannot be performed, and the smoothness of the image deformation is low.

[0025] In this embodiment, the above image processing may include but is not limited to being executed by the terminal device 102, being executed by the server 106, or being executed jointly by the terminal device 102 and the server 106. The above is only an example, and this embodiment does not impose any limitation on this.

[0026] Optionally, in this embodiment, the terminal device 102 may be a terminal device configured with a target client, which may include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, etc. The target client may be an image editing client, an image viewing client, an instant messaging client with an image processing function, a live broadcast client with an image processing function, etc. The network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The server may be a single server, or a server cluster consisting of multiple servers, or a cloud server. The above is only an example, and this embodiment does not make any limitation to this.

[0027] Optionally, as an optional implementation, as Figure 2 As shown, the above image processing method includes:

[0028] Step S202: acquiring positions of a first group of feature points in the first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions.

[0029] Step S204: moving the first group of feature points to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions.

[0030] Step S206, determining radial basis fitting parameters based on the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit the target deformation function of the first image, and the third group of positions are positions selected from the first group of positions.

[0031] Step S208, determining a target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine a movement amount corresponding to the input position according to the input position.

[0032] Step S210: according to the target deformation function, each pixel point in the first image is moved to a corresponding pixel point in the second image to obtain a deformed second image.

[0033] Optionally, in this embodiment, the above-mentioned image processing method may include but is not limited to user's arbitrary scale face deformation editing, and various interesting face special effects may be designed according to user's requirements, such as toad mouth, beautification, etc. It can also be used for face changing, face fusion, etc. It can also be used in face cartoon animation, design of exaggerated cartoon images, etc.

[0034] In this embodiment, a drag operation can be performed on the first image in the image editor to obtain a first set of positions before the first image is dragged and a second set of positions after the dragging, and a target deformation function of the first image is determined based on the first set of positions and the second set of positions. Based on the target deformation function, the deformation amount of each pixel in the first image from the first image to the second image is determined, and each pixel in the first image is moved by the corresponding deformation amount to obtain the pixel point in the second image corresponding to each pixel, thereby obtaining the second image obtained after dragging the first image.

[0035] Among them, the above-mentioned first group of feature points may include but are not limited to a group of feature points obtained by extracting facial feature points, and some feature points may also be set through user interaction, that is, a group of feature points may be obtained based on facial feature points obtained by a face recognition algorithm, or feature points in an image may be set through user interaction. The above-mentioned first group of feature points has a one-to-one corresponding first group of positions, and the coordinates of each feature point in a group of feature points obtain a group of position information, that is, a group of position information is a coordinate set. It should be noted that the number of the above-mentioned group of feature points may include but is not limited to 1, two, three, ten, etc., and in this embodiment, no specific limitation is made.

[0036] It should also be noted that the more feature points there are in the first group, the more feature points there are in the second group, and the deformation function of the first image determined according to the first group of positions and the second group of positions is closer to the deformation field of the first image, and a smoother second image can be obtained.

[0037] In this embodiment, the position of the pixel point in the first image is input into the target deformation function, and the amount of movement of the pixel point in the first image that needs to be moved can be determined when the pixel point in the first image is deformed into the pixel point in the second image. For example, by dragging the pixel point A (x1, y1) in the first image, the second image is obtained, and the corresponding pixel point A (x2, y2) in the second image, the movement amount may include Δx = x2-x1, Δy = y2-y1. The pixel point A (x1, y1) in the first image needs to be moved by Δx in the x direction and Δy in the y direction by dragging, where x and y are the coordinate positions of the pixel point in the pixel coordinates. The pixel point is deformed from the first image to be deformed to the second image, and each pixel point in the first image to be deformed needs to have a corresponding movement amount in the x direction and the y direction.

[0038] In practical applications, a group of feature points of the first image to be deformed can be obtained through a face recognition algorithm, including a feature point at the forehead, two feature points at the eyes (left and right eyes), a feature point at the nose, two feature points at the ears (left and right ears), a first feature point at the mouth, a feature point at the chin, and two feature points at the cheeks (left and right cheeks). Ten feature points form a group of feature points. The positions of the first group of 10 feature points are obtained in the first image to be deformed, and the position of each point in the 10 feature points is obtained to obtain a first group of positions that have a one-to-one correspondence with the first group of feature points. The above position information may include but is not limited to the two-dimensional coordinates of the pixel points corresponding to the first image to be deformed. The first group of position information of a group of feature points is expressed as (x1, y1), (x2, y2), (x3, y3), (x4, y4), ... (x10, y10).

[0039] It should be noted that the first group of 10 feature points can also be selected through human interaction.

[0040] In this embodiment, the positions of a group of feature points in the first image to be deformed and the positions of the second group of feature points obtained by moving a group of feature points can be obtained in advance, that is, a group of feature points with known position changes can be pre-selected, and the position information can be coordinate information. The original coordinates of a group of feature points before the change and the coordinates of a group of feature points after the change can be obtained in advance. According to the coordinates before the change and the coordinates after the change, a radial basis function is constructed to determine the target deformation function of the first image. According to the target deformation function, the movement amount of each pixel point in the first image after each drag operation on the first image can be determined, and then according to the original coordinates and the movement amount in the first image, the pixel coordinates of each pixel point in the first image corresponding to the second image are determined, that is, the second image after the first image is dragged is obtained, and the image processing is completed.

[0041] Through the embodiments provided by the present application, the positions of a first group of feature points are obtained in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions; the first group of feature points are moved to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions; radial basis fitting parameters are determined according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; according to the radial basis fitting parameters, a target deformation function is determined, wherein the target deformation function Used to determine the movement amount corresponding to the input position according to the input position; according to the target deformation function, each pixel point in the first image is moved to the corresponding pixel point in the second image to obtain the deformed second image, thereby achieving the goal of constructing a deformation field of the image based on a radial basis function according to a first group of positions and a second group of positions of a set of feature points, and generating a deformed image by transmitting the deformation field to a graphics processor to obtain a smooth deformation result. Since it is not constrained by the point spatial structure, a smooth deformation effect in a large range is produced, thereby solving the technical problems in the prior art that the spatial structure of the points is determined by the grid, arbitrary large-scale deformation cannot be performed, and the smoothness of the image deformation is low.

[0042] Optionally, determining radial basis fitting parameters according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values ​​may include:

[0043] S1, obtaining a position offset between each position in the first group of positions and each position in the third group of positions, and obtaining a group of position offsets in total;

[0044] S2, determining a first group of weight values ​​and a second group of weight values ​​according to a group of position offsets, a first group of positions, and a second group of positions, wherein the radial basis fitting parameters include the first group of weight values ​​and the second group of weight values.

[0045] In this embodiment, the offset between each position in a set of positions and each position in a third set of positions is obtained. For example, if a set of positions includes the positions of 10 feature points and the third set of positions may include the positions of 5, 6, or 10 feature points, the offset can be obtained based on the known offsets of the feature points in the third group before and after deformation.

[0046] It should be noted that, in this embodiment, when the first image to be deformed is deformed into the second image, it is necessary to determine the change amount of each pixel point in the first image when the first image changes to the second image, and the change amount includes the change amount in the X direction and the change amount in the Y direction in the pixel coordinates.

[0047] In this embodiment, the change amount of each pixel point in the first image can be determined through the target deformation function. By constructing a radial basis function, and then fitting the target deformation function according to the radial basis function. Among them, the change amount of the pixel point is a parameter in the radial basis function, and then the target deformation function is determined according to the radial basis function, that is, the change amount of each pixel point in the first image is obtained.

[0048] In this embodiment, in order to construct a globally smooth deformation field, N feature points are predefined at the first position. It is equivalent to affecting the positions of the remaining points in the first image through the changes of these feature points. Let the original positions of these N feature points be u i , i = 0, 1, … n, and the positions of these N feature points after change be v i , then the change amount M f (u i ) = v i - u i , i = 1, 2, … n. Use the radial basis function to fit M f (u i ), as shown in formula (1):

[0049]

[0050] In this embodiment, take m = 1, is the linear basis, which is α0 + α1u. When m = 2, it is a quadratic function, that is, α0 + α1u + α2u 2 , generally m is a low-degree polynomial, that is, m < n. In this embodiment, can also be replaced with an affine transformation basis, which can provide more variations for the radial basis function. Pre-set some fixed points before and after deformation, and assume them to be {(u i , v i )} i＝0，1，…N , thus obtaining the following system of equations:

[0051]

[0052] It should be noted that here N feature points N ≥ n + m. These n feature points are the constraint points for constructing the radial basis, and the other points are used to solve for m variables.

[0053] Among them, in order to conveniently calculate the target deformation function, the system of equations in the above formula (2) is transformed into a matrix. Through matrix representation, as shown in formula (3):

[0054] φ(U)·W = V (3)

[0055] Among them,

[0056]

[0057] It can be expressed as a matrix of N×(n+m),

[0058] It can be represented as a (n+m)×1 matrix, It can be represented as an N×1 matrix.

[0059] In this embodiment, in order to provide the first weight w i To ensure the robustness and smoothness of the deformation field, a regularization term is constructed.

[0060]

[0061] R is the regularization matrix. By taking the derivative, we get the following formula (4):

[0062] (φ T φ+λR T R)W=φ T V (4)

[0063] The solution to equation (4) above is:

[0064] W reg =(φ T φ+λR T R) -1 Φ T V

[0065] Perform matrix singular decomposition SVD on φ and get formula (5):

[0066] φ=X∑Y T (5)

[0067] Where X = (x1, ...x m ) and Y=(y1,…y n ) is an orthogonal matrix, where x i ,y i are respectively m-dimensional vector and n-dimensional vector Σ=diag(σ1,…σ n ),σ1≥σ2≥…σ n >0, if R=0, we can get formula (6):

[0068]

[0069] From this we can see that some singular values ​​are very small, which will lead to There are some small disturbances that will cause W reg The solution error is large and not smooth enough. Here we set R = λI n , thus we get formula (7):

[0070]

[0071] In this embodiment, not only a stable solution can be obtained, but also It also produces a smooth weight W reg , thus obtaining a smooth deformation field. The obtained deformation field is the position v of each pixel after deformation. Then the function formula (8) for calculating the deformation amount can be used to determine the deformation amount of each pixel:

[0072]

[0073] It should be noted that in order to generate a smooth deformation field, it can include but is not limited to using a smooth regularized radial basis function to generate a smooth deformation field, or using a moving least squares method mls, or inverse distance interpolation (IDW) to generate a deformation field.

[0074] It should also be noted that, in this embodiment, singular value decomposition (SVD) also decomposes the matrix, but unlike eigendecomposition, SVD does not require the decomposed matrix to be a square matrix. Assuming that matrix A is an s×p matrix, the SVD of matrix A is A=U∑V T , where U is an s×s matrix, ∑ is an s×p matrix, all elements except the elements on the main diagonal are 0, each element on the main diagonal is called a singular value, and V is a p×p matrix. U and V both satisfy U T U=I,V T V=I.

[0075] In the corresponding application, if a space-based grid division such as triangulation or quadrilateral division is established based on the image, the movement of the points will be restricted, so that arbitrary large-scale deformation cannot be performed, and the smoothness of the deformation will also be affected. In this embodiment, the above-mentioned target deformation function is used to generate this deformation field, which can not only greatly reduce the calculation speed, but also ensure any large-scale smooth deformation.

[0076] Optionally, determining a first group of weight values ​​and a second group of weight values ​​based on a group of position offsets, a first group of positions, and a second group of positions may include: determining a target matrix based on a group of position offsets and the first group of positions; decomposing the target matrix to obtain a first matrix and a second matrix; determining a first group of weight values ​​and a second group of weight values ​​based on the first matrix, the second matrix, and the second group of positions.

[0077] Decomposing the target matrix to obtain the first matrix and the second matrix may include:

[0078] The target matrix is ​​decomposed by the following formula to obtain the first matrix and the second matrix:

[0079] φ(U)=X∑Y T

[0080] The target matrix is:

[0081]

[0082] Where X represents the first matrix, Y represents the second matrix, ∑ represents an N×P matrix, in which all elements except the elements on the main diagonal are 0, and the elements on the main diagonal include Λ 11 , Λ 22 , ..., Λ PP , P=n+m, N represents the number of positions in the first group of positions, n represents the number of positions in the third group of positions, N≥n+m, N, n and m are all preset natural numbers,

[0083]

[0084] φ(u j -u i ) represents the jth position u in the first group of positions j and the i-th position u in the third group of positions i The offset between them, the value of j is 1, 2...N, the value of i is 1, 2...n, The value of s is 1, 2, …m.

[0085] It should be noted that determining the first group of weight values ​​and the second group of weight values ​​according to the first matrix, the second matrix, and the second group of positions may include:

[0086]

[0087]

[0088] Among them, x i represents the i-th column vector in the first matrix, y i represents the i-th column vector in the second matrix, The second set of positions includes v1, v2, ..., v N , The first set of weight values ​​includes w1, w2, ..., w n , the second set of weight values ​​includes α1, α2, ..., α m , σ i The elements on the main diagonal are Λ 11 , Λ 22 , ..., Λ PPthe i-th value Λ in ii , where λ is a preset constant.

[0089] Optionally, according to the radial basis fitting parameters, a target deformation function can be determined, including:

[0090] Determine the following function as the target deformation function:

[0091]

[0092] where the first set of weight values includes w1, w2,..., w n , the second set of weight values includes α1, α2,..., α m , n represents the number of positions in the third set of positions, and both n and m are preset natural numbers, w i represents the i-th weight value in the first set of weight values, α j represents the j-th weight value in the second set of weight values, u i represents the i-th position in the third set of positions, represents the position offset between the target position u in the first image and the i-th position, M f (u) represents the movement amount corresponding to the target position u,

[0093] where, in this embodiment, m = 1, is a linear basis, which is α0 + α1u, m = 2, is a quadratic function, that is, α0 + α1u + α2u 2 , usually m is a low-degree polynomial, that is, m < n. It should be noted that can also be replaced with an affine transformation basis, which can provide more variations for the radial basis function, and then the optimal movement amount corresponding to the deformation of the target position can be determined.

[0094] Optionally, according to the target deformation function, each pixel point in the first image is moved to the corresponding pixel point in the second image to obtain the deformed second image, including:

[0095] S1. Perform the following steps on each pixel point in the first image to obtain the second image. In this case, each pixel point in the first image is regarded as the first current point:

[0096] S2. Obtain the first current position of the first current point in the first image;

[0097] S3. When the first current position is the input position of the target deformation function, obtain the current movement amount output by the target deformation function corresponding to the position of the first current point;

[0098] S4, determining a second current position of a corresponding second current point in the second image according to the current movement amount and the first current position, wherein the second current point is a pixel point moved from the first current point to the second image.

[0099] Wherein, determining the second current position of the second current point corresponding to the second image according to the current movement amount and the first current position may include:

[0100] The second current position is determined by the following formula:

[0101] x′=x0+Δx

[0102] y′=y0+Δy

[0103] Among them, (x′, y′) represents the second current position of the second current point P′, (x0, y0) represents the first current position of the first current point P, and (Δx, Δy) represents the current movement amount.

[0104] In this embodiment, the movement amount of the pixel point is determined by the difference of the coordinates in the pixel point coordinates, and the movement amount may include but is not limited to the movement amount in the X direction and the movement amount in the Y direction in the pixel coordinates.

[0105] It should also be noted that, when the image is a three-dimensional stereoscopic image, the above movement amount may also include the movement amount in the Z direction in the pixel coordinates.

[0106] Optionally, according to a target deformation function, each pixel point in the first image is moved to a corresponding pixel point in the second image to obtain a deformed second image, which may include: reducing the first image to a third image according to a preset ratio; obtaining a movement amount corresponding to the position of each pixel point in the third image according to the target deformation function, wherein the movement amount corresponding to the position of each pixel point in the third image constitutes a first group of movement amounts; and according to the first group of movement amounts and the preset ratio, moving each pixel point in the first image to a corresponding pixel point in the second image.

[0107] In this embodiment, assuming that the width and height of the image are W and H, to generate a two-dimensional deformation field (x, y) of size W×H, you can first Determine the target deformation function and determine the minimum size The amount of movement of each pixel in the small-scale deformation field is The deformation at this point is solved as:

[0108]

[0109] A small-scale deformation field can be obtained. The deformation of each point in the small-scale deformation field corresponds to the large-scale one. Then, the large-scale deformation field is obtained by mapping and interpolating the small-scale deformation field. That is, for a point on the small-scale deformation field The deformation is v(x, y), which corresponds to the deformation on the large-scale deformation field (s*x, t*y). Figure 3 As shown, this is a schematic diagram of the corresponding relationship between the small-scale deformation field and the large-scale deformation field. Figure 3 There are 4 pixel points (x, y), (x+1, y), (x, y+1), and (x+1, y+1) in the small-scale deformation field.

[0110] Corresponding to any point (p, q) in the large-scale deformation field, in the large-scale deformation field, s*x≤p≤s*(x+1), t*y≤q≤s*(y+1), the deformation amount corresponding to each pixel in the large deformation field is interpolated by distance weight, that is,

[0111] v(p, q)=w1v(s*x,t*y)+w2v(s*x,t(y+1))+w3v(s*(x+1),t*y)+w4v(s*(x+1),t*(y+1))

[0112] It should be noted that, when s*x=p,t*y=q,w1 takes the value of 1, and w2, w3, w4, and w1 take the values ​​of 0. When s*x=p,q=s*(y+1), w2 takes the value of 1, and w1, w3, and w4 take the values ​​of 0. When p=s*(x+1),t*y=q,w3 takes the value of 1, and w1, w2, and w4 take the values ​​of 0. When p=s*(x+1),q=s*(y+1), w4 takes the value of 1, and w1, w2, and w3 take the values ​​of 0. That is, when (p, q) takes the same value, the large-scale deformation field corresponding to the small-scale deformation field can be directly determined through the mapping relationship.

[0113] In this embodiment, the deformation amount corresponding to each pixel point in the large deformation field is calculated by interpolation.

[0114] Optionally, moving each pixel point in the first image to a corresponding pixel point in the second image according to the first group of movement amounts and a preset ratio may include:

[0115] S1, restoring the pixels in the third image to the pixels in the first image according to a preset ratio, wherein the pixels in the first image restored to form a first group of pixels, and the second group of movement amounts corresponding to the first group of pixels are determined according to the first group of movement amounts and the preset ratio;

[0116] S2, determining a third group of movement amounts corresponding to a second group of pixel points other than the first group of pixel points in the first image according to the second group of movement amounts and the first group of pixel points, wherein the first group of pixel points and the second group of pixel points constitute the first image;

[0117] S3, moving each pixel point in the first image to a corresponding pixel point in the second image according to the second group of movement amounts and the third group of movement amounts.

[0118] It should be noted that, determining the third group of movement amounts corresponding to the second group of pixel points other than the first group of pixel points in the first image according to the second group of movement amounts and the first group of pixel points may include:

[0119] The third group of movement amounts corresponding to the second group of pixel points other than the first group of pixel points in the first image are determined by the following formula:

[0120] v(p, q)=w1v(s*x,t*y)+w2v(s*x,t(y+1))+w3v(s*(x+1),t*y)+w4v(s*(x+1),t*(y+1))

[0121] Among them, s*x≤p≤s*(x+1), t*y≤q≤s*(y+1)

[0122] The size of the first image is W×H, and the size of the third image is s>1, t>1, the second set of movement amounts includes: v(s*x, t*y), v(s*x, t(y+1)), v(s*(x+1), t*y), v(s*(x+1), t*(y+1));

[0123] (x, y) represents a pixel in the third image, v(p, q) represents the movement amount corresponding to the pixel point (p, q) in the third group of pixel points, and the first group of pixel points includes pixel points (s*x, t*y), (s*x, t(y+1)), (s*(x+1), t*y), (s*(x+1), t*(y+1));

[0124] w1, w2, w3, w4 are preset weight values.

[0125] Optionally, in this embodiment, the display parameter value of the pixel point (p, q) in the second image may be determined by the following formula:

[0126] I(p, q)=w1I(s*x,t*y)+w2I(s*x,t(y+1))+w3I(s*(x+1),t*y)+w4I(s*(x+1),t*(y+1))

[0127] in,

[0128] I(p, q) represents the display parameter value of the pixel point (p, q) in the second image,

[0129] I(s*x, t*y) represents the display parameter value of the pixel point (s*x, t*y) in the second image,

[0130] I(s*x, t(y+1)) represents the display parameter value of the pixel point (s*x, t(y+1)) in the second image,

[0131] I(s*(x+1), t*y) represents the display parameter value of the pixel point (s*(x+1), t*y) in the second image,

[0132] I(s*(x+1), t*(y+1)) represents the display parameter value of the pixel point (s*(x+1), t*(y+1)) in the second image.

[0133] It should be noted that, in this embodiment, the third group of movement amounts corresponding to the second group of pixels in the first image except the first group of pixels are calculated by interpolation, thereby determining the display parameters of all pixels in the first image. The above display parameters may include but are not limited to the brightness of the pixels.

[0134] Optionally, moving each pixel point in the first image to a corresponding pixel point in the second image according to the first group of movement amounts and the second group of movement amounts may include:

[0135] The following steps are performed on each pixel point in the first image to obtain a second image, wherein each pixel point in the first image is regarded as a third current point when the following steps are performed:

[0136] Acquire a third current position of a third current point in the first image, and a third current movement amount corresponding to the third current position in the second group of movement amounts and the third group of movement amounts;

[0137] The third current position is moved by a third current movement amount to obtain a fourth current position of a corresponding fourth current point in the second image, wherein the fourth current point is a pixel point moved from the third current point to the second image.

[0138] Optionally, as an optional implementation, Figure 4 As shown, a flowchart of the large-scale face deformation method based on deformation field.

[0139] like Figure 4As shown, the core idea of ​​the large-scale face deformation method based on deformation field is: determine the reduction ratio of the first image to be deformed, scale the first image to be deformed according to the preset reduction ratio to obtain a third image, determine the small deformation field corresponding to the third image based on the third image, and then determine the large deformation field corresponding to the first image based on the small deformation field, and then determine the deformed image of the first image according to the large deformation field.

[0140] Assuming that the width and height of the first image are W and H, to generate a two-dimensional deformation field (x, y) of size W×H, the first image is reduced by s in the W direction and t in the H direction according to the preset ratio, where (s>1, t>1), and the small size of the third image is obtained. The radial basis function is constructed according to the third image to determine the small deformation field corresponding to the third image, that is, for any point in the small-scale deformation field Find the deformation at this point A small-scale deformation field can be obtained. Since the deformation of each point in the small-scale deformation field corresponds to the large-scale one, the large-scale deformation field can be obtained by mapping and interpolating the small-scale deformation field. The deformation is v(x, y), which corresponds to the deformation on the large-scale deformation field (s*x, t*y), such as Figure 3 shown.

[0141] like Figure 5 As shown, the flowchart of large-scale face deformation based on deformation field.

[0142] Step S501, start;

[0143] Step S502, obtaining an original face image;

[0144] In step S502, the original face image is equivalent to the first image.

[0145] Step S503, setting control points and moving control points;

[0146] In step S503, the setting control point is equivalent to the feature point corresponding to the first group of positions, and the moving control point is equivalent to the feature point corresponding to the third position.

[0147] Step S504, fast smoothing of deformation field based on small-scale mapping;

[0148] In step S504, the original face image is first reduced according to a preset ratio to obtain a third image of a small size, and the small deformation field corresponding to the third image is obtained by solving. Since there is a preset ratio between the original image and the third image, the large deformation field corresponding to the original face can be restored according to the preset ratio, and then the deformed image of the original image is obtained. It should be noted that when determining the large deformation field corresponding to the original image, in order to obtain an accurate large deformation field, the deformation amount of each pixel in the original image can be determined by an interpolation algorithm. Then, based on the deformation amount and the original position in the original image, the position of the pixel after the change is determined to obtain the deformed image.

[0149] In this embodiment, the large deformation field of the original image is determined based on the deformation field determined by the small-size image, and the calculation of the deformation field is based on the small-size calculation, and the calculation speed is relatively fast. Based on the small-scale deformation field mapping, the smooth deformation field of the original image is obtained quickly.

[0150] Step S505, transmitting the deformation field to the image processor;

[0151] In step S505, the large deformation field determined based on the small deformation field is transmitted to the image processor.

[0152] Step S506, the image processor performs interpolation deformation;

[0153] In step S506, the two-dimensional deformation field consistent with the image size obtained on the CPU is transmitted to the graphics processing unit (GPU). Assuming that the original pixel coordinates of a point P on the image are (x0, y0), the corresponding deformation amount on the deformation field is (Δx, Δy), and assuming that the brightness of a point (x, y) on the image is I(x, y), then the deformed point P is P′(x′, y′).

[0154] x′=x0+Δx

[0155] y′=y0+Δy

[0156] Then in a region of (x′, y′), assuming that the nearest integer pixel is (x1, y1), bilinear interpolation is used to obtain the deformed brightness:

[0157] I(x′,y′)=w1I(x1,y1)+W2I(x1,y1+1)+w3I(x1+1,y1)+w4I(x1+1,y1+1)

[0158] In this way, each pixel of the original image can obtain the deformed position and the corresponding color value, and at the same time, a smooth image of each pixel in the image after deformation can be obtained quickly and in parallel.

[0159] Step S507, file rendering;

[0160] In step S507, the file is rendered using OpenGL (Open Graphics Library, referred to as OpenGL) to obtain a deformed image.

[0161] Step S508, end.

[0162] When the third image is a face image, determining the deformation field of the third image is described as follows.

[0163] The facial feature points are extracted through the face algorithm, and some facial feature points are selected as control points. Some control points can also be set through user interaction. Then, a small-scale deformation field is constructed based on the radial basis and a smooth regularization term is added. The specific instructions are as follows.

[0164] Pre-select N control points, and change these N control points to affect the position of each other point in the image. Suppose the original position of these N control points is u i , i = 0, 1, ... n, the position of these N control points after change is v i , then the change of N control points M f (u i )=v i -u i , i = 1, 2, ... n, using radial basis function to fit M f (u i ), which is:

[0165]

[0166] Where m = 1, is a linear basis, α0+α1u, m=2, a quadratic function, that is, α0+α1u+α2u 2 , usually m is a low-order polynomial, that is, m <n, It can also be replaced by an affine transformation basis, which can provide more variations for the radial basis function. In this embodiment, m=1, is a linear basis, and some fixed points before and after deformation are preset in advance, assuming that {(u i , v i )} i＝0，1，…N , thus obtaining the following set of equations:

[0167]

[0168] It should be noted that, in this embodiment, among the N points (N≥n+m), n points are used to construct constraint points of the radial basis, and the other points are used to solve m variables.

[0169] The above formula (2) can be simplified as:

[0170] φ(U)·W=V (3)

[0171] in,

[0172]

[0173] It can be expressed as a matrix of N×(n+m),

[0174] It can be represented as a (n+m)×1 matrix, It can be represented as an N×1 matrix.

[0175] In this embodiment, in order to provide the first weight w i To ensure the robustness and smoothness of the deformation field, a regularization term is constructed.

[0176]

[0177] R is the regularization matrix. By taking the derivative, we get the following formula (4):

[0178] (φ T φ+λR T R)W=φ T V (4)

[0179] The solution to equation (4) above is:

[0180] W reg =(φ T φ+λR T R) -1 Φ T V

[0181] Perform matrix singular decomposition SVD decomposition on φ and get

[0182] φ=X∑Y T

[0183] Where X = (x1, ...x m ) and Y=(y1,…y n ) is an orthogonal matrix, where x i ,y i are respectively m-dimensional vector and n-dimensional vector Σ=diag(σ1,…σ n ),σ1≥σ2≥…σ n >0, if R=0, we can get:

[0184]

[0185] From this we can see that some singular values ​​are very small, which will lead to There are some small disturbances that will cause W reg The solution error is large and not smooth enough. Here we set R = λI n ,thereby

[0186]

[0187] In this embodiment, not only a stable solution can be obtained, but also It also produces a smooth weight W reg , thus obtaining a smooth deformation field, the obtained deformation field is to obtain the deformed position v of each pixel point, then:

[0188]

[0189] Furthermore, by establishing a mapping rule between the small-scale deformation field and the large-scale deformation field, a large-scale deformation field that is not affected by the point space structure and maintains smooth deformation is quickly generated. By transferring the deformation field to the image processor GPU, parallel interpolation and dense mapping are used to generate a deformed image to obtain the deformation result.

[0190] The two-dimensional deformation field with the same size as the image obtained on the CPU is transferred to the GPU. Assuming that the original pixel coordinates of a point P on the image are (x0, y0), the corresponding deformation amount on the deformation field is (Δx, Δy). Assuming that the brightness of a point (x, y) on the image is I(x, y), then the point P after deformation is P′(x′, y′),

[0191] x′=x0+Δx

[0192] y′=y0+Δy

[0193] Then in a region of (x′, y′), assuming that the nearest integer pixel is (x1, y1), bilinear interpolation is used to obtain the deformed brightness:

[0194] I(x′,y′)=w1I(x1,y1)+w2I(x1,y1+1)+w3I(x1+1,y1)+w4I(x1+1,y1+1)

[0195] In this way, each pixel of the original image can obtain the deformed position and the corresponding color value, and at the same time, a smooth image of each pixel in the image after deformation can be obtained quickly and in parallel.

[0196] Through the solution of this embodiment, various natural smoothness and large-scale deformation effects can be provided for face special effects products. At the same time, face deformation provides a key basic technology for subsequent face animation, face fusion, or face replacement applications.

[0197] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0198] According to another aspect of the embodiments of the present invention, an image processing device for implementing the above-mentioned image processing method is also provided. Figure 6 As shown, the image processing device includes: a first acquiring unit 61 , a first obtaining unit 63 , a first determining unit 65 , a second determining unit 67 and a second obtaining unit 69 .

[0199] The first acquisition unit 61 is used to acquire positions of a first group of feature points in the first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions.

[0200] The first obtaining unit 63 is used to move the first group of feature points to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions.

[0201] The first determination unit 65 is used to determine radial basis fitting parameters based on the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions and the second group of position values, wherein the radial basis fitting parameters are used to fit the target deformation function of the first image, and the third group of positions are positions selected from the first group of positions.

[0202] The first determination unit 67 is used to determine a target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine a movement amount corresponding to the input position according to the input position.

[0203] The second obtaining unit 69 is used to move each pixel point in the first image to a corresponding pixel point in the second image according to the target deformation function, so as to obtain a deformed second image.

[0204] Through the embodiment provided by the present application, the first acquisition unit 61 acquires the positions of the first group of feature points in the first image to be deformed, wherein the positions of the first group of feature points constitute the first group of positions; the first obtaining unit 63 moves the first group of feature points to obtain the second group of feature points, wherein the positions of the second group of feature points constitute the second group of positions; the first determining unit 65 determines the radial basis fitting parameters according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions and the second group of position values, wherein the radial basis fitting parameters are used to fit the target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; the first determining unit 67 determines the target deformation according to the radial basis fitting parameters. Function, wherein the target deformation function is used to determine the movement amount corresponding to the input position according to the input position; the second obtaining unit 69 moves each pixel point in the first image to the corresponding pixel point in the second image according to the target deformation function, and obtains the deformed second image, thereby achieving the goal of constructing a deformation field of the image based on the radial basis function according to the first group of positions and the second group of positions of a group of feature points, and generating a deformed image by transmitting the deformation field to the graphics processor to obtain a smooth deformation result. Since it is not constrained by the point space structure, a smooth deformation effect in a large range is generated, thereby solving the technical problems in the prior art that the spatial structure of the points is determined by the grid, arbitrary large-scale deformation cannot be performed, and the smoothness of the image deformation is low.

[0205] Optionally, the first determining unit 65 may include:

[0206] The first acquisition module is used to acquire the position offset between each position in the first group of positions and each position in the third group of positions, so as to obtain a group of position offsets in total.

[0207] The first determination module is used to determine a first group of weight values ​​and a second group of weight values ​​according to a group of position offsets, a first group of positions, and a second group of positions, wherein the radial basis fitting parameters include the first group of weight values ​​and the second group of weight values.

[0208] The first determining module may include:

[0209] A first determination submodule, used to determine a target matrix according to a set of position offsets and a first set of positions;

[0210] A decomposition submodule, used for decomposing the target matrix to obtain a first matrix and a second matrix;

[0211] The second determination submodule is used to determine the first group of weight values ​​and the second group of weight values ​​according to the first matrix, the second matrix and the second group of positions.

[0212] It should be noted that the above decomposition submodule is also used to perform the following operations:

[0213] The target matrix is ​​decomposed by the following formula to obtain the first matrix and the second matrix:

[0214] φ(U)=X∑Y T

[0215] The target matrix is:

[0216]

[0217] Where X represents the first matrix, Y represents the second matrix, ∑ represents an N×P matrix, in which all elements except the elements on the main diagonal are 0, and the elements on the main diagonal include Λ 11 , Λ 22 ,…,Λ PP , P=n+m, N represents the number of positions in the first group of positions, n represents the number of positions in the third group of positions, N≥n+m, N, n and m are all preset natural numbers,

[0218]

[0219] φ(u j -u i ) represents the jth position u in the first group of positions j and the i-th position u in the third group of positions i The offset between them, the value of j is 1, 2...N, the value of i is 1, 2...n, The value of s is 1, 2, …m.

[0220] It should also be noted that the second determining submodule is also used to perform the following operations:

[0221]

[0222]

[0223] Among them, x i represents the i-th column vector in the first matrix, y i represents the i-th column vector in the second matrix, The second set of positions includes v1, v2, ..., v N , The first set of weight values ​​includes w1, w2, ..., w n , the second set of weight values ​​includes α1, α2, ..., α m , σ i The elements on the main diagonal are Λ 11 , Λ 22 , ..., Λ PP The i-th value Λ inii , λ is a preset constant.

[0224] Optionally, the second determining unit 67 may include: a third determining submodule, configured to determine the following function as the target deformation function:

[0225]

[0226] The first set of weight values ​​includes w1, w2, ..., w n , the second set of weight values ​​includes α1, α2, ..., α m , n represents the number of positions in the third group of positions, n and m are both preset natural numbers, w i represents the i-th weight value in the first set of weight values, α j represents the jth weight value in the second set of weight values, u i represents the i-th position in the third group of positions, represents the position offset between the target position u in the first image and the i-th position, M f (u) represents the movement amount corresponding to the target position u,

[0227] Optionally, the second obtaining unit 69 may include:

[0228] The following steps are performed on each pixel point in the first image to obtain a second image, wherein each pixel point in the first image is regarded as a first current point when the following steps are performed:

[0229] A second acquisition module, used for acquiring a first current position of a first current point in the first image;

[0230] A third acquisition module is used to acquire a current movement amount corresponding to the position of the first current point output by the target deformation function when the first current position is the input position of the target deformation function;

[0231] The second determination module is used to determine a second current position of a corresponding second current point in the second image according to the current movement amount and the first current position, wherein the second current point is a pixel point moved from the first current point to the second image.

[0232] The second determination module may include: a fourth determination submodule, configured to determine the second current position by the following formula:

[0233] x′=x0+Δx

[0234] y′=y0+Δy

[0235] Among them, (x′, y′) represents the second current position of the second current point P′, (x0, y0) represents the first current position of the first current point P, and (Δx, Δy) represents the current movement amount.

[0236] Optionally, the above-mentioned second obtaining unit 69 may include: a scaling module, used to reduce the first image to a third image according to a preset ratio; a fourth acquisition module, used to obtain the movement amount corresponding to the position of each pixel point in the third image according to the target deformation function, wherein the movement amount corresponding to the position of each pixel point in the third image constitutes a first group of movement amounts; a moving module, used to move each pixel point in the first image to a corresponding pixel point in the second image according to the first group of movement amounts and a preset ratio.

[0237] The mobile module may include:

[0238] a restoration submodule, configured to restore the pixels in the third image to the pixels in the first image according to a preset ratio, wherein the pixels in the first image restored constitute a first group of pixels, and the second group of movement amounts corresponding to the first group of pixels are determined according to the first group of movement amounts and the preset ratio;

[0239] a fifth determination submodule, configured to determine a third group of movement amounts corresponding to a second group of pixel points other than the first group of pixel points in the first image according to the second group of movement amounts and the first group of pixel points, wherein the first group of pixel points and the second group of pixel points constitute the first image;

[0240] The moving submodule is used to move each pixel point in the first image to a corresponding pixel point in the second image according to the second group of movement amounts and the third group of movement amounts.

[0241] It should be noted that the fifth determination submodule is further used to perform the following operation: determine a third group of movement amounts corresponding to a second group of pixel points other than the first group of pixel points in the first image by using the following formula:

[0242] v(p, q)=w1v(s*x,t*y)+w2v(s*x,t(y+1))+w3v(s*(x+1),t*y)+w4v(s*(x+1),t*(y+1))

[0243] Among them, s*x≤p≤s*(x+1), t*y≤q≤s*(y+1),

[0244] The size of the first image is W×H, and the size of the third image is s>1, t>1, the second set of movement amounts includes: v(s*x, t*y), v(s*x, t(y+1)), v(s*(x+1), t*y), v(s*(x+1), t*(y+1));

[0245] (x, y) represents a pixel in the third image, v(p, q) represents the movement amount corresponding to the pixel point (p, q) in the third group of pixel points, and the first group of pixel points includes pixel points (s*x, t*y), (s*x, t(y+1)), (s*(x+1), t*y), (s*(x+1), t*(y+1));

[0246] w1, w2, w3, w4 are preset weight values.

[0247] Optionally, the above-mentioned image processing device may further include: a sixth determination submodule, configured to determine the display parameter value of the pixel point (p, q) in the second image by using the following formula:

[0248] I(p, q)=w1I(s*x,t*y)+w2I(s*x,t(y+1))+w3I(s*(x+1),t*y)+w4I(s*(x+1),t*(y+1))

[0249] in,

[0250] I(p, q) represents the display parameter value of the pixel point (p, q) in the second image,

[0251] I(s*x, t*y) represents the display parameter value of the pixel point (s*x, t*y) in the second image,

[0252] I(s*x, t(y+1)) represents the display parameter value of the pixel point (s*x, t(y+1)) in the second image,

[0253] I(s*(x+1), t*y) represents the display parameter value of the pixel point (s*(x+1), t*y) in the second image,

[0254] I(s*(x+1), t*(y+1)) represents the display parameter value of the pixel point (s*(x+1), t*(y+1)) in the second image.

[0255] It should be noted that the above-mentioned moving submodule is also used to perform the following operations:

[0256] The following steps are performed on each pixel point in the first image to obtain a second image, wherein each pixel point in the first image is regarded as a third current point when the following steps are performed:

[0257] Acquire a third current position of a third current point in the first image, and a third current movement amount corresponding to the third current position in the second group of movement amounts and the third group of movement amounts;

[0258] The third current position is moved by a third current movement amount to obtain a fourth current position of a corresponding fourth current point in the second image, wherein the fourth current point is a pixel point moved from the third current point to the second image.

[0259] According to another aspect of the embodiments of the present invention, an electronic device for implementing the above-mentioned image processing method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a server as an example. Figure 7 As shown, the electronic device includes a memory 702 and a processor 704. The memory 702 stores a computer program, and the processor 704 is configured to execute the steps in any of the above method embodiments through the computer program.

[0260] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0261] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0262] S1, obtaining positions of a first group of feature points in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions;

[0263] S2, moving the first group of feature points to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions;

[0264] S3, determining radial basis fitting parameters according to an offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions;

[0265] S4, determining a target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine a movement amount corresponding to the input position according to the input position;

[0266] S5, according to the target deformation function, move each pixel point in the first image to a corresponding pixel point in the second image to obtain a deformed second image.

[0267] Alternatively, a person skilled in the art may understand that: Figure 7 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices. Figure 7 The electronic device and the electronic equipment described above are not limited in structure. Figure 7 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 7 Different configurations are shown.

[0268] Among them, the memory 702 can be used to store software programs and modules, such as program instructions / modules corresponding to the image processing method and device in the embodiment of the present invention. The processor 704 executes various functional applications and data processing by running the software programs and modules stored in the memory 702, that is, realizing the above-mentioned image processing method. The memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 702 may further include a memory remotely arranged relative to the processor 704, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 702 can be specifically but not limited to being used to store information such as a first image, a position of a first group of feature points, a first group of positions, a second group of feature points, a second group of positions, and a second image. As an example, if Figure 7 As shown, the memory 702 may include, but is not limited to, the first acquisition unit 61, the first obtaining unit 63, the first determination unit 65, the second determination unit 67, and the second obtaining unit 69 in the image processing device. In addition, other module units in the image processing device may also be included but are not limited to, which will not be repeated in this example.

[0269] Optionally, the transmission device 706 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 706 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 706 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0270] In addition, the electronic device further includes: a display 708 for displaying the first image and the second image; and a connection bus 710 for connecting various module components in the electronic device.

[0271] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, terminal and other electronic devices, may become a node in the blockchain system by joining the peer-to-peer network.

[0272] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image processing method provided in the above-mentioned image processing aspect or various optional implementations of the image processing aspect. The computer program is configured to execute the steps of any of the above-mentioned method embodiments when it is run.

[0273] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0274] S1, obtaining positions of a first group of feature points in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions;

[0275] S2, moving the first group of feature points to obtain a second group of feature points, wherein the positions of the second group of feature points constitute a second group of positions;

[0276] S3, determining radial basis fitting parameters according to an offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions;

[0277] S4, determining a target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine a movement amount corresponding to the input position according to the input position;

[0278] S5, according to the target deformation function, move each pixel point in the first image to a corresponding pixel point in the second image to obtain a deformed second image.

[0279] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0280] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0281] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0282] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0283] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0284] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0285] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0286] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire positions of a first group of feature points in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions; Moving the first group of feature points to obtain a second group of feature points, wherein positions of the second group of feature points constitute a second group of positions; determining radial basis fitting parameters according to an offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; Determining the target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine the movement amount corresponding to the input position according to the input position; According to the target deformation function, each pixel point in the first image is moved to a corresponding pixel point in the second image to obtain the deformed second image.

2. The method according to claim 1, characterized in that Determining the radial basis fitting parameters according to the offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values ​​comprises: Obtaining a position offset between each position in the first group of positions and each position in the third group of positions, to obtain a group of position offsets; A first group of weight values ​​and a second group of weight values ​​are determined according to the group of position offsets, the first group of positions, and the second group of positions, wherein the radial basis fitting parameters include the first group of weight values ​​and the second group of weight values.

3. The method according to claim 2, characterized in that The determining the first group of weight values ​​and the second group of weight values ​​according to the group of position offsets, the first group of positions, and the second group of positions includes: determining a target matrix based on the set of position offsets and the first set of positions; Decomposing the target matrix to obtain a first matrix and a second matrix; The first group of weight values ​​and the second group of weight values ​​are determined according to the first matrix, the second matrix, and the second group of positions.

4. The method according to claim 3, characterized in that The step of decomposing the target matrix to obtain the first matrix and the second matrix includes: The target matrix is ​​decomposed by the following formula to obtain the first matrix and the second matrix: φ(U)=X∑Y T Among them, the target matrix is: Wherein, U represents a position set including the first group of positions and the third group of positions, X represents the first matrix, Y represents the second matrix, ∑ represents an N×P matrix, in which all elements except the elements on the main diagonal are 0, and the elements on the main diagonal include Λ 11 ,Λ 22 ,…,Λ PP , P=n+m, N represents the number of positions in the first group of positions, n represents the number of positions in the third group of positions, N≥n+m, N, n and m are all preset natural numbers, φ(u j -u i ) represents the jth position u in the first group of positions j and the i-th position u in the third group of positions i The offset between them, the value of j is 1, 2...N, the value of i is 1, 2...n, The value of s is 1, 2, …m.

5. The method according to claim 4, characterized in that The determining the first group of weight values ​​and the second group of weight values ​​according to the first matrix, the second matrix and the second group of positions includes: Among them, x i represents the i-th column vector in the first matrix, y i represents the i-th column vector in the second matrix, The second group of positions includes v1, v2, ..., v N , The first set of weight values ​​includes w1, w2, ..., w n , the second set of weight values ​​includes α1, α2, …, α m , σ i is the element Λ on the main diagonal 11 ,Λ 22 ,…,Λ PP The i-th value Λ in ii , λ is a preset constant.

6. The method according to claim 2, characterized in that The step of determining the target deformation function according to the radial basis fitting parameters comprises: The following function is determined as the target deformation function: The first set of weight values ​​includes w1, w2, ..., w n , the second set of weight values ​​includes α1, α2, …, α m , n represents the number of positions in the third group of positions, n and m are both preset natural numbers, w i represents the i-th weight value in the first set of weight values, α j represents the jth weight value in the second set of weight values, u i represents the i-th position in the third set of positions, represents the position offset between the target position u in the first image and the i-th position, the M f (u) represents the movement amount corresponding to the target position u, 7. The method according to claim 1, characterized in that The step of moving each pixel point in the first image to a corresponding pixel point in the second image according to the target deformation function to obtain the deformed second image includes: The second image is obtained by performing the following steps on each pixel point in the first image, wherein each pixel point in the first image is regarded as a first current point when the following steps are performed: Obtaining a first current position of the first current point in the first image; When the first current position is the input position of the target deformation function, obtaining a current movement amount output by the target deformation function corresponding to the position of the first current point; A second current position of a second current point corresponding to the second image is determined according to the current movement amount and the first current position, wherein the second current point is a pixel point moved from the first current point to the second image.

8. The method according to claim 7, characterized in that The determining, according to the current movement amount and the first current position, a second current position of a corresponding second current point in the second image includes: The second current position is determined by the following formula: x ′ =x0+Δx y ′ =y0+Δy Among them, (x ′ ,y ′ ) represents the second current point P ′ , (x0, y0) represents the first current position of the first current point P, and (Δx, Δy) represents the current movement amount.

9. The method according to claim 1, characterized in that: The step of moving each pixel point in the first image to a corresponding pixel point in the second image according to the target deformation function to obtain the deformed second image includes: reducing the first image into a third image according to a preset ratio; Acquire a movement amount corresponding to the position of each pixel point in the third image according to the target deformation function, wherein the movement amount corresponding to the position of each pixel point in the third image constitutes a first group of movement amounts; According to the first group of movement amounts and the preset ratio, each pixel point in the first image is moved to a corresponding pixel point in the second image.

10. The method according to claim 9, characterized in that The step of moving each pixel point in the first image to a corresponding pixel point in the second image according to the first group of movement amounts and the preset ratio includes: Restoring the pixels in the third image to the pixels in the first image according to the preset ratio, wherein the pixels in the first image restored to form a first group of pixels, and the second group of movement amounts corresponding to the first group of pixels are determined according to the first group of movement amounts and the preset ratio; Determine a third group of movement amounts corresponding to a second group of pixel points other than the first group of pixel points in the first image according to the second group of movement amounts and the first group of pixel points, wherein the first group of pixel points and the second group of pixel points constitute the first image; According to the second group of movement amounts and the third group of movement amounts, each pixel point in the first image is moved to a corresponding pixel point in the second image.

11. The method according to claim 10, characterized in that The determining, according to the second group of movement amounts and the first group of pixel points, a third group of movement amounts corresponding to a second group of pixel points in the first image except the first group of pixel points, comprises: The third group of movement amounts corresponding to the second group of pixel points other than the first group of pixel points in the first image are determined by the following formula: v(p,q)=w1v(s*x,t*y)+w2v(s*x,t(y+1)) +w3v(s*(x+1),t*y)+w4v(s*(x+1),t*(y+1))where, s*x≤p≤s*(x+1),t*y≤q≤s*(y+1) The size of the first image is W×H, and the size of the third image is s>1,t>1, the second group of movement amounts includes: v(s*x,t*y), v(s*x,t(y+1)), v(s*(x+1),t*y), v(s*(x+1),t*(y+1)); (x, y) represents a pixel point in the third image, v(p,q) represents the movement amount corresponding to the pixel point (p,q) in the third group of pixel points, the first group of pixel points including pixel points (s*x,t*y), (s*x,t(y+1)), (s*(x+1),t*y), (s*(x+1),t*(y+1)); w1, w2, w3, w4 are preset weight values.

12. The method according to claim 11, characterized in that The method further comprises: The display parameter value of the pixel point (p, q) in the second image is determined by the following formula: I(p,q)=w1I(s*x,t*y)+w2I(s*x,t(y+1))+w3I(s*(x+1),t*y) +w4I(s*(x+1),t*(y+1)) in, I(p,q) represents the display parameter value of the pixel point (p,q) in the second image, I(s*x, t*y) represents the display parameter value of the pixel point (s*x, t*y) in the second image, I(s*x,t(y+1)) represents the display parameter value of the pixel point (s*x,t(y+1)) in the second image, I(s*(x+1), t*y) represents the display parameter value of the pixel point (s*(x+1), t*y) in the second image, I(s*(x+1), t*(y+1)) represents the display parameter value of the pixel point (s*(x+1), t*(y+1)) in the second image.

13. The method according to claim 11, characterized in that The step of moving each pixel point in the first image to a corresponding pixel point in the second image according to the first group of movement amounts and the second group of movement amounts includes: The second image is obtained by performing the following steps on each pixel point in the first image, wherein each pixel point in the first image is regarded as a third current point when the following steps are performed: Acquire a third current position of the third current point in the first image, and a third current movement amount corresponding to the third current position in the second group of movement amounts and the third group of movement amounts; The third current position is moved by the third current movement amount to obtain a fourth current position of a corresponding fourth current point in the second image, wherein the fourth current point is a pixel point in the second image where the third current point is moved.

14. An image processing device, characterized in that: include: A first acquisition unit, configured to acquire positions of a first group of feature points in a first image to be deformed, wherein the positions of the first group of feature points constitute a first group of positions; A first obtaining unit is used to move the first group of feature points to obtain a second group of feature points, wherein positions of the second group of feature points constitute a second group of positions; a first determining unit, configured to determine radial basis fitting parameters according to an offset between each position in the first group of positions and each position in the third group of positions, the first group of positions, and the second group of position values, wherein the radial basis fitting parameters are used to fit a target deformation function of the first image, and the third group of positions are positions selected from the first group of positions; a second determining unit, configured to determine the target deformation function according to the radial basis fitting parameters, wherein the target deformation function is used to determine a movement amount corresponding to the input position according to the input position; The second obtaining unit is used to move each pixel point in the first image to a corresponding pixel point in the second image according to the target deformation function, so as to obtain the deformed second image.

15. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 13 through the computer program.

Images