Image deformation method, device and electronic equipment

Abstract

The embodiment of the present application discloses an image deformation method, device and electronic device, the method comprising: determining an original image, and identifying the main content of the image to be deformed from the original image; dividing the main content of the image into a plurality of initial polygonal image blocks, and performing boundary expansion processing on the initial polygonal image blocks to obtain a plurality of expanded polygonal image blocks; using a first transformation model, mapping the vertices of the expanded polygonal image blocks to the deformed target image; and using a second transformation model, mapping the pixel points in the expanded polygonal image blocks to the target image according to the position correspondence information of the vertices before and after mapping. Through the embodiment of the present application, image deformation can be achieved more efficiently, and the appearance of gaps in the new image can be avoided.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image deformation method, device and electronic device. Background Art

[0002] In the product object information system, the characteristics of the product object can be conveyed to the user through the dynamic attribute expression of the product object. For example, for a product object such as shoes, the characteristics of the sole being soft and having good curvature can be expressed through animated images.

[0003] Specific dynamic attribute information can be realized by shooting videos, etc., but it requires merchants and users to spend relatively high costs. In order to save costs, algorithms can be used to process the original static images and generate dynamic images, and then the dynamic attributes can be expressed through the generated dynamic images.

[0004] In the above-mentioned process of generating the animated image, if it is necessary to reflect the soft and bendable characteristics of the sole, it is necessary to continuously deform the original image for multiple frames, and then compose the generated multiple frames of target images into an animated image. There are some image deformation processing schemes in the prior art. For example, one of the schemes is to achieve the above-mentioned purpose of deforming the image by using the moving least squares method. In this scheme, you can first select multiple anchor points in the original image, and set these anchor points at the desired positions in multiple frames of target images. Then, for each frame of the target image, you can use the moving least squares method to calculate the positions of each pixel in the original image in the target image, and then project each pixel value into the target image according to the specific position.

[0005] The above method can achieve the effect of smooth deformation of the image, but its disadvantage is that, on the one hand, it is necessary to calculate each pixel one by one and calculate the position of each pixel in the new image respectively, so the efficiency is often low. On the other hand, after the deformation process is completed, the pixels in some areas of the new image may become denser, and the image in some areas may become sparser. Among them, for areas where the pixels become sparse, gaps may appear between the pixels. Such gaps are manifested as: in some areas of the new image, it can be seen that there are multiple pixels, and there are gaps between each pixel. For example, the specific position of the gap may be displayed as a background color such as black, which obviously affects the display effect of the image.

[0006] Therefore, how to achieve image deformation more efficiently and avoid the appearance of gaps in the new image has become a technical problem that needs to be solved by those skilled in the art. Summary of the invention

[0007] The present application provides an image deformation method, device and electronic device, which can achieve image deformation more efficiently and avoid the appearance of gaps in the new image.

[0008] This application provides the following solutions:

[0009] An image deformation method, comprising:

[0010] Determine an original image, and identify the main content of the image to be deformed from the original image;

[0011] Dividing the main content of the image into a plurality of initial polygonal image blocks, and performing boundary expansion processing on the initial polygonal image blocks to obtain a plurality of expanded edge polygonal image blocks;

[0012] Using a first transformation model, mapping the vertices of the expanded polygonal image block to a deformed target image;

[0013] According to the position correspondence information of the vertices before and after mapping, the pixel points in the expanded polygonal image block are mapped to the target image using a second transformation model.

[0014] The step of dividing the main content of the image into a plurality of initial polygonal image blocks comprises:

[0015] The main content of the image is divided into a plurality of initial triangular image blocks, and the initial triangular image blocks are subjected to boundary expansion processing to obtain a plurality of expanded triangular image blocks.

[0016] The step of dividing the main content of the image into a plurality of initial triangular image blocks includes:

[0017] Performing grid division on the original image;

[0018] Traverse all meshes, retain mesh vertices containing the main content of the image and perform deduplication processing;

[0019] A preset subdivision algorithm is called according to the retained mesh vertices to obtain the multiple triangular image blocks.

[0020] The step of performing boundary expansion processing on the triangular image block to obtain a plurality of expanded triangular image blocks includes:

[0021] For each vertex of the triangular image block, the following processing is performed:

[0022] Select multiple candidate points around the vertex according to the target distance to form a candidate point set;

[0023] Using the two edges where the vertices are located as dividing lines respectively, a first candidate point subset and a second candidate point subset are selected from the candidate point set, wherein when selecting the candidate point subset using one of the edges as the dividing line, the vertex opposite to the edge is used as a reference vertex, and candidate points on the opposite side of the reference vertex are selected to form a candidate point subset;

[0024] Determine a replacement point for a vertex according to an intersection of the first candidate point subset and the second candidate point subset;

[0025] The replacement points of each vertex are connected to obtain the expanded triangle image block.

[0026] The size of the target distance is related to the distance that different image blocks may be pulled apart during the mapping process.

[0027] Wherein, the second transformation model comprises an affine transformation model;

[0028] The step of mapping the pixel points in the expanded polygonal image block to the target image by using the second transformation model includes:

[0029] Determine three groups of mapping points according to the position correspondence information of the vertices of the expanded triangle image block before and after mapping;

[0030] Based on the three groups of mapping points, the pixel points in the expanded triangle image block are mapped into the target image using an affine transformation model.

[0031] Wherein, the first transformation model comprises a moving least squares transformation model;

[0032] The method of mapping the vertices of the expanded polygonal image block to the deformed target image by using the first transformation model includes:

[0033] Based on the positions of multiple anchor points set in the original image and the expected positions corresponding to the anchor points in the target image to be generated, a moving least squares fitting function is established;

[0034] The vertices are mapped to the target image to be generated by using the moving least squares fitting function to determine the position mapping result of the vertices in the target image to be generated.

[0035] Among them, it also includes:

[0036] If there are multiple pixel points mapped to the same position in the target image, a pixel point with lower transparency is selected from the multiple pixel points and mapped to the position.

[0037] A method for generating a dynamic image for a product object, comprising:

[0038] Determine the target product object for which the animated image needs to be generated and the corresponding original image, and identify the main content of the image to be deformed from the original image;

[0039] Dividing the main content of the image into a plurality of original polygonal image blocks, and performing boundary expansion processing on the original polygonal image blocks to obtain a plurality of expanded polygonal image blocks;

[0040] Determining the positions of multiple anchor points from the main content of the image, and the expected positions of each anchor point in multiple frames of target images;

[0041] For the multiple frames of target images, the following processing is performed respectively:

[0042] For the current target image, a first transformation model is established according to the positions of the plurality of anchor points and the expected position, the vertices of the expanded polygonal image block are mapped to the current target image using the first transformation model, and according to the position correspondence information of the vertices before and after mapping, the pixel points in the expanded polygonal image block are mapped to the current target image using a second transformation model to generate the current target image;

[0043] A dynamic image of the target commodity object is generated according to the generated multiple frames of target images.

[0044] The step of determining the target commodity object for which the animated image needs to be generated and the corresponding original image includes:

[0045] Receiving page address information of a target page associated with the target product object;

[0046] The images associated with the target page are analyzed according to the page address information, and an image that meets the target condition is determined as the original image.

[0047] Among them, it also includes:

[0048] Providing an operation control for publishing the animated image;

[0049] After receiving the user operation through the operation control, the generated animated image is published to the target page.

[0050] An image deformation device, comprising:

[0051] An original image determination unit, used to determine an original image and identify the main content of the image to be deformed from the original image;

[0052] An edge expansion processing unit, used for dividing the main content of the image into a plurality of initial polygonal image blocks, and performing boundary expansion processing on the initial polygonal image blocks to obtain a plurality of edge expanded polygonal image blocks;

[0053] A first mapping unit, configured to map the vertices of the expanded polygonal image block to a deformed target image using a first transformation model;

[0054] The second mapping unit is used to map the pixel points in the expanded polygonal image block to the target image using a second transformation model according to the position correspondence information of the vertices before and after the mapping.

[0055] A device for generating a dynamic image for a commodity object, comprising:

[0056] The original image determination unit is used to determine the target commodity object for which the animated image needs to be generated and the corresponding original image, and to identify the main content of the image to be deformed from the original image;

[0057] An edge expansion processing unit, used for dividing the main content of the image into a plurality of original polygonal image blocks, and performing boundary expansion processing on the original polygonal image blocks to obtain a plurality of edge expanded polygonal image blocks;

[0058] An anchor point position determination unit, used to determine the positions of multiple anchor points from the main content of the image, and the expected positions of each anchor point in multiple frames of target images;

[0059] The target image generation unit is used to perform the following processing on the multiple frames of target images respectively: for the current target image, establish a first transformation model according to the positions of multiple anchor points and the expected position, map the vertices of the expanded polygonal image block to the current target image using the first transformation model, and map the pixel points in the expanded polygonal image block to the current target image using a second transformation model according to the position correspondence information of the vertices before and after mapping, so as to generate the current target image;

[0060] The dynamic image generation unit is used to generate a dynamic image of the target commodity object according to the generated multiple frames of target images.

[0061] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described above.

[0062] An electronic device, comprising:

[0063] one or more processors; and

[0064] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of any of the methods described above.

[0065] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0066] Through the embodiments of the present application, in the process of deforming the image, the main image content in the original image can be divided into multiple initial polygonal image blocks, and each initial polygon can be extended to obtain an extended polygonal image block. After that, the vertices of the extended polygonal image block can be mapped to the target image using the first transformation model. Combined with the second transformation model, the mapping of the pixel points inside the extended polygonal image block to the target image can be achieved, thereby generating a specific target image. In this way, since the polygonal image block is extended before the vertices of the polygonal image block are mapped, overlapping areas are generated between different polygonal image blocks. In this way, even if the distance between different extended polygonal image blocks is pulled apart during the mapping process, the existence of such overlapping areas can be used to make up for the gaps that may be caused by the pulled apart distances, thereby improving the image effect after deformation.

[0067] Specifically, the moving least squares method can be used to map the vertices of the expanded polygon, and the pixel points inside the expanded polygon only need to undergo geometric transformation, including affine transformation, to complete the mapping to the target image. Therefore, the number of moving least squares transformations can be greatly reduced, thereby improving efficiency.

[0068] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1 It is a schematic diagram of the system architecture provided by the embodiment of the present application;

[0071] Figure 2 is a flow chart of the first method provided in an embodiment of the present application;

[0072] Figures 3 to 9 is a schematic diagram of multiple states in the image processing process provided by an embodiment of the present application;

[0073] Fig.10 is a flow chart of the second method provided in an embodiment of the present application;

[0074] Fig.11is a schematic diagram of a first device provided in an embodiment of the present application;

[0075] Fig.12 is a schematic diagram of a second device provided in an embodiment of the present application;

[0076] Fig.13 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0078] First of all, it should be noted that an image is composed of coordinate points with pixel values ​​in a pixel coordinate system. Such coordinate points with pixel values ​​are also called pixel points. Distorting an image is to transform the coordinates of these pixel points, and converting specific pixel points to other positions to generate a new image. In the new image, the effect of distortion can be obtained.

[0079] The moving least square method described in the background technology can achieve the purpose of the above coordinate transformation. However, since all the pixels in the original image need to be calculated separately, that is, the number of moving least square method calculations required is the same as the number of pixels in the original image, this process is relatively time-consuming and therefore has a relatively low efficiency. In addition, there is also a disadvantage of gaps between pixels.

[0080] In order to solve the above problem, one solution is to divide the original image into multiple image blocks, for example, to perform triangulation to obtain multiple triangular image blocks. In this way, the vertices of each triangle can be transformed by moving least squares method, mapped to the target image, and the position mapping results of each vertex in the target image can be obtained. After that, for the pixel points inside the triangle, the mapping to the target image can be completed by affine transformation, and the position mapping results of each pixel point inside the triangle in the target image can be obtained. In this way, the two parts of the position mapping results are combined to generate the final target image.

[0081] Through the above method, since only the moving least squares transformation needs to be performed on the triangle vertices instead of performing the moving least squares transformation on all the pixels separately, the number of calculations of the moving least squares transformation is greatly reduced. For example, assuming that a triangular image block includes 312 pixels, if the moving least squares transformation is performed directly, then only the image block needs to perform 312 moving least squares transformations. In combination with triangulation + affine transformation, for a triangular image block, only 3 moving least squares transformations and one affine transformation need to be performed. The time required for one affine transformation operation is much less than the time required for 309 moving least squares transformations, so the efficiency can be improved overall.

[0082] In addition, in the above-mentioned process of image deformation processing by combining the moving least squares method with triangulation and affine transformation, since it is not necessary to perform the moving least squares transformation on each pixel point separately, there will be no problem of gaps between pixels. However, since the vertices of a specific image block are mapped by the moving least squares transformation, there will be a situation where the distances between some image blocks become closer and the distances between some image blocks become farther. Therefore, gaps may appear between different image blocks, which will still affect the display effect of the generated target image.

[0083] To this end, in the embodiment of the present application, on the basis of combining the moving least squares method with triangulation and affine transformation, further optimization is performed to solve the problem of gaps between different image blocks. Specifically, in the embodiment of the present application, after the original image is divided into a plurality of triangular image blocks (or quadrilaterals, etc.), the triangles can first be processed to expand the boundaries to obtain a plurality of expanded triangles. In this way, in the original image, there may be overlapping parts between different expanded triangles. After that, the moving least squares method transformation can be performed based on the vertices of such expanded triangles, and then the pixel points inside the expanded triangles are affinely transformed, so as to complete the mapping of all pixel points and generate the target image. In this way, since there are overlapping parts between different expanded triangles in the original image, after the transformation, even if the distance between some triangles is pulled away (the distance pulled away during the transformation process is usually less than the distance expanded during the expansion process), gaps between image blocks can be avoided. In addition, the workload of the expansion process is relatively small, so the impact on the processing efficiency is almost negligible.

[0084] Through the above-mentioned method provided in the embodiment of the present application, since the step of edge expansion processing is added, and the moving least squares transformation is performed based on the edge expansion processing result, the quality of the transformed image can be improved while ensuring the processing efficiency, and the appearance of gaps between image blocks can be avoided.

[0085] From the perspective of technical architecture, the embodiments of the present application can have a variety of specific architectures. For example, in one way, an image deformation processing tool can be provided. The tool can be an offline tool and can exist in the form of a client program, etc. The user can install this image deformation processing tool and input the specific image to be processed. The tool can process it and generate a deformed target image, or it can also generate multiple frames of target images and then generate a dynamic image. Afterwards, the user can apply this processed target image or dynamic image to a variety of scenarios, for example, including placing it on a certain page, etc.

[0086] Or, another way, Figure 1 As shown, an image deformation processing system can also be provided for merchant users / seller users, etc., and the system can also be connected to a specific commodity object information system. In this way, if a merchant user / seller user needs to generate a dynamic image for a published commodity object to express properties such as bendability and softness, the link to the details page of the specific commodity object can be entered through the image deformation processing system. After that, the image deformation processing system can select an image suitable as the original image from the details page, and generate a dynamic image after deformation processing in accordance with the method provided in the embodiment of the present application. Alternatively, a merchant user / seller user can also directly input a picture of the commodity object, and the image deformation processing system generates a dynamic image, and so on. After the dynamic image is generated, a preview function can also be provided to the user, so that the user can preview the generated dynamic image effect. In addition, functions such as downloading the generated dynamic image can also be provided, so that the user can download the generated dynamic image and use it. The specific use method can include: using this dynamic image as the main image of the commodity object, or it can also be published in the details page of the commodity object, for example, it can be specifically published in the commodity main image resource position in the details page, and so on.

[0087] The specific implementation scheme provided in the embodiments of the present application is described in detail below.

[0088] Embodiment 1

[0089] First, this embodiment provides an image deformation method, see Figure 2 , the method may include:

[0090] S201: Determine an original image, and identify the main content of the image to be deformed from the original image.

[0091] The original image can be input by the user, or the user can specify a specific page URL, and the image processing tool or system automatically analyzes the pictures in the page to determine a suitable original image, etc. During the automatic analysis, the judgment can be made based on information such as pre-set features. For example, the specific original image can meet the following characteristics: the specific commodity object is used as the main content of the image, the shooting angle of the main content meets the requirements (for example, if the bendable feature of the sole of the shoe needs to be expressed, the side view of the shoe can be selected as the original image), and it does not contain too much redundant content (for example, images with models can be filtered out, and images with too complex backgrounds can be filtered out, etc.), etc. In specific implementation, if the original image is input by the user, examples can also be provided to the user to help the user provide an original image that meets the requirements.

[0092] After the original image is determined, the main content of the image to be deformed can also be identified from it. Figure 3 As shown in the figure, the original image is a photo of a shoe. The part belonging to the shoe can be identified as the main content of the image, and the other parts belonging to the background do not need to participate in the subsequent calculation process. There are many algorithms for identifying the main content of the image, including a specific cutout algorithm to identify the edge of the main body, and then use the content within the edge as the main content of the image, etc. Therefore, an existing algorithm can be called to implement it.

[0093] It should be noted that in the original image, a specific pixel point may usually have RGB three-channel attributes. However, after the image main content recognition process is performed, the pixel points that are specifically identified as belonging to the image main content may have four-channel attributes, adding a transparency channel on the basis of the original RGB. This transparency information can also be used in subsequent processing.

[0094] It should also be noted that, in order to achieve better processing effects during specific implementation, some preprocessing can be performed after the main content recognition is completed. For example, the main content of the image can be rotated so that its position in the original image is more suitable for subsequent deformation processing, or it is easier to obtain a better deformation effect. For example, assuming that the original image is a photo of "shoes", the main image content of "shoes" in the original image can be rotated so that the toe of the shoe faces downward, so that the subsequently generated animated image can simulate the process of the sole of the shoe gradually bending when the shoe is worn and stepped on from the toe.

[0095] In addition, the border width of the original image can also be adjusted. Specifically, this step can be performed when necessary. For example, if the area occupied by the main content of the image in the original image is relatively large, there may not be much remaining space in the upper and lower or left and right directions of the original image. At this time, if the deformation process is directly based on the original image, the main content of the deformed image may exceed the boundary of the original image. Therefore, in an optional manner, if the area occupied by the main content of the image in the original image is relatively large, the border width of the original image can also be adjusted. For example, some background pixels can be added to the original image to make the border wider, and so on.

[0096] S202: Divide the main content of the image into a plurality of initial polygonal image blocks, and perform boundary expansion processing on the initial polygonal image blocks to obtain a plurality of expanded polygonal image blocks.

[0097] After identifying the specific main content of the image from the original image, the main content of the image can be divided into a plurality of initial polygonal image blocks. The specific polygons may include triangles, quadrilaterals, etc. In a preferred manner, this can be achieved in a triangular manner. That is, the main content of the image can be divided into a plurality of triangular image blocks, and then the triangular image blocks are subjected to a border enlargement process to obtain a plurality of expanded triangular image blocks.

[0098] Specifically, there are many ways to perform triangulation. For example, in one way, the Delaunay triangulation algorithm can be used to implement it (of course, other triangulation algorithms can also be used). The so-called Delaunay triangulation means that V is a finite point set on a two-dimensional real number field, and the edge e is a closed line segment composed of points in the point set as endpoints, and E is the set of e. Then a triangulation T = (V, E) of the point set V is a plane graph G, which satisfies the following conditions: a. Except for the endpoints, the edges in the plane graph do not contain any points in the point set; b. There are no intersecting edges; c. All faces in the plane graph are triangular faces, and the union of all triangular faces is the convex hull of the scattered point set V.

[0099] That is, for a known point set, the Delaunay triangulation algorithm can be used to form multiple triangles. Therefore, before using a specific Delaunay triangulation algorithm to triangulate the main content of an image, a point set can be first selected in the main content of the image, and then the point set can be input into the algorithm to output the triangulation result.

[0100] There are many ways to obtain the above point set. For example, in one way, the original image can be firstly divided into grids. For example, the original image can be specifically divided into uniform grids, and the grid width can be adaptively configured according to the area of ​​the main content of the image. In other words, if the area of ​​the main content of the image in the original image is relatively large, the grid width can also be larger. On the contrary, if the area of ​​the main content of the image in the original image is relatively small, the grid width can also be smaller, and so on. In the embodiment of the present application, Figure 3 In the example shown, the grid width can be set to 25 pixels. In this case, the specific grid division result can be as follows: Figure 4 shown.

[0101] After the meshing is completed, all meshes can be traversed, and the mesh vertices containing the main content of the image can be retained and deduplicated. That is, after the original image is meshed, some meshes will fall on the main content of the image, and some meshes will not fall on the main content of the image. Therefore, the meshes that fall on the main content of the image can be selected, and the vertices of such meshes can be retained and deduplicated to obtain a point set. For example, the specific method can be as follows: Figure 5 After the retained mesh vertices are used as point sets, the Delaunay triangulation algorithm can be called according to the retained mesh vertices to obtain multiple triangular image blocks. Figure 6 shown.

[0102] After the main content of the image is divided into multiple image blocks, in the embodiment of the present application, the moving least squares transform is not performed directly, but the polygon boundary is first expanded, that is, the polygon boundary is extended further outward, and different polygons can partially overlap.

[0103] Among them, there may be multiple methods for expanding the boundary of a specific polygon. For example, in the case of a triangle, it can be performed as follows: for each vertex of the triangular image block, the following processing is performed: first, multiple candidate points are selected around the vertex according to the target distance to form a candidate point set; then, the two edges where the vertex is located are used as dividing lines, and a first candidate point subset and a second candidate point subset are selected from the candidate point set, wherein, when selecting the candidate point subset with one of the edges as the dividing line, the vertex opposite to the edge is used as the reference vertex, and the candidate points on the opposite side of the reference vertex are selected to form a candidate point subset; thereafter, the replacement point of the vertex can be determined according to the intersection of the first candidate point subset and the second candidate point subset; finally, the replacement points of each vertex are connected to obtain the expanded triangular image block.

[0104] For example, Figure 7As shown, assuming that ABC is a subdivided triangle, the boundary can be expanded by the following steps:

[0105] a) For vertex A, select candidate points according to the input distance d. Taking d=1 pixel as an example, the selected candidate points may include points p1-p8; that is, the polygonal image block may be expanded according to the preset target distance d. The value of distance d should not be too large or too small. If it is too large, the area of ​​each triangular image block will be significantly larger, affecting the smoothness of the curved area of ​​the image after the final deformation. If it is too small, it may not be enough to make up for the gap formed by the distance between the image blocks being stretched. Therefore, in the specific implementation, the distance d can be slightly larger than the maximum distance that the image block may be stretched after the deformation. For example, assuming that the maximum distance stretched is two pixels, d can be three pixels, and so on.

[0106] b) Take the straight line l1 formed by edge AB as the dividing line (at this time, edge AB is the opposite side of vertex C, so vertex C can be used as the reference vertex), and retain the candidate points on the opposite side of vertex C, namely p1, p2, p3, and p8;

[0107] c) Take the straight line l2 formed by edge AC as the dividing line, and keep the candidate point on the opposite side of vertex B in the result of c), which is p8;

[0108] d) If there is only one candidate point left (such as p8 in this example), this point can be a substitute for point A. It should be noted here that after setting the value of d, each triangle can use the same d for edge expansion. Therefore, the following situation may occur. For a certain triangle, after selecting in the above manner, it is found that there are no remaining candidate points, which means that the setting of d is relatively small for the triangle. However, since the value of d should not be modified during the operation of the algorithm, point A can be directly used as its own substitute point. In addition, it is also possible that when processing a certain triangle, multiple candidate points are found to remain. In this case, the point farthest from point A can be selected from the multiple candidate points as the substitute point;

[0109] e) Repeat steps a to d for vertices B and C. The triangle formed by the three replacement points is the expanded triangle row, such as Figure 6 Shown in the dashed triangle.

[0110] S203: Mapping the vertices of the expanded polygonal image block to the deformed target image using the first transformation model.

[0111] After completing the edge expansion of the polygon, the first transformation can be performed based on the vertices of the expanded polygon to determine the position of each vertex in the target image. The target image is the deformed image to be generated. In specific implementation, the above-mentioned first transformation can be implemented in a variety of ways. For example, in a specific implementation method, it can be implemented using a moving least squares model. Specifically, in order to use the moving least squares method for transformation, multiple anchor point positions can be specified in the original image, as well as the expected position corresponding to each anchor point in the target image to be generated (this process can be independent of the aforementioned triangulation process, and therefore can be performed in parallel). The anchor point position can be selected in position and quantity based on the shape of the specific image main content, the characteristics of the required deformation, etc. The expected position is the position where the above-mentioned anchor point position appears in the target image after the transformation. In specific implementation, the specific anchor point position is a key position used to reflect the effect of the change. For example, Figure 8 As shown in the figure, assuming that the original image shows a shoe, and the deformation process needs to be used to show the curved and soft characteristics of the sole, the anchor points can be S1 at the toe, S2 at one-third of the sole (the sole is usually curved at this position), and S3 at the heel. The desired positions in the target image can be t1, t2, and t3, respectively.

[0112] It should be noted here that in the specific implementation, it may be necessary to express the characteristics of the bending of the sole through a dynamic image composed of multiple frames of target images. Therefore, multiple frames of target images can be generated based on the original image, wherein the anchor point position in the original image can remain unchanged, but the corresponding expected position in each frame of the target image can be different, thereby presenting a dynamic effect of gradual bending and deformation. Therefore, the expected position of each anchor point can be specified for different target image frames. Among them, the selection of the anchor point position in the original image and the determination of the expected position in each frame of the target image can be automatically realized by an algorithm, which will not be described in detail here.

[0113] After determining the anchor point position in the original image and the expected position in the target image, multiple groups of mapping points can be established using the anchor point position and the expected position, and a moving least squares fitting function is established based on the multiple groups of mapping points. Among them, in the moving least squares method, it is necessary to establish a fitting curve near a set of nodes at different positions, and each node has its own set of coefficients for defining the shape of the fitting curve near the position. Therefore, when calculating the fitting curve near a certain node, it is only necessary to calculate the set of coefficient values ​​of the point. The multiple groups of mapping points determined above can be used to determine the coefficient values. The specific implementation method does not belong to the focus of the embodiments of the present application, so it will not be described in detail here.

[0114] After constructing the moving least squares fitting function, the fitting function can be used to map the vertices of the specific expanded polygon to the target image to be generated, so as to determine the position mapping result of the vertices in the target image to be generated. That is to say, in the embodiment of the present application, the moving least squares transformation is not directly performed on the vertices of the polygons that have been segmented, but on the vertices of the polygons that have completed the boundary expansion.

[0115] S204: Mapping the pixel points in the expanded polygonal image block to the target image using a second transformation model according to the position correspondence information of the vertices before and after the mapping.

[0116] After mapping the vertices of the expanded polygon to the target image, the positions of the vertices in the target image can be determined. Then, other pixel points inside the polygon can be mapped to the target image according to the vertex mapping results, that is, the positions of other pixel points inside the polygon in the target image can be determined.

[0117] Specifically, when mapping other pixel points inside the polygon, it can be achieved through a variety of specific transformation models. For example, in one way, the specific second transformation model may include a geometric transformation model, that is, the geometric transformation model can be used to map the pixel points in the expanded polygon image block to the target image. Specifically, the specific geometric transformation model may include rigid transformation, affine transformation, perspective transformation, nonlinear transformation, etc. Specifically, the corresponding geometric transformation model can be determined according to the number of edges of the specifically decomposed polygon. For example, if a triangulation is performed, an affine transformation model can be used, and if a quadrilateral is performed, a perspective transformation model can be used, and so on.

[0118] Specifically, taking triangulation as an example, three groups of mapping points can be determined based on the position correspondence information of the vertices of the expanded triangular image block before and after mapping, and then, based on the three groups of mapping points, the pixel points in the expanded triangular image block are mapped to the target image using an affine transformation model. The so-called affine transformation refers to the process in geometry where a vector space is transformed into another vector space by a linear transformation followed by a translation.

[0119] After completing the mapping of the vertices of the specific expanded polygonal image block and the internal pixels to the target image, the positions of the pixels of the main content of the original image in the target image can be determined, and then the pixel values ​​of each pixel are filled into the corresponding positions in the target image to generate the target image. For example, if the original image and the anchor point position are as follows Figure 8 As shown, the target image generated after deformation processing can be Fig. 9 As shown, it can be seen that the main image in the image is bent and deformed. The target image generated after the deformation processing can be a dynamic effect image, which dynamically displays the whole process of the bending deformation of the main image. Fig. 9 It is a state diagram at a certain moment.

[0120] It should be noted here that, since the polygonal image blocks are expanded, there will be some pixel overlap between the polygons after the expansion process, that is, the same pixel may belong to different expanded polygons. After the transformation is completed, there may also be some overlap between different image blocks in the target image, especially at the edges of the image blocks. Among them, the overlap in the target image mainly refers to that two different pixel points in the original image are mapped to the same position in the target image. Therefore, in the specific implementation, this situation can also be processed. Specifically, if two pixel points are mapped to the same position in the target image, the one with lower transparency can be selected from the two pixel points and displayed at that position, and so on. Among them, the transparency information of the pixel points can be obtained in the aforementioned process of image body content recognition, or can also be obtained by other means, which is not limited here.

[0121] In summary, through the embodiments of the present application, the main image content in the original image can be divided into multiple initial polygonal image blocks during the deformation process of the image, and each initial polygon can be extended to obtain an extended polygonal image block. Afterwards, the vertices of the extended polygonal image block can be mapped to the target image using the first transformation model. Combined with the second transformation model, the pixel points inside the extended polygonal image block can be mapped to the target image, thereby generating a specific target image. In this way, since the polygonal image block is extended to its boundary, overlapping areas are generated between different polygonal image blocks. In this way, even if the distance between different extended polygonal image blocks is pulled apart during the mapping process, the existence of such overlapping areas can be used to make up for the gaps that may be caused by the pulled apart distances, thereby improving the image effect after deformation.

[0122] Embodiment 2

[0123] This second embodiment is an application of the algorithm provided in the first embodiment to the micro-motion effect attribute scenario of the commodity object, and provides a method for generating a dynamic image for the commodity object. Fig.10 , the method may include:

[0124] S1001: Determine a target product object for which an animated image needs to be generated and a corresponding original image, and identify the main content of the image to be deformed from the original image;

[0125] S1002: dividing the main content of the image into a plurality of original polygonal image blocks, and performing boundary expansion processing on the original polygonal image blocks to obtain a plurality of expanded polygonal image blocks;

[0126] S1003: determining positions of multiple anchor points from the main content of the image, and expected positions of each anchor point in multiple frames of target images;

[0127] S1004: Perform the following processing on the multiple frames of target images respectively:

[0128] For the current target image, a first transformation model is established according to the positions of the plurality of anchor points and the expected position, and the vertices of the expanded polygonal image block are mapped to the current target image using the first transformation model; and pixel points in the expanded polygonal image block are mapped to the current target image using a second transformation model according to the position correspondence information of the vertices before and after mapping, so as to generate the current target image;

[0129] S1005: Generate a dynamic image of the target product object according to the generated multiple frames of target images.

[0130] Specifically, when determining the target product object for which the animated image needs to be generated and the corresponding original image, there may be a variety of methods. For example, the user may directly input the original image, or, in another way, the user may input the page address information of the target page associated with the target product object. For example, it may be a link address of the product object details page, etc. Afterwards, the image associated with the target page may be analyzed according to the page address information, and an image that meets the target condition may be determined as the original image.

[0131] In addition, in the specific implementation, it can also be connected with the product object information system, and an operation control for publishing the animated image can be provided, so that after receiving the user operation through the operation control, the generated animated image can be published to the target page. For example, it can be published to the product main image resource position in the detail page, etc.

[0132] For the parts not described in detail in the second embodiment, please refer to the description in the first embodiment, which will not be repeated here.

[0133] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0134] Corresponding to the first embodiment, the present application embodiment further provides an image deformation device, see Fig.11 , the device may include:

[0135] The original image determining unit 1101 is used to determine the original image and identify the main content of the image to be deformed from the original image;

[0136] The edge expansion processing unit 1102 is used to divide the main content of the image into a plurality of initial polygonal image blocks, and perform boundary expansion processing on the initial polygonal image blocks to obtain a plurality of edge expanded polygonal image blocks;

[0137] A first mapping unit 1103 is used to map the vertices of the expanded polygonal image block to a deformed target image using a first transformation model;

[0138] The second mapping unit 1104 is used to map the pixel points in the expanded polygonal image block to the target image using a second transformation model according to the position correspondence information of the vertices before and after the mapping.

[0139] The edge expansion processing unit can be specifically used for:

[0140] The main content of the image is divided into a plurality of initial triangular image blocks, and the initial triangular image blocks are subjected to boundary expansion processing to obtain a plurality of expanded triangular image blocks.

[0141] Specifically, when the main content of the image is divided into a plurality of initial triangular image blocks, the edge expansion processing unit may be specifically used to:

[0142] Performing grid division on the original image;

[0143] Traverse all meshes, retain mesh vertices containing the main content of the image and perform deduplication processing;

[0144] A preset subdivision algorithm is called according to the retained mesh vertices to obtain the multiple triangular image blocks.

[0145] Specifically, the edge expansion processing unit can be used for:

[0146] For each vertex of the triangular image block, the following processing is performed:

[0147] Select multiple candidate points around the vertex according to the target distance to form a candidate point set;

[0148] Using the two edges where the vertices are located as dividing lines respectively, a first candidate point subset and a second candidate point subset are selected from the candidate point set, wherein when selecting the candidate point subset using one of the edges as the dividing line, the vertex opposite to the edge is used as a reference vertex, and candidate points on the opposite side of the reference vertex are selected to form a candidate point subset;

[0149] Determine a replacement point for a vertex according to an intersection of the first candidate point subset and the second candidate point subset;

[0150] The replacement points of each vertex are connected to obtain the expanded triangle image block.

[0151] Specifically, the size of the target distance is related to the distance that different image blocks may be pulled apart during the mapping process.

[0152] The second mapping unit may be specifically used for:

[0153] Determine three groups of mapping points according to the position correspondence information of the vertices of the expanded triangle image block before and after mapping;

[0154] Based on the three groups of mapping points, the pixel points in the expanded triangle image block are mapped into the target image using an affine transformation model.

[0155] Specifically, the first transformation model includes a moving least squares transformation model;

[0156] The first mapping unit may be specifically used for:

[0157] Based on the positions of multiple anchor points set in the original image and the expected positions corresponding to the anchor points in the target image to be generated, a moving least squares fitting function is established;

[0158] The vertices are mapped to the target image to be generated by using the moving least squares fitting function to determine the position mapping result of the vertices in the target image to be generated.

[0159] In addition, the device may also include:

[0160] The pixel point selection unit is used for selecting a pixel point with lower transparency from among the multiple pixel points and mapping it to the same position if there are multiple pixel points mapped to the same position in the target image.

[0161] Corresponding to the second embodiment, the present application embodiment also provides a device for generating a dynamic image for a commodity object, see Fig.12 , the device may include:

[0162] The original image determination unit 1201 is used to determine the target commodity object for which the animated image needs to be generated and the corresponding original image, and to identify the main content of the image to be deformed from the original image;

[0163] The edge expansion processing unit 1202 is used to divide the main content of the image into a plurality of original polygonal image blocks, and perform boundary expansion processing on the original polygonal image blocks to obtain a plurality of edge expanded polygonal image blocks;

[0164] An anchor point position determination unit 1203, used to determine the positions of multiple anchor points from the main content of the image, and the expected positions of each anchor point in multiple frames of target images;

[0165] The target image generating unit 1204 is used to perform the following processing respectively for the multiple frames of target images: for the current target image, establish a first transformation model according to the positions of the multiple anchor points and the expected position, map the vertices of the edge-expanding polygonal image block to the current target image by using the first transformation model, and map the pixel points in the edge-expanding polygonal image block to the current target image by using a second transformation model according to the position correspondence information of the vertices before and after mapping, so as to generate the current target image;

[0166] The dynamic image generation unit 1205 is used to generate a dynamic image of the target commodity object according to the generated multiple frames of target images.

[0167] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0168] And an electronic device, comprising:

[0169] one or more processors; and

[0170] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0171] in, Fig.13 The architecture of the electronic device is shown as an example, which may include a processor 1310, a video display adapter 1311, a disk drive 1312, an input / output interface 1313, a network interface 1314, and a memory 1320. The processor 1310, the video display adapter 1311, the disk drive 1312, the input / output interface 1313, the network interface 1314, and the memory 1320 may be communicatively connected via a communication bus 1330.

[0172] Among them, the processor 1310 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0173] The memory 1320 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1320 can store an operating system 1321 for controlling the operation of the electronic device 1300, and a basic input and output system (BIOS) for controlling the low-level operation of the electronic device 1300. In addition, a web browser 1323, a data storage management system 1324, and an image deformation processing system 1325, etc. can also be stored. The above-mentioned image deformation processing system 1325 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 1320 and is called and executed by the processor 1310.

[0174] The input / output interface 1313 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0175] The network interface 1314 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0176] The bus 1330 comprises a pathway for transmitting information between the various components of the device (eg, the processor 1310, the video display adapter 1311, the disk drive 1312, the input / output interface 1313, the network interface 1314, and the memory 1320).

[0177] It should be noted that, although the above device only shows a processor 1310, a video display adapter 1311, a disk drive 1312, an input / output interface 1313, a network interface 1314, a memory 1320, a bus 1330, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0178] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0179] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0180] The above is a detailed introduction to the image deformation method, device and electronic device provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. An image deformation method, characterized in that: include: Determine an original image, and identify the main content of the image to be deformed from the original image; The main content of the image is divided into a plurality of initial polygonal image blocks, and the initial polygonal image blocks are subjected to boundary expansion processing to obtain a plurality of expanded polygonal image blocks; wherein, when the boundary expansion processing is performed, a vertex replacement point is determined around each vertex of the polygonal image block according to a target distance, so as to obtain the expanded polygonal image block by connecting the replacement points of each vertex; wherein, the target distance is slightly larger than the maximum distance that the image block can be stretched after the deformation processing; Mapping the vertices of the expanded polygonal image block to the deformed target image using a first transformation model; According to the position correspondence information of the vertices before and after mapping, the pixel points in the expanded polygonal image block are mapped to the target image using a second transformation model.

2. The method according to claim 1, characterized in that The step of dividing the main content of the image into a plurality of initial polygonal image blocks comprises: The main content of the image is divided into a plurality of initial triangular image blocks, and the initial triangular image blocks are subjected to boundary expansion processing to obtain a plurality of expanded triangular image blocks.

3. The method according to claim 2, characterized in that The step of performing boundary expansion processing on the triangular image block to obtain a plurality of expanded triangular image blocks includes: For each vertex of the triangular image block, the following processing is performed: Select multiple candidate points around the vertex according to the target distance to form a candidate point set; Using the two edges where the vertices are located as dividing lines respectively, a first candidate point subset and a second candidate point subset are selected from the candidate point set, wherein when selecting the candidate point subset using one of the edges as the dividing line, the vertex opposite to the edge is used as a reference vertex, and candidate points on the opposite side of the reference vertex are selected to form a candidate point subset; Determine a replacement point for a vertex according to an intersection of the first candidate point subset and the second candidate point subset; The replacement points of each vertex are connected to obtain the expanded triangle image block.

4. The method according to claim 3, characterized in that The size of the target distance is related to the distance that different image blocks may be pulled apart during the mapping process.

5. The method according to claim 2, characterized in that: The second transformation model includes an affine transformation model; The step of mapping the pixel points in the expanded polygonal image block to the target image by using the second transformation model includes: Determine three groups of mapping points according to the position correspondence information of the vertices of the expanded triangle image block before and after mapping; Based on the three groups of mapping points, the pixel points in the expanded triangle image block are mapped into the target image using an affine transformation model.

6. The method according to any one of claims 1 to 5, characterized in that: The first transformation model comprises a moving least squares transformation model; The method of mapping the vertices of the expanded polygonal image block to the deformed target image by using the first transformation model includes: Based on the positions of multiple anchor points set in the original image and the expected positions corresponding to the anchor points in the target image to be generated, a moving least squares fitting function is established; The vertices are mapped to the target image to be generated by using the moving least squares fitting function to determine the position mapping result of the vertices in the target image to be generated.

7. The method according to any one of claims 1 to 5, characterized in that: Also includes: If there are multiple pixel points mapped to the same position in the target image, a pixel point with lower transparency is selected from the multiple pixel points and mapped to the position.

8. A method for generating a dynamic image for a commodity object, characterized in that: include: Determine the target product object for which the animated image needs to be generated and the corresponding original image, and identify the main content of the image to be deformed from the original image; The main content of the image is divided into a plurality of original polygonal image blocks, and the original polygonal image blocks are subjected to boundary expansion processing to obtain a plurality of expanded polygonal image blocks; wherein, when the boundary expansion processing is performed, a vertex replacement point is determined around each vertex of the polygonal image block according to a target distance, so as to obtain the expanded polygonal image block by connecting the replacement points of each vertex; wherein, the target distance is slightly larger than the maximum distance that the image block can be stretched after the deformation processing; Determining the positions of multiple anchor points from the main content of the image, and the expected positions of each anchor point in multiple frames of target images; For the multiple frames of target images, the following processing is performed respectively: For the current target image, a first transformation model is established according to the positions of the plurality of anchor points and the expected position, the vertices of the expanded polygonal image block are mapped to the current target image using the first transformation model, and the pixel points in the expanded polygonal image block are mapped to the current target image using a second transformation model according to the position correspondence information of the vertices before and after mapping, so as to generate the current target image; A dynamic image of the target commodity object is generated according to the generated multiple frames of target images.

9. An image deformation device, characterized in that: include: An original image determination unit, used to determine an original image and identify the main content of the image to be deformed from the original image; A border expansion processing unit is used to divide the main content of the image into a plurality of initial polygonal image blocks, and perform border expansion processing on the initial polygonal image blocks to obtain a plurality of border-expanded polygonal image blocks; wherein, when performing the border expansion processing, vertex replacement points are determined around each vertex of the polygonal image block according to a target distance, so as to obtain the border-expanded polygonal image block by connecting the replacement points of each vertex; wherein, the target distance is slightly larger than the maximum distance that the image block can be stretched after the deformation processing; A first mapping unit, used for mapping the vertices of the expanded polygonal image block to a deformed target image using a first transformation model; The second mapping unit is used to map the pixel points in the expanded polygonal image block to the target image using a second transformation model according to the position correspondence information of the vertices before and after the mapping.

10. A device for generating a dynamic image for a commodity object, characterized in that: include: The original image determination unit is used to determine the target commodity object for which the animated image needs to be generated and the corresponding original image, and to identify the main content of the image to be deformed from the original image; an edge expansion processing unit, used for dividing the main content of the image into a plurality of original polygonal image blocks, and performing boundary expansion processing on the original polygonal image blocks to obtain a plurality of edge-expanded polygonal image blocks; wherein, when performing the boundary expansion processing, vertex replacement points are determined around each vertex of the polygonal image block according to a target distance, so as to obtain the edge-expanded polygonal image block by connecting the replacement points of each vertex; wherein, the target distance is slightly larger than the maximum distance that the image block can be stretched after the deformation processing; An anchor point position determination unit, used to determine the positions of multiple anchor points from the main content of the image, and the expected positions of each anchor point in multiple frames of target images; The target image generation unit is used to perform the following processing on the multiple frames of target images respectively: for the current target image, establish a first transformation model according to the positions of multiple anchor points and the expected position, map the vertices of the expanded polygonal image block to the current target image using the first transformation model, and map the pixel points in the expanded polygonal image block to the current target image using a second transformation model according to the position correspondence information of the vertices before and after mapping, so as to generate the current target image; The dynamic image generation unit is used to generate a dynamic image of the target commodity object according to the generated multiple frames of target images.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

12. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of claims 1 to 8.

Images