Image processing method and device, electronic equipment, storage medium and product

Abstract

One or more embodiments of the invention provide an image processing method and device, equipment, a storage medium and a product. The method comprises the following steps: in response to a deformation instruction for a target object in a to-be-edited image, performing deformation processing on a grid model corresponding to the target object to obtain a deformed grid model; wherein the grid model is composed of a first grid used for representing local image content of the target object; the deformed grid model comprises a second grid obtained by performing deformation processing on the first grid; adjusting the position of a vertex in the second grid according to the difference between the second grid and the first grid corresponding to the second grid, so that the scale of the same local image content of the target object represented by the adjusted second grid and the first grid corresponding to the second grid is kept unchanged; and according to the gap region between the adjusted second grids, carrying out image content repairing processing to obtain a target image. And a natural deformation effect is realized.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of image processing technology, and in particular, to an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the continuous progress of technology, users' demand for image editing is also increasing. Among them, users hope to be able to perform deformation operations such as stretching, dragging, and twisting on objects in any image to meet personalized editing needs. To meet these needs, image deformation technology is also constantly iterating and developing.

[0003] Image deformation technology is a technology used to transform the shape or appearance of an image, which allows users to perform free deformation operations on the image. However, in actual applications, it may cause problems of image distortion, resulting in unnatural effects such as image distortion and blurring. Summary of the Invention

[0004] In view of this, one or more embodiments of this specification provide an image processing method, apparatus, electronic device, storage medium, and product.

[0005] To achieve the above object, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of this specification, an image processing method is proposed, including:

[0007] In response to a deformation instruction for a target object in an image to be edited, perform deformation processing on the mesh model corresponding to the target object to obtain a deformed mesh model; wherein, the mesh model is composed of a first mesh for representing the local image content of the target object; the deformed mesh model includes a second mesh obtained by performing deformation processing on the first mesh;

[0008] According to the difference between the second mesh and the corresponding first mesh, adjust the positions of the vertices in the second mesh so that the scales of the same local image content of the target object represented by the adjusted second mesh and the corresponding first mesh remain unchanged;

[0009] Perform image content patching processing according to the gap regions between the adjusted second meshes to obtain a target image.

[0010] In one implementation, the first mesh is divided into at least two first sub-meshes, and each first sub-mesh contains at least one coincident vertex that coincides with the vertices in the first mesh;

[0011] The second grid includes second sub - grids obtained by deforming first sub - grids in the first grid;

[0012] Adjusting the positions of the vertices in the second grid according to the difference between the second grid and the corresponding first grid includes:

[0013] Adjusting the positions of the other vertices except the coincident vertices in the second sub - network according to the difference between the second sub - grid in the second grid and the corresponding first sub - grid;

[0014] Performing image content patching processing according to the gap area between the adjusted second grids to obtain a target image includes:

[0015] Performing image content patching processing according to the gap area between the adjusted second sub - grids to obtain a target image.

[0016] In one implementation, adjusting the positions of the other vertices except the coincident vertices in the second sub - network according to the difference between the second sub - grid in the second grid and the corresponding first sub - grid includes:

[0017] Adjusting the side lengths of the sides in the second sub - grid according to the difference between the sides in the second sub - grid and the corresponding sides in the first sub - grid; and / or

[0018] Adjusting the angles of the angles corresponding to the coincident vertices in the second sub - grid according to the difference between the angles corresponding to the coincident vertices in the second sub - grid and the corresponding angles corresponding to the coincident vertices in the first sub - grid.

[0019] In one implementation, adjusting the side lengths of the sides in the second sub - grid according to the difference between the sides in the second sub - grid and the corresponding sides in the first sub - grid includes:

[0020] If the side in the second sub - grid is longer than the corresponding side in the first sub - grid, adjusting the side length of the side in the second sub - grid to be the same as the side length of the corresponding side in the first sub - grid;

[0021] If the side in the second sub - grid is shorter than or equal to the corresponding side in the first sub - grid, shortening the side length of the side in the second sub - grid by a first preset ratio.

[0022] In one implementation, adjusting the angles of the angles corresponding to the coincident vertices in the second sub - grid according to the difference between the angles corresponding to the coincident vertices in the second sub - grid and the corresponding angles corresponding to the coincident vertices in the first sub - grid includes:

[0023] If the included angle corresponding to the coincident vertex in the second sub-grid is greater than the included angle corresponding to the coincident vertex in the first sub-grid corresponding thereto, adjust the included angle corresponding to the coincident vertex in the second sub-grid to the included angle corresponding to the coincident vertex in the first sub-grid corresponding thereto;

[0024] If the included angle corresponding to the coincident vertex in the second sub-grid is less than or equal to the included angle corresponding to the coincident vertex in the first sub-grid corresponding thereto, reduce the angle of the included angle corresponding to the coincident vertex in the second sub-grid according to a second preset ratio.

[0025] In one implementation, the performing image content patching processing according to the gap region between the adjusted second sub-grids to obtain a target image includes:

[0026] Obtain a to-be-patched image represented by a mesh model composed of the adjusted second grids; wherein, the size of the mesh model composed of the adjusted second grids is the same as the size of the deformed mesh model;

[0027] Generate an initial mask image having the same size as the to-be-patched image, and pixel values of pixels in the initial mask image are all a first value;

[0028] According to positions of vertices of the adjusted second grids, identify a gap region in the mesh model composed of the adjusted second grids;

[0029] Modify pixel values of pixels corresponding to the gap region in the initial mask image from the first value to a second value to obtain a mask image for representing the gap region;

[0030] According to the mask image for representing the gap region, perform patching processing on the to-be-patched image to obtain a target image.

[0031] In one implementation, the performing patching processing on the to-be-patched image according to the mask image for representing the gap region to obtain a target image includes:

[0032] According to the mask image for representing the gap region, determine a to-be-patched region included in the to-be-patched image;

[0033] With reference to a region for displaying local image content of a target object around the to-be-patched region, perform patching processing on the to-be-patched region to obtain a target image.

[0034] In one implementation, the mesh model includes a triangular mesh model, and the first grid includes triangular meshes; wherein, the triangular mesh model is obtained by performing triangular meshing on the shape of the target object; and / or

[0035] The to-be-edited image is obtained by performing foreground segmentation on a first selected image.

[0036] In one implementation, it further includes:

[0037] Obtain a to-be-processed image, where the to-be-processed image is obtained by performing matte extraction on a selected area in a second selected image. The to-be-processed image has the same size as the second selected image, and the to-be-processed image includes a to-be-complemented area with the same position and size as the selected area of the second selected image;

[0038] Detect the color around the to-be-complemented area in the to-be-processed image, and perform a complementation process on the to-be-complemented area in the to-be-processed image according to the detection result.

[0039] In one implementation, the performing a complementation process on the to-be-complemented area in the to-be-edited image according to the detection result includes:

[0040] If the detection result indicates that the color around the to-be-complemented area in the to-be-edited image is a specified color, perform a complementation process on the to-be-complemented area in the to-be-edited image according to the specified color; wherein, the specified color includes a solid color or a gradient color.

[0041] In one implementation, it further includes:

[0042] If the detection result indicates that the color around the to-be-complemented area in the to-be-edited image is not a specified color, output reference area selection information to prompt the user to select a reference area in the to-be-processed image for complementing the to-be-complemented area;

[0043] In response to the user's selection operation of the reference area, send a first complementation request carrying the reference area and the to-be-edited image to the server, so that the server generates first complementation content for the to-be-complemented area according to the reference area, perform a complementation process on the to-be-complemented area in the to-be-edited image based on the first complementation content, and return the complemented image;

[0044] In response to the user not selecting a reference area within a preset duration, send a second complementation request carrying the to-be-edited image to the server, so that the server generates second complementation content for the to-be-complemented area according to the image content around the to-be-complemented area in the to-be-edited image, perform a complementation process on the to-be-complemented area in the to-be-edited image based on the second complementation content, and return the complemented image.

[0045] According to the second aspect of one or more embodiments of this specification, an image processing apparatus is provided, including:

[0046] A deformation processing module, configured to perform deformation processing on a mesh model corresponding to the target object in response to a deformation instruction for the target object in the image to be edited, so as to obtain a deformed mesh model; wherein, the mesh model is composed of a first mesh for representing local image content of the target object; the deformed mesh model includes a second mesh obtained by performing deformation processing on the first mesh;

[0047] A mesh adjustment module, configured to adjust the positions of the vertices in the second mesh according to the difference between the second mesh and the corresponding first mesh, so that the scales of the same local image content of the target object represented by the adjusted second mesh and the corresponding first mesh remain unchanged;

[0048] A patching processing module, configured to perform patching processing on the image content according to the gap area between the adjusted second meshes to obtain a target image.

[0049] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0050] A processor;

[0051] A memory for storing instructions executable by the processor;

[0052] Wherein, when the processor executes the executable instructions, it is configured to implement the method described in the first aspect.

[0053] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in any one of the above are implemented.

[0054] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the above are implemented.

[0055] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0056] In the embodiments of the present disclosure, for an image to be edited, in response to a deformation instruction for a target object in the image to be edited, a mesh model corresponding to the target object may be deformed to obtain a deformed mesh model. Further, the meshes in the deformed mesh model may be adjusted so that the scale of the local image content of the target object represented by the meshes remains unchanged before and after deformation. This helps to maintain the consistency between the deformed image and the original target object image, thereby avoiding the distortion problem caused by deformation. During the process of adjusting the meshes in the deformed mesh model, there may be a gap between two meshes. By patching the image content in the gap area, the final obtained target image is made more complete and natural.

[0057] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. Description of the Drawings

[0058] Figure 1 It is a schematic diagram of the effect after stretching a tree using the image deformation technology in the related art.

[0059] Figure 2 It is a flowchart of an image processing method provided by an exemplary embodiment.

[0060] Figure 3 It is a schematic diagram after grid processing provided by an exemplary embodiment.

[0061] Figure 4 It is a schematic diagram of a first mesh and a second mesh provided by an exemplary embodiment.

[0062] Figure 5 It is a schematic diagram of adjusting the side length and the included angle of a second sub-mesh in the second mesh provided by an exemplary embodiment.

[0063] Figure 6 It is provided by an exemplary embodiment for Figure 3 A schematic diagram after deforming the shown tree and adjusting the second mesh.

[0064] Figure 7 It is a schematic diagram of a target image provided by an exemplary embodiment.

[0065] Figure 8 It is a schematic diagram regarding image completion provided by an exemplary embodiment.

[0066] Figure 9 It is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment.

[0067] Figure 10It is a block diagram of an image processing device provided by an exemplary embodiment. Detailed implementation manners

[0068] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0069] It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0070] The image deformation technology is a technology used to transform the shape or appearance of an image, which allows users to perform free deformation operations on the image. However, in practical applications, it may cause problems of image distortion, resulting in unnatural effects such as image distortion and blurring. For example, please refer to Figure 1 , which is a schematic diagram of the effect after stretching a tree using the image deformation technology in related technologies. It can be seen that there are obvious distortion effects on both the left and right sides.

[0071] Based on this, the embodiments of this specification provide an image processing method. For a to-be-edited image, in response to a deformation instruction for a target object in the to-be-edited image, the grid model corresponding to the target object can be deformed to obtain a deformed grid model. Furthermore, the grids in the deformed grid model can be adjusted so that the scale of the local image content of the target object represented by the grids remains unchanged before and after deformation, which helps to maintain the consistency between the deformed image and the original to-be-edited image, thereby avoiding the distortion problem caused by deformation. During the process of adjusting the grids in the deformed grid model, there may be a gap between two grids. By performing image content repair processing on the gap area, the final obtained target image is more complete and natural.

[0072] The image processing method provided by the embodiments of this specification can be executed by an electronic device, which includes but is not limited to a smart phone / mobile phone, a tablet computer, a personal digital assistant (PDA), a laptop computer, a desktop computer, a media content player, a video game station / system, a virtual reality system, an augmented reality system, a wearable device (such as a watch, glasses, gloves, headgear (such as a hat, a helmet, a virtual reality headset, an augmented reality headset, a head-mounted device (HMD), a headband), a pendant, an armband, a leg ring, shoes, a vest), a remote control, or any other type of device.

[0073] Exemplarily, the electronic device includes a processor and a memory, and the memory stores executable instructions that can run on the processor. When the processor executes the executable instructions, the image processing method provided by the embodiments of this application is implemented.

[0074] Exemplarily, the electronic device integrates a computer program product, and when the electronic device executes the computer program product, the image processing method provided by the embodiments of this application is implemented.

[0075] Please refer to Figure 2 , Figure 2 which shows a schematic flowchart of an image processing method. The image processing method can be applied to an electronic device and includes:

[0076] In S101, in response to a deformation instruction for a target object in the image to be edited, perform a deformation process on the mesh model corresponding to the target object to obtain a deformed mesh model; wherein, the mesh model is composed of a first mesh for representing the local image content of the target object; the deformed mesh model includes a second mesh obtained by performing a deformation process on the first mesh.

[0077] Exemplarily, the electronic device can perform image segmentation on a specified position of the first selected image to obtain an image to be edited that only contains the target object; wherein, the specified position is the position of the target object specified by the user. For example, the first selected image can be displayed in the interaction interface, and the user can specify the position of the target object through operations such as box selection and clicking.

[0078] Exemplarily, the electronic device can perform foreground segmentation processing on the first selected image to obtain an image to be edited that only contains the target object, and the target object is the foreground object in the first selected image. Among them, when performing foreground segmentation processing, methods based on features such as pixel color, texture, shape, and depth can be used to distinguish the foreground and background in the image.

[0079] Exemplarily, the image to be edited is a mask image that only contains the target object. The image to be edited that only contains the target object can reduce background interference in subsequent image processing and improve the editing efficiency and accuracy.

[0080] The mesh model corresponding to the target object can be obtained by meshing the shape of the target object in the image to be edited. The mesh model consists of a number of first meshes for representing the local image content of the target object.

[0081] Exemplarily, by segmenting the contour or boundary of the target object, a set of boundary points can be obtained. Then, a regular or irregular mesh structure is constructed around these boundary points, and the boundary points are connected to form a series of meshes. These meshes can be triangles, quadrilaterals or polygons of other shapes, depending on the meshing algorithm adopted and the editing requirements.

[0082] In one example, the electronic device can perform triangulation on the shape of the target object to obtain a triangular mesh model; the triangular mesh model consists of a number of triangular meshes. That is, the above-mentioned mesh model includes the triangular mesh model, and the first mesh includes triangular meshes. Please refer to Figure 3 , Figure 3 shows a schematic diagram after meshing the tree.

[0083] Each first mesh of the mesh model contains the local image content of the target object and can be regarded as a small image area. These local image contents can be representations of colors, textures, shapes or other features. Each first mesh of the mesh model can correspond to a local part of the target object, and these local parts can be independently edited, deformed or replaced.

[0084] By meshing the shape of the target object, the obtained mesh model can provide more flexible and precise editing operations, making the image editing process more refined and controllable.

[0085] Exemplarily, the user can perform deformation operations such as stretching, dragging, and twisting on the target object according to actual needs. Then, the electronic device generates a deformation instruction based on the user's deformation operation and performs deformation processing on the mesh model. The deformation processing is achieved by changing the position, shape or size of the mesh cells in the mesh model.

[0086] For example, the deformation processing includes but is not limited to: (1) Translation: moving the mesh cell along a certain direction by a certain distance to change its position; (2) Scaling: changing the size of the mesh cell, which can make it larger or smaller; (3) Rotation: rotating the mesh cell around a certain center point to change its direction and angle; (4) Distortion: making the mesh cell bend or deform by changing its shape.

[0087] Exemplarily, an image grid deformation algorithm (such as the As-Rigid-As-Possible algorithm, the Moving least square algorithm, etc.) can be adopted to perform deformation processing on the grid model to obtain a deformed grid model. The deformed grid model includes a second grid obtained by deforming the first grid.

[0088] By performing deformation processing on the grid model, the shape adjustment and deformation effect of the target object can be achieved.

[0089] In S102, according to the difference between the second grid and the corresponding first grid, the positions of the vertices in the second grid are adjusted so that the scales of the same local image content of the target object represented by the adjusted second grid and the corresponding first grid remain unchanged.

[0090] Exemplarily, when adjusting the vertex positions in the second grid to keep the scale of the local image content unchanged, corresponding adjustments are made according to the difference between the first grid and the second grid, which can ensure that the deformed grid model can still accurately represent the same local image content of the target object, thereby maintaining the consistency between the deformed grid model and the original image and avoiding the distortion problem caused by deformation.

[0091] In S103, image content repair processing is performed according to the gap region between the adjusted second grids to obtain the target image.

[0092] Exemplarily, considering that during the process of adjusting the second grid in the deformed grid model, there may be a gap between two second grids. By performing image content repair processing on the gap region, the final target image becomes more complete and natural.

[0093] In some embodiments, the first grid in the grid model can be divided into at least two first sub-grids, and each first sub-grid contains at least one coincident vertex that coincides with the vertex in the first grid; then the second grid in the deformed grid model includes a second sub-grid obtained by deforming the first sub-grid in the first grid.

[0094] In an example, taking the grid model including a triangular grid model and the first grid including a triangular grid as an example for exemplary illustration, please refer to Figure 4 , Figure 4 (a) shown therein is the first grid. Based on the 3 midlines of the triangle, the triangle can be divided into 6 sub-triangles. Figure 4In (b), the second grid is obtained by deforming the first grid shown in (a). For example, after performing deformation processing such as translation, rotation, or scaling on the triangle shown in (a), a triangle as shown in (b) can be obtained. The triangle shown in (b) contains six sub-triangles corresponding to the six sub-triangles in the triangle shown in (a). It can be seen that each sub-triangle contains a coincident vertex that coincides with the vertex in the triangle.

[0095] Then, during the process of adjusting the second grid, the electronic device can adjust the positions of the other vertices in the second sub-grid except for the coincident vertices according to the difference between the second sub-grid in the second grid and the corresponding first sub-grid, so that the scales of the same local image content of the target object represented by the adjusted second grid and the corresponding first grid remain unchanged. In this embodiment, by adjusting the vertex positions according to the difference between the second sub-grid and the first sub-grid, it can be ensured that the adjusted second grid is consistent with the first grid in terms of the scale of the local image content, which helps to maintain the proportion and details of a specific local area of the target object and makes the editing result more accurate and natural. And by adjusting the positions of the other vertices except for the coincident vertices, the coherence between the second sub-grid and the first sub-grid after the deformation processing can be maintained, which helps to avoid discontinuous or fractured deformation effects and makes the edited grid model smoother and more continuous in shape.

[0096] Among them, the adjustment of the positions of the other vertices in the second sub-grid except for the coincident vertices can be achieved by adjusting at least one of the sides in the second sub-grid and the included angle corresponding to the coincident vertices in the second sub-grid.

[0097] In a possible implementation manner, when adjusting the vertex positions of the second sub-grid, the side length of the side in the second sub-grid can be adjusted according to the difference between the side in the second sub-grid and the corresponding side in the first sub-grid, so as to achieve the adjustment of the vertex positions of the second sub-grid. In this embodiment, by adjusting the side length of the side according to the difference between the first sub-grid and the second sub-grid, the sides in the second sub-grid have similar sizes to the sides in the first sub-grid, which helps to maintain the consistency of the geometric shape of the local area of the target object and makes the editing result more natural and accurate.

[0098] Exemplarily, if the side in the second sub-grid is longer than the corresponding side in the first sub-grid, that is, in the scenario of stretching or enlarging the target object, the electronic device can adjust the side length of the side in the second sub-grid to be the same as the side length of the corresponding side in the first sub-grid, ensuring that the proportion and shape of the object are retained during the editing process and making the editing result more natural and realistic.

[0099] If the edges in the second sub-grid are less than or equal to the corresponding edges in the first sub-grid, that is, in the scenario of compressing or shrinking the target object, the electronic device can shorten the edge lengths of the edges in the second sub-grid according to a first preset ratio, so as to shrink the object according to a preset ratio, keep the overall shape and ratio of the object unchanged, and make the editing result more accurate and in line with expectations.

[0100] In another possible implementation, when adjusting the vertex positions of the second sub-grid, the electronic device can adjust the angle of the angle corresponding to the coincident vertices in the second sub-grid according to the difference between the angle corresponding to the coincident vertices in the second sub-grid and the angle corresponding to the coincident vertices in the corresponding first sub-grid, so as to adjust the vertex positions of the second sub-grid. In this embodiment, by adjusting the angle corresponding to the coincident vertices in the second sub-grid, the adjusted second sub-grid can be made more conform to the shape and structure of the first sub-grid, thus maintaining the consistency of the overall object.

[0101] If the angle corresponding to the coincident vertices in the second sub-grid is greater than the angle corresponding to the coincident vertices in the corresponding first sub-grid, that is, in the scenario of stretching or enlarging the target object, the electronic device can adjust the angle corresponding to the coincident vertices in the second sub-grid to the angle corresponding to the coincident vertices in the corresponding first sub-grid; this embodiment realizes keeping the angle of the coincident vertices unchanged, which can ensure that the ratio and shape of the object are retained, and make the editing result more natural and real. During the process of stretching or enlarging the object, keeping the angles of the coincident vertices the same can ensure that the object is stretched or enlarged evenly in all directions, avoiding unreasonable deformations and size changes.

[0102] If the angle corresponding to the coincident vertices in the second sub-grid is less than or equal to the angle corresponding to the coincident vertices in the corresponding first sub-grid, that is, in the scenario of compressing or shrinking the target object, the electronic device can reduce the angle of the angle corresponding to the coincident vertices in the second sub-grid according to a second preset ratio; in this embodiment, by reducing the angle corresponding to the coincident vertices in the second sub-grid according to the second preset ratio, the change of the object shape can be controlled. During the process of compressing or shrinking the object, keeping the reduction ratio of the angles of the coincident vertices the same can ensure that the object is reduced evenly in all directions, avoiding unreasonable deformations.

[0103] It can be understood that the above-mentioned edge length adjustment method and the above-mentioned angle adjustment method can be selected one by one or comprehensively, and this embodiment does not make any restrictions on this.

[0104] In an example, please refer to Figure 5 , Figure 5 In (a) shown in, each sub-triangle is for Figure 4Schematic diagram of the result of performing the above-mentioned side length adjustment method on the side lengths of each sub-triangle in (b). Figure 5 Each sub-triangle shown in (b) is for Figure 5 the coincident vertices of the sub-triangles shown in (a) (i.e., Figure 5 the vertices in the circles in (b)) corresponding to the included angles to perform the above-mentioned included angle adjustment method. Finally, the adjusted result shown in (c) can be obtained. Figure 5 in

[0105] The above-mentioned side length adjustment method can be expressed by the following formula:

[0106]

[0107] where a is the side length of the side in the first sub-grid; b is the side length of the side corresponding to a in the second sub-grid; β is the above-mentioned first preset ratio, such as β = 0.5, 0.4 or 0.6; x new is the side length of the adjusted side in the second sub-grid.

[0108] The above-mentioned included angle adjustment method can be expressed by the following formula:

[0109]

[0110] where c is the angle of the included angle corresponding to the coincident vertex in the first sub-grid; d is the angle of the included angle corresponding to the coincident vertex in the second sub-grid; α is the above-mentioned second preset ratio, such as α = 0.5, 0.4 or 0.6; y new is the adjusted included angle corresponding to the coincident vertex in the second sub-grid.

[0111] Exemplarily, after the second sub-grid is adjusted, there may be gaps between the second sub-grids. Therefore, it is necessary to perform image content patching processing according to the gap areas between the adjusted second sub-grids to obtain the target image.

[0112] In one example, please refer to Figure 6 , Figure 6 which shows the schematic diagram obtained after deforming the tree shown in Figure 3 and performing the above-mentioned second grid adjustment. It can be seen that there are some gap areas between the adjusted second grids ( Figure 6 the black areas in are the gap areas), and patching processing is required.

[0113] In some embodiments, after adjusting the second grid based on the above implementation, the electronic device can obtain the image to be repaired represented by the grid model formed by the adjusted second grid; wherein, the size of the grid model formed by the adjusted second grid is the same as the size of the deformed grid model; then an initial mask image with the same size as the image to be repaired is generated, and the pixel values of the pixels in the initial mask image are all the first value; and according to the positions of the vertices of the adjusted second grid, the gap region in the grid model formed by the adjusted second grid is identified; the pixel values of the pixels in the initial mask image corresponding to the gap region are modified from the first value to the second value to obtain a mask image for representing the gap region; finally, according to the mask image for representing the gap region, the image to be repaired is repaired to obtain the target image. In this embodiment, by automatically obtaining the mask image for representing the gap region, it can help identify the region to be repaired and the non-region to be repaired, which provides a basis for subsequent repair processing, saves the time of manually marking the mask, and then by performing repair processing according to the mask image, seamless connection between the gap region and surrounding pixels can be achieved, making the repair result more natural and realistic.

[0114] Wherein, in this embodiment, no restrictions are imposed on the first value and the second value in the mask image. For example, the first value is 0 and the second value is 1.

[0115] Exemplarily, during the repair processing, the electronic device can determine the region to be repaired included in the image to be repaired according to the mask image for representing the gap region; then refer to the region for displaying the local image content of the target object around the region to be repaired, and perform repair processing on the region to be repaired to obtain the target image. In this embodiment, according to the mask image for representing the gap region, the electronic device can accurately determine the region to be repaired, which helps to avoid erroneously modifying the region that does not need to be repaired during the repair processing, improving the accuracy and efficiency of the repair processing; and by referring to the surrounding local image content, it can ensure that the repaired image is similar to the surrounding region in terms of color, texture and other features, achieving a natural and seamless repair effect and improving the quality of the image.

[0116] In an example, please refer to Figure 7 , Figure 7 The tree shown is the effect obtained after performing image content repair processing on the gap region in the tree shown in Figure 6 . It can be seen that Figure 7 Compared with Figure 1 , there is no distortion effect on the left and right sides, and the display is more natural.

[0117] In some embodiments, the electronic device can also perform other image editing functions, such as image matting. The user can select an image and the matting area in the image according to actual needs. For example, the selected image by the user can be displayed in the interaction interface, and then the user can select the matting area in the image through operations such as bounding box selection and clicking in the interaction interface. For the convenience of distinction, the image selected by the user is hereinafter referred to as the second selected image, and the matting area selected by the user is referred to as the selected area.

[0118] After the electronic device obtains the second selected image and determines the selected area in the second selected image, it can perform matting processing on the selected area in the second selected image, so as to obtain a to-be-processed image with the same size as the second selected image. The to-be-processed image includes a to-be-complemented area with the same position and size as the selected area of the second selected image. Exemplarily, an image segmentation algorithm can be used to implement the matting processing on the selected area in the second selected image. The image segmentation algorithm includes but is not limited to semantic segmentation methods based on deep learning. Then, the electronic device can detect the color around the to-be-complemented area in the to-be-processed image and perform complement processing on the to-be-complemented area in the to-be-processed image according to the detection result.

[0119] If the detection result indicates that the color around the to-be-complemented area in the to-be-edited image is a specified color, the electronic device can perform complement processing on the to-be-complemented area in the to-be-edited image according to the specified color; wherein, the specified color includes solid color or gradient color.

[0120] Exemplarily, if the detection result indicates that the color around the to-be-complemented area in the to-be-edited image is a solid color, the to-be-complemented area in the to-be-edited image is complemented according to the solid color, so as to achieve rapid filling and complement of the to-be-complemented area, make the to-be-complemented area consistent with the surrounding area, and achieve a natural fusion effect.

[0121] In one example, according to the detection result, the solid color information around the to-be-complemented area can be obtained, such as the color value in the RGB or HSV color space; then the obtained solid color information is applied to the to-be-complemented area to make it consistent with the surrounding area. For example, an algorithm similar to the fill tool can be used to set the pixel values in the to-be-complemented area to the color represented by the solid color information. And, in order to make the transition between the complemented area and the surrounding area natural, the to-be-complemented area can also be smoothed, such as using a gradient-based smoothing algorithm, a texture synthesis-based method, etc., so as to eliminate the jaggedness and defects at the edge of the complemented area and make the complemented image more natural.

[0122] Exemplarily, if the detection result indicates that the color around the area to be completed in the image to be edited is a gradient color, the area to be completed in the image to be edited is completed according to the gradient color. In this embodiment, by using gradient color filling, natural transition between the area to be completed and the surrounding area can be achieved, and the color within the area to be completed will gradually change to the surrounding color, making the completion result smoother and more coherent.

[0123] In one example, according to the detection result, the color gradient range around the area to be completed can be determined. For example, by analyzing the color change situation of the surrounding area, the starting color and ending color of the gradient color can be determined; then, according to the gradient color range, a gradient color sample including the starting color and ending color is created. For example, linear gradient, radial gradient or other gradient types can be used and selected according to requirements; finally, the gradient color sample is applied to the area to be completed to achieve the completion effect, and it can be adjusted according to parameters such as the direction, length and color breakpoints of the gradient color to obtain a better completion result. Moreover, in order to make the transition between the area to be completed and the surrounding area more natural, some smoothing processing can also be performed. For example, a blur algorithm or a blending algorithm can be used to eliminate the obvious boundary between the completed area and the surrounding area, making the completion result more coherent.

[0124] In some embodiments, if the detection result indicates that the color around the area to be completed in the image to be edited is not a specified color, that is, neither a solid color nor a gradient color, the electronic device can output reference area selection information to prompt the user to select a reference area in the image to be processed for completing the area to be completed. The user selects a suitable reference area in the image to be processed according to the reference area selection prompt provided by the electronic device. The reference area should be similar to the color and texture around the area to be completed to ensure the naturalness of the completion effect.

[0125] In a possible implementation manner, the completion process can be performed by the electronic device.

[0126] In response to the user's selection operation of the reference area, the electronic device can generate the first completion content of the area to be completed according to the reference area, and complete the area to be completed in the image to be edited based on the first completion content of the area to be completed. In this embodiment, by the user selecting the reference area, the area range that the completion algorithm needs to search can be reduced, thereby improving the completion efficiency.

[0127] Exemplarily, the electronic device can extract texture information from the reference area, and then generate the first completion content of the area to be completed based on the texture information.

[0128] In response to the user not selecting a reference area within a preset duration, another method can be adopted to generate the second completion content for the area to be completed. The electronic device can generate the second completion content for the area to be completed based on the image content around the area to be completed in the image to be edited, and perform a completion process on the area to be completed in the image to be edited based on the second completion content of the area to be completed. In this embodiment, the second completion content is automatically generated according to the image content of the surrounding area, and the user does not need to manually select a reference area, saving the user's time and operation cost. Referring to the image content around the area to be completed can also make the completion edge smoother and more natural.

[0129] Exemplarily, the electronic device can extract information such as the texture, color, and shape of the surrounding area of the area to be completed, and then generate the second completion content for the area to be completed according to the extracted information and use it to fill the area to be completed.

[0130] In another possible implementation manner, when the processing resources of the electronic device are limited, to improve the processing efficiency, it can be executed by a server communicatively connected to the electronic device.

[0131] Exemplarily, in response to the user's selection operation on the reference area, the electronic device can send a first completion request carrying the reference area and the image to be edited to the server, so that the server generates the first completion content for the area to be completed according to the reference area, performs a completion process on the area to be completed in the image to be edited based on the first completion content of the area to be completed, and returns the image after the completion process.

[0132] In response to the user not selecting a reference area within a preset duration, send a second completion request carrying the image to be edited to the server, so that the server generates the second completion content for the area to be completed according to the image content around the area to be completed in the image to be edited, performs a completion process on the area to be completed in the image to be edited based on the second completion content of the area to be completed, and returns the image after the completion process.

[0133] Among them, the above-mentioned preset duration can be specifically set according to the actual application scenario, and this embodiment does not make any restrictions on this.

[0134] In an example, please refer to Figure 8 , in Figure 8 after performing matte extraction on the rightmost flower in the right figure and performing a completion process using the above completion scheme, the left figure in Figure 8 can be obtained.

[0135] The various technical features in the above embodiments can be combined arbitrarily as long as there is no conflict or contradiction between the combinations of the features. However, due to space limitations, they are not described one by one. Therefore, any arbitrary combination of the various technical features in the above embodiments also belongs to the scope disclosed in this specification.

[0136] Figure 9 is a schematic structural diagram of a device provided by an exemplary embodiment. Please refer to Figure 9 , at the hardware level, the device includes a processor 902, an internal bus 904, a network interface 906, a memory 908, and a non-volatile memory 910. Of course, other hardware may also be included. One or more embodiments of this specification can be implemented in a software manner. For example, the processor 902 reads the corresponding computer program from the non-volatile memory 910 into the memory 908 and then runs it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementation manners, such as logic devices or a combination of software and hardware, etc. That is, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or a logic device.

[0137] Please refer to Figure 10 , the image processing device can be applied to a device as shown in Figure 9 to implement the technical solutions of this specification. Among them, the image processing device may include:

[0138] A deformation processing module 201, configured to perform deformation processing on the mesh model corresponding to the target object in response to a deformation instruction for the target object in the to-be-edited image, to obtain a deformed mesh model; wherein, the mesh model is composed of a first mesh for representing the local image content of the target object; the deformed mesh model includes a second mesh obtained by performing deformation processing on the first mesh;

[0139] A mesh adjustment module 202, configured to adjust the positions of the vertices in the second mesh according to the difference between the second mesh and the corresponding first mesh, so that the scales of the same local image content of the target object represented by the adjusted second mesh and the corresponding first mesh remain unchanged;

[0140] A patching processing module 203, configured to perform patching processing on the image content according to the gap area between the adjusted second meshes to obtain a target image.

[0141] In some embodiments, the first mesh is divided into at least two first sub-meshes, and each first sub-mesh includes at least one coincident vertex that coincides with the vertex in the first mesh. The second mesh includes a second sub-mesh obtained by performing deformation processing on the first sub-mesh in the first mesh.

[0142] The mesh adjustment module 202 is specifically configured to: adjust the positions of the other vertices except the coincident vertices in the second sub-mesh according to the difference between the second sub-mesh in the second mesh and the corresponding first sub-mesh;

[0143] The patching processing module 203 is specifically configured to: perform patching processing on the image content according to the gap area between the adjusted second sub-grids to obtain a target image.

[0144] In some embodiments, the grid adjustment module 202 includes an edge adjustment unit and / or an included angle adjustment unit.

[0145] The edge adjustment unit is configured to adjust the side length of the edge in the second sub-grid according to the difference between the edge in the second sub-grid and the corresponding edge in the first sub-grid.

[0146] The included angle adjustment unit is configured to adjust the angle of the included angle corresponding to the coincident vertex in the second sub-grid according to the difference between the included angle corresponding to the coincident vertex in the second sub-grid and the included angle corresponding to the coincident vertex in the corresponding first sub-grid.

[0147] In some embodiments, the edge adjustment unit is specifically configured to: if the edge in the second sub-grid is greater than the corresponding edge in the first sub-grid, adjust the side length of the edge in the second sub-grid to be the same as the side length of the corresponding edge in the first sub-grid; if the edge in the second sub-grid is less than or equal to the corresponding edge in the first sub-grid, shorten the side length of the edge in the second sub-grid according to a first preset ratio.

[0148] In some embodiments, the included angle adjustment unit is configured to: if the included angle corresponding to the coincident vertex in the second sub-grid is greater than the corresponding included angle in the first sub-grid, adjust the included angle corresponding to the coincident vertex in the second sub-grid to be the same as the corresponding included angle in the first sub-grid; if the included angle corresponding to the coincident vertex in the second sub-grid is less than or equal to the corresponding included angle in the first sub-grid, reduce the angle of the included angle corresponding to the coincident vertex in the second sub-grid according to a second preset ratio.

[0149] In some embodiments, the patching processing module 203 is specifically configured to: obtain a to-be-patched image represented by a mesh model formed by an adjusted second mesh; wherein, the size of the mesh model formed by the adjusted second mesh is the same as the size of the deformed mesh model; generate an initial mask image having the same size as the to-be-patched image, and pixel values of pixels in the initial mask image are all a first value; identify a gap region in the mesh model formed by the adjusted second mesh according to positions of vertices of the adjusted second mesh; modify pixel values of pixels corresponding to the gap region in the initial mask image from the first value to a second value to obtain a mask image for representing the gap region; and perform patching processing on the to-be-patched image according to the mask image for representing the gap region to obtain a target image.

[0150] In some embodiments, the patching processing module 203 is specifically configured to: determine a to-be-patched region included in the to-be-patched image according to the mask image for representing the gap region; and perform patching processing on the to-be-patched region with reference to a region for displaying local image content of a target object around the to-be-patched region to obtain a target image.

[0151] In some embodiments, the mesh model includes a triangular mesh model, and the first mesh includes triangular meshes; wherein, the triangular mesh model is obtained by performing triangulation processing on a shape of the target object; and / or the to-be-edited image is obtained by performing foreground segmentation on a first selected image.

[0152] In some embodiments, the apparatus further includes a completion module, configured to obtain a to-be-processed image, where the to-be-processed image is obtained by performing matte extraction on a selected region in a second selected image, the to-be-processed image has the same size as the second selected image, and the to-be-processed image includes a to-be-completed region having the same position and size as the selected region of the second selected image; detect colors around the to-be-completed region in the to-be-processed image, and perform completion processing on the to-be-completed region in the to-be-processed image according to a detection result.

[0153] In some embodiments, the completion module is specifically configured to, if the detection result indicates that a color around the to-be-completed region in the to-be-edited image is a specified color, perform completion processing on the to-be-completed region in the to-be-edited image according to the specified color; wherein, the specified color includes a solid color or a gradient color.

[0154] In some embodiments, the completion module is further configured to: if the result of the detection indicates that the color around the area to be completed in the image to be edited is not the specified color, output reference area selection information to prompt the user to select a reference area in the image to be processed for completing the area to be completed; in response to the user's selection operation on the reference area, send a first completion request carrying the reference area and the image to be edited to the server, so that the server generates first completion content for the area to be completed based on the reference area, perform a completion process on the area to be completed in the image to be edited based on the first completion content, and return the image after the completion process; in response to the user not selecting a reference area within a preset time period, send a second completion request carrying the image to be edited to the server, so that the server generates second completion content for the area to be completed based on the image content around the area to be completed in the image to be edited, perform a completion process on the area to be completed in the image to be edited based on the second completion content, and return the image after the completion process.

[0155] The implementation processes of the functions and roles of each module in the above device are specifically described in detail in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0156] In some embodiments, the embodiments of the present specification further provide an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein, the processor realizes the method described in any one of the above by running the executable instructions.

[0157] In some embodiments, the embodiments of the present specification further provide a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in any one of the above are realized.

[0158] In some embodiments, the embodiments of the present specification further provide a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the above are realized.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. And the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0160] The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or a combination of any several of these devices.

[0161] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0162] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0163] Computer-readable media includes both permanent and non-permanent, removable and non-removable media and can be implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage, quantum memory, graphene-based storage media, or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0164] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, commodity or device comprising the element.

[0165] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0166] The terms used in one or more embodiments of this specification are for the purpose of describing particular embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0167] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0168] The above are only the preferred embodiments of one or more embodiments of this specification and are not intended to limit one or more embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included within the scope of protection of one or more embodiments of this specification.

Claims

1. An image processing method, comprising: responding to a deformation instruction for a target object in an image to be edited, performing a deformation process on a mesh model corresponding to the target object to obtain a deformed mesh model; wherein, the mesh model is composed of a first mesh for representing local image content of the target object; the deformed mesh model includes a second mesh obtained by performing a deformation process on the first mesh; adjusting the positions of the vertices in the second mesh according to the difference between the second mesh and the corresponding first mesh, so that the scales of the same local image content of the target object represented by the adjusted second mesh and the corresponding first mesh remain unchanged; performing an image content repair process according to the gap regions between the adjusted second meshes to obtain a target image.

2. The method according to claim 1, wherein the first mesh is divided into at least two first sub-meshes, and each first sub-mesh contains at least one coincident vertex that coincides with a vertex in the first mesh; the second mesh includes a second sub-mesh obtained by performing a deformation process on the first sub-mesh in the first mesh; The adjusting the positions of the vertices in the second mesh according to the difference between the second mesh and the corresponding first mesh includes: adjusting the positions of the other vertices in the second sub-network except the coincident vertices according to the difference between the second sub-mesh in the second mesh and the corresponding first sub-mesh; The performing an image content repair process according to the gap regions between the adjusted second meshes to obtain a target image includes: performing an image content repair process according to the gap regions between the adjusted second sub-meshes to obtain a target image.

3. The method according to claim 2, wherein the adjusting the positions of the other vertices in the second sub-network except the coincident vertices according to the difference between the second sub-mesh in the second mesh and the corresponding first sub-mesh includes: adjusting the side lengths of the edges in the second sub-mesh according to the difference between the edges in the second sub-mesh and the corresponding edges in the first sub-mesh; and / or adjusting the angles of the angles corresponding to the coincident vertices in the second sub-mesh according to the difference between the angles corresponding to the coincident vertices in the second sub-mesh and the corresponding angles corresponding to the coincident vertices in the first sub-mesh.

4. The method according to claim 3, wherein the adjusting the side lengths of the edges in the second sub-mesh according to the difference between the edges in the second sub-mesh and the corresponding edges in the first sub-mesh includes: if the edge in the second sub-mesh is greater than the corresponding edge in the first sub-mesh, adjusting the side length of the edge in the second sub-mesh to be the same as the side length of the corresponding edge in the first sub-mesh; if the edge in the second sub-mesh is less than or equal to the corresponding edge in the first sub-mesh, shortening the side length of the edge in the second sub-mesh according to a first preset ratio; and / or Adjusting the angle corresponding to the coincident vertex in the second sub-grid according to the difference between the angle corresponding to the coincident vertex in the second sub-grid and the angle corresponding to the coincident vertex in the corresponding first sub-grid includes: If the angle corresponding to the coincident vertex in the second sub-grid is greater than the angle corresponding to the coincident vertex in the corresponding first sub-grid, adjusting the angle corresponding to the coincident vertex in the second sub-grid to the angle corresponding to the coincident vertex in the corresponding first sub-grid; If the angle corresponding to the coincident vertex in the second sub-grid is less than or equal to the angle corresponding to the coincident vertex in the corresponding first sub-grid, reducing the angle corresponding to the coincident vertex in the second sub-grid by a second preset ratio.

5. The method according to claim 1, wherein the mending process of the image content according to the gap area between the adjusted second sub-grids to obtain the target image includes: Obtaining the image to be mended represented by the grid model composed of the adjusted second grids; wherein, the size of the grid model composed of the adjusted second grids is the same as the size of the deformed grid model; Generating an initial mask image with the same size as the image to be mended, and the pixel values of the pixels in the initial mask image are all the first value; Identifying the gap area in the grid model composed of the adjusted second grids according to the positions of the vertices of the adjusted second grids; Changing the pixel values of the pixels corresponding to the gap area in the initial mask image from the first value to the second value to obtain a mask image for representing the gap area; Mending the image to be mended according to the mask image for representing the gap area to obtain the target image.

6. The method according to claim 5, wherein the mending process of the image to be mended according to the mask image for representing the gap area to obtain the target image includes: Determining the area to be mended included in the image to be mended according to the mask image for representing the gap area; Referring to the area for displaying the local image content of the target object around the area to be mended, and mending the area to be mended to obtain the target image.

7. The method according to any one of claims 1 to 6, wherein the mesh model includes a triangular mesh model, and the first mesh includes triangular meshes; wherein, The triangular grid model is obtained by performing triangular meshing on the shape of the target object; and / or The image to be edited is obtained by performing foreground segmentation on the first selected image.

8. The method according to claim 1, further includes: Obtaining an image to be processed, which is obtained by performing matte extraction on a selected area in a second selected image, the image to be processed has the same size as the second selected image, and the image to be processed includes a missing area with the same position and size as the selected area of the second selected image; Detecting the color around the missing area in the image to be processed, and performing a completion process on the missing area in the image to be processed according to the detection result.

9. The method according to claim 8, wherein the completion process of the missing area in the image to be edited according to the detection result includes: If the result of the detection indicates that the color around the area to be completed in the image to be edited is the specified color, complete the area to be completed in the image to be edited according to the specified color; wherein, the specified color includes solid color or gradient color.

10. The method according to claim 9, further comprising: If the result of the detection indicates that the color around the area to be completed in the image to be edited is not the specified color, output reference area selection information to prompt the user to select a reference area in the image to be processed for completing the area to be completed; In response to the user's selection operation of the reference area, send a first completion request carrying the reference area and the image to be edited to the server, so that the server generates first completion content for the area to be completed according to the reference area, complete the area to be completed in the image to be edited based on the first completion content, and return the image after the completion process; In response to the user not selecting a reference area within a preset time period, send a second completion request carrying the image to be edited to the server, so that the server generates second completion content for the area to be completed according to the image content around the area to be completed in the image to be edited, complete the area to be completed in the image to be edited based on the second completion content, and return the image after the completion process.

11. An image processing apparatus, comprising: A deformation processing module, configured to, in response to a deformation instruction for a target object in an image to be edited, perform deformation processing on a mesh model corresponding to the target object to obtain a deformed mesh model; wherein, the mesh model is composed of a first mesh for representing local image content of the target object; the deformed mesh model includes a second mesh obtained by performing deformation processing on the first mesh; A mesh adjustment module, configured to adjust the positions of the vertices in the second mesh according to the difference between the second mesh and the corresponding first mesh, so that the scales of the same local image content of the target object represented by the adjusted second mesh and the corresponding first mesh remain unchanged; A patching processing module, configured to perform patching processing on the image content according to the gap area between the adjusted second meshes to obtain a target image.

12. An electronic device, comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor realizes the method according to any one of claims 1 to 10 by running the executable instructions.

13. A computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are realized.

14. A computer program product, comprising a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are realized.

Images