Image processing method and device

Abstract

The embodiment of the present application provides an image processing method and device, if the object in the first image is located at the edge of the first image, a compensation image is obtained, and the compensation image is composed of at least the first image and the edge prediction image. Then, the object contained in the compensation image is shrunk and deformed, and in the compensation image, the edge of the first image is not the edge of the compensation image, so after the pixels of the edge of the first image are stretched to the middle of the compensation image, the edge of the first image has more edge information (i.e., the edge prediction image) that can be filled, which will not cause abnormal deformation of the edge position of the image, thereby improving the display effect of the image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to an image processing method and device. Background Art

[0002] With the continuous development of image processing applications, technologies for shrinking and parallel processing objects contained in images are emerging one after another, for example, performing face slimming operations on faces contained in images and performing body slimming operations on bodies contained in images.

[0003] After the objects contained in the image are subjected to shrinkage deformation processing, abnormally deformed images will be displayed at the edge positions of the image, resulting in poor image processing effect. Summary of the invention

[0004] In view of this, the present application provides an image processing method and device to at least solve the problem that an abnormally deformed image is displayed at the edge of the image after shrinkage and deformation processing is performed on an object contained in the image.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0007] In a case where it is detected that a shrinking and deformation operation is performed on an object included in the first image, obtaining a position of the object in the first image;

[0008] In the case where the position represents that the edge of the object and the first image meets a first distance condition, a compensation image is obtained; the compensation image is composed of at least the first image and the edge prediction image, and the edge of the object and the compensation image meets a second distance condition;

[0009] Performing shrinkage deformation on the object contained in the compensated image to obtain a second image;

[0010] The image display interface is controlled to at least display an image of a target area in the second image, where the target area is an area including the object.

[0011] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0012] A first acquisition module, configured to acquire a position of the object in the first image when detecting that a shrinkage deformation operation is performed on the object contained in the first image;

[0013] A second acquisition module is used to obtain a compensation image when the position represents that the edge of the object and the first image meets a first distance condition; the compensation image is composed of at least the first image and the edge prediction image, and the edge of the object and the compensation image meets a second distance condition;

[0014] A shrinkage deformation module, used for shrinking and deforming the object contained in the compensation image to obtain a second image;

[0015] A control display module is used to control the image display interface to at least display an image of a target area in the second image, where the target area is an area including the object.

[0016] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0017] processor;

[0018] a memory for storing instructions executable by the processor;

[0019] The processor is configured to execute the instructions to implement the image processing method as described in the first aspect.

[0020] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method as described in the first aspect.

[0021] According to the fifth aspect of the embodiments of the present disclosure, a computer program product is provided, which can be directly loaded into the internal memory of a computer. The memory is a memory included in the above-mentioned electronic device and contains software code. After the computer program is loaded and executed, it can implement the image processing method described in the first aspect.

[0022] It can be seen from the above technical solution that the embodiment of the present application provides an image processing method, if the object in the first image is located at the edge of the first image, a compensation image is obtained, and the compensation image is composed of at least the first image and the edge prediction image. Then, the object contained in the compensation image is shrunk and deformed, and the edge of the first image in the compensation image is not the edge of the compensation image, so after the pixels of the edge of the first image are stretched to the middle of the compensation image, the edge of the first image has more edge information (i.e., the edge prediction image) that can be filled, which will not cause abnormal deformation of the edge position of the image, thereby improving the display effect of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of a process of performing a shrinking and deforming operation on an object contained in an image in the related art;

[0025] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of possible situations of objects included in the first image provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of the relationship between the first image and the predicted edge image provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of a process of shrinking and deforming an object included in a compensation image provided by an embodiment of the present application;

[0029] Figure 6 A schematic diagram of the correspondence between standard key points of a face image before shrinkage and deformation and a face image after shrinkage and deformation in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the relationship between a plurality of standard key points constituting the contour and the first image provided in an embodiment of the present application;

[0031] Figure 8 A process diagram of obtaining an edge prediction image from a third image provided in an embodiment of the present application;

[0032] Fig. 9 A structural diagram of an image processing device provided in an embodiment of the present application;

[0033] Fig.10 The diagram is a structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The embodiments of the present application provide an image processing method and device. Before introducing the technical solution provided by the embodiments of the present application, the related technologies involved in the present application are first described.

[0036] In the related art, during the process of performing a shrinkage and deformation operation on an object contained in an image, if the object is located at the edge of the image, when the object is shrunk and deformed, since there is less pixel information at the edge of the image, after the pixels at the edge of the image are stretched to the middle of the image, there is no more edge information to fill in the pixels at the edge of the image. At this time, this part of the image can be filled with a local area image close to the edge object, or, alternatively, this part of the image can be filled with a preset image.

[0037] like Figure 1 As shown, it is a schematic diagram of the process of performing a shrinkage deformation operation on an object contained in an image in the related art.

[0038] Exemplarily, the electronic device that performs the shrinking and deformation operation on the object contained in the image can be any electronic product that can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, for example, a mobile phone, laptop computer, tablet computer, PDA, personal computer, wearable device, smart TV, PAD, etc.

[0039] Figure 1 The electronic device is taken as an example of a mobile phone.

[0040] Exemplarily, the electronic device 11 can capture an image of the user 12 through a camera configured thereon to obtain an original image 13, and shrink and deform the face 10 of the user 12 based on the original image 13. For example, the face of the user 12 can be shrunk and deformed based on standard key points of the face in the captured image, so that the electronic device 11 displays an image with beautification and face slimming on the display interface 16 of the screen.

[0041] Specifically, the electronic device 11 collects the original image 13 of the user 12, and performs shrinkage deformation processing on the original image 13. The dotted box indicated by the arrow illustrates the image processing process of the original image 13 by the electronic device 11, and the process can be completed by an application with a beauty function installed on the electronic device 11. Among them, the original image 13 collected by the electronic device 11 can include a face image 10 corresponding to the face of the user 12. Here, the face image 10 can be used as an object. It can be understood that if it is necessary to shrink and deform other parts of the user 12 such as arms, eyes, etc., the arms and eyes can be used as objects accordingly. Then, the electronic device 11 can use the application with a beauty function to shrink and deform the face image 10 in the original image 13. The shrinkage deformation processing here is specifically a face-thinning processing, so that a face-thinning image 14 or a face-thinning image 15 can be obtained. The face-thinning image 14 or the face-thinning image 15 includes a face image 141 or a face image 151 corresponding to the face of the user 12 after face-thinning. Next, the electronic device 11 may display the face-thinning image 14 or the face-thinning image 15 on the display interface 16 , so that the user 12 can see the face image after face-thinning on the display interface 16 .

[0042] However, since the facial image included in the original image 13 is located at the edge of the original image 13, when the facial image is shrunk and deformed, the facial image located at the edge needs to be stretched upward. Since the edge pixel information of the original image 13 is relatively small, after the facial image is stretched upward, the pixels at the edge of the image have no more edge information to be filled. The face-slimming image 14 fills the edge position of the preset image (taking the preset image as a black image as an example) with the preset image (for example, the preset image is a black image); the face-slimming image 15 fills the chin located at the edge position of the original image 13 with the edge position of the face-slimming image 15, so the chin of the face-slimming image 15 is longer.

[0043] In summary, in the related art, when shrinking and deforming an object in an image, if the object is located at the edge of the image, the edge of the image after shrinking and deforming will be abnormally deformed, resulting in poor display effect of the image.

[0044] Based on this, an embodiment of the present application provides an image processing method, in which, if an object in an original image (the original image is referred to as a first image in the embodiment of the present application) is located at the edge of the original image, a compensation image is obtained, and the compensation image is composed of at least the first image and the edge prediction image. The object is not located at the edge of the compensation image; then the object contained in the compensation image is contracted and deformed. Since the object is not located at the edge of the compensation image, after the object contained in the compensation image is contracted and deformed, abnormal deformation will not occur at the edge of the image, thereby improving the display effect of the image.

[0045] The image processing method provided in the embodiment of the present application is described below.

[0046] like Figure 2 As shown, it is a flowchart of the image processing method provided in an embodiment of the present application. The method includes the following steps S21 to S24 during implementation.

[0047] Step S21: when it is detected that a shrinking and deformation operation is performed on an object included in the first image, obtaining a position of the object in the first image.

[0048] Exemplarily, the first image may be an original image acquired by the electronic device in real time, or may be an image stored by the electronic device.

[0049] Exemplarily, the image processing method provided in the embodiment of the present application may be a processing method for images containing objects in a video, so as to ensure that the edges of a certain frame of video image will not be abnormally deformed during a live broadcast or video call.

[0050] Exemplarily, the image processing method provided in the embodiment of the present application may be a processing method for an image containing an object (the image is not an image contained in a video).

[0051] Exemplarily, a shrinking and deforming operation may be performed on an object included in the first image by a client running in an electronic device.

[0052] Exemplarily, the client may be an application client or a web client.

[0053] Exemplarily, the object may be any one of the targets to be reduced, such as a face, an arm, a leg, a nose, a mouth, etc., or the object may be a local area of ​​the target to be reduced. The objects included in the first image are described below with reference to examples.

[0054] like Figure 1 As shown, if the first image is the original image 13, the object contained in the first image is the entire face.

[0055] like Figure 3 , which is a schematic diagram of possible situations of objects contained in the first image provided in an embodiment of the present application.

[0056] like Figure 3 As shown, if the first image is image 31 , then the object contained in the first image is the half face of the person framed by a dotted line in image 31 .

[0057] In summary, the object contained in the first image mentioned in the embodiment of the present application refers to a target located in the first image and needs to be shrunk and deformed.

[0058] Step S22: obtaining a compensated image when the position representing the object and the edge of the first image satisfy a first distance condition.

[0059] The compensated image is composed of at least the first image and an edge prediction image, and the object and the edge of the compensated image meet a second distance condition.

[0060] Exemplarily, the size of the compensation image is equal to the sum of the size of the first image and the size of the edge prediction image. Exemplarily, the size of the compensation image is less than or equal to the sum of the size of the first image and the size of the edge prediction image.

[0061] The size of the compensation image may be determined based on actual conditions and will not be described here.

[0062] The first distance condition is described below.

[0063] Since the standard object corresponding to the object is needed, the standard object corresponding to the object is explained first.

[0064] Exemplarily, the object contained in the first image may be a local area of ​​a target, for example, Figure 3 The object contained in the image 31 shown is a partial area of ​​a human face. In the embodiment of the present application, the target corresponding to the "object" is referred to as a standard object. If the object contained in the image 31 is half a human face, the standard object corresponding to the object is the entire human face; if the object contained in the original image 13 is the entire human face, the standard object corresponding to the object is the object itself.

[0065] Exemplarily, the first distance condition includes: the standard object corresponding to the object is cut off by the edge of the first image. Figure 3 As shown, the entire face is cut off by the edge of image 31.

[0066] Exemplarily, the first distance condition includes: the standard object corresponding to the object is not cut off by the edge of the first image (such as Figure 1 The original image shown contains a human face), and the standard object is located at the edge of the first image.

[0067] The second distance condition is described below.

[0068] The second distance condition at least includes: the object in the compensated image is not located at the edge of the compensated image.

[0069] Since the object contained in the compensated image is not located at the edge of the compensated image, performing shrinkage deformation processing on the object in the compensated image will not cause abnormal shrinkage deformation at the edge of the image.

[0070] The edge prediction image is described below.

[0071] If the object and the target edge of the first image meet the first distance condition, then the edge prediction image includes an image located outside the target edge of the first image.

[0072] Exemplarily, the first image may include a plurality of edges, and the target edge includes one or more edges.

[0073] Exemplarily, the number of edges included in the first image is related to the shape of the first image. For example, the number of edges included in the first image is equal to the number of sides included in the shape of the first image. For example, if the first image is a rectangle, the number of edges included in the first image is 4.

[0074] Exemplarily, the shape of the first image may be a triangle, a quadrilateral, a pentagon, etc. The embodiment of the present application does not describe the shape of the first image.

[0075] The following description is made by taking the first image as a rectangle as an example. The number of edges included in the first image is 4, which are the first edge, the second edge, the third edge and the fourth edge. Exemplarily, the object included in the first image satisfies the first distance condition with at least one edge of the first image.

[0076] like Figure 4 , which is a schematic diagram of the relationship between the first image and the predicted edge image provided in an embodiment of the present application.

[0077] Figure 4 In , if the first image is image 31, then the object contained in the first image and the first edge (that is, the target edge is the first edge) meet the first distance condition, and the edge prediction image can be as follows Figure 4 The edge prediction image 41 is shown.

[0078] Figure 4 In the example, if the first image is the original image 13, then the object contained in the first image and the fourth edge (that is, the target edge is the fourth edge) meet the first distance condition, and the edge prediction image can be as follows: Figure 4 An edge predicted image 42 is shown.

[0079] The compensated image including the image 31 and the edge prediction image 41 is as follows: Figure 4 The compensated image 43 shown. The compensated image including the original image 13 and the edge prediction image 42 is shown as Figure 4 Compensated image 44 is shown.

[0080] The dotted lines in the compensated image 43 and the compensated image 44 represent that the compensated images include the first image and the edge prediction image, and the dotted lines may not exist in actual applications.

[0081] Step S23: shrinking and deforming the object contained in the compensated image to obtain a second image.

[0082] The compensated image contains the same object as the first image.

[0083] like Figure 5 , which is a schematic diagram of the process of shrinking and deforming the object contained in the compensation image provided by an embodiment of the present application.

[0084] Figure 5 The compensation image 44 is taken as an example for explanation.

[0085] The face included in the compensated image 44 is contracted and deformed to obtain a second image 51 .

[0086] The first region above the dotted line in the compensated image 44 is where the first image is located, and the second region below the dotted line is where the edge prediction image is located. The first region corresponds to the first region, and the second region corresponds to the west of the dotted line in the second image 51.

[0087] It can be seen from the middle of the dotted line in the second image 51 that the neck part below the face is stretched up. Since the edge of the first image in the compensated image is not the edge of the compensated image, after the edge pixels of the first image are stretched to the middle of the compensated image, the edge of the first image has more edge information (i.e., edge prediction image) to fill in, so there will be no abnormal deformation.

[0088] Step S24: controlling the image display interface to at least display an image of a target area in the second image, where the target area is an area including the object.

[0089] Exemplarily, the target area is a local area or the entire area where the second image is located.

[0090] Exemplarily, the size of the compensation image and the second image is the sum of the size of the first image and the edge prediction image. The compensation image includes two areas, namely, the first area where the first image is located and the second area where the edge prediction image is located. Since the second image corresponds to the compensation image, the second image also includes the first area and the second area. Exemplarily, the target area is the first area.

[0091] Exemplarily, if the size of the compensated image is larger or smaller than the sum of the sizes of the first image and the edge prediction image, the compensated image includes a first region corresponding to the first image and a second region corresponding to the edge prediction image. The image content in the first region is the image content of the first image, and the image content in the second region is the image content in the edge prediction image. Since the second image corresponds to the compensated image, the second image also includes the first region and the second region. Exemplarily, the target region is the first region.

[0092] Exemplarily, the image in the target area may be enlarged or reduced to the size of the first image and then displayed.

[0093] The embodiment of the present application provides an image processing method, if the object in the first image is located at the edge of the first image, a compensation image is obtained, and the compensation image is composed of at least the first image and the edge prediction image. Then, the object contained in the compensation image is shrunk and deformed, and the edge of the first image in the compensation image is not the edge of the compensation image, so after the pixels of the edge of the first image are stretched to the middle of the compensation image, the edge of the first image has more edge information (i.e., the edge prediction image) that can be filled, which will not cause abnormal deformation of the edge position of the image, thereby improving the display effect of the image.

[0094] In an optional implementation, the image processing method provided in an embodiment of the present application includes a step of determining that the position representing the object and the edge of the first image satisfy a first distance condition. There are multiple ways to implement this step. An embodiment of the present application provides but is not limited to the following method, which includes steps A1 to A2.

[0095] Step A1: Determine the outline of a standard object corresponding to the object included in the first image, where the object included in the first image is a partial area or the entire area of ​​the standard object.

[0096] The relationship between the object and the standard object can be found in the above description and will not be repeated here.

[0097] Exemplarily, if the standard object corresponding to the object is located in the first image, the outline of the standard object is the outline of the object; if the standard object corresponding to the object is cut off by the edge of the first image, the outline of the standard object includes the outline of the object.

[0098] Step A2: when the contour satisfies a preset condition, determining that the position represents that the edge of the object and the first image satisfy a first distance condition.

[0099] The preset conditions include: a part of the contour is located outside the first image, and / or the contour is completely located in the first image, and a distance between the part of the contour and a boundary of the first image is less than or equal to a first threshold.

[0100] Exemplarily, a part of the contour is located outside the first image, that is, the standard object is cut off by the edge of the first image.

[0101] Exemplarily, the first threshold is determined based on the number of stretched pixels included in the shrinkage deformation degree parameter corresponding to the shrinkage deformation operation. Exemplarily, the first threshold is less than or equal to the number of stretched pixels.

[0102] Exemplarily, the user may perform a shrinkage deformation operation on an object included in the first image, and during the shrinkage deformation operation, a shrinkage deformation degree parameter may be selected.

[0103] For example, the electronic device may display a shrinkage deformation bar, and the user may drag a shrinkage deformation button on the shrinkage deformation bar. The shrinkage deformation button is located at different positions on the shrinkage deformation bar, and the shrinkage deformation degree of the object is different. Exemplarily, the shrinkage deformation degree parameter includes the number of stretched pixels.

[0104] The number of stretched pixels refers to the number of pixels required to stretch the boundary of the object toward the center of the image during the process of shrinking and deforming the object.

[0105] Exemplarily, the first threshold may be a fixed value, for example, the fixed value may be the maximum value of the number of stretched pixels included in the shrinkage deformation degree parameter corresponding to the shrinkage deformation operation.

[0106] The first threshold can be determined based on actual conditions and is not limited here.

[0107] Exemplarily, the second distance condition includes that the minimum distance between the object and the edge of the compensated image is greater than or equal to the number of stretched pixels.

[0108] In an optional implementation, there are multiple implementation methods for step A1. The embodiment of the present application provides but is not limited to the following method, which includes the steps of: acquiring multiple standard key points of the standard object corresponding to the object contained in the first image to obtain the contour composed of the multiple standard key points.

[0109] In an optional implementation, there are multiple implementations for obtaining multiple standard key points of the standard object corresponding to the object contained in the first image. The embodiment of the present application provides but is not limited to the following method, which includes steps B1 to B2.

[0110] Step B1: Obtain a standard object model corresponding to the target object type to which the object belongs.

[0111] The standard object model includes a plurality of standard key points on the standard object belonging to the target object type, wherein the plurality of standard key points constitute a contour of the standard object.

[0112] Exemplarily, the object type includes, but is not limited to: a face, a nose, a mouth, an arm, a leg, or any other type of target that can be shrunk.

[0113] Step B2: Marking a plurality of standard key points of the standard object corresponding to the object contained in the first image based on the standard object model.

[0114] Among them, each object type may correspond to at least one standard object model, and the standard object model may include a standard key point set of the corresponding standard object. The standard object can be marked by multiple standard key points included in the standard key point set.

[0115] Exemplarily, the standard key points in the standard object before shrinking and deforming correspond to the standard key points of the standard object after shrinking and deforming. During the shrinking and deforming process, the standard key points in the standard object before shrinking and deforming may be moved to obtain the standard key points of the standard object after shrinking and deforming, thereby obtaining the object after shrinking and deforming. The following description is made by taking the object type as a human face as an example.

[0116] like Figure 6 As shown, the standard key points of the face image 61 before shrinkage and deformation and the face image 62 after shrinkage and deformation are one-to-one corresponding. For example, each standard key point is set with a corresponding serial number. For example, the face image 61 before shrinkage and deformation includes 20 standard key points, and the serial numbers of the 20 standard key points are A1, A2, ..., A20; the face image 62 after shrinkage and deformation includes 20 standard key points, and the serial numbers of the 20 standard key points are B1, B2, ..., B20; wherein, the standard key point Ai corresponds to the standard key point Bi, i = 1, 2, ..., 20. During the shrinkage and deformation process, the electronic device can obtain the standard key point Bi of the face image 62 after shrinkage and deformation by moving the position of the standard key point Ai of the face image 61 before shrinkage and deformation.

[0117] In an optional implementation, the method for determining whether the outline of a standard object corresponding to an object contained in a first image is located in the first image includes the following steps C1 and C2.

[0118] Step C1: when at least one of the plurality of standard key points constituting the contour is located outside the first image, determining that a part of the contour is located outside the first image.

[0119] Step C2: when a plurality of standard key points constituting the contour are all located on the first image, determining that the contour is completely located in the first image.

[0120] like Figure 7 , which is a schematic diagram of the relationship between a plurality of standard key points constituting the contour and the first image provided in an embodiment of the present application.

[0121] like Figure 7The first image 71 shown on the left (the image inside the rectangle is the first image) includes a partial face. Among the multiple standard key points of the standard object corresponding to the partial face, 8 standard key points are not located in the first image, that is, located outside the first image. At this time, it is determined that the part of the contour is located outside the first image.

[0122] like Figure 7 The first image 72 shown on the right (the image inside the rectangle is the first image) includes a whole face. A plurality of standard key points of the standard object corresponding to the whole face are all located in the first image, and it is determined that the contour is completely located in the first image.

[0123] In an optional implementation, there are multiple implementations of step S22, and the embodiments of the present application provide but are not limited to the following two.

[0124] The first implementation method of step S22 includes the following steps D11 to D13.

[0125] Step D11: Acquire a third image, wherein the third image includes the edge prediction image.

[0126] For example, if the first image is an image in a video, the object is moving during the process of the electronic device capturing the video, that is, although the object in the first image meets the first distance condition with the edge of the first image, there is a third image in the video, and the object contained in the third image and the edge of the third image do not meet the first distance condition. That is, the object contained in the third image is not located at the edge of the third image. Therefore, the edge prediction image can be obtained from the third image.

[0127] Exemplarily, a user may use an electronic device to take multiple images containing the object, and the third image may exist among the multiple images containing the object.

[0128] Step D12: acquiring the edge prediction image from the third image when the edge between the position representing the object and the first image satisfies a first distance condition.

[0129] Exemplarily, step D12 may include: determining a target boundary of the object, where the target boundary and an edge of the first image satisfy a first distance condition; and obtaining an image located outside the target boundary of the object from the third image.

[0130] like Figure 8 As shown, it is a process diagram of obtaining an edge prediction image from a third image provided by an embodiment of the present application.

[0131] Figure 8The first image is taken as the original image 13 as an example for explanation. The target boundary of the object contained in the first image is as follows: Figure 8 As shown by the dotted line in the original image 13.

[0132] Assuming that the third image is image 81 , the predicted edge image may be the portion framed by a dotted line in image 81 .

[0133] Step D13: splicing the edge prediction image with the first image to obtain the compensated image.

[0134] Exemplarily, step D13 may include: determining, from the first image, a target edge that satisfies a first distance condition from the object; and splicing the edge prediction image at the target edge of the first image.

[0135] Exemplarily, if the third image is not stored in the electronic device, the edge prediction image can be obtained through the second implementation method.

[0136] The second implementation method of step S22 includes the following steps D21 to D22.

[0137] Step D21: repairing the image to be repaired based on the first image to obtain the edge prediction image.

[0138] Image restoration refers to the process of reconstructing the lost or damaged parts of an image, that is, using complex algorithms to replace the lost or damaged image data.

[0139] In the embodiment of the present application, the edge image to be repaired is used as the lost or damaged part of the compensation image to be repaired. The compensation image to be repaired is composed of at least the first image and the edge image to be repaired. The process of performing image repair on the compensation image to be repaired is the process of repairing the edge image to be repaired based on the first image.

[0140] Exemplarily, step D21 includes: determining a target edge that satisfies a first distance condition from the object from the first image; and repairing an image outside the target edge of the first image based on the first image to obtain an edge prediction image. The edge image to be repaired is an image outside the target edge of the first image.

[0141] Step D22: splicing the edge prediction image with the first image to obtain the compensated image.

[0142] The method is described in detail in the embodiments disclosed in the above-mentioned application. The method of the application can be implemented by various forms of devices, so the application also discloses various devices. Specific embodiments are given below for detailed description.

[0143] like Fig. 9 , which is a structural diagram of an image processing device provided in an embodiment of the present application, the image processing device includes: a first acquisition module 91, a second acquisition module 92, a shrinkage and deformation module 93 and a control and display module 94, wherein:

[0144] A first acquisition module 91 is used to acquire the position of the object in the first image when it is detected that a shrinkage deformation operation is performed on the object contained in the first image;

[0145] A second acquisition module 92 is used to obtain a compensation image when the position represents that the edge of the object and the first image meets a first distance condition; the compensation image is composed of at least the first image and the edge prediction image, and the edge of the object and the compensation image meets a second distance condition;

[0146] A shrinkage deformation module 93, configured to shrink and deform the object contained in the compensation image to obtain a second image;

[0147] The control display module 94 is used to control the image display interface to at least display an image of a target area in the second image, where the target area is an area including the object.

[0148] In an optional implementation, the method further includes:

[0149] A first determining module, configured to determine an outline of a standard object corresponding to an object included in the first image, wherein the object included in the first image is a partial area or the entire area of ​​the standard object;

[0150] A second determination module is used to determine that the position representation of the object and the edge of the first image meet a first distance condition when the contour meets a preset condition; the preset condition includes: a part of the contour is located outside the first image, and / or the contour is completely located in the first image, and the distance between the part of the contour and the boundary of the first image is less than or equal to a first threshold.

[0151] In an optional implementation, the first determining module includes:

[0152] The first acquisition unit is used to acquire a plurality of standard key points of the standard object corresponding to the object included in the first image, so as to obtain the contour formed by the plurality of standard key points.

[0153] In an optional implementation, the first acquiring unit includes:

[0154] A first acquisition subunit is configured to acquire a standard object model corresponding to a target object type to which the object belongs, wherein the standard object model includes a plurality of standard key points on the standard object belonging to the target object type, wherein the plurality of standard key points constitute an outline of the standard object;

[0155] A marking subunit is used to mark multiple standard key points of the standard object corresponding to the object contained in the first image based on the standard object model.

[0156] In an optional implementation, the method further includes:

[0157] A third determining module, configured to determine that a part of the contour is located outside the first image when at least one of the plurality of standard key points constituting the contour is located outside the first image;

[0158] The fourth determination module is used to determine that the contour is completely located in the first image when multiple standard key points constituting the contour are all located on the first image.

[0159] In an optional implementation, the second acquisition module includes:

[0160] A second acquisition unit, used to acquire a third image, wherein the third image includes the edge prediction image;

[0161] A fourth acquisition unit, configured to acquire the edge prediction image from the third image when the edge between the position characterizing the object and the first image satisfies a first distance condition;

[0162] The first splicing unit is used to splice the edge prediction image with the first image to obtain the compensated image.

[0163] In an optional implementation, the shrinkage deformation operation corresponds to a shrinkage deformation degree parameter, and the shrinkage deformation operation parameter includes the number of stretched pixels; the second distance condition includes that the minimum distance between the object and the edge of the compensated image is greater than or equal to the number of stretched pixels.

[0164] In an optional implementation, the second acquisition module includes:

[0165] A fifth acquisition unit, configured to repair the image to be repaired based on the first image to obtain the edge prediction image;

[0166] The second splicing unit is used to splice the edge prediction image with the first image to obtain the compensated image.

[0167] In an optional implementation, the target area is a local area or the entire area where the second image is located.

[0168] Fig.10 The diagram is a structural diagram of an electronic device according to an exemplary embodiment.

[0169] like Fig.10 As shown, the electronic device includes but is not limited to: a processor 1001 , a memory 1002 , a network interface 1003 , an I / O controller 1004 and a communication bus 1005 .

[0170] It should be noted that those skilled in the art can understand that Fig.10 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include Fig.10 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0171] Combine the following Fig.10 The components of the electronic device 11 are described in detail:

[0172] The processor 1001 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 1001 may include one or more processing units; optionally, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.

[0173] The processor 1001 may be a central processing unit (CPU), or an application specific integrated circuit ASIC (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0174] The memory 1002 may include a memory, such as a high-speed random access memory (RAM) 1021 and a read-only memory (ROM) 1022, and may also include a large-capacity storage device 1023, such as at least one disk storage, etc. Of course, the memory 1002 may also include hardware required by other services.

[0175] The memory 1002 is used to store instructions executable by the processor 1001. The processor 1001 is configured to execute any step in the embodiment of the image processing method applied to an electronic device.

[0176] A wired or wireless network interface 1003 is configured to connect the electronic device 11 to a network.

[0177] The processor 1001, the memory 1002, the network interface 1003 and the I / O controller 1004 may be interconnected via a communication bus 1005, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0178] In an exemplary embodiment, the electronic device 11 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned image processing method.

[0179] In an exemplary embodiment, a computer storage medium is also provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can perform the above-mentioned image processing method.

[0180] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer. The memory is a memory included in the above-mentioned electronic device and contains software code. After the computer program is loaded and executed, the above-mentioned image processing method can be implemented.

[0181] It should be noted that the features described in the various embodiments in this specification can be replaced or combined with each other. For the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0182] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0183] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0184] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, comprising: In a case where it is detected that a shrinking and deformation operation is performed on an object included in the first image, obtaining a position of the object in the first image; In the case where the position represents that the edge of the object and the first image meet a first distance condition, obtaining a compensated image; The compensation image is composed of at least the first image and the edge prediction image, and the edge of the object and the compensation image meets a second distance condition, wherein the first distance condition includes: the standard object corresponding to the object is cut by the edge of the first image, or the standard object corresponding to the object is not cut by the edge of the first image, and the standard object is located at the edge of the first image; the second distance condition includes at least: the object in the compensation image is not located at the edge of the compensation image; Performing shrinkage deformation on the object contained in the compensated image to obtain a second image; The image display interface is controlled to at least display an image of a target area in the second image, where the target area is an area including the object.

2. According to the image processing method of claim 1, the step of determining that the position represents that the object and the edge of the first image meet a first distance condition comprises: Determine a contour of a standard object corresponding to an object included in the first image, wherein the object included in the first image is a partial area or the entire area of ​​the standard object; In the case where the contour satisfies a preset condition, determining that the position represents that the edge of the object and the first image satisfy a first distance condition; The preset conditions include: a part of the contour is located outside the first image, and / or the contour is completely located in the first image, and a distance between the part of the contour and a boundary of the first image is less than or equal to a first threshold.

3. The image processing method according to claim 2, wherein the step of determining the contour of the standard object corresponding to the object contained in the first image comprises: Acquire a plurality of standard key points of the standard object corresponding to the object included in the first image to obtain the contour formed by the plurality of standard key points.

4. According to the image processing method of claim 3, the step of obtaining a plurality of standard key points of a standard object corresponding to the object contained in the first image comprises: Acquire a standard object model corresponding to the target object type to which the object belongs, wherein the standard object model includes a plurality of standard key points on the standard object belonging to the target object type, wherein the plurality of standard key points constitute the outline of the standard object; A plurality of standard key points of the standard object corresponding to the object included in the first image are marked based on the standard object model.

5. The image processing method according to claim 3 or 4, further comprising: When at least one of the plurality of standard key points constituting the contour is located outside the first image, determining that a part of the contour is located outside the first image; When a plurality of standard key points constituting the contour are all located on the first image, it is determined that the contour is completely located in the first image.

6. According to any one of the image processing methods of claim 1 to 4, when the edge of the object represented at the position and the edge of the first image meet a first distance condition, the step of obtaining the compensated image comprises: Acquire a third image, wherein the third image includes the edge prediction image; In a case where the position represents that the edge of the object and the edge of the first image meet a first distance condition, acquiring the edge prediction image from the third image; The edge prediction image is spliced ​​with the first image to obtain the compensated image.

7. According to the image processing method of claim 6, the shrinkage deformation operation corresponds to a shrinkage deformation degree parameter, and the shrinkage deformation degree parameter includes the number of stretched pixels; the second distance condition includes that the minimum distance between the object and the edge of the compensated image is greater than or equal to the number of stretched pixels.

8. According to any one of the image processing methods of claims 1 to 4, when the edge of the object represented at the position and the edge of the first image meet a first distance condition, the step of obtaining the compensated image comprises: Repairing the edge image to be repaired based on the first image to obtain the edge prediction image; The edge prediction image is spliced ​​with the first image to obtain the compensated image.

9. According to any one of the image processing methods of claims 1 to 4, the target area is a local area or the entire area where the second image is located.

10. An image processing device, comprising: A first acquisition module, configured to acquire a position of the object in the first image when detecting that a shrinkage deformation operation is performed on the object contained in the first image; A second acquisition module, configured to obtain a compensated image when the position characterizing the edge of the object and the first image satisfy a first distance condition; The compensation image is composed of at least the first image and the edge prediction image, and the edge of the object and the compensation image meets a second distance condition, wherein the first distance condition includes: the standard object corresponding to the object is cut by the edge of the first image, or the standard object corresponding to the object is not cut by the edge of the first image, and the standard object is located at the edge of the first image; the second distance condition includes at least: the object in the compensation image is not located at the edge of the compensation image; A shrinkage deformation module, used for shrinking and deforming the object contained in the compensation image to obtain a second image; A control display module is used to control the image display interface to at least display an image of a target area in the second image, where the target area is an area including the object.

Images