Image processing method, video processing method, device, storage medium and program product

Abstract

Provided in the embodiments of the present application are an image processing method, a video processing method, a device, a storage medium and a program product. In a scenario where contour deformation is performed on a target object in an image, the embodiments of the present application introduce outside the outer contour of the target object the concept of external boundary to define a range of background points that follow the contour deformation, further introduce between the outer contour and the external boundary the concept of buffer boundary and, when contour points of the outer contour are deformed, use extension points on the buffer boundary as buffers to deform the deformed contour points and the background points within the range defined by the external boundary, thus keeping to a certain extent the relative positional relationship between the background points within the range defined by the external boundary before and after the deformation, keeping the continuity between the target object and a background region, mitigating the problem of background distortions caused by changes of the outer contour, and improving the credibility of a contour deformation result of the target object.

Description

Image and video processing method and device, storage medium and program product

[0001] Cross-reference to related applications

[0002] This application is based on the Chinese Patent Application No. 2024106601621 entitled "Image and video processing method and device, storage medium and program product" filed on May 24, 2024, which is incorporated by reference in its entirety into this application. TECHNICAL FIELD

[0003] The present application relates to the technical field of computer, and particularly relates to an image and video processing method and device, a storage medium and a program product. BACKGROUND

[0004] In live broadcast, short video, film special effect or image beautification application scenarios, people want to change the face contour in the video, such as slim face, narrow face, small face or thin chin, so that the face looks more beautiful. At present, the change of the face contour is mainly to determine the deformation parameters of the face contour according to the beautification demand, and to adjust the positions of the related contour points on the face according to the deformation parameters, so as to achieve the beautification effect of slim face, narrow face, small face or thin chin. However, the current scheme will cause the related background area to be distorted while beautifying the face, thereby reducing the credibility of face deformation, and even causing the background area in the video to be discontinuous or inconsistent in time. SUMMARY

[0005] Aspects of the present application provide an image and video processing method, device, storage medium and program product to alleviate the background distortion problem in the contour deformation process.

[0006] An embodiment of the present application provides an image and video processing method, device, storage medium and program product to alleviate the background distortion problem in the contour deformation process.

[0007] The embodiment of the present application further provides a method for obtaining video data comprising a plurality of images, wherein the plurality of images comprise at least one to-be-processed image comprising a face region, the face region in each to-be-processed image comprises a face contour, the face contour has a plurality of contour points thereon, an image region outside the face contour is a background region, and the background region comprises background points; in the background region, an outer boundary and a buffer boundary are determined to be adaptive to the face contour, the outer boundary and the buffer boundary respectively have a plurality of extension points corresponding to the plurality of contour points thereon, and the extension points belong to the background points; and in response to a morphing operation on the face contour, the target contour points subjected to the morphing operation and the background points within the outer boundary are subjected to morphing processing with the extension points on the buffer boundary as a buffer to obtain a target image.

[0008] The embodiment of the present application further provides an electronic device comprising a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and used to execute the computer program to implement the steps in the image processing method and the video processing method provided by the embodiment of the present application.

[0009] The embodiment of the present application further provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor is caused to implement each step in the image processing method and the video processing method provided by the embodiment of the present application.

[0010] The embodiment of the present application further provides a computer program product comprising computer programs / instructions, when the computer programs / instructions are executed by a processor, the processor is caused to implement each step in the image processing method and the video processing method provided by the embodiment of the present application.

[0011] In the embodiment of the present application, in the scene of contour morphing on a target object in an image, the concept of an outer boundary outside the outer contour of the target object is introduced to limit the range of the background points following the contour morphing, and the concept of a buffer boundary between the outer contour and the outer boundary is further introduced, in the case that the contour points of the outer contour are subjected to morphing, the extension points on the buffer boundary are used as a buffer to perform morphing on the contour points subjected to the morphing and the background points within the range limited by the outer boundary, the relative position relationship between the background points within the range limited by the outer boundary before and after the morphing is maintained to a certain extent, the continuity of the target object and the background region is maintained, the background distortion problem caused by the change of the outer contour is alleviated, and the credibility of the contour morphing result of the target object is improved. Further, the background distortion problem can be alleviated and the image processing efficiency can be improved without increasing the algorithm complexity. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are included to provide a further understanding of the present application and constitute a part of this application, illustrate certain illustrative embodiments of the present application and are used to explain the present application, but do not limit the present application. In the drawings:

[0013] FIG. 1a is a schematic diagram of background warping of a face region according to an example embodiment of the present application;

[0014] FIG. 1b is a schematic diagram of face key points and protection extension points according to an example embodiment of the present application;

[0015] FIG. 1c is a schematic diagram of a triangle face according to an example embodiment of the present application;

[0016] FIG. 1d is a schematic diagram of contour points and background points before and after morphing according to an example embodiment of the present application;

[0017] FIG. 2a is a flowchart of an image processing method according to an example embodiment of the present application;

[0018] FIG. 2b is a flowchart of determining a second morphing parameter of a first extension point according to an example embodiment of the present application;

[0019] FIG. 2c is a flowchart of another method of determining a second morphing parameter of a first extension point according to an example embodiment of the present application;

[0020] FIG. 2d is a schematic diagram of morphing of a first extension point according to an example embodiment of the present application;

[0021] FIG. 2e is a flowchart of a video processing method according to an example embodiment of the present application;

[0022] FIG. 3a is a schematic diagram of contour points, first extension points and second extension points according to an example embodiment of the present application;

[0023] FIG. 3b is a schematic diagram of a reference point according to an example embodiment of the present application;

[0024] FIG. 3c is a schematic diagram of background warping optimization effect according to an example embodiment of the present application;

[0025] FIG. 4 is a schematic diagram of a structure of an image processing apparatus according to an example embodiment of the present application;

[0026] FIG. 5 is a schematic diagram of a structure of an electronic device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0028] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0029] In the traditional scheme of adjusting the position of the contour point, the pixel points in the background region will move with the movement of the contour point, changing the relative position relationship between the pixel points in the background region, so that the deformed background is distorted. In view of the above background distortion problem, several solutions are proposed in the embodiments of the present application.

[0030] In one solution A1, the target object (such as a face region) in the image to be processed is separated from the background region by means of a segmentation algorithm, and the pixel points in the background region can be referred to as background points or background pixel points; the texture of the background region is filled after the target object (such as the face region) is deformed. In this solution A1, the calculation amount of the segmentation algorithm is large, and the texture filling of the background region is easy to cause the discontinuity of the texture of the background region and the region to which the target object belongs.

[0031] In another solution A2, the process of face deformation is described taking the face image as an example.

[0032] 1) According to the input face image, the deformation key points in the face image are constructed. In an optional implementation, the deformation key points can include face contour points on the face region and protection extension points on the background region. The protection extension points are located in the background region of the image and correspond one-to-one to the contour points of the target object (face), and the protection extension points belong to the background points. The protection extension points limit the range of the background points in the background region that need to follow the deformation of the face contour, and the background points outside the protection extension points do not follow the deformation of the face contour in the background region.

[0033] 2) Triangular subdivision is performed according to the deformation key points to construct a plurality of triangular faces with the deformation key points as the vertices. For example, each triangular face includes 2 face contour points and 1 protection extension point, or each triangular face includes 1 face contour point and 2 protection extension points.

[0034] The triangulation refers to a process of dividing the plane into a plurality of triangular faces based on the given plane M and the point set P on the plane. The plurality of triangular faces divided form a triangular face set T, which should satisfy the following requirements: 1) the vertices of all the triangular faces exactly constitute the point set P; 2) the edges of any two triangular faces in the set T do not intersect; and 3) the set T of all the triangular faces constitutes a convex hull of the point set P, which refers to a minimum convex polygon.

[0035] The plane M can be an image to be processed, and the point set P can include face contour points and protection extension points. Adjacent protection extension points are connected by edges of triangular faces to form a protection contour ring, which surrounds a target object. In order to play the role of the protection contour ring, the positions of the protection extension points on the protection contour ring remain unchanged.

[0036] 3) Adjusting the positions of part of the deformation key points according to a preset face deformation mode. In a possible implementation, the adjustment direction and distance of part of the face key points can be determined according to different face deformation modes; and the positions of the face key points are adjusted according to the determined adjustment direction and distance to obtain new positions. The face deformation mode can be, but is not limited to, a thin face or a narrow face.

[0037] 4) Updating the pixel values in the triangular face according to the new positions of the deformation key points that have occurred deformation.

[0038] According to the principle that the barycentric coordinate system is unchanged before and after deformation, when the positions of the vertices of any triangular face change, the pixel values of each pixel point in the triangular face are updated. The coordinates of any pixel point in the triangular face can be expressed by linear combination of the coordinates of the three vertices of the triangular face, and the weights of the three vertices can be regarded as the coordinates of the pixel point in the barycentric coordinate system of the current triangular face. As shown in FIG. 1c, the coordinates of three vertices A, B and C of a given triangular face are given, and a pixel point D in the triangular face can be expressed as a linear combination of the coordinates of the three vertices, that is, D = αA + βB + γC, and α + β + γ = 1. At this time, the coordinates of the pixel point D in the barycentric coordinate system of ΔABC are (α, β, γ). Wherein α, β, γ respectively correspond to the areas of the three triangular faces, that is, α = S ΔDBC , β = S ΔDAC , γ = S ΔDABWherein, Δ represents a triangular face, and S represents an area. In FIG. 1c, for any one triangular face before deformation, taking the triangular face ΔABC as an example, the coordinates of the three vertices A, B and C are adjusted to obtain the triangular face ΔA'B'C' after deformation. For a pixel point D in the triangular face ΔABC, the corresponding pixel point in the triangular face ΔA'B'C' is denoted as D', and the coordinates of the pixel point D' in the barycentric coordinate system of the triangular face ΔA'B'C' are equal to the coordinates of the pixel point D in the barycentric coordinate system of the triangular face ΔABC, so the coordinates of the pixel point D' are (α, β, γ), D' = αA' + βB' + γC'.

[0039] In the method for limiting the background distortion range by the protection extension point, the positions of the pixel points in the triangular face of the new position of the deformation key point are adjusted, so that the background points within the range limited by the protection extension point can be deformed along with the deformation of the face contour, the reliability of the contour deformation result is improved, and the background points outside the protection extension point do not follow the deformation, which can to some extent alleviate the background distortion problem, but the effect is not particularly ideal. This is because, in the solution A2, the background distortion problem still exists for the background points within the range limited by the protection extension point. This is analyzed and explained as follows:

[0040] In the face deformation, the positions of the face key points are adjusted according to a predetermined face deformation mode, and the position of the protection extension point is kept unchanged. This way is easy to change the relative position relationship between the pixel points in the triangular face, resulting in texture distortion in the triangular face before and after deformation. As shown in FIG. 1d, L1 and L2 are adjacent face contour key points in ΔL1L2C, and C is the protection extension point. E and F are any two points on the edges L1C and L2C of the triangular face respectively, and can be expressed in the barycentric coordinate system of ΔL1L2C as follows: E = xL2 + (1-x)C F = yL1 + (1-y)C

[0041] L1 and L2 are adjusted to L'1 and L'2 according to the predetermined deformation mode, the triangular face ΔL1L2C is deformed to the triangular face ΔL'1L'2C, and E and F are deformed to E' and F'. According to the principle that the coordinates in the barycentric coordinate system before and after deformation are unchanged, the coordinates of E' and F' are expressed as follows: E' = xL2' + (1-x)C F' = yL'1 + (1-y)C

[0042] The displacement amounts of E and F before and after deformation are expressed as follows: E' - E = x(L2 - L'2) F' - F = y(L1 - L'1)

[0043] If it is desired to keep the texture in the triangular face unchanged as much as possible, the displacement amounts of E and F need to be as consistent as possible. According to this principle, a target function is constructed as follows:

[0044] wherein x, y are any value in the range of [0, 1], the offset of E, F is mainly affected by the offset of L1, L2. Therefore, the final target function is:

[0045] However, since the offset of L1, L2 mainly depends on the preset face morphing mode, it cannot be ensured that the offset of L1, L2 is exactly the same, which changes the relative position relationship between E, F, and causes the texture in the triangular face to be distorted.

[0046] As mentioned above, the face protection contour composed of protection extension points can reduce the range of background distortion, but there is no way to alleviate the background distortion problem between the face contour points and the protection extension points. If the background distortion in this part of the region is serious, it will also reduce the credibility of the contour morphing effect. For video data, if there is serious background distortion when each frame of image is subjected to contour morphing, it will cause the problem of discontinuity of video effect.

[0047] In view of the above analysis, the embodiment of the present application also provides a solution A3. In the solution A3, in the scenario of contour morphing for a target object in an image, the concept of an outer boundary is introduced outside the outer contour of the target object to limit the range of background points following the contour morphing. Further, the concept of a buffer boundary is introduced between the outer contour and the outer boundary. In the case that the contour points of the outer contour are morphed, the extension points on the buffer boundary are used as a buffer to morph the contour points that are morphed and the background points within the range limited by the outer boundary, to maintain the relative position relationship between the background points within the range limited by the outer boundary before and after morphing to a certain extent, keep the continuity of the target object and the background region, alleviate the background distortion problem caused by the change of the outer contour, and improve the credibility of the contour morphing result of the target object. Meanwhile, compared with the solution A1, the solution A3 can alleviate the background distortion problem and improve the image processing efficiency without increasing the algorithm complexity.

[0048] The solution A3 provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0049] FIG. 2a is a flowchart of an image processing method provided by an exemplary embodiment of the present application. As shown in FIG. 2a, the method comprises:

[0050] 201a, acquiring a to-be-processed image, the to-be-processed image comprising a target object, the target object having an outer contour, the outer contour having a plurality of contour points, and an image region outside the outer contour being a background region, the background region comprising background points;

[0051] 202a, in the background region, an outer boundary and a buffer boundary are determined, which are fitted to the outer contour, and the outer boundary and the buffer boundary have a plurality of extension points corresponding to a plurality of contour points respectively, and the extension points belong to background points;

[0052] 203a, in response to the deformation operation on the outer contour, the target contour points subjected to deformation and the background points within the outer boundary are subjected to deformation processing with the extension points on the buffer boundary as a buffer to obtain a target image.

[0053] In this embodiment, the manner of obtaining the image to be processed is not limited. For example, the execution subject of the image processing method can be a terminal device, which can collect the image to be processed through a camera, or the terminal device can accept the image to be processed sent by other terminal devices, and the terminal device can obtain a target image after performing the contour deformation provided in this embodiment. For another example, the execution subject of the image processing method is a server device, which can receive the image to be processed sent by the terminal device, perform the contour deformation provided in this embodiment on the image to be processed to obtain a target image, and then return the target image to the terminal device.

[0054] In this embodiment, the image to be processed includes at least one object, which can be a subject object such as a portrait, an animal, a building, a vehicle, and a plant, or a text object such as a single or combined text object of a word, a letter, or a symbol. For the convenience of distinguishing and describing, the object in the image to be processed on which the contour deformation operation can be performed is referred to as a target object. The target object can be any object in the image to be processed.

[0055] In a case where the to-be-processed image contains one object, the object can be directly determined as the target object; in a case where the to-be-processed image contains multiple objects, the objects can be identified, and the target object can be determined from the multiple objects according to positions of the objects in the to-be-processed image and / or sizes of the objects. For example, an object that is relatively centered in position can be taken as the target object, or an object that is relatively centered in position and relatively large in size can be taken as the target object, or the largest object can be taken as the target object, and the like. Of course, in a case where the to-be-processed image contains multiple objects, the target object can also be manually set by a user. Alternatively, the terminal device supports a touch operation, and in a case where the terminal device displays the to-be-processed image, the user can click the target object on the to-be-processed image; correspondingly, the terminal device can determine a screen position of the click operation on the screen in response to a click operation of the user on the screen, and then map the screen position of the click operation on the screen to a click position on the to-be-processed image according to a display position of the to-be-processed image on the screen, and take an object located at the click position as the target object. Alternatively, the terminal device displays a setting control while displaying the to-be-processed image, and the user can set the target object through the setting control. Specifically, the terminal device can display a setting interface in response to a click operation of the user on the setting control, and the setting interface is provided with a position information item and / or a name information item, where the position information item is used for the user to set a position range of the target object in the to-be-processed image, for example, a top-left corner, a bottom-right corner, a left region, a middle region, and the like; the name information item is used for the user to set a name of the target object, for example, a dog, a cat, a face, and the like; the terminal device obtains position information and / or name information of the target object set by the user in response to an input operation of the user in the position information item and / or the name information item; and the terminal device determines the target object from the multiple objects according to the position information and / or the name information of the target object set by the user.

[0056] The target object corresponds to a key point, which is a pixel point that can reflect a feature of the target object. For example, in a case where the target object is implemented as a face, the face key point is a point set that can describe positions of key parts of the face. For example, the face key point includes pixel points corresponding to eyes, a nose, a mouth, and a face contour, as shown in FIG. 1b.

[0057] The target object in the image to be processed has an outer contour. For example, if the target object is a face, the outer contour can be the contour of the face; if the target object is a "leg", the outer contour can be the contour of the "leg"; if the target object is a character, the outer contour can be the contour of the character. The outer contour includes a plurality of contour points, which are key points of the target object. The area outside the outer contour is a background area, i.e., the area of the image other than the area where the target object (e.g., the face area) is located can be referred to as the background area, which includes background points. The background points and the key points are both pixel points in the image to be processed. The background area can present a pattern or a design. When the contour of the target object is deformed (e.g., the shape is changed), the texture in the background area is stretched or changed, which will cause the shape of the texture to change, i.e., the background is distorted. As shown in FIG. 1a, the left image is a face image before deformation, and the right image is a face image after deformation of the background.

[0058] In this embodiment, in the background area, an outer boundary and a buffer boundary that adapt to the outer contour are determined, and a plurality of extension points corresponding to the plurality of contour points are present on the outer boundary and the buffer boundary. The outer boundary is used to limit the range of the background points in the background area that need to follow the deformation of the contour of the target object; the buffer boundary is located between the outer contour of the target object and the outer boundary, and functions to buffer the background points within the range limited by the outer boundary during the deformation process of the contour of the target object, so as to reduce the deformation amplitude of the background points within the range limited by the outer boundary and alleviate the problem of background distortion. It is noted that the extension points on the outer boundary and the buffer boundary are both background points, and for the purpose of distinction and description, the plurality of background points on the buffer boundary corresponding to the plurality of contour points can be referred to as first extension points, and the plurality of background points on the outer boundary corresponding to the plurality of contour points can be referred to as second extension points. The first extension points and the second extension points are both background points. The contour points correspond one-to-one to the first extension points, and the contour points correspond one-to-one to the second extension points. As shown in FIG. 3a, the outer contour, the buffer boundary and the outer boundary, and the contour points, the first extension points and the second extension points are shown.

[0059] In this embodiment, the user corresponding to the terminal device can initiate a deformation operation on the outer contour. The deformation operation can be that the user manually drags the outer contour, for example, in the case where the target object is a face, the outer contour of the face is pushed inward to achieve the effect of "face slimming", or the outer contour of the face is dragged outward to achieve the effect of "face plumping" or "face fattening". In addition, the deformation operation can also be that the user triggers a deformation control on the page to achieve the effect of "one-key" contour change. The deformation control can include but is not limited to a "face slimming" or "small face" control, etc.

[0060] In the embodiment, in the case that the morphing operation is initiated for the outer contour, the target contour points and the background points within the outer boundary can be morphed with the expansion points on the buffer boundary as a buffer in response to the morphing operation for the outer contour to obtain the target image. The expansion points on the buffer boundary can buffer the change of the background between the outer contour and the outer boundary. By balancing the morphing of the expansion points on the buffer boundary, the morphing amount of the background points between the outer contour and the buffer boundary and the morphing amount of the background points between the buffer boundary and the outer boundary are small, and the background distortion between the outer contour and the outer boundary can be effectively alleviated.

[0061] In the embodiment, in the case that the morphing operation is initiated for the outer contour, the target contour points and the background points within the outer boundary can be morphed with the expansion points on the buffer boundary as a buffer in response to the morphing operation for the outer contour to obtain the target image. The expansion points on the buffer boundary can buffer the change of the background between the outer contour and the outer boundary. By balancing the morphing of the expansion points on the buffer boundary, the morphing amount of the background points between the outer contour and the buffer boundary and the morphing amount of the background points between the buffer boundary and the outer boundary are small, and the background distortion between the outer contour and the outer boundary can be effectively alleviated.

[0062] In an optional embodiment, the implementation of morphing the target contour points and the background points within the outer boundary with the expansion points on the buffer boundary as a buffer in response to the morphing operation for the outer contour to obtain the target image is not limited. The order of morphing the contour points on the outer contour, the expansion points on the buffer boundary and the other background points within the outer boundary is not limited. The other background points are the background points within the outer boundary except the expansion points on the buffer boundary.

[0063] In example G1, in response to the morphing operation for the outer contour, the expansion points on the buffer boundary are morphed according to the morphing parameters of the morphing operation, the other background points within the outer boundary are morphed based on the morphing of the expansion points on the buffer boundary, and the target contour points on the outer contour that are morphed are morphed based on the morphing parameters of the morphing operation to obtain the target image.

[0064] In example G2, in response to the morphing operation for the outer contour, the target contour points on the outer contour that are morphed are morphed according to the morphing parameters of the morphing operation, the expansion points on the buffer boundary are morphed based on the morphing of the target contour points, and the other background points within the outer boundary are morphed according to the morphing of the expansion points on the buffer boundary to obtain the target image.

[0065] For example G2, in particular, in response to the contour deformation operation on the target object, at least one target contour point in the plurality of contour points is deformed according to the contour deformation parameter corresponding to the contour deformation operation; based on the deformation parameter of the at least one target contour point, the background points within the outer boundary are deformed by taking the expansion point on the buffer boundary as the buffer between the at least one target contour point and the outer boundary, and a target image is obtained.

[0066] Wherein, the user corresponding to the terminal device, for example, the host, the short video shooting user or the image beautification user, can initiate a contour deformation operation on the target object in the image to be processed. The contour deformation operation is an operation that can change the position, angle, size, shape, etc. of the contour of the target object in the image to be processed. For example, if the target object is a face, the contour deformation operation can be a "slim face", "narrow face" or "small head" operation, or it can be a "height increase", "thin legs" and "thin arms" operation. For example, if the target object is text, the contour deformation operation can be an operation that changes the contour by gradually increasing the text "from left to right", gradually decreasing the text "from left to right", or narrowing the text as a whole or in part. Wherein, the contour deformation operation corresponds to a contour deformation parameter, which indicates how to deform the outer contour of the target object, for example, the contour deformation parameter corresponding to the contour deformation operation indicates that the outer contour satisfies a certain contour proportion relationship. For example, in the case of a face as a target object, the width of the cheeks is changed and the length of the chin is lengthened, so that the face contour conforms to the set proportion relationship.

[0067] Wherein, in response to the contour deformation operation on the target object, at least one target contour point in the plurality of contour points is deformed according to the contour deformation parameter corresponding to the contour deformation operation. Wherein, the deformation processing on the target contour point can change the position of the target contour point. Alternatively, the deformation processing on the target contour point can not change the pixel value of the target contour point, but is not limited thereto. The at least one target contour point can be part of the plurality of contour points, or it can be all contour points, and the specific type of contour deformation operation is not limited.

[0068] Optionally, a buffer border is added between the outer border and the outer contour, and a first extension point on the buffer border is taken as a deformation buffer object between a second extension point on the outer border and a target contour point on the outer contour, and a relative deformation variable between the target contour point and the first extension point is taken as a target to satisfy a set condition, and a deformation of a background point inside the outer border is determined, so that a deformation amplitude of the background point inside the outer border is relatively small, thereby relieving the background distortion problem of the background point inside the outer border. Specifically, based on a position of at least one target contour point after deformation, a relative deformation variable between the at least one target contour point and a plurality of first extension points on the buffer border is taken as a target to satisfy a set condition, and a background point inside the outer border is deformed to obtain a deformed target image.

[0069] Wherein, the determination manner of the relative deformation variable between the at least one target contour point and the plurality of first extension points on the buffer border is not limited. For example, the relative deformation variable can be realized as a first relative deformation variable between a deformation variable of the target contour point B1 and a deformation variable of the first extension point C1 corresponding to the target contour point B1; for another example, the relative deformation variable can be realized as a second relative deformation variable between the deformation variable of the first extension point C1 corresponding to the target contour point B1 and a neighboring extension point C2 of the first extension point C1; for another example, the relative deformation variable can be realized as the first relative deformation variable and the second relative deformation variable; for another example, the relative deformation variable can be realized as a third relative deformation variable between the deformation variable of the target contour point B1 and the deformation variable of the neighboring extension point C2.

[0070] Wherein, the relative deformation variable satisfies the set condition, and the set condition can be that the relative deformation variable is 0, or the first relative deformation variable is within a set numerical range. Wherein, different relative deformation variables can satisfy the same set condition, of course, different relative deformation variables can satisfy different set conditions. For example, the first relative deformation variable satisfies a first set condition, and the first set condition is realized as that the first relative deformation variable is 0, or the first relative deformation variable is less than a set first threshold value, and the first threshold value can be 0.1, 0.8 or 1.5, etc., and the unit of the first threshold value can be pixel, millimeter, centimeter or meter, etc. For another example, the second relative deformation variable satisfies a second set condition, and the second relative deformation variable is 0, or the second relative deformation variable is less than a set second threshold value, and the second threshold value can be 0.2, 0.9 or 2.1, etc., and the unit of the second threshold value can be pixel, millimeter, centimeter or meter, etc.

[0071] In the embodiments of the present application, the implementation of determining the outer boundary and the buffer boundary is not limited. In an optional embodiment, the outer boundary can be determined according to the color change of the background points in the background region and / or the degree of association between the background information presented by the background points in the background region and the target object; and the buffer boundary can be randomly determined between the outer boundary and the outer contour. For example, a continuous region with the strongest degree of association between the background information and the target object can be selected, one boundary of the continuous region is the outer contour, and the other boundary of the continuous region is the outer boundary; or the region boundary where the color change is obviously different in the background region can be taken as the outer boundary, and the like.

[0072] In another optional embodiment, the reference point of the target object can be calculated according to the target key point corresponding to the target object; and the first extension point on the buffer boundary and the second extension point on the outer boundary can be determined according to the reference point of the target object.

[0073] The target key point corresponding to the target object includes a plurality of contour points, or the target key point includes at least one internal key point, or the target key point includes both a plurality of contour points and at least one internal key point. Taking the face region as an example, the internal key point can include but is not limited to eyebrow key points, eye key points, nose key points, and mouth key points, etc. For different target objects, the implementation of the internal key point will be different. For example, if the target object is a table, the internal key point can be but is not limited to a key point on the table top and a key point on the table leg; if the target object is a bottle, the internal key point can be but is not limited to various key points on the bottle body.

[0074] The reference point of the target object can be a relatively central position point on the target object, for example, can be a center point of the target object or a point on the horizontally aligned center line or a point on the vertically aligned center line, without limitation. The implementation of calculating the reference point of the target object according to the object key point of the target object is not limited. For example, in the case where the target key point includes a plurality of contour points, the position information of the plurality of contour points is averaged to obtain the reference point of the target object. For another example, in the case where the target key point includes a plurality of contour points and at least one internal key point, the position information of the plurality of contour points and the at least one internal key point is averaged to obtain the reference point of the target object.

[0075] The first extension point on the buffer boundary and the second extension point on the outer boundary can be determined in sequence on the extension line of the reference point and any one contour point. Each contour point corresponds to a first extension point and a second extension point. The first extension points corresponding to all contour points form the buffer boundary, and the second extension points corresponding to all contour points form the outer boundary. As shown in FIG. 3b, the reference point, the first extension point and the second extension point are exemplarily shown, but are not limited thereto.

[0076] Optionally, a first position is selected from the extension line of the reference point and any one contour point, and a background point at the first position is taken as the first extension point on the buffer boundary; a second position is selected from the extension line of the reference point and any one contour point, and a background point corresponding to the second position is taken as the second extension point on the outer boundary; wherein the second distance from the second position to the reference point is greater than the first distance from the first position to the reference point, and the first distance and the second distance satisfy a set proportional relationship. For example, the ratio of the first distance from the first position to the reference point to the reference distance from the reference point to the contour point is a first ratio k1, the ratio of the second distance from the second position to the reference point to the reference distance from the reference point to the contour point is a second ratio k2, and k2>k1, and k1 and k2 are positive real numbers. The first ratio k1 and the second ratio k2 are preset values, for example, the first ratio can include but is not limited to 1.05, 1.10 or 1.20, etc. The second ratio can include but is not limited to 1.25, 1.30 or 1.31, etc. The proportional relationship that the first distance and the second distance should satisfy can be represented by the first ratio k1 and the second ratio k2. Based on this, the first position on the extension line can be determined according to the first ratio k1, and the second position on the extension line can be determined according to the second ratio k2.

[0077] In an optional embodiment, the contour deformation operation corresponds to a contour deformation parameter, which indicates how to deform the outer contour as a whole. Based on the contour deformation parameter and the positions of the contour points in the outer contour, it can be determined how to deform the target contour points that need to be deformed. For the sake of distinguishing and description, the contour points in the outer contour that need to be deformed are referred to as target key points.

[0078] Specifically, in response to the contour deformation operation on the target object, the contour deformation parameter corresponding to the contour deformation operation is obtained, the contour deformation parameter including a contour deformation type and a contour deformation ratio; at least one target contour point that needs to be deformed is determined from the plurality of contour points according to the contour deformation type; the first deformation parameter of each of the at least one target contour point is determined according to the contour deformation ratio and the position of the at least one target contour point; and the position of the at least one target contour point is adjusted according to the first deformation parameter of each of the at least one target contour point to obtain the deformed position of the at least one target contour point.

[0079] The contour deformation parameter includes a contour deformation type and a contour deformation ratio. The contour deformation type indicates the type of the contour deformation operation. For example, when the target object is a face region, the contour deformation type can include, but is not limited to, a slim face, a slim chin, a small head, long legs, thin legs, and the like. Correspondingly, the contour deformation ratio represents the proportion of the contour. For example, the contour deformation ratio can include, but is not limited to, the length-width ratio of the cheeks of the face, the proportion of the length of the chin to the length of the face, or the proportion of the eyes to the face, and the like. For another example, when the target object is an S-shaped vase, the contour deformation type can include, but is not limited to, reducing the width of the overall bottle body, reducing the width of the lower bottle body, shortening the height of the bottle body, lengthening the height of the bottle body, and the like. Correspondingly, the contour deformation ratio can include, but is not limited to, the height-width ratio of the overall bottle body, the proportion of the width of the lower bottle body, or the height ratio between the upper bottle body and the lower bottle body, and the like.

[0080] The at least one target contour point to which the deformation needs to occur is determined from the plurality of contour points according to the contour deformation type in the contour deformation parameter. The contour deformation type and the contour point to which the deformation needs to occur have a corresponding relationship, and the corresponding relationship can be preset and saved. For example, when the target object is a face and the contour deformation type is a small head, the target contour point is all contour points in the outer contour. For another example, when the target object is a face and the contour deformation type is a slim chin, the target contour point is the contour point corresponding to the chin in the outer contour. For another example, when the target object is a human body and the contour deformation type is a waist, the target contour point is the contour point corresponding to the waist in the outer contour. For another example, when the target object is a human body and the contour deformation type is a slim leg, the target contour point is the contour point corresponding to the leg in the outer contour.

[0081] The first deformation parameter of each of the at least one target contour point is determined according to the contour deformation ratio and the position of the at least one target contour point. For example, the target object corresponds to an initial contour ratio. According to the size relationship between the initial contour ratio and the contour deformation ratio, the first deformation parameter of the at least one target contour point is determined. For example, the contour deformation ratio is realized as that the length of the chin accounts for 1 / 3 of the length of the face, and the initial contour ratio indicates that the chin accounts for 1 / 4 of the length of the face. Then, the chin can be lengthened, that is, the at least one target contour point corresponding to the chin is moved downward by a set distance, so that the length of the chin after the movement accounts for 1 / 3 of the length of the face. The first deformation parameter includes a deformation direction and a deformation distance. The deformation directions of different target contour points can be different, and the deformation distances are also different. For example, the contour deformation operation is "face slimming", and the at least one contour point corresponding to the face is moved towards the reference point, and the at least one contour point surrounds the reference point. At this time, the moving directions of different target contour points are different. Meanwhile, the face curve can be the curve between the "ear" and the "chin". When the face is slimmed, the deformation distance of the target contour point close to the "ear" and the "chin" is small, and the deformation distance of the target contour point far away from the "ear" and the "chin" is large.

[0082] In an optional embodiment, in order to facilitate the distinction and description, the first extension point on the buffer boundary corresponding to the at least one target contour point is referred to as a target extension point. That is, the plurality of first extension points on the buffer boundary include target extension points and non-target extension points; the target extension point is the first extension point corresponding to the target contour point, and the non-target extension point is the first extension point corresponding to the non-target contour point. Based on this, based on the position of the at least one target contour point after deformation, the relative deformation variable between the at least one target contour point and the plurality of extension points on the buffer boundary satisfies the set condition as the target, the background points within the outer boundary are deformed to obtain a target image. An embodiment includes: the relative deformation variable between the at least one target contour point and the at least one target extension point corresponding to the at least one target contour point on the buffer boundary satisfies the first set condition, and the relative deformation variable between the at least one target extension point and the adjacent extension point of the at least one target extension point on the buffer boundary satisfies the second set condition as the constraint condition, and the second deformation parameter of the at least one target extension point and the adjacent extension point thereof is calculated; wherein the first set condition and the second set condition can be the same or different, and the details can be referred to the foregoing embodiments, which will not be described here. According to the second deformation parameter of the at least one target extension point and the adjacent extension point thereof, the positions of the at least one target extension point and the adjacent extension point thereof are adjusted to obtain the positions of the at least one target extension point and the adjacent extension point thereof after deformation. Based on the positions of the at least one target contour point and the at least one target extension point and the adjacent extension point thereof after deformation, the other background points within the outer boundary are deformed to obtain a target image. The other background points refer to the background points within the outer boundary except the target extension points.

[0083] Optionally, the background region can be triangulated with the contour points on the outer contour and the second expansion points on the outer boundary as vertices, and in the case of deformation of some or all of the contour points, the pixels inside the triangular faces obtained by triangulation are deformed, which may cause background distortion in the background region between the outer contour and the outer boundary. In order to alleviate the problem of background distortion, the first expansion points on the buffer boundary are used instead of the contour points to construct new triangular faces with the second expansion points on the outer boundary, and at the same time, triangular faces between the contour points and the first expansion points are constructed. By adjusting the position of the first expansion point, the problem of background distortion between the outer contour and the outer boundary is alleviated. Since the contour deformation operation cannot guarantee that the deformation amounts of different contour points are the same, in order to keep the relative positions between the background points in the triangular face with the first expansion point and the second expansion point as vertices unchanged, the deformation amounts of adjacent second expansion points need to be consistent when adjusting the position of the first expansion point. In order to keep the relative positions between the background points in the triangular face with the contour point and the first expansion point as vertices unchanged, the deformation amount of the contour point needs to be consistent with the deformation amount of the corresponding first expansion point when adjusting the position of the first expansion point.

[0084] Based on this, for any target contour point, the second deformation parameters of the target expansion point and its adjacent expansion points can be calculated with the constraint condition that the difference between the deformation amount of any target contour point and its corresponding target expansion point on the buffer boundary is within a first preset range, and the difference between the deformation amount of the target expansion point and its adjacent expansion point is within a second preset range. The first preset range can be [-0.5, 0.5], [-0.3, 0.2] or [-1, 0.8] and the like, with the unit being pixels (px), and the second preset range can be [-0.3, 0.5], [-0.7, 0.2] or [-0.6, 0.4] and the like.

[0085] For example, the constraint condition one can be established with the difference between the deformation amount of the target expansion point and its adjacent expansion point being within the second preset range, and the constraint condition two can be established with the difference between the deformation amount of any target contour point and its corresponding target expansion point on the buffer boundary being within the first preset range. Based on the constraint condition one and the constraint condition two, the deformation amount of the first expansion point on the buffer boundary is solved. For the constraint condition one and the constraint condition two, please refer to the subsequent embodiments.

[0086] Further optionally, an embodiment for calculating the second deformation parameters of at least one target expansion point and its adjacent expansion points is also provided, as shown in FIG. 2b, the method comprises:

[0087] 201b, according to the positional relationship between the plurality of contour points on the outer contour and the corresponding relationship between the plurality of contour points and the expansion points on the buffer boundary, the plurality of contour points and the plurality of first expansion points on the buffer boundary are assigned serial numbers;

[0088] 202b, for the first target contour point, according to the first deformation parameter of the first target contour point and the above constraint condition, determine the current candidate deformation parameter of the first target extension point and its adjacent extension point corresponding to the first target contour point; The first target contour point is any target contour point;

[0089] Wherein, the first target extension point and its adjacent extension point belong to the first extension point, in order to facilitate the description of the simplification.

[0090] 203b, if the serial number of the first target extension point or its adjacent extension point belongs to the first serial number set, the current candidate deformation parameter of the first target extension point or its adjacent extension point is taken as the second deformation parameter;

[0091] 204b, if the serial number of the first target extension point or its adjacent extension point belongs to the second serial number set, the second deformation parameter is calculated according to the current candidate deformation parameter and the previous candidate deformation parameter of the first extension point or its adjacent extension point; The previous candidate deformation parameter is calculated according to the first deformation parameter of the previous contour point of the first target contour point.

[0092] In the embodiment shown in Figure 2b, the deformation parameter (i.e. the second deformation parameter) of the corresponding target extension point and the adjacent extension point of the target extension point is calculated based on the target contour point. In actual implementation process, the target contour point can be determined from a plurality of contour points, and then the second deformation parameter of the corresponding target extension point and the adjacent extension point of the target extension point of each target contour point is calculated according to the embodiment shown in Figure 2b. Wherein, the determination method of the target contour point can be based on the serial number assigned to the plurality of contour points in advance, the serial number corresponding to the target contour point can be recorded, and the target contour point can be identified based on the serial number. It is explained that the calculation of the second deformation parameter of the corresponding target extension point and the adjacent extension point of the target extension point of each target contour point is not limited to the embodiment shown in Figure 2b, but also can adopt the embodiment shown in Figure 2c:

[0093] 201c, according to the position relationship between the plurality of contour points on the outer contour and the corresponding relationship between the plurality of contour points and the extension points on the buffer boundary, a target serial number set is assigned to the plurality of contour points and the plurality of extension points on the buffer boundary;

[0094] 202c, for any contour point, according to the first deformation parameter of any contour point and the constraint condition, the first candidate deformation parameter of the extension point corresponding to any contour point and the second candidate deformation parameter of the next serial number adjacent extension point of the extension point are calculated, and the first deformation parameter of the non-target contour point is 0;

[0095] 203c, for any extension point on the buffer boundary, it is judged whether any extension point belongs to the first serial number set.

[0096] 204c, if the serial number of the any expansion point on the buffer boundary belongs to the first serial number set, then the second deformation parameter of the any expansion point is calculated according to the first candidate deformation parameter of the any expansion point;

[0097] 205c, if the serial number of the any expansion point on the buffer boundary does not belong to the first serial number set, i.e. the any expansion point belongs to the second serial number set, then the second deformation parameter of the any expansion point is calculated according to the first candidate deformation parameter and the second candidate deformation parameter of the any expansion point, the second candidate deformation parameter is a candidate deformation parameter calculated according to the first deformation parameter of the last adjacent contour point corresponding to the any expansion point or a default value; the union of the first serial number set and the second serial number set is the target serial number set.

[0098] In the case of calculating the second deformation parameter of each expansion point according to the method shown in FIG. 2c, the second deformation parameters of the target expansion point and its adjacent expansion points which need to be deformed can be obtained. Among them, the target expansion point can be selected after all the first expansion points are calculated; or the target expansion point and its adjacent expansion points can be selected while calculating the second deformation parameters of the expansion points.

[0099] Among them, for step 201c, the target serial number set is determined for the plurality of contour points and the plurality of expansion points on the buffer boundary according to the positional relationship between the plurality of contour points on the outer contour and the corresponding relationship between the plurality of contour points and the plurality of expansion points on the buffer boundary, the plurality of contour points and the plurality of expansion points on the buffer boundary share the same target serial number set. For example, there are 20 contour points on the outer contour, and correspondingly, there are 20 first expansion points on the buffer boundary. The 20 contour points can be represented by Li, i taking values from 0 to 19, and the 20 first expansion points are represented by Pi, i taking values from 0 to 19, wherein the value set of i is the serial number set.

[0100] For step 202c, for any contour point, the candidate deformation parameter of the expansion point corresponding to the any contour point and the candidate deformation parameter of the next serial number adjacent expansion point of the expansion point are calculated according to the first deformation parameter of the any contour point and the constraint condition. For example, in order to make the deformation amount of the any contour point and the expansion point corresponding to it on the buffer boundary the same, the first deformation parameter of the any contour point can be directly used as the candidate deformation parameter of the expansion point corresponding to the any contour point. For another example, in order to make the deformation amount of the first expansion point and its adjacent expansion point the same, the candidate deformation parameter of the expansion point corresponding to the any contour point can be used as the candidate deformation parameter of the next serial number adjacent expansion point of the expansion point.

[0101] It should be noted that the first deformation parameter of the target contour point is a non-zero value, and the first deformation parameter of the non-target contour point is 0, i.e. the non-target contour point does not deform.

[0102] For step 203c, if the serial number of any extension point belongs to the first serial number set, the second deformation parameter of any extension point is calculated according to the first candidate deformation parameter of the contour point corresponding to any extension point, for example, the first deformation parameter of the contour point corresponding to any extension point is directly taken as the second deformation parameter of the extension point, or the second deformation parameter of any extension point can be obtained by adding a set deformation threshold to the candidate deformation parameter of the contour point corresponding to any extension point, and the deformation threshold can be 0.1px, 0.3px or 0.5px, etc.

[0103] For step 204c, if the serial number of any extension point belongs to the second serial number set, the second deformation parameter of any extension point is calculated according to the first candidate deformation parameter and the second candidate deformation parameter of any extension point, and the second candidate deformation parameter is the first deformation parameter of the previous adjacent contour point corresponding to the contour point of any extension point. Wherein, if there is no previous adjacent contour point corresponding to the contour point of any extension point, the second candidate deformation parameter can take a default value, for example, the default value can be 0, at this time the first candidate deformation parameter of any extension point can be taken as the second deformation parameter of any extension point. For another example, the default value can be the first candidate deformation parameter, at this time the first candidate deformation parameter and the default value can be averaged to obtain the first candidate deformation parameter, that is, the second deformation parameter of any extension point is the first candidate deformation parameter.

[0104] In the case that there is a previous adjacent contour point corresponding to the contour point of any extension point, for example, any extension point D1 belongs to the second serial number set, the contour point corresponding to the extension point D1 is C1, and the previous serial number adjacent contour point of the contour point C1 is C0, the linear difference between the first deformation parameter e1 of the contour point C1 and the first deformation parameter e2 of the adjacent contour point C0 is performed to obtain the second deformation parameter of the any extension point D1. The second deformation parameter can be the average of e1 and e2, or the 3rd or 4th division point between e1 and e2, as the second deformation parameter.

[0105] The first serial number set and the second serial number set are not limited, and the union of the first serial number set and the second serial number set is the target serial number set. For example, taking the outer contour including 20 contour points and the buffer boundary including 20 first extension points as an example, the first serial number set can be an even set {0, 2, …, 18, 20}, and the second serial number set can be an odd set {1, 3, …, 17, 19}. For another example, the first serial number set can be {0, 3, 6, 9, 12, 15, 18}, and the second serial number set can be {1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19, 20}. For yet another example, the first serial number set can be {1, 4, 9, 13, 17, 19}, and the second serial number set can be {0, 2, 3, 5, 6, 7, 8, 10, 11, 12, 14, 15, 16, 18, 20}.

[0106] The following is an exemplary description.

[0107] The background region of the image to be processed is triangulated according to the contour points and the extension points. For example, a first type of triangular face is constructed in the region between the outer contour and the buffer boundary, taking the extension points on the buffer boundary and the plurality of contour points as vertices; and a second type of triangular face is constructed in the region between the buffer boundary and the outer boundary, taking the extension points on the outer boundary and the extension points on the buffer boundary as vertices.

[0108] As described above, since the contour morphing operation can make the morphing amount (offset amount) of different contour points different, so as to not guarantee that the morphing amount between adjacent contour points is the same, the first extension point on the buffer boundary is used instead of the contour point to construct a new triangular face with the second extension point on the outer boundary. In order to keep the relative positions between the background points (pixel points) in the second triangular face with the first extension point and the second extension point as vertices unchanged, it is necessary to ensure that the morphing amount of adjacent second extension points is consistent. This is constraint condition one, that is:

[0109] In order to keep the relative positions between the pixel points in the first triangular face with the contour point and the first extension point as vertices unchanged, it is necessary to ensure that the morphing amount of the contour point is consistent with the morphing amount of the corresponding first extension point, which is constraint condition two, that is:

[0110] Wherein, N is the number of contour points, N is a positive integer, and the value range of i is 0~N-1;

[0111] L i represents the coordinates of the i th face contour key point before morphing, L i is a known quantity;

[0112] L′ i represents the coordinates of the i th face contour key point after morphing, L′ iThe deformation of the first extension point on the buffer boundary can be calculated based on the contour deformation operation.

[0113] P i P i is a known quantity.

[0114] P i ′P i ′ is an unknown quantity and needs to be solved.

[0115] min is a minimum value.

[0116] The deformation of the first extension point on the buffer boundary can be obtained by solving the constraint condition one and the constraint condition two. In one possible implementation, first, the deformation of at least one target contour point on the outer contour is calculated based on the contour deformation operation, and details can be referred to the foregoing embodiments, which will not be described here. Wherein, the deformation of the non-target contour point is 0, that is, the target contour point is deformed, and the non-target contour point is not deformed. The deformation of the target contour point is substituted into the constraint condition two as (L i -L′ i ), and a local optimal solution P i ′ of the first extension point corresponding to the target contour point can be obtained, which is recorded as P′ i,i (the first candidate deformation parameter), representing the coordinate of the i-th first extension point after deformation calculated according to the deformation of the i-th contour point.

[0117] The local optimal solution P i ′ is substituted into the constraint condition one, and P′ i+1 is obtained, which is temporarily recorded as P′ i,i+1 (the second candidate deformation parameter), representing the coordinate of the i+1-th first extension point after deformation calculated according to the deformation of the i-th contour point. Finally, the coordinates of all first extension points after deformation are iteratively solved. For example, when i is the first serial number set (for example, even number), P i ′ takes P′ i,i , representing that when i is even, P′ i,i calculated according to the i-th contour point is directly taken as the deformation of the i-th first extension point; when i is the second serial number set (for example, odd number), P i ′ is linearly interpolated between P′ i,i and P′ i-1,i , representing that when i is odd, P′ i,i calculated according to the i-th contour point and P′ i,i-1 calculated according to the i-1-th contour point are linearly interpolated. Wherein, when linearly interpolated, the mean value, the bisection point, the trisection point or the quarter point, etc. can be taken.

[0118] As shown in Fig. 2d, an exemplary illustration is given for the case of i = 0, 1. In which, the contour points are represented by circles, and the first extension points are represented by triangular faces. The contour point L0 is deformed, which will affect both the first extension points P0 and P1. The deformation variable of L0 is In order to keep the relative deformation variable of L0 and P0 the same, the candidate deformation variable of P0 is also In order to keep the candidate deformation variable of P0 and P1 the same, the candidate deformation variable of P1 is also In which, the second candidate deformation parameter of P0 is The contour point L1 is also deformed, and the deformation variable is The deformation of L1 will affect the deformation of P1. In order to keep the deformation variable of L1 and P1 the same, the candidate deformation variable of P1 is also At this time, P1 is affected by the deformation of both L0 and L1, then the second deformation parameter of P1 can be linearly interpolated between and to obtain the second deformation parameter of P1.

[0119] In an alternative embodiment, the implementation of the triangular faces between the outer contour and the outer boundary is not limited. For example, the first type of triangular faces are constructed in the area between the outer contour and the buffer boundary, with the extension points on the buffer boundary and a plurality of contour points as vertices; the second type of triangular faces are constructed in the area between the buffer boundary and the outer boundary, with the extension points on the outer boundary and the extension points on the buffer boundary as vertices.

[0120] Correspondingly, the implementation of deforming other background points within the outer boundary based on the positions of at least one target contour point, at least one target extension point and its adjacent extension points after deformation, to obtain a target image, includes: determining target triangular faces which are deformed according to at least one target contour point, at least one target extension point and its adjacent extension points. The target triangular faces are triangular faces whose vertex positions are changed before and after deformation. In which, the target triangular faces can include the first type of triangular faces, or the second type of triangular faces, or both the first type of triangular faces and the second type of triangular faces. For example, each vertex in each first type of triangular face and each second type of triangular face is known, if the contour points, the first extension points and their adjacent extension points are deformed, then the triangular faces to which the deformed contour points, the first extension points and their adjacent extension points belong can be determined, and these triangular faces are taken as the target triangular faces. The number of the target triangular faces can be one, or multiple, for example, 2, 10 or 50, etc. With the relative positions of the background points within the target triangular faces and their vertices unchanged as the target, the positions of the background points within the target triangular faces are adjusted based on the positions of at least one target contour point after deformation, the positions of at least one target extension point and its adjacent extension points after deformation, to obtain a target image.

[0121] The relative position of the target triangular face internal background point and its vertex is unchanged, which can be achieved by keeping the barycentric coordinate system of each target triangular face unchanged before and after deformation. For example, according to the principle of keeping the barycentric coordinate system of the target triangular face unchanged before and after deformation, when the position of any vertex of the target triangular face changes, the position of each background point in the target triangular face is updated. For example, any target triangular face ΔABC before deformation, vertex A is a target contour point that deforms, vertex B is a first expansion point corresponding to the target contour point, and vertex C is a first expansion point adjacent to the next sequence number of the first expansion point. Any background point D in the target triangular face ΔABC is αA+βB+γC, and α+β+γ=1. After the vertices A, B and C deform, the deformed vertices A', B' and C' are obtained, and D' is any background point in ΔA'B'C'. The barycentric coordinate system coordinates of the D' point are equal to the barycentric coordinate system coordinates of the D point in ΔABC, and the barycentric coordinate system coordinates of the D' point are (α, β, γ), that is, D' = αA'+βB'+γC'. In this way, according to the principle of keeping the barycentric coordinate system of the target triangular face ΔABC before deformation unchanged after deformation, the position of the background point in the target triangular face can be updated to obtain the deformed target triangular face ΔA'B'C'.

[0122] When the background points in the target triangular face are deformed in the above manner for each target triangular face, that is, the position of the background point in the target triangular face is updated, the deformed background region following the contour deformation of the target object can be obtained. At this time, the deformed background region, the non-deformed background region and the contour-deformed target object constitute the target image. During the entire deformation process, the deformation of the background points in the outer boundary limited range based on the expansion points on the buffer boundary plays a deformation buffer role, to a certain extent, maintains the relative position relationship between the background points in the outer boundary limited range before and after deformation, maintains the continuity of the target object and the background region, alleviates the background distortion problem caused by the change of the outer contour, and improves the credibility of the contour deformation result of the target object.

[0123] The application also provides a video processing method, as shown in FIG. 2e, which comprises:

[0124] 201e, acquiring video data comprising multiple images, the multiple images comprising: at least one image to be processed containing a face region, the face region in the at least one image to be processed comprising a face contour, the face contour having a plurality of contour points, and an image region outside the face contour being a background region, the background region comprising background points;

[0125] 202e, in the background region, determine an outer boundary and a buffer boundary that fit the face contour, the outer boundary and the buffer boundary respectively have a plurality of extension points corresponding to a plurality of contour points, and the extension points belong to background points;

[0126] 203e, in response to the deformation operation on the face contour, taking the extension points on the buffer boundary as the buffer, performing deformation processing on the target contour points and the background points within the outer boundary to obtain a target image.

[0127] Among them, the video data can be live data, or short video data, which will be described below. Wherein, the video processing method can be executed on the terminal device, or executed on the server, which is not limited.

[0128] The following is an example of a live scene, where the video processing method is executed on the server.

[0129] The anchor records the live video in real time through the camera of the terminal device (referred to as the anchor end), and displays the live video corresponding to the live picture on the anchor end, the live picture includes the face region of the anchor and the "one-key beautifying", "slim face", "slim chin" or "small head" and other controls that can change the face contour; the anchor end can respond to the trigger operation of any control to obtain the target contour deformation parameter bound to the control; or, in response to the trigger operation of any control, a contour deformation parameter setting page corresponding to the control is displayed, and the adjusted target contour deformation parameter is obtained in response to the adjustment operation of the contour deformation parameter; the target contour deformation parameter and the current live picture are provided to the server, and the server executes the video processing method, that is, the target contour deformation parameter is used to deform the face region in the current live picture, and the background distortion problem is processed. The server provides the processed live picture to the viewing user corresponding to the viewing end. Further, the processed live picture can also be provided to the anchor end. For details of the server executing the video processing method, please refer to the foregoing embodiments, which will not be repeated here.

[0130] The following is an example of a video processing method executed on a terminal device in a short video scenario. A short video user captures a short video through the camera of the terminal device and displays a video page corresponding to the short video on the terminal device, which includes a leg region. In addition, the terminal device also includes various beautification controls, such as "one-key shaping", "slim legs", "straight legs", or "long legs", and the like, which are leg contour beautification controls. The terminal device can obtain target contour deformation parameters bound to the control in response to a triggering operation of the user on any control, or display a contour deformation parameter setting page corresponding to the control in response to a triggering operation on any control, and obtain adjusted target contour deformation parameters in response to an adjustment operation on the contour deformation parameters. Based on the target contour deformation parameters, the terminal device performs a contour deformation operation on the leg region in the video page and processes the background distortion problem through the video processing method. The terminal device can upload the short video after background distortion processing to the server, which is provided to the terminal devices of other viewers.

[0131] As shown in FIG. 3c, taking the target object as a face region and the background region as straight lines as an example, a "slim face" operation is performed on the face region. The left image in FIG. 3c is before the "slim face" operation is performed, and the background does not distort, and the lines in the background region are straight lines. The middle image in FIG. 3c is after the "slim face" operation is performed on the face region, and the background distorts, and the lines in the background region deform and become curves. At this time, the background distortion problem is not optimized. The right image in FIG. 3c is the image after the "slim face" operation is performed, and the method shown in FIG. 2a or FIG. 2c is performed to optimize the background distortion problem. In the image, the curves in the background region become straight lines, and the background distortion problem is alleviated. In the images in FIG. 3c, the background region in the lower right corner of the frame is taken as an example to illustrate the background deformation distortion effect. In order to more clearly highlight the degree of background distortion, the key deformation region is marked with a "black thick line" in the lower right corner of the frame in FIG. 3c. As shown by the "black thick line", it can be clearly seen that the method provided by the present application can significantly improve the background distortion problem. The image processing algorithm provided by the present application has the following advantages:

[0132] (1) A buffer boundary is introduced between the outer contour and the outer boundary, that is, a first extension point is introduced between the contour point and the second extension point. Further, the background region between the contour point and the second extension point is divided by the buffer boundary, and the range of the background region affected when the contour is deformed is reduced. Among them, the background points between the outer contour and the buffer boundary are more affected by the deformation of the contour point than the background points between the buffer boundary and the outer boundary. Compared with before the first extension point is introduced, the background distortion problem is alleviated;

[0133] (2) Based on the target of not changing the relative position relationship between the background points in the background area before and after deformation, a target function (constraint condition one and constraint condition two) is constructed to solve the offset of the first expansion point of the buffer boundary and to relieve the background distortion problem between the contour point and the second expansion point.

[0134] (3) In solving the background distortion problem, the conventional algorithm often selects to introduce a segmentation algorithm (such as a face segmentation algorithm) with higher complexity. In comparison, in the embodiments of the present application, the background distortion problem of the contour deformation is relieved without increasing the algorithm complexity, and the processing efficiency of relieving the background distortion problem is improved.

[0135] The detailed implementation and beneficial effects of each step in the method shown in FIGS. 2a-2c provided by the embodiments of the present application have been described in detail in the foregoing embodiments, and will not be described in detail here.

[0136] It should be noted that the execution subject of each step of the method provided by the above embodiments can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps 201a-203a can be device A; for another example, the execution subject of steps 201a and 202a can be device A, and the execution subject of step 203a can be device B; and the like.

[0137] In addition, in some of the processes described in the foregoing embodiments and the accompanying drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed or executed in parallel without the order in which they appear in this text. The serial numbers of the operations, such as 201a, 202a, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this text are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not of different types.

[0138] FIG. 4 is a structural schematic diagram of an image processing device provided by an exemplary embodiment of the present application, as shown in FIG. 4, the device includes an acquisition module 41, a determination module 42, and a deformation module 43.

[0139] The acquisition module 41 is configured to acquire a to-be-processed image, the to-be-processed image including a target object, the target object having an outer contour, the outer contour having a plurality of contour points, an image region outside the outer contour being a background region, the background region including background points.

[0140] The determining module 42 is configured to determine, in the background region, an outer boundary and a buffer boundary that fit the outer contour, the outer boundary and the buffer boundary having a plurality of extension points corresponding to a plurality of contour points respectively, the extension points belonging to background points; and the morphing module 43 is configured to, in response to a morphing operation on the outer contour, perform morphing processing on the target contour points and the background points within the outer boundary with the extension points on the buffer boundary as a buffer to obtain a target image.

[0141] In an optional embodiment, the morphing module is specifically configured to, in response to a contour morphing operation on the target object, perform morphing processing on at least one target contour point in the plurality of contour points according to a contour morphing parameter corresponding to the contour morphing operation; and perform morphing processing on the background points within the outer boundary with the extension points on the buffer boundary as a buffer between the at least one target contour point and the outer boundary to obtain a target image.

[0142] Optionally, the morphing module is specifically configured to, based on a position of the at least one target contour point after morphing, perform morphing processing on the background points within the outer boundary to obtain a target image, with a relative morphing variable between the at least one target contour point and the plurality of extension points on the buffer boundary satisfying a set condition as a target.

[0143] In an optional embodiment, the determining module is specifically configured to: calculate a reference point of the target object according to a target key point corresponding to the target object, the target key point including the plurality of contour points and / or at least one internal key point of the target object; and for any contour point, determine the extension points on the buffer boundary and the extension points on the outer boundary on an extension line of the reference point and the any contour point in sequence.

[0144] Optionally, the determining module is specifically configured to: take a background point corresponding to a first position on the extension line of the reference point and the any contour point as the extension point on the buffer boundary; and take a background point corresponding to a second position on the extension line of the reference point and the any contour point as the extension point on the outer boundary; wherein a second distance from the second position to the reference point is greater than a first distance from the first position to the reference point, and the first distance and the second distance satisfy a set proportional relationship.

[0145] In an optional embodiment, the deformation module is specifically configured to: in response to the contour deformation operation on the target object, acquire contour deformation parameters corresponding to the contour deformation operation, the contour deformation parameters including a contour deformation type and a contour deformation ratio; determine at least one target contour point that needs to be deformed from the plurality of contour points according to the contour deformation type; determine a first deformation parameter of each of the at least one target contour point according to the contour deformation ratio and a position of the at least one target contour point; and adjust the position of the at least one target contour point according to the first deformation parameter of each of the at least one target contour point to obtain a deformed position of the at least one target contour point.

[0146] In an optional embodiment, the deformation module is specifically configured to: calculate a second deformation parameter of each of the at least one target expansion point and the adjacent expansion point of the at least one target expansion point, with a relative deformation variable between the at least one target contour point and the at least one target expansion point corresponding to the at least one target contour point on the buffer boundary satisfying a first set condition and a relative deformation variable between the at least one target expansion point and the adjacent expansion point of the at least one target expansion point on the buffer boundary satisfying a second set condition as a constraint condition; adjust the position of the at least one target expansion point and the adjacent expansion point of the at least one target expansion point according to the second deformation parameter of each of the at least one target expansion point and the adjacent expansion point of the at least one target expansion point to obtain a deformed position of the at least one target expansion point and the adjacent expansion point of the at least one target expansion point; and perform deformation processing on other background points within the outer boundary based on the deformed position of the at least one target contour point, the at least one target expansion point and the adjacent expansion point of the at least one target expansion point to obtain the target image, the other background points being background points within the outer boundary except the target expansion points.

[0147] Optionally, the deformation module is specifically configured to: calculate a second deformation parameter of the target expansion point and the adjacent expansion point of the target expansion point for any target contour point, with a deformation variable difference between the target contour point and the target expansion point corresponding to the target contour point on the buffer boundary being within a first preset range and a deformation variable difference between the target expansion point and the adjacent expansion point of the target expansion point being within a second preset range as a constraint condition.

[0148] Further optionally, the morphing module is specifically configured to: assign serial numbers to the plurality of contour points and the plurality of extension points on the buffer boundary according to the positional relationship between the plurality of contour points on the outer contour and the corresponding relationship between the plurality of contour points and the plurality of extension points on the buffer boundary; for a first target contour point, determine a current candidate morphing parameter of a first target extension point corresponding to the first target contour point and neighboring extension points of the first target extension point according to a first morphing parameter of the first target contour point and a constraint condition; the first target contour point is any target contour point; if the serial number of the first target extension point or the neighboring extension points belongs to a first serial number set, the current candidate morphing parameter of the first target extension point or the neighboring extension points is taken as a second morphing parameter; if the serial number of the first target extension point or the neighboring extension points belongs to a second serial number set, the second morphing parameter is calculated according to the current candidate morphing parameter and a previous candidate morphing parameter of the first target extension point or the neighboring extension points; the previous candidate morphing parameter is calculated according to a first morphing parameter of a previous contour point of the first target contour point.

[0149] Further optionally, the apparatus further comprises: a construction module; the construction module is configured to: construct first type of triangular faces in a region between the outer contour and the buffer boundary with the plurality of extension points on the buffer boundary and the plurality of contour points as vertices; construct second type of triangular faces in a region between the buffer boundary and the outer boundary with the plurality of extension points on the outer boundary and the plurality of extension points on the buffer boundary as vertices; the morphing module is specifically configured to: determine a target triangular face in which morphing occurs according to at least one target contour point, at least one target extension point and neighboring extension points of the at least one target extension point, the target triangular face comprising the first type of triangular face and / or the second type of triangular face; adjust the positions of background points inside the target triangular face based on the positions of the at least one target contour point after morphing and the positions of the at least one target extension point and the neighboring extension points of the at least one target extension point after morphing, so as to obtain a target image, with the relative positions of the background points inside the target triangular face and the vertices of the target triangular face being unchanged.

[0150] The embodiments of the present application also provide a video processing apparatus, which comprises: an acquisition module, a determination module and a morphing module. The apparatus is the same as or similar to the apparatus shown in FIG. 4, and details can be referred to FIG. 4.

[0151] The acquisition module is configured to acquire video data comprising a plurality of images, the plurality of images comprising: at least one to-be-processed image comprising a face region, the face region in the at least one to-be-processed image comprising a face contour, the face contour having a plurality of contour points, and an image region outside the face contour being a background region, the background region comprising background points;

[0152] The determination module is configured to determine an outer boundary and a buffer boundary that are adapted to the face contour in the background region, the outer boundary and the buffer boundary respectively having a plurality of extension points corresponding to the plurality of contour points, the extension points belonging to the background points;

[0153] The deformation module is configured to perform deformation processing on the target contour point and the background point within the outer boundary in response to a deformation operation on the face contour, with the expansion point on the buffer boundary as a buffer, to obtain a target image.

[0154] The detailed implementation and beneficial effects of each step in the above device provided by the embodiments of the present application have been described in detail in the foregoing embodiments, and will not be described in detail here.

[0155] FIG. 5 is a structural schematic diagram of an electronic device provided by an exemplary embodiment of the present application. As shown in FIG. 5, the device includes a memory 54 and a processor 55.

[0156] The memory 54 is configured to store computer programs and can be configured to store various data to support operations on the electronic device. Examples of the data include instructions of any application program or method for operating on the electronic device, etc.

[0157] The processor 55 is coupled to the memory 54 and is configured to execute the computer programs in the memory 54, to: acquire a to-be-processed image, the to-be-processed image including a target object, the target object having an outer contour, the outer contour having a plurality of contour points thereon, an image region outside the outer contour being a background region, the background region including background points; in the background region, determine an outer boundary and a buffer boundary that fit the outer contour, the outer boundary and the buffer boundary respectively having a plurality of expansion points corresponding to the plurality of contour points thereon, the expansion points being background points; and in response to a deformation operation on the outer contour, perform deformation processing on a target contour point that is deformed and the background points within the outer boundary, with the expansion point on the buffer boundary as a buffer, to obtain a target image.

[0158] In an optional embodiment, when the processor 55 performs deformation processing on a target contour point that is deformed and the background points within the outer boundary, with the expansion point on the buffer boundary as a buffer, to obtain a target image in response to a deformation operation on the outer contour, the processor 55 is specifically configured to: in response to a contour deformation operation on the target object, perform deformation processing on at least one target contour point in the plurality of contour points according to a contour deformation parameter corresponding to the contour deformation operation, perform deformation processing on the background points within the outer boundary based on a deformation parameter of the at least one target contour point, with the expansion point on the buffer boundary as a buffer between the at least one target contour point and the outer boundary, to obtain a target image.

[0159] Optionally, when the processor 55 performs morphing processing on the background points within the outer boundary based on the morphing parameters of the at least one target contour point, and the buffer between the at least one target contour point and the outer boundary is the buffer point on the buffer boundary, the processor 55 is specifically configured to: perform morphing processing on the background points within the outer boundary based on the relative morphing variables between the at least one target contour point and the plurality of buffer points on the buffer boundary satisfying a set condition as the target, to obtain the target image.

[0160] In an optional embodiment, when the processor 55 determines the outer boundary and the buffer boundary that fit the outer contour in the background region, the processor 55 is specifically configured to: calculate a reference point of the target object according to the target key points corresponding to the target object, the target key points including the plurality of contour points and / or at least one internal key point of the target object; and for any contour point, sequentially determine the buffer point on the buffer boundary and the extension point on the outer boundary on the extension line of the reference point and the contour point.

[0161] Optionally, when the processor 55 determines the first extension point on the buffer boundary and the second extension point on the outer boundary on the extension line of the reference point and each contour point, the processor 55 is specifically configured to: take the background point corresponding to the first position on the extension line of the reference point and any contour point as the extension point on the buffer boundary; and take the background point corresponding to the second position on the extension line of the reference point and any contour point as the extension point on the outer boundary, wherein the second distance from the second position to the reference point is greater than the first distance from the first position to the reference point, and the first distance and the second distance satisfy a set proportional relationship.

[0162] In an optional embodiment, when the processor 55 performs morphing processing on the target contour points in the plurality of contour points according to the contour morphing parameters corresponding to the contour morphing operation in response to the contour morphing operation on the target object, the processor 55 is specifically configured to: obtain the contour morphing parameters corresponding to the contour morphing operation in response to the contour morphing operation on the target object, the contour morphing parameters including a contour morphing type and a contour morphing ratio; determine at least one target contour point that needs to be morphed from the plurality of contour points according to the contour morphing type; determine a first morphing parameter of each of the at least one target contour point according to the contour morphing ratio and the position of the at least one target contour point; and adjust the position of the at least one target contour point according to the first morphing parameter of each of the at least one target contour point to obtain the position of the at least one target contour point after morphing.

[0163] In an optional embodiment, when the processor 55 performs the morphing process on the background points within the outer boundary based on the deformed positions of the at least one target contour point, and the relative morphing variables between the at least one target contour point and the plurality of expansion points on the buffer boundary satisfy the set condition, to obtain the target image, the processor 55 is specifically configured to: calculate the second morphing parameters of the at least one target expansion point and the adjacent expansion points thereof, with the relative morphing variable between the at least one target contour point and the at least one target expansion point corresponding thereto on the buffer boundary satisfying a first set condition, and the relative morphing variable between the at least one target expansion point and the adjacent expansion points thereof on the buffer boundary satisfying a second set condition as constraint conditions; adjust the positions of the at least one target expansion point and the adjacent expansion points thereof according to the second morphing parameters of the at least one target expansion point and the adjacent expansion points thereof, to obtain the deformed positions of the at least one target expansion point and the adjacent expansion points thereof; and perform the morphing process on other background points within the outer boundary based on the deformed positions of the at least one target contour point, the at least one target expansion point and the adjacent expansion points thereof, to obtain the target image, wherein the other background points refer to the background points within the outer boundary except the target expansion points.

[0164] Optionally, when the processor 55 calculates the second morphing parameters of the at least one target expansion point and the adjacent expansion points thereof, with the relative morphing variable between the at least one target contour point and the at least one target expansion point corresponding thereto on the buffer boundary satisfying the first set condition, and the relative morphing variable between the at least one target expansion point and the adjacent expansion points thereof on the buffer boundary satisfying the second set condition as constraint conditions, the processor 55 is specifically configured to: for any target contour point, calculate the second morphing parameters of the target expansion point and the adjacent expansion points thereof, with the difference between the morphing variable of any target contour point and the target expansion point corresponding thereto on the buffer boundary being within a first preset range, and the difference between the morphing variable of the target expansion point and the adjacent expansion points thereof being within a second preset range as constraint conditions.

[0165] Further optionally, the processor 55 is configured to, when calculating the second morphing parameter of the at least one target extension point and its adjacent extension points, with the same morphing amount of any target contour point and its corresponding target extension point on the buffer boundary and the same morphing amount of the target extension point and its adjacent extension points as the target, specifically configured to: assign serial numbers to the plurality of contour points on the outer contour and the plurality of extension points on the buffer boundary according to the positional relationship between the plurality of contour points on the outer contour and the corresponding relationship between the plurality of contour points and the extension points on the buffer boundary; determine the current candidate morphing parameter of the first target extension point and its adjacent extension points corresponding to the first target contour point according to the first morphing parameter of the first target contour point and the constraint condition; the first target contour point is any target contour point; if the serial number of the first target extension point or its adjacent extension points belongs to the first serial number set, the current candidate morphing parameter of the first target extension point or its adjacent extension points is taken as the second morphing parameter; if the serial number of the first target extension point or its adjacent extension points belongs to the second serial number set, the second morphing parameter is calculated according to the current candidate morphing parameter and the previous candidate morphing parameter of the first target extension point or its adjacent extension points; the previous candidate morphing parameter is calculated according to the first morphing parameter of the previous contour point of the first target contour point.

[0166] Further optionally, the processor 55 is further configured to: construct a first type of triangular face in the region between the outer contour and the buffer boundary with the extension points on the buffer boundary and the plurality of contour points as vertices; construct a second type of triangular face in the region between the buffer boundary and the outer boundary with the extension points on the outer boundary and the extension points on the buffer boundary as vertices; when the processor 55 performs morphing processing on other background points within the outer boundary based on the positions of the at least one target contour point, the at least one target extension point and its adjacent extension points after morphing to obtain a target image, the processor 55 is specifically configured to: determine the target triangular face that occurs morphing according to the positions of the at least one target contour point, the at least one target extension point and its adjacent extension points after morphing, the target triangular face including the first type of triangular face and / or the second type of triangular face; adjust the positions of the background points inside the target triangular face based on the positions of the at least one target contour point after morphing and the positions of the at least one target extension point and its adjacent extension points after morphing, with the relative positions of the background points inside the target triangular face and their vertices unchanged as the target, to obtain the target image. The detailed implementation and beneficial effects of each step of the device shown in FIG. 5 provided by the embodiments of the present application have been described in detail in the foregoing embodiments, and will not be described in detail here.

[0167] Further, as shown in FIG. 5, the electronic device further includes a communication component 56, a display 57, a power supply component 58, an audio component 59, and other components. The components shown in FIG. 5 are only schematic and the electronic device can include components other than those shown in FIG. 5. In addition, the components in the dashed box in FIG. 5 are optional components and not mandatory components, and the specific components can be determined according to the product form of the electronic device. The electronic device of the present embodiment can be implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, or an IOT device, or a server device such as a general server, a cloud server, or a server array. If the electronic device of the present embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, or a smart phone, the electronic device can include the components in the dashed box in FIG. 5. If the electronic device of the present embodiment is implemented as a server device such as a general server, a cloud server, or a server array, the electronic device can not include the components in the dashed box in FIG. 5.

[0168] The present embodiment also provides an electronic device, which has the same or similar implementation structure as the electronic device shown in FIG. 5 and can be implemented with reference to the structure of the electronic device shown in FIG. 5. The electronic device provided by the present embodiment is mainly different from the electronic device in the embodiment shown in FIG. 5 in that the processor implements different functions by executing the computer program stored in the memory. For the electronic device provided by the present embodiment, the processor executing the computer program stored in the memory can be used to: obtain video data including multiple images, the multiple images including at least one image to be processed containing a face region, the face region in the at least one image to be processed including a face contour, the face contour having multiple contour points thereon, an image region outside the face contour being a background region, and the background region including background points; in the background region, determine an outer boundary and a buffer boundary that are adapted to the face contour, the outer boundary and the buffer boundary respectively having multiple extension points corresponding to the multiple contour points, and the extension points being background points; in response to a morphing operation on the face contour, perform morphing processing on a target contour point that has undergone morphing and a background point inside the outer boundary with the extension points on the buffer boundary as a buffer to obtain a target image.

[0169] The detailed implementation and advantages of the above electronic device provided by the present embodiment have been described in detail in the foregoing embodiments, and will not be described in detail here.

[0170] Correspondingly, the present embodiment also provides a computer readable storage medium storing a computer program, which can implement each step that can be executed by the electronic device in the method embodiments shown in FIGS. 2a to 2c and 2e when the computer program is executed.

[0171] Accordingly, the embodiments of the present application also provide a computer program product, including computer programs / instructions, which, when executed by a processor, cause the processor to implement each step of the method embodiments shown in FIGS. 2a-2c and 2e that can be performed by the electronic device.

[0172] The memory described above can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0173] The communication component described above is configured to facilitate communication between the device where the communication component is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, 2G, 3G, 4G / LTE, 5G, or the like mobile communication network, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel.

[0174] The display described above includes a screen, which can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect a duration and a pressure related to a touch or a slide operation.

[0175] The power supply component described above provides power to various components of the device where the power supply component is located. The power supply component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0176] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when the device in which the audio component is included is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in a memory or transmitted via a communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals.

[0177] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0178] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0179] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0181] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0182] Memory can include non-persistent memory and / or volatile memory, representing an example of computer readable media. Non-persistent memory and / or volatile memory can include, for example, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and / or the like. Memory is an example of computer readable media.

[0183] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0184] It should also be noted that the terms "comprising," "including," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0185] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0186] The above merely provides an example of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a to-be-processed image, the to-be-processed image comprising a target object, the target object having an outer contour, the outer contour having a plurality of contour points thereon, and an image region outside the outer contour being a background region, the background region comprising background points; in the background region, determining an outer boundary and a buffer boundary that fit the outer contour, the outer boundary and the buffer boundary respectively having a plurality of extension points corresponding to the plurality of contour points thereon, the extension points belonging to background points; in response to a deformation operation on the outer contour, performing deformation processing on a target contour point that is subjected to deformation and a background point inside the outer boundary with the extension points on the buffer boundary as a buffer to obtain a target image.

2. The method of claim 1, wherein, in response to a deformation operation on the outer contour, performing deformation processing on a target contour point that is subjected to deformation and a background point inside the outer boundary with the extension points on the buffer boundary as a buffer to obtain a target image, comprising: in response to a contour deformation operation on the target object, performing deformation processing on at least one target contour point in the plurality of contour points according to a contour deformation parameter corresponding to the contour deformation operation; based on the deformation parameter of the at least one target contour point, performing deformation processing on the background points inside the outer boundary with the extension points on the buffer boundary as a buffer between the at least one target contour point and the outer boundary to obtain a target image.

3. The method of claim 2, wherein, based on the deformation parameter of the at least one target contour point, performing deformation processing on the background points inside the outer boundary with the extension points on the buffer boundary as a buffer between the at least one target contour point and the outer boundary to obtain a target image, comprising: based on the position of the at least one target contour point after deformation, performing deformation processing on the background points inside the outer boundary to obtain a target image, with a relative deformation variable between the at least one target contour point and the plurality of extension points on the buffer boundary satisfying a set condition as a target.

4. The method of claim 1, wherein, in the background region, determining an outer boundary and a buffer boundary that fit the outer contour, comprising: calculating a reference point of the target object according to a target key point corresponding to the target object, the target key point comprising the plurality of contour points and / or at least one internal key point of the target object; for any contour point, determining the extension points on the buffer boundary and the extension points on the outer boundary on an extension line of the reference point and the any contour point in sequence.

5. The method of claim 4, wherein, determining a first extension point on the buffer boundary and a second extension point on the outer boundary on an extension line of the reference point and each contour point, comprising: taking a background point corresponding to a first position on the extension line of the reference point and the any contour point as the extension point on the buffer boundary; taking a background point corresponding to a second position on the extension line of the reference point and the any contour point as the extension point on the outer boundary; wherein a second distance from the second position to the reference point is greater than a first distance from the first position to the reference point, and the first distance and the second distance satisfy a set proportional relationship.

6. The method of claim 1, wherein, In response to a contour morphing operation on the target object, the target contour points in the plurality of contour points are morphed according to a contour morphing parameter corresponding to the contour morphing operation, including: In response to a contour morphing operation on the target object, the contour morphing parameter corresponding to the contour morphing operation is obtained, and the contour morphing parameter includes a contour morphing type and a contour morphing ratio; According to the contour morphing type, at least one target contour point that needs to be morphed is determined from the plurality of contour points; According to the contour morphing ratio and the position of the at least one target contour point, the first morphing parameter of the at least one target contour point is determined; According to the first morphing parameter of the at least one target contour point, the position of the at least one target contour point is adjusted to obtain the position of the at least one target contour point after morphing.

7. The method according to any one of claims 1 to 6, characterized in that, Based on the position of the at least one target contour point after morphing, the relative morphing variable between the at least one target contour point and the plurality of expansion points on the buffer boundary satisfies a set condition, and the background points inside the outer boundary are morphed to obtain a target image, including: The relative morphing variable between the at least one target contour point and the at least one target expansion point corresponding to the at least one target contour point on the buffer boundary satisfies a first set condition, and the relative morphing variable between the at least one target expansion point and its adjacent expansion point on the buffer boundary satisfies a second set condition as a constraint condition, and the second morphing parameter of the at least one target expansion point and its adjacent expansion point is calculated; According to the second morphing parameter of the at least one target expansion point and its adjacent expansion point, the position of the at least one target expansion point and its adjacent expansion point is adjusted to obtain the position of the at least one target expansion point and its adjacent expansion point after morphing; Based on the position of the at least one target contour point, the at least one target expansion point and its adjacent expansion point after morphing, the other background points inside the outer boundary are morphed to obtain a target image, and the other background points refer to the background points inside the outer boundary except the target expansion points.

8. The method of claim 7, wherein, The relative morphing variable between the at least one target contour point and the at least one target expansion point corresponding to the at least one target contour point on the buffer boundary satisfies a first set condition, and the relative morphing variable between the at least one target expansion point and its adjacent expansion point on the buffer boundary satisfies a second set condition as a constraint condition, and the second morphing parameter of the at least one target expansion point and its adjacent expansion point is calculated, including: For any target contour point, the difference between the morphing variable of the target contour point and the target expansion point corresponding to the target contour point on the buffer boundary is within a first preset range, and the difference between the morphing variable of the target expansion point and its adjacent expansion point is within a second preset range as a constraint condition, and the second morphing parameter of the target expansion point and its adjacent expansion point is calculated.

9. The method of claim 8, wherein, The second morphing parameter of the at least one target extension point and its adjacent extension points is calculated, including: According to the position relationship between the plurality of contour points on the outer contour and the corresponding relationship between the plurality of contour points and the extension points on the buffer boundary, the plurality of contour points and the plurality of extension points on the buffer boundary are assigned serial numbers; For the first target contour point, the current candidate morphing parameter of the first target extension point corresponding to the first target contour point and its adjacent extension points is determined according to the first morphing parameter of the first target contour point and the constraint condition; the first target contour point is any target contour point; If the serial number of the first target extension point or its adjacent extension points belongs to the first serial number set, the current candidate morphing parameter of the first target extension point or its adjacent extension points is taken as the second morphing parameter; if the serial number of the first target extension point or its adjacent extension points belongs to the second serial number set, the second morphing parameter is calculated according to the current candidate morphing parameter and the previous candidate morphing parameter of the first extension point or its adjacent extension points; the previous candidate morphing parameter is calculated according to the first morphing parameter of the previous contour point of the first target contour point.

10. The method of claim 9, wherein, Further comprising: A first type of triangular face is constructed in the region between the outer contour and the buffer boundary with the extension points on the buffer boundary and the plurality of contour points as vertices; A second type of triangular face is constructed in the region between the buffer boundary and the outer boundary with the extension points on the outer boundary and the extension points on the buffer boundary as vertices; Based on the positions of the at least one target contour point, the at least one target extension point and its adjacent extension points after morphing, the other background points inside the outer boundary are morphed to obtain a target image, including: According to the at least one target contour point, the at least one target extension point and its adjacent extension points, a target triangular face that occurs morphing is determined, and the target triangular face includes the first type of triangular face and / or the second type of triangular face; The relative positions of the background points inside the target triangular face and their vertices are kept unchanged, and based on the positions of the at least one target contour point after morphing and the positions of the at least one target extension point and its adjacent extension points after morphing, the positions of the background points inside the target triangular face are adjusted to obtain the target image.

11. A method for video processing, comprising: Including: Video data including a plurality of images is acquired, the plurality of images including: at least one image to be processed containing a face region, the face region in the at least one image to be processed including a face contour, the face contour having a plurality of contour points, and an image region outside the face contour being a background region, the background region including background points; In the background region, an outer boundary and a buffer boundary that adapt to the face contour are determined, the outer boundary and the buffer boundary respectively having a plurality of extension points corresponding to the plurality of contour points, and the extension points belonging to background points; In response to the morphing operation on the human face contour, the target contour points and the background points within the outer boundary are morphed with the extended points on the buffer boundary as a buffer to obtain a target image.

12. An electronic device, comprising: Comprising: a memory and a processor; the memory, configured to store a computer program; and the processor, coupled to the memory, configured to execute the computer program to implement the steps in the method of any one of claims 1-10 and claim 11.

13. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to implement the steps in the method of any one of claims 1-10 and claim 11.

14. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, cause the processor to implement the steps in the method of any one of claims 1-10 and claim 11.

Images