Image processing method, server and storage medium

Abstract

The embodiment of the present application discloses an image processing method, device, server and storage medium, which includes: obtaining a target image and determining a driving video selected by a user; when it is detected that the facial image in the target image is in a tilted state, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is intercepted to obtain a first facial image. The first facial image and the driving video are input into a first-order motion model, and the first facial image is driven by the first-order motion model to simulate the driving video movement to obtain a first facial image driving video. The first facial image driving video is pasted back to the position of the facial image in the target image to obtain a target driving video. By adopting the embodiment of the present application, the driving effect on the target image can be improved, and the applicability is high.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, a server, and a storage medium. Background Art

[0002] With the continuous development of society, electronic devices are playing an increasingly important role in people's learning, entertainment, work, etc. These electronic devices are usually equipped with cameras, which can be used for taking pictures, recording videos or live broadcasts. Among them, in applications such as making expressions or special effects, the user uploads an original facial image that meets the specifications. The application drives the original facial image through a first-order motion model based on the original facial image and the driving video, so that the facial state in the driving video is expressed on the original facial image, that is, the static original facial image is moved. However, this places strict requirements on the original facial image uploaded by the user. When the original facial image has a small facial proportion, the face is tilted, or the facial position is significantly different from the facial position in the driving video, the driving effect is poor, and even the portrait may be distorted. Based on this, how to improve the driving effect of the original facial image has become one of the current problems that need to be solved urgently. Summary of the Invention

[0003] The embodiments of the present application provide an image processing method, device, server, and storage medium, which can improve the driving effect of a target image and have high applicability.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] Acquire target images and determine driving videos;

[0006] When it is detected that the facial image in the target image is in a tilted state, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is intercepted to obtain a first facial image;

[0007] Inputting the first facial image and the driving video into a first-order motion model, so that the first-order motion model drives the first facial image to simulate the motion of the driving video, thereby obtaining a first facial image driving video;

[0008] Paste the first facial image driven video back to the target image at the location of the facial image to obtain a target driven video.

[0009] In conjunction with the first aspect, in a possible implementation, determining the driving video includes:

[0010] Acquire an initial driving video, wherein the initial driving video includes multiple reference image frames, each of the multiple reference image frames includes a reference facial image;

[0011] When it is detected that the reference facial image included in the first reference image frame among the multiple reference image frames is in a tilted state, rotating each reference image frame among the multiple reference image frames so that the angle of the reference facial image is the preset angle;

[0012] The driving video is obtained based on the rotated multiple-frame reference image.

[0013] In combination with the first aspect, in a possible implementation, obtaining the driving video based on the rotated multiple reference image frames includes:

[0014] intercepting a reference facial image included in each frame of the rotated multiple reference image frames to obtain multiple reference facial images;

[0015] A driving video is generated according to a preset frame rate and the plurality of reference facial images.

[0016] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0017] Performing facial key point detection processing on the target image to obtain position information corresponding to a plurality of facial key points included in the target image;

[0018] It is determined whether the facial image in the target image is in a tilted state according to the position information of the multiple facial key points.

[0019] In conjunction with the first aspect, in one possible implementation, the multiple facial key points include a left eye and a right eye, and the position information of each facial key point includes a horizontal coordinate value and a vertical coordinate value; and determining whether the facial image in the target image is tilted based on the position information of the multiple facial key points includes:

[0020] Determine an angle between a line connecting the left eye and the right eye and a horizontal direction according to an absolute value of a difference between the abscissa value of the left eye and the abscissa value of the right eye, and an absolute value of a difference between the ordinate value of the left eye and the ordinate value of the right eye;

[0021] If the angle is greater than or equal to a first preset threshold, it is determined that the facial image in the target image is in a tilted state.

[0022] In combination with the first aspect, in one possible implementation, the first facial image driven video includes multiple frames of facial driven images;

[0023] Pasting the first facial image driven video back to the target image at the location of the facial image to obtain the target driven video includes:

[0024] Acquire a target mask image corresponding to a target facial drive image in the first facial image driven video, where the target facial drive image is any one of the multiple facial drive images;

[0025] fusing the target facial drive image and the rotated target image according to the target mask image to obtain a fused image corresponding to the target facial drive image;

[0026] A target driving video is generated according to the multiple frames of fused images corresponding to the multiple frames of facial driving images.

[0027] In conjunction with the first aspect, in one possible implementation, obtaining a target mask image corresponding to a target facial driving image in the first facial image driving video includes:

[0028] performing semantic segmentation processing on the target facial driving image in the first facial image driving video to obtain an initial mask image;

[0029] The initial target mask image is subjected to a feathering process, and the initial target mask image after the feathering process is determined as the target mask image corresponding to the target facial driving image.

[0030] In combination with the first aspect, in one possible implementation, fusing the target facial driving image and the rotated target image according to the target mask image to obtain a fused image corresponding to the target facial driving image includes:

[0031] Obtain the Gaussian pyramid Gm corresponding to the target mask image;

[0032] Obtaining a first Laplacian pyramid La corresponding to the target facial drive image and a second Laplacian pyramid Lb corresponding to the rotated target image;

[0033] A third Laplacian pyramid Ls is generated according to the Gaussian pyramid Gm, the first Laplacian pyramid La, and the second Laplacian pyramid Lb, so as to determine the fused image through the third Laplacian pyramid Ls.

[0034] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0035] A transceiver unit, used to acquire target images and determine driving video;

[0036] a processing unit configured to, when detecting that the facial image in the target image is in a tilted state, rotate the target image to correct the angle of the facial image to a preset angle, and intercept the facial image in the rotated target image to obtain a first facial image;

[0037] The processing unit is configured to input the first facial image and the driving video into a first-order motion model, so that the first-order motion model drives the first facial image to simulate the motion of the driving video, thereby obtaining the first facial image driving video;

[0038] The processing unit is configured to paste the first facial image driven video back to the position of the facial image in the target image to obtain a target driven video.

[0039] In conjunction with the second aspect, in one possible implementation, the processing unit is configured to:

[0040] Acquire an initial driving video, wherein the initial driving video includes multiple reference image frames, each of the multiple reference image frames includes a reference facial image;

[0041] When it is detected that the reference facial image included in the first reference image frame among the multiple reference image frames is in a tilted state, rotating each reference image frame among the multiple reference image frames so that the angle of the reference facial image is the preset angle;

[0042] The driving video is obtained based on the rotated multiple-frame reference image.

[0043] In conjunction with the second aspect, in one possible implementation, the processing unit is configured to:

[0044] intercepting a reference facial image included in each frame of the rotated multiple reference image frames to obtain multiple reference facial images;

[0045] A driving video is generated according to a preset frame rate and the plurality of reference facial images.

[0046] In conjunction with the second aspect, in one possible implementation, the processing unit is configured to:

[0047] Performing facial key point detection processing on the target image to obtain position information corresponding to a plurality of facial key points included in the target image;

[0048] It is determined whether the facial image in the target image is in a tilted state according to the position information of the multiple facial key points.

[0049] In conjunction with the second aspect, in one possible implementation, the plurality of facial key points include a left eye and a right eye, and the position information of each facial key point includes a horizontal coordinate value and a vertical coordinate value; and the processing unit is configured to:

[0050] Determine an angle between a line connecting the left eye and the right eye and a horizontal direction according to an absolute value of a difference between the abscissa value of the left eye and the abscissa value of the right eye, and an absolute value of a difference between the ordinate value of the left eye and the ordinate value of the right eye;

[0051] If the angle is greater than or equal to a first preset threshold, it is determined that the facial image in the target image is in a tilted state.

[0052] In conjunction with the second aspect, in one possible implementation, the first facial image driven video includes multiple frames of facial driven images;

[0053] The processing unit is used for:

[0054] Acquire a target mask image corresponding to a target facial drive image in the first facial image driven video, where the target facial drive image is any one of the multiple facial drive images;

[0055] fusing the target facial drive image and the rotated target image according to the target mask image to obtain a fused image corresponding to the target facial drive image;

[0056] A target driving video is generated according to the multiple frames of fused images corresponding to the multiple frames of facial driving images.

[0057] In conjunction with the second aspect, in one possible implementation, the processing unit is configured to:

[0058] performing semantic segmentation processing on the target facial driving image in the first facial image driving video to obtain an initial mask image;

[0059] The initial target mask image is subjected to a feathering process, and the initial target mask image after the feathering process is determined as the target mask image corresponding to the target facial driving image.

[0060] In conjunction with the second aspect, in one possible implementation, the processing unit is configured to:

[0061] Obtain the Gaussian pyramid Gm corresponding to the target mask image;

[0062] Obtaining a first Laplacian pyramid La corresponding to the target facial drive image and a second Laplacian pyramid Lb corresponding to the rotated target image;

[0063] A third Laplacian pyramid Ls is generated according to the Gaussian pyramid Gm, the first Laplacian pyramid La, and the second Laplacian pyramid Lb, so as to determine the fused image through the third Laplacian pyramid Ls.

[0064] In a third aspect, an embodiment of the present application provides a terminal device, comprising a processor, a memory, and a transceiver, the processor, the memory, and the transceiver being interconnected. The memory is configured to store a computer program that supports the terminal device in executing the method provided in the first aspect and / or any possible implementation of the first aspect, the computer program comprising program instructions, and the processor and the transceiver are configured to invoke the program instructions to execute the method provided in the first aspect and / or any possible implementation of the first aspect.

[0065] In a fourth aspect, an embodiment of the present application provides a server, comprising a processor, a memory, and a transceiver, the processor, the memory, and the transceiver being interconnected. The memory is configured to store a computer program that supports the server in executing the method provided by the first aspect and / or any possible implementation of the first aspect, the computer program comprising program instructions, and the processor and the transceiver are configured to invoke the program instructions to execute the method provided by the first aspect and / or any possible implementation of the first aspect.

[0066] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program includes program instructions, which, when executed by a processor, enable the processor to execute the method provided by the above-mentioned first aspect and / or any possible implementation method of the first aspect.

[0067] In an embodiment of the present application, a target image is obtained and a driving video selected by a user is determined; when it is detected that the facial image in the target image is in a tilted state, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is intercepted to obtain a first facial image. The first facial image and the driving video are input into a first-order motion model, and the first facial image is driven by the first-order motion model to simulate the driving video movement to obtain a first facial image driving video. The first facial image driving video is pasted back to the position of the facial image in the target image to obtain a target driving video. In an embodiment of the present application, in order to ensure the quality of the facial image, it is necessary to first perform rotation correction and alignment on the facial image in the target image, and crop out the facial image in the target image to ensure that no matter whether the face in the input image is tilted or the facial image accounts for a very small proportion, the input image can be effectively driven, thereby improving the driving effect on the target image and having high applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0069] Figure 1 This is a schematic diagram of an image processing scenario provided by an embodiment of the present application;

[0070] Figure 2 This is a flow chart of an image processing method provided by an embodiment of the present application;

[0071] Figure 3 This is a flow chart of automatic face alignment based on face detection provided by an embodiment of the present application;

[0072] Figure 4 is a schematic structural diagram of a first-order motion model provided in an embodiment of the present application;

[0073] Figure 5 This is a schematic diagram of a scene for semantic segmentation of a portrait provided by an embodiment of the present application;

[0074] Figure 6 Schematic diagram of the image processing process provided by the embodiment of the present application;

[0075] Figure 7 is a structural diagram of an image processing device provided in an embodiment of the present application;

[0076] Figure 8 It is a structural diagram of the network device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0078] The embodiments of the present application relate to artificial intelligence (AI) and machine learning (ML). Among them, AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It mainly produces a new intelligent machine that can respond in a similar way to human intelligence by understanding the essence of intelligence, so that the intelligent machine has multiple functions such as perception, reasoning and decision-making.

[0079] AI technology is a comprehensive discipline encompassing several key areas, including computer vision (CV), speech processing, natural language processing, and machine learning (ML) / deep learning. Computer vision is the study of how machines can "see." Specifically, it involves using cameras and computers to replace the human eye in identifying, tracking, and measuring objects. This is followed by further image processing, which transforms the computer's image into an image more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. This discipline typically encompasses techniques such as image processing, video processing, video semantic understanding, and video content / behavior recognition.

[0080] Machine learning is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of AI and the fundamental way to make computers intelligent. Its applications span all areas of artificial intelligence. Machine learning / deep learning typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0081] Based on computer vision technology and machine learning technology in AI technology, an embodiment of the present application provides an image processing method, which includes: obtaining a target image and determining a driving video; when it is detected that the facial image in the target image is in a tilted state, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is intercepted to obtain a first facial image. The first facial image and the driving video are input into a first-order motion model, and the first facial image is driven by the first-order motion model to simulate the driving video movement to obtain a first facial image driving video. The first facial image driving video is pasted back to the position of the facial image in the target image to obtain a target driving video.

[0082] See Figure 1 , Figure 1 This is a schematic diagram of a scene of image processing provided by an embodiment of the present application. Figure 1 As shown, the image processing scenario includes a terminal device 101 and a server 102. The terminal device 101 is a device used by a user, and may include, but is not limited to, a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a portable personal computer, a mobile Internet device (MID), and the like; the terminal device is equipped with a display device, which may be a display, a display screen, a touch screen, etc., and the touch screen may be a touch screen, a touch panel, etc., which is not limited in the present embodiment.

[0083] The server 102 refers to a background device that can process the target image and the selected driving video provided by the terminal device 101. After obtaining the target driving video based on the target image and the driving video, the server 102 can return the target driving video to the terminal device 101. The server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. In addition, multiple servers can be grouped into a blockchain network, and each server is a node in the blockchain network. The terminal device 101 and the server 102 can be directly or indirectly connected through wired communication or wireless communication, and this application does not limit this.

[0084] It should be noted that Figure 1The number of terminal devices and servers in the model processing scenario shown is for example only. For example, there can be multiple terminal devices and servers, and this application does not limit the number of terminal devices and servers. The methods provided in the embodiments of this application can be applied to servers as well as terminal devices, and are not limited here. For ease of description, the embodiments of this application may collectively refer to terminal devices and servers as network devices, and the following description will take network devices as an example.

[0085] The following will be combined Figures 2 to 8 The methods and related devices provided in the embodiments of the present application are described in detail respectively.

[0086] See Figure 2 , Figure 2 A flowchart of an image processing method provided in an embodiment of the present application is provided. The method provided in an embodiment of the present application may include the following steps S201 to S204:

[0087] S201: Acquire a target image and determine a driving video.

[0088] In some feasible implementations, the network device obtains a target image and determines a driving video. Specifically, the network device can obtain a target image uploaded by a user or selected in a preset image library, and determine a driving video uploaded by a user or selected in a preset video library. The driving video includes multiple frames of reference images, and each frame of the reference image includes a reference image object. For example, when a user wants to drive a still image of a person in a picture to move, the user can upload a picture including a person image (for example, the person image can be the user himself, or it can also be the user's friend, or it can also be the user's favorite star, etc.), and select a video that wants to imitate the action. Generally speaking, the above-mentioned picture including the person image is the target image, and the selected video is the driving video. Therefore, the network device can process the target image and driving video uploaded by the user.

[0089] It is understandable that the driving video can be a video pre-stored in the network device, so the user can select the video he wants to imitate from the driving video list or from the driving video library as the driving video. Generally speaking, the various videos included in the driving video list are strictly screened and have been pre-evaluated, that is, videos that meet the conditions of being a driving video. Optionally, if there is no driving video that the user wants to imitate in the pre-established driving video list, the user can also upload a video that he wants to imitate as the driving video. Generally speaking, if the user uploads a video as the driving video, the video uploaded by the user needs to be tested, evaluated and screened first, and when it is determined that the uploaded video meets the conditions of being a driving video, the target image is processed based on the video uploaded by the user to drive the movement of the person in the target image. For the convenience of description, the video uploaded by the user can be described as the initial driving video.

[0090] The detection, evaluation, and screening of the user-uploaded video (i.e., the initial driving video) can be understood as follows: after obtaining the initial driving video uploaded by the user, the initial driving video can be deframed to obtain multiple frames of images that constitute the initial driving video. Images in the multiple frames that include the reference facial image are then used as reference images to obtain multiple reference images. In other words, images in the multiple frames that do not include the reference facial image (e.g., a face) can be discarded, and only images that include the reference facial image are retained as reference images. Then, the first frame in the multiple reference images is detected, and when the reference facial image included in the first frame is detected to be tilted, each reference image in the multiple reference images is rotated so that the angle of the reference facial image is a preset angle. Furthermore, the driving video can be obtained based on the rotated multiple reference images. Alternatively, each frame in the multiple reference images can be detected, and when the reference facial image included in any of the multiple reference images is detected to be tilted, the reference image is rotated to ensure that the angle of the reference facial image in each frame in the multiple reference images is a preset angle. For ease of understanding, this application uses the example of detecting the first frame image in multiple reference images, and when it is detected that the reference facial image included in the first frame reference image in the multiple reference images is in a tilted state, rotating each frame reference image in the multiple reference images so that the angle of the reference facial image is a preset angle.

[0091] The first frame of a plurality of reference images is detected, and when it is detected that the reference facial image included in the first frame of the plurality of reference images is tilted, each reference frame in the plurality of reference images is rotated so that the angle of the reference facial image is a preset angle. This can be understood as: performing facial key point detection on the first frame of the plurality of reference images to obtain position information of multiple facial key points included in the first frame. Key points representing the left and right eyes are selected from the plurality of facial key points, and the tilt angle of the line connecting the two points relative to the horizontal is calculated. Each reference frame in the plurality of reference images is then rotated based on the tilt angle so that the angle of the reference facial image in the plurality of reference images is a preset angle. This can also be understood as causing the tilt angle of the line connecting the two eyes in the plurality of reference images to be a preset angle relative to the horizontal. The preset angle can be set to 0 degrees, for example, and is not limited here. It can be understood that the position information of each facial key point includes a horizontal coordinate value and a vertical coordinate value. Therefore, the tilt angle of the line connecting the left and right eyes relative to the horizontal can satisfy: Wherein, θ is the tilt angle, d2 represents the absolute value of the difference between the horizontal coordinate value of the left eye and the horizontal coordinate value of the right eye, and d1 represents the absolute value of the difference between the vertical coordinate value of the left eye and the vertical coordinate value of the right eye.

[0092] The aforementioned process of obtaining a driving video based on the rotated multiple reference image frames can be understood as follows: intercepting the reference facial image included in each of the rotated multiple reference image frames to obtain multiple reference facial images, and generating a driving video based on a preset frame rate and the multiple reference facial images. To distinguish it from other driving videos such as the initial driving video, the driving video generated from the reference facial images can be referred to as a reference driving video. The driving video input into the first-order motion model in the subsequent step 203 is the reference driving video. Specifically, the driving video can be generated by playing the multiple reference facial images at the preset frame rate. The preset frame rate can be the frame rate of the initial driving video, which is not a limitation here.

[0093] S202: When it is detected that the facial image in the target image is tilted, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is captured to obtain a first facial image.

[0094] In some feasible implementations, when the network device detects that the facial image in the target image is tilted, it rotates the target image to correct the facial image by a predetermined angle, and then crops the facial image in the rotated target image to obtain a first facial image. In other words, to ensure the quality of the facial image, it is necessary to first perform rotation correction and alignment on the facial image in the target image, and then crop the facial image in the target image to ensure that the input image can be effectively driven regardless of whether the face in the input image is tilted or the facial image accounts for a small proportion.

[0095] The process of detecting whether the facial image in the target image is in a tilted state can be understood as: performing facial key point detection processing on the target image to obtain position information corresponding to multiple facial key points included in the target image. Determine whether the facial image in the target image is in a tilted state based on the position information of the multiple facial key points. The multiple facial key points include a left eye and a right eye, and the position information of each facial key point includes a horizontal coordinate value and a vertical coordinate value. Determining whether the facial image in the target image is in a tilted state based on the position information of the multiple facial key points can be understood as: determining the angle between the line connecting the left eye and the right eye and the horizontal direction based on the absolute value of the difference between the horizontal coordinate value of the left eye and the horizontal coordinate value of the right eye, and the absolute value of the difference between the vertical coordinate value of the left eye and the vertical coordinate value of the right eye. If the angle is greater than or equal to a first preset threshold, it is determined that the facial image in the target image is in a tilted state. Then, the target image is rotated to correct the angle of the facial image to a preset angle. The angle between the line connecting the left eye and the right eye of the facial image in the target image and the horizontal direction satisfies: Wherein, θ is the tilt angle (i.e., the included angle), d2 represents the absolute value of the difference between the abscissa value of the left eye and the abscissa value of the right eye, and d1 represents the absolute value of the difference between the ordinate value of the left eye and the ordinate value of the right eye.

[0096] For example, see Figure 3 , Figure 3 FIG is a flow chart of automatic face alignment based on face detection provided by an embodiment of the present application. Figure 3 Figure (a) shows that after performing facial key point detection on the target image, multiple facial key points are obtained. Among them, the key points representing the left eye and the right eye are selected, and the tilt angle of the line between the two points compared to the horizontal is calculated. Therefore, the image (i.e., the target image) can be rotated according to the tilt angle so that the tilt angle of the line between the two eyes compared to the horizontal becomes 0 degrees (as shown in Figure 1). Figure 3 (b) shown in FIG). Further, according to the face range parameters obtained based on facial key point detection, a face interception function is written, and the range of the intercepted face is adjusted according to the face range parameters to intercept the face included in the rotated target image (such as Figure 3 (c) shown in Figure ).

[0097] S203: Input the first facial image and the driving video into a first-order motion model, so that the first-order motion model drives the first facial image to simulate the driving video motion, thereby obtaining the first facial image driving video.

[0098] In some feasible implementations, after the network device obtains the facial image (i.e., the first facial image) in the rotated target image, it can input the first facial image and the driving video into a first-order motion model. The first-order motion model drives the first facial image to simulate the motion of the driving video, thereby generating the first facial image-driven video. The first-order motion model utilizes a set of self-learned key points and local affine transformations to build a complex motion model. This is designed to address the poor generation quality of traditional models when the pose of large objects changes dynamically. Furthermore, by introducing an occlusion-aware generator, it can indicate portions of objects that are not visible in the original image, thereby leveraging context to infer the content of these portions.

[0099] That is to say, in an embodiment of the present application, the input of the first-order motion model can be a source image (i.e., the first facial image in the target image in the present application) and a driving video provided by the user. Then, the first-order motion model can drive the source image to perform the same action according to the action in the provided driving video.

[0100] See Figure 4 , Figure 4 This is a schematic diagram of the structure of the first-order motion model provided in the embodiment of this application. Figure 4 As shown in the figure, the first-order motion model consists of two main components: a motion estimation module and an image generation module. In the motion estimation module, the model separates the appearance and motion information of the target object through self-supervised learning and performs feature representation. In the image generation module, the model models the occlusion that occurs during the target motion, and then extracts the appearance information from the given celebrity image, and combines it with the previously obtained feature representation to perform video synthesis. In other words, the first-order motion model uses a large number of video sequences of the same object extracted from the same video to reconstruct the training video by combining single frames and the learned potential representation of motion in the video. Among them, the motion estimation module in the first-order motion model can predict a frame in the driving video (driving video) to the source video (source image). The dense motion field TS←D between these images is obtained in the following steps: The first step is to learn the trained key point detector network (i.e., Figure 4The KeypointDetecotor in the CNN predicts the positions of the key points respectively, and then approximates the two transformations with a set of sparse trajectories obtained through the key points. This sparse motion representation is very suitable for animation because the key points of the source image can be moved using the key point trajectories in the driving video during testing. The motion near each key point is then modeled using a local affine transformation. Compared with using only key point displacements, local affine transformations allow the model to model a larger family of transformations. After obtaining the key point positions and the parameters of each affine transformation, Taylor's first-order expansion is used to represent TD←R and TS←R through a set of key point positions and affine transformations. In the second step, the dense motion network ( Figure 4 The Dense Motion Network in

[15] combines the local approximations to obtain a dense motion field TS←D. In the third step, in addition to the dense motion field, the dense motion network also outputs an occlusion mask OS←D, which not only indicates which parts of the figure are occluded, but also can reconstruct the occluded parts of the image by deforming the source image (i.e., inferred from the context semantics). Finally, the image generation module renders the image according to the driving source object of the action provided in the driving video. Here, a generator network G (encoder-decoder structure) is used, which warps the source image according to the motion field TS←D and patches the occluded image parts in the source image to obtain the driven image. Therefore, based on the multiple frames of images obtained after driving, a new driving video (i.e., the first facial image driving video) can be generated.

[0101] S204: Paste the first facial image driven video back to the position of the facial image in the target image to obtain a target driven video.

[0102] In some feasible implementations, the network device pastes the first facial image-driven video back to the location of the facial image in the target image to obtain the target-driven video. It is understood that the first facial image-driven video includes multiple frames of facial image-driven images, i.e., the first facial image-driven video is composed of multiple frames of facial image-driven images. Pasting the first facial image-driven video back to the location of the facial image in the target image to obtain the target-driven video can be understood as obtaining a target mask image corresponding to the target facial image in the first facial image-driven video. Based on the target mask image, the target facial image (or the target facial image after border filling) and the rotated target image (i.e., the target image obtained by rotating the target image before intercepting the first facial image in step S202) are fused to obtain a fused image corresponding to the target facial image. The target-driven video is then generated based on the multiple fused images corresponding to the multiple frames of facial image-driven images. The multiple frames of facial image-driven images include the target facial image, i.e., the target facial image is any one of the multiple frames of facial image-driven images. That is to say, each frame of the facial-driven image in the multiple frames can be used as the target facial-driven image in turn, and an operation of obtaining a target mask image corresponding to the target facial-driven image is performed, and an operation of fusing the target facial-driven image and the rotated target image according to the target mask image is performed to obtain a multi-frame fused image corresponding to the multiple frames of facial-driven images.

[0103] Specifically, obtaining a target mask image corresponding to a target facial driving image in the first facial image-driven video can be understood as: performing semantic segmentation on the target facial driving image in the first facial image-driven video to obtain an initial mask image, performing feathering on the initial target mask image, and determining the feathered initial target mask image as the target mask image corresponding to the target facial driving image. Furthermore, fusing the target facial driving image and the rotated target image based on the target mask image to obtain a fused image corresponding to the target facial driving image can be understood as: obtaining a Gaussian pyramid Gm corresponding to the target mask image, obtaining a first Laplacian pyramid La corresponding to the target facial driving image (or the target facial driving image after border filling), and obtaining a second Laplacian pyramid Lb corresponding to the rotated target image. A third Laplacian pyramid Ls is generated based on the Gaussian pyramid Gm, the first Laplacian pyramid La, and the second Laplacian pyramid Lb, so as to determine the fused image using the third Laplacian pyramid Ls.

[0104] That is to say, after completing the face driving, it is necessary to paste the video after driving (i.e., the first facial image driving video) back to the target image to achieve the overall display effect. Among them, in order to eliminate the seams and color differences caused by pasting the video after driving back to the original image, the embodiment of the present application can obtain an initial mask image by performing semantic segmentation processing on the target facial driving image in the first facial image driving video. Furthermore, in order to reduce the sense of tearing between the original image and the driven image at the seam position, a feathering operation can be performed on the result of portrait segmentation (i.e., the initial mask image), that is, the initial target mask image is feathered, and the initial target mask image after feathering is determined as the target mask image corresponding to the target facial driving image. The target facial driving image and the rotated target image are then fused according to the target mask image.

[0105] Specifically, the embodiment of the present application can use the Unet-based portrait semantic segmentation model to perform semantic segmentation processing on the target facial driving image. The model can determine whether each pixel in the target facial driving image belongs to a person, so as to obtain a binary image (i.e., the initial mask image) of the same size as the original image (i.e., the target facial driving image). For example, see Figure 5 , Figure 5 : is a scene diagram of the semantic segmentation of portrait provided by the embodiment of this application. Figure 5 The middle (a) figure is the target facial driving image, such as Figure 5 The middle (b) figure is the initial mask image corresponding to the target facial driving image.

[0106] Optionally, after obtaining the target mask image, the target mask image can be further padded to its boundaries, so that the target mask image is the same size as the target image and the face position in the target mask image is at the corresponding position in the target image. Furthermore, the target facial driving image and the rotated target image are fused based on the target mask image to obtain a fused image corresponding to the target facial driving image. Specifically, the image fusion process can be understood as follows: First, Gaussian pyramids Ga and Gb are established for the target facial driving image (referred to as image a) and the rotated target image (referred to as image b), respectively, and Laplacian pyramids La and Lb are calculated. Second, a Gaussian pyramid Gm is established for the target mask image (referred to as image m). The Gaussian pyramid Gm is used to more smoothly fuse the two input images (i.e., image a and image b). Third, La and Lb are fused using Gm as a weight to form a new Laplacian pyramid Ls, where the new Laplacian pyramid Ls satisfies the following: Ls = Gm*La + (1-Gm)*Lb. The fourth step is to obtain the final fused image through the new Laplacian pyramid Ls (that is, at each scale, Gm is multiplied by Ga, Gb and then added to Ls, and then up-sampled and added to the previous scale Ls. This operation is repeated until the fused image of the original size (that is, the size of the target image) is obtained).

[0107] In some feasible implementations, each facial driving image frame included in the first facial image-driven video is used as a target facial driving image, and after performing the above-mentioned image fusion operation, a multi-frame fused image corresponding to the multiple facial driving images can be obtained. Furthermore, the target driving video can be generated based on the multi-frame fused images corresponding to the multiple facial driving images. Generating the target driving video based on the multi-frame fused images corresponding to the multiple facial driving images can be understood as rotating the multi-frame fused images so that the angle of the facial image in the multi-frame fused image is the same as the angle of the facial image in the target image uploaded by the user, and then processing the rotated multi-frame fused images according to the preset frame rate to obtain the target driving video.

[0108] For example, see Figure 6 , Figure 6 This is a scene diagram of the image processing process provided by the embodiment of this application. Figure 6 As shown in Figure 2, in order to ensure the quality of the face image, it is necessary to crop the target image (such as Figure 6 (a) shown in Figure ) and the initial drive video (as shown in Figure ) Figure 6 The face images in (b) are rotated and aligned to obtain the first face image in the rotated target image (as shown in FIG. Figure 6 (c) shown in Figure ) and driving video (as shown in Figure Figure 6Then, the first facial image is driven by the first-order motion model to simulate the motion of the driving video, and the driven video (i.e., the first facial image driven video involved in the embodiment of the present application) can be obtained. Figure 6 As shown in Figure (e), for each frame of facial driving image included in the first facial image driving video (for the convenience of description, the embodiment of the present application takes one frame of facial driving image (i.e., target facial driving image) as an example), the target facial driving image is sequentially subjected to semantic segmentation processing, feathering processing, and boundary filling processing, and a target mask image corresponding to the target facial driving image can be obtained (e.g., Figure 6 Finally, the target face driving image (as shown in (f)) after filling the boundary according to the target mask image is Figure 6 (g) shown in Figure ) and the rotated target image (as shown in Figure ) Figure 6 By fusing the target facial driving image (h), the fused image corresponding to the target facial driving image can be obtained. By rotating the fused image, the following Figure 6 The fused and rotated fused image shown in (i) is used to generate the target driving video.

[0109] In an embodiment of the present application, a target image is acquired and a driving video selected by a user is determined; when a facial image in the target image is detected to be tilted, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is captured to obtain a first facial image. The first facial image and the driving video are input into a first-order motion model, and the first facial image is driven by the first-order motion model to simulate the driving video movement to obtain a first facial image driving video. The first facial image driving video is pasted back to the position of the facial image in the target image to obtain a target driving video. By adopting the embodiment of the present application, the driving effect on the target image can be improved, and the applicability is high.

[0110] See Figure 7 , Figure 7 Schematic diagram of the structure of the image processing device provided in the embodiment of the present application. The image processing device provided in the embodiment of the present application includes:

[0111] The transceiver unit 71 is used to obtain the target image and determine the driving video;

[0112] a processing unit 72 configured to, when detecting that the facial image in the target image is tilted, rotate the target image to correct the angle of the facial image to a preset angle, and intercept the facial image in the rotated target image to obtain a first facial image;

[0113] The processing unit 72 is configured to input the first facial image and the driving video into a first-order motion model, so that the first-order motion model drives the first facial image to simulate the motion of the driving video, thereby obtaining a first facial image driving video;

[0114] The processing unit 72 is configured to paste the first facial image driven video back to the location of the facial image in the target image to obtain a target driven video.

[0115] In a possible implementation, the processing unit 72 is configured to:

[0116] Acquire an initial driving video, wherein the initial driving video includes multiple reference image frames, each of the multiple reference image frames includes a reference facial image;

[0117] When it is detected that the reference facial image included in the first reference image frame among the multiple reference image frames is in a tilted state, rotating each reference image frame among the multiple reference image frames so that the angle of the reference facial image is the preset angle;

[0118] The driving video is obtained based on the rotated multiple-frame reference image.

[0119] In a possible implementation, the processing unit 72 is configured to:

[0120] intercepting a reference facial image included in each frame of the rotated multiple reference image frames to obtain multiple reference facial images;

[0121] A driving video is generated according to a preset frame rate and the plurality of reference facial images.

[0122] In a possible implementation, the processing unit 72 is configured to:

[0123] Performing facial key point detection processing on the target image to obtain position information corresponding to a plurality of facial key points included in the target image;

[0124] It is determined whether the facial image in the target image is in a tilted state according to the position information of the multiple facial key points.

[0125] In a possible implementation, the plurality of facial key points include a left eye and a right eye, and the position information of each facial key point includes a horizontal coordinate value and a vertical coordinate value; the processing unit 72 is configured to:

[0126] Determine an angle between a line connecting the left eye and the right eye and a horizontal direction according to an absolute value of a difference between the abscissa value of the left eye and the abscissa value of the right eye, and an absolute value of a difference between the ordinate value of the left eye and the ordinate value of the right eye;

[0127] If the angle is greater than or equal to a first preset threshold, it is determined that the facial image in the target image is in a tilted state.

[0128] In a possible implementation, the first facial image driven video includes multiple frames of facial driven images;

[0129] The processing unit 72 is configured to:

[0130] Acquire a target mask image corresponding to a target facial drive image in the first facial image driven video, where the target facial drive image is any one of the multiple facial drive images;

[0131] fusing the target facial drive image and the rotated target image according to the target mask image to obtain a fused image corresponding to the target facial drive image;

[0132] A target driving video is generated according to the multiple frames of fused images corresponding to the multiple frames of facial driving images.

[0133] In a possible implementation, the processing unit 72 is configured to:

[0134] performing semantic segmentation processing on the target facial driving image in the first facial image driving video to obtain an initial mask image;

[0135] The initial target mask image is subjected to a feathering process, and the initial target mask image after the feathering process is determined as the target mask image corresponding to the target facial driving image.

[0136] In a possible implementation, the processing unit 72 is configured to:

[0137] Obtain the Gaussian pyramid Gm corresponding to the target mask image;

[0138] Obtain a first Laplacian pyramid La corresponding to the target facial drive image and a second Laplacian pyramid Lb corresponding to the target image;

[0139] A third Laplacian pyramid Ls is generated according to the Gaussian pyramid Gm, the first Laplacian pyramid La, and the second Laplacian pyramid Lb, so as to determine the fused image through the third Laplacian pyramid Ls.

[0140] In an embodiment of the present application, an image processing device can obtain a target image and determine a driving video selected by a user; when a facial image in the target image is detected to be tilted, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is captured to obtain a first facial image. The first facial image and the driving video are input into a first-order motion model, and the first facial image is driven by the first-order motion model to simulate the driving video movement to obtain a first facial image driving video. The first facial image driving video is pasted back to the position of the facial image in the target image to obtain a target driving video. By adopting the embodiment of the present application, the driving effect on the target image can be improved, and the applicability is high.

[0141] See Figure 8 , Figure 8 This is a schematic diagram of the structure of the network device provided in the embodiment of the present application. Figure 8 As shown, the network device in this embodiment may include: one or more processors 801, memory 802, and transceiver 803. The processor 801, memory 802, and transceiver 803 are connected via a bus 804. The memory 802 is used to store a computer program, which includes program instructions. The processor 801 and transceiver 803 are used to execute the program instructions stored in the memory 802 to perform the following operations:

[0142] Transceiver 803, used to obtain the target image and determine the driving video;

[0143] Processor 801 is configured to, when detecting that the facial image in the target image is tilted, rotate the target image to correct the angle of the facial image to a preset angle, and intercept the facial image in the rotated target image to obtain a first facial image;

[0144] Processor 801 is configured to input the first facial image and the driving video into a first-order motion model, so that the first-order motion model drives the first facial image to simulate the motion of the driving video, thereby obtaining a first facial image driving video;

[0145] The processor 801 is configured to paste the first facial image driven video back to the location of the facial image in the target image to obtain a target driven video.

[0146] In a possible implementation, the processor 801 is configured to:

[0147] Acquire an initial driving video, wherein the initial driving video includes multiple reference image frames, each of the multiple reference image frames includes a reference facial image;

[0148] When it is detected that the reference facial image included in the first reference image frame among the multiple reference image frames is in a tilted state, rotating each reference image frame among the multiple reference image frames so that the angle of the reference facial image is the preset angle;

[0149] The driving video is obtained based on the rotated multiple-frame reference image.

[0150] In a possible implementation, the processor 801 is configured to:

[0151] intercepting a reference facial image included in each frame of the rotated multiple reference image frames to obtain multiple reference facial images;

[0152] A driving video is generated according to a preset frame rate and the plurality of reference facial images.

[0153] In a possible implementation, the processor 801 is configured to:

[0154] Performing facial key point detection processing on the target image to obtain position information corresponding to a plurality of facial key points included in the target image;

[0155] It is determined whether the facial image in the target image is in a tilted state according to the position information of the multiple facial key points.

[0156] In a possible implementation, the plurality of facial key points include a left eye and a right eye, and the position information of each facial key point includes a horizontal coordinate value and a vertical coordinate value; the processor 801 is configured to:

[0157] Determine an angle between a line connecting the left eye and the right eye and a horizontal direction according to an absolute value of a difference between the abscissa value of the left eye and the abscissa value of the right eye, and an absolute value of a difference between the ordinate value of the left eye and the ordinate value of the right eye;

[0158] If the angle is greater than or equal to a first preset threshold, it is determined that the facial image in the target image is in a tilted state.

[0159] In a possible implementation, the first facial image driven video includes multiple frames of facial driven images;

[0160] The processor 801 is configured to:

[0161] Acquire a target mask image corresponding to a target facial drive image in the first facial image driven video, where the target facial drive image is any one of the multiple facial drive images;

[0162] fusing the target facial drive image and the rotated target image according to the target mask image to obtain a fused image corresponding to the target facial drive image;

[0163] A target driving video is generated according to the multiple frames of fused images corresponding to the multiple frames of facial driving images.

[0164] In a possible implementation, the processor 801 is configured to:

[0165] performing semantic segmentation processing on the target facial driving image in the first facial image driving video to obtain an initial mask image;

[0166] The initial target mask image is subjected to a feathering process, and the initial target mask image after the feathering process is determined as the target mask image corresponding to the target facial driving image.

[0167] In a possible implementation, the processor 801 is configured to:

[0168] Obtain the Gaussian pyramid Gm corresponding to the target mask image;

[0169] Obtain a first Laplacian pyramid La corresponding to the target facial drive image and a second Laplacian pyramid Lb corresponding to the target image;

[0170] A third Laplacian pyramid Ls is generated according to the Gaussian pyramid Gm, the first Laplacian pyramid La, and the second Laplacian pyramid Lb, so as to determine the fused image through the third Laplacian pyramid Ls.

[0171] It should be understood that in some feasible implementations, the processor 801 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. The memory 802 may include a read-only memory and a random access memory, and provide instructions and data to the processor 801. A portion of the memory 802 may also include a non-volatile random access memory. For example, the memory 802 may also store device type information.

[0172] In a specific implementation, the above network equipment can execute the above-mentioned functions through its built-in functional modules. Figures 2 to 6 For the implementation methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0173] In an embodiment of the present application, a network device can obtain a target image and determine a driving video selected by a user; when a facial image in the target image is detected to be tilted, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is captured to obtain a first facial image. The first facial image and the driving video are input into a first-order motion model, and the first facial image is driven by the first-order motion model to simulate the driving video movement to obtain a first facial image driving video. The first facial image driving video is pasted back to the position of the facial image in the target image to obtain a target driving video. By adopting the embodiment of the present application, the driving effect on the target image can be improved, and the applicability is high.

[0174] The present invention also provides a computer-readable storage medium that stores a computer program. The computer program includes program instructions that are executed by a processor to implement Figures 2 to 6 For the image processing methods provided in each step, please refer to the implementation methods provided in the above steps, which will not be repeated here.

[0175] The above-mentioned computer-readable storage medium can be the internal storage unit of the image processing device provided in any of the aforementioned embodiments or the above-mentioned terminal device, such as the hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (smartmedia card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium can also include both the internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0176] The terms "first," "second," "third," "fourth," and the like in the claims, specification, and drawings of this application are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0177] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the present application. The appearance of such a phrase in various locations in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, that the embodiments described herein may be combined with other embodiments. The term "and / or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, including such combinations. Those skilled in the art will appreciate that the elements and algorithmic steps of the various examples described in connection with the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the above description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0178] The methods and related devices provided by the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided by the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 The flow or flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire target images and determine driving videos; When it is detected that the facial image in the target image is in a tilted state, the target image is rotated to correct the angle of the facial image to a preset angle, and the facial image in the rotated target image is intercepted to obtain a first facial image; Inputting the first facial image and the driving video into a first-order motion model, so that the first-order motion model drives the first facial image to simulate the motion of the driving video, thereby obtaining a first facial image driving video; Paste the first facial image driven video back to the target image at the location of the facial image to obtain a target driven video.

2. The method according to claim 1, characterized in that The determining of the driving video includes: Acquire an initial driving video, wherein the initial driving video includes multiple reference image frames, each of the multiple reference image frames includes a reference facial image; When it is detected that the reference facial image included in the first reference image frame among the multiple reference image frames is in a tilted state, rotating each reference image frame among the multiple reference image frames so that the angle of the reference facial image is the preset angle; The driving video is obtained based on the rotated multiple-frame reference image.

3. The method according to claim 2, characterized in that The step of obtaining the driving video based on the rotated multi-frame reference image comprises: intercepting a reference facial image included in each frame of the rotated multiple reference image frames to obtain multiple reference facial images; A driving video is generated according to a preset frame rate and the plurality of reference facial images.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Performing facial key point detection processing on the target image to obtain position information corresponding to a plurality of facial key points included in the target image; It is determined whether the facial image in the target image is in a tilted state according to the position information of the multiple facial key points.

5. The method according to claim 4, characterized in that The plurality of facial key points include a left eye and a right eye, and the position information of each facial key point includes a horizontal coordinate value and a vertical coordinate value; and determining whether the facial image in the target image is in a tilted state according to the position information of the plurality of facial key points includes: Determine an angle between a line connecting the left eye and the right eye and a horizontal direction according to an absolute value of a difference between the abscissa value of the left eye and the abscissa value of the right eye, and an absolute value of a difference between the ordinate value of the left eye and the ordinate value of the right eye; If the angle is greater than or equal to a first preset threshold, it is determined that the facial image in the target image is in a tilted state.

6. The method according to any one of claims 1 to 3, characterized in that The first facial image driven video includes multiple frames of facial driven images; Pasting the first facial image driven video back to the target image at the location of the facial image to obtain the target driven video includes: Acquire a target mask image corresponding to a target facial drive image in the first facial image driven video, where the target facial drive image is any one of the multiple facial drive images; fusing the target facial drive image and the rotated target image according to the target mask image to obtain a fused image corresponding to the target facial drive image; A target driving video is generated according to the multiple frames of fused images corresponding to the multiple frames of facial driving images.

7. The method according to claim 6, characterized in that The obtaining of a target mask image corresponding to a target facial driving image in the first facial image driving video includes: performing semantic segmentation processing on the target facial driving image in the first facial image driving video to obtain an initial mask image; The initial mask image is subjected to a feathering process, and the initial mask image after the feathering process is determined as a target mask image corresponding to the target facial driving image.

8. The method according to claim 6, characterized in that The step of fusing the target facial drive image and the rotated target image according to the target mask image to obtain a fused image corresponding to the target facial drive image includes: Obtain the Gaussian pyramid Gm corresponding to the target mask image; Obtaining a first Laplacian pyramid La corresponding to the target facial drive image and a second Laplacian pyramid Lb corresponding to the rotated target image; A third Laplacian pyramid Ls is generated according to the Gaussian pyramid Gm, the first Laplacian pyramid La, and the second Laplacian pyramid Lb, so as to determine the fused image through the third Laplacian pyramid Ls.

9. A server, characterized in that: comprising a processor, a memory and a transceiver, wherein the processor, the memory and the transceiver are connected to each other; The memory is used to store a computer program, the computer program includes program instructions, and the processor and the transceiver are configured to call the program instructions to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 8.

Images