Human body posture transfer method, electronic device, and computer storage medium

Abstract

An embodiment of the present application provides a method for human body posture migration, an electronic device, and a computer storage medium, the method comprising: acquiring multiple pictures, different pictures among the multiple pictures presenting different angles of a target human body; performing segmentation processing on each of the multiple pictures to obtain multiple human body segmentation maps; generating a second target picture based on the multiple pictures, the multiple human body segmentation maps, and a first target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture.

Description

Technical Field

[0001] The embodiments of the present application relate to data transmission technology, and specifically to a human body posture migration method, an electronic device, and a computer storage medium. Background Art

[0002] Human posture transfer refers to transferring a specific posture to a certain human body, so as to achieve the effect of presenting the human body in different postures.

[0003] At present, the input of human body posture transfer technology is a single human body picture. If the human body picture is occluded, or the back of the human body appears in the new posture to be transferred (while the input human body picture shows a posture other than the back of the human body), the effect of human body posture transfer is poor due to the lack of more human body information. Summary of the invention

[0004] The embodiments of the present application provide a human body posture migration method, an electronic device, and a computer storage medium, which can solve the problem of poor effect of human body posture migration.

[0005] The human body posture transfer method provided in the embodiment of the present application includes:

[0006] Acquire multiple pictures, where different pictures in the multiple pictures present different angles of the target human body;

[0007] Performing segmentation processing on each of the multiple images to obtain multiple human body segmentation images;

[0008] A second target picture is generated based on the multiple pictures, the multiple human body segmentation maps, and the first target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture.

[0009] The human body posture migration method provided in the embodiment of the present application includes:

[0010] Generate a three-dimensional (3D) appearance flow image based on a first target image and an input image; wherein the first target image is used to determine a first posture; and the input image has a target human body;

[0011] A second target image is generated based on the first target image, the input image, and the 3D appearance flow image, wherein the second target image has the target human body presented in the first posture.

[0012] The electronic device provided in the embodiment of the present application includes:

[0013] An acquisition unit, used to acquire a plurality of pictures, wherein different pictures in the plurality of pictures present different angles of the target human body;

[0014] A processing unit is used to perform segmentation processing on each of the multiple pictures to obtain multiple human body segmentation maps; based on the multiple pictures, the multiple human body segmentation maps and the first target picture, generate a second target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture.

[0015] The electronic device provided in the embodiment of the present application includes:

[0016] A first generating unit is configured to generate a three-dimensional 3D appearance flow image based on a first target image and an input image; wherein the first target image is used to determine a first posture; and the input image has a target human body;

[0017] The second generating unit is configured to generate a second target image based on the first target image, the input image, and the 3D appearance flow image, wherein the second target image has the target human body presented in the first posture.

[0018] The electronic device provided in an embodiment of the present application includes: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor is used to execute the above-mentioned human body posture migration method when running the computer program.

[0019] The computer storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the above-mentioned human body posture migration method is implemented.

[0020] In the technical solution of the embodiment of the present application, the human body can be effectively distinguished from the background by segmenting the image, and effective human body information can be retained. Further, the migration of human body posture based on the human body segmentation map can improve the effect of human body posture migration, especially for human body images with occlusion, which improves the posture migration effect. On the other hand, the human body posture migration can be assisted by 3D appearance flow images, which can improve the effect of human body posture migration, especially for human body images with occlusion, which improves the posture migration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the process of the human body posture transfer method provided in the embodiment of the present application Figure 1 ;

[0022] Figure 2 Schematic diagram of the process of the human body posture transfer method provided in the embodiment of the present application Figure 2 ;

[0023] Figure 3 A schematic diagram of the human body posture migration provided in the embodiment of the present application;

[0024] Figure 4 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 1 ;

[0025] Figure 5 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 2 ;

[0026] Figure 6 Schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application Figure 1 ;

[0027] Figure 7 Schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0028] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0029] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0030] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0031] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0032] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0033] The embodiments of the present application can be applied to electronic devices, such as computer systems / servers, which can operate with many other general or special computing system environments or configurations. Examples of well-known computing systems, environments and / or configurations suitable for use with computer systems / servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0034] Computer systems / servers may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

[0035] Figure 1 Schematic diagram of the process of the human body posture transfer method provided in the embodiment of the present application Figure 1 ,like Figure 1 As shown, the human body posture migration method comprises the following steps:

[0036] Step 101: Acquire multiple pictures, where different pictures in the multiple pictures present different angles of the target human body; perform segmentation processing on each picture in the multiple pictures to obtain multiple human body segmentation maps.

[0037] In the embodiment of the present application, obtaining multiple pictures includes: obtaining N pictures, where N is a positive integer greater than or equal to 2. It should be noted that the technical solution of the embodiment of the present application does not limit the value of N. In an optional implementation, the value of N can be determined according to the processing capability of the electronic device. Generally, the higher the processing capability of the electronic device, the larger the value of N. Conversely, the lower the processing capability of the electronic device, the smaller the value of N.

[0038] In the embodiment of the present application, different pictures in the plurality of pictures present different angles of the target body. Taking N=4 as an example, the pictures obtained are: Picture 1 (presenting the front of the target body), Picture 2 (presenting the back of the target body), Picture 3 (presenting the left side of the target body), and Picture 4 (presenting the right side of the target body).

[0039] In the embodiment of the present application, after acquiring multiple pictures, it is necessary to perform segmentation processing on the multiple pictures to extract effective human body information.

[0040] In an optional implementation manner, each of the multiple pictures may be segmented in the following manner:

[0041] 1) extracting a target area from each of the multiple images, where the target area is an area where the target human body is present;

[0042] 2) Segmenting the target area of ​​each of the multiple images into multiple parts to obtain the multiple human body segmentation images.

[0043] In one example, the BodyPix2.0 model can be used to process each of the multiple images to distinguish the target human body in the image from the environment, and then the target area where the target human body is located is divided into multiple parts to form a human body segmentation map. Here, the target area can be divided into multiple parts, which can be 24 parts. It should be noted that the number of target areas segmented in the embodiment of the present application is not limited to the above 24, and can also be other numbers, such as 18, 36, etc.

[0044] For example, the target area can be segmented into: head, neck, torso, arms, legs and feet, etc.

[0045] Step 102: Generate a second target picture based on the multiple pictures, the multiple human body segmentation maps and the first target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture.

[0046] In the embodiment of the present application, the first target image is used to determine a first posture, and the first posture is the posture that needs to be transferred to the target human body.

[0047] In an optional implementation of the present application, the first target image is a frame image in a video. For example, a short dance video includes M frames of images, where M is a positive integer, and each frame of the M frames of images presents a specific dance posture, wherein the first target image is a frame image in the M frames of images, and the posture presented by the frame image is the first posture.

[0048] In order to achieve the migration of the first posture in the first target image to the target human body, it is necessary to input the multiple images, the multiple human body segmentation maps and the first target image into a human body posture migration model (or a posture generation model), and process the multiple images, the multiple human body segmentation maps and the first target image through the human body posture migration model to obtain a second target image, wherein the second target image has the target human body presented in the first posture, thereby achieving the migration of the first posture in the first target image to the target human body.

[0049] Furthermore, each frame of the video can be used as the above-mentioned first target image, so that the posture of each frame of the video can be transferred to the target human body, generating a series of second target images of the target human body in various postures, and finally composing a series of generated second target images into a video, which is the video after posture transfer.

[0050] The above-mentioned technical scheme of the embodiment of the present application can effectively distinguish the human body from the background by segmenting the image, retaining effective human body information, and further realizing the migration of human body posture based on the human body segmentation map to improve the effect of human body posture migration, especially for human body images with occlusion, which improves the posture migration effect.

[0051] Figure 2 Schematic diagram of the process of the human body posture transfer method provided in the embodiment of the present application Figure 2 ,like Figure 2 As shown, the human body posture migration method comprises the following steps:

[0052] Step 201: Generate a 3D appearance flow picture based on a first target picture and an input picture; wherein the first target picture is used to determine a first posture; and the input picture has a target human body.

[0053] In the embodiment of the present application, the 3D appearance flow image is generated based on the 3D appearance flow model, and the training of the 3D appearance flow model can be achieved in the following ways:

[0054] 1) Acquire multiple pictures, where different pictures in the multiple pictures present different angles of the target human body.

[0055] Specifically, N pictures are obtained, where N is a positive integer greater than or equal to 2. It should be noted that the technical solution of the embodiment of the present application does not limit the value of N. In an optional implementation, the value of N can be determined according to the processing capability of the electronic device. Generally, the higher the processing capability of the electronic device, the larger the value of N. Conversely, the lower the processing capability of the electronic device, the smaller the value of N.

[0056] In the embodiment of the present application, different pictures in the plurality of pictures present different angles of the target body. Taking N=4 as an example, the pictures obtained are: Picture 1 (presenting the front of the target body), Picture 2 (presenting the back of the target body), Picture 3 (presenting the left side of the target body), and Picture 4 (presenting the right side of the target body).

[0057] 2) Generate 3D human body models corresponding to the multiple images respectively.

[0058] In one example, a DenseBody model can be used to generate a corresponding human 3D model from a picture. For multiple pictures, a large number of human 3D model data sets can be obtained.

[0059] 3) Calculating a 3D appearance flow image based on every two images in the plurality of images and the corresponding 3D human body model.

[0060] Here, feature point matching and visibility analysis methods are used to obtain 3D appearance flow images, wherein the role of visibility analysis is to know which part of the new posture (the posture corresponding to one of the two pictures) and the old posture (the posture corresponding to the other of the two pictures) is visible and which part is invisible.

[0061] 4) The two posture images and 3D appearance flow images corresponding to the two images are used to train the 3D appearance flow model.

[0062] Here, the image can be processed by using a posture estimation model to obtain a posture graph of the image. In one example, the posture graph is represented by 18 human body key points, and the 18 human body key points are connected based on the human body structure to form a posture graph. It should be noted that the number of human body key points is not limited to the above 18, and other numbers such as 24, 32, etc. can also be used.

[0063] In the embodiment of the present application, after the 3D appearance flow model is trained in the above manner, the first target image and the input image are processed by the 3D appearance flow model to obtain a 3D appearance flow image.

[0064] Here, the first target image is a frame image in the video. For example, a short dance video includes M frames of images, where M is a positive integer, and each frame of the M frames of images presents a specific dance posture, wherein the first target image is a frame image in the M frames of images, and the posture presented by the frame image is the first posture.

[0065] In an optional implementation manner of the present application, generating a three-dimensional 3D appearance flow picture based on the first target picture and the input picture includes:

[0066] Based on the first target image, generating a first posture graph;

[0067] Based on the input graph, generating a second posture graph;

[0068] A 3D appearance flow picture is generated based on the first posture graph and the second posture graph.

[0069] Here, the image can be processed by using a posture estimation model to obtain a posture graph of the image. In one example, the posture graph is represented by 18 human body key points, and the 18 human body key points are connected based on the human body structure to form a posture graph. It should be noted that the number of human body key points is not limited to the above 18, and other numbers such as 24, 32, etc. can also be used.

[0070] Step 202: Generate a second target image based on the first target image, the input image, and the 3D appearance flow image, wherein the second target image has the target human body presented in the first posture.

[0071] In an embodiment of the present application, in order to achieve the migration of the first posture in the first target image to the target human body, it is necessary to input the first target image, the input image and the 3D appearance flow image into a human body posture migration model (or a posture generation model), and process the first target image, the input image and the 3D appearance flow image through the human body posture migration model to obtain a second target image, wherein the second target image has the target human body presented in the first posture, thereby achieving the migration of the first posture in the first target image to the target human body.

[0072] In an optional implementation of the present application, a second target image is generated based on the first posture graph, the input graph, and the 3D appearance flow image. In a specific implementation, the first posture graph, the input graph, and the 3D appearance flow image are used to generate a second target image and input it into a human posture transfer model (or a posture generation model). The first posture graph, the input graph, and the 3D appearance flow image are processed by the human posture transfer model to obtain a second target image, and the second target image has the target human body presented in the first posture.

[0073] For example: Refer to Figure 3 , by processing the input image through the posture estimation model, the posture graph of the input image can be obtained; by processing a frame in the video through the posture estimation model, the posture graph of a frame in the video can be obtained. 1) By processing the posture graph of the input image and the posture graph of a frame in the video through the 3D appearance flow model, a 3D appearance flow image can be obtained. 2) By processing the input image, the posture graph of a frame in the video, and the 3D appearance flow image through the posture transfer model (or posture generation model), the migrated posture graph (i.e., a picture of the target human body presenting the posture of a frame in the video) can be obtained.

[0074] The above technical solution of the embodiment of the present application can improve the effect of human body posture migration by assisting human body posture migration through 3D appearance flow images, especially improving the posture migration effect when the human body image has occlusion.

[0075] Figure 4 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 1 ,like Figure 4 As shown, the electronic device includes:

[0076] An acquisition unit 401 is used to acquire a plurality of pictures, wherein different pictures in the plurality of pictures present different angles of the target human body;

[0077] The processing unit 402 is used to perform segmentation processing on each of the multiple pictures to obtain multiple human body segmentation maps; based on the multiple pictures, the multiple human body segmentation maps and the first target picture, generate a second target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture.

[0078] In an optional embodiment, the processing unit 402 is used to extract a target area from each of the multiple pictures, where the target area is the area where the target human body is presented; and divide the target area of ​​each of the multiple pictures into multiple parts to obtain the multiple human body segmentation maps.

[0079] In an optional implementation manner, the first target image is a frame of image in a video.

[0080] In the embodiment of the present application, the functions implemented by each unit in the electronic device can be understood with reference to the relevant description of the aforementioned human posture migration method. In specific implementation, the calculation unit, adjustment unit, and control unit in the electronic device can be implemented by a processor in the electronic device, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), or a programmable gate array (FPGA); the communication unit in the electronic device can be implemented by a communication module (including: basic communication kit, operating system, communication module, standardized interface and protocol, etc.) and a transceiver antenna, and the detection unit in the electronic device can be implemented by a temperature sensor.

[0081] Figure 5 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 2 ,like Figure 5 As shown, the electronic device includes:

[0082] A first generating unit 501 is configured to generate a three-dimensional 3D appearance flow image based on a first target image and an input image; wherein the first target image is used to determine a first posture; and the input image has a target human body;

[0083] The second generating unit 502 is configured to generate a second target image based on the first target image, the input image, and the 3D appearance flow image, wherein the second target image has the target human body presented in the first posture.

[0084] In an optional implementation, the first generating unit 501 is configured to generate a first posture graph based on a first target image; generate a second posture graph based on an input image; and generate a 3D appearance flow image based on the first posture graph and the second posture graph.

[0085] In an optional implementation, the second generating unit 502 is configured to generate a second target image based on the first posture image, the input image, and the 3D appearance flow image.

[0086] In an optional implementation manner, the first target image is a frame of image in a video.

[0087] In the embodiment of the present application, the functions implemented by each unit in the electronic device can be understood by referring to the relevant description of the aforementioned human posture migration method. In specific implementation, the calculation unit, adjustment unit, and control unit in the electronic device can be implemented by a processor in the electronic device, such as a CPU, DSP, MCU, or FPGA; the communication unit in the electronic device can be implemented by a communication module (including: basic communication kit, operating system, communication module, standardized interface and protocol, etc.) and a transceiver antenna, and the detection unit in the electronic device can be implemented by a temperature sensor.

[0088] It should be noted that the division of the above-mentioned units is only exemplary. In actual applications, the internal structure of the electronic device can be divided into different units to complete all or part of the functions described above. In addition, the electronic device and the human body posture transfer method embodiment provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0089] Based on the hardware implementation of the above device, the embodiment of the present application also provides an electronic device, Figure 6 Schematic diagram of the hardware structure of the electronic device of the embodiment of the present application. Figure 6 As shown, the electronic device includes a memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor; as a first implementation, when the processor 602 of the electronic device executes the program, the following steps are implemented:

[0090] Acquire multiple pictures, where different pictures in the multiple pictures present different angles of the target human body;

[0091] Performing segmentation processing on each of the multiple images to obtain multiple human body segmentation images;

[0092] A second target picture is generated based on the multiple pictures, the multiple human body segmentation maps, and the first target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture.

[0093] In an optional implementation manner, when the processor 602 executes the program, the processor 602 further implements the following steps:

[0094] Extracting a target area from each of the multiple images, where the target area is an area where a target human body is present;

[0095] The target area of ​​each of the multiple images is segmented into multiple parts to obtain the multiple human body segmentation images.

[0096] In an optional implementation manner, the first target image is a frame of image in a video.

[0097] It can be understood that the electronic device also includes a bus system 603; the various components in the electronic device are coupled together through the bus system 603. It can be understood that the bus system 603 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 603 also includes a power bus, a control bus and a status signal bus.

[0098] Based on the hardware implementation of the above device, the embodiment of the present application also provides an electronic device, Figure 7 Schematic diagram of the hardware structure of the electronic device of the embodiment of the present application. Figure 7 As shown, the electronic device includes a memory 701, a processor 702, and a computer program stored in the memory 701 and executable on the processor; as a first implementation, when the processor 702 of the electronic device executes the program, the following steps are implemented:

[0099] Generate a three-dimensional 3D appearance flow picture based on a first target picture and an input picture; wherein the first target picture is used to determine a first posture; and the input picture has a target human body;

[0100] A second target image is generated based on the first target image, the input image, and the 3D appearance flow image, wherein the second target image has the target human body presented in the first posture.

[0101] In an optional implementation manner, when the processor 702 executes the program, the processor 702 further implements the following steps:

[0102] Based on the first target image, generating a first posture graph;

[0103] Based on the input graph, generating a second posture graph;

[0104] A 3D appearance flow picture is generated based on the first posture graph and the second posture graph.

[0105] In an optional implementation manner, when the processor 702 executes the program, the processor 702 further implements the following steps:

[0106] A second target picture is generated based on the first posture picture, the input picture, and the 3D appearance flow picture.

[0107] In an optional implementation manner, the first target image is a frame of image in a video.

[0108] It can be understood that the electronic device also includes a bus system 703; the various components in the electronic device are coupled together through the bus system 703. It can be understood that the bus system 703 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 703 also includes a power bus, a control bus and a status signal bus.

[0109] It can be understood that the memory in this embodiment can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0110] The method disclosed in the above embodiment of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0111] The present application also provides a computer storage medium, specifically a computer-readable storage medium, on which computer instructions are stored. As a first implementation, when the computer storage medium is located in an electronic device, the computer instructions are executed by a processor to implement the following steps:

[0112] Acquire multiple pictures, where different pictures in the multiple pictures present different angles of the target human body;

[0113] Performing segmentation processing on each of the multiple images to obtain multiple human body segmentation images;

[0114] A second target picture is generated based on the multiple pictures, the multiple human body segmentation maps, and the first target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture.

[0115] In an optional implementation, when the computer instructions are executed by a processor, the following steps are further implemented:

[0116] Extracting a target area from each of the multiple images, where the target area is an area where a target human body is present;

[0117] The target area of ​​each of the multiple images is segmented into multiple parts to obtain the multiple human body segmentation images.

[0118] In an optional implementation manner, the first target image is a frame of image in a video.

[0119] The present application also provides a computer storage medium, specifically a computer-readable storage medium, on which computer instructions are stored. As a first implementation, when the computer storage medium is located in an electronic device, the computer instructions are executed by a processor to implement the following steps:

[0120] Generate a three-dimensional 3D appearance flow picture based on a first target picture and an input picture; wherein the first target picture is used to determine a first posture; and the input picture has a target human body;

[0121] A second target image is generated based on the first target image, the input image, and the 3D appearance flow image, wherein the second target image has the target human body presented in the first posture.

[0122] In an optional implementation, when the computer instructions are executed by a processor, the following steps are further implemented:

[0123] Based on the first target image, generating a first posture graph;

[0124] Based on the input graph, generating a second posture graph;

[0125] A 3D appearance flow picture is generated based on the first posture graph and the second posture graph.

[0126] In an optional implementation, when the computer instructions are executed by a processor, the following steps are further implemented:

[0127] A second target picture is generated based on the first posture picture, the input picture, and the 3D appearance flow picture.

[0128] In an optional implementation manner, the first target image is a frame of image in a video.

[0129] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0130] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0131] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or at least two units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0132] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0133] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0134] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A human body posture migration method, characterized in that: The method comprises: Acquire multiple pictures, where different pictures in the multiple pictures present different angles of the target human body; Performing segmentation processing on each of the multiple images to obtain multiple human body segmentation images; Generate a second target picture based on the multiple pictures, the multiple human body segmentation maps, and the first target picture, wherein the first target picture is used to determine a first posture, and the second target picture has the target human body presented in the first posture; The method comprises: Generate a three-dimensional 3D appearance flow picture based on a first target picture and an input picture; wherein the first target picture is used to determine a first posture; and the input picture has a target human body; Based on the first target image, the input image and the 3D appearance flow image, a second target image is generated, wherein the second target image has the target human body presented in the first posture; wherein The step of generating a three-dimensional 3D appearance flow picture based on the first target picture and the input picture includes: Based on the first target image, a first posture graph is generated; based on the input image, a second posture graph is generated; the first posture graph and the second posture graph are input into a 3D appearance flow model to generate a 3D appearance flow image; The generating a second target picture based on the first target picture, the input picture and the 3D appearance flow picture comprises: The first target image, the input image and the 3D appearance flow image are input into a human posture transfer model, and the first target image, the input image and the 3D appearance flow image are processed by the human posture transfer model to obtain a second target image.

2. The method according to claim 1, characterized in that: The step of performing segmentation processing on each of the plurality of images to obtain a plurality of human body segmentation images includes: Extracting a target area from each of the multiple images, where the target area is an area where a target human body is present; The target area of ​​each of the multiple images is segmented into multiple parts to obtain the multiple human body segmentation images.

3. The method according to claim 1 or 2, characterized in that: The first target image is a frame of image in a video.

4. The method according to claim 1, characterized in that The generating a second target picture based on the first target picture, the input picture and the 3D appearance flow picture comprises: A second target picture is generated based on the first posture picture, the input picture, and the 3D appearance flow picture.

5. The method according to claim 1 or 4, characterized in that: The first target image is a frame of image in a video.

6. An electronic device, characterized in that: The electronic device comprises: An acquisition unit, used to acquire a plurality of pictures, wherein different pictures in the plurality of pictures present different angles of the target human body; a processing unit, configured to segment each of the plurality of images to obtain a plurality of human body segmentation maps; and generate a second target image based on the plurality of images, the plurality of human body segmentation maps, and a first target image, wherein the first target image is used to determine a first posture, and the second target image has the target human body presented in the first posture; A first generating unit is configured to generate a three-dimensional 3D appearance flow image based on a first target image and an input image; wherein the first target image is used to determine a first posture; and the input image has a target human body; A second generating unit, configured to generate a second target image based on the first target image, the input image, and the 3D appearance flow image, wherein the second target image has the target human body presented in the first posture; The first generating unit is further configured to generate a first posture graph based on the first target image; generate a second posture graph based on the input image; input the first posture graph and the second posture graph into a 3D appearance flow model to generate a 3D appearance flow image; The second generating unit is further used to input the first target image, the input image and the 3D appearance flow image into a human posture transfer model, and process the first target image, the input image and the 3D appearance flow image through the human posture transfer model to obtain a second target image.

7. A computer storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: The electronic device comprises: a processor and a memory for storing a computer program that can be run on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 5.

Images