Pose transfer method and device for 3D virtual human, equipment and storage medium

Abstract

The application discloses a pose migration method and device for a 3D virtual person, equipment and a storage medium. An rgb image of a target object is collected by a monocular camera; a target detection algorithm is used to detect the target object in the rgb image to obtain target frame information; a target image is cropped from the rgb image according to the target frame information, and first feature information of the target image is extracted; the target frame information and the first feature information are subjected to feature fusion processing to obtain target parameters; the target parameters include position parameters, deformation parameters and camera angle parameters; the target parameters are input into a pre-established body prior model to obtain 3D point cloud data returned by the body prior model and including a plurality of vertices, and a 3D pose of a virtual person is reconstructed according to the 3D point cloud data. The pose reconstruction method has high accuracy and is beneficial to improving the effect of simulation simulation. The application can be widely applied in the technical field of three-dimensional models.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional model, in particular to a pose transfer method and device for 3D virtual person, equipment and storage medium. BACKGROUND

[0002] Under the background of the metaverse, the interaction between virtual people in three-dimensional virtual space and real people is particularly important. And it has great research value to capture real person actions through a monocular camera and reconstruct 3D virtual people. Compared with traditional wearable motion capture devices, capturing animations through a single camera combined with deep learning graphics algorithms can not only reduce the user threshold, but also is a breakthrough in motion capture in the field of image algorithms.

[0003] However, there are many challenges to overcome and complete for a monocular camera to complete motion capture, such as occlusion, depth data, etc. The situation may involve motion coherence, resulting in low accuracy of the reconstructed pose and poor effect. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art.

[0005] To this end, an object of an embodiment of the present application is to provide a pose transfer method, device, equipment and storage medium for 3D virtual people.

[0006] In order to achieve the above technical purpose, the technical solution adopted by the embodiments of the present application comprises:

[0007] On the one hand, the embodiments of the present application provide a pose transfer method for 3D virtual people, the method comprising:

[0008] capturing an rgb image of a target object through a monocular camera;

[0009] detecting the target object in the rgb image through a target detection algorithm to obtain target box information;

[0010] cropping a target image from the rgb image according to the target box information, and extracting first feature information of the target image;

[0011] performing feature fusion processing on the target box information and the first feature information to obtain target parameters; the target parameters include position parameters, deformation parameters and camera angle parameters;

[0012] inputting the target parameters into a pre-established body prior model to obtain 3D point cloud data returned by the body prior model, the 3D point cloud data including a plurality of vertices, and reconstructing a 3D pose of a virtual person according to the 3D point cloud data.

[0013] In addition, the method for pose migration of a 3D virtual person according to the above-mentioned embodiment of the present application can further have the following additional technical features.

[0014] Further, in an embodiment of the present application, the target frame information includes first coordinate information of the top-left vertex and second coordinate information of the bottom-right vertex of the rectangular frame in which the target object is located.

[0015] Further, in an embodiment of the present application, after the step of obtaining the target frame information, the method further includes:

[0016] According to the first coordinate information and the second coordinate information, the center point coordinate of the rectangular frame and the width information of the rectangular frame are calculated;

[0017] According to the length and width of the rgb image, the focal length of the monocular camera is determined;

[0018] According to the center point coordinate and the focal length, the global rotation information of the target object relative to the monocular camera is extracted;

[0019] The global rotation information and the width information are taken as the pre-processed target frame information.

[0020] Further, in an embodiment of the present application, the target detection algorithm is a yolov3 algorithm.

[0021] Further, in an embodiment of the present application, the method further includes:

[0022] The hand region of the target object in the rgb image is detected by a target detection algorithm, and a hand image of the target object is cropped;

[0023] The hand image is processed by an end-to-end hand neural network model, and a hand parameter of the target object is extracted;

[0024] According to the hand parameter, the pose of the virtual person is migrated.

[0025] Further, in an embodiment of the present application, the hand neural network model includes an encoder and a decoder, and the processing of the hand image by the end-to-end hand neural network model to extract the hand parameter of the target object includes:

[0026] The hand image is input to the encoder for processing to obtain encoded features;

[0027] The encoded features are regressed by the decoder to obtain the hand parameter of the target object.

[0028] In another aspect, an embodiment of the present application provides a device for pose transfer of a 3D virtual person, the device comprising:

[0029] a collection module configured to collect an rgb image of a target object by using a monocular camera;

[0030] a detection module configured to detect the target object in the rgb image by using a target detection algorithm to obtain target bounding box information;

[0031] an extraction module configured to crop a target image from the rgb image according to the target bounding box information and extract first feature information of the target image;

[0032] a fusion module configured to perform feature fusion processing on the target bounding box information and the first feature information to obtain target parameters; the target parameters include position parameters, deformation parameters and camera angle parameters;

[0033] a reconstruction module configured to input the target parameters into a pre-established body prior model to obtain 3D point cloud data including a plurality of vertices returned by the body prior model, and reconstruct a 3D pose of a virtual person according to the 3D point cloud data.

[0034] Further, in an embodiment of the present application, the target bounding box information includes first coordinate information of a top-left vertex and second coordinate information of a bottom-right vertex of a rectangular bounding box in which the target object is located; the device further comprises:

[0035] a preprocessing module, which is specifically configured to:

[0036] calculate a center point coordinate of the rectangular bounding box and width information of the rectangular bounding box according to the first coordinate information and the second coordinate information;

[0037] determine a focal length of the monocular camera according to a length and a width of the rgb image;

[0038] extract global rotation information of the target object relative to the monocular camera according to the center point coordinate and the focal length;

[0039] use the global rotation information and the width information as preprocessed target bounding box information.

[0040] In another aspect, an embodiment of the present application provides a computer device, comprising:

[0041] at least one processor;

[0042] at least one memory configured to store at least one program;

[0043] The at least one program, when executed by the at least one processor, causes the at least one processor to implement the posture migration method for a 3D virtual person.

[0044] In another aspect, the embodiments of the present application also provide a computer readable storage medium, which stores a processor executable program, and the processor executable program, when executed by a processor, is used to implement the posture migration method for a 3D virtual person.

[0045] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be learned by the practice of the present application:

[0046] The posture migration method for a 3D virtual person disclosed by the embodiments of the present application comprises the following steps: acquiring an rgb image of a target object by a monocular camera; detecting a target object in the rgb image by a target detection algorithm to obtain target frame information; cropping a target image from the rgb image according to the target frame information and extracting first feature information of the target image; performing feature fusion processing on the target frame information and the first feature information to obtain target parameters; the target parameters comprise position parameters, deformation parameters and camera angle parameters; inputting the target parameters into a pre-established body prior model to obtain 3D point cloud data comprising a plurality of vertices returned by the body prior model, and reconstructing a 3D posture of a virtual person according to the 3D point cloud data. The method supports posture reconstruction of a virtual person by a single rgb photo, can be applied to character motion capture and migration on a 3D virtual person, and has high posture reconstruction accuracy, which is beneficial to improving the effect of simulation simulation. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing part of the embodiments of the technical solutions of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0048] Figure 1 The flowchart of the posture migration method for a 3D virtual person provided in the embodiments of the present application;

[0049] Figure 2 The data processing schematic diagram of the posture migration method for a 3D virtual person provided in the embodiments of the present application;

[0050] Figure 3 The structural schematic diagram of a computer device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0051] The present application will be further described with reference to the drawings and specific examples described below. The described embodiments should not be considered limiting the present application, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0052] In the following description, reference is made to "some embodiments", which describes a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0054] Artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other fields.

[0055] Under the background of metaverse, the interaction between virtual people in three-dimensional virtual space and real people in real scene is particularly important. And it has great research value to capture real person action through monocular camera and reconstruct 3D virtual person. Compared with traditional wearable motion capture devices, through single camera to capture animation and deep learning graphics algorithm, not only can reduce the use threshold of users, but also is a breakthrough of motion capture in image algorithm field.

[0056] However, there are many challenges to be overcome and completed for monocular camera to complete motion capture, such as occlusion, depth data, etc., which may involve motion coherence, resulting in low accuracy of posture reconstruction for migration.

[0057] Therefore, the embodiment of the present application provides a posture migration method for a 3D virtual person, which comprises the following steps:

[0058] Please refer to Figure 1 , Figure 1 is a flowchart of a posture migration method for a 3D virtual person provided by the embodiment of the present application, and the posture migration method for a 3D virtual person comprises but is not limited to: Figure 1

[0059] Step 110, acquiring an rgb image of a target object by a monocular camera;

[0060] Step 120, detecting a target object in the rgb image by a target detection algorithm to obtain target frame information;

[0061] Step 130, cropping a target image from the rgb image according to the target frame information and extracting first feature information of the target image;

[0062] Step 140, performing feature fusion processing on the target frame information and the first feature information to obtain target parameters; the target parameters comprise position parameters, deformation parameters and camera angle parameters;

[0063] Step 150, inputting the target parameters into a pre-established body prior model to obtain 3D point cloud data returned by the body prior model, the 3D point cloud data comprising a plurality of vertices, and reconstructing a 3D posture of a virtual person according to the 3D point cloud data.

[0064] ​In the embodiments of the present application, the rgb image of the target object is collected by a monocular camera, where the target object can be a person of any age and gender. After obtaining the rgb image of the target object, a top-down 3D pose estimation method can be used to extract the pose information of the human body. Specifically, in the embodiments of the present application, the position of the human body in the rgb image can be detected first, that is, the target object in the rgb image can be detected by a target detection algorithm to determine the region of interest and obtain a target box information. For example, in some embodiments, the target detection algorithm can use a classic yolov3 algorithm, and the basic structure of the model uses a Darknet-53 model. The network mainly consists of a series of 1*1 and 3*3 convolutional layers, and the final output is the target box information, which can be represented as bbox=[n_x, n_y, m_x, m_y], representing the coordinate information of the left top and right bottom vertices of the cropped human target box on the cropped rgb image, where (n_x, n_y) can represent the first coordinate information of the left top vertex, and (m_x, m_y) can represent the second coordinate information of the right bottom vertex.

[0065] In the embodiments of the present application, after obtaining the target box information, the image content containing the target object can be cropped from the rgb image, and the image is recorded as a target image. For the target image, its size can be first adjusted to a fixed size, for example, 224*224, of course, the present application is not limited to this. Then, the target image can be input into a convolutional encoder, which can be used to extract the feature information of the target image, recorded as first feature information. Then, the first feature information and the aforementioned target box information can be subjected to feature fusion processing, for example, the two can be spliced, and the dimensions of the two information need to be stretched to one dimension before splicing. After merging, the feature information is subjected to dimension reduction by two fully connected layers, and then subjected to three times of fully connected processing of three branches respectively, a series of parameters based on the body prior model can be obtained, including position parameter pose, deformation parameter shape and camera angle parameter cam, which are collectively referred to as target parameters in the embodiments of the present application.

[0066] It should be noted that the body prior model is a three-dimensional human body model with parameters, which provides a differentiable function, including input parameters Θ∈{θ, β}, and P={s, t x ,t y}, which can return a 3D point cloud data including 6890 vertices, where θ∈R 24×3 is the angle of the model skeleton, β∈R 10 is the deformation of the model, s is the scale parameter, t x and t yis the translation along the x and y axes relative to the camera root position. The initial state file of the body prior model can be downloaded through the relevant channel, which contains three key data: position pose ∈ R 24×6 , shape ∈ R 10 , and camera angle cam ∈ R 3 .

[0067] Based on the above description, it can be understood that the body prior model is a parameterized three-dimensional human body model, and after the target parameters are derived, the target parameters can be input as input values into the pre-established body prior model, which provides a differentiable function derivation as described above, and a 3D point cloud data V ∈ R 6890×3 of 6890 vertices can be returned. In the embodiment of the application, the 3D point cloud data V ∈ R 6890×3 and the face arrangement data of the three-dimensional human body model can be used to reconstruct the 3D pose of the rgb image of the target object based on the three-dimensional human body model, that is, the 3D pose of the virtual person is reconstructed.

[0068] In some embodiments, after the step of obtaining the target box information, the method further comprises:

[0069] According to the first coordinate information and the second coordinate information, the center point coordinate of the rectangular box and the width information of the rectangular box are calculated;

[0070] According to the length and width of the rgb image, the focal length of the monocular camera is determined;

[0071] According to the center point coordinate and the focal length, the global rotation information of the target object relative to the monocular camera is extracted;

[0072] The global rotation information and the width information are taken as the preprocessed target box information.

[0073] In the embodiment of the application, it should be noted that in the rgb image containing the target object in the continuous frame, the position information of the rectangular box in which the target object is located in the cropped rgb image should not be discarded, and this information can be used to estimate the global rotation value of the original camera coordinate system relative to the entire rgb image. Therefore, in the embodiment of the application, the target box information as the input value of the inference model can be simply preprocessed before use. Specifically, the bbox can be converted into three groups of parameters [cx, cy, b], where (cx, cy) represents the center point coordinate of the rectangular box, and b is the width of the rectangular box. Here, the center point coordinate and the width information of the rectangular box can be calculated by the first coordinate information and the second coordinate information, and the focal length can be approximated by the square root of the width and height of the cropped rgb image, and the calculation formula is as follows:

[0074]

[0075] In the formula, f represents the focal length of the monocular camera, w represents the width of the rgb image, and h represents the height of the rgb image.

[0076] By the center point coordinates and the focal length, the conversion from the coordinate distance to the angle information can be completed, so that the global rotation information of the person relative to the camera can be provided to the subsequent regressor for extraction.

[0077] Specifically, the global rotation information can include and The finally obtained preprocessed target frame information is

[0078] In some embodiments, the method further includes:

[0079] detecting the hand region of the target object in the rgb image by a target detection algorithm, and cropping to obtain the hand image of the target object;

[0080] processing the hand image by an end-to-end hand neural network model to extract the hand parameters of the target object;

[0081] According to the hand parameters, the posture of the virtual person is migrated.

[0082] In the embodiments of the application, the yolov3 algorithm can be used to crop the hand region of the target object, and then the hand image I H of the target object can be cropped. H For the hand image, an end-to-end hand neural network model M h is constructed, which specifically includes an encoder and a decoder. The encoder is used to output encoded features based on the input hand image, and the network structure of the encoder can be Resnet50. The decoder is used to regress hand parameters from the encoded features, and the decoder can be composed of a group of fully connected layers. The process can be represented by the following formula:

[0083] [φ h ,θ h ,β h ,c H ]=M H (I h )

[0084] In the formula, φ 3 ∈R h represents the global direction of the hand mesh, θ 15×3 ∈R h represents the posture parameter of the hand, and β 10 ∈Ra shape parameter of the hand, c h is a camera parameter.

[0085] In the embodiments of the present application, with reference to Figure 2 In actual implementation, a virtual person can be created according to the appearance and gender of the virtual person, for example, the skin mesh of the virtual person contains 18303 vertices and 18224 faces. The skeleton information of the aforementioned body prior model is used to bind the skeleton, and for the convenience of migration, the initial value (T-pose) of the skeleton of the virtual person can be exactly the same as that of the body prior model. By using a monocular camera to collect the action video of the whole body of a model (target object), animation frame pictures of various actions can be obtained, and the values of the body part θ∈R 24×3 , hand θ h ∈R 15 ×3 , θ∈R 24×3 are the rotation Euler angles of 24 bones based on the body prior model in the action posture of a single picture, θ h ∈R 15×3 are the rotation Euler angles of 15 joints of a single hand based on the hand prior model, and cam∈R 3 is the value (displacement of the target object relative to the camera), which corresponds to the displacement value of the hip skeleton in the body prior model. In this way, the posture of the target object can be migrated to the 3D virtual person by one-to-one correspondence.

[0086] The embodiments of the present application also provide a posture migration device for a 3D virtual person, which comprises:

[0087] A collection module is configured to collect an rgb image of a target object by using a monocular camera.

[0088] A detection module is configured to detect the target object in the rgb image by using a target detection algorithm to obtain target box information.

[0089] An extraction module is configured to crop a target image from the rgb image according to the target box information and extract first feature information of the target image.

[0090] A fusion module is configured to perform feature fusion processing on the target box information and the first feature information to obtain target parameters; the target parameters comprise a position parameter, a shape parameter and a camera angle parameter.

[0091] A reconstruction module is configured to input the target parameters into a pre-established body prior model to obtain 3D point cloud data comprising a plurality of vertices returned by the body prior model, and reconstruct a 3D posture of a virtual person according to the 3D point cloud data.

[0092] Furthermore, in one embodiment of this application, the target bounding box information includes the first coordinate information of the upper left vertex and the second coordinate information of the lower right vertex of the rectangle containing the target object; the device further includes:

[0093] The preprocessing module is specifically used for:

[0094] Based on the first coordinate information and the second coordinate information, calculate the coordinates of the center point of the rectangle and the width information of the rectangle;

[0095] The focal length of the monocular camera is determined based on the length and width of the RGB image.

[0096] Based on the center point coordinates and the focal length, extract the global rotation information of the target object relative to the monocular camera;

[0097] The global rotation information and the width information are used as the preprocessed target bounding box information.

[0098] Understandable, Figure 1 The content of the embodiment of the pose transfer method for 3D virtual humans shown is applicable to the embodiment of the pose transfer device for 3D virtual humans. The specific functions implemented by the embodiment of the pose transfer device for 3D virtual humans are the same as those shown in this embodiment. Figure 1 The embodiment shown is the same as that for pose transfer of a 3D virtual human, and the beneficial effects achieved are the same. Figure 1 The beneficial effects achieved by the embodiment of the pose transfer method for 3D virtual humans shown are also the same.

[0099] Reference Figure 3 This application also discloses a computer device, including:

[0100] At least one processor 201;

[0101] At least one memory 202 is used to store at least one program;

[0102] When at least one program is executed by at least one processor 201, such that at least one processor 201 performs as follows: Figure 1 An embodiment of a pose transfer method for 3D virtual humans is shown.

[0103] It is understandable that, such as Figure 1 The content of the embodiment of the pose transfer method for 3D virtual human shown is applicable to the embodiment of this computer device. The specific functions implemented by the embodiment of this computer device are the same as those shown below. Figure 1 The embodiment shown is the same as that for pose transfer of a 3D virtual human, and the beneficial effects achieved are the same as those described. Figure 1The beneficial effects achieved by the embodiment of the pose transfer method for 3D virtual humans shown are also the same.

[0104] This application also discloses a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement, for example... Figure 1 An embodiment of a pose transfer method for 3D virtual humans is shown.

[0105] It is understandable that, such as Figure 1 The content of the embodiment of the pose transfer method for 3D virtual human shown is applicable to the embodiment of this computer-readable storage medium. The specific functions implemented by the embodiment of this computer-readable storage medium are the same as those shown below. Figure 1 The embodiment shown is the same as that for pose transfer of a 3D virtual human, and the beneficial effects achieved are the same as those described above. Figure 1 The beneficial effects achieved by the embodiment of the pose transfer method for 3D virtual humans shown are also the same.

[0106] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0107] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical system and / or software module, or one or more functions and / or features may be implemented in a separate physical system or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the system disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0108] If the functions are implemented in software, the functions can be stored in or implemented as one or more instructions or code on a computer-readable medium. Computer-readable media include both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage medium can be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, or twisted pair, then the coaxial cable, fiber optic cable, or twisted pair are included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), and Blu-Ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0109] In other words, like a human driver of a vehicle, an autonomous vehicle can be programmed to follow traffic laws and to make decisions based on its environment. For example, an autonomous vehicle can be programmed to follow a speed limit, to stop at a stop sign, to yield to a pedestrian, to merge onto a highway, to change lanes, to park, and so on. In some embodiments, an autonomous vehicle can be programmed to follow traffic laws and to make decisions based on its environment using a machine learning algorithm. For example, an autonomous vehicle can be programmed to follow a speed limit, to stop at a stop sign, to yield to a pedestrian, to merge onto a highway, to change lanes, to park, and so on using a machine learning algorithm.

[0110] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical), and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0111] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination of them. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or their combinations can be used: a discrete logic circuit having logic gates for implementing logic functions on data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), and the like.

[0112] In the above description of the present specification, the description referring to the terms "one embodiment", "another embodiment" or "certain embodiments" or the like means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative expressions of the above terms do not necessarily refer to the same embodiment or example in the present specification. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner.

[0113] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, alternatives and variations can be made thereto without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.

[0114] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present application

[0115] In the above description of the present specification, the description referring to the terms "one embodiment", "another embodiment" or "certain embodiments" or the like means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative expressions of the above terms do not necessarily refer to the same embodiment or example in the present specification. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner.

[0116] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, alternatives and variations can be made thereto without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A pose transfer method for a 3D virtual human, characterized in that, The method comprises: acquiring an rgb image of a target object by a monocular camera; detecting the target object in the rgb image by a target detection algorithm to obtain target frame information; the target frame information comprises first coordinate information of a top-left vertex and second coordinate information of a bottom-right vertex of a rectangular frame in which the target object is located; cropping a target image from the rgb image according to the target frame information and extracting first feature information of the target image; performing feature fusion processing on the target frame information and the first feature information to obtain target parameters; the target parameters comprise a position parameter, a deformation parameter and a camera angle parameter; inputting the target parameters into a pre-established body prior model to obtain 3D point cloud data returned by the body prior model, the 3D point cloud data comprising a plurality of vertices, and reconstructing a 3D pose of a virtual person according to the 3D point cloud data; after the step of obtaining the target frame information, the method further comprises: calculating a center point coordinate of the rectangular frame and width information of the rectangular frame according to the first coordinate information and the second coordinate information; determining a focal length of the monocular camera according to a length and a width of the rgb image; extracting global rotation information of the target object relative to the monocular camera according to the center point coordinate and the focal length; taking the global rotation information and the width information as preprocessed target frame information. 2.The method for pose transfer of 3D virtual human according to claim 1, wherein, The target detection algorithm is a yolov3 algorithm. 3.The method for pose transfer of 3D virtual human according to claim 1, wherein, The method further comprises: detecting a hand region of the target object in the rgb image by a target detection algorithm to crop a hand image of the target object; processing the hand image by an end-to-end hand neural network model to extract hand parameters of the target object; performing pose migration on the virtual person according to the hand parameters.

4. The pose transfer method for 3D virtual humans of claim 3, wherein, The hand neural network model comprises an encoder and a decoder, and the processing of the hand image by the end-to-end hand neural network model to extract the hand parameters of the target object comprises: inputting the hand image into the encoder for processing to obtain encoded features; regressing the encoded features by the decoder to obtain the hand parameters of the target object.

5. An apparatus for pose transfer of a 3D virtual human, characterized by, The device comprises: an acquisition module configured to acquire an rgb image of a target object by a monocular camera; a detection module configured to detect the target object in the rgb image by a target detection algorithm to obtain target frame information; the target frame information comprises first coordinate information of a top-left vertex and second coordinate information of a bottom-right vertex of a rectangular frame in which the target object is located; an extraction module configured to crop a target image from the rgb image according to the target frame information and extract first feature information of the target image; a fusion module configured to perform feature fusion processing on the target frame information and the first feature information to obtain target parameters; the target parameters comprise a position parameter, a deformation parameter and a camera angle parameter; The reconstruction module is configured to input the target parameter into a pre-established body prior model to obtain 3D point cloud data including a plurality of vertices returned by the body prior model, and to reconstruct a 3D pose of a virtual person according to the 3D point cloud data; The device is further configured to: calculate a center point coordinate of the rectangular frame and width information of the rectangular frame according to the first coordinate information and the second coordinate information; determine a focal length of the monocular camera according to a length and a width of the rgb image; extract global rotation information of the target object relative to the monocular camera according to the center point coordinate and the focal length; use the global rotation information and the width information as preprocessed target frame information.

6. The apparatus for pose transfer of a 3D virtual human of claim 5, wherein, The target frame information includes first coordinate information of a top-left vertex and second coordinate information of a bottom-right vertex of a rectangular frame in which the target object is located; and the device further includes: a preprocessing module, which is specifically configured to: calculate a center point coordinate of the rectangular frame and width information of the rectangular frame according to the first coordinate information and the second coordinate information; determine a focal length of the monocular camera according to a length and a width of the rgb image; extract global rotation information of the target object relative to the monocular camera according to the center point coordinate and the focal length; use the global rotation information and the width information as preprocessed target frame information.

7. A computer device, comprising: comprise: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor is caused to implement a pose migration method for a 3D virtual person according to any one of claims 1-4.

8. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that: The program executable by the processor, when executed by the processor, is configured to implement a pose migration method for a 3D virtual person according to any one of claims 1-4.

Images