Virtual human body gait clothes changing method and device, electronic equipment and medium

Abstract

The invention provides a virtual human body gait garment changing method and device, electronic equipment and a medium, and effectively solves the problem that a gait recognition technology is limited under garment changing conditions. The method comprises the steps that a multi-view two-dimensional gait joint point sequence of a human body is acquired, a three-dimensional gait joint point sequence is recovered, a three-dimensional human body grid is recovered, and human body skeleton parameters and human body shape parameters corresponding to the three-dimensional human body grid are calculated; adjusting a preset virtual human body template to obtain a virtual human body, and obtaining a target virtual human body; constructing a rotation space of joint points of the target virtual human body, and mapping a rotation vector corresponding to the rotation space to the target virtual human body; capturing reloading gait data generated when the target virtual human body is driven, and rendering the reloading gait data to generate multiple frames of reloading gait pictures so as to complete gait reloading of the virtual human body based on the multiple frames of reloading gait pictures.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more particularly, to a method, apparatus, electronic device, and medium for changing the clothing of a virtual human gait. Background Art

[0002] Gait recognition is a biometric recognition technology based on an individual's walking posture. It identifies a person's identity by analyzing features such as the posture and pace when walking, and is often used in multiple fields such as security monitoring, medical rehabilitation, and human-computer interaction. Although existing gait recognition methods have made rapid progress, they also face many challenges. For example, in the case of gait recognition under difficult conditions such as clothing changes, current gait recognition cannot demonstrate obvious advantages. An important reason is that it is difficult to collect gait data of the same person wearing different styles of clothing throughout the year, which limits the research on gait recognition across different clothing situations. Some current technologies have also started to study using synthetic data to solve the problem of insufficient cross-clothing data, but the issue of accurately defining the identity consistency between virtual and real people has not been addressed, resulting in limitations in gait recognition. Summary of the Invention

[0003] In view of this, the purpose of the present application is to provide a method, apparatus, electronic device, and medium for changing the clothing of a virtual human gait, which effectively solves the problem of limitations in gait recognition technology under clothing change conditions.

[0004] In a first aspect, an embodiment of the present application provides a method for changing the clothing of a virtual human gait, the method including:

[0005] Obtain a multi-view two-dimensional gait joint point sequence of a human body, restore the multi-view two-dimensional gait joint point sequence to a three-dimensional gait joint point sequence, and respectively restore a three-dimensional human body mesh and calculate human body bone parameters based on the three-dimensional gait joint point sequence, as well as human body shape parameters corresponding to the three-dimensional human body mesh;

[0006] Adjust a preset virtual human template based on the human body shape parameters and the human body bone parameters to obtain a virtual human, and make the virtual human wear a target virtual clothing to obtain a target virtual human; there are multiple sets of the target virtual clothing;

[0007] Construct a rotation space of the joints of the target virtual human based on the three-dimensional gait joint point sequence, and map the rotation vector corresponding to the rotation space to the target virtual human;

[0008] Capture the dressing gait data generated when driving the target virtual human body, and render the dressing gait data to generate multiple frames of dressing gait pictures, so as to complete the gait dressing of the virtual human body based on the multiple frames of dressing gait pictures.

[0009] Combined with the first aspect, the embodiments of the present application provide a first possible implementation manner of the first aspect. After rendering the dressing gait data to generate multiple frames of dressing gait pictures, it includes:

[0010] Randomly shoot the gait video of the human body to be queried, and compare the gait video of the human body to be queried with the multiple frames of dressing gait pictures to obtain a comparison result;

[0011] Based on the comparison result, determine that the human body to be queried matches the target human body corresponding to the multiple frames of dressing gait pictures.

[0012] Combined with the first aspect, the embodiments of the present application provide a second possible implementation manner of the first aspect. The comparison of the gait video of the human body to be queried with the multiple frames of dressing gait pictures to obtain a comparison result includes:

[0013] The gait sequence of the human body to be queried and the dressing gait sequence corresponding to the multiple frames of dressing gait pictures are respectively extracted by a picture comparison model pre-trained based on virtual synthesis data;

[0014] Calculate the similarity of the gait sequence features corresponding to the gait sequence of the human body to be queried and the dressing gait sequence, and generate the comparison result based on the similarity.

[0015] Combined with the first aspect, the embodiments of the present application provide a third possible implementation manner of the first aspect. Making the virtual human body wear the target virtual clothing to obtain the target virtual human body includes:

[0016] Obtain multiple groups of attribute parameters of the virtual clothing, and determine whether the multiple groups of attribute parameters match the body shape parameters and body bone parameters of the virtual human body;

[0017] If not, adjust the virtual clothing to obtain the target virtual clothing that matches the virtual human body.

[0018] Combined with the first aspect, the embodiments of the present application provide a fourth possible implementation manner of the first aspect. Adjusting the virtual clothing includes:

[0019] Determine the target attribute parameters with abnormal matching, and calculate the difference between the target attribute parameters and the corresponding body shape parameters;

[0020] Perform multi-dimensional analysis on the difference value to obtain an analysis result, and adjust the target attribute parameters of the virtual clothing based on the analysis result.

[0021] Combined with the first aspect, the embodiments of the present application provide a fifth possible implementation manner of the first aspect. Among them, the dressing gait data generated when the capture drives the target virtual human body includes:

[0022] Based on the relationship between the rotation vector and the joint points of the target virtual human body, determine the rotation joint points;

[0023] Control the rotation joint points to perform gait movement based on the rotation vector to generate dressing gait data.

[0024] Combined with the first aspect, the embodiments of the present application provide a sixth possible implementation manner of the first aspect. Among them, the adjustment of the preset virtual human body template based on the human body shape parameters and the human body bone parameters to obtain a virtual human body includes:

[0025] Call the template human body shape parameters and template human body bone parameters of the preset virtual human body template, and iteratively align the human body shape parameters and the human body bone parameters with the template human body shape parameters and the template human body bone parameters respectively to generate an iterative alignment result;

[0026] Based on the iterative alignment result, construct the virtual human body corresponding to the human body.

[0027] In the second aspect, the embodiments of the present application provide a dressing device for virtual human body gait. The device includes:

[0028] An acquisition module, configured to acquire a multi-view two-dimensional gait joint point sequence of a human body, restore the multi-view two-dimensional gait joint point sequence to a three-dimensional gait joint point sequence, and respectively restore a three-dimensional human body mesh and calculate human body bone parameters based on the three-dimensional gait joint point sequence, as well as human body shape parameters corresponding to the three-dimensional human body mesh;

[0029] A dressing module, configured to adjust a preset virtual human body template based on the human body shape parameters and the human body bone parameters to obtain a virtual human body, and make the virtual human body wear a target virtual clothing to obtain a target virtual human body; there are multiple sets of the target virtual clothing;

[0030] A mapping module, configured to construct a rotation space of the joint points of the target virtual human body based on the three-dimensional gait joint point sequence, and map the rotation vector corresponding to the rotation space to the target virtual human body;

[0031] A driving module, configured to capture the dressing gait data generated when driving the target virtual human body, and render the dressing gait data to generate multiple frames of dressing gait pictures, so as to complete the gait dressing of the virtual human body based on the multiple frames of dressing gait pictures.

[0032] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of any one of the methods for dressing the gait of a virtual human body are executed.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the methods for dressing the gait of a virtual human body are executed.

[0034] An embodiment of the present application provides a method for dressing the gait of a virtual human body. The method first obtains a multi-view two-dimensional gait joint point sequence of a human body, restores the multi-view two-dimensional gait joint point sequence to a three-dimensional gait joint point sequence, and respectively restores a three-dimensional human body mesh and calculates human body bone parameters based on the three-dimensional gait joint point sequence, as well as human body shape parameters corresponding to the three-dimensional human body mesh; secondly, adjusts a preset virtual human body template based on the human body shape parameters and the human body bone parameters to obtain a virtual human body, and makes the virtual human body wear a target virtual clothing to obtain a target virtual human body; the target virtual clothing has multiple sets; then constructs a rotation space of the joints of the target virtual human body based on the three-dimensional gait joint point sequence, maps the rotation vector corresponding to the rotation space to the target virtual human body; finally, captures the dressing gait data generated when driving the target virtual human body, and renders the dressing gait data to generate multiple frames of dressing gait pictures, so as to complete the gait dressing of the virtual human body based on the multiple frames of dressing gait pictures, thereby realizing the acquisition of dressing gait data when the virtual human body wears multiple sets of clothing, thus solving the limitation problem existing in the data acquisition of the existing gait recognition technology, and can realize the acquisition of dressing gait data when the human body wears different styles of clothing, thereby reducing the limitation received by the gait recognition research in the case of cross-dressing. Description of the Drawings

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1 Shows a schematic flowchart of the first method for changing clothes of a virtual human gait provided by an embodiment of the present application;

[0037] Figure 2 Shows another schematic flowchart of the first method for changing clothes of a virtual human gait provided by an embodiment of the present application;

[0038] Figure 3 Shows a schematic flowchart of the process for constructing a virtual human body provided by an embodiment of the present application;

[0039] Figure 4 Shows a structural block diagram of the first device for changing clothes of a virtual human gait provided by an embodiment of the present application;

[0040] Figure 5 Shows a structural block diagram of the first electronic device provided by an embodiment of the present application. Detailed implementation manners

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application show the operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0042] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the protection scope of the present application.

[0043] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0044] The acquisition of gait recognition data under clothing change conditions is quite difficult. It is hard to collect gait data of the same person wearing different styles of clothing throughout the four seasons, which limits the research on gait recognition across different clothing. Currently, some technologies have started to study using synthetic data to solve the problem of insufficient cross-clothing data, but the issue of accurately defining the identity consistency between virtual and real people has not been resolved, resulting in limitations in gait recognition.

[0045] Based on this, the embodiments of the present application provide a method, device, electronic device, and medium for changing the clothing of a virtual human gait, which will be described below through embodiments.

[0046] Embodiment 1

[0047] To facilitate the understanding of this embodiment, first, a method for changing the clothing of a virtual human gait disclosed in the embodiments of the present application will be introduced in detail. As Figure 1 shown in the flowchart of a method for changing the clothing of a virtual human gait, and as Figure 2 shown in another schematic flowchart of a method for changing the clothing of a virtual human gait, a method for changing the clothing of a virtual human gait provided by the present application includes:

[0048] S101. Obtain a multi-view two-dimensional gait joint point sequence of a human body, restore the multi-view two-dimensional gait joint point sequence to a three-dimensional gait joint point sequence, and respectively restore a three-dimensional human body mesh and calculate human body bone parameters based on the three-dimensional gait joint point sequence, as well as the human body shape parameters corresponding to the three-dimensional human body mesh;

[0049] S102. Adjust a preset virtual human template based on the human body shape parameters and the human body bone parameters to obtain a virtual human, and make the virtual human wear a target virtual clothing to obtain a target virtual human; the target virtual clothing has multiple sets;

[0050] S103. Construct a rotation space of the joints of the target virtual human based on the three-dimensional gait joint point sequence, and map the rotation vectors corresponding to the rotation space to the target virtual human;

[0051] S104. Capture the clothing change gait data generated when driving the target virtual human, and render the clothing change gait data to generate multiple frames of clothing change gait pictures, so as to complete the gait clothing change of the virtual human based on the multiple frames of clothing change gait pictures.

[0052] In step S101, the present application collects multiple frames of RGB images of a human body during gait through six perspective cameras. The six perspective cameras achieve a 360-degree surround of the human body, and it is possible to set that each perspective camera occupies a 60-degree viewing angle. The RGB images are two-dimensional, and the RGB images are output to the 3D virtual engine blender. The 3D virtual engine blender extracts multiple joint points in the multiple frames of RGB images based on a preset multi-perspective two-dimensional human pose estimation algorithm to obtain a multi-perspective two-dimensional gait joint point sequence during gait. The multi-perspective two-dimensional gait joint point sequence is two-dimensional, that is, the joint points in the multi-perspective two-dimensional gait joint point sequence are represented by (x, y). The joint points can be the head, shoulders, thighs, calves, ankles, etc. The multi-perspective two-dimensional gait joint point sequence is restored to a three-dimensional gait joint point sequence through a multi-perspective three-dimensional human pose estimation algorithm, that is, triangulation, in combination with the parameters of the perspective cameras. That is, the joint points in the three-dimensional gait joint point sequence are represented by (x, y, z). The parameters of the perspective cameras include internal parameters and external parameters, which are measured during the setup of the perspective cameras and used to process the three-dimensional gait joint point sequence with a three-dimensional human mesh estimation algorithm to restore the three-dimensional human mesh. The human bone parameters are also obtained by calculating the three-dimensional gait joint point sequence. The human bone parameters are the lengths of the human bones. The lengths of the human bones and the joint points involved in this bone can be obtained by subtracting the coordinate values. In addition, the human body shape parameters corresponding to the three-dimensional human mesh are obtained. The human body shape parameters include 12 common human body shape parameters such as waist circumference, chest circumference, hip circumference, arm circumference, and leg circumference.

[0053] In step S102, the present application pre-sets a virtual human template, and it is necessary to adjust the pre-set virtual human template based on the human body shape parameters and the human bone parameters to obtain a virtual human. At this time, the virtual human corresponds to the human body in step S101 and is exactly the same as the human body. Therefore, the three-dimensional virtual engine blender sends the calculated human body shape parameters and human bone parameters to the virtual engine MAKE HUMAN. Multiple sets of virtual clothes are set on the virtual engine MAKE HUMAN. Therefore, the target virtual clothes for the virtual human are selected. The present application sets multiple sets of virtual clothes, including upper clothes, lower clothes, hats, accessories, and various carrying items such as backpacks, which can be worn throughout the year. Then, the virtual clothes are adjusted to obtain the target virtual clothes, so that the virtual human wears the target virtual clothes, that is, the adjusted virtual clothes, to obtain the target virtual human. That is, the target virtual human is the virtual human wearing the appropriate virtual clothes after parameter adjustment. 100 sets of different virtual clothes are randomly matched. The number of sets of virtual clothes can also be set according to the actual situation. And the size of the virtual clothes is adjusted according to the body shape of the virtual human to ensure that the virtual clothes fit well on the virtual human. There are multiple sets of the target virtual clothes, and the target virtual clothes are the virtual clothes after parameter adjustment. That is, the present application sets that the virtual human can wear one set of the target virtual clothes each time and can wear it multiple times. That is, the target virtual human of the present application can wear 100 sets of virtual clothes, initially solving the problem that it is difficult to collect the same person wearing different styles of clothes throughout the year. The virtual engine MAKEHUMAN sends multiple target virtual humans wearing the target virtual clothes to the three-dimensional virtual engine blender.

[0054] In a specific implementation process of step S102, there is an embodiment as follows: As Figure 3 shown, the adjusting the pre-set virtual human template based on the human body shape parameters and the human bone parameters to obtain a virtual human includes:

[0055] S10211. Invoke the template human body shape parameters and template human bone parameters of the pre-set virtual human template, and generate an iterative alignment result based on the iterative alignment of the human body shape parameters and the human bone parameters with the template human body shape parameters and the template human bone parameters respectively;

[0056] S10212. Based on the iterative alignment result, construct the virtual human corresponding to the human body.

[0057] In steps S10211 - S10212, the present application calls various parameters of a preset virtual human template, including the body shape parameters and bone parameters of the template human body, and preliminarily matches the body shape parameters of the human body, the body shape parameters of the template human body, the bone parameters of the human body, and the bone parameters of the template human body respectively as the starting point of iterative alignment. An iterative algorithm (such as gradient descent, genetic algorithm, particle swarm optimization, etc.) is used to gradually adjust the input body shape parameters and bone parameters of the template human body. The iterative alignment results include the differences between the body shape parameters of the human body and the body shape parameters of the template human body, and the differences between the bone parameters of the human body and the bone parameters of the template human body. The alignment target of the iterative alignment is set such that the differences between the body shape parameters of the human body and the body shape parameters of the template human body, and the differences between the bone parameters of the human body and the bone parameters of the template human body are all less than a preset threshold. The preset threshold can be 0.1 cm. If the generated iterative alignment result reaches the alignment target, the iteration ends. At this time, the virtual human template after iterative alignment is the virtual human, that is, the virtual human construction is completed. If the generated iterative alignment result shows that the alignment target is not reached, the iteration continues until the iterative alignment result reaches the alignment target. During the iterative alignment process, the constraint conditions of the human body structure, such as bone length ratio, joint range of motion, etc., are considered to ensure that the adjusted parameters still conform to the ergonomic principle, and the iterative alignment result that reaches the alignment target is applied to the virtual human template to adjust the body shape and bone structure of the virtual human template to construct a virtual human for the target human body.

[0058] In the specific implementation process of step S102, there is another embodiment: making the virtual human wear the target virtual clothing to obtain the target virtual human body includes:

[0059] S10221. Obtain multiple groups of attribute parameters of the virtual clothing, and determine whether the multiple groups of attribute parameters match the body shape parameters and bone parameters of the virtual human body;

[0060] S10222. If not, adjust the virtual clothing to obtain the target virtual clothing that matches the virtual human body.

[0061] In steps S10221 - S10222, multiple groups of attribute parameters of the virtual clothing are obtained, such as clothing length, chest circumference, waist circumference, hip circumference, sleeve length, shoulder width, cutting, fabric thickness, etc. And according to the principles of clothing design and ergonomics, matching rules for attribute parameters, human body shape parameters, human bone parameters, and the dynamic changes of the human body during movement on the clothing wearing effect are used to determine whether the multiple groups of attribute parameters match the human body shape parameters and human bone parameters of the virtual human body. If they all match, the matching is normal, and there is no need to adjust the target virtual clothing. At this time, the virtual clothing is the target virtual clothing, which can make the virtual human body produce a suitable and comfortable effect. However, if a certain attribute parameter does not match the human body shape parameters and human bone parameters of the virtual human body, that is, the matching is abnormal, then based on the unmatched attribute parameter, adjustments are made so that the adjusted virtual clothing matches the target virtual clothing with the virtual human body, thus obtaining the target virtual human body. And during the adjustment process, it is also necessary to ensure that the selected target virtual clothing has sufficient flexibility and adjustability to adapt to the dynamic changes of the human body.

[0062] In the specific implementation process of step S10222, there is an embodiment: The adjustment of the virtual clothing includes:

[0063] S102221. Determine the target attribute parameter with abnormal matching, and calculate the difference between the target attribute parameter and the corresponding human body shape parameter;

[0064] S102222. Analyze the difference in multiple dimensions to obtain an analysis result, and based on the analysis result, adjust the target attribute parameter of the target virtual clothing.

[0065] In steps S102221 - S102222, let an attribute parameter that does not match the human body shape parameters and human bone parameters of the virtual human body be the target attribute parameter. Then calculate the difference between the target attribute parameter and the corresponding body shape parameter of the virtual human body, and based on the multi - dimensional analysis of the difference, obtain an analysis result. The dimensions respectively include: key dimension difference, dynamic difference, and fabric property difference. Among them, the key dimension difference is to calculate the difference of core parameters such as chest circumference, waist circumference, and hip circumference. The dynamic difference is the difference between the extension amount of the clothing when simulating human movement and the actual requirement (such as the change in sleeve length when the arm is raised). The fabric property difference is obtained by comparing the matching degree between the elastic modulus of the clothing fabric and the movement range of the human body. Corresponding weights are set for each dimension above, and after performing multiplication operations on the weights and the differences of the corresponding dimensions and then adding them together, the analysis result is obtained. The analysis result can be "It is recommended to shorten the sleeve length by 2 cm, or choose a fabric with higher elasticity", etc. Then, based on the analysis result, the target attribute parameter of the target virtual clothing is adjusted to make the effect of the target virtual clothing on the virtual human body suitable.

[0066] In step S103, the 3D virtual engine Blender defines a local coordinate system for each joint point. Usually, the parent joint point is used as the origin, the orientation of the joint point is the Z-axis, the lateral direction is the X-axis, and the vertical direction is the Y-axis. The rotation state of the joint point is represented by a quaternion or a rotation matrix. Key frames are extracted from the 3D gait joint point sequence to determine the rotation state of the joint point at different time points. The rotation transition between key frames is calculated using a preset interpolation method such as linear interpolation (LERP) or spherical linear interpolation (SLERP) to construct a continuous rotation space. The rotation space is usually a 3D space, and each dimension corresponds to a degree of rotational freedom (such as rotation around the X, Y, and Z axes). The rotation vector of each joint point is extracted from the rotation space, and the rotation vector corresponding to the rotation space is mapped to the target virtual human according to the correspondence of the joint points. For cases where precise control of the end position is required (such as when the foot touches the ground), the inverse kinematics algorithm is used to adjust the joint rotation to ensure the accuracy of the end position. During the mapping process, collision detection between the virtual human and the environment is considered to avoid penetration or unreasonable postures.

[0067] In step S104, the 3D virtual engine Blender applies the rotation vector to the corresponding joint points of the dressing model frame by frame, and calculates the global position and orientation drive of each joint point through the bone hierarchy relationship between the parent and child joint points, so that each joint point of the target virtual human can be transformed according to the rotation vector, thereby achieving the effect of driving the target virtual human, and capturing the dressing gait data generated when the target virtual human is driven. It also processes the bone offset caused by the volume difference of the clothing during the driving process (such as the shoulders moving up due to a thick coat). A virtual camera is also set to surround the target virtual human to capture the gait from different angles, and screen space ambient occlusion (SSAO) and dynamic shadows are added to enhance the realism. Each frame is rendered to generate dressing gait pictures, and based on the dressing gait pictures, the gait dressing of the target virtual human is completed, so as to obtain multiple frames of dressing gait pictures and dressing gait data of the virtual human wearing different target virtual clothes and different gaits, solving the problem that it is difficult to collect gait recognition data under dressing conditions, achieving the effect of collecting gait data of the same person wearing different styles of clothes throughout the year, and also avoiding the problem of inconsistent identities defined between virtual characters and real characters. The multiple dressing gait pictures are stored in a preset database for calling when needed.

[0068] In a specific implementation process of step S104, there is an embodiment: The capturing of the dressing gait data generated when driving the target virtual human includes:

[0069] S10411. Determine the rotated joint points based on the relationship between the rotation vector and the joint points of the target virtual human body;

[0070] S10412. Control the rotated joint points to perform gait movement based on the rotation vector to generate dressing gait data.

[0071] In steps S10411 - S10412, for the rotation vector of the joint points extracted and represented by quaternion, which is usually stored by time frame, import it onto the target virtual human body, ensure that its bone structure is consistent with the reference model corresponding to the rotation vector, align the initial pose (such as T-pose) of the target virtual human body with the reference pose of the rotation vector to eliminate the initial offset. According to the bone hierarchy relationship, map each element in the rotation vector to the corresponding joint points of the target virtual human body, such as the root node → the pelvis of the target virtual human body, the left arm → the root of the left sleeve of the target virtual human body, and reassign the rotation weights for the joint points affected by dressing (such as the shoulders moving up due to a thick coat), ensure that the clothing moves synchronously with the body of the target virtual human body, store the joint point positions and rotation angles of each frame as gait data. Among them, key frame sampling can also be performed on the rotation vector, only store the frames with significant changes to reduce the computational complexity, and record the vertex positions and wrinkle states of the target virtual clothing in each frame as supplementary data for the dressing gait data. During this process, perform real-time compensation for the joint point offsets caused by dressing (such as a backpack causing the spine to lean forward). When the joint points rotate, synchronously update the contact points between the target virtual clothing and the skin (such as the armpits and elbows) to prevent penetration, and also adjust the clothing dynamics according to the gait speed (such as the hem of the coat swinging violently when running fast).

[0072] In the specific implementation process of step S104, there is another embodiment: after rendering the dressing gait data to generate multiple frames of dressing gait pictures, it includes:

[0073] S10421. Randomly shoot the gait video of the human body to be queried, and compare the gait video of the human body to be queried with the multiple frames of dressing gait pictures to obtain a comparison result;

[0074] S10422. Based on the comparison result, determine the matching between the human body to be queried and the target human body corresponding to the multiple frames of dressing gait pictures.

[0075] In steps S10421 - S10422, in actual use of the present application, a gait video of the human body to be queried can be randomly captured. The human body to be queried is a random human body whose identity is unknown. Then, the gait video of the human body to be queried is converted into multiple pictures stored frame by frame. Based on a pre - trained picture comparison model established based on a convolutional neural network, the gait video of the human body to be queried is compared with the multiple pictures of the changed - clothes gait to obtain a comparison result. The shooting angle of the changed - clothes gait pictures is the same as that of the real video, and the resolution and color space of the gait video of the human body to be queried are aligned with those of the changed - clothes gait pictures. If the comparison result shows that the similarity exceeds a preset similarity threshold, it is determined that the human body to be queried matches the target human body corresponding to the multiple pictures of the changed - clothes gait, that is, the human body to be queried and the target human body corresponding to the multiple pictures of the changed - clothes gait are the same human body, thereby confirming the identity of the human body to be queried. Based on this method, the accuracy of the method provided by the present application can be verified, and it can also be applied to various occasions. For example, when the person to be queried is a vagrant, if it is determined that the gait video of the person to be queried corresponds to multiple pictures of the changed - clothes gait existing in the vagrant image library, where the multiple pictures of the changed - clothes gait are generated by vagrants in a preset image library. The image library includes, but is not limited to, the vagrant image library, and can also be an image library of missing persons and an image library of lost children. If the comparison result shows that the similarity does not exceed the preset similarity threshold, the gait of the next human body to be queried is captured again, and at the same time, the diversity of the gait data of the target human body is also expanded.

[0076] In a specific implementation process of step S10421, there is an embodiment: The comparison of the gait video of the human body to be queried with the multiple pictures of the changed - clothes gait to obtain a comparison result includes:

[0077] S104211. Respectively extract the gait sequence of the human body to be queried and the changed - clothes gait sequence corresponding to the multiple pictures of the changed - clothes gait from the picture comparison model pre - trained based on virtual synthetic data;

[0078] S104212. Calculate the similarity of the gait sequence features corresponding to the gait sequence of the human body to be queried and the changed - clothes gait sequence, and generate the comparison result based on the similarity.

[0079] In steps S104211 - S104212, the to - be - queried human gait video is pre - converted into multiple frames of to - be - queried pictures. Based on the trained picture comparison model, gait sequences corresponding to the multiple frames of to - be - queried pictures and multiple frames of dressed - up gait pictures are respectively extracted to obtain the to - be - queried human gait sequence and the dressed - up gait sequence. The picture comparison model is trained based on the pre - obtained virtual synthetic data. And gait sequence features corresponding to the to - be - queried human gait sequence and the dressed - up gait sequence are extracted. That is, pose estimation tools (such as OpenPose, MediaPipe) are used to obtain the joint point coordinates in the to - be - queried pictures and the multiple frames of dressed - up gait pictures respectively, and the joint point coordinates are respectively normalized (such as scaled relative to the height) to eliminate the scale difference. Among them, the joint point coordinates in the extracted dressed - up gait pictures are ensured to be consistent with the definition of the joint point coordinates in the to - be - queried human gait video. The normalized to - be - queried human gait sequence and the dressed - up gait sequence are input into the trained picture comparison model. The picture comparison model outputs gait feature vectors such as 512 - dimensional embedding vectors for each frame. The feature vectors within the gait cycle are aggregated using methods such as average pooling, LSTM encoding, etc. to generate sequence - level features, and a similarity calculation network including cosine similarity, Euclidean distance, or dynamic time warping (DTW), etc. is used to calculate the similarity of the sequence features. Then, a comparison result is generated based on the similarity. Based on the comparison result, it can be known whether the to - be - queried human gait video is similar to the multiple frames of dressed - up gait pictures.

[0080] Embodiment 2

[0081] The present application also provides a dressing - up device for virtual human gait, such as Figure 4 shown as a block diagram of a dressing - up device for virtual human gait. The functions implemented by the dressing - up device for virtual human gait correspond to the steps of executing a dressing - up method for virtual human gait on a terminal device. This device can be understood as a component of a server including a processor. The dressing - up device for virtual human gait described in the present application, the device includes:

[0082] An acquisition module 401, configured to acquire a multi - perspective two - dimensional gait joint point sequence of a human body, restore the multi - perspective two - dimensional gait joint point sequence to a three - dimensional gait joint point sequence, and respectively restore a three - dimensional human body mesh and calculate human body bone parameters based on the three - dimensional gait joint point sequence, as well as human body shape parameters corresponding to the three - dimensional human body mesh;

[0083] A dressing - up module 402, configured to adjust a preset virtual human template based on the human body shape parameters and the human body bone parameters to obtain a virtual human, and make the virtual human wear a target virtual clothing to obtain a target virtual human; there are multiple sets of the target virtual clothing;

[0084] A mapping module 403, configured to construct a rotation space for the joints of the target virtual human body based on the three-dimensional gait joint point sequence, and map the rotation vectors corresponding to the rotation space onto the target virtual human body;

[0085] A driving module 404, configured to capture the dressing gait data generated when driving the target virtual human body, and render the dressing gait data to generate multiple frames of dressing gait pictures, so as to complete the gait dressing of the virtual human body based on the multiple frames of dressing gait pictures.

[0086] In a feasible implementation manner, the driving module includes:

[0087] A shooting module, configured to randomly shoot a gait video of the human body to be queried, and compare the gait video of the human body to be queried with the multiple frames of dressing gait pictures to obtain a comparison result;

[0088] A first determination module, configured to determine that the human body to be queried matches the target human body corresponding to the multiple frames of dressing gait pictures based on the comparison result.

[0089] In a feasible implementation manner, the driving module further includes:

[0090] An extraction module, configured to respectively extract a gait sequence of the human body to be queried and a dressing gait sequence corresponding to the multiple frames of dressing gait pictures based on a picture comparison model pre-trained with virtual synthesis data;

[0091] A generation module, configured to calculate the similarity of the gait sequence features corresponding to the gait sequence of the human body to be queried and the dressing gait sequence, and generate the comparison result based on the similarity.

[0092] In a feasible implementation manner, the dressing module includes:

[0093] A judgment module, configured to obtain multiple groups of attribute parameters of the virtual clothing, and judge whether the multiple groups of attribute parameters match the body shape parameters and body bone parameters of the virtual human body;

[0094] An adjustment module, configured to, if not, adjust the virtual clothing to obtain a target virtual clothing that matches the virtual human body.

[0095] In a feasible implementation manner, the dressing module further includes:

[0096] A calculation module, configured to determine the target attribute parameters with abnormal matching, and calculate the difference between the target attribute parameters and the corresponding body shape parameters;

[0097] An analysis module is used to perform multi-dimensional analysis on the difference to obtain an analysis result, so as to adjust the target attribute parameters of the target virtual clothing based on the analysis result.

[0098] In a feasible implementation manner, the driving module also includes:

[0099] A second determination module is used to determine a rotating joint point based on the relationship between the rotation vector and the joint points of the target virtual human body;

[0100] A movement module is used to control the rotating joint point to perform gait movement based on the rotation vector, so as to generate dressing gait data.

[0101] In a feasible implementation manner, the wearing module also includes:

[0102] A calling module is used to call the template human body shape parameters and template human body bone parameters of the preset virtual human body template, and generate an iterative alignment result based on the iterative alignment of the human body shape parameters and the human body bone parameters with the template human body shape parameters and the template human body bone parameters respectively;

[0103] A construction module is used to construct the virtual human body corresponding to the human body based on the iterative alignment result.

[0104] Embodiment 3

[0105] The present application also provides an electronic device, as Figure 5 shown, including: a processor 501, a memory 502, and a bus 503. The memory 502 stores machine-readable instructions executable by the processor 501. When the electronic device runs, the processor 501 communicates with the memory 502 through the bus 503. When the machine-readable instructions are executed by the processor 501, the steps of any one of the described methods for changing clothes with a virtual human body gait are executed.

[0106] Embodiment 4

[0107] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the described methods for changing clothes with a virtual human body gait are executed.

[0108] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated herein. In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0109] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0111] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0112] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for changing clothes of a virtual human gait, characterized in that, The method includes: Obtaining a multi-view two-dimensional gait joint point sequence of a human body, restoring the multi-view two-dimensional gait joint point sequence to a three-dimensional gait joint point sequence, and respectively restoring a three-dimensional human body mesh and calculating human body bone parameters based on the three-dimensional gait joint point sequence, as well as human body shape parameters corresponding to the three-dimensional human body mesh; Adjusting a preset virtual human template based on the human body shape parameters and the human body bone parameters to obtain a virtual human, and dressing the virtual human in a target virtual clothing to obtain a target virtual human; there are multiple sets of the target virtual clothing; Constructing a rotation space of the joints of the target virtual human based on the three-dimensional gait joint point sequence, and mapping the rotation vectors corresponding to the rotation space to the target virtual human; Capturing the dressing gait data generated when driving the target virtual human, and rendering the dressing gait data to generate multiple frames of dressing gait pictures, so as to complete the gait dressing of the virtual human based on the multiple frames of dressing gait pictures.

2. The method according to claim 1, characterized in that, After rendering the dressing gait data to generate multiple frames of dressing gait pictures, it includes: Randomly shooting a gait video of a human body to be queried, and comparing the gait video of the human body to be queried with the multiple frames of dressing gait pictures to obtain a comparison result; Based on the comparison result, determining that the human body to be queried matches the target human body corresponding to the multiple frames of dressing gait pictures.

3. The method according to claim 2, wherein The comparing the gait video of the human body to be queried with the multiple frames of dressing gait pictures to obtain a comparison result includes: Respectively extracting a gait sequence of the human body to be queried and a dressing gait sequence corresponding to the multiple frames of dressing gait pictures by a picture comparison model pre-trained based on virtual synthesis data; Calculating the similarity of the gait sequence features corresponding to the gait sequence of the human body to be queried and the dressing gait sequence, and generating the comparison result based on the similarity.

4. The method according to claim 1, characterized in that, The dressing the virtual human in a target virtual clothing to obtain a target virtual human includes: Obtaining multiple groups of attribute parameters of the virtual clothing, and determining whether the multiple groups of attribute parameters match the human body shape parameters and the human body bone parameters of the virtual human; If not, adjusting the virtual clothing to obtain a target virtual clothing that matches the virtual human.

5. The method according to claim 4, characterized in that, The adjusting the virtual clothing includes: Determining the target attribute parameters with abnormal matching, and calculating the difference between the target attribute parameters and the corresponding human body shape parameters; Analyzing the difference in multiple dimensions to obtain an analysis result, so as to adjust the target attribute parameters of the virtual clothing based on the analysis result.

6. The method according to claim 1, wherein The capturing the dressing gait data generated when driving the target virtual human includes: Determining rotation joints based on the relationship between the rotation vector and the joints of the target virtual human; Controlling the rotation joints to perform gait movement based on the rotation vector to generate dressing gait data.

7. The method according to claim 1, characterized in that, The adjusting a preset virtual human template based on the human body shape parameters and the human body bone parameters to obtain a virtual human includes: Call the template human body shape parameters and template human body bone parameters of the preset virtual human template, and generate an iterative alignment result based on the iterative alignment of the human body shape parameters and the human body bone parameters with the template human body shape parameters and the template human body bone parameters respectively; Based on the iterative alignment result, construct the virtual human corresponding to the human body.

8. A dressing device for virtual human gait, characterized in that, The device includes: An acquisition module, configured to acquire a multi-view two-dimensional gait joint point sequence of a human body, restore the multi-view two-dimensional gait joint point sequence to a three-dimensional gait joint point sequence, and respectively restore a three-dimensional human body mesh and calculate human body bone parameters based on the three-dimensional gait joint point sequence, as well as the human body shape parameters corresponding to the three-dimensional human body mesh; A dressing module, configured to adjust a preset virtual human template based on the human body shape parameters and the human body bone parameters to obtain a virtual human, and dress the virtual human in a target virtual clothing to obtain a target virtual human; there are multiple sets of the target virtual clothing; A mapping module, configured to construct a rotation space of the joints of the target virtual human based on the three-dimensional gait joint point sequence, and map the rotation vector corresponding to the rotation space to the target virtual human; A driving module, configured to capture and drive the dressing gait data generated when driving the target virtual human, and render the dressing gait data to generate multiple frames of dressing gait pictures, so as to complete the gait dressing of the virtual human based on the multiple frames of dressing gait pictures.

9. An electronic device, characterized in that, Includes: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of a method for dressing a virtual human gait according to any one of claims 1 to 7 are executed.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of a method for dressing a virtual human gait according to any one of claims 1 to 7 are executed.

Images