Interaction method of virtual human body system, electronic device and computer readable medium

Abstract

The present invention discloses an interactive method, electronic device and computer-readable medium of a virtual human body system, wherein the interactive method comprises: obtaining body shape information of a target human body, constructing a human body virtual model of the target human body according to the body shape information; receiving a target associated object associated with the human body virtual model and indication information for indicating the morphological characteristics and position of the target associated object, matching the target associated object to the human body virtual model according to the indication information; in response to a morphological change of the human body virtual model, adjusting the morphological characteristics and position of the target associated object according to the morphological change of the human body virtual model. The present invention constructs a human body virtual model of a fitting person and matches the changes of associated objects in real time according to the changes of human body morphology, thereby effectively presenting a dressing process close to reality without the fitting person actually trying on clothes, allowing the fitting person to feel the dressing effect more timely and intuitively, thereby improving the customer shopping experience and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an interaction method for a virtual human body system, an electronic device and a computer-readable medium. Background Art

[0002] At present, when customers choose clothes in clothing stores, they often cannot choose the clothes they like in time because of too many varieties and styles, unclear display locations, excessive flow of people, long queues in changing rooms and other problems. As a result, they waste time and energy, affecting customers' shopping experience and efficiency.

[0003] Moreover, with the popularity of various online shopping platforms, more and more customers choose to select clothing through online shopping platforms. However, due to the limitations of online shopping, customers cannot view the physical clothing in time to try it on, and can only select by viewing pictures or videos of models trying on the clothes, thus affecting the customer's shopping experience and efficiency. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defect in the prior art that the person trying on clothes cannot feel the dressing effect in a timely and intuitive manner, which affects the customer's shopping experience and efficiency, and to provide an interactive method, electronic device and computer-readable medium for a virtual human body system.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A method for interacting with a virtual human body system, comprising:

[0007] Acquire body shape information of a target person, and construct a human body virtual model of the target person according to the body shape information;

[0008] receiving a target associated object associated with the human body virtual model and indication information for indicating the morphological characteristics and position of the target associated object, and matching the target associated object to the human body virtual model according to the indication information; and

[0009] In response to the morphological change of the human virtual model, the morphological features and position of the target associated object are adjusted according to the morphological change of the human virtual model.

[0010] Optionally, the step of acquiring body shape information of a target person and constructing a human body virtual model of the target person according to the body shape information comprises:

[0011] Scanning the target human body in all directions through the image acquisition module to acquire an image of the target human body;

[0012] Parsing the body shape information of the target human body from the image, wherein the body shape information includes joint feature parameters and body representation parameters, using a preset reference object as a coordinate origin, identifying the human body joints in the image to extract the joint feature parameters for constructing a human body virtual model;

[0013] Constructing a human body virtual model of the target human body according to the body shape information, wherein the human body virtual model includes a two-dimensional or three-dimensional human body virtual model;

[0014] By training a neural network, at least a portion of the human body virtual model is adjusted according to the body representation parameters to improve the human body virtual model.

[0015] Optionally, the step of constructing a human body virtual model of the target human body comprises:

[0016] A human virtual model of the target human body is constructed by training one or more neural networks, wherein the one or more neural networks are trained on multiple training images based on the human body to identify features presented in the training images, and infer joint angles associated with a root kinematic chain based on the identified features, and the one or more neural networks are also trained to infer joint angles associated with a head or limb kinematic chain based on the identified features and the joint angles associated with the root kinematic chain, wherein the root kinematic chain includes at least a chest region or a pelvic region of the human body, and the head or limb kinematic chain includes at least a head region or a limb region of the human body.

[0017] Optionally, the one or more neural networks are also trained to infer joint angles associated with the root kinematic chain or the end effector of the head or limb kinematic chain based on one or more other joints associated with the root kinematic chain or the head or limb kinematic chain, and the one or more neural networks are also trained to infer joint angles associated with a joint at or near the starting segment of the root kinematic chain or the head or limb kinematic chain based at least on the end effector associated with the root kinematic chain or the head or limb kinematic chain.

[0018] Optionally, it also includes:

[0019] An associated object is selected from an associated database as a target associated object and outputted, wherein the associated database is used to store a plurality of associated objects with different attribute information.

[0020] Optionally, it also includes:

[0021] By training a neural network, an associated object is recommended from the associated database as a target associated object according to the acquired body shape information and outputted.

[0022] Optionally, it also includes:

[0023] Acquire the environmental background of the human body virtual model;

[0024] In response to changes in the environmental background, the morphological features or positions of the target associated objects are adjusted according to the changes in the environmental background.

[0025] Optionally, the morphological change of the human virtual model includes any one or more of a body posture change, a body size change, and a body part shape change.

[0026] An electronic device includes a camera, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the camera is configured to collect the current body shape and form of a target human body and output them to the memory and the processor, and the processor is configured to implement the steps of the above-mentioned method for interacting with a virtual human body system when executing the computer program.

[0027] A computer-readable medium stores computer instructions, which, when executed by a processor, implement the steps of the above-mentioned method for interacting with a virtual human body system.

[0028] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present invention.

[0029] The positive and progressive effects of the present invention are:

[0030] The present invention constructs a virtual human model of the person trying on clothes and matches the changes of related objects in real time according to the changes in human body shape. It can effectively present a dressing process that is close to the real thing without the person trying on clothes actually trying on clothes, so that the person trying on clothes can feel the dressing effect more timely and intuitively, thereby improving the customer's shopping experience and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The features and advantages of the present invention can be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or characteristics may have the same or similar reference numerals.

[0032] Figure 1 4 is a flow chart of an interaction method of a virtual human body system according to an embodiment of the present invention.

[0033] Figure 2 FIG. 4 is a schematic diagram of the structure of an interactive device for a virtual human body system according to another embodiment of the present invention.

[0034] Figure 3The figure is a schematic diagram of the structure of an electronic device for implementing an interactive method of a virtual human body system according to another embodiment of the present invention.

[0035] Figure 4a An example block diagram illustrating the root kinematic chain of a human body is shown as an example.

[0036] Figure 4b An example block diagram illustrating a limb kinematic chain of a human body is shown as an example.

[0037] Figure 5 An example block diagram illustrating six kinematic chains of the human body is shown as an example.

[0038] Figure 6 A diagram illustrating an example neural network training process. DETAILED DESCRIPTION

[0039] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0040] In order to overcome the above-mentioned defects that currently exist, the present embodiment provides an interaction method of a virtualized human body system, including: acquiring body shape information of a target human body, and constructing a human virtual model of the target human body according to the body shape information; receiving a target associated object associated with the human virtual model and indication information for indicating the morphological characteristics or position of the target associated object, and matching the target associated object to the human virtual model according to the indication information; and, in response to a morphological change of the human virtual model, adjusting the morphological characteristics or position of the target associated object according to the morphological change of the human virtual model.

[0041] The interactive method provided in this embodiment can be effectively applied to the fitting system to effectively present a dressing process close to reality without the need for the person fitting the clothes to actually try them on, so that the person fitting the clothes can feel the dressing effect more timely and intuitively, thereby improving the customer shopping experience and efficiency. However, this embodiment does not specifically limit the application scenarios of the interactive method, and corresponding adjustments and selections can be made according to actual needs.

[0042] In this embodiment, the associated objects may include any one or more of trademark logos, advertising logos, and clothing virtual models, but the types of the associated objects are not specifically limited and may be adjusted and selected accordingly according to actual needs.

[0043] Specifically, as an embodiment, Figure 1 As shown, the interaction method of the virtual human body system provided in this embodiment mainly includes the following steps:

[0044] Step 101: Obtain body shape information of a target person.

[0045] In this step, the target human body and the environment are scanned 360 degrees by the image acquisition module to acquire images of the target human body and the environmental background, and the body shape information of the target human body is parsed from the images.

[0046] In this embodiment, the image acquisition module includes but is not limited to an instrument camera and / or various image acquisition related sensors.

[0047] In this embodiment, the body shape information includes but is not limited to joint feature parameters and body characterization parameters (height, gender characteristics, etc.), and of course, corresponding selection and adjustment can be made according to actual needs.

[0048] In this step, the preset reference object is used as the coordinate origin to identify the human body joints in the image to extract the joint feature parameters used to construct the human body virtual model.

[0049] Specifically, a scanned image is input, and a plurality of features Φ can be extracted from the image, for example, using an encoder module. The extracted features can be provided to a hierarchical motion posture / shape regression module, which can be configured to infer parameters from the extracted features so as to restore a human body virtual model. The inferred parameters can include, for example, one or more posture parameters Θ and one or more shape parameters β, which can indicate the posture and shape of an individual's body, respectively. Step 102, construct a human body virtual model of the target human body.

[0050] In this step, a corresponding human body virtual model is constructed according to the joint feature parameters of the target human body. Preferably, the human body virtual model is a three-dimensional human body virtual model, but it can also be a two-dimensional human body virtual model.

[0051] Specifically, the joint positions can be determined based on an image of the person, such as an image including color and / or depth information representing physical features of the person. The joint positions can be a subset of all joint positions of the person (e.g., excluding joints that are occluded or otherwise unknown). The processor implements an artificial neural network and provides information related to one or more joint positions of the person to the artificial neural network. The artificial neural network can determine a first plurality of parameters associated with the person's posture and a second plurality of parameters associated with the person's shape based on the information related to the one or more joint positions of the person. Based on the first plurality of parameters and the second plurality of parameters, one or more human models representing the person's posture and shape can be generated.

[0052] The artificial neural network can be trained using training data that includes joint positions of a human body. During training, the artificial neural network can predict posture and shape parameters associated with the human body based on the joint positions included in the training data. The artificial neural network can then infer the joint positions of the human body from the predicted posture and shape parameters, and adjust (e.g., optimize) operating parameters (e.g., weights) of the artificial neural network based on the difference between the inferred joint positions and the joint positions included in the training data. In an example, the training data may also include posture and shape parameters associated with the joint positions of the human body, and the artificial neural network may also adjust (e.g., optimize) its operating parameters based on the difference between the predicted posture and shape parameters and the posture and shape parameters included in the training data.

[0053] Each of the one or more neural networks may include multiple layers, such as an input layer, one or more convolutional layers, one or more non-linear activation layers, one or more pooling layers, one or more fully connected layers, and / or an output layer. Each of the layers may correspond to multiple filters (e.g., kernels), and each filter may be designed to detect (e.g., learn) a set of key points that collectively represent a corresponding feature or pattern. The filters may be associated with corresponding weights that, when applied to the input, produce an output indicating whether certain visual features or patterns have been detected. The weights associated with the filters may be learned by the neural network through a training process that includes: inputting a large number of images from one or more training data sets to the neural network, calculating the difference or loss resulting from the weights currently assigned to the filters (e.g., based on an objective function such as mean squared error or L1 norm, a loss function based on a margin, etc.), and updating the weights assigned to the filters so as to minimize the difference or loss (e.g., based on stochastic gradient descent). Once trained (e.g., having learned to recognize features and / or patterns in training images), the neural network can take an image at an input layer, extract and / or classify visual features or patterns from the image, and provide an indication of the identified features or feature classes at an output layer. The identified features can be indicated, for example, by feature descriptors or feature vectors.

[0054] One or more neural networks may also be trained to, for example, infer posture and shape parameters for restoring a three-dimensional human virtual model based on features extracted from an input image. For example, one or more neural networks may be trained to determine the joint angles of multiple joints of a person depicted in an input image based on a data set that covers a wide range of human subjects, human activities, background noise, shape and / or posture changes, camera motion, and the like. The multiple joints may include, for example, 23 joints and a root joint included in a skeletal device, and the posture parameters derived therefrom may include 72 parameters (e.g., 3 parameters for each of the 23 joints, and 3 parameters for the root joint, where each parameter corresponds to an axis-angle rotation starting from the root orientation). The neural network may also learn to determine one or more shape parameters based on a training data set, and the one or more shape parameters are used to predict a mixed shape of a person based on an image of the person. For example, the neural network may learn to determine shape parameters by performing principal component analysis (PCA), and the shape parameters determined thereby may include multiple coefficients (e.g., the first 10 coefficients) of the PCA space. Once the pose and shape parameters are determined, a plurality of vertices (e.g., 6,890 vertices based on 82 shape and pose parameters) may be obtained for constructing a representation (e.g., a 3D mesh) of the human body. Each vertex may include respective position, normal, texture, and / or shading information. Using these vertices, a 3D mesh of the person may be created, for example, by connecting a plurality of vertices with edges to form polygons (e.g., triangles), connecting a plurality of polygons to form surfaces, using a plurality of surfaces to determine a 3D shape, and applying textures and / or shading to the surfaces and / or shapes.

[0055] Figure 4a is an example block diagram illustrating a root kinematic chain 300a of a human body, Figure 4b 300b is an example block diagram illustrating a limb (e.g., right shoulder) kinematic chain 300b of a human body. The root kinematic chain 300a may include multiple joints in the core portion of the human body, such as joints 302a in the pelvis region, one or more joints 304a in the spine, joints 306a in the chest region, and an end effector 308a (e.g., an end region of the root kinematic chain 300a that may be connected to the next kinematic chain of the human body). Similarly, the limb kinematic chain 300b may include multiple joints along the right arm of the human body, such as a shoulder joint 302b, an elbow joint 304b, a wrist 306b, and an end effector 308b (e.g., a fingertip of the right hand).

[0056] As shown in the figure, the positions and / or joint angles of each corresponding kinematic chain (e.g., θ of the root kinematic chain) are pelvis , Θ spine , Θchest, and Θ of the limb kinetic chainshoulder , Θ elbow , Θ wrist ) can be associated with each other, thus, it can be seen that knowledge about the positions and / or joint angles of a subset of joints in the kinematic chain can be used to estimate the positions and / or joint angles of a second subset of joints in the kinematic chain. The estimation can be performed in a forward direction and / or a backward direction. In an example forward estimation involving a root kinematic chain 300a, the corresponding positions and / or joint angles of the pelvic joint 302a, the spinal joint 304a, and the thoracic joint 306a can be used to estimate the position and / or joint angles of the end effector 308a. Similarly, in an example forward estimation involving a limb kinematic chain 300b, the corresponding positions and / or joint angles of the shoulder joint 302b, the spinal joint 304b, and the thoracic joint 306b can be used to estimate the position and / or joint angles of the end effector 308b (e.g., a fingertip). In the reverse direction, the positions and / or joint angles of a subset of joints at or near the end effector can be used to estimate the positions and / or joint angles of the joints at or near the beginning segment of the kinematic chain. For example, with respect to the limb kinematic chain 300b, inverse estimation may include estimating the position and / or joint angle of the shoulder joint 302b based on the positions and / or angles of other joints in the kinematic chain, including, for example, one or more of the end effector 308b, the wrist 306b, or the elbow joint 304b.

[0057] In addition to the structural correlation between joints within a kinematic chain, the position and / or joint angle of a joint in a kinematic chain may also be affected by the position and / or joint angle of another kinematic chain. Figure 5 4 is an example block diagram illustrating six kinematic chains of a human body, including a root kinematic chain 402, a head kinematic chain 404, and four limb kinematic chains (e.g., a left arm kinematic chain 406, a right arm kinematic chain 408, a left leg kinematic chain 410, and a right leg kinematic chain 412). As described above, each kinematic chain may include multiple joints and end effectors, such as Figure 4a The pelvis 302a, spine 304a, chest 306a and end effector 308a are shown as well as Figure 4b 302b, elbow 304b, wrist 306b, and end effector 308b are shown. The kinematic chains can be interrelated with each other. For example, the joint angles of the head kinematic chain 404 and / or the joint angles in the limb kinematic chains 406, 408, 410, and 412 can depend on the joint angles of the root kinematic chain 402, for example, Figure 5 As shown by the arrows in FIG. 4 , for example, when the joint angle of the root kinematic chain 402 is Figure 5, the human body can be in an upright position, and the joint angles of other kinematic chains can be restricted to corresponding value ranges specified by the upright position of the body. Figure 5 As the illustrated state moves away, for example, as the body tilts to the side, the joint angles of the other kinematic chains may have different value ranges dictated by the new position of the body.

[0058] One or more neural networks (referred to herein as "neural networks") may be trained to learn structural correlations between joints and / or kinematic chains. Figure 6 is a diagram illustrating an example training process of a neural network. Figure 6 As shown, a neural network can be trained to estimate a posture parameter Θ and a shape parameter β associated with a human virtual model through an iterative process. For example, during an initial iteration t-1, the neural network can receive an input image and extract a plurality of features Φ from the input image. The neural network can also initialize the shape parameter to have a value β t-1 , and the posture parameters of each of the root chain 502, the head chain 504, the right arm chain 506, the left arm chain 508, the right leg chain 510 and the left leg chain 512 are initialized to have values ​​Θ respectively t-1 Root , Θ t-1 Head , Θ t-1 R.Arm , Θ t-1 L.Arm , Θ t-1 R.Leg and θ t-1 L.Leg These initial values ​​of the shape and / or posture parameters may be set, for example, based on a normal distribution (e.g., with a certain standard deviation). The neural network may then be based on the initial values ​​θ t-1 Root , initial shape parameter β t-1 and features Φ extracted from the input image to estimate (eg, predict) initial values ​​Θ for the pose parameters for the root chain 502 t-1 Root The adjustment ΔΘ can be applied to the initial value Θ t-1 Root , to derive an updated version of the root chain's pose parameters Θ t Root (For example, based on Θ t Root =Θ t-1 Root +ΔΘ). Using the updated posture parameter Θ for the root chain 502 t Root, the neural network may predict corresponding updated posture parameters for one or more (e.g., each) of the head or limb chains 504, 506, 508, 510, or 512. For example, the neural network may be based on the updated root chain posture parameter θ t Root , the initial pose parameter Θ of the head chain t-1 Head , initial shape parameter β t-1 and the features Φ extracted from the input image to predict updated posture parameters of the head motion chain 504. As another example, the neural network can be based on the updated root chain posture parameters Θ t Root , the initial posture parameter Θ of the right arm chain t-1 R.Arm , initial shape parameter β t-1 and the features Φ extracted from the input image to predict the updated posture parameters of the right arm kinematic chain 506. The neural network can also update the posture parameters of other kinematic chains in a similar manner.

[0059] Once the posture parameters θ (e.g., for the corresponding root chain 502 and head / limb chains 504-512) have been updated, the neural network can continue to update the posture parameters θ based on the initial shape parameters β. t-1 , updated posture parameter Θ t , and the feature Φ extracted from the input image to predict the shape parameter β t The neural network can then use the updated pose parameters Θ t (e.g., for the corresponding root chain 502 and head / limb chains 504-512), the updated shape parameters β t The and feature Φ are used as inputs for the next iteration of training, and the above estimation operation is repeated until one or more training termination criteria are met (e.g., after a predetermined number of training iterations are completed, when it is determined that the change in the objective function falls below a predetermined threshold, etc.). The neural network can optimize its parameters using an objective function based on one or more of the mean square error, the L1 norm, etc.

[0060] The neural network can also be trained to perform forward and reverse estimation in each of the kinematic chains 502, 504, 506, 508, 510, or 512 through an iterative process. For example, in the forward direction, the neural network can learn to predict the position and / or joint angle of the end effector in the kinematic chain based on the other joint positions and / or joint angles in the kinematic chain. Using the right arm kinematic chain 506 as an example, the neural network can receive an image of a person (e.g., a feature Φ extracted from the image), information about the positions and / or joint angles of other joints (e.g., including one or more joints of the right arm kinematic chain 506 and / or the joints of the root kinematic chain 502), and / or information about the shape of the person as input. Based on these inputs, the neural network can estimate the position and / or joint angle of the end effector (e.g., fingertips) of the right arm kinematic chain. The neural network can compare the estimated position and / or joint angle of the end effector with the real data of the position and / or joint angle, and determine the update of the relevant filter weights or parameters (e.g., related to the right arm kinematic chain) based on the objective function (e.g., loss function). The objective function can be implemented, for example, based on one or more of a mean square error, an L1 norm, etc., and the neural network can use a back-propagation process to update the filter weights or parameters (e.g., by determining the gradient of the objective function with respect to the current filter weights or parameters). The neural network can then repeat the foregoing process using the updated parameters until one or more training termination criteria are met (e.g., after a predetermined number of training iterations are completed, when the change in the value of the objective function between training iterations falls below a predetermined threshold, etc.).

[0061] In the reverse direction, the neural network can be trained in a similar manner to predict the position and / or joint angle of the joint at or near the beginning segment of the kinematic chain based on the end effector of the kinematic chain and / or other joints included in the chain. For example, information about the position and / or angle of the fingertips and other joints (e.g., the right wrist, right hand and / or right elbow included in the kinematic chain and / or the joints in the root kinematic chain 502) can be given to the neural network, and the neural network can use the given joint information together with the feature Φ and the shape of the person to learn parameters for predicting the position and / or joint angle of the right shoulder. This reverse training can help improve the ability of the neural network to estimate the joint position and / or joint angle in the forward direction, for example, when a portion of the human body is covered or blocked. More details about this feature will be provided in the examples below.

[0062] In order to acquire the ability to predict pose and shape parameters based on partial knowledge about the joint positions of a person (e.g., some joint positions of a person may be occluded, unobservable, or otherwise unknown to the neural network), training the artificial neural network may involve providing a subset of joint positions to the artificial neural network and forcing the artificial neural network to use the subset of joint positions to predict pose and shape parameters. For example, the training may utilize an existing parametric human body model associated with the human body to determine multiple joint positions of the human body, and then randomly exclude a subset of multiple joint positions from the input of the artificial neural network (e.g., by artificially treating the subset of joint positions as unobserved and unavailable).

[0063] The joint positions described herein may include two-dimensional (2D) and / or three-dimensional (3D) joint positions of a person. When at least 2D joint positions are used for training, the artificial neural network may predict posture and shape parameters based on the 2D joint positions during training, infer the 3D joint positions of the human body using the predicted posture and shape parameters, and project the 3D joint positions into the image space to obtain corresponding 2D joint positions. The artificial neural network may then adjust its operating parameters based on the difference between the projected 2D joint positions and the 2D joint positions included in the training data.

[0064] The pose and shape parameters may be recovered separately (eg, independently of each other). For example, the recovered pose parameters may be independent of body shape (eg, independent of a person's height and weight).

[0065] In this step, at least a part of the human body virtual model is adjusted according to the body representation parameters by training a neural network to improve the human body virtual model. The customer's body representation parameters can be used to match the specific size of the clothing; through the image, the representation parameters of each part of the customer's body are analyzed by artificial intelligence, and the customer's human body virtual model is improved by the body representation parameters; the clothing suitable for the human body virtual model is selected or automatically matched by the customer.

[0066] Step 103: Select or recommend a target associated object.

[0067] In this step, based on the received user instruction, an associated object is selected from the associated database as a target associated object and outputted. The associated database is used to store a plurality of associated objects with different attribute information.

[0068] In this step, by training a neural network, an associated object can be recommended from the associated database as a target associated object based on the acquired body shape information and output. For example, clothing suitable for the height can be automatically recommended based on parameters such as height, and suitable clothing can also be automatically recommended based on parameters such as gender characteristics and age.

[0069] Step 104: Match the target associated object to the human body virtual model.

[0070] In this step, based on user instructions or preset rules, indication information for indicating the morphological features or positions of the target associated object is obtained, and the target associated object is matched to the human body virtual model according to the indication information, and the human body virtual model matching the target associated object is output to the display module for display. For example, by using the existing three-dimensional model of clothes (geometric information and material information) and cooperating with the physical simulation of flexible materials of the rendering engine, the three-dimensional effect (with wrinkles) of clothes worn on the human body can be presented in real time.

[0071] In this embodiment, the display module is preferably a touch display screen that can provide a user interaction interface. Of course, other options can also be made according to actual needs.

[0072] Step 105: collect the morphological changes of the target human body in real time.

[0073] In this step, the current shape and dynamic changes of the shape of the target human body are captured by the image acquisition module, and the actual shape changes of the target human body are associated with the human body virtual model, so that the displayed human body virtual model also makes corresponding shape changes.

[0074] In this embodiment, the morphological change of the human body virtual model of the target human body includes but is not limited to any one or more of a body posture change, a body size change, and a body part shape change.

[0075] Step 106: adjusting the shape or position of the human body virtual model and the target associated objects in real time according to the shape changes of the target human body.

[0076] In this step, in response to the morphological change of the human virtual model, the morphological features or position of the target associated object are adjusted according to the morphological change of the human virtual model, and the morphological features or position of the target associated object matching the current morphological change is displayed through the display module.

[0077] Specifically, the human body parameters are fed back in real time, and the manipulable three-dimensional human body virtual model will be updated in real time in the rendering engine. The shape of the clothing will be updated in real time using flexible physical simulation (the shape of the clothing will change with the movement of the human body, including floating during rapid movement, etc.). The camera parameters are used to build a virtual camera in the rendering engine, and the three-dimensional shape of the clothing is filtered out by the human body model. The image of the clothing is rendered through the virtual camera, and the image of the clothing is overlapped on the image obtained by the real camera to create a virtual try-on effect.

[0078] Preferably, as an embodiment, in this step, in response to changes in the environmental background of the target human body, the morphological features or positions of the target associated objects are adjusted according to the changes in the environmental background.

[0079] The following specifically describes the functions and beneficial effects that can be achieved by using the above-mentioned interaction method.

[0080] 1. The display module can present a close-to-real appearance of the person being tried on. The person being tried on can perform various operations such as zooming in, zooming out, and rotating the virtual human body model by touching the display module.

[0081] 2. The person trying on clothes can browse all clothing styles in the system database and find out whether the corresponding size of the style is in stock. The person trying on clothes can change clothes on the virtual human model by clicking on a specific piece of clothing. The background will automatically fit and adjust the clothes to the virtual human model through a neural network, making the dressing effect more natural and closer to reality, thereby effectively saving the customer the process of going to the dressing room to change clothes. The customer can also experience the effect of trying on clothes on the online shopping platform.

[0082] 3. The person trying on the clothes can choose a virtual human model wearing the clothes and pose in different postures (sitting, standing, walking, running, etc.). Through artificial intelligence, the clothes will change with the changes in human body shape.

[0083] 4. Merchants can enter models wearing different styles into the system in the form of virtual human models for reference by those who try on clothes.

[0084] 5. The system database classifies different clothing styles according to style, target age and suitable body shape. The person trying on clothes can click "recommendation" to obtain the recommended clothing given by the system based on the reminder parameters, or they can choose the style type by themselves.

[0085] 6. The system can set different environmental backgrounds (including light brightness, indoor or outdoor, etc.) on the interface, so that the person trying on clothes can feel the dressing effect more intuitively.

[0086] 7. Not only clothing, but also trademark logos or advertising logos can be matched with the virtual human body model as related objects and the position, size, shape, etc. of the trademark logos or advertising logos can be adjusted according to the human body shape, so as to meet diverse needs and achieve a more dynamic and intuitive model effect.

[0087] Figure 2 The structure diagram of the interactive device of the virtual human body system provided according to another embodiment of the present invention is as follows. The interactive device uses the above-mentioned interactive method, and includes an image acquisition module 21 , an interactive module 22 and a display module 23 .

[0088] The image acquisition module 21 is mainly configured to acquire images of the target human body and the environmental background in real time and output them to the interaction module 22 .

[0089] The interaction module 22 is mainly configured to obtain the body shape information of the target human body, construct a human virtual model of the target human body according to the body shape information and output it to the display module 23; receive a target associated object associated with the human virtual model and indication information for indicating the morphological characteristics or position of the target associated object, match the target associated object to the human virtual model according to the indication information and output it to the display module 23; in response to a morphological change of the human virtual model, adjust the morphological characteristics or position of the target associated object according to the morphological change of the human virtual model.

[0090] The display module 23 is mainly configured to display the information received from the interaction module 22 in real time so that the user can view the changes of the human virtual model and associated objects in real time.

[0091] Figure 3 The present invention is a schematic diagram of the structure of an electronic device provided according to another embodiment of the present invention. The electronic device includes a camera, a memory, a processor, and a computer program stored in the memory and executable on the processor, the camera is configured to collect the current body shape and form of the target human body and output them to the memory and the processor, and the processor is configured to implement the steps of the above-mentioned virtual human body system interaction method when executing the computer program. Figure 3 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0092] like Figure 3 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0093] The bus 33 includes a data bus, an address bus, and a control bus.

[0094] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0095] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0096] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as the interaction method of the virtualized human body system in the above embodiment of the present invention.

[0097] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 3 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0098] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0099] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the interactive method of the virtual human body system in the above embodiment are implemented.

[0100] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0101] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the steps in the interaction method of the virtualized human body system in the above embodiment.

[0102] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0103] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. An interactive method for a virtual human body system, It is characterized in that include: Acquire body shape information of a target person, and construct a human body virtual model of the target person according to the body shape information; receiving a target associated object associated with the human body virtual model and indication information for indicating the morphological characteristics and position of the target associated object, and matching the target associated object to the human body virtual model according to the indication information; as well as Collecting the morphological changes of the target human body in real time, and associating the morphological changes of the target human body with the human body virtual model; In response to a morphological change of the virtual human model, adjusting the morphological features and position of the target associated object according to the morphological change of the virtual human model; The step of constructing a human body virtual model of the target human body comprises: A human virtual model of the target human body is constructed by training one or more neural networks, wherein the one or more neural networks are trained on multiple training images based on the human body to identify features presented in the training images, and infer joint angles associated with a root kinematic chain based on the identified features, and the one or more neural networks are also trained to infer joint angles associated with a head or limb kinematic chain based on the identified features and the joint angles associated with the root kinematic chain, wherein the root kinematic chain includes at least a chest region or a pelvic region of the human body, and the head or limb kinematic chain includes at least a head region or a limb region of the human body.

2. The interactive method according to claim 1, It is characterized in that The step of obtaining the body shape information of the target human body and constructing a human body virtual model of the target human body according to the body shape information comprises: Scanning the target human body in all directions through the image acquisition module to acquire an image of the target human body; Parsing the body shape information of the target human body from the image, wherein the body shape information includes joint feature parameters and body representation parameters, using a preset reference object as a coordinate origin, identifying the human body joints in the image to extract the joint feature parameters for constructing a human body virtual model; Constructing a human body virtual model of the target human body according to the body shape information, wherein the human body virtual model includes a two-dimensional or three-dimensional human body virtual model; By training a neural network, at least a portion of the human body virtual model is adjusted according to the body representation parameters to improve the human body virtual model.

3. The interactive method according to claim 1, It is characterized in that The one or more neural networks are also trained to infer joint angles associated with the root kinematic chain or the end effector of the head or limb kinematic chain based on one or more other joints associated with the root kinematic chain or the head or limb kinematic chain, and the one or more neural networks are also trained to infer joint angles associated with joints at or near the starting segment of the root kinematic chain or the head or limb kinematic chain based at least on the end effector associated with the root kinematic chain or the head or limb kinematic chain.

4. The interactive method according to claim 1, It is characterized in that Also includes: An associated object is selected from an associated database as a target associated object and outputted, wherein the associated database is used to store a plurality of associated objects with different attribute information.

5. The interactive method according to claim 4, It is characterized in that Also includes: By training a neural network, an associated object is recommended from the associated database as a target associated object according to the acquired body shape information and outputted.

6. The interactive method according to claim 1, It is characterized in that Also includes: Acquire the environmental background of the human body virtual model; In response to changes in the environmental background, the morphological features or positions of the target associated objects are adjusted according to the changes in the environmental background.

7. The interactive method according to claim 1, It is characterized in that The morphological change of the human virtual model includes any one or more of a body posture change, a body size change, and a body part shape change.

8. An electronic device comprising a camera, a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that The camera is configured to capture the current body shape and form of the target human body and output them to the memory and the processor, and the processor is configured to implement the steps of the interaction method of the virtualized human body system as described in any one of claims 1 to 7 when executing the computer program.

9. A computer readable medium having computer instructions stored thereon, It is characterized in that When the computer instructions are executed by a processor, the computer instructions implement the steps of the interaction method of a virtual human body system as claimed in any one of claims 1 to 7.

Images