Digital human generation method, platform, electronic device and storage medium

Abstract

The present disclosure provides a method, platform, electronic device and storage medium for generating digital humans, which involve technical fields such as augmented reality, virtual reality, computer vision, and deep learning, and can be applied to scenes such as the metaverse and virtual digital humans. The specific implementation scheme includes: based on the picture of the digital human to be generated, obtaining the corresponding target object model; based on the key features of the head in the picture, obtaining the point cloud of the corresponding key features of the head from a preconfigured feature library; fusing the point cloud of the key features of the head into the target object model to obtain the image of the digital human. The technology disclosed in the present disclosure can automatically realize the production of digital humans and obtain the image of digital humans, which can effectively save the production cost and cycle of the digital human image; moreover, it can also effectively improve the accuracy of the generated digital human image and the generation efficiency of the digital human image.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer technology and artificial intelligence technology, specifically to the fields of augmented reality, virtual reality, computer vision, deep learning, etc., and can be applied to scenes such as metaverse and virtual digital human. Specifically, it relates to a method, platform, electronic device and storage medium for generating digital human. Background Art

[0002] Traditional digital human image production, especially the production of three-dimensional hyper-realistic digital human images, requires extremely professional original painting, modeling, binding, animation and other stages. The manual production cycle usually takes several months, the investment cost is extremely high, the effect can only be displayed in stages, and modification, optimization and other operations require a lot of additional investment.

[0003] Moreover, with traditional technologies, it is difficult to restore the ideal human features in the real world, such as eye shape and lip shape, of artificially created digital human images, and the final presentation effect is greatly limited by the personal ability levels of each link in the digital human production. Summary of the invention

[0004] The present disclosure provides a method, platform, electronic device and storage medium for generating a digital human.

[0005] According to one aspect of the present disclosure, a method for generating a digital human is provided, comprising:

[0006] Based on the image of the digital human to be generated, obtain the corresponding target object model;

[0007] Based on the key features of the head in the image, obtaining a point cloud of the corresponding key features of the head from a preconfigured feature library;

[0008] The point cloud of the key features of the head is fused into the target object model to obtain a digital human image.

[0009] According to another aspect of the present disclosure, a digital human generation platform is provided, comprising:

[0010] A model acquisition module, used to acquire a corresponding target object model based on the image of the digital human to be generated;

[0011] A feature acquisition module, used to acquire a point cloud of the corresponding key features of the head from a preconfigured feature library based on the key features of the head in the image;

[0012] The fusion module is used to fuse the point cloud of the key features of the head into the target object model to obtain a digital human image.

[0013] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0017] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method of the above-mentioned aspect and any possible implementation manner.

[0018] According to yet another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned aspects and any possible implementation method.

[0019] According to the technology disclosed in the present invention, the production of digital humans can be automatically realized to obtain digital human images, overcoming the problems of high cost and poor accuracy in the traditional technology of manually producing digital humans in multiple stages, and being able to effectively save the production cost and cycle of digital human images; moreover, it can also effectively improve the accuracy of the generated digital human images and the efficiency of generating digital human images.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0022] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0024] Figure 3 This is a schematic diagram of an interface display of attribute information of key features of a head provided by the present disclosure;

[0025] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;

[0026] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0027] Figure 6 The block diagram is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] Obviously, the described embodiments are only part of the embodiments of the present disclosure, but not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure.

[0030] It should be noted that the terminal devices involved in the embodiments of the present disclosure may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0031] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0032] The production of digital humans using traditional technology basically relies on manual work, which not only has high production costs, but also produces digital humans with low accuracy and poor results.

[0033] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure; Figure 1 As shown, this embodiment provides a method for generating a digital human, which may specifically include the following steps:

[0034] S101, based on the image of the digital human to be generated, obtaining a corresponding target object model;

[0035] S102, based on the key features of the head in the image, obtaining a point cloud of the corresponding key features of the head from a preconfigured feature library;

[0036] S103, integrating the point cloud of key features of the head into the target object model to obtain a digital human image.

[0037] The execution subject of the digital human generation method of this embodiment may be a digital human generation platform, which may be a software integrated platform that can automatically generate digital humans. Alternatively, a corresponding electronic device may be configured based on the platform to generate digital humans.

[0038] In this embodiment, the corresponding digital human image can be automatically generated based on the picture of the digital human to be generated. The picture of the digital human to be generated is a two-dimensional 2D picture. The digital human image can be a 3D digital human image.

[0039] Since the features of the digital human in the generated digital human image are mainly concentrated in the head and the facial area of ​​the head, in this embodiment, the image of the digital human to be generated can be a frontal image including the head of the person, for example, it can be only a head portrait, or an upper body image including the features of the head of the person.

[0040] The target object model of this embodiment can be a pre-created prototype of a digital human model. In order to improve the efficiency of digital human generation, in this embodiment, a digital human model prototype does not need to be created, and the corresponding digital human image is directly generated based on the acquired target object model.

[0041] Since the features of the digital human are mainly concentrated on the head, in this embodiment, for each key feature of the head in the image, the point cloud of the corresponding key feature of the head can be obtained from the feature library and fused into the target object model to finally obtain the image of the digital human.

[0042] In this embodiment, the key features of the head can be defined according to actual needs, including at least one feature such as eyebrows, eyes, nose, mouth, and face shape.

[0043] The method for generating digital humans in this embodiment can automatically realize the production of digital humans and obtain digital human images through the above technical solutions, thus overcoming the problems of high cost and poor accuracy in the traditional technology of manually producing digital humans in multiple stages, and can effectively save the production cost and cycle of digital human images; moreover, it can also effectively improve the accuracy of the generated digital human images and the generation efficiency of the digital human images, so that the generated digital human images have a hyper-realistic performance effect; at the same time, it can also effectively lower the threshold for digital human production and promote the progress of the digital industry.

[0044] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure; Figure 2 As shown, the method for generating a digital human in this embodiment, in the above Figure 1Based on the technical solutions of the embodiments shown in the figure, the technical solutions of the present disclosure are further described in more detail. Figure 2 As shown, the method for generating a digital human in this embodiment may specifically include the following steps:

[0045] S201, extracting attribute features of the digital human based on the image of the digital human to be generated;

[0046] Since the main features of the digital human are concentrated on the head, the image of the digital human to be generated in this embodiment is a frontal image including the head, which can clearly show the key feature areas of the head on the front side of the person's head.

[0047] For example, in this embodiment, a pre-trained attribute feature extraction model can be used to extract the attribute features of a digital person. The attribute feature extraction model is a pre-trained neural network model. When used, a picture of a digital person is input to the attribute feature extraction model, and the attribute feature extraction model can extract the attribute features of the person in the picture based on the picture. For example, the attribute features of this embodiment can include middle-aged women, middle-aged men, girls, boys, elderly men or elderly women, etc.

[0048] In addition, optionally, in this embodiment, other algorithms may also be used to analyze the image of the digital human to be generated to obtain the attribute characteristics of the digital human, which is not limited here.

[0049] S202, based on the attribute characteristics of the digital human, obtaining a corresponding target object model from a preset model library;

[0050] Since characters with different attribute characteristics have different facial area characteristics, in order to more accurately generate a digital human image, in this embodiment, a model library may be pre-set, and multiple object models may be pre-set in the model library, such as at least one of a middle-aged female model, a middle-aged male model, a girl model, a boy model, an elderly male model, and an elderly female model. For each object model, corresponding attribute characteristics may be set when pre-set. Therefore, according to the attribute characteristics of the digital human, the corresponding target object model can be obtained from the preset model library. Compared with the prior art, the original drawing and modeling processes can be directly omitted, and the pre-set matching target object model can be directly used, which can effectively improve the production efficiency of the digital human image.

[0051] Optionally, in an embodiment of the present disclosure, if the attribute features of the digital human are not extracted based on the image of the digital human to be generated, a pre-configured standard model can be used as the target object model. The standard model can be a model in a preset model library, such as a middle-aged female model. Alternatively, a separate standard model can be pre-set to be used when the attribute features cannot be extracted, so as to ensure that a reasonable target object model can be obtained in any case.

[0052] S203, based on the multiple head key features included in the image, detect whether there are any unfused head key features; if so, execute step S204, otherwise, execute step S210;

[0053] Specifically, since there may be multiple key features of the head in the image, in order to facilitate fusion into the target object model one by one, each time a key feature of the head is fused, it can be marked to avoid repeated operations.

[0054] S204, obtaining target attribute information of an unfused head key feature in the image; executing step S205;

[0055] The key features of the head in this embodiment may be eyebrows, eyes, nose, mouth, ears or face shape, etc. The corresponding target attribute information may be used to define the key features of the head, such as the attribute information of the eyes may include almond eyes, phoenix eyes, eagle eyes, slender eyes, squinting eyes, etc.

[0056] In this embodiment, an attribute extraction model for each key feature of the head may be pre-trained to extract the target attribute information of the key feature of the head. Alternatively, the target attribute information of the key feature of the head in the picture selected by the user may be obtained.

[0057] S205, based on the target attribute information of the key features of the head, obtaining the point cloud of the corresponding key features of the head from the preconfigured feature library; executing step S206;

[0058] Before step S205 , the method may include: collecting point clouds of multiple key head features of each of the multiple characters and attribute information of each key head feature, and storing them in a feature library.

[0059] In this embodiment, the collected multiple characters include characters with different attribute information, such as middle-aged women, middle-aged men, girls, boys, elderly men or elderly women, etc. Moreover, the attribute information of the same key head feature in multiple characters is as different as possible. It is necessary to include rich key head features with various types of attribute information in the feature library, so as to provide effective feature data support for the generation of digital humans. For example, for eyebrows, the point clouds of eyebrows of characters with different eyebrow shapes and the corresponding eyebrow shape information are collected as much as possible. For eyes, the point clouds of eyes of characters with different eye shapes and the corresponding eye shape information are collected as much as possible. For mouths, the point clouds of mouths of characters with different mouth shapes and the corresponding mouth shape information are collected as much as possible, and so on. Based on the feature library established above, when step S205 is specifically implemented, the digital human generation platform can automatically match the point clouds of the key head features corresponding to the target attribute information according to the target attribute information of the key head features and all the target attribute information of the key head features in the feature library.

[0060] For example, Figure 3 This is a schematic diagram of an interface display of attribute information of a key feature of a head provided by the present disclosure. Figure 3 As shown, in the digital human generation platform, taking the nose as an example, the key feature of the head is Figure 3 The various nose shapes shown in the figure are the nose shapes supported by the feature library in the digital human generation platform. When the user clicks on the nose shape, Figure 3 In the interface shown, the user can select a nose shape that best matches the nose shape in the image of the digital person to be generated, and the user can also reset the currently selected nose shape to 0 and reselect a nose shape.

[0061] S206, registering the point cloud of the key features of the head with the target object model; executing step S207;

[0062] The registration of this embodiment can be adjusted by at least one operation mode such as displacement, rotation and scaling. For example, the point cloud of the key features of the head can be adjusted so that the coordinate system of the point cloud of the key features of the head is consistent with the target object model, and the size of the point cloud of the key features of the head is consistent with the regional size of the key features of the head of the target object model, so that the point cloud of the key features of the head can be accurately and efficiently matched with the target object model.

[0063] S207, migrating the point cloud of the key features of the head to the region of the corresponding key features of the head in the target object model; executing step S208;

[0064] S208, whether the similarity between the key features of the head in the target object model after detection and the key features of the head in the picture is greater than or equal to a preset similarity threshold; if so, return to step S203; otherwise, execute step S209;

[0065] For example, the key features of the head in the front view of the fused target object model and the key features of the head in the picture can be respectively intercepted, and then a pre-trained similarity calculation model is used to calculate the similarity between the two. Alternatively, a pre-trained feature expression model can be used to extract the feature expression of the key feature region of the head in the front view of the fused target object model and the feature expression of the key features of the head in the picture, and then the similarity between the feature expressions of the two can be calculated as the similarity between the two.

[0066] Based on this step, it can be known that in actual applications, there are also situations where the ideal effect is achieved after the key features of the head are integrated. At this time, the face pinching operation is no longer performed for the key features of the head.

[0067] S209, based on the user's trigger, performing a face pinching operation on the point cloud of the key features of the head fused to the target object model; returning to step S208;

[0068] The face pinching operation in this embodiment is based on the pre-established implicit constraint surface and the preset dynamic curve as constraints, and based on the user's trigger, the face pinching operation is performed on the point cloud of the key features of the head fused to the target object model. Specifically, the following steps may be included:

[0069] (1) Obtaining action information of the first controller configured on the implicit constraint surface triggered by the user;

[0070] Before the first controller is adjusted, the implicit constraint surface is a surface consistent with the topological structure of the surface of the target object model, and the identification and position of the points included are exactly the same; multiple first controllers are configured on the implicit constraint surface, but the implicit constraint surface is invisible to the user; there is a first motion mapping relationship between each first controller and the point on the implicit constraint surface, so that when the first controller is actuated, it can drive the point on the implicit constraint surface to move.

[0071] In the disclosed embodiment, since the main features of the digital human image are concentrated on the head, the fusion and face-pinching operations are mainly performed on the key features of the head of the target object model. At this time, the topological structure of the implicit constraint surface and the target object model is limited to the head of the digital human. In this embodiment, the action information of the first controller may include the direction and displacement of the movement of the first controller.

[0072] (2) based on a pre-established first motion mapping relationship between a first controller on an implicit constraint surface and a point on the implicit constraint surface, and the motion information of the first controller, obtaining motion information of the triggered point on the implicit constraint surface;

[0073] Optionally, in this embodiment, more than 85 available first controllers can be designed on the implicit constraint surface based on facial features, muscle direction, bone root position, etc. There can be more than 20,000 point clouds on the target object model. Therefore, one first controller can trigger the movement of multiple points on the target object model. When the user triggers the first controller, the movement of only one first controller can be triggered, or the movement of multiple first controllers can be triggered at the same time. For example, the control interface at this time can first prompt the user to select the first controller to be triggered, and after confirmation, the user can trigger the movement of all the selected first controllers at the same time. In this embodiment, the action information of the point may also include the movement direction and displacement of the point.

[0074] Specifically, the action information of each point can be determined by weighted summing up the motion information of the point triggered by each first controller among multiple first controllers and the weight of each first controller to determine the final action information of the point and then determine the motion position of the point.

[0075] The main purpose of the implicit constraint surface of this embodiment is to solve the unreasonable situations caused by large body changes, such as changes in the top of the skull, which will be transmitted to the forehead, sides, back of the head, and neck through this constraint, so that the physics and aesthetics of the target object model are guaranteed to a certain extent, which can effectively improve the aesthetics of the generated digital human.

[0076] (3) determining, based on the action information of the triggered point on the implicit constraint surface, the action information of a second controller at the same position as the point on the target object model;

[0077] In this embodiment, part of the second controllers are deployed on the points of the target object model. During the control process, after the point on the implicit constraint surface is triggered to move, the second controller deployed on the point with the same mark on the target object model should follow the action of the point on the implicit constraint surface to perform synchronous action.

[0078] (4) controlling the motion information of the point on the target object model based on a pre-established second motion mapping relationship between a second controller configured on the target object model and a point on the target object model, and the motion information of the second controller;

[0079] In this embodiment, more than 550 second controllers are configured on the target object model. Similarly, each second controller also establishes a binding relationship with multiple points of the target object model, such as establishing the above-mentioned second motion mapping relationship. Compared with the number of first controllers, the number of second controllers is greater, which can achieve more precise control of the points on the target object model.

[0080] Based on step (3), it can be known that after the action of the triggered point on the implicit constraint surface, the second controller deployed on the point with the same mark on the target object model should follow the action of the point and move synchronously, and then according to the second motion mapping relationship between the second controller configured on the target object model and the point on the target object model, the action information of all points bound to the second controller on the target object model can be obtained, and then the action of the corresponding point can be controlled based on the action information of the point on the target object model. This process can transfer the action of the first controller on the implicit constraint surface to the action of the point on the target object model, so as to achieve precise adjustment of the target object model.

[0081] (5) Using a preset dynamic curve as a constraint, the point cloud of the key features of the head fused to the target object model is adjusted in position.

[0082] For example, when step (5) is implemented specifically, it may include the following steps:

[0083] (a) detecting whether the second controller is on a preset dynamic curve; at least two second controllers are configured on each preset dynamic curve;

[0084] The preset dynamic curves in this embodiment can be some natural curves of the face of the target object model, such as the jaw line, the chin line, or the nasolabial folds. Since it is not easy for a single controller to achieve a continuous, smooth, and natural topological effect when adjusting the jaw line, the chin line, the nasolabial folds, and other areas during the face pinching optimization process. Therefore, in this embodiment, a dynamic curve is introduced, so that by controlling the movement of at least two second controllers on the dynamic curve, the dynamic curve can be made more continuous, smooth, and natural.

[0085] (b) in response to the second controller being on a preset dynamic curve, based on the action information of the second controller, acquiring action information of other second controllers on the dynamic curve;

[0086] (c) Based on the motion information of the second controller and the motion information of other second controllers on the dynamic curve, the point cloud of the key features of the head fused to the target object model is adjusted in position.

[0087] Furthermore, in response to the second controller not being on the preset dynamic curve, the method may also include: adjusting the position of the point cloud of the key features of the head fused to the target object model based on the action information of the second controller.

[0088] Based on the above, it can be understood that the core of dynamic curve constraint is to connect multiple continuous second controllers of the same level in series using a space curve. When the endpoint is fixed, the position change and rotation change of each second controller are transmitted to its surroundings. The closer the second controller is to it, the greater the weight. On the contrary, the farther away from it and the closer to the endpoint, the smaller the weight. The relationship between weight and distance is also pre-configured.

[0089] In this embodiment, based on the motion information of the second controller and the motion information of other second controllers on the dynamic curve, when the face pinching operation is performed on the point cloud of the key features of the head fused to the target object model, the points around the adjusted points are also smoothed. In this embodiment, the adjustment is performed with the feature curve where the point is located as a constraint.

[0090] For example, the surface of the target object model itself is a grid structure consisting of multiple points and edges between adjacent points. Therefore, when adjusting a certain point, at the topological level, the feature curve where the point is located can be taken as the center, and its adjacent topological lines also need to be updated to ensure the uniformity and smoothness of the topology. This process mainly uses dynamic weights to transition from the feature curve to the contour, and uses tangents, normals, and UVs for auxiliary smoothing detection. In the process of transitioning from the feature curve to the contour, the adjustment amplitude is attenuated outward. In other words, in order to achieve smoother adjustment, in practical applications, the change of a feature curve can attenuate a preset number of feature curves outward. The preset number can be set according to actual needs, for example, 6, 8, 10 or other values. The feature curve where the point is located has the largest adjustment amplitude, and the adjacent preset number of feature curves, as the distance from the feature curve where the point is located increases, the adjustment amplitude gradually decreases until the adjacent preset number of feature curves completes the last step of this round of adjustment.

[0091] The dynamic curve constraint of this embodiment ensures that all second controllers on the curve are smooth at all times during operation. Furthermore, the control of the second controller can be transmitted to the surface and point, which ensures the topological uniformity of the target object model. In real-time operation, it allows users to complete special and large-scale operations, and also ensures the standardization of the topology at the modeling level.

[0092] In this embodiment, binding logic is established between the point cloud of the implicit constraint surface and the target object model. These logics are simple but very effective. For example, a binding relationship is established between the first controller on the implicit constraint surface and the point on the implicit constraint surface, and a binding relationship is also established between the second controller on the target object model and the point on the target object model. The action of the point on the implicit constraint surface can be transmitted to the second controller located at the point on the target object model, which is equivalent to a binding relationship between a second controller on the target object model and a point on the implicit constraint surface. Then, the position change of the point on the implicit constraint surface will be affected by the weight and transmitted to the corresponding second controller located at the point. At the same time, the changes in the tangent and normal of the point will also be transmitted to the rotation of the corresponding second controller through the weight, thereby driving the movement of other points bound to the second controller on the target object model.

[0093] Based on the above, it can be understood that, in this embodiment, after the point cloud is updated, more than 85 first controllers established on the implicit constraint surface can be mapped to more than 550 second controllers on the target object model surface through binding relationships, and then further updated to more than 20,000 point clouds on the target object model surface, thereby achieving more precise control and making the generated digital human more vivid and accurate.

[0094] When performing face pinching, this embodiment uses implicit constraint surfaces and dynamic curve constraints, so that the topological effect of the target object model is always constrained during the operation, ensuring the quick use of the product and the universality and standardization of the product output, effectively ensuring the accuracy and standardization of the generated digital human, and effectively improving the efficiency of generating digital humans.

[0095] S210, detecting whether the attachment template of the target object model fits the generated digital human image; if so, obtaining the digital human image; if not, executing step S211;

[0096] S211. Adjust the attachment template in the digital human image so that the attachment template fits the digital human image.

[0097] In this embodiment, in response to the attachment template of the target object model not fitting the generated digital human image, step S211 is executed. If they fit, a perfect digital human image is obtained.

[0098] The attachment template of this embodiment includes at least one of the eyeball, teeth, AO and lacrimal gland. AO refers to the eyeshell, which is referred to as AO in the object model of the digital human. For example, when the eyeball template is too large, it may cause the eyes of the digital human to be out of focus when the eyes are closed. In this case, the eyeball template needs to be reduced and adjusted. When the eyeball template is too small, it may cause a gap between the eyeball template and the image of the digital human when the digital human opens his eyes. In this case, the eyeball template needs to be enlarged and adjusted. Similarly, when the tooth template is too large, it may cause the mouth of the digital human to be not closed tightly when the mouth is closed. In this case, the tooth template needs to be reduced and adjusted. When the tooth template is too small, it may cause the gap between the mouth and the tooth template when the digital human opens his mouth. In this case, the tooth template needs to be enlarged and adjusted. Similarly, the adjustment principles of the AO template and the lacrimal gland template are the same, which will not be repeated here. After the processing of this step, the final digital human image can be made more accurate, more vivid and more perfect.

[0099] As of step S211, a perfect digital human image can be obtained. Since the digital human image has been subjected to the above-mentioned feature fusion and face pinching operation, it can match the features in the picture, and the face pinching adjustment constrained by implicit constraint surfaces and dynamic curves can effectively standardize the topological structure of the generated digital human image, thereby ensuring the accuracy of the generated digital human image.

[0100] It should be noted that the above step S203 is based on the example that multiple key head features included in the picture can be fused. In actual applications, there may be key head features that do not include the target attribute information in the feature library. At this time, the point cloud of the key head features in the target object model can be pinched based on the user's trigger. The method of the face twisting operation refers to the above-mentioned related records and will not be repeated here. Stop pinching the face until the similarity between the key head features in the target object model and the key head features in the picture after the face twisting is greater than or equal to the preset similarity threshold.

[0101] Compared with traditional technologies, the digital human generation method of this embodiment greatly reduces the image production cost. The traditional digital human image production process requires a lot of manpower and takes several months. Using this technical solution, the features of the scanned body can be quickly and reasonably combined, and the face-pinching tool can be used to gradually modify it, greatly shortening the image generation cycle to within a few days.

[0102] Moreover, the digital human generation method of this embodiment can effectively improve the quality of the digital human image. Based on the preset target object model, the digital human image is generated through fusion and face pinching operations. Under the premise that the efficiency of digital human generation is greatly improved, the quality of the generated digital human is still maintained. In addition, the generated digital human image is tested for fit with the attachment template, and adjustments are made when it is not fit, further ensuring the quality of the generated digital human.

[0103] Figure 4 is a schematic diagram according to the third embodiment of the present disclosure; Figure 4 As shown, this embodiment provides a digital human generation platform 400, including:

[0104] The model acquisition module 401 is used to acquire the corresponding target object model based on the image of the digital human to be generated;

[0105] A feature acquisition module 402 is used to acquire a point cloud of the corresponding key head features from a preconfigured feature library based on the key head features in the image;

[0106] The fusion module 403 is used to fuse the point cloud of the key features of the head into the target object model to obtain a digital human image.

[0107] The digital human generation platform 400 of this embodiment realizes the generation of digital human by adopting the above modules, and the realization principle and technical effect are the same as those of the above related method embodiments. For details, please refer to the records of the above related method embodiments, which will not be repeated here.

[0108] Figure 5 is a schematic diagram according to the fourth embodiment of the present disclosure; the digital human generation platform 500 of this embodiment is in the above Figure 4 Based on the embodiments shown, the technical solution of the present disclosure is further described in more detail. First, as Figure 5 As shown, the digital human generation platform 500 of this embodiment includes the above Figure 4 The same modules with the same functions of the digital human generation platform shown are: model acquisition module 501, feature acquisition module 502 and fusion module 503.

[0109] like Figure 5 As shown, the digital human generation platform 500 of this embodiment further includes:

[0110] A first detection module 504 is used to detect whether the similarity between the key features of the head in the fused target object model and the key features of the head in the image is greater than or equal to a preset similarity threshold;

[0111] The face pinching module 505 is used to perform a face pinching operation on the point cloud of the key head features fused to the target object model based on a user trigger in response to the similarity being less than a preset similarity threshold.

[0112] Further optionally, in one embodiment of the present disclosure, the face pinching module 505 is used to:

[0113] With the pre-established implicit constraint surface and the preset dynamic curve as constraints, based on the user's trigger, the face pinching operation is performed on the point cloud of the key features of the head fused to the target object model.

[0114] Further optionally, in one embodiment of the present disclosure, the face pinching module 505 is used to:

[0115] Acquiring action information of a first controller configured on the implicit constraint surface triggered by the user; the implicit constraint surface is a surface that is completely consistent with the surface topology structure of the target object model before being controlled; the implicit constraint surface is invisible to the user;

[0116] Based on a pre-established first motion mapping relationship between a first controller on the implicit constraint surface and a point on the implicit constraint surface, and the action information of the first controller, acquiring action information of a triggered point on the implicit constraint surface;

[0117] Determining action information of a second controller at the same position as the point on the target object model based on action information of the triggered point on the implicit constraint surface;

[0118] Controlling the motion information of the point on the target object model based on a pre-established second motion mapping relationship between a second controller configured on the target object model and a point on the target object model, and the motion information of the second controller;

[0119] The point cloud of the key features of the head fused to the target object model is positionally adjusted using a preset dynamic curve as a constraint.

[0120] Further optionally, in one embodiment of the present disclosure, the face pinching module 505 is used to:

[0121] Detecting whether the second controller is on a preset dynamic curve; configuring at least two of the second controllers on each preset dynamic curve;

[0122] In response to the second controller being on a preset dynamic curve, based on the action information of the second controller, acquiring action information of other second controllers on the dynamic curve;

[0123] Based on the motion information of the second controller and the motion information of other second controllers on the dynamic curve, the point cloud of the key features of the head fused to the target object model is adjusted in position.

[0124] Further optionally, in one embodiment of the present disclosure, the face pinching module 505 is further used to:

[0125] In response to the second controller not being on a preset dynamic curve, a position adjustment is performed on the point cloud of the key features of the head fused to the target object model based on the motion information of the second controller.

[0126] Further optionally, if Figure 5 As shown, in one embodiment of the present disclosure, the digital human generation platform 500 further includes:

[0127] A second detection module 506 is used to detect whether the attachment template of the target object model fits the digital human image;

[0128] The adjustment module 507 is used to adjust the attachment template in the digital human image in response to the attachment template of the target object model being matched with the digital human image.

[0129] Further optionally, in one embodiment of the present disclosure, the model acquisition module 501 is used to:

[0130] Extracting attribute features of the digital person based on the image of the digital person to be generated;

[0131] Based on the attribute characteristics of the digital human, the corresponding target object model is obtained from a preset model library; the model library includes a plurality of object models.

[0132] Further optionally, in one embodiment of the present disclosure, the model acquisition module 501 is used to:

[0133] Based on the image of the digital human to be generated, attribute features of the digital human are not extracted, and a preconfigured standard model is used as a target object model.

[0134] Further optionally, in one embodiment of the present disclosure, the feature acquisition module 502 is used to:

[0135] Obtain target attribute information of key features of the head in the image;

[0136] Based on the target attribute information of the key features of the head, a point cloud of the corresponding key features of the head is obtained from the feature library.

[0137] Further optionally, in one embodiment of the present disclosure, the face pinching module 505 is further used to:

[0138] If the feature library does not include the key head feature corresponding to the target attribute information, based on the user's trigger, a face pinching operation is performed on the point cloud of the key head feature in the target object model.

[0139] Further optionally, if Figure 5 As shown, in one embodiment of the present disclosure, the digital human generation platform 500 further includes:

[0140] The acquisition module 508 is used to acquire point clouds of multiple key head features of each of the multiple characters and attribute information of each key head feature, and store them in the feature library.

[0141] Further optionally, in one embodiment of the present disclosure, the fusion module 503 is used to:

[0142] Registering the point cloud of the key features of the head with the target object model;

[0143] The point cloud of the key features of the head is migrated to a region of the key features of the head corresponding to the target object model.

[0144] The digital human generation platform 500 of this embodiment realizes the generation of digital human by adopting the above modules, and the realization principle and technical effect are the same as those of the above related method embodiments. For details, please refer to the records of the above related method embodiments, which will not be repeated here.

[0145] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0146] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0147] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0148] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0149] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the above-mentioned methods of the present disclosure. For example, in some embodiments, the above-mentioned methods of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the above-mentioned methods of the present disclosure described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the above-mentioned methods of the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0150] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0152] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0154] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0155] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0156] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0157] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a digital human, comprising: Based on the image of the digital human to be generated, obtain the corresponding target object model; Based on the key features of the head in the image, obtaining a point cloud of the corresponding key features of the head from a preconfigured feature library; Fusion of the point cloud of the key features of the head into the target object model to obtain a digital human image; The method further comprises: Detecting whether the similarity between the key features of the head in the fused target object model and the key features of the head in the image is greater than or equal to a preset similarity threshold; In response to the similarity being less than a preset similarity threshold, based on a user trigger, performing a face pinching operation on the point cloud of the key head features fused to the target object model; Wherein, based on the user's trigger, performing a face pinching operation on the point cloud of the key features of the head fused to the target object model includes: With the pre-established implicit constraint surface and the preset dynamic curve as constraints, based on the user's trigger, the face pinching operation is performed on the point cloud of the key features of the head fused to the target object model.

2. The method according to claim 1, wherein: Using the pre-established implicit constraint surface and the preset dynamic curve as constraints, based on the user's trigger, a face pinching operation is performed on the point cloud of the key features of the head fused to the target object model, including: Acquiring action information of a first controller configured on the implicit constraint surface triggered by the user; the implicit constraint surface is a surface that is completely consistent with the surface topology structure of the target object model before being controlled; the implicit constraint surface is invisible to the user; Based on a pre-established first motion mapping relationship between a first controller on the implicit constraint surface and a point on the implicit constraint surface, and the action information of the first controller, acquiring the action information of the triggered point on the implicit constraint surface; Determining action information of a second controller at the same position as the point on the target object model based on action information of the triggered point on the implicit constraint surface; Controlling the motion information of the point on the target object model based on a pre-established second motion mapping relationship between a second controller configured on the target object model and a point on the target object model, and the motion information of the second controller; The point cloud of the key features of the head fused to the target object model is positionally adjusted using a preset dynamic curve as a constraint.

3. The method according to claim 2, wherein: Using a preset dynamic curve as a constraint, position adjustment is performed on the point cloud of the key features of the head fused to the target object model, including: Detecting whether the second controller is on a preset dynamic curve; configuring at least two of the second controllers on each preset dynamic curve; In response to the second controller being on a preset dynamic curve, based on the action information of the second controller, acquiring action information of other second controllers on the dynamic curve; Based on the motion information of the second controller and the motion information of other second controllers on the dynamic curve, the point cloud of the key features of the head fused to the target object model is adjusted in position.

4. The method according to claim 3, wherein: The method further comprises: adjusting the position of the point cloud of the key features of the head fused to the target object model using a preset dynamic curve as a constraint, and further comprising: In response to the second controller not being on a preset dynamic curve, a position adjustment is performed on the point cloud of the key features of the head fused to the target object model based on the motion information of the second controller.

5. The method according to any one of claims 2 to 4, wherein: After performing a face pinching operation on the point cloud of the key features of the head fused to the target object model based on a user trigger, the method further includes: Detecting whether the attachment template of the target object model fits the digital human image; In response to the attachment template of the target object model not fitting the digital human image, the attachment template in the digital human image is adjusted.

6. The method according to any one of claims 1 to 4, wherein: Based on the image of the digital human to be generated, the corresponding target object model is obtained, including: Extracting attribute features of the digital person based on the image of the digital person to be generated; Based on the attribute characteristics of the digital human, the corresponding target object model is obtained from a preset model library; the model library includes a plurality of object models.

7. The method according to claim 6, wherein: Based on the image of the digital human to be generated, the corresponding target object model is obtained, including: Based on the image of the digital human to be generated, attribute features of the digital human are not extracted, and a preconfigured standard model is used as a target object model.

8. The method according to any one of claims 1 to 4, wherein: Based on the key features of the head in the image, a point cloud of the corresponding key features of the head is obtained from a preconfigured feature library, including: Obtain target attribute information of key features of the head in the image; Based on the target attribute information of the key features of the head, a point cloud of the corresponding key features of the head is obtained from the feature library.

9. The method according to claim 8, wherein: The method further comprises: If the feature library does not include the key head feature corresponding to the target attribute information, based on the user's trigger, a face pinching operation is performed on the point cloud of the key head feature in the target object model.

10. The method according to any one of claims 1 to 4, wherein: Based on the key features of the head in the image, before obtaining the point cloud of the corresponding key features of the head from a preconfigured feature library, the method further includes: Point clouds of multiple key head features of each of the multiple characters and attribute information of each key head feature are collected and stored in the feature library.

11. The method according to any one of claims 1 to 4, wherein: The step of fusing the point cloud of the key features of the head into the target object model includes: Registering the point cloud of the key features of the head with the target object model; The point cloud of the key features of the head is migrated to a region of the key features of the head corresponding to the target object model.

12. A digital human generation platform, comprising: A model acquisition module, used to acquire a corresponding target object model based on the image of the digital human to be generated; A feature acquisition module, used to acquire a point cloud of the corresponding key features of the head from a preconfigured feature library based on the key features of the head in the image; A fusion module, used for fusing the point cloud of the key features of the head into the target object model to obtain a digital human image; The platform further includes: A first detection module is used to detect whether the similarity between the key features of the head in the fused target object model and the key features of the head in the image is greater than or equal to a preset similarity threshold; A face pinching module, configured to, in response to the similarity being less than a preset similarity threshold, perform a face pinching operation on the point cloud of the key features of the head fused to the target object model based on a user trigger; Wherein, the face pinching module is used to: With the pre-established implicit constraint surface and the preset dynamic curve as constraints, based on the user's trigger, the face pinching operation is performed on the point cloud of the key features of the head fused to the target object model.

13. The platform according to claim 12, wherein: The face pinching module is used to: Acquiring action information of a first controller configured on the implicit constraint surface triggered by the user; the implicit constraint surface is a surface that is completely consistent with the surface topology structure of the target object model before being controlled; the implicit constraint surface is invisible to the user; Based on a pre-established first motion mapping relationship between a first controller on the implicit constraint surface and a point on the implicit constraint surface, and the action information of the first controller, acquiring the action information of the triggered point on the implicit constraint surface; Determining action information of a second controller at the same position as the point on the target object model based on action information of the triggered point on the implicit constraint surface; Controlling the motion information of the point on the target object model based on a pre-established second motion mapping relationship between a second controller configured on the target object model and a point on the target object model, and the motion information of the second controller; The point cloud of the key features of the head fused to the target object model is positionally adjusted using a preset dynamic curve as a constraint.

14. The platform according to claim 13, wherein: The face pinching module is used to: Detecting whether the second controller is on a preset dynamic curve; configuring at least two of the second controllers on each preset dynamic curve; In response to the second controller being on a preset dynamic curve, based on the action information of the second controller, acquiring action information of other second controllers on the dynamic curve; Based on the motion information of the second controller and the motion information of other second controllers on the dynamic curve, the point cloud of the key features of the head fused to the target object model is adjusted in position.

15. The platform according to claim 14, wherein: The face pinching module is also used for: In response to the second controller not being on a preset dynamic curve, a position adjustment is performed on the point cloud of the key features of the head fused to the target object model based on the motion information of the second controller.

16. The platform according to any one of claims 12 to 15, wherein: The platform also includes: A second detection module is used to detect whether the attachment template of the target object model fits the digital human image; The adjustment module is used to adjust the attachment template in the digital human image in response to the attachment template of the target object model not fitting the digital human image.

17. The platform according to any one of claims 12 to 15, wherein: The model acquisition module is used to: Extracting attribute features of the digital person based on the image of the digital person to be generated; Based on the attribute characteristics of the digital human, the corresponding target object model is obtained from a preset model library; the model library includes a plurality of object models.

18. The platform according to claim 17, wherein: The model acquisition module is used to: Based on the image of the digital human to be generated, attribute features of the digital human are not extracted, and a preconfigured standard model is used as a target object model.

19. The platform according to any one of claims 12 to 15, wherein: The feature acquisition module is used to: Obtain target attribute information of key features of the head in the image; Based on the target attribute information of the key features of the head, a point cloud of the corresponding key features of the head is obtained from the feature library.

20. The platform according to claim 19, wherein: The face pinching module is also used for: If the feature library does not include the key head feature corresponding to the target attribute information, based on the user's trigger, a face pinching operation is performed on the point cloud of the key head feature in the target object model.

21. The platform according to any one of claims 12 to 15, wherein: The platform also includes: The acquisition module is used to acquire point clouds of multiple key head features of each of the multiple characters and attribute information of each key head feature, and store them in the feature library.

22. The platform according to any one of claims 12 to 15, wherein: The fusion module is used to: Registering the point cloud of the key features of the head with the target object model; The point cloud of the key features of the head is migrated to a region of the key features of the head corresponding to the target object model.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

25. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images