Digital human generation method, platform, electronic device, and storage medium

Abstract

To provide a method for generating a digital human applicable to scenes such as a virtual digital human and a metaverse which can automatically produce a digital human, acquire a digital human image, and effectively save the production cost and cycle of the digital human image, a platform, an electronic device, and a storage medium.SOLUTION: A digital human generation method includes the steps of: acquiring a corresponding target object model on the basis of an image of the digital human to be generated; acquiring a point group of a corresponding head key feature from a preset feature library on the basis of the head key feature in the image; and acquiring a digital human image by fusing the point group of the head key feature into the target object model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to computer technology and artificial intelligence technology fields, particularly to technical fields such as augmented reality, virtual reality, computer vision, and deep learning, and can be applied to scenes such as meta-universes and virtual digital humans. In particular, the present disclosure relates to a method, platform, electronic device, and storage medium for generating a digital human. [Background technology]

[0002] In the traditional production of digital human images, especially in the production of 3D ultra-realistic digital human images, highly specialized input is required in multiple stages, such as original drawing, modeling, binding, and animation. The manual production cycle usually lasts for several months, the input costs are extremely high, and the effects can only be displayed in stages. Operations such as correction and optimization all require a large amount of additional input.

[0003] In addition, in the conventional technology, it is difficult to restore the ideal features of a real-world person, such as the shape of the eyes and lips, in an artificially created digital human image, and the final effect is largely limited by the individual skill level at each stage of the digital human production. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a method, platform, electronic device, and storage medium for generating a digital human. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided a method for generating a digital human, comprising: Obtaining a corresponding target object model based on an image of the digital human to be generated; obtaining a point cloud of corresponding head key features from a predefined feature library based on the head key features in the image; and fusing the point cloud of the head key features with the target object model to obtain a digital human image.

[0006] According to another aspect of the present disclosure, there is provided a digital human generation platform, comprising: a model acquisition module for acquiring a corresponding target object model based on an image of the digital human to be generated; a feature acquisition module for acquiring a point cloud of corresponding head key features from a predefined feature library based on the head key features in the image; and a fusion module for fusing the point cloud of the head key features with the target object model to obtain a digital human image.

[0007] According to another aspect of the present disclosure, there is provided an electronic device, comprising: At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the methods of the aspects and any possible implementations.

[0008] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing the computer to perform the method of the aspect and any possible implementation thereof.

[0009] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of the aspect and any possible implementation manner.

[0010] The technology disclosed in the present disclosure can automatically realize the creation of a digital human and obtain a digital human image, overcome the problems of the prior art that involve multiple manual steps to create a digital human, such as high cost and poor accuracy, effectively save the cost and time required to create a digital human image, and also effectively improve the accuracy of the generated digital human image and the generation efficiency of the digital human image.

[0011] It should be understood that the contents described herein are not intended to identify key or important features of the embodiments of the present disclosure, nor should they be used to limit the scope of the present disclosure. Other features of the present disclosure can be readily understood through the following specification. [Brief description of the drawings]

[0012] The drawings are for a better understanding of the present application and are not intended to limit the present application. [Figure 1] FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure. [Diagram 3] FIG. 13 is a schematic diagram of an interface display of attribute information of head key features provided by the present disclosure. [Figure 4] FIG. 11 is a schematic diagram according to a third embodiment of the present disclosure. [Diagram 5] FIG. 11 is a schematic diagram according to a fourth embodiment of the present disclosure. [Figure 6] FIG. 1 is a block diagram of an electronic device for implementing a method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, exemplary embodiments of the present application will be described based on the drawings. For ease of understanding, various details of the embodiments of the present application are included, and they should be regarded as merely examples. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of simplicity, the following description omits the description of well-known functions and structures.

[0014] Apparently, the described embodiments are some embodiments of the present application, but not all embodiments. According to the embodiments of the present application, all other embodiments that a person skilled in the art can obtain without creative labor all belong to the scope of protection of the present application.

[0015] In addition, the terminal device according to the embodiment of the present disclosure may include, but is not limited to, a mobile phone, a personal digital assistant (PDA (registered trademark)), a wireless handheld device, a tablet computer, and other smart devices. The display device may include, but is not limited to, a personal computer, a television, and other devices having a display function.

[0016] Additionally, the term "and / or" in this specification describes only the relation between related objects and indicates that three types of relations can exist, for example, A and / or B can indicate three cases: only A exists, A and B exist simultaneously, or only B exists. It should be understood that the sign " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0017] The digital human production in the prior art is basically completed manually, which not only results in high production costs, but also in low accuracy and poor effect of the produced digital human.

[0018] 1 is a schematic diagram according to a first embodiment of the present disclosure. As shown in FIG. 1, the present embodiment provides a method for generating a digital human, which specifically includes the following steps: S101, obtaining a corresponding target object model according to an image of a digital human to be generated; S102, based on the head key features in the image, obtain a point cloud of corresponding head key features from a pre-defined feature library; S103, the point cloud of head key features is fused with the target object model to obtain a digital human image.

[0019] The execution body of the digital human generation method of this embodiment can be a digital human generation platform, which can be a platform integrated with software to automatically realize the generation of the digital human, or can configure a corresponding electronic device based on the platform to realize the generation of the digital human.

[0020] In this embodiment, a corresponding digital human image can be automatically generated based on the image of the digital human to be generated, and the image of the digital human to be generated is a two-dimensional (2D) image, or the digital human image can be a 3D digital human image.

[0021] In the generated digital human image, the features of the digital human are mainly concentrated in the head and facial area of ​​the head, so in this embodiment, the image of the digital human to be generated may be a frontal image including a person's head, for example, only a head image, or an upper body image including the person's head features.

[0022] The target object model in this embodiment can be a prototype of a pre-created digital human model. In order to improve the efficiency of digital human generation, this embodiment does not need to create a digital human model prototype, and directly generates a corresponding digital human image based on the obtained target object model.

[0023] Since the features of a digital human are mainly concentrated in the head, in this embodiment, for each head key feature in the image, a point cloud of the corresponding head key feature is obtained from the feature library, and then merged into the target object model, so as to finally obtain a digital human image.

[0024] In this embodiment, the head key features may be limited parts according to actual needs, including at least one feature such as eyebrows, eyes, nose, mouth, and face shape.

[0025] The digital human generation method of this embodiment, through the above-mentioned technical solution, can automatically realize the production of digital humans and obtain digital human images, overcome the problems of manual, multi-step digital human production in the prior art, high cost, poor accuracy, etc., effectively save the cost and period of digital human image production, and effectively improve the accuracy of the generated digital human image and the generation efficiency of the digital human image, so that the generated digital human image has a hyper-realistic expression effect, and at the same time, effectively lower the entry requirements for digital human production and promote the progress of the digital industry.

[0026] 2 is a schematic diagram according to a second embodiment of the present disclosure. As shown in FIG. 2, the digital human generating method of this embodiment is based on the technical solution of the embodiment shown in FIG. 1 above, and the technical solution of this disclosure is further described in detail. As shown in FIG. 2, the digital human generating method of this embodiment can specifically include the following steps: S201, extracting attribute features of the digital human according to an image of the digital human to be generated; Since the main features of a digital human are all concentrated in the head, the image of the digital human to be generated in this embodiment may be a frontal image including the head, which can clearly show each key head feature area in the front of the person's head.

[0027] For example, in this embodiment, a pre-trained attribute feature extraction model can be used to realize the extraction of attribute features of a digital human. The attribute feature extraction model may be a pre-trained neural network model, and when used, an image of a digital human is input to the attribute feature extraction model, and the attribute feature extraction model can extract attribute features of the person in the image based on the image. For example, the attribute features in this embodiment can include middle-aged women, middle-aged men, girls, boys, elderly men, or elderly women, etc.

[0028] Alternatively, in this embodiment, other algorithms can be used to analyze the image of the digital human to be generated to obtain the attribute features of the digital human, which is not limited here.

[0029] S202, obtaining a corresponding target object model from a pre-defined model library according to the attribute features of the digital human; Since people with different attribute features have different facial region features, in order to generate digital human images more accurately, in this embodiment, a model library can also be preset, and a plurality of object models can also be preset in the model library, such as at least one of a middle-aged female model, a middle-aged male model, a girl model, a boy model, an elderly male model, and an elderly female model. For each object model, a corresponding attribute feature can be set when presetting. Therefore, according to the attribute features of the digital human, a corresponding target object model can be obtained from the preset model library, which can directly save the processes of original drawing and modeling, compared with the prior art, and can directly use the preset matching target object model, which can effectively improve the production efficiency of the digital human image.

[0030] Optionally, in one embodiment of the present disclosure, if the attribute features of the digital human are not extracted based on the image of the digital human to be generated, then a preset standard model can be taken as the target object model. The standard model can be one model in a preset model library, such as a middle-aged female model. Or another individual standard model can be preset and used when the attribute features cannot be extracted, to ensure that a reasonable target object model can be obtained in any case.

[0031] S203, based on the plurality of head key features included in the image, detect whether there is an unfused head key feature, if there is, execute step S204, if not, execute step S210; Specifically, since an image may contain multiple head key features, in order to facilitate one-by-one fusion into the target object model, one head key feature can be fused and then labeled each time to avoid repeated operations.

[0032] S204, obtain target attribute information of one unfused head key feature in the image, and execute step S205; In this embodiment, the head key feature may be eyebrows, eyes, nose, mouth, ears, or face shape, etc. Corresponding target attribute information can be used to define the head key feature, for example, eye attribute information can include apricot eyes, red phoenix eyes, hawk eyes, elongated eyes, narrow eyes, etc.

[0033] In this embodiment, an attribute extraction model of each head key feature is pre-trained to extract target attribute information of the head key feature, or the target attribute information of the head key feature in the image selected by the user based on the image can be obtained.

[0034] S205, according to the target attribute information of the head key feature, obtain the point cloud of the corresponding head key feature from the pre-defined feature library, and execute step S206; Before step S205, a step of collecting point clouds of a plurality of head key features of each person among the plurality of people and attribute information of each head key feature and storing them in a feature library may be included.

[0035] In this embodiment, the collected multiple people include people with different attribute information, such as middle-aged women, middle-aged men, girls, boys, elderly men, or elderly women. In addition, the attribute information of the same head key features among the multiple people should be as different as possible, so that the feature library should contain rich head key features with various types of attribute information, so as to provide effective feature data support for digital human generation. For example, for eyebrows, collect eyebrow point clouds and corresponding eyebrow shape information of people with as different eyebrow shapes as possible; for eyes, collect eye point clouds and corresponding eye shape information of people with as different eyebrow shapes as possible; for mouths, collect mouth point clouds and corresponding mouth shape information of people with as different mouth shapes as possible, etc. Based on the above-mentioned established feature library, when step S205 is specifically implemented, the digital human generation platform can automatically accurately and efficiently match the point cloud of head key features corresponding to the target attribute information based on the target attribute information of the head key features and all the target attribute information of the head key features in the feature library.

[0036] For example, Fig. 3 is a schematic diagram of an interface display of attribute information of head key features provided by the present disclosure. As shown in Fig. 3, in a digital human generation platform, the head key feature is taken as an example of nose, and the various nose shapes shown in Fig. 3 are nose shapes supported and used in the feature library in the digital human generation platform. When a user clicks on a nose shape, the interface shown in Fig. 3 can be displayed, and the user can select one best matching nose shape based on the nose shape in the image of the digital human to be generated, and the user can also reset the currently selected nose shape to 0 and reselect one nose shape.

[0037] S206, perform registration between the point cloud of the head key features and the target object model, and then execute step S207; The registration in this embodiment can be adjusted by at least one operation manner of displacement, rotation, scaling, etc. For example, specifically, by adjusting the point cloud of the head key feature, the coordinate system of the point cloud of the head key feature is made to match with the target object model, the size of the point cloud of the head key feature is made to match with the area size of the head key feature of the target object model, and the point cloud of the head key feature can be accurately and efficiently matched with the target object model.

[0038] S207, transferring the point cloud of the head key feature to the region of the corresponding head key feature in the target object model, and executing step S208; S208, detect whether the similarity between the head key feature in the processed target object model and the head key feature in the image is equal to or greater than a preset similarity threshold, if so, return to step S203, if not, execute step S209; For example, specifically, the head key feature in the front view of the fused target object model and the head key feature in the image can be extracted, and then a similarity between the two can be calculated using one pre-trained similarity calculation model. Or, the feature representation of the head key feature region in the front view of the fused target object model and the feature representation of the head key feature in the image can be extracted, and then the similarity between the two feature representations can be calculated as the similarity between the two.

[0039] Based on this step, it can be seen that in practical applications, there may be cases where an ideal effect is obtained after combining the head key features, and in this case, the avatar creation operation is not performed for the head key features.

[0040] S209, based on a user's trigger, perform an avatar creation operation on the point cloud of the head key features fused to the target object model, and return to step S208; The avatar creation operation of this embodiment uses the pre-established implicit constraint surface and the pre-set dynamic curve as constraints, and performs the avatar creation operation on the point cloud of the head key features merged with the target object model based on the user's trigger. Specifically, the avatar creation operation can include the following steps: (1) acquiring motion information of a first controller set on an implicit constraint surface triggered by a user; Before adjusting the first controller, the implicit constraint surface is a surface that matches the topological structure of the surface of the target object model, and the identifiers and positions of the included points are completely the same, and multiple first controllers are set on the implicit constraint surface, but the implicit constraint surface is invisible to the user, and a first motion mapping relationship that exists between each first controller and the points on the implicit constraint surface can drive the movement of the points on the implicit constraint surface when the first controller is operated.

[0041] In the embodiment of the present disclosure, since the main features of the digital human image are concentrated on the head, the fusion and avatar creation operation is also mainly performed on the head key features of the target object model. At this time, the implicit constraint surface and the topology structure of the target object model can be limited to only the head of the digital human. In this embodiment, the motion information of the first controller can include the direction and displacement of the first controller movement.

[0042] (2) obtaining motion information of the triggered point on the implicit constraint surface based on a first motion mapping relationship between a first controller on the pre-established implicit constraint surface and a point on the implicit constraint surface, and motion information of the first controller; Alternatively, in this embodiment, 85 or more usable first controllers can be designed on the implicit constraint surface based on facial features, muscle flow, and skeletal root position. There can be 20,000 or more point clouds in the target object model. Therefore, one first controller can trigger the movement of multiple points on the target object model. When a user triggers a first controller, only one first controller movement can be triggered, and multiple first controller movements can also be triggered simultaneously. For example, the control interface at this time can first instruct the user to select the first controller to be triggered, and after the determination, the user can trigger movements for all the selected first controllers simultaneously. The motion information of the point in this embodiment can also include the motion direction and displacement of the point.

[0043] Specifically, the motion information of each point is determined by a weighted addition method based on the motion information of each first controller among the multiple first controllers that triggers the point and the weight of each first controller, and the final motion information of the point is determined, and further the motion position of the point is determined.

[0044] The main purpose of the implicit constraint surface in this embodiment is to resolve unreasonable situations caused by large shape changes, such as the change of the skull top, which is transmitted to the forehead, sides, back of the head and neck through this constraint, thereby ensuring the physics and aesthetics of the target object model to a certain extent, and effectively improving the aesthetics of the generated digital human.

[0045] (3) determining motion information of a second controller that is in the same position as the triggered point on the target object model based on the motion information of the triggered point on the implicit constraint surface; In this embodiment, in the target object model, some second controllers are deployed at the points of the target object model. In the process of control, after a point on the implicit constraint surface is triggered to move, the second controller deployed at the point of the same identifier in the target object model at this time will perform synchronous operation according to the movement of the point on the implicit constraint surface.

[0046] (4) controlling the motion information of the point on the target object model based on a second motion mapping relationship between the second controller and the point on the target object model that is set in advance for the target object model and the motion information of the second controller; In this embodiment, more than 550 second controllers are set on the target object model. Similarly, each second controller also establishes a binding relationship with a plurality of points on the target object model, for example, establishes the above-mentioned second motion mapping relationship. Compared with the number of first controllers, the number of second controllers is more, and can realize finer control of the points on the target object model.

[0047] Based on step (3), it can be seen that after the motion of the triggered point on the implicit constraint surface, the second controller deployed on the point with the same identifier on the target object model performs a synchronous motion according to the motion of the point, and further, based on the second motion mapping relationship between the second controller set on the target object model and the point on the target object model, can obtain the motion information of all points bound by the second controller on the target object model, and further, based on the motion information of the points on the target object model, control the motion of the corresponding points. In this process, the motion of the first controller on the implicit constraint surface is transferred to the motion of the points on the target object model to realize accurate adjustment to the target object model.

[0048] (5) Using a preset dynamic curve as a constraint, a position adjustment is performed on the point cloud of the head key features fused to the target object model.

[0049] For example, when implementing step (5), the following steps may be included: (a) detecting whether a second controller is on a preset dynamic curve, and setting at least two second controllers for each preset dynamic curve; The preset dynamic curves in this embodiment may be some natural curves of the face of the target object model, such as the jaw line, the mandibular line, or the nasal crest. In the avatar creation optimization process, when adjusting the jaw line, the mandibular line, the nasal crest, and other areas, a single controller is difficult to achieve a continuous, smooth, and natural topology effect. Therefore, in this embodiment, a dynamic curve is introduced, so that the dynamic curve can be made more continuous, smooth, and natural by controlling the movement of at least two second controllers on the dynamic curve.

[0050] (b) in response to the second controller being on the preset dynamic curve, obtaining operation information of another second controller on the dynamic curve based on the operation information of the second controller; (c) performing position adjustment on the point cloud of head key features fused to the target object model based on the motion information of the second controller and the motion information of another second controller on the dynamic curve;

[0051] Furthermore, the method may further include a step of, in response to the second controller not being on a preset dynamic curve, performing a position adjustment on the point cloud of the head key features fused to the target object model based on the motion information of the second controller.

[0052] Based on the above, it can be understood that the core of dynamic curve constraint is that multiple consecutive, same-level secondary controllers are connected in series using one space curve. When the end points are fixed, the position and rotation changes of each secondary controller are all transmitted to its surroundings, and the closer the secondary controller is, the higher the weight is, and conversely, the farther away the secondary controller is, the lower the weight is for the secondary controller closer to the end point. The relationship between weight and distance is also preset.

[0053] In this embodiment, when performing an avatar creation operation on the point cloud of the head key feature fused to the target object model based on the motion information of the second controller and the motion information of another second controller in the dynamic curve, a smoothing process is further performed on the points around the adjusted points. In this embodiment, the feature curve on which the points are located is adjusted as a constraint.

[0054] For example, the surface of the target object model itself is a grid structure that includes edge configurations between multiple points and adjacent points. When adjusting a point, at the topology level, the feature curve on which the point is located can be the center, and its adjacent topology lines also need to be updated at a certain level to ensure the uniformity and smoothness of the topology. This process mainly uses dynamic weights to transition from the feature curve to the outline, and simultaneously uses tangents, normals, and UVs to perform auxiliary smooth detection. In the process of transitioning from the feature curve to the outline, the adjustment amplitude is attenuated outward. That is, in order to achieve a smoother adjustment, in actual applications, the change of one feature curve can be attenuated outward by a preset number of feature curves. The preset number can be set according to actual needs, and may be, for example, 6, 8, 10, or other values. The adjustment amplitude of the feature curve on which the point is located is the largest, and with the increase in the distance of each feature curve in the adjacent preset number of feature curves, the adjustment amplitude is gradually decreased until the th feature curve of the adjacent preset number completes the adjustment of the last step of this wheel adjustment.

[0055] The dynamic curve constraint of this embodiment ensures that all the second controller's current operations on the curve are always smooth. Furthermore, the control of the second controller can be transmitted to faces and points, ensuring the uniformity of the topology of the target object model. In real-time operation, it allows the user to complete specific and large-range operations, and ensures the modeling level and topology regularity.

[0056] In this embodiment, the points of the implicit constraint surface establish binding logic with the target object model, and these logics are simple but very effective. For example, a binding relationship is established between a first controller on the implicit constraint surface and a point on the implicit constraint surface, and a binding relationship is also established between a second controller on the target object model and a point on the target object model, and the motion of the point on the implicit constraint surface can be transmitted to the second controller located at that point on the target object model, and it corresponds to a binding relationship also exists between a second controller on the target object model and a point on the implicit constraint surface, so that the position change of the point on the implicit constraint surface is transmitted to the second controller located at the corresponding point under the influence of weight. At the same time, the tangent and normal changes of the point are also transmitted to the rotation of the corresponding second controller through weight, which further causes the motion of other points bound by the second controller on the target object model.

[0057] Based on the above, it can be understood that in this embodiment, the 85 or more first controllers established on the implicit constraint surface can be mapped to the 550 or more second controllers in the target object model through binding relationships after updating the point cloud, and further updated to the 20,000 or more point cloud in the target object model, so as to realize finer control, and make the generated digital human more realistic and accurate.

[0058] In this embodiment, when creating the avatar, by using the implicit constraint surface and dynamic curve constraint, the topology effect of the target object model can be constantly constrained during operation, ensuring the speed of product use and the versatility and standardity of product output, effectively ensuring the accuracy and standardity of the generated digital human, and effectively improving the efficiency of digital human generation.

[0059] S210, detecting whether the attached template of the target object model is suitable for the generated digital human image, and if so, obtaining the digital human image; if not, executing step S211; S211, adjusting the attached template in the digital human image so that the added template is suitable for the digital human image.

[0060] In this embodiment, in response to the attached template of the target object model not being suitable for the generated digital human image, step S211 is performed, otherwise, a perfect digital human image is obtained.

[0061] The attached template in this embodiment includes at least one of an eyeball, a tooth, an AO, and a lacrimal gland. AO refers to an eyeshell, and is abbreviated as AO in the object model of a digital human. For example, if the eyeball template is too large, the digital human may not be able to close his eyes when he closes them, and in this case, a reduction adjustment needs to be made to the eyeball template. If the eyeball template is too small, there may be a gap between the eyeball template and the digital human image when the digital human opens his eyes, and in this case, a magnification adjustment needs to be made to the eyeball template. Similarly, if the tooth template is too large, the digital human may not be able to close his mouth when he closes his mouth, and in this case, a reduction adjustment needs to be made to the tooth template. If the tooth template is too small, there may be too much of a gap between the mouth and the tooth template when the digital human opens his mouth, and in this case, a magnification adjustment needs to be made to the tooth template. Similarly, the adjustment principles of the AO template and the lacrimal gland template are the same, and will not be described in detail here. Through the processing steps, the final captured digital human image can be more accurate, more representative and more complete.

[0062] By step S211, a perfect digital human image can be obtained, which can be matched with features in the image through the above-mentioned feature fusion and avatar creation operations, and the topology structure of the generated digital human image can be effectively standardized through the avatar creation adjustment of the implicit constraint surface and dynamic curve constraint, so as to ensure the accuracy of the generated digital human image.

[0063] It should be noted that the above step S203 is taken as an example where multiple head key features contained in an image can all be merged, and in actual applications, the feature library may not contain a head key feature corresponding to the target attribute information, and in this case, an avatar creation operation can be performed on the point cloud of the head key feature in the target object model based on a user's trigger. The avatar creation operation method is referred to the above related description and will not be described in detail here. After the avatar is created, the avatar creation is stopped until the similarity between the head key feature in the target object model and the head key feature in the image is equal to or greater than a preset similarity threshold.

[0064] The digital human generation method of this embodiment greatly reduces the image production cost compared with the prior art, since the traditional digital human image production process requires a large amount of manpower and takes several months, but by using this technical solution, the features of the scanned body can be quickly and reasonably combined, and the avatar creation tool can be used to make step-by-step corrections, greatly shortening the image generation cycle to within a few days.

[0065] In addition, the digital human generation method of this embodiment can effectively improve the quality of the digital human image. Based on the preset target object model, a digital human image is generated through fusion and avatar creation operations, and the efficiency of digital human generation is greatly improved while still maintaining the quality of the generated digital human. In addition, the generated digital human image is subjected to suitability detection with the attached template, and if not, adjustment is made to further ensure the quality of the generated digital human.

[0066] 4 is a schematic diagram according to a third embodiment of the present disclosure. As shown in FIG. 4, this embodiment provides a digital human generation platform 400, which includes a model acquisition module 401, a feature acquisition module 402, and a fusion module 403. The model acquisition module 401 is used for acquiring a corresponding target object model based on an image of a digital human to be generated; The feature acquisition module 402 is used to acquire a point cloud of corresponding head key features from a pre-defined feature library according to the head key features in the image; A fusion module 403 is used to fuse the point cloud of head key features to the target object model to obtain a digital human image.

[0067] The digital human generation platform 400 of this embodiment is the same as the implementation of the above-mentioned related method embodiment by using the above-mentioned modules to realize the implementation principle and technical effect of digital human generation, for which details may refer to the description of the above-mentioned related method embodiment and will not be described in detail here.

[0068] 5 is a schematic diagram according to a fourth embodiment of the present disclosure. The digital human generation platform 500 of this embodiment will be described in more detail based on the embodiment shown in FIG. 4. First, as shown in FIG. 5, the digital human generation platform 500 of this embodiment includes the same modules with the same functions as the digital human generation platform shown in FIG. 4: a model acquisition module 501, a feature acquisition module 502, and a fusion module 503.

[0069] As shown in FIG. 5 , the digital human generation platform 500 of this embodiment further includes a first detection module 504 and an avatar creation module 505; a first detection module 504 for detecting whether a similarity between a head key feature in the target object model after fusion and a head key feature in the image is equal to or greater than a preset similarity threshold; The avatar creation module 505 is used to perform an avatar creation operation on the point cloud of the head key features fused to the target object model based on a user trigger in response to the similarity being less than a pre-set similarity threshold.

[0070] Further optionally, in one embodiment of the present disclosure, the avatar creation module 505 includes: It is used to perform an avatar creation operation on the point cloud of the head key features fused to the target object model based on a user trigger, with pre-established implicit constraint surfaces and pre-set dynamic curves as constraints.

[0071] Further optionally, in one embodiment of the present disclosure, the avatar creation module 505 includes: Obtain operation information of a first controller set on the implicit constraint surface triggered by the user, the implicit constraint surface being a surface that completely matches a surface topological structure of the target object model before being controlled, the implicit constraint surface being invisible to the user; Obtain motion information of a triggered point on the implicit constraint surface based on a first motion mapping relationship between a first controller on the implicit constraint surface and a point on the implicit constraint surface that is pre-established, and motion information of the first controller; determining motion information of a second controller that is co-located with the point on the target object model based on motion information of the triggered point on the implicit constraint surface; Controlling the motion information of the point on the target object model based on a second motion mapping relationship between a second controller and the point on the target object model that is set in advance for the target object model and the motion information of the second controller; A predefined dynamic curve is used as a constraint to perform position adjustment on the point cloud of the head key features merged with the target object model.

[0072] Further optionally, in one embodiment of the present disclosure, the avatar creation module 505 includes: Detect whether the second controller is on a preset dynamic curve, and set at least two second controllers for each preset dynamic curve; In response to the second controller being on a preset dynamic curve, acquiring operation information of another second controller on the dynamic curve based on operation information of the second controller; It is used to perform position adjustment on the point cloud of the head key features fused to the target object model based on the motion information of the second controller and the motion information of another second controller in the dynamic curve.

[0073] Further optionally, in one embodiment of the present disclosure, the avatar creation module 505 further comprises: In response to the second controller not being on a preset dynamic curve, a position adjustment is made to the point cloud of the head key features fused to the target object model based on the motion information of the second controller.

[0074] Optionally, as shown in FIG. 5 , in an embodiment of the present disclosure, the digital human generation platform 500 further includes a second detection module 506 and an adjustment module 507; A second detection module 506 is used to detect whether the attached template of the target object model is suitable for the digital human image; The adjustment module 507 is used to adjust the attached template in the digital human image in response to the attached template of the target object model being compatible with the digital human image.

[0075] Further optionally, in one embodiment of the present disclosure, the model acquisition module 501 includes: Extracting attribute features of the digital human based on the image of the digital human to be generated; Based on the attribute features of the digital human, a corresponding target object model is obtained from a pre-defined model library, where the model library includes a plurality of object models.

[0076] Further optionally, in one embodiment of the present disclosure, the model acquisition module 501 includes: Based on the image of the digital human to be generated, attribute features of the digital human are not extracted, but are used to take a pre-defined standard model as a target object model.

[0077] Further optionally, in one embodiment of the present disclosure, the feature acquisition module 502 includes: obtaining target attribute information of head key features in the image; It is used to obtain a point cloud of the corresponding head key feature from the feature library based on the target attribute information of the head key feature.

[0078] Further optionally, in one embodiment of the present disclosure, the avatar creation module 505 further comprises: If the feature library does not contain a head key feature corresponding to the target attribute information, it is used to perform an avatar creation operation on the point cloud of the head key feature in the target object model based on a user trigger.

[0079] Optionally, as shown in FIG. 5 , in one embodiment of the present disclosure, the digital human generation platform 500 further includes a collection module 508: The collection module 508 is used for collecting the point clouds of multiple head key features of each person in the multiple people and the attribute information of each head key feature, and storing them in the feature library.

[0080] Further optionally, in one embodiment of the present disclosure, the fusion module 503 is performing registration of the head key feature point cloud with the target object model; It is used to transfer the points of the head key features to the regions of corresponding head key features in the target object model. The digital human generation platform 500 of this embodiment is the same as the implementation of the above-mentioned related method embodiment by using the above-mentioned modules to realize the implementation principle and technical effect of digital human generation, for which details may refer to the description of the above-mentioned related method embodiment and will not be described in detail here.

[0081] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0082] As shown in FIG. 6, a block diagram of an electronic device 600 for implementing an embodiment of the present disclosure is shown. The electronic device is intended to represent various types of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, mobile phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure as described herein and / or claimed herein.

[0083] 6, the device 600 includes a computing unit 601, which can perform various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 602 or loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 can also store various programs and data required for the device 600 to operate. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0084] Several components in device 600 are connected to an I / O interface 605, including input units 606 such as a keyboard, mouse, etc., output units 607 such as various types of displays, speakers, etc., storage units 608 such as a disk, optical disk, etc., and communication units 609 such as a network card, modem, wireless communication transceiver, etc. The communication units 609 enable device 600 to exchange information / data with other devices via a computer network, such as the Internet, and / or various telecommunication networks.

[0085] The computing unit 601 is a general-purpose and / or dedicated processing component with various processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, computing units that execute various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the method of constructing a terrain map. For example, in some embodiments, the method of constructing a terrain map can be realized as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, some or all of the computer program is loaded and / or installed in the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the above-mentioned methods can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the method of constructing a terrain map via any other suitable manner (e.g., by firmware).

[0086] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), programming programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include being implemented in one or more computer programs that may be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be an application specific or general purpose programmable processor, and that may receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0087] The program codes for implementing the methods of the present disclosure can be written using any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus such that the functions / operations specified in the flowcharts and / or block diagrams are implemented when the program codes are executed by the processor or controller. The program codes can be executed entirely on the machine, partially on the machine, partially on the machine as a separate software package and partially on a remote machine, or entirely on a remote machine or server.

[0088] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use with or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

[0089] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other types of devices can also be used to provide interaction with a user, for example, feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback) and can receive input from the user in any form (including acoustic input, speech input, and haptic input).

[0090] The systems and techniques described herein may be implemented in a computing system including a back-end component (e.g., a data server), or a middleware component (e.g., an application server), or a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user interacts with the implementation of the systems and techniques described herein), or any combination of such back-end, middleware, and front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0091] The computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is created by computer programs running on corresponding computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server for combining blockchains.

[0092] It should be understood that steps can be rearranged, added, or deleted using various types of flows shown above. For example, each step described in the present disclosure can be performed in parallel, sequentially, or in a different order, but is not limited herein as long as the technical solution disclosed in the present disclosure can achieve the desired results.

[0093] The above specific implementation methods do not constitute a limitation of the protection scope of the present disclosure. Those skilled in the art can make various modifications, combinations, subcombinations, and substitutions based on design requirements and other factors. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. 1. A method for generating a digital human, comprising: Obtaining a corresponding target object model based on an image of the digital human to be generated; obtaining a point cloud of corresponding head key features from a predefined feature library based on the head key features in the image; fusing the point cloud of the head key features with the target object model to obtain a digital human image. How to generate a digital human.

2. After fusing the point cloud of the head key features to the target object model, and before the step of acquiring a digital human image, the method further comprises: detecting whether a similarity between a head key feature in the fused target object model and a head key feature in the image is equal to or greater than a preset similarity threshold; and performing a create avatar operation on the point cloud of the head key features fused to the target object model based on a user trigger in response to the similarity being less than a preset similarity threshold. The method of generating a digital human according to claim 1 .

3. performing an avatar creation operation on the point cloud of the head key features fused to the target object model based on a user trigger, performing an avatar creation operation on the point cloud of the head key features fused to the target object model based on a user trigger, using a pre-established implicit constraint surface and a pre-set dynamic curve as constraints; The method for generating a digital human according to claim 2 .

4. performing an avatar creation operation on the point cloud of the head key features fused to the target object model based on a user trigger, with a pre-established implicit constraint surface and a pre-set dynamic curve as constraints; Obtaining operation information of a first controller set on the implicit constraint surface triggered by the user, the implicit constraint surface being a surface that completely matches a surface topological structure of the target object model before being controlled, and the implicit constraint surface being invisible to the user; obtaining motion information of a triggered point on the implicit constraint surface based on a first motion mapping relationship between a first controller on the implicit constraint surface and a point on the implicit constraint surface that is pre-established, and motion information of the first controller; determining motion information of a second controller that is co-located with the point on the target object model based on motion information of the triggered point on the implicit constraint surface; Controlling motion information of a point on the target object model based on a second motion mapping relationship between a second controller and a point on the target object model that is set in advance for the target object model and motion information of the second controller; and performing position adjustment on the point cloud of the head key features merged with the target object model using a preset dynamic curve as a constraint. The method for generating a digital human according to claim 3 .

5. performing position adjustment on the point cloud of the head key features merged with the target object model using a preset dynamic curve as a constraint; Detecting whether the second controller is on a preset dynamic curve, and setting at least two of the second controllers for each preset dynamic curve; acquiring operation information of another second controller on the dynamic curve based on operation information of the second controller in response to the second controller being on a preset dynamic curve; and performing position adjustment on the point cloud of the head key features fused to the target object model based on the motion information of the second controller and the motion information of another second controller in the dynamic curve. The method for generating a digital human according to claim 4.

6. performing position adjustment on the point cloud of the head key features merged with the target object model using a preset dynamic curve as a constraint; and, in response to the second controller not being on a preset dynamic curve, performing a position adjustment on the point cloud of the head key features fused to the target object model based on motion information of the second controller. The method for generating a digital human according to claim 5 .

7. After performing an avatar creation operation on the point cloud of the head key features fused to the target object model based on a user trigger, the method further comprises: detecting whether an attached template of the target object model is suitable for the digital human image; and adjusting the attachment template in the digital human image in response to the attachment template of the target object model being incompatible with the digital human image. A method for generating a digital human according to any one of claims 2 to 6.

8. The step of obtaining a corresponding target object model based on an image of the digital human to be generated includes: Extracting attribute features of the digital human based on an image of the digital human to be generated; and obtaining the corresponding target object model from a pre-defined model library based on the attribute features of the digital human, the model library including a plurality of object models. A method for generating a digital human according to any one of claims 1 to 6.

9. The step of obtaining a corresponding target object model based on an image of the digital human to be generated includes: and when an attribute feature of the digital human is not extracted based on the image of the digital human to be generated, a preset standard model is set as a target object model. The method of generating a digital human according to claim 8.

10. The step of obtaining a point cloud of a corresponding head key feature from a pre-established feature library based on the head key feature in the image includes: obtaining target attribute information of head key features in the image; and acquiring a point cloud of a corresponding head key feature from the feature library based on target attribute information of the head key feature. A method for generating a digital human according to any one of claims 1 to 6.

11. The method comprises: and performing an avatar creation operation on a point cloud of the head key feature in the target object model based on a user's trigger when the feature library does not include a head key feature corresponding to the target attribute information. The method of generating a digital human according to claim 10.

12. Prior to the step of obtaining a point cloud of corresponding head key features from a pre-established feature library based on the head key features in the image, the method further comprises: Further comprising a step of collecting a point cloud of a plurality of head key features of each person among the plurality of people and attribute information of each head key feature, and storing the point cloud in the feature library. A method for generating a digital human according to any one of claims 1 to 6.

13. The step of blending the head key feature point cloud to the target object model comprises: registering the head key feature point cloud to the target object model; transferring the point cloud of the head key feature to a region of a corresponding head key feature in the target object model. A method for generating a digital human according to any one of claims 1 to 6.

14. A digital human generation platform, a model acquisition module for acquiring a corresponding target object model based on an image of the digital human to be generated; a feature acquisition module for acquiring a point cloud of corresponding head key features from a predefined feature library based on the head key features in the image; and a fusion module for fusing the point cloud of the head key features with the target object model to obtain a digital human image. A platform for generating digital humans.

15. The generation platform comprises: a first detection module for detecting whether a similarity between a head key feature in the target object model after fusion and a head key feature in the image is equal to or greater than a preset similarity threshold; and an avatar creation module for performing an avatar creation operation on the point cloud of the head key features merged with the target object model based on a user trigger in response to the similarity being less than a preset similarity threshold. The digital human generation platform of claim 14.

16. The avatar creation module includes: and performing an avatar creation operation on the point cloud of the head key features fused to the target object model based on a user trigger, with a pre-established implicit constraint surface and a pre-set dynamic curve as constraints. The digital human generation platform of claim 15.

17. The avatar creation module includes: Obtain operation information of a first controller set on the implicit constraint surface triggered by the user, the implicit constraint surface being a surface that completely matches a surface topological structure of the target object model before being controlled, the implicit constraint surface being invisible to the user; Obtaining motion information of a triggered point on the implicit constraint surface based on a first motion mapping relationship between a first controller on the implicit constraint surface and a point on the implicit constraint surface that is pre-established, and motion information of the first controller; determining motion information of a second controller that is co-located with the point on the target object model based on motion information of the triggered point on the implicit constraint surface; Controlling the motion information of the point on the target object model based on a second motion mapping relationship between a second controller and a point on the target object model that is set in advance for the target object model and motion information of the second controller; A preset dynamic curve is used as a constraint to perform position adjustment on the point cloud of the head key features fused to the target object model. The digital human generation platform of claim 16.

18. The avatar creation module includes: Detect whether the second controller is on a preset dynamic curve, and set at least two of the second controllers for each preset dynamic curve; In response to the second controller being on a preset dynamic curve, acquiring operation information of another second controller on the dynamic curve based on operation information of the second controller; and performing position adjustment on the point cloud of the head key features fused to the target object model based on the motion information of the second controller and the motion information of another second controller in the dynamic curve. The digital human generation platform of claim 17.

19. The avatar creation module further includes: and in response to the second controller not being on a preset dynamic curve, performing position adjustments on the point cloud of the head key features fused to the target object model based on motion information of the second controller. The digital human generation platform of claim 18.

20. The generation platform comprises: a second detection module for detecting whether an attached template of the target object model is suitable for the digital human image; and an adjustment module for adjusting the attached template in the digital human image in response to the attached template of the target object model being matched to the digital human image. A platform for generating a digital human according to any one of claims 15 to 19.

21. The model acquisition module includes: Extracting attribute features of the digital human based on the image of the digital human to be generated; and obtaining the corresponding target object model from a predefined model library based on the attribute features of the digital human, the model library including a plurality of object models. A platform for generating a digital human according to any one of claims 14 to 19.

22. The model acquisition module includes: When the attribute features of the digital human are not extracted based on the image of the digital human to be generated, a preset standard model is used as a target object model. The digital human generation platform of claim 21.

23. The feature acquisition module includes: obtaining target attribute information of head key features in the image; and obtaining a point cloud of the corresponding head key feature from the feature library based on the target attribute information of the head key feature. A platform for generating a digital human according to any one of claims 15 to 19.

24. The avatar creation module further includes: if the feature library does not include a head key feature corresponding to the target attribute information, a point cloud of the head key feature in the target object model is used to perform an avatar creation operation based on a user's trigger. The digital human generation platform of claim 23.

25. The generation platform comprises: A collection module is further included for collecting a point cloud of a plurality of head key features of each person among the plurality of people and attribute information of each head key feature, and storing the point cloud in the feature library. A platform for generating a digital human according to any one of claims 14 to 19.

26. The fusion module comprises: performing registration of the head key feature point cloud with the target object model; used to transfer the point cloud of the head key feature to a region of a corresponding head key feature in the target object model. A platform for generating a digital human according to any one of claims 14 to 19.

27. An electronic device, At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 6. electronic equipment.

28. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions cause a computer to carry out the method according to any one of claims 1 to 6. A non-transitory computer-readable storage medium.

29. A computer program comprising: The computer program, when executed by a processor, implements the method according to any one of claims 1 to 6. Computer program.

Images