Three-dimensional human motion generation method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of computer vision, in particular to a three-dimensional human action generation method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: identifying a dependency relationship between a plurality of constraints in a target constraint set, and constructing a core difficult constraint subset based on the dependency relationship; then matching action samples of the core difficult constraint subset; generating diffusion noise of a human action diffusion generation model according to the action samples; further fusing the diffusion noise and random noise by using a mask to meet a preset semantic condition; and iteratively optimizing the diffusion noise by using an optimizer to minimize errors under the target constraint set and generate an action sequence of a three-dimensional human pose representation. Thus, the problems that, due to random initialization of noise in the related art, constraints are difficult to meet, errors are large and visual quality is low when a high-difficulty action generation task is processed are solved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, electronic device and storage medium for generating three-dimensional human motion. Background Technology

[0002] 3D human motion generation is an important task in fields such as computer vision and computer graphics. Its goal is to transform high-level semantics and behavioral objectives into continuous, natural, and physically consistent 3D human motion sequences by modeling and learning from large-scale human behavior data. It has important applications in fields such as virtual reality, embodied intelligence, and film and television production, and also provides key support for embodied intelligent agents such as humanoid robots to achieve a closed loop from perception to executable actions.

[0003] In related technologies, control signals used for motion generation are mainly divided into two categories: trajectory signals and constraint functions. Trajectory signals can be used as conditional inputs in ControlNet-like model architectures; constraint functions provide a flexible representation for arbitrary zero-shot generation tasks, and are compatible with training-free diffusion noise optimization methods, while also integrating trajectory signals as optimization targets.

[0004] However, due to the random initialization of noise, the relevant diffusion noise optimization method lacks the action priors required for handling high-difficulty tasks. As a result, when dealing with high-difficulty action generation tasks in the real world, such as moving in an extremely restricted physical space or applying counting constraints such as step control, problems such as scene clipping, difficulty in satisfying constraints, large errors, and low visual quality occur, which urgently need to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for generating three-dimensional human motion, in order to solve the problems of difficulty in meeting constraints, large errors, and low visual quality when processing high-difficulty motion generation tasks due to random initialization of noise in related technologies.

[0006] The first aspect of this application provides a method for generating three-dimensional human motion, comprising the following steps: identifying the dependencies between multiple constraints in a target constraint set, and constructing a core difficult constraint subset based on the dependencies; matching motion samples from the core difficult constraint subset, and generating diffusion noise of a human motion diffusion generation model based on the motion samples; fusing the diffusion noise and random noise using a mask to satisfy preset semantic conditions, and iteratively optimizing the diffusion noise using an optimizer to minimize the error under the target constraint set, thereby generating a motion sequence representing a three-dimensional human pose.

[0007] Based on the above technical means, this application embodiment constructs a core difficult constraint subset and matches action samples to provide the diffusion model with action priors that meet the task requirements, avoids random initialization of noise, and uses a mask to fuse diffusion noise and random noise. While retaining the effective action priors of the retrieved diffusion noise, random noise is introduced to improve diversity. Furthermore, the fusion results are constrained by preset semantic conditions to ensure the semantic consistency of the action generation results. By using an optimizer to iteratively optimize the diffusion noise, the error is minimized under the target constraint set, effectively improving the quality of the 3D human action sequence generation.

[0008] Optionally, in one embodiment of this application, the step of constructing a core difficult constraint subset based on the dependency relationship includes: sorting the constraints according to the difficulty of each constraint among the plurality of constraints to generate a difficulty ranking sequence of the plurality of constraints; and based on the difficulty ranking sequence, determining the constraint with the highest difficulty and other constraints that satisfy a preset association condition with the constraint with the highest difficulty to construct the core difficult constraint subset.

[0009] Based on the above technical means, the embodiments of this application construct a subset of core difficult constraints as a reference for subsequent action sample retrieval, avoiding the use of the entire target constraint set for retrieval, and only selecting the core difficult constraints in the task to construct a subset for retrieval, effectively improving the targeting and retrieval efficiency of the retrieval target.

[0010] Optionally, in one embodiment of this application, the action sample matching the core difficult constraint subset includes: determining the constraint loss function corresponding to the core difficult constraint subset; filtering a sample candidate set that meets preset filtering conditions from a preset action dataset based on the constraint loss function; and determining the action sample based on the text description of the sample candidate set.

[0011] Based on the above technical means, the embodiments of this application filter sample candidate sets that meet the preset filtering conditions from the preset action dataset based on the constraint loss function, and determine the action samples based on the text description of the sample candidate sets. This can use the constraint loss function as a quantitative basis, avoid the subjectivity of sample retrieval, and further improve the semantic adaptability of samples and tasks by combining text description filtering.

[0012] Optionally, in one embodiment of this application, the formula for solving the optimal mask of the mask may be, but is not limited to, the following: , in, For the optimal mask, For the mask candidate set, For the target constraint set, Generate a model for the diffusion of human motion. ( ) represents the preset reward cost function. This is the result of fusing the diffused noise and the random noise.

[0013] Based on the above technical means, the embodiments of this application accurately select the mask that minimizes the sum of constraint loss and quality cost by combining the optimization objectives of constraint loss and quality cost, while preserving the prior knowledge of effective actions of the search samples and taking into account the generation flexibility brought by random noise.

[0014] Optionally, in one embodiment of this application, the optimization formula of the optimizer may be, but is not limited to, the following: , in, To optimize the update amount for diffused noise, For the target constraint set, Generate a model for the diffusion of human motion. This is the result of fusing the diffused noise and the random noise.

[0015] Based on the above technical means, the embodiments of this application calculate the gradient of the target constraint function on the optimization update amount of the diffusion noise through backpropagation of the optimizer, and iteratively update the diffusion noise based on the gradient to obtain the optimized diffusion noise. This provides a quantifiable and reproducible mathematical basis for the iterative optimization of diffusion noise, ensuring the rigor and operability of the optimization process.

[0016] A second aspect of this application provides a three-dimensional human motion generation device, comprising: a recognition module for recognizing dependencies between multiple constraints in a target constraint set and constructing a core difficult constraint subset based on the dependencies; a first generation module for matching motion samples from the core difficult constraint subset and generating diffusion noise for a human motion diffusion generation model based on the motion samples; and a second generation module for fusing the diffusion noise and random noise using a mask to satisfy preset semantic conditions and iteratively optimizing the diffusion noise using an optimizer to minimize the error under the target constraint set, thereby generating a motion sequence representing a three-dimensional human pose.

[0017] Based on the above technical means, this application embodiment constructs a core difficult constraint subset and matches action samples to provide the diffusion model with action priors that meet the task requirements, avoids random initialization of noise, and uses a mask to fuse diffusion noise and random noise. While retaining the effective action priors of the retrieved diffusion noise, random noise is introduced to improve diversity. Furthermore, the fusion results are constrained by preset semantic conditions to ensure the semantic consistency of the action generation results. By using an optimizer to iteratively optimize the diffusion noise, the error is minimized under the target constraint set, effectively improving the quality of the 3D human action sequence generation.

[0018] Optionally, in one embodiment of this application, the identification module includes: a sorting unit, configured to sort the multiple constraints according to the difficulty of each constraint to generate a difficulty sorting sequence of the multiple constraints; and a construction unit, configured to determine the constraint with the highest difficulty and other constraints that satisfy a preset association condition with the constraint with the highest difficulty based on the difficulty sorting sequence, so as to construct the core difficult constraint subset.

[0019] Based on the above technical means, the embodiments of this application construct a subset of core difficult constraints as a reference for subsequent action sample retrieval, avoiding the use of the entire target constraint set for retrieval, and only selecting the core difficult constraints in the task to construct a subset for retrieval, effectively improving the targeting and retrieval efficiency of the retrieval target.

[0020] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the three-dimensional human motion generation method as described in the above embodiments.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for generating three-dimensional human motion.

[0022] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, implements the above-described method for generating three-dimensional human motion.

[0023] This application's embodiments construct a core difficult constraint subset and match it with action samples to provide the diffusion model with action priors that meet task requirements, avoiding random initialization of noise. By using a mask to fuse diffusion noise and random noise, it retains the effective action priors retrieved from the diffusion noise while introducing random noise to enhance diversity. Furthermore, by using preset semantic conditions to constrain the fusion results, it ensures the semantic consistency of the action generation results. By using an optimizer to iteratively optimize the diffusion noise, it achieves error minimization under the target constraint set, effectively improving the quality of 3D human action sequence generation. Thus, it solves the problems of difficulty in satisfying constraints, large errors, and low visual quality in handling high-difficulty action generation tasks caused by the random initialization of noise in related technologies.

[0024] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a three-dimensional human motion generation method provided according to an embodiment of this application; Figure 2 This is a block diagram of a three-dimensional human motion generation device provided according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application.

[0026] Figure label: 20-Three-dimensional human motion generation device; 100-Recognition module; 200-First generation module; 300-Second generation module; 301-Memory; 302-Processor; 303-Communication interface. Detailed Implementation

[0027] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0028] In 3D human motion generation, a core challenge is how to handle open generation tasks without retraining the model, such as movement in a constrained physical space or imposing counting constraints like step counts. This requires addressing two key issues.

[0029] First, due to the random initialization of noise, relevant diffusion noise optimization methods often fail to include the action priors required to handle these difficult generation tasks. This results in generation results that often fail to meet the required constraints well, exhibiting large errors and significant motion defects such as large inter-frame discontinuities. To address this, this application proposes a retrieval method to obtain the necessary motion patterns for the corresponding difficult tasks, providing better initial values ​​for noise optimization of the diffusion model. Specifically, it proposes to improve the targeting by parsing the core difficulty subset of a given generation task.

[0030] Second, retrieval-based methods often suffer from deficiencies in diversity and semantic consistency. To address this, this application proposes a masking fusion method using diffused noise and random noise. On one hand, introducing random noise increases the uncertainty of the retrieved diffused noise, enhancing local diversity. On the other hand, the generated results corresponding to random noise are closer to the original data distribution and better conform to the text prior. Therefore, by masking and fusing the retrieval diffused noise and random noise, the original text conditions can be better satisfied while maintaining the new necessary motion patterns.

[0031] The following description, with reference to the accompanying drawings, describes a method, apparatus, electronic device, and storage medium for generating three-dimensional human motion according to embodiments of this application.

[0032] Figure 1 This is a flowchart of a three-dimensional human motion generation method provided according to an embodiment of this application.

[0033] like Figure 1 As shown, the method for generating three-dimensional human motion includes the following steps: In step S101, the dependencies between multiple constraints in the target constraint set are identified, and a subset of core difficult constraints is constructed based on the dependencies.

[0034] In the embodiments of this application, the target constraint set refers to the set of all constraints preset by the action generation task to limit the action generation result, and is the basis for action generation.

[0035] In actual execution, the embodiments of this application first determine the target constraint set corresponding to an action generation task, and use a language model to perform correlation analysis on the target constraint set to clearly identify the interrelationships between multiple constraints (specifically divided into three types: closely related, conflicting, and unrelated).

[0036] Furthermore, in one embodiment of this application, constructing a core difficult constraint subset based on dependency relationships includes: sorting the constraints according to their difficulty to generate a difficulty ranking sequence of the constraints; and based on the difficulty ranking sequence, determining the constraint with the highest difficulty and other constraints that satisfy preset association conditions with the constraint with the highest difficulty to construct a core difficult constraint subset.

[0037] In the embodiments of this application, the preset association condition is a judgment criterion used to screen the constituent elements of the core difficult constraint subset. Specifically, it refers to the preset judgment rule used to determine whether there is a close relationship between other constraints (excluding the most difficult constraint) and the most difficult constraint in the difficulty ranking sequence. Only other constraints that meet the preset association condition can form the core difficult constraint subset together with the most difficult constraint.

[0038] In actual implementation, after completing the correlation analysis, this embodiment of the application evaluates the difficulty of each constraint in the target constraint set and sorts them in descending order of difficulty to obtain a difficulty ranking sequence. Then, based on the difficulty ranking sequence, it obtains the core difficult constraint subset of the action generation task. This core difficult constraint subset specifically includes the most difficult constraint and other constraints closely related to the most difficult constraint.

[0039] This application embodiment constructs a subset of core difficult constraints as a reference for subsequent action sample retrieval, avoiding the use of the entire target constraint set for retrieval. Instead, it selects only the core difficult constraints in the task to construct a subset for retrieval, effectively improving the targeting and efficiency of the retrieval target.

[0040] In step S102, action samples of the core difficult constraint subset are matched, and diffusion noise of the human action diffusion generation model is generated based on the action samples.

[0041] Specifically, in one embodiment of this application, matching action samples of a subset of core difficult constraints includes: determining the constraint loss function corresponding to the subset of core difficult constraints; filtering a candidate set of samples that meet preset filtering conditions from a preset action dataset based on the constraint loss function; and determining action samples based on the text description of the candidate set of samples.

[0042] In the embodiments of this application, the preset screening criteria refer to the explicit judgment criteria used to screen the sample candidate set from the preset action dataset, specifically the screening principle of minimizing the constraint loss function corresponding to the subset of core difficult constraints.

[0043] In actual implementation, this embodiment uses the obtained core difficult constraint subset as a retrieval reference to retrieve action samples with high matching degree to the core difficult constraint subset, providing effective action pattern support for subsequent initial diffusion noise fitting. The specific retrieval process is as follows: According to the principle of minimizing the constraint loss function corresponding to the core difficult constraint subset, a top-k sample candidate set is obtained from a preset large-scale human action dataset (where the top-k sample candidate set refers to the k action samples with the smallest constraint loss function value, and k is a preset positive integer). Before calculating the constraint loss function, the action samples in the dataset can be subjected to equivalent geometric transformations on the horizontal plane (this transformation is a translation, rotation, or other transformation that does not change the essential characteristics of the action, used to eliminate matching deviations caused by differences in the pose angles of the action samples). Subsequently, based on the target text description of the action generation task and the text descriptions of each action sample in the top-k sample candidate set, semantic consistency screening is performed to remove samples with large semantic deviations. Finally, the optimal sample is selected from the filtered samples as the retrieved action sample.

[0044] The expression for the action sample retrieval process can be, but is not limited to, as follows: , in, For the retrieved action samples, For the equivalent geometric transformation on the horizontal plane, As a subset of core difficulty constraints, This is a minimize operation.

[0045] This application embodiment filters a candidate set of samples that meet preset filtering conditions from a preset action dataset based on a constraint loss function, and determines action samples based on the text description of the candidate set. It can use the constraint loss function as a quantitative basis, avoid the subjectivity of sample retrieval, and further improve the semantic adaptability of samples to tasks by combining text description filtering.

[0046] Furthermore, in this embodiment, the DDIM-inversion method (i.e., the standard algorithm for reverse conversion from action samples to noise in the diffusion model) is used to fit the retrieved action samples to diffusion noise, and convert them into initial diffusion noise that can be directly recognized by the pre-trained human action diffusion generation model.

[0047] In step S103, diffuse noise and random noise are fused using a mask to meet preset semantic conditions, and the diffuse noise is iteratively optimized using an optimizer to minimize the error under the target constraint set, thereby generating a three-dimensional human pose representation action sequence.

[0048] In the embodiments of this application, the preset semantic conditions refer to the judgment criteria used to constrain the effect of noise generation after mask fusion. Specifically, they refer to the preset requirements related to the semantic requirements and visual quality of the action generation task. They are quantitatively evaluated through a preset reward cost function and used to select the optimal mask in the mask fusion stage to ensure that the actions generated by the fused noise not only conform to the semantic logic of the task but also have a natural and smooth quality.

[0049] In actual implementation, this embodiment first initializes a random noise that follows a Gaussian distribution (the Gaussian distribution is the standard input noise distribution for the diffusion model). Next, a set of candidate binary temporal and spatial masks is constructed using a uniform partitioning method (each element in the mask takes only 0 or 1 values; the temporal mask controls the noise fusion weights at different time steps, and the spatial mask controls the noise fusion weights at different joints of the human body). Subsequently, a task-related semantic loss function and a motion quality function are constructed. These two functions are used as the minimum loss function, and the optimal mask with the minimum loss value is selected from the mask candidate set. This mask is then used to fuse the obtained initial diffusion noise with the initialized random noise to obtain the updated diffusion noise.

[0050] It should be noted that the mask candidate set The expression can be, but is not limited to, as: , in, For the mask candidate set, For a single binary temporal or spatial mask, It is a single element.

[0051] In addition, in one embodiment of this application, the formula for solving the optimal mask can be, but is not limited to, the following: , in, For the best mask, For the mask candidate set, For the target constraint set, Generate a model for the diffusion of human motion. ( ) represents the preset reward cost function. This is the result of fusing diffuse noise and random noise.

[0052] In actual implementation, the embodiments of this application use the candidate mask set as the optimization space, quantify the constraint satisfaction of the generated action with the complete target constraint set, and quantify the semantic consistency and quality of the action with the preset reward cost function, thereby selecting the mask that minimizes the sum of constraint loss and quality cost.

[0053] This application embodiment precisely selects the mask that minimizes the sum of constraint loss and quality cost by jointly optimizing the constraint loss and quality cost. This preserves the prior knowledge of effective actions in the search samples while taking into account the generation flexibility brought by random noise.

[0054] Furthermore, the expression for the fusion result of diffused noise and random noise can be, but is not limited to, as follows: , in, The result is a fusion of diffuse noise and random noise. This is the fitting result of the target constraint set based on the obtained diffused noise. This represents the fitting result of the target constraint set based on the initialized random noise. This can be solved using the general gradient descent method, and its expression can be, but is not limited to, as follows: , in, The obtained diffused noise, For the target constraint set, Generate a model for the diffusion of human motion.

[0055] Similarly, This can be solved using the general gradient descent method, and its expression can be, but is not limited to, as follows: , in, For the obtained random noise, For the target constraint set, Generate a model for the diffusion of human motion.

[0056] Furthermore, in this embodiment, a differentiable optimizer (e.g., the Adam optimizer based on gradient descent, which can adaptively adjust the learning rate to improve the efficiency of iterative optimization) is used to calculate the gradient of the target constraint function (a mathematical function constructed based on the target constraint set, used to quantify the degree of deviation between the action generation result and each constraint of the target constraint set) on the updated diffusion noise through backpropagation, and the diffusion noise is iteratively updated based on the gradient, so that the action generation result corresponding to the diffusion noise achieves the minimum error under the target constraint set.

[0057] It should be noted that, in one embodiment of this application, the optimization formula of the optimizer may be, but is not limited to, the following: , in, To optimize the update amount for diffused noise, For the target constraint set, Generate a model for the diffusion of human motion. This is the result of fusing diffuse noise and random noise.

[0058] In actual implementation, this embodiment starts with the fusion result of diffused noise and random noise, calculates the gradient of the objective constraint function with respect to diffused noise through backpropagation of the optimizer, and continuously adjusts the value of the diffused noise optimization update amount until the diffused noise optimization update amount that minimizes the error is found, and outputs the optimized diffused noise at this time. .

[0059] In this embodiment, the gradient of the target constraint function on the optimization update of the diffuse noise is calculated by backpropagation of the optimizer, and the diffuse noise is iteratively updated based on the gradient to obtain the optimized diffuse noise. This provides a quantifiable and reproducible mathematical basis for the iterative optimization of diffuse noise, ensuring the rigor and operability of the optimization process.

[0060] Furthermore, this embodiment utilizes a human motion diffusion generation model to perform a standard denoising process on the optimized diffusion noise (i.e., the calculation process in the diffusion model that gradually restores the motion generation result from the noise), generating a motion sequence represented by a three-dimensional human posture. The three-dimensional human posture specifically includes human joint coordinates (the specific position coordinates of each joint in three-dimensional space), joint velocities (the magnitude and direction of the speed of each joint's movement), and joint rotations (the rotation angle of each joint around its corresponding axis). Finally, by combining the standard SMPL (Skinned Multi-Person Linear Model) human model with joint rotation information, a complete and explicit three-dimensional surface mesh representation of the human posture motion sequence is obtained. The SMPL human model is a commonly used parametric three-dimensional human model that can generate a three-dimensional surface mesh through joint rotation parameters. A three-dimensional surface mesh refers to a three-dimensional structure composed of a series of vertices, edges, and faces, which can intuitively present the complete shape and motion posture of the human body.

[0061] The following is an illustrative description of the three-dimensional human motion generation method proposed in this application, using a specific embodiment as an example.

[0062] This application embodiment performs correlation analysis on the target constraint set of the action generation task, identifies the dependencies between constraints, and sorts the difficulty of each constraint to obtain a core difficult constraint subset. This subset includes the most difficult constraint and other constraints that satisfy preset correlation conditions with the most difficult constraint.

[0063] Furthermore, in this embodiment of the application, the action sample with the highest matching degree to the core difficult subset is retrieved from a large-scale dataset, and then diffusion noise is fitted to the retrieved action sample to transform it into the initial diffusion noise of the human action diffusion generation model.

[0064] Furthermore, in this embodiment, diffuse noise and random noise are fused by masking to make the action generation result meet the preset semantic conditions as much as possible. Then, the diffuse noise is iteratively optimized by an optimizer to minimize the error of the action generation result under the target constraint set, so as to obtain the optimized diffuse noise. Finally, the human motion diffusion generation model is used to perform a denoising process on the optimized diffuse noise to generate a three-dimensional human pose representation of the action sequence.

[0065] The 3D human motion generation method proposed in this application constructs a core difficult constraint subset and matches motion samples to provide the diffusion model with action priors that meet task requirements, avoiding random initialization of noise. By fusing diffusion noise and random noise using a mask, it retains the effective action priors retrieved from the diffusion noise while introducing random noise to enhance diversity. Furthermore, it ensures the semantic consistency of the motion generation results by constraining the fusion results through preset semantic conditions. By iteratively optimizing the diffusion noise using an optimizer, it achieves error minimization under the target constraint set, effectively improving the quality of 3D human motion sequence generation. This solves the problems of difficulty in satisfying constraints, large errors, and low visual quality in handling high-difficulty motion generation tasks caused by the random initialization of noise in related technologies.

[0066] Next, the three-dimensional human motion generation device according to the embodiments of this application is described with reference to the accompanying drawings.

[0067] Figure 2 This is a block diagram of a three-dimensional human motion generation device provided according to an embodiment of this application.

[0068] like Figure 2 As shown, the three-dimensional human motion generation device 20 includes: a recognition module 100, a first generation module 200, and a second generation module 300.

[0069] The identification module 100 is used to identify the dependency relationships between multiple constraints in the target constraint set and to construct a subset of core difficult constraints based on the dependency relationships.

[0070] The first generation module 200 is used to match action samples of the core difficult constraint subset and generate diffusion noise of the human action diffusion generation model based on the action samples.

[0071] The second generation module 300 is used to fuse diffuse noise and random noise using a mask to meet preset semantic conditions, and to use an optimizer to iteratively optimize the diffuse noise in order to minimize the error under the target constraint set, thereby generating a three-dimensional human pose representation action sequence.

[0072] Optionally, in one embodiment of this application, the identification module 100 includes a sorting unit and a construction unit.

[0073] The sorting unit is used to sort the constraints according to their difficulty to generate a difficulty sorting sequence of the constraints.

[0074] The building unit is used to determine the most difficult constraint and other constraints that satisfy preset association conditions with the most difficult constraint based on the difficulty ranking sequence, so as to build a core difficult constraint subset.

[0075] Optionally, in one embodiment of this application, the first generation module 200 includes: a first determining unit, a filtering unit, and a second determining unit.

[0076] The first determining unit is used to determine the constraint loss function corresponding to the subset of core difficult constraints.

[0077] The filtering unit is used to filter a candidate set of samples that meet the preset filtering conditions from the preset action dataset based on the constraint loss function.

[0078] The second determining unit is used to determine action samples based on the text description of the sample candidate set.

[0079] Optionally, in one embodiment of this application, the formula for solving the optimal mask can be, but is not limited to, the following: , in, For the best mask, For the mask candidate set, For the target constraint set, Generate a model for the diffusion of human motion. ( ) represents the preset reward cost function. This is the result of fusing diffuse noise and random noise.

[0080] Optionally, in one embodiment of this application, the optimization formula of the optimizer may be, but is not limited to, the following: , in, To optimize the update amount for diffused noise, For the target constraint set, Generate a model for the diffusion of human motion. This is the result of fusing diffuse noise and random noise.

[0081] It should be noted that the foregoing explanation of the embodiment of the three-dimensional human motion generation method also applies to the three-dimensional human motion generation device of this embodiment, and will not be repeated here.

[0082] The 3D human motion generation device proposed in this application constructs a core difficult constraint subset and matches motion samples to provide the diffusion model with action priors that meet task requirements, avoiding random initialization of noise. By fusing diffusion noise and random noise using a mask, it retains the effective action priors retrieved from the diffusion noise while introducing random noise to enhance diversity. Furthermore, by constraining the fusion results through preset semantic conditions, it ensures the semantic consistency of the motion generation results. By iteratively optimizing the diffusion noise using an optimizer, it achieves error minimization under the target constraint set, effectively improving the quality of 3D human motion sequence generation. This solves the problems of difficulty in satisfying constraints, large errors, and low visual quality in handling high-difficulty motion generation tasks caused by the random initialization of noise in related technologies.

[0083] Figure 3 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. The electronic device may include: The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0084] When the processor 302 executes the program, it implements the three-dimensional human motion generation method provided in the above embodiments.

[0085] Furthermore, electronic devices also include: Communication interface 303 is used for communication between memory 301 and processor 302.

[0086] The memory 301 is used to store computer programs that can run on the processor 302.

[0087] The memory 301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0088] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0089] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0090] Processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0091] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for generating three-dimensional human motion.

[0092] This application also provides a computer program product, including a computer program that, when executed, implements the above-described method for generating three-dimensional human motion.

[0093] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0094] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0095] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0096] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0097] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0098] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0099] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0100] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for generating three-dimensional human motion, characterized in that, Includes the following steps: Identify the dependencies between multiple constraints in the target constraint set, and construct a subset of core difficult constraints based on the dependencies; Match action samples from the core difficult constraint subset, and generate diffusion noise for the human action diffusion generation model based on the action samples; The diffuse noise and random noise are fused using a mask to satisfy preset semantic conditions, and the diffuse noise is iteratively optimized using an optimizer to minimize the error under the target constraint set, thereby generating a three-dimensional human pose representation action sequence.

2. The method according to claim 1, characterized in that, The construction of the core hard constraint subset based on the dependency relationship includes: The constraints are sorted according to their difficulty to generate a difficulty ranking sequence of the constraints. Based on the difficulty ranking sequence, the constraint with the highest difficulty and other constraints that satisfy preset association conditions with the constraint with the highest difficulty are determined to construct the core difficult constraint subset.

3. The method according to claim 1, characterized in that, The action samples that match the subset of core difficult constraints include: Determine the constraint loss function corresponding to the subset of core difficult constraints; Based on the constraint loss function, a candidate set of samples that meet the preset screening conditions is selected from the preset action dataset; The action sample is determined based on the text description of the candidate sample set.

4. The method according to claim 1, characterized in that, The formula for finding the optimal mask is: ， in, For the optimal mask, For the mask candidate set, For the target constraint set, Generate a model for the diffusion of human motion. ( ) represents the preset reward cost function. This is the result of fusing the diffused noise and the random noise.

5. The method according to claim 1, characterized in that, The optimization formula of the optimizer is: ， in, To optimize the update amount for diffused noise, For the target constraint set, Generate a model for the diffusion of human motion. This is the result of fusing the diffused noise and the random noise.

6. A three-dimensional human motion generation device, characterized in that, include: The identification module is used to identify the dependencies between multiple constraints in the target constraint set, and to construct a subset of core difficult constraints based on the dependencies. The first generation module is used to match action samples of the core difficult constraint subset and generate diffusion noise of the human action diffusion generation model based on the action samples. The second generation module is used to fuse the diffused noise and random noise using a mask to meet preset semantic conditions, and to use an optimizer to iteratively optimize the diffused noise to minimize the error under the target constraint set, thereby generating a three-dimensional human pose representation action sequence.

7. The apparatus according to claim 6, characterized in that, The identification module includes: A sorting unit is used to sort the multiple constraints according to the difficulty of each constraint to generate a difficulty sorting sequence of the multiple constraints. The construction unit is used to determine the constraint with the highest difficulty and other constraints that satisfy preset association conditions with the constraint with the highest difficulty based on the difficulty ranking sequence, so as to construct the core difficult constraint subset.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the three-dimensional human motion generation method as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the three-dimensional human motion generation method as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the three-dimensional human motion generation method as described in any one of claims 1-5.

Images