A text-driven human motion editing method based on omnidirectional positive and negative supervised learning

Abstract

This invention discloses a text-driven human motion editing method based on comprehensive positive and negative supervised learning, belonging to the fields of artificial intelligence and computer vision. The method comprises two stages: model training and application. In the training stage, the original motion, target motion, and positive and negative text instructions are acquired. After feature extraction and fusion, these are input into a diffusion Transformer model for denoising and prediction. The core lies in simultaneously calculating three losses: retrospective feature supervision, motion preservation, and triple semantic alignment, and jointly optimizing them with the model's main loss to obtain a comprehensively supervised motion editing model. In the application stage, user instructions and the motion to be edited are input into the model, which outputs a target motion that accurately responds to semantics while highly maintaining the coherence of non-edited regions. This invention significantly improves editing accuracy, preservation capability, and generation stability through a multi-layered, positive-negative combined comprehensive supervision mechanism.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, specifically to a text-driven human motion editing method based on omnidirectional positive and negative supervised learning. Background Technology

[0002] Human motion generation and editing technology has broad application prospects in fields such as 3D animation, virtual reality, human-computer interaction, and game development. With the development of natural language processing technology, text-driven human motion editing has become a research hotspot, aiming to semantically modify a given source human motion sequence based on the natural language description input by the user.

[0003] Existing text-driven human motion editing methods face a core challenge: how to precisely modify target regions based on text semantics while perfectly preserving the original motion continuity and realism in unmentioned regions—that is, balancing "editing" and "preservation." Most current mainstream methods employ a global conditional diffusion model, using textual instructions as conditional input to guide the generation of the entire motion sequence. However, this global adjustment mechanism lacks explicit distinction between edited and non-edited regions, easily leading to fine-grained semantic ambiguity. Specifically, modifying local actions can inadvertently disrupt the smoothness and physical structure of the overall motion, producing unnatural or even distorted movements.

[0004] To alleviate these problems, some studies have attempted to introduce action similarity-based auxiliary supervision to constrain the consistency between the generated action and the source action in the unedited region. However, these methods typically have limitations in hierarchical feature alignment and deep semantic consistency, making it difficult to maintain consistency across multiple scales from coarse to fine. Other methods use dynamic data augmentation strategies to enrich training samples and improve the model's coherence in complex temporal sequences. However, when faced with strict and complex text-action semantic alignment requirements, they still lack effective, fine-grained joint positive and negative supervision mechanisms, failing to achieve precise control over the edited region and lossless preservation of the non-edited region.

[0005] In summary, existing technologies in text-driven human motion editing struggle to balance the precise editing of target regions (that need to be changed) with the preservation of the continuity of non-edited regions (that need to be preserved), and local editing can easily disrupt the overall motion structure. Therefore, there is an urgent need for a text-driven motion editing method that can deeply integrate multi-level feature supervision, explicit motion preservation mechanisms, and refined semantic alignment strategies to overcome the aforementioned problems of existing technologies. Summary of the Invention

[0006] The purpose of this invention is to provide a text-driven human motion editing method based on comprehensive positive and negative supervised learning to overcome the above-mentioned problems in the existing technology. By constructing multi-layer backtracking feature supervision, fine-grained motion preservation mechanism and triple semantic alignment strategy, it realizes multi-faceted constraints on the generation process, thereby accurately responding to text instructions and highly maintaining the motion structure of non-editing areas. Under the premise of ensuring that the editing instructions are executed accurately, it maintains the overall fluency and naturalness of the original motion to the greatest extent.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A text-driven human motion editing method based on omnidirectional positive and negative supervised learning includes a model training phase and a model application phase, wherein: The model training phase includes: Acquire model training data, including original action sequences, target action sequences, positive text instructions corresponding to the target action sequences, and sample negative text instructions; extract features from the data to obtain original action features, positive text features, and negative text features; The original action features and the positive text features are fused across modalities to obtain fused features; Based on a preset noise scheduling mechanism, the denoising step and its time step information corresponding to the current training iteration are determined; the fused features, time step information, and the target action sequence with added noise are input into the diffusion Transformer model for denoising prediction to obtain the predicted action sequence. Based on the predicted action sequence and the target action sequence, calculate the main diffusion loss and related classification loss of the diffusion Transformer model; Based on the outputs of multiple intermediate layers of the diffusion Transformer model, the retrospective feature supervision loss between the target action sequence and the model is calculated. Based on the original action sequence and the target action sequence, frame-level similarity and action signal-to-noise ratio are calculated, and action retention loss is calculated according to the relationship between the action signal-to-noise ratio and a preset threshold and the predicted action sequence. Based on the positive text features, negative text features, and action feature embeddings extracted from the predicted action sequence, the triple semantic alignment loss is calculated; By combining the main diffusion loss, classification loss, retrospective feature supervision loss, action preservation loss, and triple semantic alignment loss of the aforementioned diffusion Transformer model, the model is jointly optimized to obtain a well-trained action editing model. The model application phase includes: The source action sequence to be edited, along with the natural language editing instructions input by the user, are input into the trained action editing model, which then outputs the edited target action sequence.

[0008] Further, feature extraction is performed on the data, including: The original motion sequence is represented as original motion features in the form of a frame-level time series, wherein the original motion features include global velocity, global orientation, local joint rotation, and local joint position. A pre-trained text encoder is used to extract the semantic features of the positive text instructions as positive text features, and the semantic features of the negative text instructions are extracted as negative text features.

[0009] Furthermore, the cross-modal fusion is specifically performed by: using an information fusion module based on the Transformer architecture to interactively calculate the original action features and the positive text features, and outputting the fused features.

[0010] Furthermore, the preset noise scheduling mechanism is a cosine noise scheduling mechanism.

[0011] Further, the calculation of the retrospective feature supervision loss includes: According to the diffusion Transformer model, multiple intermediate layers are pre-selected, and for each selected intermediate Transformer block, the following operations are performed: a lightweight prediction head is attached after the intermediate layer; the output of the intermediate layer is mapped to the corresponding intermediate action prediction through the lightweight prediction head; the error between the intermediate action prediction and the target action sequence is calculated as the retrospective feature supervision loss component of the intermediate layer. The retrospective feature supervision loss is determined by calculating a weighted sum based on the retrospective feature supervision loss components corresponding to all selected intermediate layers.

[0012] Furthermore, the plurality of intermediate layers includes a second, a fourth, and a sixth intermediate layer.

[0013] Furthermore, the calculation of frame-level similarity and action signal-to-noise ratio specifically includes: Using a sliding window mechanism, the frame-level similarity between the original action sequence and the target action sequence in rotation space and position space is calculated, and weighted combination and normalization are performed to obtain the similarity curve in the time dimension. Based on the similarity curve, the frames are sorted according to their similarity values, and the top similarity frame set and the bottom similarity frame set are determined respectively. The ratio of the sum of all elements in the top similarity frame set to the sum of all elements in the bottom similarity frame set is used to obtain the action signal-to-noise ratio.

[0014] Furthermore, the calculation action retains the loss, specifically including: When the signal-to-noise ratio of the action is greater than a preset threshold, the corresponding sample is determined to be a high signal-to-noise ratio sample; For the high signal-to-noise ratio sample, the reconstruction error between the predicted action sequence and the original action sequence is calculated and used as the action retention loss; The reconstruction error is the mean square error between the predicted action sequence and the original action sequence in the corresponding frame.

[0015] Furthermore, the calculation of the semantic alignment loss of the triples specifically involves: Calculate the first distance between the action feature embedding and the positive text feature, and the second distance between the action feature embedding and the negative text feature; Based on the difference between the first distance and the second distance, and in conjunction with a preset boundary margin, the semantic alignment loss of the triple is calculated to drive the first distance to be less than the second distance.

[0016] Furthermore, the joint optimization of the model specifically includes: According to the preset weighting coefficients, the main diffusion loss, classification loss, retrospective feature supervision loss, action retention loss and triple semantic alignment loss are weighted and summed to construct the total loss function; Based on the total loss function, an optimizer is used to perform end-to-end parameter updates on the model.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. Significantly improves the semantic alignment accuracy of action editing, overcoming semantic ambiguity. By introducing a triplet semantic alignment module combining positive and negative text, attraction and repulsion operations are performed simultaneously in the feature space. This mechanism not only brings the generated action closer to the positive instruction features but also actively distances it from irrelevant or opposite negative instruction features, thus constructing a more precise semantic boundary. This effectively enhances the correspondence between generated actions and natural language instructions, avoiding semantic ambiguity that may result from solely relying on positive conditional supervision, and achieving a deeper and more robust understanding and response to text instructions.

[0018] 2. Achieved precise modification of the edited area while maintaining a high degree of preservation of the non-edited area. This invention innovatively proposes an action preservation mechanism based on frame-level similarity and action signal-to-noise ratio. This mechanism can automatically identify high signal-to-noise ratio areas (i.e., non-edited subjects) that are not mentioned in the editing instructions and should remain unchanged, and apply strong explicit reconstruction constraints to them. This allows the model to clearly distinguish between the editing target and the preservation target during training and inference, thereby faithfully executing the text instructions to modify specific areas while perfectly maintaining the temporal coherence of the original action and the physical smoothness of the overall motion, fundamentally solving the problem of the difficulty in simultaneously achieving editing and preservation.

[0019] 3. Enhanced stability and multi-scale consistency in multi-layer feature generation, improving the overall reliability of the solution. Addressing the limitations of existing methods in hierarchical feature alignment, this invention introduces retrospective feature supervision into several key intermediate layers of the diffusion Transformer model. This mechanism ensures that the hidden representations of the intermediate layers continuously align with the final ideal target action distribution during training, guiding the model to establish and solidify coarse-to-fine feature correspondences from high-level semantics to low-level details. This multi-scale supervision throughout the generation process significantly improves the stability of model parameter optimization and final inference generation, ensuring consistent high quality and naturalness in the output action across overall pose, local details, and temporal transitions, thus enhancing the robustness and practicality of the entire technical solution. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] This embodiment provides a text-driven human motion editing method based on omnidirectional positive and negative supervised learning, such as... Figure 1 As shown, the specific implementation process includes a model training phase and a model application phase. The model training phase aims to build an action editing model that can accurately respond to text editing instructions and highly maintain the coherence of non-editing areas; the model application phase uses the trained model to process the user-input source actions and natural language editing instructions to generate the edited target action sequence. The two phases are described in detail below.

[0023] 1. Model training phase The core of the model training phase is to jointly optimize the generative model based on Diffusion Transformer (DiT) by introducing multi-faceted, positive and negative combined supervision signals, so that it internalizes the ability to accurately edit and effectively preserve data.

[0024] 1.1 Obtain model training data, including original action sequences, target action sequences, positive text instructions corresponding to the target action sequences, and sample negative text instructions; extract features from the data to obtain original action features, positive text features, and negative text features.

[0025] First, obtain data samples for model training. Each sample includes: an original action sequence X and a target action sequence. and a description from X to A positive natural language description of change, i.e., a positive text instruction. The target action sequence. That is, we hope that the model will learn the edited sequence of actions generated by the model.

[0026] Feature extraction includes the processing of action data and text data, as follows: 1) Motion Feature Extraction. The original motion sequence is represented as a frame-level time series. Specifically, the motion features of each frame are formalized as a feature containing global velocity. Global Orientation Local joint rotation and local joint position The combined vector is shown in the following equation. This representation method can comprehensively capture the spatiotemporal dynamics and structural information of the action. Wherein, global velocity... and global orientation Each dimension is 3, representing the motion of the root joint in three-dimensional space; local joint rotation. and local joint position The dimension J is determined by the number of joints in the human skeletal model. In a typical setting of this embodiment, the spliced ​​dimension J is 207. The specific expression is: in, It is a feature of the i-th frame, the entire sequence T is the sequence length. This processing is applied to the original action sequence X to obtain the original action features. Subsequent calculations of rotational spatial distance are based solely on the features. Partially performed, location and spatial distance calculation is based only on Partially carried out.

[0027] 2) Text Feature Extraction. A pre-trained text encoder, such as the CLIP text encoder, is used to extract semantic features from the forward text instructions, thus obtaining the forward text features. Meanwhile, to construct a positive-negative contrast learning mechanism, a random sample of a natural language description unrelated to the semantics of the current action is used as a negative text instruction. Its semantic features are then extracted using the same encoder to obtain the negative text features. This step lays the foundation for subsequent semantic alignment of triples.

[0028] 1.2 The original action features and the positive text features are fused across modally to obtain fused features. The cross-modal fusion is performed by an information fusion module based on the Transformer architecture, which interactively calculates the original action features and the positive text features to output the fused features.

[0029] To effectively combine the editing intent (text) with the context of the source action, this implementation constructs an information fusion module based on the Transformer architecture. The original action feature X obtained in step 1.1 is combined with the positive text feature... The inputs are shared into this module. Through a multi-head self-attention mechanism within the module, deep interaction and alignment of action features and text features are achieved. The module ultimately outputs a fused feature. This feature integrates the spatiotemporal structure information of the source action with the semantic information of the positive text instruction, providing clear, instruction-aware conditional guidance for the subsequent generation process.

[0030] 1.3 Based on a preset noise scheduling mechanism, the denoising step and its time step information corresponding to the current training iteration are determined; the fused features, time step information, and the target action sequence with added noise are input into the diffusion Transformer model for denoising prediction to obtain the predicted action sequence. The preset noise scheduling mechanism is a cosine noise scheduling mechanism.

[0031] A DiT model, comprising multiple consecutive Transformer blocks, is constructed as the core generator; in this embodiment, eight consecutive Transformer blocks are used. The model training follows a denoising diffusion probability model framework, including noise scheduling and time steps, noise addition and input, and denoising prediction. In a specific embodiment, a cosine noise scheduling mechanism is used to define the diffusion process. In each training iteration, a denoising step, i.e., time step t, is randomly sampled from a preset schedule, and its corresponding time step information is encoded into a vector. Based on time step t and the cosine scheduling, a clean target action sequence is generated. Adding appropriate levels of Gaussian noise yields a noisy sequence. The fusion features obtained in step 1.2 The embedding vector (time step information) at time step t and the noisy action sequence The common input is the DiT model. The DiT model performs progressive denoising calculations through multiple Transformer blocks within it, ultimately outputting a predicted action sequence. The core task of the model is to learn from... The original clean sequence was predicted in the middle. .

[0032] 1.4 Comprehensive Positive and Negative Supervision Loss Calculation. To ensure that the generated actions conform to text semantics while maintaining the structure of non-edited regions, this invention introduces three core supervision losses on top of the diffusion loss, collectively constituting comprehensive positive and negative supervision. These include: calculating the main diffusion loss and related classification loss of the diffusion Transformer model based on the predicted action sequence and the target action sequence; calculating the retrospective feature supervision loss between the target action sequence and the output of multiple intermediate layers of the diffusion Transformer model; calculating frame-level similarity and action signal-to-noise ratio based on the original action sequence and the target action sequence, and calculating the action retention loss based on the relationship between the action signal-to-noise ratio and a preset threshold and the predicted action sequence; and calculating the triplet semantic alignment loss based on the positive text features, negative text features, and action feature embeddings extracted from the predicted action sequence. Specifically: 1) Action sequences based on model prediction With the actual target action sequence The fundamental training objective of the computational model is the main diffusion loss. This loss measures the difference between the predicted data and the actual data (e.g., mean squared error). Simultaneously, a related classification loss is calculated. This loss is used to help improve the model's semantic discriminative ability at the feature level. Specifically, the classification loss... In the information fusion stage, it is achieved by connecting a transformer encoder to a classifier.

[0033] 2) To address the consistency issue of generated actions across multiple scales, this invention introduces retrospective feature supervision into multiple intermediate layers of the DiT model. First, intermediate layer feature capture is performed: Let... This represents the hidden representation output by the l-th intermediate Transformer block of the DiT model, where l is the layer index variable, B is the batch size, T is the sequence length, and D is the hidden dimension. The specific implementation steps are as follows: Pre-selection of supervision layers: In the DiT model, multiple intermediate Transformer blocks are pre-selected as supervision points. In this embodiment, the 2nd, 4th, and 6th Transformer blocks are selected.

[0034] Attaching Prediction Headers and Mapping: Attach a lightweight prediction head after each selected intermediate layer. Let the output hiding representation of the l-th selected intermediate layer be... The corresponding prediction head is used to map the action back to the action feature space to generate intermediate action predictions for this layer. Its expression is: Calculate components and weighted sums: Calculate the prediction for each intermediate action. With the actual target action sequence The error between them (e.g., mean squared error) is used as a retrospective feature of the supervised loss component of this layer. Its expression is: Where J represents the dimension of the action feature.

[0035] Finally, a weighted sum is calculated based on the loss components corresponding to all selected intermediate layers to obtain the total retrospective feature supervision loss. Its expression is: in, These are the preset weight coefficients for the corresponding layers. In one specific implementation of this embodiment, the weight coefficients for each layer are all 1. This mechanism forces the feature representations of the intermediate layers of the model to continuously align with the final target, effectively guiding the feature generation process from coarse to fine, significantly enhancing the stability of model optimization and inference, and ensuring the consistency of multi-scale actions.

[0036] 3) To explicitly protect regions not mentioned in the text instructions that should not be modified, this invention designs an edit-preservation mechanism based on motion signal-to-noise ratio (MotionSNR). This mechanism automatically identifies high-SNR frames (i.e., non-edited regions) that should remain unchanged during the editing process by comparing the original action sequence with the target action sequence, and applies reinforcement reconstruction constraints to these regions during training. The loss is calculated as follows: Calculate frame-level similarity and action signal-to-noise ratio. A sliding window mechanism is used to calculate the similarity between the original action sequence X and the target action sequence. The frame-level similarity between frames is calculated in both rotation and position spaces. The similarities in the two spaces are weighted, combined, and normalized to obtain a similarity curve in the time dimension. Based on this similarity curve, frames are sorted according to their similarity values, and a top and bottom similarity frame set are determined. The ratio of the sum of all elements in the top set to the sum of all elements in the bottom set is calculated to obtain the action signal-to-noise ratio.

[0037] To explicitly protect areas not mentioned in text instructions that should not be modified, this invention designs an edit-preservation mechanism based on motion signal-to-noise ratio (MotionSNR), the loss calculation of which specifically includes: To identify frames in the training data that should remain unchanged (i.e., non-edit regions), the original action sequence is first calculated. With target action sequence The frame-level similarity between the frames is used to assess the degree of modification in each frame. Using a sliding window of pre-defined size w, X and X are calculated in both rotation and position spaces. The frame-by-frame similarity between the frames is calculated. The similarity in rotation space is calculated as follows: in, This represents a distance metric computed in rotation space (such as quaternion representation). Similarly, similarity is computed in location space. Subsequently, using preset weights... and The similarity between the two spaces is weighted and combined to obtain the combined similarity of each frame. As shown in the following formula: In this embodiment, it can be set to = =1, to balance the importance of rotation and position information.

[0038] The above original similarity Normalized to the range [0,1], a smooth temporal similarity curve is generated. Based on this curve, all frames are sorted according to their similarity values, and a top set of high-similarity frames is defined. (For example, the top k most similar frames) and the bottom set of low similarity frames. (For example, the k frames with the lowest similarity). Here, k is the number of frames selected, which in this embodiment can be 5, meaning the 5 frames with the highest and lowest similarity are selected to form separate sets. The motion signal-to-noise ratio (MotionSNR) is extracted by calculating the ratio of the sum of all elements in these two sets, as shown in the following formula: in, and They respectively represent belonging to the set and The combined similarity value corresponding to the frames.

[0039] Motion SNR reflects the contrast between unedited regions (high similarity) and edited regions (low similarity) in a sample. When the MotionSNR of an action sample exceeds a preset threshold... When the value is typically 1.5, the corresponding sample is identified as a high signal-to-noise ratio (SNR) sample. For these high SNR samples, the model should preserve the original actions to the greatest extent possible. Therefore, an indicator function is introduced to impose constraints on high SNR samples, and the predicted action sequence is calculated. The reconstruction error between the original action sequence X and the original action sequence X is used as the action preservation loss. As shown in the following formula: in, The indicator function is 1 when the condition is true and 0 otherwise. The reconstruction error is the mean square error between the predicted action sequence and the original action sequence in the corresponding frame. This mechanism can intelligently identify samples with high signal-to-noise ratio (i.e., samples where the original action and the target action are highly similar and correspond to non-edited regions) and impose strong explicit reconstruction constraints on them. This forces the model to accurately learn and maintain the action structure of the unedited region during training, achieving perfect preservation of the spatiotemporal coherence and naturalness of the original action while accurately executing text editing instructions.

[0040] 4) To enhance the deep semantic consistency between generated actions and text instructions, triplet contrastive learning is introduced: Extracting action feature embeddings involves performing average pooling on the output hidden representation of the last layer (Lth layer) of the DiT model along the time dimension to obtain the global action feature embeddings for the predicted action sequence. : Calculate the triplet loss and the action feature embedding. With the positive text features The first distance between them, and the embedding of the action features With the negative text features The second distance between them. This embodiment uses the square of the Euclidean distance. Based on the difference between the first distance and the second distance, and combined with a preset boundary margin. (Typically set to 0.2), the semantic alignment loss of the triple is calculated. Its expression is: Where B is the batch size, and i is the index of the sample within the batch. , , Let represent the action feature embedding, positive text feature, and negative text feature of the i-th sample, respectively. This loss causes the first distance to be smaller than the second distance, that is, it brings the generated action closer to the positive instruction and pushes it further away from the negative instruction within the feature space. This effectively constructs a more accurate semantic boundary, overcomes the semantic ambiguity that may be caused by relying solely on positive supervision, and significantly improves the semantic alignment accuracy of editing actions.

[0041] 1.5 By combining the main diffusion loss, classification loss, retrospective feature supervision loss, action preservation loss, and triple semantic alignment loss of the aforementioned diffusion Transformer model, the model is jointly optimized to obtain a trained action editing model.

[0042] Specifically, after obtaining the aforementioned losses, the model undergoes end-to-end joint optimization. The main diffusion loss, classification loss, retrospective feature supervision loss, action preservation loss, and triple semantic alignment loss are weighted and summed according to preset weight coefficients to construct the total loss function. Its expression is: in, , , , These are the preset weight coefficients for classification loss, retrospective feature supervision loss, action preservation loss, and triple semantic alignment loss, respectively. In a specific training configuration, the weight coefficients for each loss can be set as follows: =0.01, =0.1, =1.0, =0.01. These coefficients can be determined by performing a grid search on the validation set.

[0043] Specifically, the model is trained through multiple iterations. Each training iteration includes the following steps: sampling a batch of sample data (including the original action sequence, the target action sequence, and positive and negative text instructions) from the training dataset; performing steps 1.1 to 1.4 above to complete the forward propagation and calculate the total loss. Then, backpropagation is performed to calculate the gradient of the loss relative to the model parameters. Based on the total loss function, an optimizer (such as AdamW) is used to perform end-to-end parameter updates on the model, minimizing... Through iterative training, the model gradually learns to generate actions that are both semantically accurate and maintain the structure of non-editable regions under the guidance of text instructions, ultimately resulting in a well-trained action editing model.

[0044] 2. Model Application Stage In the model application (inference) phase, the trained action editing model is used to process the user's editing requests. The model application phase includes: inputting the source action sequence to be edited and the natural language editing instructions input by the user into the trained action editing model, and outputting the edited target action sequence.

[0045] The specific steps are as follows: The user provides a source action sequence to be edited and a natural language editing instruction describing the desired modifications. The system first performs the same preprocessing and feature extraction on the instruction and source action sequence as in the training phase, and then fuses the resulting features through an information fusion module. Next, the fused features are input into the trained action editing model for denoising and inference. Guided by the positive text instruction, and combining its internalized action preservation and semantic alignment capabilities, the model automatically modifies the target region while maintaining the coherence of the source action in unmentioned areas. Finally, the model outputs an edited target action sequence that accurately responds to the text instruction while maintaining a high degree of naturalness and fluency.

[0046] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0048] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0049] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0050] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A text-driven human motion editing method based on omnidirectional positive and negative supervised learning, characterized in that, It includes the model training phase and the model application phase, in which: The model training phase includes: Acquire model training data, including original action sequences, target action sequences, positive text instructions corresponding to the target action sequences, and sample negative text instructions; extract features from the data to obtain original action features, positive text features, and negative text features; The original action features and the positive text features are fused across modalities to obtain fused features; Based on a preset noise scheduling mechanism, the denoising step and its time step information corresponding to the current training iteration are determined; the fused features, time step information, and the target action sequence with added noise are input into the diffusion Transformer model for denoising prediction to obtain the predicted action sequence. Based on the predicted action sequence and the target action sequence, calculate the main diffusion loss and related classification loss of the diffusion Transformer model; Based on the outputs of multiple intermediate layers of the diffusion Transformer model, the retrospective feature supervision loss between the target action sequence and the model is calculated. Based on the original action sequence and the target action sequence, frame-level similarity and action signal-to-noise ratio are calculated, and action retention loss is calculated according to the relationship between the action signal-to-noise ratio and a preset threshold and the predicted action sequence. Based on the positive text features, negative text features, and action feature embeddings extracted from the predicted action sequence, the triple semantic alignment loss is calculated; By combining the main diffusion loss, classification loss, retrospective feature supervision loss, action preservation loss, and triple semantic alignment loss of the aforementioned diffusion Transformer model, the model is jointly optimized to obtain a well-trained action editing model. The model application phase includes: The source action sequence to be edited, along with the natural language editing instructions input by the user, are input into the trained action editing model, which then outputs the edited target action sequence.

2. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, Feature extraction of the data includes: The original motion sequence is represented as original motion features in the form of a frame-level time series, wherein the original motion features include global velocity, global orientation, local joint rotation, and local joint position. A pre-trained text encoder is used to extract the semantic features of the positive text instructions as positive text features, and the semantic features of the negative text instructions are extracted as negative text features.

3. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, The cross-modal fusion is specifically performed by using an information fusion module based on the Transformer architecture to interactively calculate the original action features and the positive text features, and output the fused features.

4. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, The preset noise scheduling mechanism is a cosine noise scheduling mechanism.

5. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, The calculation of the retrospective feature supervision loss includes: According to the diffusion Transformer model, multiple intermediate layers are pre-selected, and for each selected intermediate Transformer block, the following operations are performed: a lightweight prediction head is attached after the intermediate layer; the output of the intermediate layer is mapped to the corresponding intermediate action prediction through the lightweight prediction head; the error between the intermediate action prediction and the target action sequence is calculated as the retrospective feature supervision loss component of the intermediate layer. The retrospective feature supervision loss is determined by calculating a weighted sum based on the retrospective feature supervision loss components corresponding to all selected intermediate layers.

6. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 5, characterized in that, The plurality of intermediate layers includes the 2nd, 4th and 6th intermediate layers.

7. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, The calculation of frame-level similarity and action signal-to-noise ratio specifically includes: Using a sliding window mechanism, the frame-level similarity between the original action sequence and the target action sequence in rotation space and position space is calculated, and weighted combination and normalization are performed to obtain the similarity curve in the time dimension. Based on the similarity curve, the frames are sorted according to their similarity values, and the top similarity frame set and the bottom similarity frame set are determined respectively. The ratio of the sum of all elements in the top similarity frame set to the sum of all elements in the bottom similarity frame set is used to obtain the action signal-to-noise ratio.

8. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, The calculation of loss retention specifically includes: When the signal-to-noise ratio of the action is greater than a preset threshold, the corresponding sample is determined to be a high signal-to-noise ratio sample; For the high signal-to-noise ratio sample, the reconstruction error between the predicted action sequence and the original action sequence is calculated and used as the action retention loss; The reconstruction error is the mean square error between the predicted action sequence and the original action sequence in the corresponding frame.

9. The text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, The calculation of the semantic alignment loss of triples is specifically as follows: Calculate the first distance between the action feature embedding and the positive text feature, and the second distance between the action feature embedding and the negative text feature; Based on the difference between the first distance and the second distance, and in conjunction with a preset boundary margin, the semantic alignment loss of the triple is calculated to drive the first distance to be less than the second distance.

10. A text-driven human motion editing method based on omnidirectional positive and negative supervised learning according to claim 1, characterized in that, The joint optimization of the model specifically includes: According to the preset weighting coefficients, the main diffusion loss, classification loss, retrospective feature supervision loss, action retention loss and triple semantic alignment loss are weighted and summed to construct the total loss function; Based on the total loss function, an optimizer is used to perform end-to-end parameter updates on the model.

Images