Digital character motion synthesized video training data generation method and system

Abstract

The application relates to a digital role action synthesized video training data generation method and system. The digital role action synthesized video training data generation method comprises the following steps: writing a rendering task of an action synthesized video into a shared task queue, wherein a combination of film semantic parameters of the rendering task is obtained by values of shooting dimensions conforming to a cinematography specification; making multiple rendering nodes concurrently take rendering tasks from the shared task queue by executing an atomic declaration mechanism; each rendering node executes a rendering work according to a configuration parameter to generate a corresponding frame image sequence; converting the frame image sequence into a video file, packing the video file and a structured annotation record to obtain a training data entry, and obtaining a training data set composed of multiple training data entries. The application solves the problem of poor production efficiency of a synthesized video training data generation system.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for generating training data for digital character motion synthesis videos. Background Technology

[0002] Training AI visual perception models (including human pose estimation, action recognition, video generation, and camera motion prediction) relies on large-scale, high-quality, and diverse datasets of human motion videos. However, datasets based on real-world shooting have significant limitations in terms of acquisition costs, control of shooting conditions, privacy protection, and annotation accuracy. Using synthetic video training data (also known as synthetic human motion datasets) can overcome these limitations. Therefore, synthetic video training data generation technology has become an important research direction in this field of visual perception models. However, existing systems for generating synthetic video training data have relatively low production efficiency. Summary of the Invention

[0003] This application provides a method and system for generating training data for digital character motion synthesis videos, which at least solves the problem of poor production efficiency in the generation system for generating synthesis video training data.

[0004] According to one aspect of this application, a method for generating training data for digital character motion synthesis videos is provided, comprising: The rendering task of motion-synthesized video is written into a shared task queue. The role configuration parameters of the rendering task include a role identifier, a motion capture sequence identifier, and a combination of movie semantic parameters. The combination of movie semantic parameters is constructed by the values ​​of shooting dimensions that conform to the cinematography specifications. The shooting dimensions include camera motion type dimension, shot size dimension, and shooting angle dimension. By executing an atomic declaration mechanism, multiple rendering nodes concurrently receive the rendering tasks from the shared task queue. The atomic declaration mechanism is such that each of the multiple rendering nodes can successfully declare at most one rendering task at the same time, and the rendering tasks declared by the multiple rendering nodes are different from each other. After the declaration is successful, each of the rendering nodes performs rendering work according to the configuration parameters to generate the corresponding frame image sequence; Each of the frame image sequences is converted into a video file, and the video file is packaged with the structured annotation records to obtain training data entries, thereby obtaining a training dataset consisting of multiple training data entries.

[0005] Furthermore, the steps for performing the atomicity declaration mechanism include: In a transaction of the database storing the shared task queue, the rendering node applies a row-level exclusive lock to the target row containing the selected rendering task.

[0006] Furthermore, the method also includes: During transaction scanning, if a row already locked by the row-level exclusive lock is encountered, the locked row is skipped based on the semantics of skipping locked rows, and the scanning continues until an unlocked target row is found; and / or, When the rendering node terminates abnormally, the row-level exclusive lock held by the rendering node is automatically released when the rendering node closes its connection with the database.

[0007] Furthermore, the plurality of rendering nodes are located in a rendering cluster, the rendering cluster being configured with a shared network file system for the plurality of rendering nodes to share predetermined data, and the rendering nodes running a rendering engine; the method further includes: When the rendering cluster starts, the target rendering node writes the shader compilation artifacts to a predetermined storage location in the shared network file system so that other rendering nodes can read the compilation artifacts from the predetermined storage location. The target rendering node is the first node to complete the rendering engine initialization operation.

[0008] Furthermore, the method also includes: After the compiled output has been written to the predetermined storage location, a warm-up completion marker is also written to the predetermined storage location. This ensures that if the warm-up completion marker is detected when other rendering nodes start, the compilation process for the shader is skipped and the compiled output is read; and / or, During the initialization phase of the rendering engine at each rendering node, the temporary compilation file directory of the shaders is redirected to the local storage path of the rendering node.

[0009] Furthermore, the step of the rendering node performing the rendering task according to the configuration parameters includes: Define the calculation rules for the camera physical pose parameters corresponding to the values ​​of each shooting dimension in the movie semantic parameter combination; Based on the camera physical pose parameter calculation rules, geometric inverse kinematics calculation is performed on the combination of movie semantic parameters to obtain shooting parameters. Based on the shooting parameters, the camera object is configured and baked to generate the camera motion keyframes required for the frame image sequence. The shooting parameters include the camera's spatial position coordinates, orientation vector, and focal length.

[0010] Furthermore, the method also includes: The shooting parameters are persistently cached in a predetermined cache location, where the cache value is the shooting parameter and the cache key is the first hash digest of the calculated combination of the movie semantic parameters. Before the rendering node performs the rendering operation: The rendering node calculates a second hash digest of the movie semantic parameter combination of the current rendering task. When the rendering node finds a first hash digest that is the same as the second hash digest in the predetermined cache location, it reads the shooting parameters corresponding to the first hash digest.

[0011] Furthermore, when the rendering node performs rendering work, the method further includes: Load the skeletal network model of the digital character according to the character identifier, and load the motion capture sequence of the digital character according to the motion capture sequence identifier; Obtain pre-configured skeleton correction parameters, which include at least one of pairing layer correction parameters and character layer correction parameters and motion layer correction parameters. The character layer correction parameters are used to adjust the deviation between the skeleton network model and the standard reference skeleton. The motion layer correction parameters are used to eliminate the recording deviation of the motion capture sequence itself. The pairing layer correction parameters are used to correct the distortion problem after the digital character is combined with the motion capture sequence. The skeleton network model and the motion capture sequence are adjusted using skeleton correction parameters to obtain a skeleton driving sequence, so that the rendering node renders the frame image sequence based on the skeleton driving sequence.

[0012] Furthermore, before the rendering node performs the rendering work, the method further includes: A snapshot saving operation is performed on the paired layer correction parameters, and the saved paired layer correction parameters are marked as locked. When the rendering node performs rendering work, if the rendering node detects that the paired layer correction parameters of the digital character are locked, the skeletal network model and the motion capture sequence are adjusted using the paired layer correction parameters from the snapshot; and / or, Before the rendering node renders the frame image sequence based on the bone-driven sequence, the bone-driven sequence is preprocessed. The preprocessing steps include: identifying and deleting the position displacement animation tracks of non-root bones and non-pelvic bone nodes, and / or correcting the rotation interpolation data in the bone animation to eliminate bone flip jitter artifacts.

[0013] Furthermore, the steps for correcting the rotation interpolation data in the skeletal animation include: Traverse the rotation path of each bone joint and perform correction operations sequentially along the rotation path. The correction operation is as follows: calculate the inner product of the quaternions of two adjacent keyframes; when the inner product is negative, invert the quaternion of one of the keyframes to make the inner product positive; and / or, The structured annotation record includes the character identifier, the character information of the digital character, the motion capture sequence identifier, the motion type, the combination of movie semantic parameters, the skeleton correction parameters, the character orientation, and the storage path of the video file.

[0014] Furthermore, the method also includes: Obtain the third hash digest based on the character configuration parameters and the bone correction parameters. If a completed rendering task with the same third hash digest is found in the historical task record library, skip the rendering work for the current rendering task and use the training data entry of the completed rendering task as the output of the current rendering task.

[0015] Furthermore, before writing the rendering task into the shared task queue, the method further includes: When adding motion assets, including motion capture sequences, to the motion asset library, structured text information is extracted from the motion assets. The structured text information is mapped into asset semantic vectors using a language embedding model, and the asset semantic vectors are associated with and stored with the action assets. Upon receiving a natural language query text, the language embedding model is used to map the query text into a query semantic vector, and based on the similarity between the query semantic vector and the asset semantic vector, the target action asset that matches the query semantic vector is returned. The rendering task is created based on the target motion asset, the selected character asset, and the combination of movie semantic parameters.

[0016] Further, the step of returning the target action asset matching the query semantic vector based on the similarity between the query semantic vector and the asset semantic vector includes: Calculate the cosine similarity between the query semantic vector and each of the asset semantic vectors, and select the action assets with a cosine similarity higher than a predetermined value as the target action assets.

[0017] Furthermore, the method also includes: Register a frame update callback function in the rendering engine running on the rendering node; During the rendering process of the rendering node, the rendering engine calls the frame update callback function every time a frame is updated, so as to detect the progress status of the rendering task through the frame update callback function and keep the main event loop of the rendering engine running continuously.

[0018] According to another aspect of this application, a digital character motion synthesis video training data generation system is provided, the digital character motion synthesis video training data generation system being used to perform the digital character motion synthesis video training data generation method.

[0019] The method for generating training data for digital character synthesis videos provided in this application enables multiple rendering nodes to retrieve different rendering tasks from a shared task queue through an atomic declaration mechanism. This achieves conflict-free and non-blocking task retrieval by multiple rendering nodes, avoiding repeated execution of the same rendering task and time consumption caused by individual rendering nodes waiting, significantly improving the generation efficiency of batch synthesis video training data. Simultaneously, it can automatically package video files and structured annotation records into training data entries, eliminating the manual annotation step and achieving end-to-end automation from rendering to dataset construction, enhancing the reusability of the training dataset in downstream multi-task scenarios. Secondly, since the combination of film semantic parameters is constructed based on a descriptive language conforming to cinematography standards for camera motion type, shot size, and shooting angle dimensions, the film semantic parameter combination of the rendering task has clear physical meaning and balanced coverage, ensuring that the generated training dataset has greater diversity, controllability, and interpretability in viewpoint distribution. This avoids the time consumption caused by manual adjustments, improves the quality of the training dataset, and further ensures the production efficiency of the training dataset. Attached Figure Description

[0020] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other embodiments based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a method for generating training data for digital character motion synthesis video according to an embodiment of this application.

[0022] Figure 2 This is a single-node complete rendering process based on the combination of movie semantic parameters provided in one embodiment of this application. Detailed Implementation

[0023] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0024] Currently, the production efficiency of systems that generate synthetic video training data is relatively poor. Research has revealed that existing synthetic human motion dataset generation systems suffer from technical deficiencies in several aspects, hindering the production efficiency and even the quality of synthetic video training datasets: Regarding camera semantic description, existing synthetic video training data generation systems generally employ random sampling or manual specification of parameters such as camera position, focal length, and angle of view. This approach lacks a systematic mapping relationship with the language system of professional film cinematography, resulting in severe randomness and imbalance in the distribution of shooting angles in the generated training dataset. It fails to guarantee balanced coverage of the shot size dimension, nor can it guarantee systematic sampling of the shooting angle and camera motion type dimensions. Consequently, the trained generative model lacks sufficient generalization ability for specific shooting styles, and the training data lacks shooting semantic labels that can be used for downstream tasks.

[0025] In terms of camera parameter calculation, given the target person's body parameters, desired framing, and shooting angle, the optimal camera distance, focal length, and spatial position must be determined using inverse geometry algorithms. This calculation process involves multiple steps of trigonometric geometry operations, with each iteration typically taking several seconds. Existing systems embed this calculation process into the main loop of the rendering engine, resulting in unacceptable cumulative computational latency when generating tens of thousands to hundreds of thousands of training data points in batches.

[0026] In terms of skeletal animation, skeletal animation from motion capture databases (such as AMASS, AIST++, etc.) exhibits systematic joint angle biases during the transfer of digital character models across different skeletal body types. Existing systems lack the ability to independently correct layered biases for "character-level skeletal biases," "action-level skeletal biases," and "character-action pairing layer skeletal biases," resulting in an irreconcilable contradiction between precise control of human posture and data diversity in mass-produced data.

[0027] In distributed rendering task scheduling, when multiple rendering workers simultaneously compete for a shared task queue, if the task declaration operation lacks atomicity guarantees, the same rendering task may be repeatedly claimed by multiple workers, resulting in a serious waste of computing resources. Existing systems typically introduce centralized task queue services, but the centralized scheduling service itself becomes a single point of failure in the system.

[0028] Regarding conflicts in the shared file system of rendering nodes, when multiple rendering nodes are running in a shared network file system environment, the concurrent writing of the 3D rendering engine to the shader temporary directory causes serious file lock contention, which may result in shader compilation failure or even cause the rendering node process to crash and exit.

[0029] In terms of action asset retrieval, the management of large-scale action asset databases relies on precise string matching retrieval. When users describe the desired action in natural language, the existing system cannot perform semantic approximation retrieval, resulting in extremely low asset discovery efficiency. In multilingual scenarios with mixed Chinese and English annotations, the retrieval accuracy is particularly low.

[0030] Regarding the completeness of training data annotation, existing synthetic datasets typically only record action category labels and keypoint coordinates, lacking a structured and complete record of shooting and production parameters, which greatly limits the reusability of training datasets.

[0031] To address the aforementioned issues, the first embodiment of this application provides a method for generating training data for digital character motion synthesis videos. Please refer to [link to relevant documentation]. Figure 1 The generation method includes the following steps: Step S11: Write the motion-synthesized video rendering task into the shared task queue. The rendering task's character configuration parameters include a character identifier, a motion capture sequence identifier, and a combination of film semantic parameters. The film semantic parameter combination is constructed based on the values ​​of shooting dimensions conforming to cinematography standards. The shooting dimensions include camera motion type dimension, shot size dimension, and shooting angle dimension. That is, the film semantic parameter combination is composed of values ​​of camera motion type dimension, shot size dimension, and shooting angle dimension, which are pre-discretely defined according to cinematography standards.

[0032] Here, the so-called cinematography standards refer to the conventional classification system for camera composition, movement, and position in professional cinematography practice. Its core concepts (shot size classification, shooting angle categories, and camera movement naming) have broad consensus in professional cinematography textbooks, industry training systems, and film and television production standards, and are a descriptive language independent of any specific rendering engine or hardware platform. This application maps the aforementioned natural language form of cinematography classification system into discrete coded combinations of film semantic parameters, enabling these combinations to be parameterized by computer systems.

[0033] Step S12: By implementing an atomic declaration mechanism, multiple rendering nodes concurrently receive rendering tasks from the shared task queue. The atomic declaration mechanism ensures that each of the multiple rendering nodes successfully declares at most one rendering task at any given time, and the rendering tasks declared by the multiple rendering nodes are different from each other. This avoids the same rendering task being claimed by multiple workers, avoids repeated execution and time consumption, saves computing resources, and avoids introducing a single point of failure node such as a centralized scheduling service.

[0034] Step S13: After successful declaration, each rendering node performs rendering work according to the configuration parameters to generate the corresponding frame image sequence.

[0035] Step S14: Convert each frame image sequence into a video file, and package the video file with the structured annotation record to obtain training data entries, so as to obtain a training dataset consisting of multiple training data entries.

[0036] Following steps S11 to S14, the method for generating training data for digital character synthesis videos provided in this application can, through the execution of an atomicity declaration mechanism, enable multiple rendering nodes to retrieve different rendering tasks from a shared task queue. This achieves conflict-free and non-blocking task retrieval by multiple rendering nodes, avoiding repeated execution of the same rendering task and the time consumption caused by individual rendering nodes waiting, significantly improving the generation efficiency of batch synthesis video training data. Simultaneously, it can automatically package video files and structured annotation records into training data entries, eliminating the manual annotation step and achieving end-to-end automation from rendering to dataset construction. Furthermore, since the combination of film semantic parameters is constructed based on a descriptive language conforming to cinematography standards for camera motion type, shot size, and shooting angle dimensions, the combination of film semantic parameters for the rendering task has clear physical meaning and balanced coverage. This ensures that the generated training dataset has greater diversity, controllability, and interpretability in viewpoint distribution, avoiding the time consumption caused by manual adjustments, improving the quality of the training dataset, and further ensuring the production efficiency of the training dataset.

[0037] In step S12, the steps for performing the atomicity declaration mechanism include: Within a database transaction storing a shared task queue, a rendering node applies a row-level exclusive lock to the target row containing the selected rendering task. By applying this lock within the database transaction, it ensures that the same rendering task is locked and executed by at most one rendering node at any given time, thus preventing duplicate execution of the same task. Furthermore, since the locking operation of a row-level exclusive lock is performed only at the row level, different rendering nodes can concurrently lock different rows, achieving parallel and non-blocking rendering task acquisition. A database transaction is an atomic logical unit in the execution process of a database management system (DBMS), consisting of a finite sequence of database operations (e.g., one or more SQL statements).

[0038] During transaction scanning, if a row is encountered that is already locked by a row-level exclusive lock, the locked row is skipped based on the semantics of skipping locked rows, and the scanning continues until an unlocked target row is found. This avoids mutual blocking between rendering nodes, realizes a high-concurrency, wait-free task acquisition method, and thus significantly improves the throughput efficiency of rendering tasks.

[0039] When a rendering node terminates abnormally, the row-level exclusive lock held by the rendering node is automatically released when the rendering node closes its connection to the database. In other words, when a rendering node terminates abnormally, the row-level exclusive lock it holds is automatically released when the database connection is closed, without manual intervention. This allows the locked rendering tasks to automatically return to a state where they can be claimed by other normal rendering nodes, thus avoiding permanent task blocking caused by rendering node failure and ensuring continuous production capability and high availability.

[0040] Multiple rendering nodes reside in a rendering cluster, which is configured with a shared network file system for sharing predetermined data among the multiple rendering nodes. Each rendering node runs a rendering engine. The generation method provided in this application further includes the following steps: When the rendering cluster starts up, the target rendering node writes the shader compilation artifacts to a predetermined storage location in the shared network file system, allowing other rendering nodes to read the compiled artifacts from this location. The target rendering node is the first node to complete the rendering engine initialization. This operation eliminates the need for other rendering nodes in the cluster to recompile; they can directly read and reuse the pre-compiled artifacts from the shared storage location. This eliminates the compilation overhead and time consumption of each individual rendering node, significantly shortening the overall startup warm-up time of the rendering cluster and further improving production efficiency. The compiled artifacts are binary machine code files that can be directly executed by accelerated processors (such as GPUs).

[0041] After the compiled artifacts have been written to the predetermined storage location, a warm-up completion flag is also written to the predetermined storage location. This allows other rendering nodes to skip the shader compilation process and read the compiled artifacts when they start up if the warm-up completion flag is detected. This application uses the warm-up completion flag as a signal that compiled artifacts already exist in the predetermined storage location. This ensures that subsequent rendering nodes only skip local compilation and directly reuse the shared compiled artifacts when the compiled artifacts are complete and available. This avoids rendering nodes reading incomplete data or incorrectly executing the compilation process due to incomplete artifacts, thereby ensuring the reliability of shared compiled artifacts and the consistency of the rendering cluster startup.

[0042] During the rendering engine initialization phase of each rendering node, the shader compilation temporary file directory is redirected to the local storage path of the rendering node. This application avoids write contention caused by multiple rendering nodes simultaneously writing to the shared network file system by redirecting the compilation temporary file directory to the local storage path of each rendering node. This redirection can be achieved through file system operations such as mounting bindings or symbolic links, without modifying the shader compilation logic of the rendering engine. The compilation temporary file directory is a dedicated folder used by the rendering engine to store intermediate artifacts and cached results during shader compilation.

[0043] The steps a rendering node takes to perform rendering tasks based on configuration parameters include: Define the calculation rules for the camera physical pose parameters corresponding to the values ​​of each shooting dimension in the combination of movie semantic parameters; Based on the camera's physical pose parameter calculation rules, geometric inverse kinematics is performed on the combination of movie semantic parameters to obtain shooting parameters. Based on these shooting parameters, the camera object is configured and baked to generate the camera motion keyframes required for the frame image sequence. The shooting parameters include the camera's spatial position coordinates, orientation vector, and focal length. Geometric inverse kinematics achieves a fully automated conversion from high-level semantic description to low-level shooting parameters, eliminating the tedious manual frame-by-frame camera setup.

[0044] The structured annotation records include character identifiers, character information of digital characters, motion capture sequence identifiers, motion types, combinations of movie semantic parameters, skeletal correction parameters, character orientation, and the storage path of video files. By including these structured annotation records, machine-readable, multi-dimensional semantic labels are provided for each training data entry sample, enabling the generated training dataset to be directly used for supervised training of various downstream visual models, thus enhancing the reusability of the training dataset.

[0045] The character information includes structured information such as identity, character type, appearance attributes, or body shape characteristics. Action types include movement types, posture types, interaction types, dance types, and sports types. Movement types include walking, running, jumping, and crawling. Posture types include standing, sitting, lying down, and bending over. Interaction types include waving, grabbing, pushing, and pulling. Dance types include dance movements based on musical characteristics. Sports types include kicking a ball, swinging a racket, and throwing.

[0046] In some implementations, the steps for performing inverse geometry calculations may be as follows: First, obtain the scene proportion + the target digital character's height H, then calculate the shooting parameters according to the following camera physical pose parameter calculation rules to obtain the spatial position coordinates by determining the distance the camera should be placed at: d = H / (2 × tan(Fov / 2) × percentage range).

[0047] d is the straight-line distance from the camera to the digital character (along the camera's optical axis), and Fov is the camera's vertical field of view (usually in degrees or radians), determining the vertical range of the camera's field of view. The function tan() is the tangent of the half-vertical field of view, used to convert the angle into a distance ratio, reflecting the degree of vertical spread of the camera's field of view. Proportion: The percentage of the digital character's height in the total height of the frame (e.g., approximately 10%~20% in a wide shot, and over 80% in a close-up), determined by the user-selected shot size. In other words, based on the target's height, shot size proportion, and lens field of view, the appropriate camera placement distance is deduced.

[0048] The camera orientation vector is a three-dimensional vector that describes the direction the camera lens is facing. It is typically represented by three components (X, Y, Z), pointing from the camera position to the target point being filmed (or representing the camera's own orientation). Specifically, the camera's placement and orientation can be determined based on the desired image effect (shot size, shooting angle). Input parameters: target digital character height H (meters), shot size percentage, shooting angle: determines whether the viewpoint is overhead, level, or low-angle, camera field of view (FOV): the rendering engine's preset vertical field of view (e.g., 30°). For example, a medium shot (percentage R=0.5), level shot, character height H=1.7m, FOV=30°, character located at the origin. Camera distance: d=1.7 / (2×tan15°×0.5) =1.7 / (2×0.268×0.5) = 1.7 / 0.268 ≈ 6.34 meters. Camera position: (0, 1.6, 6.34), orientation vector: (0, 1.6, 0) - (0, 1.6, 6.34) = (0, 0, -6.34) → can be normalized to (0, 0, -1).

[0049] When performing inverse geometry calculations, the camera's focal length in the shooting parameters is inferred from the scene size and sensor size. The calculation rules for the camera's physical pose parameters corresponding to the focal length f are as follows: f=( ×d) / d is the distance to the camera. Sensor height, The actual height of the subject in the scene on the plane perpendicular to the optical axis.

[0050] In the general case, given the camera position coordinates Pcam = (0, , d) and the gaze target point T = (0,H×0.93, 0) (H is the height of the digital character, H×0.93 is the eyeline height), the corresponding calculation rule for the camera physical pose parameters for calculating the camera orientation vector is: D = (0, H×0.93 , d) / sqrt((H×0.93 ) 2 + d 2 ) in Let be the camera height obtained by inverse solving from the shooting angle dimension, and d be the camera horizontal distance obtained by inverse solving from the shot size dimension. When the shooting angle is level, = H×0.93, the above equation degenerates into D = (0, 0, 1) The result is consistent with the example above.

[0051] This application can also persistently cache shooting parameters in a predetermined cache location, where the cache value is the shooting parameter and the cache key is the first hash digest of the calculated combination of movie semantic parameters. Before the rendering node performs rendering: The second hash digest of the movie semantic parameter combination of the current rendering task is calculated by the rendering node. When the rendering node finds a first hash digest with the same second hash digest in the predetermined cache location, the shooting parameters corresponding to the first hash digest are read out.

[0052] Therefore, this application uses the first hash digest as the cache key to cache the shooting parameters, so that the shooting parameters corresponding to the same combination of movie semantic parameters do not need to be calculated repeatedly, reducing the number of geometric inverse solution calculations in batch rendering tasks, reducing the cumulative calculation latency, and improving the startup efficiency of rendering tasks.

[0053] When the rendering node finds that no first hash digest with the same second hash digest exists at the predetermined cache location, it performs inverse geometric solving and writes the obtained shooting parameters into the predetermined cache location. In other words, when no first hash digest with the same second hash digest exists, the shooting parameters are obtained by performing inverse geometric solving and the calculation result is written into the predetermined cache location. This achieves dynamic filling of the cached shooting parameters, ensuring that subsequent rendering tasks with the same combination of movie semantic parameters can be directly reused, avoiding repeated calculations in subsequent tasks, and improving the overall processing efficiency in batch rendering scenarios.

[0054] It is important to note that the film semantic parameter combination established in this application is, in essence, a parameterized coding system that directly maps camera semantic descriptions to rules for calculating camera physical pose parameters, rather than a simple information classification and labeling system. Specifically: each discrete value in the shot size dimension (e.g., "medium shot") directly corresponds to a quantized range of the target character's proportion in the frame (e.g., 40% to 55% of the frame height). This proportion range forms a unique mathematical mapping with the physical distance from the camera to the target object through trigonometric tangent relationships, allowing the computer system to automatically perform geometric inverse kinematics calculations of the camera position. The values ​​in the shooting angle dimension directly correspond to the physical angular constraints between the camera's optical axis and the horizontal plane. The values ​​in the camera motion type dimension correspond to a defined 3D keyframe trajectory generation algorithm. Therefore, each discrete coordinate point in the film semantic parameter combination uniquely determines a set of shooting parameter calculation methods.

[0055] For the camera motion type dimension of film semantic parameter combinations: Define the camera's motion along the timeline, covering at least twenty predefined motion types, including static observation, zoom in / out, zoom out, push-in, pull-out, left pan, right pan, left tilt, right pan, tilt up / down, vertical ascent, vertical descent, follow motion, handheld follow, surround view, over-the-shoulder static observation, and first-person movement. Each camera motion type corresponds to a specific camera trajectory generation algorithm, capable of automatically calculating the sequence of keyframe positions for camera motion in the 3D rendered scene.

[0056] For the shot size dimension of film semantic parameter combination: define the proportion of the target character in the frame, and discretize it into six levels according to film cinematography standards: full shot, long shot, medium shot, close-up, extreme close-up, and extreme close-up. Each shot size corresponds to the proportion range of the target character's body on the vertical coordinate of the frame. Taking a medium shot as an example, the area from the knees to the top of the head of the target character should occupy approximately 40% to 55% of the height of the frame.

[0057] For the shooting angle dimension of the combination of semantic parameters in film: define the relative relationship between the camera optical axis and the horizontal plane, and divide it into three types: overhead shot (camera is above the target's line of sight), level shot (camera is at the same height as the target's line of sight), and low shot (camera is below the target's line of sight), and correspondingly define the reference position range of the target character's eye line in the vertical direction of the screen under each shooting angle.

[0058] The discrete sets of values ​​for the three dimensions of film semantic parameters are independent and do not overlap. Each valid parameter combination corresponds to a unique film semantic encoding identifier, which can be used as a semantic annotation field in the structured annotation records of the generated training data entries. This encoding system allows the training dataset to be systematically and evenly sampled according to photographic semantics, improving the generalization ability of downstream visual models to different shooting styles.

[0059] Based on the aforementioned combination of cinematic semantic parameters, this application, given the target character's screen proportion range determined by the shot size parameter, the eyeliner reference position determined by the shooting angle parameter, and the target character's body parameters (height, key joint coordinates), automatically determines the camera's spatial position coordinates, orientation vector, and focal length, among other shooting parameters, using an inverse geometric solution algorithm. To avoid redundant calculations of the same combination of cinematic semantic parameters in batch rendering scenarios, this application persistently caches the aforementioned shooting parameters. The cache key is a first hash digest of the cinematic semantic parameter combination. Upon cache hit, the shooting parameters already present at the predetermined cache location are directly read, skipping the inverse geometric solution calculation step.

[0060] When the rendering node performs rendering work, the generation method provided in this application further includes the following steps: Step S21: Load the skeletal network model of the digital character according to the character identifier, and load the motion capture sequence of the digital character according to the motion capture sequence identifier.

[0061] Step S22: Obtain pre-configured skeleton correction parameters. Skeletal correction parameters include at least one of the following: pairing layer correction parameters, character layer correction parameters, and motion layer correction parameters. Character layer correction parameters are used to adjust the deviation between the skeleton network model and the standard reference skeleton. Motion layer correction parameters are used to eliminate the recording deviation of the motion capture sequence itself. Pairing layer correction parameters are used to correct the distortion problem after the digital character and motion capture sequence are combined.

[0062] Step S23: Adjust the skeletal network model and motion capture sequence using skeletal correction parameters to obtain a skeletal driving sequence, so that the rendering node renders a frame image sequence based on the skeletal driving sequence.

[0063] Through steps S21 to S23, the independent configuration and superposition adjustment of the three layers of skeletal correction parameters (character layer, action layer, and pairing layer) are achieved, thereby improving the quality of skeletal animation reuse and posture consistency across characters and actions.

[0064] Before the rendering node performs the rendering work, the generation method provided in this application also includes the following steps: A snapshot is saved for the paired layer correction parameters, and the saved paired layer correction parameters are marked as locked. When the rendering node performs rendering work, if the rendering node detects that the paired layer correction parameters of the digital character are locked, the paired layer correction parameters in the snapshot are used to adjust the skeletal network model and motion capture sequence.

[0065] Therefore, this application saves the pairing layer correction parameters by taking a snapshot and marking them as locked, so that the corresponding digital character-action combination always uses fixed pairing layer correction parameters in subsequent rendering. Even if the parameters of the character layer or action layer change, it will not affect the skeletal pose of the locked combination, thereby ensuring that the skeletal pose of a specific character-action combination in the batch production data characters is strictly consistent and avoiding data inconsistency caused by parameter misadjustment.

[0066] Before rendering the frame image sequence based on the bone-driven sequence at the rendering node, the bone-driven sequence is preprocessed. The preprocessing steps include: identifying and removing positional displacement animation tracks from non-root bones and non-pelvic bone nodes, and / or correcting the rotation interpolation data in the bone animation to eliminate bone flip jitter artifacts. Here, this application removes unnecessary positional displacement animation tracks, ensuring that global displacement is carried only by the root bone and pelvic bone nodes, conforming to the standard bone-driven specifications for digital characters. Simultaneously, the rotation interpolation data is corrected to eliminate bone flip jitter artifacts. This preprocessing improves the quality and stability of the bone-driven sequence, ensuring that the rendering engine can output smooth and natural character animations.

[0067] The digital character in this application may be at least one of the following: humanoid character, animal character, robot character, etc.

[0068] The steps for correcting rotational interpolation data in skeletal animation include: Traverse the rotation path of each bone joint and perform correction operations sequentially along the rotation path. The correction operation is as follows: calculate the inner product of the quaternions of two adjacent keyframes. When the inner product is negative, invert the quaternion of one of the keyframes to make the inner product positive.

[0069] This application calculates the inner product of quaternions in adjacent keyframes by traversing the rotational orbit of the skeletal joints, and inverts one of the quaternions when the inner product is negative, so that the spherical linear interpolation (SLERP) always follows the shortest path (minor arc), thereby eliminating the instantaneous bone jitter artifact caused by the quaternion hemisphere flip from the root and ensuring the continuity and smoothness of skeletal animation interpolation.

[0070] As can be seen, this application addresses the systematic joint angle deviation problem in cross-bone body shape transfer during motion capture skeletal animation by designing a skeletal correction parameter system composed of pairing layer correction parameters, character layer correction parameters, and motion layer correction parameters. In this application, the joint angle correction amount of each layer of correction parameters is defined in the local coordinate system of each bone node, i.e., with reference to the coordinate system of the parent bone node of the current joint. The superposition of the three layers of correction parameters is achieved by adding rotation offsets in the local coordinate system. The final rotation value applied to the joint is equal to the result of the animation base rotation plus the sum of the correction amounts from each layer, regardless of the specific bone binding implementation. The parent bone node is the joint located "above" the current joint in the bone hierarchy tree and controls it.

[0071] Specifically, the Character Layer correction parameters store global joint angle correction parameters for a specific digital character's skeletal network model. These corrections apply uniformly to all motion capture sequences driven by that digital character. When a digital character's skeletal network model structure exhibits systematic deviations relative to a standard reference skeleton (e.g., a larger shoulder width ratio leading to a deviation in arm resting angle), the corresponding correction amount can be recorded in the Character Layer correction parameters. These correction parameters are effective for all rendering tasks using that digital character.

[0072] For the Motion Layer correction parameters: These parameters store global joint angle correction parameters for a specific motion capture sequence (a continuous, time-sequential segment of human motion data recorded by a motion capture system). These correction parameters apply uniformly to all digital characters using the motion capture sequence. When a motion capture sequence itself has systematic recording biases (such as a bias in the anterior pelvic tilt angle of a specific dataset), the correction amount can be recorded in the Motion Layer correction parameters.

[0073] For the Pair Layer correction parameters under the character-action combination: This layer stores the exclusive joint angle correction parameters for a specific digital character and a specific motion capture sequence combination (referred to as a specific character-action combination). The correction parameters of this layer are only effective for this specific combination of a specific digital character and a specific motion capture sequence.

[0074] The final correction amount applied to the skeletal joints of the skeletal network model in this application is calculated by superimposing the correction parameters of the above three layers. The superposition relationship is: final correction amount = character layer correction parameters + action layer correction parameters + pairing layer correction parameters.

[0075] This application also supports snapshot saving operations on paired layer correction parameters (different versions of paired layer correction parameters are all saved). Once the skeletal fine-tuning parameters of a specific character-action combination have been accurately calibrated and confirmed, a snapshot of the paired layer correction parameters can be recorded and marked as locked. After locking, subsequent changes to the character layer correction parameters or action layer correction parameters will no longer affect the actual rendering parameters of that specific character-action combination. The rendering node will preferentially use the parameter values ​​of the paired layer correction parameters in the locked snapshot, thereby ensuring strict consistency of the skeletal poses of the calibrated combinations in the batch production data.

[0076] Automatic preprocessing of FBX bone tracks: Before the bone drive sequence is applied to the 3D rendering engine, this application performs automatic preprocessing on the input bone animation file containing the bone drive sequence, specifically including: (1) identifying and deleting the position displacement animation tracks of non-root bones and non-pelvic nodes to ensure that the global displacement is only carried by root bones and pelvic nodes, so as to adapt to the standard humanoid bone drive specifications; (2) performing quaternion continuity correction on the rotation interpolation data in the bone animation (the rotation value of the intermediate frame calculated by mathematical algorithm between two known keyframe poses): traversing the rotation track of each bone joint, calculating the inner product (dot product) of the quaternions of adjacent keyframes. When the inner product of the quaternions of adjacent keyframes is negative, it is determined that there is a quaternion hemisphere flipping problem in the keyframe pair. The negative inner product indicates that the rotation of the two keyframes is distributed in different hemispheres of the unit quaternion hypersphere. Directly performing spherical linear interpolation (SLERP) on them will be carried out along the reverse path of the minor arc direction of the rotating sphere, resulting in rapid flipping jitter artifacts in the bones. The correction method is to invert the quaternion of one of the keyframes (e.g., q→). q, because q and q represents the same rotation (inverting it does not change the final orientation of the joint), ensuring that the quaternion inner product of adjacent frames is positive, guaranteeing that SLERP is executed along the minor arc direction, thereby eliminating interpolation jump artifacts. This correction is executed consistently along the track to maintain the continuity of quaternion signs throughout the animation.

[0077] FBX, short for Filmbox, is a universal 3D file format widely used for exchanging data between 3D animation, game engines, and digital content creation software.

[0078] In this application, non-root bones and non-pelvic nodes refer to "all other bones" besides root bones and pelvic nodes, such as: Spine, Left Arm, Right Leg, Head, Left Toe, etc.

[0079] The generation method provided in this application also includes the following steps: Obtain the third hash digest based on the character configuration parameters and bone correction parameters. If a completed rendering task with the same third hash digest is found in the historical task record library, skip the rendering work for the current rendering task and use the training data entry of the completed rendering task as the output of the current rendering task.

[0080] This application queries historical rendering tasks based on a third hash digest. When a completed rendering task is found, the training data entry is directly reused, avoiding the repeated execution of rendering tasks with the same character configuration parameters, eliminating redundant computational overhead, and improving overall production efficiency and resource utilization in batch data generation scenarios. Furthermore, the source of the third hash digest (character configuration parameters and skeleton correction parameters, etc.) can improve the accuracy of the query results. In some embodiments, the source of the third hash digest may also include a scene identifier.

[0081] Before writing the rendering task to the shared task queue, the generation method provided in this application also includes the following steps: Step S31: When adding motion assets including motion capture sequences to the motion asset library, extract structured text information from the motion assets; the structured text information may include name, motion type, text description and music features.

[0082] Step S32: Use a language embedding model to map structured text information into asset semantic vectors, and store the asset semantic vectors in association with action assets.

[0083] Step S33: Upon receiving a natural language query text, a language embedding model is used to map the query text into a query semantic vector, and based on the similarity between the query semantic vector and the asset semantic vector, the target action asset that matches the query semantic vector is returned.

[0084] Step S34: Create a rendering task based on the target motion assets, the selected character assets, and the combination of movie semantic parameters.

[0085] Based on steps S31 to S34 above, this application achieves natural language fuzzy retrieval of action assets by mapping the structured text information of action assets into asset semantic vectors and performing similarity matching based on the query semantic vectors of the query text. The retrieval results can directly drive the batch creation of rendering tasks, completing end-to-end automation from semantic query to synthetic data production, improving the efficiency of action asset discovery and the intelligence of training dataset construction.

[0086] The steps for returning target action assets that match the query semantic vector based on the similarity between the query semantic vector and the asset semantic vector include: calculating the cosine similarity between the query semantic vector and each asset semantic vector, and taking action assets with a cosine similarity higher than a predetermined value as target action assets.

[0087] Therefore, this application finds the target action asset that best matches the query text by calculating the cosine similarity between the query semantic vector and the asset semantic vector and filtering out action assets with similarity higher than a predetermined value, thereby improving the accuracy, flexibility and efficiency of action asset retrieval.

[0088] As can be seen, this application integrates semantic vector retrieval capabilities, supporting fuzzy semantic retrieval of the action asset library using query text in natural language form. During the action asset ingestion phase, structured text information such as the action asset's name, action category, text description, and musical features are mapped into high-dimensional real-valued asset semantic vectors using a language embedding model, and these asset semantic vectors are stored in a database supporting vector similarity retrieval. During the retrieval phase, the query text is mapped into a query semantic vector using the same language embedding model. Approximate nearest neighbor retrieval is performed using cosine similarity calculation, returning a candidate list of action assets most semantically similar to the query text. This retrieval mechanism supports mixed Chinese and English input, and the retrieval results can directly drive the batch creation of rendering tasks, achieving end-to-end automation from semantic query to batch production of synthetic data. The text description can explain details such as the action's style, speed, emotion, and applicable scenarios, enriching the semantic information.

[0089] This application also provides a partitioned collaborative annotation workflow, supporting multiple annotators to perform quality review and semantic annotation on the synthesized video file data in parallel. The data assets are divided into a fixed number of annotation partitions according to a predetermined number of partitions. Each partition supports independent claiming, annotation progress tracking, quality rating and scoring, and secondary review operations. The final evaluation results are written into the structured annotation record for use in downstream training dataset selection.

[0090] In this application, cosine similarity is used to determine the degree of similarity between the query semantic vector and the asset semantic vector by calculating the cosine of the angle between them in the vector space. The closer the value is to 1, the more similar they are. For example, if the query semantic vector is an n-dimensional vector A and the asset semantic vector is an n-dimensional vector B, the cosine similarity cosθ = (A·B) / (|A| ×|B|). Where: A·B is the dot product; |A| is the magnitude of vector A; and |B| is the magnitude of vector B.

[0091] The generation method provided in this application also includes the following steps: Register frame update callback functions in the rendering engine running on the rendering node. During the rendering work performed by the rendering node, the rendering engine calls the frame update callback function every time a frame is updated, so as to detect the progress status of the rendering task and keep the main event loop of the rendering engine running continuously.

[0092] This application employs a mechanism where the 3D rendering engine calls frame update callback functions to drive the rendering task state machine, rather than using thread sleep or blocking. The frame update callback function is automatically invoked every frame the rendering engine updates, and the rendering engine's main event loop continues to run throughout the entire rendering task execution, enabling timely processing of completion signals and status callbacks from the rendering subprocesses. This mechanism avoids UI freezing or task interruption due to prolonged rendering node occupation, improving the responsiveness and stability of the rendering process, thereby further ensuring a significant increase in productivity.

[0093] The main event loop is the core driving mechanism of rendering engines (such as Unreal Engine, Unity, Blender, etc.). It is an infinite loop (usually running on the main thread) responsible for continuously and sequentially processing operating system window messages, user input, timers, network events, etc., and driving the state updates and rendering of the entire engine.

[0094] In one embodiment, this application processes the rendered frame image sequence into final training data entries through two independent stages. The first stage, a post-processing worker (an independently running process or service instance), is responsible for synthesizing the frame image sequence into a video file according to preset video encoding parameters. The second stage, an export worker, is responsible for packaging the video file with complete structured annotation records to generate training data entries that conform to the training dataset format requirements. The two processing stages are handled in parallel by independent worker processes. The second-stage export worker utilizes a database advisory lock mechanism to implement distributed locking; the advisory lock is automatically released when the worker process terminates abnormally, supporting automatic recovery of the export task.

[0095] Video encoding parameters are a pre-defined set of configuration options that control video quality, file size, compression efficiency, and playback compatibility when compressing the original rendered frame sequence (such as PNG images) into the final video file. These parameters determine the final video's clarity, smoothness, file size, and decoding compatibility. As can be seen from the above, the overall technical approach of this application is as follows: A three-dimensional independent combination of cinematic semantic parameters, consisting of camera motion type, shot size, and shooting angle dimensions, is constructed within a digital asset management platform. The core technology of this combination lies in the fact that each value in each dimension directly corresponds to a defined rule for calculating camera physical parameters. This allows the synthetic video training data generation system to automatically perform inverse geometric calculations (a process of deducing camera parameters from the "final image effect") based on any semantically encoded coordinates, obtaining uniquely determined camera spatial positions (e.g., starting point (X=0, Y=1.2, Z=2.5) → ending point (X=0, Y=1.3, Z=1.5)), orientation (starting elevation angle 15° → ending elevation angle 25°), and focal length (50mm), among other shooting parameters. This achieves fully automatic conversion from cinematographic semantic descriptions to camera physical pose parameters. Furthermore, inverse geometric calculation results with the same combination of cinematic semantic parameters can be persistently cached (i.e., the shooting parameters are cached). The motion capture skeletal animation is processed by applying joint angle corrections to the skeletal parameters at three independent levels: character layer, motion layer, and character-motion pairing layer. Independent management and version snapshots of the correction parameters for each layer are supported. 3D rendering is performed concurrently via a distributed rendering cluster based on an atomic declaration mechanism, while isolating the shader compilation temporary file directories of each rendering node to eliminate shared memory contention. The rendered frame image sequence is converted into video files through a two-stage post-processing pipeline and associated with complete film semantic parameters, skeletal adjustment parameters, and character motion metadata to generate a structured training dataset where each record is an independent row.

[0096] Compared with the prior art, this application has at least the following beneficial effects: 1. By constructing a combination of movie semantic parameters consisting of three independent dimensions—camera motion type, shot size, and shooting angle—the shooting perspective coverage of the synthetic training dataset is transformed from random distribution to systematic and balanced coverage. Furthermore, complete structured annotation records are automatically appended to each training data entry, thereby enhancing the reusability of the training dataset in downstream multi-task scenarios.

[0097] 2. Through a persistent caching mechanism for shooting parameters, shooting parameters configured for the same camera can be reused across multiple rendering tasks within the same batch. The geometric inverse kinematics calculation of shooting parameters typically takes over 7 seconds per run. In batch scenarios producing 100,000 training data points, cache reuse can reduce the number of configuration calculations by approximately 90%, saving over 200 CPU hours of total configuration calculation time. This significantly improves the overall throughput efficiency of the digital character motion synthesis video training data generation system. The CPU refers to the central processing unit.

[0098] 3. By using a three-layer skeleton correction parameter and snapshot locking mechanism, this application enables the correction of skeleton joint angles to be applied and managed independently under the correction parameters of the three dimensions of character layer, action layer and pairing layer. It supports flexible and differentiated posture adjustment without destroying the consistency of the calibrated parameters, effectively solving the contradiction between mass production and precise posture control.

[0099] 4. By employing an atomic declaration mechanism based on database row-level locks, the security of rendering task declarations in a multi-rendering node concurrent environment is guaranteed, eliminating the risk of rendering tasks being executed repeatedly. Simultaneously, the automatic release mechanism of row-level exclusive locks enables automatic task recovery without manual intervention after abnormal termination of a rendering node. Compared to existing solutions that introduce centralized task queue services such as Celery and Ray, this application eliminates the need to deploy additional scheduling service processes, removing the single point of failure of the centralized scheduler itself and reducing the operational complexity of the distributed rendering cluster.

[0100] 5. By isolating the temporary compilation file directory of the shaders of the rendering nodes and sharing the preheating mechanism of the compilation artifacts, this application eliminates the problem of concurrent write contention among multiple rendering nodes in a shared network file system environment, and significantly shortens the overall startup time of the rendering cluster with multiple rendering nodes, thereby reducing the operational complexity of the distributed rendering infrastructure.

[0101] 6. By using a two-stage distributed post-processing pipeline and outputting structured annotation records in JSON line format, this application parallelizes the generation of video files for rendered frame image sequences with the packaging of training data entries, and provides each training data entry with a machine-readable structured annotation record containing complete semantic parameters, skeleton adjustment amounts, and metadata, thereby improving the end-to-end efficiency of the data production pipeline.

[0102] 7. Through semantic vector retrieval mechanism and partitioned collaborative annotation workflow, this application supports the rapid discovery and batch rendering of large-scale action assets driven by natural language queries, and provides a complete workflow for multi-person parallel annotation and quality review, which improves the production and quality management efficiency of large-scale synthetic training datasets.

[0103] The aforementioned technical features generate synergistic effects at the system level that exceed the sum of their individual components: the combination of cinematic semantic parameters under the 3D independent parameter encoding system provides a structured parameter enumeration space for rendering tasks, resulting in an order-of-magnitude improvement in the hit rate of rendering configuration hash deduplication in batch production scenarios. The independence of the three-layer skeletal correction parameters ensures that once the paired layer correction parameters are locked by snapshots, the skeletal parameters do not become invalid with adjustments to the character layer correction parameters or the action layer correction parameters, thus guaranteeing the consistency of skeletal poses in the historical rendering results reused by rendering configuration hash deduplication. The synergistic effect of the batch task creation driven by semantic vector retrieval and the atomic declaration mechanism enables the entire process from user semantic input to large-scale concurrent rendering completion to run stably without human intervention. The combination of these three features achieves, for the first time in a synthetic training data production scenario, a unified 3D optimization of cinematographic semantic annotation, precise skeletal pose control, and large-scale batch production efficiency.

[0104] Based on the above embodiments, a second embodiment of this application provides a specific implementation method to further illustrate the specific implementation process of the digital character motion synthesis video training data generation method: Example 1: Complete Single-Node Rendering Process Based on Movie Semantic Parameter Combination like Figure 2 As shown below, the technical process of a single rendering node executing a complete rendering task is explained.

[0105] Step S101: The digital asset management platform receives a rendering task creation request. The role configuration parameters of the rendering task include a role identifier, a motion capture sequence identifier, and a combination of movie semantic parameters consisting of camera motion type dimension, shot size dimension, and shooting angle dimension.

[0106] Step S102: The rendering node calculates the second hash digest of the movie semantic parameter combination based on the target digital character shape parameters (loaded from the character asset library) corresponding to the movie semantic parameter combination and the character identifier, and checks whether a matching record exists in the persistent pre-defined cache location. If the cache is hit, the shooting parameters are read directly. If the cache is not hit, geometric inverse kinematics calculation is performed to calculate the camera's spatial position coordinates, orientation vector, focal length, and other shooting parameters, and the shooting parameters are written to the pre-defined cache location.

[0107] Step S103: Calculate the third hash digest of the complete character configuration parameters for this rendering task, and check if there is a matching completed rendering task in the historical rendering task records. If a match exists, skip steps S104 to S109, directly associate with the historical rendering result (training data entry), and jump to step S110. If no match exists, continue with the subsequent steps.

[0108] Step S104: Create a rendering task record in the shared task queue database with a row-level exclusive lock. The initial state of the rendering task is the pending declaration state.

[0109] Step S105: The worker process on the rendering node atomically declares a rendering task to be executed from the shared task queue by executing a transaction with row-level exclusive lock queries using "skip locked rows" semantics, and updates the rendering task status to rendering in progress.

[0110] Step S106: The worker process loads the 3D scene, replaces the digital character model in the 3D scene with the character specified by the character identifier, loads the corresponding motion capture sequence, and sequentially applies the bone joint angle correction parameters of the character layer correction parameters, motion layer correction parameters, and character-motion pairing layer correction parameters to generate the corrected bone drive sequence.

[0111] Step S107: Based on the shooting parameters and trajectory algorithm corresponding to the camera motion type determined in step S102, the worker process configures the camera objects in the 3D scene and bakes the camera motion keyframe sequence.

[0112] Step S108: The worker process triggers the execution of the 3D rendering pipeline by registering the rendering engine frame update callback function. The frame update callback function detects the progress status of the rendering task during each frame update. The main event loop of the rendering engine continues to run throughout the entire rendering task execution.

[0113] Step S109: After rendering is complete, the worker process submits the output frame image sequence to the two-stage post-processing queue, and the task status is updated to "Post-processing in progress." The post-rendering worker then composites the frame image sequence into a video file, and the rendering task status is updated to "Completed."

[0114] Step S110: Generate structured annotation records for this rendering task. The structured annotation records include character identifiers, character information, action type, movie semantic parameter combinations (camera motion type, shot size, shooting angle), three-layer skeleton correction parameters, character orientation, and video file storage path. The structured annotation records are written to the metadata file of the training dataset in JSON line format.

[0115] Example 2: Concurrent Rendering with Multiple Rendering Nodes and Distributed Task Scheduling The following describes the distributed scheduling process for multiple rendering nodes to execute rendering tasks concurrently.

[0116] Step S201: When the rendering cluster starts, the first rendering node to complete the initialization of the 3D rendering engine (including shader compilation) writes the compiled shader output to the derived data cache directory of the predetermined storage location in the shared network file system. After writing, it writes a preheating completion marker file to the predetermined storage location in the shared network file system.

[0117] Step S202: During the rendering engine initialization phase, each rendering node redirects the shader compilation temporary file directory of the rendering engine to a local storage path that is independent and private for this rendering node through file system binding mount or symbolic link operation, so as to ensure that the shader compilation temporary file writing operations of each rendering node are isolated from each other.

[0118] Step S203: When the preheating completion marker is detected at startup, all rendering nodes except the first rendering node skip the local shader compilation process and directly read the compilation artifacts from the derived data cache directory.

[0119] Step S204: After each rendering node completes initialization, it starts its own independent worker process. Each worker process simultaneously initiates a task declaration transaction with the shared task queue database using the "skip locked rows" semantic. Each transaction acquires a row-level exclusive lock for at most one rendering task. Multiple concurrent requests from rendering nodes are resolved at the database level, and each rendering task is declared successfully by at most one worker on a rendering node.

[0120] Step S205: If a rendering node terminates abnormally due to process crash or system failure, the connection between the rendering node and the database will be closed, the row-level exclusive lock it holds will be automatically released, the corresponding rendering task will be restored to a state that can be claimed by other rendering nodes, and the remaining normally running rendering node worker processes will automatically take over the rendering task in the next rendering task declaration cycle without manual intervention.

[0121] Step S206: The worker processes of each rendering node independently execute the rendering and post-processing flow from steps S106 to S110 in Example 1. The rendering task status updates are all written to the shared task queue database, and the rendering output frame image sequence of each rendering node is written to its own local output path.

[0122] Step S207: The export worker performs batch export jobs on the rendered tasks that have been rendered through the database query lock mechanism, summarizes the video files distributed in the local output paths of each rendering node, and packages them with the structured annotation records to generate complete training data entries. This generates a JSON row-format metadata file containing all training data entries (the field structure includes the identifier of the rendering task, character identifier, camera motion type, shot size, shooting angle, skeleton correction parameters, character orientation, quality rating, video file storage path, etc.).

[0123] Example 3: Three-layer skeleton fine-tuning calibration and snapshot locking process The following describes the calibration and production application process of skeletal fine-tuning parameters.

[0124] Step S301: Load a specific digital character skeletal network model in the 3D preview interface, apply angle corrections to the joints of the digital character model's skeletal network model such as the shoulder, arm, and leg using the joint angle adjustment controls, adjust the systematic deviation between the digital character and the standard reference skeleton, and save the correction parameters as the character layer correction parameters for the character (such as shoulder width deviation, arm resting angle, etc. correction amount).

[0125] Step S302: Load a specific motion capture sequence in the 3D preview interface, apply global angle correction to joints such as anterior pelvic tilt and spinal curvature in the motion capture sequence through the joint angle adjustment control, eliminate the systematic recording deviation of the motion capture sequence itself, and save the correction parameters as the motion layer correction parameters of the motion capture sequence.

[0126] Step S303: Simultaneously load the combination of a specific digital character and a specific motion capture sequence in the 3D preview interface. The system automatically overlays the character layer correction parameters from step S301 and the motion layer correction parameters from step S302. By observing the rendered preview, identify clipping or pose distortion issues specific to this character-motion combination. Apply exclusive correction parameters to this specific character-motion combination using the joint angle adjustment controls, and save them as the paired layer correction parameters for this specific character-motion combination.

[0127] Step S304: After confirming that the pairing layer correction parameters in step S303 have achieved the expected rendering effect for a specific character-action combination, perform a snapshot saving operation on the pairing layer correction parameters. Store the current parameter values ​​as a version snapshot and set the locking status of the pairing to locked. The snapshot saving operation allows the selection and use of an earlier historical version of the pairing layer correction parameters, which is not the most recent modification, at any subsequent point in time.

[0128] Step S305: In subsequent rendering tasks, when executing step S106, the system checks the locking status of the paired layer correction parameters of the target character-action combination. If it is a locked state, the paired layer correction parameter values ​​recorded in the snapshot are directly used as the bone correction amount to ensure strict consistency of the calibrated bone poses in mass production.

[0129] Step S306: When the character layer correction parameters or motion layer correction parameters change due to overall calibration adjustment, the locked paired layer correction parameter snapshots are not affected, and the rendering bone parameters of the target character-motion combination remain unchanged, thereby avoiding the disruption of pose consistency between batches.

[0130] Example 4: Natural Language Query Text Retrieval Drives Batch Training Data Production The following describes the process of retrieving motion assets using natural language and creating rendering tasks in batches.

[0131] Step S401: When motion assets are added to the database, the system extracts fields such as name, motion category, text description and music features (beat type, energy level, tempo, etc.) for each motion asset, concatenates them into structured text information, calculates the asset semantic vector of the structured text information through a pre-trained language embedding model, and stores the asset semantic vector in the semantic vector index in association with the motion asset record.

[0132] Step S402: The user inputs natural language query text (such as "walking slowly" or "light Latin dance steps"). The query text is mapped to query semantic vectors through the same language embedding model. Approximate nearest neighbor retrieval is performed on the semantic vector index based on cosine similarity, and a list of candidate action assets with the most similar semantics is returned. Mixed Chinese and English input is supported.

[0133] Step S403: The user selects the target motion asset from the motion asset candidate list and specifies the character asset list and the range of movie semantic parameters (such as specifying two shot types: medium shot and close-up shot, two shooting angles: level shot and low shot, and two camera movement types: push-in shot and follow shot).

[0134] Step S404: The system creates rendering task records in batches for all legal combinations of the selected action assets, character assets and movie semantic parameters. It performs hash digest calculation on the character configuration parameters of each rendering task to obtain a third hash digest, automatically skips character configuration parameters that are duplicated with historically completed rendering tasks, and writes the remaining rendering tasks into the shared task queue.

[0135] Step S405: The distributed rendering cluster executes batch rendering tasks in parallel according to the concurrent scheduling process described in Example 2. After completion, the export worker generates an export package of training data entries containing complete structured annotation records. Each record in the metadata file of the training dataset contains structured text information from step S401.

[0136] Example 5: Implementation of an Alternative Rendering Engine Based on the Blender Python API for 3D Computer Graphics Software The following describes an alternative technical solution for implementing the core method of this application in the Blender rendering engine, proving that the technical solution of this application is independent of a specific rendering engine.

[0137] Step S501: The system constructs a 3D scene through the scripted 3D rendering engine interface, imports the skeletal mesh model of the target digital character, and imports the corresponding skeletal animation sequence from the motion asset library.

[0138] Step S502: The system applies character layer correction parameters, action layer correction parameters, and pairing layer correction parameters to each bone level of the bone mesh model through the bone constraint interface provided by the rendering engine, according to the bone fine-tuning mechanism of Embodiment 3. The superposition relationship of the three correction parameters is consistent with the implementation in Embodiments 1 to 4, and is independent of the bone binding method of the specific rendering engine.

[0139] Step S503: Perform inverse geometric calculation on the combination of movie semantic parameters to obtain shooting parameters (camera spatial position coordinates, orientation vector and focal length), set the camera spatial pose through the camera programming interface of the rendering engine, and generate motion trajectory sequences corresponding to various camera motion types through the keyframe interpolation interface.

[0140] Step S504: The system triggers the rendering pipeline to execute through the batch frame rendering interface of the rendering engine, and outputs a frame image sequence file. The atomicity mechanism declaration and state management of the rendering task are implemented through the same shared task queue as described in Example 2. Rendering workers of different rendering engines share the same task scheduling layer, and the scheduling mechanism is independent of the rendering engine.

[0141] Step S505: The rendered frame image sequence is synthesized into a video file and a structured annotation record is generated through a two-stage post-processing pipeline. The structured annotation record format is fully compatible with the formats in Examples 1 to 4, and the downstream training dataset consumer does not need to be aware of the specific type of rendering engine.

[0142] The construction and application of the cinematic semantic parameter combination in this application are not limited to the Unreal Engine rendering engine. The three-dimensional independent parameter encoding system for camera motion type, shot size, and shooting angle can be implemented on any 3D rendering engine that supports programmable camera control (including but not limited to Unity, Blender, Arnold, etc.). The specific set of values ​​for the camera motion type dimension can be expanded or reduced according to the target application scenario, as long as it contains at least two independent parameter dimensions and the set of values ​​for each dimension is pre-defined by discretization according to cinematography standards.

[0143] The parameter hierarchy of skeletal correction parameters is not limited to a three-layer implementation. The skeletal layer correction concept in this application can be implemented as a two-layer system (character layer correction parameters and pairing layer correction parameters, omitting action layer correction parameters) or extended to a four-layer system (adding scene layer correction).

[0144] The hash algorithm used for the deduplication mechanism of the configured rendering tasks and the caching of shooting parameters is not limited to SHA-256. Any deterministic hash function with collision resistance (including SHA-3, BLAKE2, etc.) can be used instead. The combination of input parameters for hashing can add additional dimensions according to actual needs (such as scene lighting parameters, background asset identifiers, etc.), as long as the hash value can uniquely identify the complete configuration of a rendering task.

[0145] Distributed atomicity declaration mechanisms are not limited to row-level exclusive lock implementations in relational databases. Any data storage system that provides equivalent semantics (including Redis's SETNX mechanism, ZooKeeper's ephemeral nodes, etcd's transactional writes, etc.) can be used as an alternative implementation, as long as the two core constraints of "the same rendering task can be declared successfully by at most one rendering node" and "the task automatically recovers to an claimable state after the holder terminates abnormally" are met.

[0146] The language embedding model used for action asset retrieval is not limited to a locally deployed specific model; any pre-trained language model capable of mapping natural language text to a fixed-dimensional real-valued vector space can be used instead. The semantic vector index storage layer is not limited to pgvector; any database or vector retrieval engine that supports vector similarity retrieval (including Milvus, Weaviate, FAISS, etc.) can be used instead.

[0147] The third embodiment of this application also provides a system for generating training data for digital character motion synthesis videos. This system is used to execute a method for generating training data for digital character motion synthesis videos. For the specific process of the method for generating training data for digital character motion synthesis videos, please refer to the content provided in the first embodiment of this application; it will not be repeated here.

[0148] In some embodiments, the generation system provided in this application may include a semantic encoding module, a camera parameter calculation module, a skeleton layer-by-layer fine-tuning module, a distributed rendering scheduling module, and a structured annotation generation module. The semantic encoding module is used to construct a combination of film semantic parameters based on a descriptive language conforming to cinematography standards, encompassing camera motion type, shot size, and shooting angle dimensions. The distributed rendering scheduling module is connected to the semantic encoding module and is used to enable multiple rendering nodes to concurrently receive rendering tasks from a shared task queue by executing an atomic declaration mechanism. The camera parameter calculation module is connected to the rendering nodes and is used to perform inverse geometric calculations on the film semantic parameter combination to obtain shooting parameters. The structured annotation generation module is connected to the semantic encoding module and is used to generate structured annotation records. The skeleton layer-by-layer fine-tuning module is used to adjust the skeleton network model and motion capture sequence using skeleton correction parameters to obtain a skeleton-driven sequence.

[0149] It should be noted that the term "comprising" and its variations used in the embodiments of this invention are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "a plurality" mentioned in the embodiments of this invention are illustrative and not restrictive, and those skilled in the art should understand that unless explicitly indicated otherwise in the context, they should be understood as "one or more".

[0150] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0151] The steps described in the method embodiments provided by the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0152] The term "embodiment" in this specification refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily imply the same embodiment, nor does it imply independence or alternativeity from other embodiments. The various embodiments in this specification are described in a related manner, with reference to each other for similar or identical parts. In particular, for apparatus, device, and system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, and relevant details are referred to in the description of the method embodiments.

[0153] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for generating training data for digital character motion synthesis videos, characterized in that, include: The rendering task of motion-synthesized video is written into a shared task queue. The role configuration parameters of the rendering task include a role identifier, a motion capture sequence identifier, and a combination of movie semantic parameters. The combination of movie semantic parameters is constructed by the values ​​of shooting dimensions that conform to the cinematography specifications. The shooting dimensions include camera motion type dimension, shot size dimension, and shooting angle dimension. By executing an atomic declaration mechanism, multiple rendering nodes concurrently receive the rendering tasks from the shared task queue. The atomic declaration mechanism is such that each of the multiple rendering nodes can successfully declare at most one rendering task at the same time, and the rendering tasks declared by the multiple rendering nodes are different from each other. After the declaration is successful, each of the rendering nodes performs rendering work according to the configuration parameters to generate the corresponding frame image sequence; Each of the frame image sequences is converted into a video file, and the video file is packaged with the structured annotation records to obtain training data entries, thereby obtaining a training dataset consisting of multiple training data entries.

2. The method for generating training data for digital character motion synthesis video according to claim 1, characterized in that, The steps for performing the atomicity declaration mechanism include: In a transaction of the database storing the shared task queue, the rendering node applies a row-level exclusive lock to the target row containing the selected rendering task.

3. The method for generating training data for digital character motion synthesis video according to claim 2, characterized in that, The method further includes: During transaction scanning, if a row already locked by the row-level exclusive lock is encountered, the locked row is skipped based on the semantics of skipping locked rows, and the scanning continues until an unlocked target row is found; and / or, When the rendering node terminates abnormally, the row-level exclusive lock held by the rendering node is automatically released when the rendering node closes its connection with the database.

4. The method for generating training data for digital character motion synthesis video according to any one of claims 1 to 3, characterized in that, The multiple rendering nodes are located in a rendering cluster, the rendering cluster is configured with a shared network file system for the multiple rendering nodes to share predetermined data, and the rendering nodes run a rendering engine. The method further includes: When the rendering cluster starts, the target rendering node writes the shader compilation artifacts to a predetermined storage location in the shared network file system so that other rendering nodes can read the compilation artifacts from the predetermined storage location. The target rendering node is the first node to complete the rendering engine initialization operation.

5. The method for generating training data for digital character motion synthesis video according to claim 4, characterized in that, The method further includes: After the compiled output has been written to the predetermined storage location, a warm-up completion marker is also written to the predetermined storage location. This ensures that if the warm-up completion marker is detected when other rendering nodes start, the compilation process for the shader is skipped and the compiled output is read; and / or, During the initialization phase of the rendering engine at each rendering node, the temporary compilation file directory of the shaders is redirected to the local storage path of the rendering node.

6. The method for generating training data for digital character motion synthesis video according to claim 1, characterized in that, The steps for the rendering node to perform rendering work according to the configuration parameters include: Define the calculation rules for the camera physical pose parameters corresponding to the values ​​of each shooting dimension in the movie semantic parameter combination; Based on the camera physical pose parameter calculation rules, geometric inverse kinematics calculation is performed on the combination of movie semantic parameters to obtain shooting parameters. Based on the shooting parameters, the camera object is configured and baked to generate the camera motion keyframes required for the frame image sequence. The shooting parameters include the camera's spatial position coordinates, orientation vector, and focal length.

7. The method for generating training data for digital character motion synthesis video according to claim 6, characterized in that, The method further includes: The shooting parameters are persistently cached in a predetermined cache location, where the cache value is the shooting parameter and the cache key is the first hash digest of the calculated combination of the movie semantic parameters. Before the rendering node performs the rendering operation: The rendering node calculates a second hash digest of the movie semantic parameter combination of the current rendering task. When the rendering node finds a first hash digest that is the same as the second hash digest in the predetermined cache location, it reads the shooting parameters corresponding to the first hash digest.

8. The method for generating training data for digital character motion synthesis video according to claim 1, characterized in that, When the rendering node performs rendering work, the method further includes: Load the skeletal network model of the digital character according to the character identifier, and load the motion capture sequence of the digital character according to the motion capture sequence identifier; Obtain pre-configured skeleton correction parameters, which include at least one of pairing layer correction parameters and character layer correction parameters and motion layer correction parameters. The character layer correction parameters are used to adjust the deviation between the skeleton network model and the standard reference skeleton. The motion layer correction parameters are used to eliminate the recording deviation of the motion capture sequence itself. The pairing layer correction parameters are used to correct the distortion problem after the digital character is combined with the motion capture sequence. The skeleton network model and the motion capture sequence are adjusted using skeleton correction parameters to obtain a skeleton driving sequence, so that the rendering node renders the frame image sequence based on the skeleton driving sequence.

9. The method for generating training data for digital character motion synthesis video according to claim 8, characterized in that, Before the rendering node performs the rendering work, the method further includes: A snapshot saving operation is performed on the paired layer correction parameters, and the saved paired layer correction parameters are marked as locked. When the rendering node performs rendering work, if the rendering node detects that the paired layer correction parameters of the digital character are locked, the skeletal network model and the motion capture sequence are adjusted using the paired layer correction parameters from the snapshot; and / or, Before the rendering node renders the frame image sequence based on the bone-driven sequence, the bone-driven sequence is preprocessed. The preprocessing steps include: identifying and deleting the position displacement animation tracks of non-root bones and non-pelvic bone nodes, and / or correcting the rotation interpolation data in the bone animation to eliminate bone flip jitter artifacts.

10. The method for generating training data for digital character motion synthesis video according to claim 9, characterized in that, The steps for correcting rotational interpolation data in skeletal animation include: Traverse the rotation path of each bone joint and perform correction operations sequentially along the rotation path. The correction operation is as follows: calculate the inner product of the quaternions of two adjacent keyframes; when the inner product is negative, invert the quaternion of one of the keyframes to make the inner product positive; and / or, The structured annotation record includes the character identifier, the character information of the digital character, the motion capture sequence identifier, the motion type, the combination of movie semantic parameters, the skeleton correction parameters, the character orientation, and the storage path of the video file.

11. The method for generating training data for digital character motion synthesis video according to claim 8, characterized in that, The method further includes: Obtain the third hash digest based on the character configuration parameters and the bone correction parameters. If a completed rendering task with the same third hash digest is found in the historical task record library, skip the rendering work for the current rendering task and use the training data entry of the completed rendering task as the output of the current rendering task.

12. The method for generating training data for digital character motion synthesis video according to claim 1, characterized in that, Before writing the rendering task to the shared task queue, the method further includes: When adding motion assets, including motion capture sequences, to the motion asset library, structured text information is extracted from the motion assets. The structured text information is mapped into asset semantic vectors using a language embedding model, and the asset semantic vectors are associated with and stored with the action assets. Upon receiving a natural language query text, the language embedding model is used to map the query text into a query semantic vector, and based on the similarity between the query semantic vector and the asset semantic vector, the target action asset that matches the query semantic vector is returned. The rendering task is created based on the target motion asset, the selected character asset, and the combination of movie semantic parameters.

13. The method for generating training data for digital character motion synthesis video according to any one of claims 1 to 3, characterized in that, The method further includes: Register a frame update callback function in the rendering engine running on the rendering node; During the rendering process of the rendering node, the rendering engine calls the frame update callback function every time a frame is updated, so as to detect the progress status of the rendering task through the frame update callback function and keep the main event loop of the rendering engine running continuously.

14. A system for generating training data for digital character motion synthesis videos, characterized in that, The digital character motion synthesis video training data generation system is used to execute the digital character motion synthesis video training data generation method according to any one of claims 1 to 13.

Images