Digital human generation method based on video generation large model

Abstract

A digital human generation method based on a video generation large model includes five steps: S1: receiving an original video signal from a video data source, performing time sequence frame decomposition and key action extraction operations on the original video signal, and forming a preprocessed signal containing human posture features and facial expression features; S2: inputting the preprocessed signal into a pre-trained video generation large model, and generating an initial digital human motion trajectory signal and an appearance rendering signal through the space-time attention mechanism of the model; S3: fusing the initial digital human motion trajectory signal and a preset voice driving signal, and generating a lip shape synchronous control signal by using a cross-modal alignment module; S4: generating a high-real-sense digital human video stream signal by using an adaptive lighting rendering engine according to the appearance rendering signal and the lip shape synchronous control signal; and S5: outputting the high-real-sense digital human video stream signal to a display terminal, simultaneously generating a dynamic detail enhancement feedback signal, and iteratively optimizing the parameter weight of the video generation large model. The digital human generation method based on the video generation large model can solve the problems that it is difficult to eliminate the experience dependence of manual adjustment and optimization, and the die casting parameters cannot be accurately self-optimized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence digital human generation technology, specifically to a digital human generation method based on a large video generation model. Background Technology

[0002] Traditional digital human generation primarily relies on 3D modeling and motion capture technologies, requiring specialized equipment to collect facial marker data and driving the skeletal model through inverse kinematics algorithms. This method has significant limitations: high equipment costs hinder widespread adoption, marker occlusion leads to data loss, and manually binding skeletal weights is time-consuming and prone to joint clipping issues. While single-image-based reconstruction methods reduce hardware dependence, the generated results lack temporal coherence and struggle to represent the micro-expression changes required for natural dialogue. Recent advancements in neural rendering technology, utilizing generative adversarial networks to synthesize facial images, have made progress in static image quality, but dynamic video generation still faces issues such as stiff expressions and abnormal blinking frequencies. Voice-driven solutions typically employ simple phoneme-to-view mapping, ignoring the non-linear impact of speech rate changes on lip shape and failing to consider the muscle movement differences in dialect pronunciation. Existing video generation models often exhibit physical inconsistencies when handling full-body movements, such as fingers appearing in the background and clothing overlapping, and insufficient lighting adaptation results in a stiff integration of virtual digital humans with real-world scenes. Furthermore, existing methods lack continuous optimization mechanisms, failing to improve quality based on feedback after the initial generation. Patent documents propose digital human animation methods based on keyframe interpolation, but the interpolation algorithms result in overly mechanical movements; while deep learning is used to generate lip-sync animation, the problem of matching facial expressions with the emotional nuances of speech remains unresolved. These technical shortcomings limit the effectiveness of digital humans in scenarios such as remote conferencing and virtual live streaming. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a digital human generation method based on a large video generation model, which solves the problems of stiff movements, distorted expressions, and asynchronous speech and lip movements in existing digital human generation methods. This invention extracts spatiotemporal features from real-life videos using a large video generation model, and establishes a cross-modal mapping relationship between movements, expressions, and speech: First, it decomposes video frames to extract human posture and micro-expression parameters; then, it uses a pre-trained model to generate basic digital human motion trajectories; finally, it combines speech signals with cross-modal alignment to achieve precise lip-sync; then, it uses a physically based lighting model to render skin and hair details; and finally, it continuously optimizes the generation effect through a dynamic feedback mechanism to achieve end-to-end generation of highly realistic digital humans.

[0004] This invention provides a digital human generation method based on a large video generation model, comprising: S1: Receive the raw video signal from the video data source, perform time-series frame decomposition and key action extraction on the raw video signal to form a preprocessed signal containing human posture features and facial expression features. S2: Input the preprocessed signal into the pre-trained video generation model, and generate the initial digital human motion trajectory signal and appearance rendering signal through the model's spatiotemporal attention mechanism; S3: Integrates the initial digital human motion trajectory signal with the preset voice driving signal, and uses the cross-modal alignment module to generate lip-sync control signal; S4: Based on the appearance rendering signal and the lip-sync control signal, a high-fidelity digital human video stream signal is generated through the adaptive lighting rendering engine; S5: Outputs a high-fidelity digital human video stream signal to the display terminal, while generating dynamic detail enhancement feedback signals and iteratively optimizing the parameter weights of the large video generation model.

[0005] In one embodiment of the present invention, the temporal frame decomposition operation includes the process of establishing a dynamic topological structure of key points of the human skeleton, extracting motion differential features between consecutive frames through a convolutional long short-term memory network, mapping non-rigid deformation parameters to latent space representation vectors, and using an adversarial training strategy to eliminate jitter noise interference from the video acquisition device. Finally, a motion coding sequence with spatiotemporal consistency is constructed as the core component of the preprocessed signal. This process also includes a robust correction mechanism for regions with sudden changes in illumination.

[0006] In one embodiment of the present invention, the large video generation model includes a hierarchical cascaded generator architecture. The first-level generator generates a basic human geometric mesh based on pose features. The second-level generator synthesizes skin details at the pore level through a texture mapping network. The third-level generator introduces a physics engine to simulate dynamic fabric wrinkles. The output of each generator is residually connected to the signal of the previous stage. At the same time, a gated recurrent unit is used to supervise the coherence of details across frames. The training process of the model includes adversarial regularization constraints based on perceptual loss.

[0007] In one embodiment of the present invention, the cross-modal alignment module performs a joint embedding operation of speech phonemes and temporal expression units, constructs a phoneme-view position probability transfer matrix as the alignment basis, calculates the phase offset between the speech signal and the video signal through a dynamic time warping algorithm, generates continuous control parameters for the opening and closing of the corners of the mouth and the position of the tongue using differentiable rendering technology, and establishes a nonlinear mapping relationship between the emotion intensity coefficient and the activation intensity of the facial action unit.

[0008] In one embodiment of the present invention, the adaptive lighting rendering engine performs real-time calculation of ambient occlusion, uses spherical harmonic function to decompose the scene light source distribution characteristics, dynamically adjusts the subsurface scattering coefficient according to the material properties of the digital human, performs Monte Carlo sampling simulation of corneal refraction light path for the eye area, and uses anisotropic bidirectional scattering distribution function for beam ray tracing for the hair area, and finally outputs a rendering signal that conforms to the laws of physical optics.

[0009] In one embodiment of the present invention, the dynamic detail enhancement feedback signal includes a motion blur compensation sub-signal and a micro-expression enhancement sub-signal. The former generates a motion trajectory compensation map through motion vector field estimation, while the latter uses a facial motion coding system to mark unconventional expression frames and improve the texture resolution of the corresponding regions. The feedback signal adjusts the convolution kernel weight distribution of the large video generation model through the backpropagation path, and the optimization process adopts a teacher-student distillation architecture to achieve knowledge transfer.

[0010] In one embodiment of the present invention, the texture mapping network implements multi-layer material separation rendering, decomposing the skin surface into three physical layers: the epidermis, the dermis, and the subcutaneous tissue. The epidermis uses a Fresnel reflection model to handle the oil gloss effect, the dermis uses a blood vessel distribution map to control the blood color diffusion parameters, and the subcutaneous tissue simulates the surface deformation fluctuations caused by muscle contraction. The outputs of each layer are rendered and fused using a ray travel algorithm.

[0011] In one embodiment of the present invention, the nonlinear mapping relationship is used to encode the emotional intensity through a spiking neural network, which transforms the fundamental frequency variance of the speech signal into an emotional pulse sequence. A specific combination of facial action units is driven by a synaptic weight matrix. For joy, the contraction signal of the zygomaticus major muscle and the vibration frequency of the orbicularis oculi muscle are enhanced. For anger, the activation intensity of the depressor supercilii muscle and the tremor component of the orbicularis oris muscle are enhanced.

[0012] In one embodiment of the present invention, the anisotropic bidirectional scattering distribution function includes a hair axis offset angle parameterization module, calculates the light penetration depth based on the hair follicle density distribution map, simulates the multiple scattering effect between hair strands using the Kelvin-Voyt model, optimizes the light projection in the spiral coordinate system for curly hair, and finally generates a physically realistic rendering result containing specular stripes and edge halos.

[0013] In one embodiment of the present invention, the volume rendering fusion process adopts a deep learning-accelerated ray sampling strategy, constructs an octree structure to store the medium density features of the space, reduces invalid ray tracing paths through importance sampling, performs a neural network approximation calculation of Monte Carlo integral for semi-transparent tissue regions, and verifies the material realism of the fusion result through an adversarial discriminator.

[0014] This invention provides a digital human generation method based on a large video generation model. The method extracts spatiotemporal features from real-life videos using a large video generation model and establishes a cross-modal mapping relationship between actions, expressions, and speech. First, video frames are decomposed to extract human posture and micro-expression parameters. A pre-trained model is used to generate basic digital human motion trajectories. Combined with speech signals, cross-modal alignment is used to achieve precise lip-sync. A physically based lighting model is used to render skin and hair details. Finally, a dynamic feedback mechanism is used to continuously optimize the generation effect, achieving end-to-end generation of highly realistic digital humans. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating a method for generating digital humans based on large video models. Figure 2 This is a schematic diagram illustrating the steps of a digital human generation method based on video-generated large models. Detailed Implementation

[0017] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0018] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0019] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0020] Please see Figure 1-2The image shows the digital human generation method based on a large video generation model according to the present invention. The digital human generation method based on a large video generation model of the present invention includes five steps: S1: Receive the raw video signal from a video data source, perform temporal frame decomposition and key action extraction operations on the raw video signal to form a preprocessed signal containing human posture features and facial expression features; S2: Input the preprocessed signal into a pre-trained large video generation model, and generate an initial digital human motion trajectory signal and an appearance rendering signal through the model's spatiotemporal attention mechanism; S3: Fuse the initial digital human motion trajectory signal with a preset voice-driven signal, and generate a lip-sync control signal using a cross-modal alignment module; S4: Generate a high-fidelity digital human video stream signal through an adaptive lighting rendering engine based on the appearance rendering signal and the lip-sync control signal; S5: Output the high-fidelity digital human video stream signal to a display terminal, while simultaneously generating a dynamic detail enhancement feedback signal and iteratively optimizing the parameter weights of the large video generation model.

[0021] like Figure 1As shown, a core closed-loop process for digital human generation based on a large video generation model is constructed, the technical essence of which is reflected in five progressive signal conversion stages. The first stage acquires the mold's identity feature signal through a mold identification device. This signal is essentially a digital carrier uniquely identifying the mold, and can take the form of electromagnetic encoding from an RFID chip, the geometric topological matrix of a QR code, or the curvature feature value of a 3D visual point cloud. After the identity feature signal is transmitted to the central controller via the industrial bus, it triggers the second stage parameter matching mechanism: the central controller parses the mold classification identification code in the signal, indexes the best historical records with the same identification code in the initial parameter database, and generates an initial mold adjustment parameter signal containing the target value of clamping force, the set value of mold capacity, and the set value of ejection position. Here, the database adopts a tree-like index structure, with mold tonnage as the main branch and gate type as the sub-branch to achieve millisecond-level retrieval. The third stage converts the parameter signals into physical actions: After receiving the signal, the servo drive device decomposes it into a pulse sequence to drive the servo motor, which in turn drives the moving template displacement through the ball screw transmission system. Simultaneously, real-time monitoring by the tie rod strain sensor and displacement sensor is initiated. The former converts the micro-strain of the tie rod into a voltage signal through a Wheatstone bridge and then calibrates it into a clamping force feedback signal. The latter generates a template position signal through grating ruler moiré fringe counting. Both types of signals are transmitted back to the central controller at a frequency of thousands of times per second. The fourth stage enables dynamic compensation: The central controller's built-in digital comparator performs difference calculations between the clamping force feedback signal and the target value to generate a clamping force error signal. At the same time, it compares the template position signal with the setpoint of the capacity modulus using a sliding window to generate a position error signal. The dual-path error signal is input into the adaptive control algorithm (such as variable gain PID or fuzzy RBF neural network). The algorithm dynamically adjusts the proportional-integral-derivative weights according to the error change rate and outputs a closed-loop compensation signal containing velocity feedforward and torque compensation components. This signal is injected back into the servo drive device to form a closed loop. The fifth stage achieves system self-evolution: During trial die casting, the injection curve acquisition module captures the slow injection filling curve and the fast injection pressure peak through a pressure transmitter and a magnetostrictive displacement sensor, synthesizing a three-dimensional process curve signal of time-pressure-speed; the visual inspection device uses a multi-angle linear array camera to acquire images of the casting surface, and identifies defects such as flash and shrinkage cavities through edge detection and morphological operations to generate quality evaluation signals; the central controller maps the process curve stability coefficient and quality defect level to parameter correction quantities through a weighted fusion algorithm (such as entropy weight TOPSIS), generates parameter optimization command signals to update the database, so that the initial parameters of the next mold adjustment of the same mold automatically approach the optimal solution. The whole process forms an intelligent closed loop of "identity recognition → parameter matching → execution feedback → dynamic compensation → data evolution", compressing the traditional mold adjustment time of 3 hours to within 20 minutes and increasing the first-piece pass rate to over 90%.

[0022] Furthermore, in the S1 stage, 3D machine vision technology is introduced to achieve high-precision mold authentication. Specifically, a sinusoidal fringe grating is projected onto the mold parting surface using a structured light projector. After a binocular camera acquires the deformed fringe image, a phase unwrapping algorithm is used to reconstruct 3D point cloud data of the mold surface with millimeter-level precision. This point cloud data constitutes the core carrier of the identity feature signal. After receiving the point cloud data, the central controller performs triple feature extraction: first, the parting surface region in the point cloud is segmented using a region growing algorithm, and its normal vector inclination angle and area are calculated using least squares plane fitting; second, the cylindricity parameters and center coordinates of the gate sleeve point cloud cluster are detected based on Hough transform; finally, the curvature histogram is used to statistically analyze the geometric complexity of the cavity surface. The extracted feature vectors are matched with a pre-stored mold topology database: the topology database uses a graph database to store the relationships between nodes (such as ejector pin holes, cooling channels) and edges (such as distances, angles) of historical molds. The matching process uses a graph isomorphism algorithm to calculate the similarity between the current feature vector and the templates in the database. When the similarity exceeds a dynamic threshold (initially set at 85%, adaptively adjusted with the number of learning iterations), the central controller activates the mold's access to the initial parameter database. In special cases, if neither the new mold's point cloud nor the records in the database reach the threshold, the system initiates an incremental database building mode: after guiding the operator to input basic parameters to complete the first round of mold adjustment, the 3D point cloud and the final optimized parameters are automatically stored in the topology database and an associated index is established. This solution overcomes the limitations of traditional ID recognition; even if the mold nameplate is damaged or the RFID tag is detached, accurate authentication can still be achieved through physical features, keeping the identification error rate below 0.1%.

[0023] In one embodiment of the invention, stage S2 proposes a dynamic parameter generation mechanism based on case-based reasoning. The initial parameter database is essentially a structured collection of successful historical mold adjustment cases. Each case contains a four-dimensional data set: mold identity feature code, die-casting machine model fingerprint, mold adjustment parameter set (clamping force gradient value, capacity compensation coefficient, etc.), and production effect score (calculated based on scrap rate and equipment wear). After receiving the identity feature signal, the central controller performs multi-level retrieval: the first round filters similar cases using the mold classification code as the key; the second round sorts cases by die-casting machine model similarity, with model similarity calculated by comparing the characteristic values ​​of the stiffness matrix of the mold clamping mechanism; finally, the clamping force data of the top 5% of high-scoring cases is extracted to construct a probability distribution model. The generation of the clamping force gradient distribution adopts the kernel density estimation method: using the case clamping force value as a sample point, a continuous probability density curve is generated using a Gaussian kernel function as the weight, and the density peak interval is taken as the recommended clamping force target value range. The calculation of the ejection displacement compensation coefficient considers the mold thermal deformation effect: the coordinate difference of the ejector pin position in the case study under cold mold and thermal steady state is extracted, and a mapping relationship between the compensation coefficient and mold quality is established through linear regression. The dynamic generation process introduces real-time constraints: the clamping force safety threshold is calculated based on the rigidity parameters of the current die-casting machine model (such as tie rod yield strength and template flatness). If the gradient distribution recommended in the case study exceeds the threshold, the penalty function method is used to compress the distribution range; simultaneously, the hydraulic system oil temperature is monitored, and if the temperature is lower than the preheating standard, the mold capacity setting value is automatically increased by 0.5%-1.2% to reserve thermal expansion margin. The final output initial mold adjustment parameter signal is actually a multi-dimensional vector.

[0024] Furthermore, the S3 stage innovatively deploys a dual closed-loop control architecture, the essence of which lies in upgrading the traditional open-loop mold adjustment to a multi-level feedback precision control system. The inner loop control of the servo drive device focuses on the nanometer-level accuracy of the moving template displacement trajectory: the absolute encoder built into the servo motor provides real-time feedback of the rotor angular displacement signal, which is decomposed into fundamental and harmonic components through elliptic Fourier transform; after the harmonic components are eliminated by a notch filter, the remaining fundamental signal is differentially compared with the capacitive modulus setting value in the initial mold adjustment parameter signal to generate a motor torque compensation command that is directly injected into the current loop, so that the actual displacement deviation of the moving template is controlled within ±5 micrometers. The outer loop control focuses on the stability of the clamping force: the strain rosette sensor group attached to the four tie rods collects multi-directional micro-strain, and the signal conditioning circuit uses a temperature self-compensated Wheatstone bridge to convert the strain value into a differential voltage; the central controller fuses the four voltage signals through Kalman filtering and removes equipment vibration noise, and then calculates the clamping force in real time based on the elastic modulus and cross-sectional area parameters of the tie rods, and finally outputs the clamping force feedback signal. To overcome the uneven loading of the template caused by the asynchronous movement of the Togolin columns, a four-channel grating ruler matrix layout is adopted for the displacement sensors: magnetic grating linear encoders are installed at the four corners of the moving template to monitor the displacement values ​​at each position in real time; the central controller calculates the diagonal position difference to generate a synchronization deviation signal. When any diagonal deviation exceeds 10 micrometers, a pressure redistribution algorithm is triggered, which adjusts the q-axis current component in the servo motor vector control to balance the force on the Togolin columns. This dual closed-loop architecture decomposes the template adjustment process into two orthogonal dimensions: displacement precision control and force balance. While ensuring the accuracy of the capacity setting, it reduces the clamping force fluctuation rate from ±15% to ±1.8%.

[0025] like Figure 1As shown, stage S4 proposes a hybrid adaptive algorithm combining fuzzy rules and proportional-integral-differential parameter self-tuning. Its core lies in transforming manual mold-adjusting experience into a mathematical decision-making model. The fuzzy rule base is constructed based on an expert experience matrix: the universe of discourse for the clamping force error signal is defined as [-1000kN, 1000kN], divided into seven fuzzy subsets: "negative large," "negative medium," "negative small," "zero," "positive small," "positive medium," and "positive large." The proportional-integral-differential parameter self-tuning module takes the derivative of the position error signal as input, and its universe of discourse is divided into five subsets: "rapidly moving away," "slowly moving away," "stable," "slowly approaching," and "rapidly approaching." When the clamping force error signal falls into the "negative large" range, the fuzzy rule base outputs a 0.9-level pressure compensation weight coefficient, instructing the system to prioritize increasing the clamping cylinder pressure. If the position error derivative is simultaneously detected as "rapidly moving away," the proportional-integral-differential parameter self-tuning module increases the proportional gain by 40% and compresses the integral time constant to 30% of its original value, generating a compensation signal for a high-speed approximation command. The fusion of the two types of instructions adopts a membership-weighted method: the activation intensity of the pressure compensation weight output by the fuzzy rules and the velocity feedforward instruction generated by the proportional-integral-derivative (PID) module are calculated separately, and the final closed-loop compensation signal is generated by defuzzifying using the centroid method. This algorithm innovatively introduces a bivariate collaborative decision-making mechanism in the field of die casting mold adjustment, enabling the system to focus on rapid position tracking in the initial stage of mold capacity adjustment (PID-dominated), and switch to fine control of clamping force (fuzzy rule-dominated) when approaching the set value, thus reducing the overall mold adjustment time by 37%.

[0026] Specifically, the S5 phase proposes a dynamic weight allocation mechanism for parameter optimization based on defect type. Its core technology lies in establishing a causal mapping relationship between casting quality defects and mold adjustment parameters. After receiving real-time process curve signals and casting quality evaluation signals, the central controller initiates multi-dimensional fusion analysis: first, feature extraction is performed on the process curve signals to identify the frequency of metal flow peak fluctuations during the slow injection stage and the pressure overshoot at the end of the fast injection stage; simultaneously, defect classification is performed on the quality evaluation signals, quantifying typical defects such as flash, cold shuts, and porosity according to their severity. The fusion process employs a weighted decision engine, which has a built-in expert knowledge graph storing the association rules between different defect patterns and mold adjustment parameters. When the vision inspection device detects flash defects and the flash thickness exceeds the safety threshold, the decision engine automatically increases the weight coefficient of the clamping force target value in the parameter optimization command signal to the highest priority. At this time, the system generates a clamping force strengthening command, driving the clamping force target value to increase by 3%-8% in the next mold adjustment. If the process curve signal shows that there are insufficient filling characteristics in the slow injection stage (such as the front peak speed being more than 15% lower than the standard value), the decision engine determines that the mold capacity setting deviation causes the injection start point to lag, and immediately increases the compensation range of the mold capacity setting value, adjusting the mold capacity benchmark value down by 0.5-1.2 mm through a reverse compensation algorithm. When porosity defects and injection pressure overshoot occur simultaneously, the engine reduces the ejection position weight and activates the boost pressure optimization channel. The entire decision-making process adopts fuzzy confidence evaluation and initiates a voting arbitration mechanism for contradictory multi-source signals to ensure the scientific nature of the parameter optimization command signal. Actual production line verification shows that this solution reduces the secondary occurrence rate of flash defects from 35% to 1.2%, and the scrap rate caused by insufficient filling decreases by 90%.

[0027] like Figure 2As shown, a continuous evolution mechanism for constructing the initial parameter database enables the system to continuously self-optimize as production data accumulates. The parameter optimization command signal triggers the incremental learning process of the database: the central controller encapsulates key data from the current mold adjustment process into a sample set, including the actual set values ​​of the initial mold adjustment parameter signals, the cumulative effect of the closed-loop compensation signal, the convergence time of the error signal from triggering to zero, and the defect index of the casting quality evaluation signal. The sample set is then transmitted to the cloud training platform after being bound to the mold's identity characteristics via timestamps. The training platform uses a support vector machine regression model for data analysis. This model uses the clamping force gradient distribution in historical samples as the input vector and the reduction in scrap rate in actual production as the optimization objective. It searches for the optimal segmentation hyperplane in high-dimensional space through kernel function transformation. Model updates focus on two core parameters: first, the recommended clamping force range for similar molds, dynamically expanding or compressing the original range based on the correlation between the actual clamping force value and the quality score in the new samples; second, the capacity compensation benchmark value, establishing a nonlinear mapping table between the compensation value and the mold tonnage by analyzing the effectiveness of thermal compensation under different ambient temperatures. The updated model parameters are fed back to the field-level initial parameter database, and the latest recommended values ​​are prioritized when adjusting the mold in the following text. To prevent overfitting, the system includes a sample validity verification step: when the error convergence time of a new sample exceeds twice the standard deviation of the historical mean, the learning weight of that sample is automatically frozen. This mechanism enabled the system to improve the hit rate of the first round of mold adjustment parameters from 72% to 98% after 50 iterations of learning.

[0028] Furthermore, a mold life protection system is introduced during the mass production stage to solve the problem of abnormal mold wear in traditional die casting production. After mold adjustment and entering mass production mode, the central controller continuously collects the clamping force fluctuation signal, and obtains the force value oscillation waveform by sampling 2000 times per second through high-frequency dynamic strain sensors installed on four gate pillars. The signal processing unit performs a fast Fourier transform on the original waveform to extract the energy distribution characteristics in the 0-100Hz frequency domain, paying particular attention to the resonance peak changes in the 10-25Hz range. The mold protection engine compares the current fluctuation energy spectrum with the historical baseline spectrum in real time. When it detects that the energy increase in a specific frequency band (such as 18Hz) continuously exceeds the fatigue limit threshold of the mold material, it determines that the mold is at risk of plastic deformation. At this point, the system executes a three-level response: the primary response generates an early warning signal and displays the stress concentration location on the human-machine interface; if the fluctuating energy does not decay within 10 minutes, the intermediate response automatically reduces the injection speed by 20% to reduce the impact load; when the energy increase exceeds the red warning line, the advanced response immediately activates the parameter rollback mechanism, retrieving the optimal mold adjustment parameter signal from the mold's dedicated historical database (usually the parameter set from the last 100,000 consecutive mold runs without failure), and forcibly overwriting the current working parameters. The rollback process employs seamless switching technology: during the interval between two mold closing operations, key parameters such as the clamping force target value and the mold capacity setting value are reloaded to avoid production line downtime. Simultaneously, the system records the process status during abnormal fluctuations (such as aluminum liquid temperature and injection speed curves), generating a mold health diagnosis report for maintenance reference. This solution reduces the accidental mold damage rate by 80% and extends the average lifespan to 450,000 mold runs.

[0029] This invention discloses a digital human generation method based on a large video model. It automatically matches historical optimal parameters with mold identity features to drive a servo mechanism to perform mold adjustment; it collects clamping force and position signals in real time to generate dual-path errors, and dynamically compensates for the motion trajectory through an adaptive algorithm; after trial die casting, it integrates process curves and casting quality signals to generate parameter optimization instructions to update the database, forming a closed loop of "recognition-execution-compensation-learning". This allows the mold adjustment accuracy to break free from the constraints of human experience and realize the autonomous evolution of die casting parameters.

[0030] Therefore, the digital human generation method based on video-generated large models of the present invention can solve the problem of difficulty in eliminating the dependence on manual mold adjustment experience and the inability to achieve accurate self-optimization of die casting parameters.

[0031] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for generating digital humans based on a large video generation model, characterized in that, include: S1: Receive the raw video signal from the video data source, perform time-series frame decomposition and key action extraction on the raw video signal to form a preprocessed signal containing human posture features and facial expression features. S2: Input the preprocessed signal into the pre-trained video generation model, and generate the initial digital human motion trajectory signal and appearance rendering signal through the spatiotemporal attention mechanism of the model; S3: The initial digital human motion trajectory signal is fused with the preset voice driving signal, and a lip-sync control signal is generated using the cross-modal alignment module; S4: Based on the appearance rendering signal and the lip-sync control signal, generate a high-fidelity digital human video stream signal through the adaptive lighting rendering engine; S5: Output the high-fidelity digital human video stream signal to the display terminal, and at the same time generate a dynamic detail enhancement feedback signal and iteratively optimize the parameter weights of the video generation model.

2. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The temporal frame decomposition operation includes the process of establishing a dynamic topological structure of key points of the human skeleton, extracting motion differential features between consecutive frames through a convolutional long short-term memory network, mapping non-rigid deformation parameters to latent space representation vectors, and using an adversarial training strategy to eliminate jitter noise interference from the video acquisition device. Finally, a motion coding sequence with spatiotemporal consistency is constructed as the core component of the preprocessed signal. This process also includes a robust correction mechanism for regions with sudden changes in illumination.

3. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The video generation model comprises a hierarchical cascaded generator architecture. The first-level generator generates a basic human geometric mesh based on pose features. The second-level generator synthesizes skin details at the pore level through a texture mapping network. The third-level generator introduces a physics engine to simulate dynamic fabric wrinkles. The output of each generator is residually connected to the signal of the previous stage. At the same time, a gated recurrent unit is used to supervise the coherence of details across frames. The training process of the model includes adversarial regularization constraints based on perceptual loss.

4. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The cross-modal alignment module performs joint embedding operations of speech phonemes and temporal expression units, constructs a phoneme-view position probability transfer matrix as the alignment basis, calculates the phase offset between speech signals and video signals through a dynamic time warping algorithm, generates continuous control parameters for mouth corner opening and tongue position using differentiable rendering technology, and establishes a nonlinear mapping relationship between the emotion intensity coefficient and the activation intensity of facial action units.

5. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The adaptive lighting rendering engine performs real-time calculation of ambient occlusion, uses spherical harmonic functions to decompose the scene light source distribution characteristics, dynamically adjusts the subsurface scattering coefficient according to the material properties of the digital human, performs Monte Carlo sampling simulation of corneal refraction light path for the eye area, and uses anisotropic bidirectional scattering distribution function for beam ray tracing for the hair area, and finally outputs a rendering signal that conforms to the laws of physical optics.

6. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The dynamic detail enhancement feedback signal includes a motion blur compensation sub-signal and a micro-expression enhancement sub-signal. The former generates a motion trajectory compensation map through motion vector field estimation, while the latter uses a facial motion coding system to mark unconventional expression frames and improve the texture resolution of the corresponding regions. The feedback signal adjusts the convolution kernel weight distribution of the large video generation model through the backpropagation path. The optimization process adopts a teacher-student distillation architecture to achieve knowledge transfer.

7. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The texture mapping network implements multi-layer material separation rendering, decomposing the skin surface into three physical layers: the epidermis, the dermis, and the subcutaneous tissue. The epidermis uses a Fresnel reflection model to handle the oil gloss effect, the dermis uses a blood vessel distribution map to control the blood color diffusion parameters, and the subcutaneous tissue simulates the surface deformation fluctuations caused by muscle contraction. The outputs of each layer are rendered and fused using a ray-traveling algorithm.

8. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The nonlinear mapping relationship is used to encode emotional intensity through a spiking neural network, which transforms the fundamental frequency variance of the speech signal into an emotional pulse sequence. The synaptic weight matrix drives a specific combination of facial action units. For joy, it strengthens the contraction signal of the zygomaticus major muscle and the vibration frequency of the orbicularis oculi muscle, while for anger, it enhances the activation intensity of the depressor supercilii muscle and the tremor component of the orbicularis oris muscle.

9. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The anisotropic bidirectional scattering distribution function includes a hair axis offset angle parameterization module, calculates the light penetration depth based on the hair follicle density distribution map, simulates the multiple scattering effect between hair strands using the Kelvin-Voyt model, optimizes the light projection in the spiral coordinate system for curly hair, and finally generates a physically realistic rendering result containing specular stripes and edge halos.

10. The digital human generation method based on a large video generation model according to claim 1, characterized in that, The volumetric rendering fusion process employs a deep learning-accelerated ray sampling strategy, constructs an octree structure to store the medium density features, reduces invalid ray tracing paths through importance sampling, performs Monte Carlo integral neural network approximation calculations for semi-transparent tissue regions, and verifies the material realism of the fusion results through an adversarial discriminator.

Images