A method for rapid switching and reuse of digital human images

Abstract

The present invention discloses a method for rapid switching and reuse of digital human images, which relates to the field of digital human technology. The method includes: pre-constructing a digital human image template parameter file, and aligning the key points of different digital human images to a standard reference coordinate system through an affine transformation matrix; mapping the key point data of a driving source to the coordinate system of a target digital human image to generate a target key point set; utilizing a resource cache mechanism to manage the texture and template data of the digital human image, and reducing resource loading delays during switching through a cache reuse strategy; based on the target key point set, synthesizing digital human image frames through a layered rendering engine, mapping the key point data of the same driving source to multiple digital human images through differentiated parameter fine-tuning, and independently controlling the skeletal deformation ratio and expression amplitude of each image. The present application solution can significantly reduce switching delays, improve resource reuse rates, and support multi-role parallel driving and voice-synchronized lip generation.

Description

Technical Field

[0001] The present invention relates to the field of digital human technology, and in particular to a method for rapid switching and multiplexing of digital human images. Background Art

[0002] With breakthroughs in generative artificial intelligence and real-time rendering technology, digital human technology has gradually penetrated into fields such as virtual live broadcasting, online education, and customer service, becoming an important vehicle for human-computer interaction. Its core goal is to achieve high-fidelity image generation, low-latency interaction, and personalized adaptation. However, existing technologies still face significant bottlenecks in multimodal data processing, real-time optimization, and privacy protection. For example, traditional digital human systems rely on high-precision three-dimensional modeling and motion capture, resulting in low generation efficiency and huge consumption of computing resources. The system needs to repeatedly load large-volume models and textures during each switch, and the data volume of a single high-precision digital human model can reach several GB, resulting in frame rate drops, excessive memory usage, and switching lags, making it difficult to achieve real-time interaction. At the same time, the driving mechanism based on fixed templates cannot meet the needs of diverse scenarios, and personalized expression is limited. Therefore, based on the above difficulties, the present invention proposes a method for rapid switching and reuse of digital human images. Summary of the Invention

[0003] Purpose of the Invention

[0004] To address the above-mentioned issues, the present invention aims to provide a method for rapid switching and reuse of digital human images. This method achieves modular reuse of digital human resources through template standardization and a multi-layer architecture, significantly reducing duplicate data processing and improving the efficiency of switching between different character images. Furthermore, the dynamic fault-tolerant strategy introduced ensures the continuity and stability of the switching process, avoiding issues such as freezes and interruptions that may occur during switching, significantly improving the reliability and real-time performance of digital human image switching.

[0005] Technical Solution

[0006] To achieve the above objectives, the present invention provides a method for rapidly switching and reusing digital human images. By standardizing digital human image templates, preconfiguring mapping parameters for multiple roles, and applying a multi-layer mechanism, this method enables rapid replacement of digital human images and component reuse. Furthermore, a dynamic fault-tolerance strategy is introduced to ensure the stability and continuity of the switching process. Specifically, the method establishes a digital human base template with a unified skeletal structure, with each preset digital human role corresponding to a set of mapping parameters. Using this base template and mapping parameters, a digital human image is constructed using a multi-layer architecture. When a role switch is required, the mapping parameters of the target role are called to update the relevant layers of the base template, thereby rapidly switching the current digital human image to the target role. Resource loading is monitored during the switching process, and if loading fails or delays occur, a fault-tolerance strategy is automatically executed to ensure a smooth switching process.

[0007] In a first aspect, the present invention provides a method for rapidly switching and reusing digital human images, comprising:

[0008] Pre-constructing a digital human image template parameter file in a unified format, which includes a set of facial key feature point coordinates, basic texture mapping data, and local material layering information, and aligning the key points of different digital human images to a standard reference coordinate system through an affine transformation matrix;

[0009] Map the key point data of the driving source to the coordinate system of the target digital human image to generate a target key point set;

[0010] Use resource caching mechanisms to manage the texture and template data of digital human images, and use cache reuse strategies to reduce resource loading delays during switching;

[0011] Based on the target key point set, a layered rendering engine is used to synthesize digital human image frames, and key point sequence smoothing and anomaly compensation strategies are combined to ensure the continuity and stability of the animation. The layered rendering engine divides the digital human face into multiple independent layers, and each layer is synthesized into a complete image according to a preset occlusion order and transparency blending rules, and supports the rapid replacement and reuse of local textures.

[0012] The method supports parallel driving of multiple characters. By fine-tuning differentiated parameters, key point data of the same driving source is mapped to multiple digital human images, and the skeletal deformation ratio and expression amplitude of each image are independently controlled.

[0013] Furthermore, the digital human image template parameter file is a universal digital human model with a unified skeletal structure and interface definition. The unified skeletal structure includes predefined positions and names of main human skeletal nodes, and the interface definition includes a standardized parameter interface of appearance features.

[0014] Furthermore, the affine transformation matrix is ​​generated as follows:

[0015] The optimal transformation matrix is ​​solved by the least squares method to minimize the mapping error between the key point set of the source digital human image and the corresponding key points of the standard reference coordinate system.

[0016] Furthermore, the resource cache mechanism manages cache data through a unified resource ID index, supports asynchronous loading and frame-by-frame batch loading to achieve smooth transition during switching, maintains a dynamic resource cache pool, and adopts an LRU strategy to eliminate unused resources.

[0017] Furthermore, the key point sequence smoothing process adopts the exponential moving average algorithm, and its recursive formula is:

[0018] (3);

[0019] Where, is the key point position after smoothing; is the frame index; is the smoothing coefficient; is the original key point position of the current frame; is the key point position after smoothing in the previous frame.

[0020] Furthermore, the abnormality compensation strategy includes:

[0021] When a key point is missing, the smoothed coordinates of the previous frame are used to fill it in.

[0022] If multiple consecutive frames are missing, the static neutral expression coordinates preset in the template file are called as replacement values.

[0023] Furthermore, the layered rendering engine splits the digital human image into at least a basic image layer, a clothing layer, and an expression layer. Each layer is superimposed in a preset alignment method to form a complete image. Switching between different digital human characters is achieved by replacing one or more layers.

[0024] Furthermore, it also includes a voice-driven lip synchronization step, which converts the audio input into a sequence of mouth key points of the target digital human image through phoneme recognition and predefined lip shape mapping, and generates collaborative facial movements in combination with the default expression curve.

[0025] Furthermore, the method supports parallel driving of multiple characters. By fine-tuning differentiated parameters, key point data of the same driving source is mapped to multiple digital human images, and the skeletal deformation ratio and expression amplitude of each image are independently controlled.

[0026] Furthermore, the implementation of the method for adapting to multiple platforms includes:

[0027] Adopt a cross-platform graphics rendering framework and dynamically load rendering pipelines adapted to different terminals;

[0028] The coordinate system and resolution are uniformly converted through the interface adaptation middle platform module to ensure the consistency of cross-platform animation output.

[0029] In a second aspect, the present invention further provides a system for rapidly switching and reusing digital human images. The system is based on the method described in the first aspect and includes:

[0030] The template preprocessing module is used to parse each digital human image into a template parameter file;

[0031] A key point alignment module, used to calculate the affine transformation matrix of each digital human image relative to the standard reference coordinate system based on the template parameter file, so as to achieve standardized alignment of the key point set;

[0032] A driving mapping module is used to map the key point data of the driving source to the coordinate system of the target digital human image through affine transformation to generate a target key point set;

[0033] Resource cache management module, used to maintain a dynamic resource cache pool, manage texture and template data through the LRU strategy, and support asynchronous loading and reuse;

[0034] The layered rendering engine module is used to divide the digital face into multiple independent layers based on the target key point set and reuse resources, and synthesize the images in a preset order, combining smoothing processing and anomaly compensation mechanisms to output continuous animation frames;

[0035] The interface adaptation middle platform module is used to shield the underlying platform differences and realize unified scheduling across terminal resolutions, coordinate systems and rendering pipelines by dynamically loading adaptation sub-modules.

[0036] Furthermore, it also includes a speech analysis module for performing phoneme recognition on the audio data when the driving source is audio input, and generating a mouth key point sequence corresponding to the target digital human image.

[0037] In a third aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for rapid switching and multiplexing of digital human images is executed.

[0038] The present invention constructs a standardized template file containing a set of facial key feature point coordinates, basic texture maps, and layered materials, and combines it with an affine transformation matrix to achieve alignment of the key point coordinate systems of multiple human figures; in the real-time driving stage, a dynamic resource cache pool and an LRU strategy are used to manage texture and template data, and the face is divided into independent layers for synthesis through a layered rendering engine. An exponential moving average algorithm is introduced to smooth the key point sequence, and a priority compensation strategy is combined to solve the problem of abnormal data. In addition, the system dynamically loads a cross-platform rendering pipeline through an interface adaptation middleware module to achieve multi-terminal resolution and coordinate system unification. This solution significantly reduces switching delays, increases resource reuse rate to more than 90%, and supports multi-role parallel driving and voice-synchronized lip generation, solving the core defects of high latency, resource redundancy, and poor cross-platform adaptability in traditional technologies, and achieves the technical effects of high fluency, low resource consumption, and multi-terminal consistency in scenarios such as virtual live broadcasts and online education.

[0039] Beneficial effects

[0040] By implementing the method for rapid switching and reuse of digital human images provided by the present invention, the following technical effects are achieved:

[0041] (1) This application achieves unified alignment of key point coordinate systems for different digital human images by constructing a standardized template parameter file and combining it with an affine transformation matrix. This mechanism breaks through the constraints of model topology differences in traditional multi-image switching, enabling seamless migration of drive data between heterogeneous digital human images, solving the problem of cross-image continuity of movements and expressions, and laying the foundation for the universalization of the core drive process.

[0042] (2) Based on the LRU strategy and a dynamic resource management mechanism for asynchronous loading, a multi-level cache pool is constructed to implement on-demand scheduling of texture and template data. This architecture significantly reduces memory usage and IO load during switching by intelligently eliminating redundant resources and reusing high-frequency data. This eliminates the delays and resource waste caused by repeated loading of large models in traditional technologies at the system level.

[0043] (3) The digital face is divided into independent functional layers such as the skin layer, eye layer, and mouth layer, and rendered and synthesized layer by layer using a preset occlusion order and transparency blending rules. This technology achieves decoupled control of facial regions and rapid replacement of local resources. While ensuring high-fidelity visual effects, it significantly improves the efficiency of texture updates and local feature switching, providing refined operation space for multi-style image reuse.

[0044] (4) An exponential moving average algorithm is used to low-pass filter the key point trajectory, and a priority compensation strategy is combined to handle abnormal data. This mechanism effectively suppresses the interference of high-frequency noise and sudden loss of the driving signal on the continuity of the animation. Through dynamic correction and fault tolerance, it ensures the stable output of digital human expressions and movements in complex interactive scenarios.

[0045] (5) Compared with end-to-end image generation methods, such as deep learning face-swapping, this method ensures the controllability of details during switching and smooth transitions between frames through preprocessing and layer rendering, avoiding the distortion and delay that may occur in end-to-end methods, and has better real-time performance and stability.

[0046] (6) By configuring multiple sets of differentiated mapping parameters for a single driving source, multiple digital human characters can be driven in parallel, achieving synchronized animation of "one person driving multiple people." This mechanism ensures that the skeletal deformation ratio and expression amplitude of each character conform to their own image characteristics, expanding the application scenarios of digital humans. For example, in a virtual meeting, a single user's facial expression can simultaneously drive multiple digital human images of different styles, achieving collaborative performance.

[0047] (7) Design a universal lightweight cross-platform architecture, shield the underlying platform differences through the interface adaptation middle platform, and ensure that the real-time drive and image switching process of this method can be smoothly executed on various terminal devices such as mobile terminals, web terminals and desktop terminals. Animation performance and efficiency on different platforms

[0048] The results are kept consistent and the development cost of multi-platform adaptation is reduced.

[0049] (8) A pure voice-driven mechanism is introduced. Through audio analysis and phoneme-lip shape mapping, the digital human's mouth movements can be driven even with only microphone frequency input, and the default expression curve is used to ensure the natural and coordinated overall facial expression. Therefore, in camera-free scenarios such as conference calls and live voice broadcasts, this method can still achieve real-time synchronization between the digital human's mouth shape and the voice content, expanding the applicability of this method in multiple input modes.

[0050] (9) The basic template predefines the location and naming of the main skeletal nodes of the human body, as well as the parameter interface of the appearance characteristics, so that different digital human characters can be constructed based on this template. By adopting a standardized basic template, the differences between different characters only need to be described through parameterized data, achieving a unified standardization of digital human image construction, facilitating subsequent rapid switching and reuse, and enabling the system to manage and apply various character characteristics based on the template. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] To make the above-mentioned method of rapid switching and reuse of digital human images of the present invention more clear and easy to understand, the following briefly introduces the drawings required for use in the specific implementation of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 It represents the flow chart of the present application method;

[0053] Figure 2 Represents the system architecture diagram of this application. DETAILED DESCRIPTION

[0054] Without departing from the core concept of this invention, those skilled in the art may also devise various variations. For example, mapping drive data between different digital human images could be accomplished by establishing a unified skeletal binding relationship or a standardized expression parameter system (e.g., FACS) to achieve key point alignment, thus achieving a similar cross-image drive effect. For another example, in the generation of digital human images, a pre-trained generative adversarial network model could be used to directly map the drive source to the animation frames of the target image, achieving real-time "face swapping" without explicit key point conversion. The present invention is equally applicable to these alternative implementations, and the technical effects achieved by them are also within the scope of this invention.

[0055] Example 1:

[0056] A method for rapid switching and reuse of digital human images is provided. The method flow is as follows: Figure 1As shown, it includes: pre-building a digital human image template parameter file in a unified format, the template parameter file includes a facial key feature point coordinate set, basic texture mapping data and local material layering information, and aligning the key points of different digital human images to a standard reference coordinate system through an affine transformation matrix; mapping the key point data of the driving source to the coordinate system of the target digital human image to generate a target key point set; using a resource caching mechanism to manage the texture and template data of the digital human image, and reducing the resource loading delay during switching through a cache reuse strategy; based on the target key point set, synthesizing the digital human image frame through a layered rendering engine, and combining key point sequence smoothing and anomaly compensation strategies to ensure the continuity and stability of the animation.

[0057] A system for rapid switching and reuse of digital human images is provided. The system is based on the above-mentioned method and has the following architecture: Figure 2 As shown, it includes: interface adaptation middle platform module, shielding the differences of underlying platforms (PC, mobile terminal, Web, etc.), providing a unified API interface, and being responsible for sitting

[0058] Standard system and resolution conversion; image processing and key point extraction module, which is responsible for preprocessing the key points of the digital human image template in the offline stage, and performing facial key point detection, preprocessing and normalization in the real-time stage; key point alignment module, which aligns the key points of each digital human image template to the standard reference coordinate system based on the affine transformation matrix; drive mapping module, which maps the real-time drive source key points to the coordinate system of the target digital human image through affine transformation to generate a target key point set; cache and resource management module, which maintains a dynamic resource cache pool, adopts LRU strategy and asynchronous loading, manages texture and template data, realizes resource reuse and reduces loading delay; layered rendering engine module, which splits the digital human face into multiple layers, synthesizes images in a preset order, and combines key point sequence smoothing and anomaly compensation strategy to output continuous animation frames; output animation frames / video streams, which provide the continuous image frames generated by the rendering engine to the upper-layer application or encoder to form a real-time video stream for display.

[0059] The details are as follows.

[0060] 1. Technical solution

[0061] The core idea of ​​the method is to introduce a unified image template parameter structure and key point mapping driving mechanism, so that different digital human images can be quickly interchanged under the same set of driving data while maintaining consistent movements and expressions. The method includes the following steps:

[0062] Step 1: Image template preprocessing:

[0063] First, a standardized digital human base template is established. This base template is a universal human body model with a unified skeleton and interface definition. For example, the base template predefines the location and naming of the main skeletal nodes of the human body, as well as the parameter interface of the appearance characteristics, so that different digital human characters can be constructed based on this template. By adopting a standardized base template, the differences between different characters only need to be described through parameterized data, achieving unified standardization of digital human image construction. This measure provides a foundation for subsequent rapid switching and reuse, enabling the system to manage and apply various character features with the template as the center. Specifically including:

[0064] In the offline phase, each digital human image is parsed into a template parameter file in a unified format. The template content includes:

[0065] Facial key feature point coordinate set: The coordinates of the main feature points of the face of the image are extracted as the reference skeleton markers representing the face shape.

[0066] Basic texture map data: The skin / face texture resource of the image, used to present the details of the character's appearance.

[0067] Local mask or layered material: Material layering or masking information for specific parts, such as a transparent mask for the mouth area, a highlight layer for the eyes, a translucent texture layer for the hair, etc.

[0068] The template parameters are stored in a unified data structure to ensure consistent correspondence between key parameters across different images. After preprocessing, a standard template file is generated for each image, recording the locations of its feature points in a standard reference coordinate system and the associated texture resource index, facilitating subsequent rapid loading and mapping.

[0069] Step 2: Affine alignment of key points:

[0070] The coordinate transformation relationship of each image relative to the standard reference coordinate system is calculated, specifically including: taking the key point set of the image extracted from the template as input, solving an affine transformation matrix that maps it to the corresponding key points in the reference coordinate system. Preferably, the least squares method is used to solve the transformation matrix that minimizes the error, for example, solving the matrix To minimize , where is the frame index; is the affine transformation matrix to be solved; is the key point set of the source digital human image; The optimal affine matrix is ​​obtained by aligning the source image's coordinate system to the unified reference frame, thereby standardizing the initial poses of different images. The calculated affine matrix for each image is stored in its template file for future use.

[0071] Step 3: Drive key point mapping:

[0072] During real-time operation, each frame obtains the current set of facial key points from the driving source, whose format is consistent with the key points in the template. To apply this driving action to the new target image, the coordinates of the source key points need to be converted to the coordinate system of the target image. Assume that the affine matrix of the target digital human image relative to the reference coordinate system is , and the matrix of the current driving source corresponding image is . The relationship between the mapping of the current frame source key points to the target image coordinate system can be expressed as:

[0073] (1);

[0074] The above formula means: First, use the transformation matrix of the source image The set of driving key points captured in the current frame Mapped to the standard reference coordinate system, and then through the matrix of the target image Convert the reference coordinates to the target image's own coordinate space to obtain the corresponding target image key point set After this coordinate system conversion, the new digital human image can accurately reproduce the driver's facial expressions and movements at the moment, laying the foundation for subsequent rendering.

[0075] Step 4: Resource loading and cache reuse:

[0076] To enable fast image switching and avoid repetitive loading of large amounts of resources, the system has designed an efficient texture / model caching mechanism. A cache pool of image resources is maintained at runtime. The loading logic is as follows: When rendering an image, the system first checks whether the image's resources are already in the cache. If a cache hit is found, the existing data is reused without reloading. If a cache miss is found, the image's texture and template data are read from the storage medium, and their affine alignment matrix is ​​calculated before adding this data to the cache. To prevent unlimited cache growth, cache management employs a LRU strategy, eliminating unused image resources when memory usage approaches the upper limit. Furthermore, images are managed using a unified resource ID index, ensuring efficient and reliable search and retrieval. This strategy eliminates the need to reload frequently used digital human images, significantly reducing switching latency and memory overhead. For large texture or model data, background asynchronous loading or batch loading, frame by frame, is used to smooth the transition and avoid lag during switching.

[0077] Step 5: Image synthesis and rendering:

[0078] The rendering engine is based on the key point set of the target image And the corresponding texture resources, the digital human is drawn to generate the current frame image. The drawing function of the rendering engine is , then the output image of each frame is expressed as:

[0079] (2);

[0080] Where, is the output digital human image frame; Texture resources for the target digital human image;

[0081] Upcoming Texture Resources According to the key point collection After positioning and deformation, it is attached to the digital human model to generate the image for that frame. During the rendering process, the facial texture undergoes affine deformation based on the key point positions, synchronizing the facial expressions and texture deformation. Appropriate lighting effects and local masking are also superimposed to ensure realistic visuals. This process is repeated for consecutive frames, forming an animation sequence in which the target digital human changes with the driving source's movements.

[0082] Step 6: Smoothing of key point sequence:

[0083] Since the key point sequence output by the actual driving source contains slight jitter noise, if used directly, it will cause slight jitter in the rendering result. Therefore, the method introduces a low-pass filtering smoothing strategy for the key points of consecutive frames, preferably using the exponential moving average method. The recursive formula is:

[0084] (3);

[0085] Where, is the key point position after smoothing; is the smoothing coefficient, ranging from 0 to 1; is the original key point position of the current frame; is the key point position after smoothing in the previous frame. When it is smaller, it emphasizes the position of the previous frame and achieves stronger smoothing filtering; When it is close to 1, it will follow the changes of the current frame more closely and respond to large movements in a timely manner. After EMA smoothing, the position changes of key points in each frame are more continuous, eliminating high-frequency jitter, making the rendered animation transition smooth and jitter-free.

[0086] Step 7: Compensation for missing key points:

[0087] In response to coordinate loss or anomalies that occur during the detection of driving key points, the method sets a priority compensation strategy to avoid deformed rendering output. When some key points are detected to be invalid in a certain frame, the system fills the gaps in the following order: First, the previous frame is used: If a key point is missing in the current frame but there is a valid value in the previous frame, the coordinates of the point after smoothing in the previous frame are used to ensure the continuity of movement; if multiple consecutive frames are missing and the "previous frame" has no valid reference, the template default value is used: the initial coordinates of the point in the target image template are used to fill in to ensure the integrity of the facial structure. Without loss of generality, the key points are described in mathematical form In the Frame compensation results :

[0088] If the current frame detection is valid, the indicator function , then directly use the smoothed current value: .

[0089] If the current frame is missing and the previous frame has a valid value, that is, and , then the smoothing value of the previous frame is used: .

[0090] If the current frame is missing and there is no previous frame reference, If there is no valid value before, the template default value is used: .

[0091] Based on these rules, we can find reasonable replacement coordinates for each abnormal key point, avoiding local distortion of the output portrait due to missing individual points. After inter-frame smoothing and missing key point compensation, the key point set is used to replace the original input for subsequent rendering, ensuring a stable and coherent animation sequence during the switching process.

[0092] In addition, it should be noted that the smoothing and compensation strategies of the present invention are not limited to the methods used above. For example, in order to smooth the driving key point sequence to avoid high-frequency jitter, in addition to using the exponential moving average algorithm, Kalman filtering, bilateral filtering, or noise reduction filtering based on frequency domain analysis can also be used to achieve similar smoothing effects. Similarly, for the compensation of missing key points, in addition to giving priority to filling with the smoothing value of the previous frame or the default value of the preset template, other alternatives can also be used as needed, such as using the positional relationship between adjacent feature points for interpolation prediction, or using the symmetry of the face to infer the missing point value from the corresponding point on the other side. Regardless of the specific smoothing algorithm or compensation method used, as long as it can achieve continuous and stable changes in the driving key point sequence, thereby ensuring a smooth and coherent switching process for digital human animation, it falls within the scope of protection of the present invention.

[0093] Furthermore, to enhance the robustness of the switching process, the system incorporates a dynamic fault-tolerance strategy to handle anomalies that may arise during the switching process. Specifically, when executing a digital human avatar switch, the system continuously monitors the loading status of resources on each layer. If it detects a resource loading failure for a layer (e.g., missing or corrupted file) or fails to load after a predetermined time, the system automatically triggers a fault-tolerance mechanism. For critical layer resource loading failures, pre-set default resources are substituted, such as replacing a specific clothing model that failed to load with a default garment. For non-critical layers or resources that can be temporarily omitted, the system can skip loading these resources to prioritize the switching of the primary avatar. This dynamic fault-tolerance strategy ensures that even when some resources experience anomalies, the digital human avatar switch can be completed continuously and stably, preventing the entire switching process from being interrupted by a single module failure. This fault-tolerance design further ensures the reliability of the present method in practical applications.

[0094] 2. System architecture and collaboration process

[0095] This application adopts a loosely coupled system architecture, with multiple functional modules working together to achieve real-time driving and image switching of digital humans. The responsibilities of each module are clearly defined, and they interact through standard interfaces. The overall workflow is as follows:

[0096] The rendering engine module is responsible for the final rendering of the digital human image. It receives the keypoint locations and texture resources of the target image in the current frame, deforms and fits the texture according to the keypoints, and overlays the corresponding masks and lighting effects to output the digital human image frame. The rendering engine includes efficient texture mapping and mesh deformation shaders, enabling real-time deformation and smooth transitions of the model mesh in localized facial areas. This ensures that facial expressions and details flow naturally during image transitions, generating highly realistic animation frames.

[0097] Image Processing and Keypoint Extraction Module: This module includes two functions: offline template preprocessing and real-time keypoint detection. In the offline phase, this module uses a facial keypoint detection algorithm to extract the characteristic keypoints of each digital human image template, aligns them with reference coordinates, and generates a standardized image parameter file for the system to load and use. In the real-time phase, this module obtains the raw driving data of each frame from the driving source, performs facial keypoint recognition or expression parameter calculations, and obtains the keypoint set of the driving character in the current frame. The keypoint data is then subjected to necessary normalization and coordinate transformation before being passed through an interface to the subsequent keypoint mapping algorithm. If necessary, the module also preprocesses the input image to improve the accuracy and robustness of keypoint detection.

[0098] Interface adaptation middle platform module: As a bridge between the system and upper-level applications, it provides a unified API interface for front-ends of different platforms to call digital human-driven rendering and image switching functions. Regardless of whether the front-end is a web page, mobile app or desktop application, the middle platform module shields the differences in the underlying platforms and provides consistent services. This module is responsible for the conversion of coordinate systems of different platforms, resolution adaptation, and data synchronization between the UI interface layer and the rendering engine to ensure that the switching process presents consistent effects on various devices. By dynamically loading the adapter sub-module, the middle platform applies corresponding optimization measures for different operating environments, thereby achieving smooth cross-platform docking.

[0099] Cache and resource management module: responsible for loading, caching, and managing digital human image template data and texture resources. This module works in conjunction with the rendering engine and image processing module: when a request to switch images is received, it first queries whether the target image resource is already in the cache. If so, it directly returns the resource handle to avoid repeated loading; if not, it triggers the asynchronous loading process, reads the texture and template data required for the new image into memory, and updates the cache after completion for use by the rendering engine. Cache management uses the aforementioned unified index and LRU strategy to control memory usage, ensuring that required resources can still be quickly retrieved and scheduled when multiple image resources coexist. Resources are delivered through standardized data interfaces, and this module is decoupled from other modules. It does not rely on specific implementation details, but can efficiently support resource reuse and switching.

[0100] The aforementioned modules work together through interfaces. The overall operational process is as follows: First, the image processing module continuously acquires the driving character's facial keypoint data from camera video or other inputs. After preprocessing, it is transmitted to the rendering engine via the interface middleware. The rendering engine uses the currently active digital human image template to perform rendering, generating and outputting continuous animation frames. When an image switch is required, the upper-layer application issues a switch command, which is captured by the interface middleware and notifies the cache management module to prepare the target image resources. The cache module loads the new image's texture and template data in the background. Simultaneously, the keypoint mapping algorithm converts the keypoints of the current driving frame to the new image's coordinate system according to step three above. Then, starting from the next frame, the rendering engine uses the new image's texture resources and mapped keypoints for rendering. Thanks to keypoint smoothing / compensation and rendering transition strategies, the transition between frames remains seamless: the action positions of the old and new images remain continuous, and the scene background is not abruptly cleared or redrawn, thus avoiding flickering or pauses. The entire switch does not affect users and upper-level applications. The system still maintains real-time performance under the module decoupling architecture, and the collaboration of various units ensures smooth image replacement.

[0101] 4. Extended functionality

[0102] Based on the aforementioned solutions, extended functions are also provided to enhance the flexibility and applicability of digital human driving and rendering.

[0103] Parallel Multi-Character Driving: The system supports simultaneous control of multiple digital human avatars using driver input, enabling parallel driving of multiple characters. In scenarios such as virtual meetings and group performances, a single user's expressions or movements can be simultaneously mapped to multiple virtual characters of varying styles. To ensure that each character's performance aligns with its own unique image characteristics, the system incorporates a differentiated parameter fine-tuning strategy: a set of mapping parameters is pre-configured for each target virtual character, including a facial proportion adjustment coefficient, an expression amplitude scaling factor, and an offset correction value. After the driver generates a series of facial keypoints, the system sends this keypoint sequence to multiple character mapping modules in parallel. Each character module applies its own set of parameters to transform and fine-tune the input keypoints, generating animation control signals tailored to the character's skeletal / mesh structure. For example, for characters with larger facial skeletons, keypoint displacements are amplified to maintain a consistent expression amplitude. For cartoon-style characters, an exaggeration factor is applied to enhance their expression variations, resulting in exaggerated, cartoon-like movements. Each character's driving process operates independently and in parallel, without interfering with each other. This parallel driving capability meets the requirements for synchronized performances of multiple digital humans in virtual scenes. Even characters with different appearances can maintain synchronized and distinctive movements based on the same input.

[0104] Layered rendering mechanism: The system implements this by breaking down the digital human image into multiple layers (hierarchies). Preferably, the digital human image includes at least a base image layer, a clothing layer, and an expression layer, each carrying different types of content: the base image layer presents the digital human's basic physical appearance, such as body shape and basic features; the clothing layer overlays the character's clothing and accessories; and the expression layer overlays facial expressions or dynamic effects. The layers are stacked together using predefined alignments to form a complete digital human image. When switching digital human characters, the system replaces the resources of the corresponding layers layer by layer for the changed parts, such as replacing the data in the clothing and expression layers, while keeping the base image layer unchanged, thereby rapidly generating the new character's image. This multi-layered mechanism ensures maximum resource reuse, ensuring that only the necessary changes are updated during the switching process, significantly improving switching efficiency.

[0105] In the process of splitting the face into multiple layers according to functional areas and drawing and synthesizing them separately, the system divides the digital face into the bottom skin layer and several independent upper layers such as lips, teeth, eyes, eyebrows, and hair according to the facial structure. Each layer maintains the texture and geometric information of the corresponding area. During rendering, each layer is drawn in sequence according to the preset occlusion relationship and order, and is blended and superimposed using transparency to finally generate a complete facial image. For example, after drawing the hair as an independent layer on the skin layer, the translucent effect of the hair is presented through blending to ensure that the hair is correctly obscured behind the face; for example, in the mouth processing, the lips and teeth are treated as different sub-layers: when the character opens his mouth, the opacity of the teeth / tongue layer is increased to make it clearly visible, and when the mouth is closed, its transparency is reduced so that the teeth are obscured by the lips layer. Through layered drawing, decoupled control of each facial area is achieved: modifying or replacing a layer will not affect the display of other layers. This greatly improves the efficiency of local adjustment and switching of the image, and supports rapid replacement of local features on demand to reuse image resources. In addition, each layer uses a unified Blending rules ensure that the final composite image has smooth transitions at edges and realistic connections between different areas. A layered rendering mechanism enables the system to flexibly adjust facial details, correctly handling layered occlusion in complex scenes while enhancing the realism of digital human expressions and providing smoother visual transitions when switching between images.

[0106] Speech-Based Lip Synchronization: This system further provides methods for lip syncing a digital human when driven solely by speech. Even without camera video input, the system can synchronize the digital human's mouth shapes with the spoken content in real time based on pure audio. When driven by audio, the speech analysis submodule is activated. It first performs speech recognition or feature extraction on the input speech to obtain a phoneme-level timeline, dividing the continuous speech into a series of time-labeled phoneme segments. Next, the system maps the phoneme sequence to a predefined standard lip animation sequence. Specifically, the system pre-establishes a speech expression library that stores keyframe mappings between key phonemes / syllables and corresponding lip shapes. For example, vowels correspond to open lip shapes, bilabial sounds correspond to pouting or closed lip shapes, and dental fricatives correspond to exposed teeth. The speech-driven module sequentially extracts the lip shape keyframes corresponding to each phoneme in the speech based on the audio timeline. These keyframes are interpolated and smoothly transitioned based on the phoneme duration to generate a continuous and natural lip movement sequence. At the same time, to avoid only mouth movements while the rest of the face appears stiff, the system combines the default facial expression curves in the expression library to apply basic coordinated movements to other facial areas. Ultimately, the lip synchronization module outputs a sequence of key mouth points that is strictly aligned with the input voice, driving the digital human to produce accurate speaking lip shapes. For example, when a greeting is input, the system extracts the continuous sequence of phonemes and drives the digital human to open and close its mouth, smile with teeth, and other movements in sequence, precisely and synchronously corresponding to the voice content. In this way, even in scenes without video, the digital human's lip movements can be consistent with the voice, presenting a natural and smooth speaking effect. This voice-driven mechanism significantly improves the expressiveness of the digital human in pure audio scenes, enabling the digital human image to achieve realistic lip synchronization in various input modes.

[0107] 5. Cross-platform applicability and exception handling

[0108] In order to enable this method to run stably on a variety of hardware platforms and complex environments, the system is designed with emphasis on cross-platform compatibility and is equipped with a comprehensive exception handling mechanism.

[0109] The system uses a highly portable graphics rendering framework and standardized interface programming, so that the core rendering and driver code can maintain consistent performance on different operating systems such as Windows, Linux, macOS, Android, and iOS. During initialization, the interface adaptation center dynamically loads the corresponding adaptation module according to the current platform environment. For example, it enables a streamlined rendering pipeline on the mobile side to reduce power consumption, and calls high-performance shaders on the desktop side to fully utilize GPU performance. For graphics hardware from different manufacturers, the system uses common shading language features and standard extensions to avoid relying on the private functions of specific devices to enhance cross-hardware compatibility. Through the above measures, the entire system has good compatibility and real-time performance on various platforms and devices, ensuring that the digital human switching function has wide applicability.

[0110] For abnormal situations that may occur during operation, design an exception handling strategy to ensure stable and continuous operation of the system:

[0111] If network delays cause remote texture resource acquisition to fail or local files are missing, the system will detect this and promptly enable alternative solutions, such as switching to low-resolution alternative textures or using preset placeholder textures instead, to avoid rendering interruptions and asynchronously retry loading the original resources in the background.

[0112] If real-time keypoint data shows anomalies, the system activates a fault-tolerance strategy. For minor fluctuations, the system uses the previous frame's smoothed data for transitions, or temporarily reduces the drive frame rate to allow for data recovery. For severely distorted frames, the system discards the data to prevent exaggerated or distorted expressions.

[0113] If certain rendering effects are unsupported on a specific device, the system pre-sets multiple rendering schemes for switching. When compatibility errors occur, it will automatically downgrade to a conservative compatibility mode, such as disabling advanced lighting effects or switching to the software rendering path, to ensure normal operation of basic functions.

[0114] The system allows you to set a maximum switching delay threshold. If an image switching operation exceeds the preset time and is still not completed, the system will automatically terminate the current switching process and roll back to the state before the switch to maintain the continuity of the user's visual experience and prevent the interface from staying in an unfinished transition for a long time.

[0115] Through these exception handling strategies, the system maintains continuous and stable operation even in the face of network instability, significant hardware discrepancies, or sensor data anomalies. Users are virtually unaware of the internal failure and recovery process, and always see a smooth and natural digital human presentation.

[0116] In the event of anomalies in resource loading or real-time driving, the present invention can also take other equivalent fault-tolerant measures. For example, when remote texture resource acquisition fails or local files are missing, in addition to enabling low-definition textures or placeholder textures as a substitute, the system can also call the last version of the valid resource screen cached locally, if conditions permit, or directly skip the rendering output of the frame in extreme cases, to avoid screen freezes and interruptions. For example, for frames with severely distorted camera data, in addition to directly discarding them, an expression and action database can be pre-established to replace the erroneous key points identified in the abnormal frames with the most approximate legal expression and posture data. All these modifications are intended to ensure that the system can still operate continuously and stably under abnormal circumstances without affecting the switching effect of the digital human. These equivalent variations based on the concept of the present invention also fall within the scope of protection required by the present invention.

[0117] Example 2:

[0118] To address abnormal fluctuations or omissions that may occur during the real-time acquisition of key point data, a key point protection mechanism has been added to ensure the stability and coherence of digital human animation. A protection point refers to a safe alternative value or compensation measure used when the key point detection results are abnormal, that is, a fault-tolerant protection for key points. The system first performs anomaly detection on the key point set input for each frame to determine whether certain key points have failed or are abnormal. For example, when the camera capture quality is poor or the user's movements are too fast, the coordinates of individual key points may have abnormal values ​​or be directly missing. Once an anomaly in key point data is detected, the system activates the protection mechanism and corrects the key points through filtering and compensation strategies: minor jitter noise is buffered through sequence smoothing, and serious omissions are filled using historical data or template data. The entire protection process is completed in milliseconds, effectively preventing distortion or incoherence in digital human expressions caused by key point anomalies.

[0119] For detected minor anomalies, the system introduces low-pass filtering to smooth the key point sequence, preferably using the exponential moving average algorithm. By smoothing high-frequency noise, overly sensitive minor jitter is weakened, making the position change of key points more continuous. The recursive formula is expressed as:

[0120] (4);

[0121] Where, are the coordinates of the key points after smoothing; is the smoothing coefficient, ranging from 0 to 1; For the The original key point coordinates obtained by frame detection; The key point coordinates after smoothing in the previous frame. The value makes the smoothing result more biased towards the previous frame data, thereby filtering out small high-frequency jitter; A value closer to 1 makes the result closer to the original value of the current frame, allowing for timely response to significant changes in the user's expression. For the initial frame, the smoothed coordinates can be directly set equal to the original detection value to begin the recursive calculation. After EMA smoothing, the keypoint trajectories of successive frames are more stable, eliminating the impact of subtle jitter on rendering and ensuring natural, smooth transitions in digital human animation.

[0122] If anomaly detection finds significant errors or omissions in keypoint data for certain frames, the system uses a priority compensation strategy to provide alternative coordinates for the outliers. This strategy uses historically valid data or template reference values ​​as "protection points" to prevent distortion in the rendered output. The compensation mechanism searches for the best replacement values ​​in order from newest to oldest, and from dynamic data to static templates. The specific rules are as follows:

[0123] Use Previous Frame: If a keypoint in the current frame is missing or invalid, but valid coordinates exist for that point in the previous frame, the smoothed position of that keypoint in the previous frame will be used to replace the current frame's value to ensure the continuity of the motion trajectory. This allows for a smooth transition during brief detection failures, without the user noticing the loss of a single frame.

[0124] Template Defaults: If keypoints are missing in the current frame and there's no valid reference from the previous frame, pre-defined template default coordinates are used. These defaults are derived from the target digital human's keypoint template and represent reasonable static positions. Using template values ​​to fill in the gaps ensures the digital human's facial structure is intact, preventing mesh stretching or distortion caused by missing coordinates.

[0125] The above compensation rule can also be described in mathematical form: Let The original key point coordinates obtained by frame detection are , and its effectiveness indicator function is , the smoothed key point coordinates are , the template default value is The final coordinates of the key point output by the protection mechanism are Expressed as:

[0126] (5);

[0127] When the current frame's detection value is valid, the smoothed result of the current frame is directly used. When the current frame is missing but the previous frame has a reference, the protected coordinates of the previous frame are used. If consecutive key points are missing, the static coordinates in the template are used as replacements. This strategy ensures that whenever an abnormal key point appears, the system can always select a reasonable replacement position for it, thus ensuring the integrity and stability of digital human facial animation.

[0128] The above smoothing and compensation mechanisms together constitute the keypoint protection strategy. In practice, different types of keypoint anomalies are promptly corrected to avoid negatively impacting the animation. For example, when a user blinks rapidly, eye features change dramatically, potentially preventing the corner keypoints of the eyes from being properly recognized in the current frame. In this case, the system detects a "missing corner keypoint" anomaly and automatically activates a protection mechanism: if the eye corner coordinates exist in the previous frame, the smoothing values ​​from the previous frame are used to maintain a continuous transition of eye position. If the eye corners cannot be detected in consecutive frames, the reference coordinates of the eye corners in the template are temporarily used to maintain the digital human's eyes in a naturally open state. This ensures that regardless of the user's rapid eye closing and opening, the digital human's eye movements remain smooth and seamless, without anomalies such as collapse or jumping. For another example, when a user laughs or speaks exaggeratedly, the corners of the mouth can sometimes be misdetected due to facial angle or occlusion. If the coordinates of the right corner of the mouth are abnormally offset or even missing in a given frame, the system will prioritize using the smoothed coordinates of that corner from the previous frame to continue driving the mouth shape. If the anomaly persists for multiple frames, the system will place the right corner of the mouth in the neutral position defined in the template. As a result, the digital human character's mouth deformation will pause briefly but maintain its normal shape, without causing mouth deformities due to errors in a single frame of data. Through this strategy, even if individual feature points are temporarily lost or abnormal, the various parts of the digital human's face can still maintain their proper expression, form, and relative position, making the overall animation effect appear natural and credible to the viewer.

[0129] In summary, the system drives digital human animation through the precise extraction and mapping of key points, and incorporates a protection point mechanism to ensure the stability and reliability of key point data. On the one hand, standardized key points serve as a unified driving medium, enabling smooth transitions between expressions and movements across multiple digital human images. On the other hand, the system employs protection strategies such as exponential smoothing and graded compensation to correct random jitter and abnormal omissions in key point sequences in real time. This ensures that animation frames remain continuous and complete during the switching process, eliminating distortion and jitter caused by sensor data anomalies. The key points and protection points work together to perform the dual functions of driving animation and providing fault-tolerant voltage stabilization. They ensure highly accurate movement reproduction when switching between different images, while also providing a safety net for each key feature through anomaly detection and compensation mechanisms. Ultimately, this ensures a smooth, stable, realistic, and reliable digital human image switching process.

[0130] Example 3:

[0131] Based on the above embodiments, the specific process of the method for quickly switching digital human images is described in combination with a virtual live broadcast scene:

[0132] Step 1: Preload the image template:

[0133] Before the live broadcast begins, the host prepares two digital human images A and B. The system performs offline preprocessing on images A and B respectively, generates standardized key point templates and texture data files, and loads these two sets of template resources into the system cache in advance for backup.

[0134] Step 2: Start live broadcast and drive animation:

[0135] At the start of a live stream, the host selects avatar A as their initial digital human. The system activates avatar A's template parameters and begins capturing the host's facial expressions and movements through the camera. It calculates facial key points in real time and drives avatar A to move in sync. The rendering engine continuously outputs animated images of avatar A as it changes with the host's expressions. The host's avatar A responds smoothly to the host's movements in real time.

[0136] Step 3: Trigger image switching:

[0137] During a live broadcast, the host decides to change the digital human avatar from A to B and sends a switch command through the operation interface. After the interface adaptation center captures the command, it immediately notifies the system to prepare to switch the current digital human avatar from A to B and dispatches relevant resources.

[0138] Step 4: Load and map the new image:

[0139] Upon receiving the switch command, the system quickly retrieves the texture and template data for Image B from the cache module. Simultaneously, the image processing module invokes a keypoint mapping algorithm to convert the current keypoint set of the anchor's face from Image A to the coordinate system of Image B. In the very next frame, the rendering engine begins rendering using Image B's texture and mapped keypoints. Because the driver data from Image A is seamlessly mapped to Image B, Digital Human Image B immediately takes over the screen.

[0140] Step 5: Complete the switch presentation:

[0141] At the moment of switching, the system employed a smooth transition strategy to ensure visual continuity: when the rendering engine switched frames to draw character B, it retained the background and scene state from the previous frame, without abruptly clearing or resetting the image, thus avoiding any flickering or frame skipping. After the switch, the digital human in the live broadcast seamlessly transitioned from A to B, with facial expressions and body movements remaining consistent; only the appearance changed.

[0142] Viewers will see the host's digital human avatar switch from A to B in a remarkably fast time, with the entire process smooth and fluid, and the switching delay barely perceptible to the naked eye. The switched avatar B immediately retains the host's pre-switch expressions and movements, without any pauses or lags. This demonstrates the efficiency and stability of this solution, ensuring that even in scenarios like live streaming, which place extremely high demands on real-time performance, rapid switching of digital human avatars is seamless and seamless, without compromising the user experience.

[0143] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media containing computer-usable program code.

[0144] The present invention can provide computer program instructions to a management platform of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the management platform of the computer or other programmable data processing device produce a device for implementing the system.

[0145] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device that implements the functions of the system.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions of the described system.

Claims

1. A method for rapid switching and reuse of digital human images, characterized in that: include: Pre-build the digital human image template parameter file and align the key points of different digital human images to the standard reference coordinate system through the affine transformation matrix; Map the key point data of the driving source to the coordinate system of the target digital human image to generate a target key point set; Use resource caching mechanisms to manage the texture and template data of digital human images, and use cache reuse strategies to reduce resource loading delays during switching; Based on the target key point set, digital human image frames are synthesized through a layered rendering engine, and key point sequence smoothing and anomaly compensation strategies are combined to ensure the continuity and stability of the animation; The layered rendering engine divides the digital face into multiple independent layers, and each layer is synthesized into a complete image according to the preset occlusion order and transparency blending rules, and supports the rapid replacement and reuse of local textures; The method supports parallel driving of multiple characters. By fine-tuning differentiated parameters, key point data of the same driving source is mapped to multiple digital human images, and the skeletal deformation ratio and expression amplitude of each image are independently controlled.

2. The method according to claim 1, wherein: The digital human image template parameter file is a universal digital human model with a unified bone structure and interface definition. The unified bone structure includes predefined positions and names of the main bone nodes of the human body, and the interface definition includes a standardized parameter interface of appearance features.

3. The method according to claim 1, characterized in that The method for generating the affine transformation matrix is: The optimal transformation matrix is ​​solved by the least squares method to minimize the mapping error between the key point set of the source digital human image and the corresponding key points of the standard reference coordinate system.

4. The method according to claim 1, wherein: The resource cache mechanism manages cache data through a unified resource ID index, supports asynchronous loading and frame-by-frame batch loading, maintains a dynamic resource cache pool, and adopts an LRU strategy to eliminate unused resources.

5. The method according to claim 1, wherein: The key point sequence smoothing process adopts the exponential moving average algorithm, and its recursive formula is: (3)； Where, is the key point position after smoothing; is the frame index; is the smoothing coefficient; is the original key point position of the current frame; is the key point position after smoothing in the previous frame.

6. The method according to claim 1, wherein The abnormality compensation strategy includes: When a key point is missing, the smoothed coordinates of the previous frame are used to fill it in. If multiple consecutive frames are missing, the static neutral expression coordinates preset in the template file are called as replacement values.

7. The method according to claim 1, wherein: The layered rendering engine splits the digital human image into at least a basic image layer, a clothing layer, and an expression layer. Each layer is superimposed in a preset alignment method to form a complete image. Switching between different digital human characters is achieved by replacing one or more layers.

8. The method according to claim 1, wherein: It also includes a voice-driven lip synchronization step, which converts the audio input into a sequence of mouth key points of the target digital human image through phoneme recognition and predefined lip shape mapping, and generates collaborative facial movements in combination with the default expression curve.

9. A system for rapid switching and reuse of digital human images, characterized by: The system operation is to execute the method according to any one of claims 1 to 8: The system comprises: The template preprocessing module is used to parse each digital human image into a template parameter file; A key point alignment module, used to calculate the affine transformation matrix of each digital human image relative to the standard reference coordinate system based on the template parameter file, so as to achieve standardized alignment of the key point set; A driving mapping module is used to map the key point data of the driving source to the coordinate system of the target digital human image through affine transformation to generate a target key point set; Resource cache management module, used to maintain a dynamic resource cache pool, manage texture and template data through the LRU strategy, and support asynchronous loading and reuse; The layered rendering engine module is used to divide the digital face into multiple independent layers based on the target key point set and reuse resources, and synthesize the images in a preset order, combining smoothing processing and anomaly compensation mechanisms to output continuous animation frames; The interface adaptation middle platform module is used to shield the underlying platform differences and realize unified scheduling across terminal resolutions, coordinate systems and rendering pipelines by dynamically loading adaptation sub-modules.

10. The system according to claim 9, characterized in that: It also includes a speech analysis module for performing phoneme recognition on the audio data when the driving source is audio input, and generating a mouth key point sequence corresponding to the target digital human image.

Images