A security control middleware system for large language model driven digital person and a method thereof
By using an adaptive instruction safety control system to dynamically verify pipeline assembly and resource priority scheduling, the verification adaptability and resource scheduling problems of LLM-driven digital human systems are solved, improving interaction stability and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 李建华
- Filing Date
- 2026-06-04
- Publication Date
- 2026-07-21
AI Technical Summary
In existing technologies, LLM-driven digital human systems suffer from poor adaptability to fixed rule verification, crude handling of high-risk instructions, and unreasonable resource scheduling, resulting in unstable interactive experiences and low efficiency.
An adaptive command safety control system is adopted, which achieves scene adaptive verification and interaction priority guarantee through dynamic verification of pipeline assembly, hierarchical repair and resource priority scheduling.
It improved verification efficiency, reduced false interception rate and interaction latency, enhanced system throughput and stability, and ensured the continuity and efficiency of the core interactive experience.
Smart Images
Figure CN122433712A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of digital human intelligent control and information security technology. Specifically, it relates to a system and method for providing security control middleware for digital humans driven by Large Language Model (LLM), which can be widely used in scenarios such as virtual live streaming, virtual companions, VR game NPC interaction, and virtual education lecturers. Background Technology
[0002] With the rapid development of Large Language Model (LLM) technology, using LLM to drive real-time interaction of 3D digital humans has become an industry trend. Digital humans typically have a complete human skeletal model, facial expression system, and rendering pipeline, and can execute corresponding limb movements, facial expressions, and lip-syncing according to instructions.
[0003] However, LLM outputs inherently possess uncertainty. Control instructions generated by LLM may contain formatting errors, parameter out-of-bounds errors, physical logic conflicts, and other issues. If these insufficiently validated instructions are directly issued to the digital human terminal for execution, it may result in distorted digital human movements and screen stuttering, or even system crashes or rendering pipeline malfunctions.
[0004] Existing technologies include methods for format validation of LLM outputs and solutions for physical limit constraints on robot movements. However, these solutions have the following drawbacks: First, existing solutions all use fixed rule validation, which cannot adapt to the rich semantic variations of LLM output. All instructions go through the same fixed validation process, resulting in the lack of key constraints in certain specific scenarios (such as the lack of physical limit validation for dance movements), while unnecessary validation overhead exists in other scenarios (such as limb collision detection in dialogue scenarios), resulting in a high false interception rate and low validation efficiency.
[0005] Second, high-risk instructions that fail validation are typically discarded and the LLM is requested to regenerate them. This approach leads to interrupted interaction, increased latency, and reduced system throughput. Moreover, requiring the LLM to rewrite the entire instruction may generate new errors, creating a vicious cycle that severely impacts the user experience.
[0006] Third, existing resource scheduling strategies do not consider the priority of digital human interaction experience. When the terminal rendering load or action queue load is too high, the general resource allocation often leads to a serious degradation of core interaction experience such as lip-syncing asynchrony and stiff facial expressions, while some non-critical rendering tasks consume valuable resources.
[0007] The aforementioned problems have become the core bottleneck restricting the transition of LLM-driven digital humans from experimental demonstrations to large-scale commercial applications. Summary of the Invention
[0008] The technical problem to be solved by this invention is to provide an adaptive command safety control system and method for LLM-driven digital humans, addressing the shortcomings of existing fixed rule verification schemes such as poor adaptability, crude handling of high-risk commands, and failure to consider the interactive experience of digital humans in resource scheduling.
[0009] To address the numerous shortcomings of existing technologies, this invention solves them one by one through three sets of core technical solutions. The corresponding logic of each innovation is as follows: Addressing the issue of poor adaptability of fixed verification processes Technical issues: Existing technologies use a unified and fixed verification process to process all instructions, which has weak semantic scene differentiation capabilities, is prone to verification redundancy or lack of constraints, and has low verification efficiency and high false interception rate.
[0010] Technical means: This system parses semantic intent tags of action, interaction and composite scene. The dynamic instruction verification unit selects the corresponding component from the component pool according to the tag and assembles it to form a scene-specific real-time verification pipeline.
[0011] Technical effect: On-demand matching of verification rules eliminates unnecessary computational overhead, while ensuring the effective execution of key constraints in different scenarios, thereby improving the overall performance of verification.
[0012] The problem of crude handling of high-risk instructions Technical issue: Existing solutions directly discard non-compliant instructions and require the entire LLM to be regenerated, causing interaction delays, reduced system throughput, and a vicious cycle of repeated errors.
[0013] Technical measures: This system performs automatic repair operations on high-risk instructions; if the repair is ineffective, it only generates a local regeneration request containing specific parameters and correction suggestions, and after reflow, it only triggers local content modification of the LLM.
[0014] Technical effects: Significantly reduces the number of instruction regenerations, lowers interaction latency, improves system throughput, and ensures the continuity of human-computer interaction.
[0015] Addressing the issues of unreasonable resource scheduling and lack of system adaptability Technical issues: The existing scheduling strategy does not distinguish between interaction priorities, resulting in a severe degradation of the core experience under high load; at the same time, the system cannot autonomously adjust the management and control strategy according to the running status.
[0016] Technical means: This system dynamically schedules instructions based on the priority of interactive experience, prioritizing core instructions such as lip-sync, facial expression, and voice synchronization when resources are scarce; and constructs a closed loop through a full-link status feedback unit to dynamically adjust the system's processing strategy.
[0017] Technical effect: Maintains the core interactive experience under high load scenarios, while achieving adaptive optimization of the entire system process and improving overall operational stability.
[0018] To solve the above-mentioned technical problems, the present invention adopts the following technical solution.
[0019] Firstly, an adaptive instruction safety control system for LLM-driven digital humans is provided, comprising: an input adaptation processing unit, a dynamic instruction verification unit, a digital human execution control unit, and a full-link status feedback unit.
[0020] The input adaptation processing unit receives the natural language text stream output by the large language model, performs byte-level cleaning, length control, and violation feature filtering on the text stream, outputs structured intermediate data, and parses semantic intent tags from the structured intermediate data. The semantic intent tags are at least categorized into action-related, interaction-related, and composite scene-related tags.
[0021] It should be noted that, in this specification, "byte-level cleaning" refers to the process of removing illegal byte sequences from the input UTF-8 text stream, replacing invisible control characters, and standardizing the newline character format.
[0022] The dynamic instruction verification unit is the core innovative unit of this system. This unit connects to the input adaptation processing unit and pre-stores multiple independent verification components. These independent verification components include, but are not limited to: a general format verification component, a parameter range verification component, a human skeleton physical limit component, a motion collision detection component, an expression / lip-syncing component, and a motion rate limiting component. Each component provides a unified, callable interface, supporting on-demand loading and dynamic assembly.
[0023] The dynamic instruction verification unit receives structured intermediate data output by the input adaptation processing unit. Based on the semantic intent tags parsed from this data, it dynamically selects and assembles components from the verification component pool to form a real-time verification pipeline corresponding to the current instruction. The assembly rules include: When the semantic intent tag indicates an action-related scene (such as dancing, turning around, raising an arm, or walking), the assembled real-time verification pipeline includes the human skeleton physical limit component, the action collision detection component, and the motion rate limiting component, but does not include the facial expression and lip-sync component. When the semantic intent label indicates an interactive scenario (such as dialogue, broadcast, or question and answer), the assembled real-time verification pipeline includes an expression and lip-sync component, but does not include a human skeleton physical limit component, a motion collision detection component, or a motion rate limiting component. When the semantic intent label indicates a complex scenario (such as speaking while performing actions), the assembled real-time verification pipeline includes all of the above components, forming a hybrid verification pipeline.
[0024] The assembly described above is performed in real time, and the verification pipeline structure can be different for each instruction. Different semantic scenarios correspond to different combinations of verification components, verification order, and verification weights, achieving scenario-adaptive verification strategies.
[0025] The dynamic instruction verification unit, based on the assembled real-time verification pipeline, verifies the instruction execution format, parameter validity, and the physical movement limits and speed limits of the digital human skeleton joints. For high-risk instructions detected during verification, the dynamic instruction verification unit prioritizes automatic repair. The criteria for determining a high-risk instruction include at least one of the following: the instruction action parameters exceed the physical movement limits of the digital human skeleton; there are conflicting action combinations (such as simultaneous high-speed displacement and extreme torsion); the instruction timing is disordered or sub-instructions conflict with each other; or parameter mutations exceed the preset LLM output distribution range.
[0026] The automatic repair methods include: parameter limiting correction (truncating over-limit parameters to the safe upper limit), parameter softening processing (smoothing the transition of abruptly changed parameters), conflicting sub-instruction elimination (removing mutually exclusive sub-instructions from the instruction queue), and parameter quantization correction (quantizing continuous parameter values into discrete safe levels).
[0027] For high-risk instructions that cannot be automatically corrected after attempts, the dynamic instruction verification unit generates a partial regeneration request and sends it back to the input adaptation processing unit. It should be noted that a "partial regeneration request" refers to a request that only includes the specific parameter fields that need correction and correction suggestions, as opposed to a complete regeneration request that calls for the LLM to regenerate the entire instruction. Upon receiving this request, the input adaptation processing unit triggers the LLM to perform targeted partial content regeneration.
[0028] One of the core innovative mechanisms of this invention lies in its hierarchical processing strategy for high-risk instructions: "automatic repair first, then partial regeneration if repair fails." This strategy differs from the existing approach of directly discarding and requesting a complete rewrite, significantly reducing the number of invalid regenerations, minimizing interaction latency, and improving the overall throughput of the LLM-digital human system.
[0029] The digital human execution control unit is connected to the dynamic instruction verification unit, and is used to parse verified instructions into action control signals and / or rendering control signals that the digital human can execute. The action control signals are used to drive the digital human's skeletal animation system, and the rendering control signals are used to control the digital human's facial expressions, lip movements, and scene rendering.
[0030] The digital human execution control unit is also used to dynamically schedule the execution order of instructions based on the priority of the digital human's interactive experience. Specific scheduling rules include: when the terminal rendering load exceeds a preset threshold (e.g., 80%) or the action queue load exceeds a preset threshold, priority is given to ensuring the execution of lip-sync instructions, facial micro-expression instructions, and voice synchronization instructions; global lighting and shadow rendering instructions, scene rendering instructions, and background effects instructions are delayed; and unnecessary rendering precision is automatically reduced when resources are scarce. Once the load returns to normal levels, full-effect rendering is automatically restored.
[0031] The end-to-end status feedback unit is connected to the input adaptation processing unit, the dynamic instruction verification unit, and the digital human execution control unit. This unit collects the processing status and anomaly information of each unit in real time (including output latency, success rate, filtering interception rate, verification pass rate, repair success rate, rendering frame rate, and load metrics) through a log collection probe, forming a timestamped associated log. When an anomaly is detected, a feedback control signal is sent to the upstream scheduling unit to dynamically adjust processing parameters. For example, when the repair success rate decreases, feedback is sent to the dynamic instruction verification unit to appropriately adjust parameters and soften the boundaries; when the rendering load is too high, feedback is sent to the digital human execution control unit to trigger experience priority scheduling. The second core innovative mechanism of this invention lies in: forming a closed-loop control through end-to-end status feedback, enabling the verification strategy and scheduling strategy to be dynamically adjusted according to the real-time system status.
[0032] In a preferred embodiment, the system is configured as a general security control middleware between a large language model server and a digital human rendering client, providing unified instruction security verification and repair services for multiple concurrent digital human interaction scenarios.
[0033] Secondly, an adaptive instruction safety control method for LLM-driven digital humans is provided, applied to the aforementioned system, including the following steps: Step S1: Receive the natural language text stream output by the large language model, perform byte-level cleaning, length control, and violation feature filtering, output structured intermediate data, and parse semantic intent tags from the structured intermediate data; the semantic intent tags include at least action, interaction, and composite scene categories; Step S2: Based on the semantic intent tag, dynamically select and assemble multiple pre-stored independent verification components to form a real-time verification pipeline corresponding to the current instruction. Step S3: Utilize the real-time verification pipeline to perform format verification, parameter validity verification, and physical movement limit and movement speed limit verification of the digital human skeleton joints; perform automatic repair on high-risk instructions detected by the verification; for high-risk instructions that cannot be automatically repaired, generate a local regeneration request and backflow it to trigger the large language model to perform local regeneration. Step S4: Parse the verified instructions into action control signals and / or rendering control signals that can be executed by the digital human, and dynamically schedule the execution order of instructions according to the priority of the digital human's interactive experience. Step S5: Collect the status and anomaly information of each processing node, form a related log, and feed it back to the upstream scheduling unit to dynamically adjust the processing strategy.
[0034] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method.
[0035] Beneficial effects Compared with the prior art, the present invention has the following beneficial effects: (1) By using a dynamic verification pipeline assembly mechanism based on semantic intent tags, the verification strategy is made scene-adaptive. For action-type scenarios, physical limits and collision detection components are automatically loaded; for interaction-type scenarios, lip-sync components are automatically loaded. This avoids the computational overhead of irrelevant verifications and ensures the effective execution of key constraints in specific scenarios, greatly reducing the false interception rate and improving verification efficiency.
[0036] (2) By using a graded repair and local regeneration backflow mechanism for high-risk instructions, the crude approach of directly discarding and requesting a complete rewrite in the existing technology is changed. Automatic repair is performed first, and when repair fails, only the LLM is requested to perform local regeneration of specific parameters, which greatly reduces the number of invalid regenerations, reduces interaction latency, improves system throughput, and ensures the continuity of interaction.
[0037] (3) By using a dynamic scheduling mechanism based on the priority of digital human interaction experience, core interaction commands such as lip-sync and micro-expression are prioritized when resources are scarce, while non-critical rendering tasks are postponed, ensuring that the core human-computer interaction experience is not degraded.
[0038] (4) Through the closed loop of full-link status feedback, the verification strategy and scheduling strategy can be dynamically adjusted according to the real-time operating status of the system, thus realizing the adaptive optimization of the system.
[0039] (5) This invention can be implemented as a general security control middleware, providing a unified security verification infrastructure for various digital human interaction scenarios, and has good versatility and scalability. Attached Figure Description
[0040] Figure 1 This is a diagram showing the overall system architecture and data flow of the present invention; Figure 2 This is a flowchart of the dynamic assembly process of the verification pipeline based on semantic tags. Figure 3 Flowchart for graded repair and partial regeneration of backflow signaling for high-risk instructions;
[0041] Figure 4 This is a schematic diagram of the joints in a digital human skeletal model and their corresponding verification rules. Detailed Implementation
[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only preferred embodiments of the present invention and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.
[0043] The overall architecture of this invention is a security control middleware deployed between the LLM server and the digital human rendering client. The overall working logic is as follows: After the LLM outputs a natural language command stream, the input adaptation processing unit performs cleaning, structuring, and semantic classification; the dynamic command verification unit adaptively assembles a verification pipeline based on semantic tags to complete command security verification and risk mitigation; compliant commands are parsed and prioritized for rendering by the digital human execution control unit; finally, the end-to-end state feedback unit achieves closed-loop adaptive optimization throughout the entire process. The above process corresponds to the appendix... Figure 1 The overall architecture is shown below.
[0044] This embodiment discloses the complete hardware and software deployment method and basic data processing logic of the system, corresponding to claims 1, 8 and appendices. Figure 1 .
[0045] This system adopts a middleware deployment architecture that separates the front-end and back-end, and can be deployed independently on cloud servers, local workstations or embedded devices, adapting to various digital human scenarios such as virtual live streaming, educational explanations, and intelligent interactive NPCs.
[0046] The input adaptation processing unit is deployed as a server process, monitoring the text output interface of the LLM server in real time through a background service to receive natural language text command streams. The byte-level cleaning process specifically performs the following operations: identifying and removing illegal control characters in the range 0x00 to 0x1F; standardizing the newline character format between Windows and Linux systems; and filtering abnormal byte sequences encoded in UTF-8. The length control process sets a maximum threshold of 4096 characters for a single command; excessively long texts are automatically truncated while retaining the core command fields. Violation feature filtering uses a built-in keyword dictionary and regular expression matching rules to initially intercept vulgar and illegal commands.
[0047] After preprocessing, the input adaptation processing unit parses the text intent using a lightweight semantic classification model, outputting three core semantic tags: action tags (body movements, posture adjustments, dance moves, etc.), interaction tags (voice broadcasting, dialogue and Q&A, facial expression interaction, etc.), and composite scenario tags (speaking while performing actions, facial expressions combined with body movements, etc.). Finally, structured intermediate data carrying semantic tags, instruction parameters, and instruction types is generated and pushed to the dynamic instruction verification unit.
[0048] The dynamic instruction verification unit adopts an FPGA chip hardware acceleration architecture and has six independently callable verification components. Each component is modularly packaged, supporting hot-swapping and dynamic calling, and can be started, stopped, and its parameters adjusted without restarting the system. The digital human execution control unit is equipped with a GPU rendering module and an inverse kinematics skeleton drive controller, supporting real-time skeleton animation calculation and 3D scene rendering. The end-to-end status feedback unit uses a distributed log collection probe to collect the operating data of each unit at the millisecond level, ensuring real-time controllability of the entire process status.
[0049] The overall data flow of the system strictly follows the appendix Figure 1 The logic shown forms a complete closed-loop link from LLM text output, adaptation preprocessing, dynamic verification, instruction execution to status feedback, realizing standardized and secure control over the nondeterministic output of LLM.
[0050] This embodiment details the rules for verifying the dynamic assembly of pipelines, corresponding to claims 2, 3 and appendices. Figure 2 , 4 This is the core implementation method of adaptive verification in this invention.
[0051] The six components in the system's verification component pool are all independently encapsulated, with separate calling interfaces and verification logic. They can be freely combined according to semantic tags, thus overcoming the limitations of traditional fixed verification processes.
[0052] When the input adaptation processing unit parses action-related semantic tags, the system determines that the current scene is a pure body interaction scene, requiring no facial or lip-sync verification. It automatically calls the general format verification component, parameter range verification component, human skeleton physical limit component, action collision detection component, and motion rate limiting component to assemble a dedicated action-related verification pipeline. Combined with the attached... Figure 4 The skeletal verification rules shown comprehensively verify the core joint parameters of the digital human, including the neck, shoulders, spine, hips, knees, and ankles. For example, they limit the neck yaw rotation angle to no more than ±90°, the pitch angle to no more than ±60°, and the roll angle to no more than ±45°; and limit the knee flexion and extension range to between 0° and 180°. Simultaneously, they detect issues such as limb movement collisions and exceeding movement speed limits, thoroughly avoiding limb distortion, clipping, and motion breakdown.
[0053] When the system parses the interactive semantic tags, it determines that the current scenario is a pure dialogue broadcast, with no need for body movements. It only calls the general format verification component, parameter range verification component, and facial expression / lip-sync component, simplifying the verification process. In this scenario, redundant steps such as body physical verification and collision detection are skipped. While ensuring real-time synchronization between lip movements and speech and compliance of facial expression parameters, the system significantly reduces GPU and CPU computing overhead and improves interactive response speed.
[0054] When semantic tags of composite scenarios are parsed, corresponding to the mixed interaction scenario of the digital human speaking and acting at the same time, the system loads all six major verification components to build a full-dimensional hybrid verification pipeline. At the same time, it completes the physical constraints of the limbs, the control of the action rate, the synchronization of lip movements and facial expressions, and the verification of format parameters, taking into account both the safety of the action and the smoothness of the interaction, and adapting to complex commercial interaction scenarios.
[0055] The dynamic assembly mechanism in this embodiment achieves "scenario adaptation and on-demand verification". Compared with the traditional fixed verification scheme, the redundant calculation overhead is significantly reduced and the instruction false interception rate is greatly reduced, effectively solving the problems of rigid and inefficient verification in the existing technology.
[0056] This embodiment discloses a specific implementation method for the layered processing of high-risk instructions, corresponding to claims 4, 5, 6 and the appendix. Figure 3 This addresses the problems of crude instruction processing methods and high interaction latency in existing technologies.
[0057] This system has four preset high-risk instruction judgment criteria, which are fully matched with the physical rules of digital human operation and the output characteristics of LLM, including: skeletal parameters exceeding limits, action combination conflicts, instruction timing disorder, and abnormal parameter mutations.
[0058] For commands deemed high-risk, the system prioritizes automatic repair, matching corresponding repair strategies based on the anomaly type. For parameter overruns, parameter limiting correction is used, automatically truncating excessive joint angles and motion rates to preset safety thresholds. For stuttering caused by parameter mutations, parameter softening is employed, performing smooth inter-frame interpolation on mutated parameters to prevent abrupt transitions. For multi-subcommand conflicts, conflicting subcommands are automatically eliminated, retaining only valid and compliant commands. For continuous parameter precision anomalies, parameter quantization correction normalizes floating-point parameters into safe discrete increments.
[0059] After automatic repair is complete, the system performs a second verification on the corrected command. If the verification passes, the command is executed normally. In cases where automatic repair is not possible due to severe parameter corruption or deep conflicts between multiple commands, the system does not discard the entire command but generates a partial regeneration request. This request only marks the abnormal parameter fields, error type, and correction suggestions, such as only marking "shoulder joint rotation angle exceeds the limit and needs to be adjusted to within ±120°", rather than requiring the entire interaction command to be rewritten.
[0060] After receiving the feedback request, the input adaptation processing unit precisely triggers the LLM to perform local parameter correction, regenerating only abnormal fields while retaining the original compliant instruction content. Compared to traditional whole-sentence regeneration schemes, this mechanism significantly reduces instruction regeneration time from hundreds of milliseconds, significantly lowers the interaction interruption rate, and effectively improves system throughput and interaction continuity. The above process perfectly matches the appendix. Figure 3 The signaling logic is shown below.
[0061] This embodiment discloses a system resource scheduling and end-to-end adaptive optimization mechanism, corresponding to claims 1 and 7, to ensure the core interactive experience under high load scenarios.
[0062] The system has a preset load threshold: when the GPU rendering load reaches 80% or the action command queue backlog reaches 20, the priority scheduling mechanism will be triggered.
[0063] The system categorizes digital human interaction commands into four priority levels, from highest to lowest: Level 1 core commands (lip-sync, facial micro-expressions, voice synchronization), Level 2 important commands (head posture, core gestures), Level 3 secondary commands (finger detail movements, limb fine-tuning), and Level 4 non-core commands (global lighting, background effects, particle rendering). When the load exceeds the limit, the system prioritizes ensuring a stable frame rate for Level 1 commands to guarantee a smooth core human-computer dialogue experience and accurate lip-sync; it appropriately reduces the frame rate for Level 2 and Level 3 commands; and it pauses or downgrades Level 4 non-core rendering tasks to free up GPU computing resources.
[0064] When the system load drops below 60% and remains stable for more than 3 seconds, it automatically resumes full-dimensional rendering and full-command frame rate, achieving dynamic adaptive scheduling.
[0065] Meanwhile, the end-to-end status feedback unit collects key indicators such as verification success rate, repair success rate, rendering frame rate, and load utilization in real time, forming a time-series log. When a continuous decline in the repair success rate is detected, the parameter softening threshold is automatically relaxed; when action over-limit issues occur frequently, the skeleton verification boundary is fine-tuned to achieve dynamic iterative optimization of the system strategy.
[0066] The application scenarios that this invention can support include, but are not limited to, the following types.
[0067] Virtual human live streaming scenario: The system can stably manage the interactive commands generated by LLM 24 / 7, eliminating digital human motion distortion and screen stuttering issues, and greatly improving the stability of live streaming.
[0068] Virtual education instructor scenario: The system can precisely constrain the skeletal parameters of teaching demonstration movements to ensure that teaching movements are standardized and avoid demonstration errors.
[0069] VR game NPC interaction scenarios: The system can support concurrent interaction of multiple NPCs, balancing interaction flexibility and operational stability.
[0070] Virtual companion scenario: The system continuously ensures the smoothness and naturalness of daily conversations and physical interactions.
[0071] This invention employs a lightweight middleware architecture design, requiring no modification to existing LLM models and digital human rendering terminals. It boasts strong compatibility, low deployment costs, and high commercial viability. Any scenario requiring the reliable application of nondeterministic AI output to deterministic virtual human or robot terminals can benefit from this invention.
[0072] The above description is merely a preferred embodiment of the present invention and does not limit the scope of patent protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An adaptive instruction safety control system for a large language model (LLM) driven digital human, comprising an input adaptation processing unit, a dynamic instruction verification unit, a digital human execution control unit, and a full-link status feedback unit, characterized in that: The input adaptation processing unit is used to receive the natural language text stream output by the large language model, perform byte-level cleaning, length control and violation feature filtering on the text stream, output structured intermediate data, and parse semantic intent tags from the structured intermediate data; the semantic intent tags include at least action, interaction and composite scene categories; The dynamic instruction verification unit, connected to the input adaptation processing unit, pre-stores multiple independent verification components. Based on the semantic intent tag, it dynamically selects and assembles these components to form a real-time verification pipeline corresponding to the current instruction. This real-time verification pipeline is used to verify the instruction execution format, parameter validity, and the physical movement limits and movement rate limits of the digital human's skeletal joints. The dynamic instruction verification unit also automatically repairs high-risk instructions detected during verification, and generates a partial regeneration request for high-risk instructions that cannot be automatically repaired, which is then fed back to the input adaptation processing unit. The digital human execution control unit is connected to the dynamic instruction verification unit and is used to parse the verified instructions into action control signals and / or rendering control signals that can be executed by the digital human, and dynamically schedule the instruction execution order according to the priority of the digital human interaction experience. The end-to-end status feedback unit is connected to the input adaptation processing unit, the dynamic instruction verification unit, and the digital human execution control unit, respectively. It is used to collect the processing status and abnormal information of each unit, form an associated log, and feed it back to the upstream scheduling unit to dynamically adjust the processing strategy.
2. The system according to claim 1, characterized in that, The multiple independent verification components include: a general format verification component, a parameter range verification component, a human skeleton physical limit component, a motion collision detection component, an expression and lip-syncing component, and a motion rate limiting component; the dynamic instruction verification unit assembles differentiated real-time verification pipelines for different scenarios based on semantic intent tags.
3. The system according to claim 2, characterized in that, The rules by which the dynamic instruction verification unit dynamically assembles the real-time verification pipeline based on the semantic intent tags include: When the semantic intent tag indicates an action-related scene, the real-time verification pipeline includes the human skeleton physical limit component, the action collision detection component, and the motion rate limiting component, but does not include the facial expression and lip-sync component. When the semantic intent tag indicates an interactive scenario, the real-time verification pipeline includes the facial expression and lip-sync component, but does not include the human skeleton physical limit component, the motion collision detection component, and the motion rate limiting component. When the semantic intent label indicates a composite scene, the real-time verification pipeline simultaneously includes the human skeleton physical limit component, the motion collision detection component, the motion rate limiting component, and the facial expression lip-sync component.
4. The system according to claim 1, characterized in that, The dynamic instruction verification unit adopts a layered processing mechanism for high-risk instructions: first, it performs automatic repair, and the repair methods include at least one of parameter limiting correction, parameter softening processing, conflict sub-instruction elimination, and parameter quantization correction; for instructions that fail to be repaired, it generates a partial regeneration request containing only the parameter field to be corrected and the correction suggestion and reflows it.
5. The system according to claim 1, characterized in that, The conditions for the dynamic instruction verification unit to determine a high-risk instruction include at least one of the following: The command action parameters exceeded the physical movement limits of the digital human skeleton. There are conflicting action combinations; Instruction timing is disordered or sub-instructions conflict with each other; The parameter mutation exceeds the preset output distribution range of the large language model.
6. The system according to claim 1, characterized in that, The local regeneration request generated by the dynamic instruction verification unit only includes the specific parameter fields that need to be corrected and the correction suggestions; after receiving the local regeneration request, the input adaptation processing unit triggers the large language model to perform targeted local content regeneration, rather than overall instruction rewriting.
7. The system according to claim 1, characterized in that, The digital human execution control unit schedules the execution order of instructions according to the priority of interactive experience: when the terminal rendering load or action queue load exceeds the preset threshold, the execution of lip-sync instructions, facial micro-expression instructions and voice synchronization instructions is prioritized, while the execution of global lighting and shadow rendering instructions, scene rendering instructions and background special effects instructions is delayed, and unnecessary rendering precision is automatically reduced when resources are scarce.
8. The system according to any one of claims 1 to 7, characterized in that: The input adaptation processing unit is implemented by a network gateway device or a server process; The dynamic instruction verification unit is implemented by an FPGA chip or a software verification module; The digital human execution control unit includes a GPU rendering module and a skeleton drive controller.
9. An adaptive instruction safety control method for LLM-driven digital humans, applied to the system described in any one of claims 1 to 8, characterized in that, include: The system receives a natural language text stream output from a large language model, performs byte-level cleaning, length control, and violation feature filtering, outputs structured intermediate data, and parses semantic intent tags from the structured intermediate data; the semantic intent tags include at least action-type, interaction-type, and composite scene-type tags. Based on the semantic intent tag, a real-time verification pipeline corresponding to the current instruction is dynamically selected and assembled from multiple pre-stored independent verification components. The real-time verification pipeline is used to verify the execution format of instructions, the validity of parameters, and the physical movement limits and movement speed limits of the digital human skeleton joints. High-risk instructions found by the verification are automatically repaired. For high-risk instructions that cannot be automatically repaired, a local regeneration request is generated and returned to trigger the large language model to perform local regeneration. The verified instructions are parsed into action control signals and / or rendering control signals that can be executed by the digital human, and the execution order of the instructions is dynamically scheduled according to the priority of the digital human's interactive experience. Collect the status and anomaly information of each processing node, form a related log, and feed it back to the upstream scheduling unit to dynamically adjust the processing strategy.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method of claim 9.