A virtual character behavior action prediction method and system based on a neural network

Abstract

The application discloses a virtual role behavior action prediction method and system based on a neural network, which introduces physical rules in the form of executable codes and semantic constraints in the form of hot update rule tables for double checking after model output; under the premise of not retraining the model, the mechanism of combining the high generalization ability of the neural network with the certainty of the rule engine effectively guarantees the legality, rationality and authenticity of the virtual role behavior; through integrated Grad-CAM or SHAP feature attribution analysis, a logical link from environment perception to behavior execution is established, so that the developer can accurately quantify the contribution of position and posture factors to the decision, and through the introduction of controlled disturbance at the input end, the stability is screened, so that the parameters can still remain stable under abnormal interference, and the role jitter and collapse caused by network fluctuation or abnormal operation online are avoided, and the engineering reliability of the system in a complex interactive environment is greatly enhanced.

Description

Technical Field

[0001] This invention relates to the field of virtual character behavior technology, specifically to a method and system for predicting virtual character behavior and actions based on neural networks. Background Technology

[0002] With the deep integration of computer graphics and artificial intelligence technologies, the intelligent behavior of virtual characters (including game NPCs, digital humans, and simulated agents) has become a core indicator for measuring product immersion and interactive experience. Deep learning-based behavior prediction technology, due to its ability to fit complex human operating patterns and physical interaction logic, has been widely used in recent years.

[0003] Existing neural network prediction models are usually data-driven. Although they can generate seemingly natural action sequences, they often lack respect for physical laws and semantic logic. The models are prone to outputting behaviors that violate physical laws, such as clipping, levitation, teleportation, or unnatural joint distortion, as well as errors such as waving weapons or running away from enemies in non-combat situations, which violate scene logic. Traditional post-processing methods mostly use simple collision box detection or hard-coded interception, which not only have narrow coverage but also cannot cope with complex multimodal interaction scenarios, resulting in a significant reduction in the realism of virtual characters. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for predicting the behavior and actions of virtual characters based on neural networks, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting the behavior and actions of virtual characters based on neural networks, comprising the following steps:

[0006] S1. Collect multimodal behavior information of virtual characters within a preset time period, perform time synchronization, formatting and cleaning, and generate reference information based on preset behavior targets before passing it to S2.

[0007] S2. Based on the preprocessed data of S1, behavioral features are extracted using a temporal processing structure. After combining multi-source information fusion and action category semantic encoding, feature data is generated and input into S3.

[0008] S3. Using the feature data from S2, a training task is constructed by dividing the data subset and using a sliding window. Combined with adaptive adjustment and multi-task parallel processing, an optimized processing mechanism is generated through joint evaluation of multi-objective errors and iterative optimization.

[0009] S4. Based on the preliminary behavioral information output by S3, the results are double-checked and adjusted using physical rules in the form of executable code and semantic constraints in the form of hot-update rule tables. After removing illegal and illogical behaviors, the results are used as the input of S5.

[0010] S5. Based on the behavioral information verified by S4, reverse analyze the intermediate state from S2 to S4, quantify the feature contribution and map it to the intention category, and establish a logical link from environmental perception to behavior execution.

[0011] S6. Regarding the processing mechanism of S5 output, abnormal interactive interference within a preset amplitude range is introduced into the input for testing. Versions with parameter fluctuations not exceeding the threshold are selected to form the final runnable version.

[0012] S7. Input the real-time state sequence into the processing mechanism of S6 to output behavioral instructions and drive the role, while dynamically adjusting the input sequence through feedback correction strategy.

[0013] Preferably, step S1 specifically includes the following steps:

[0014] S11. Collect multimodal behavior information of the virtual character within a preset time period, including three-dimensional position change trajectory, skeletal posture data, user input interaction signals and environmental status information; the skeletal posture data includes joint angles and angular velocities; the interaction signals include keyboard commands, mouse commands, touch commands or voice commands; the environmental status information includes terrain type, obstacle distribution and lighting conditions;

[0015] S12. Synchronize the time of each data source to align different modal data on the time axis, and unify the data format and sampling frequency to 30Hz or 60Hz; using a right-handed three-dimensional coordinate system with the system master clock as the time reference, process the numerical features and normalize them to the [0,1] interval or standardize them to a mean of 0 and a variance of 1.

[0016] S13. Complete missing data frames using linear interpolation or spline interpolation; remove outliers using threshold filtering or statistical outlier detection.

[0017] S14. Establish reference information based on expected behavior within a preset time period in the future, including movement direction vector, action category identifier, and target position coordinates; the action category identifier includes walking, running, and jumping; the reference information is appended to the input data of the corresponding time period as input to S2.

[0018] Preferably, step S2 specifically includes the following steps:

[0019] S21. Use a temporal processing structure to extract the dependencies of behavior in the time dimension; the temporal processing structure is selected from LSTM, GRU, and Transformer, and the features of multiple consecutive frames are input in time order, and the output is a temporal feature vector of fixed dimension.

[0020] S22. Using a multi-source information fusion method, position information, attitude information, and environmental information are mapped into feature vectors of the same dimension through independent encoders, and then mapped to a unified feature representation space through concatenation and multilayer perceptron.

[0021] S23. In the fusion process, combine the weight allocation strategy, calculate the weight coefficients of different frames or information sources through the attention mechanism and normalize them through softmax, or set fixed importance coefficients according to the task requirements using a configuration file.

[0022] S24. Semantically encode the action categories. After one-hot encoding, the discrete action categories are connected to the embedding layer and converted into continuous embedding vectors. The dimension of the embedding vectors is consistent with the dimension of the hidden layer of the backbone network. Generate feature data and input it into S3.

[0023] Preferably, step S3 specifically includes the following steps:

[0024] S31. Divide the preprocessed multimodal behavioral information into multiple processing subsets according to action type and environmental scene. Each subset independently maintains a training sample pool and evaluation index. Use a sliding window method to generate input-output pairs.

[0025] Using historical frames of a fixed length as input, the window step size supports overlapping or non-overlapping configurations, and the next future frame immediately outside the window is the prediction target. Multimodal behavior information of the following M frames is predicted from the character state of the previous N frames.

[0026] S32. In the processing, an adaptive adjustment strategy is introduced: in the case of small sample new scenarios, the parameters of the backbone network are frozen and only the output layer is fine-tuned, or the learning rate is adjusted to 0.1 to 1 times the initial value according to the real-time prediction error in online inference; at the same time, through multi-task parallel processing, the same backbone network is shared and connected to independent action classification head and position regression head respectively, and the two outputs are generated synchronously at the same time step, while predicting action category and position change.

[0027] S33. Multi-objective joint error evaluation is adopted, using cross-entropy loss as action classification loss and mean squared error loss as position regression loss. The two types of losses are added together with fixed weights to obtain the total loss.

[0028] S34. Adjust the internal parameters iteratively through backpropagation and configure an early stopping strategy during training: terminate training when the validation set loss no longer decreases within 5-10 consecutive training cycles, thus obtaining the optimized processing mechanism.

[0029] Preferably, step S4 specifically includes the following steps:

[0030] S41. Based on the preliminary behavioral information output by S3, trigger the verification process and load predefined physical rules and semantic constraints; the physical rules are expressed in the form of numerical inequalities or kinematic equations, and the semantic constraints are stored in the form of rule tables, which together constitute the verification benchmark.

[0031] S42. Perform compliance checks on the predicted behavior parameters using physical rules in the form of executable code, and remove objects that penetrate, large displacements, or unnatural joint movements.

[0032] S43. Perform logical verification based on semantic constraints in the form of a rule table. The rule table supports hot updates and does not require retraining of the processing mechanism. Remove attack actions in non-combat states or actions that conflict with scene targets.

[0033] S44. The multimodal behavior information adjusted by physical rule compliance verification and semantic constraint logic verification is used as the input of S5.

[0034] Preferably, step S5 specifically includes the following steps:

[0035] S51. Add a feature attribution analysis step to capture and lock the intermediate state data of S2-S4, including time-series feature vectors, fused feature data, and behavioral parameters before and after verification.

[0036] S52. Perform reverse analysis on the intermediate state, calculate the contribution weight of the feature map or input features to the prediction result, and output it in the form of feature contribution ranking; map the high-dimensional features to human-understandable intention categories, including probing, avoidance, aggression or cooperation, and convert the numerical calculation results into behavioral motivation labels.

[0037] S53. Analyze the causal relationship between input information and predicted behavior, quantify the influence of location, posture and environmental information on the final decision, and establish a logical link from environmental perception, intention inference to behavior execution.

[0038] Preferably, step S6 specifically includes the following steps:

[0039] S61. Regarding the processing mechanism for the output of S5, a controlled disturbance is introduced into the input sequence. The disturbance intensity is controlled by a percentage and does not exceed 30% of the input amplitude.

[0040] S62. The injected disturbances include sudden occlusion, high-frequency operation fluctuations, or transmission delay noise: the sudden occlusion is the random discarding of part of the joint pose data in consecutive frames; the high-frequency operation fluctuation is the simulation of continuous key input exceeding a threshold within a unit time; the transmission delay noise is the introduction of random time offsets into the data frames.

[0041] S63. By repeatedly processing the perturbed input, the processing mechanism maintains stable feature extraction capability and consistent output behavior under uncertain interference; the processing parameters verified by stability testing are locked and solidified to form the final runnable version.

[0042] Preferably, step S7 specifically includes the following steps:

[0043] S71. During the operation of the virtual character, the state sequence of N frames before the current moment is obtained in real time, including position, posture and environment information, and input into the final runnable version processing mechanism of S6.

[0044] S72. The processing mechanism is based on the input state sequence and outputs the distribution of possible behaviors or specific behavior parameters for future moments, including the target position and action category probability; the prediction results are converted into animation system control commands to drive the character to perform walking, jumping, attacking or dodging.

[0045] S73. Compare the actual motion completion rate returned by the animation system with the collision detection results and the predicted values. The deviation threshold is set according to the historical statistical distribution. When the deviation exceeds the preset threshold, the prediction is judged to be abnormal and the correction process is triggered.

[0046] S74. In the correction process, the current frame and several frames before and after it are added back to the input sequence and the input buffer is updated. The weights and composition of the input sequence are adjusted according to environmental changes.

[0047] Preferably, the timing processing structure, encoder, multilayer perceptron, action classification head and position regression head are all implemented by neural networks and run on GPU or dedicated AI acceleration chip.

[0048] The multimodal behavioral information, reference information, and intermediate feature data are all represented in tensor form and are computed and fused within the same feature space;

[0049] The reverse analysis uses Grad-CAM or SHAP values ​​to perform feature attribution analysis on the intermediate state of the model, quantifying the contribution of each input feature to the prediction result.

[0050] During online inference, the state sequence of the past N frames is acquired in real time as input, and the behavioral instructions for future moments are output. Through a feedback correction strategy, the actual action completion degree returned by the animation system and the collision detection results are compared with the predicted values, and the input sequence is dynamically adjusted according to the deviation.

[0051] A virtual character behavior prediction system based on neural networks, characterized in that it includes:

[0052] The data acquisition module is used to receive multimodal behavior information of virtual characters within a preset time period, perform time synchronization, formatting and cleaning processing, and generate reference information based on the preset future time period behavior target;

[0053] The feature fusion module is used to receive preprocessed data and reference information output by the data acquisition module, call the temporal processing structure to extract behavioral features, and output feature data containing temporal and semantic information through multi-source information fusion and action category semantic encoding.

[0054] The model optimization module receives the feature data output by the feature fusion module, constructs a training task by dividing the data subset and sliding window, combines adaptive adjustment and multi-task parallel processing mechanism, generates and outputs the optimized processing mechanism through multi-objective error joint evaluation and iterative optimization.

[0055] The dual verification module receives the preliminary behavioral information output by the model optimization module, performs dual verification and adjustment through physical rules in the form of executable code and semantic constraints in the form of hot update rule tables, removes behaviors that violate physical laws and non-logical behaviors, and outputs the verified behavioral information.

[0056] An interpretability analysis module is used to receive behavioral information output by the dual verification module, perform reverse analysis on intermediate state data in the feature extraction, fusion and verification process, quantify feature contributions and map them to intent categories, and establish and output a logical link from environmental perception to behavior execution.

[0057] The stability verification module is used to receive the logical link and processing mechanism output by the interpretability analysis module, introduce abnormal interactive interference within a preset amplitude range into the input sequence for testing, screen out the version whose parameter fluctuations do not exceed the threshold under the interference conditions, and solidify it into the final runnable version.

[0058] An online feedback module is used to receive the final runnable version output by the stability verification module, input the real-time state sequence into the processing mechanism to output behavioral instructions and drive the role, and dynamically adjust the input sequence through a feedback correction strategy.

[0059] A general execution unit is used to carry out the operation of the feature fusion module, model optimization module, dual verification module and interpretability analysis module, and to perform calculations on a GPU or a dedicated AI acceleration chip.

[0060] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0061] Compared with existing technologies, the beneficial effects of this invention are: by introducing physical rules in the form of executable code (such as numerical inequalities and kinematic equations) and semantic constraints in the form of hot-updated rule tables after the model output, dual verification is performed; this not only intercepts explicit violations such as penetrating objects and unnatural joint movements from a mathematical perspective, but also quickly corrects logical errors of attacks in non-combat states by updating the rule table without retraining the model. The mechanism of combining the high generalization ability of neural networks with the determinism of rule engines effectively ensures the legality, rationality and realism of virtual character behavior;

[0062] By integrating Grad-CAM or SHAP feature attribution analysis, high-dimensional features are mapped into human-understandable intention categories such as probing, avoidance, and aggressive attacks. This establishes a logical link from environmental perception to behavior execution, enabling developers to accurately quantify the contribution of position and posture factors to decision-making. This addresses the pain points of uncontrollable and difficult-to-explain AI behavior. By introducing controlled perturbations (such as sudden occlusion and transmission delay) at the input end for stability screening, it ensures that the final solidified version can maintain parameter stability under abnormal interference, avoiding character jitter and crashes caused by network fluctuations or abnormal operations online. This greatly enhances the engineering reliability of the system in complex interactive environments. Attached Figure Description

[0063] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of the method steps of the present invention. Detailed Implementation

[0065] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses consistent with some aspects of this disclosure as detailed in the appended claims.

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0067] Example 1:

[0068] See Figure 1 As shown in the figure, a method for predicting the behavior of virtual characters based on neural networks according to an embodiment of the present invention includes the following steps:

[0069] S1. Collect multimodal behavior information of virtual characters within a preset time period, perform time synchronization, formatting and cleaning, and generate reference information based on preset behavior targets before passing it to S2.

[0070] S2. Based on the preprocessed data of S1, behavioral features are extracted using a temporal processing structure. After combining multi-source information fusion and action category semantic encoding, feature data is generated and input into S3.

[0071] S3. Using the feature data from S2, a training task is constructed by dividing the data subset and using a sliding window. Combined with adaptive adjustment and multi-task parallel processing, an optimized processing mechanism is generated through joint evaluation of multi-objective errors and iterative optimization.

[0072] S4. Based on the preliminary behavioral information output by S3, the results are double-checked and adjusted using physical rules in the form of executable code and semantic constraints in the form of hot-update rule tables. After removing illegal and illogical behaviors, the results are used as the input of S5.

[0073] S5. Based on the behavioral information verified by S4, reverse analyze the intermediate state from S2 to S4, quantify the feature contribution and map it to the intention category, and establish a logical link from environmental perception to behavior execution.

[0074] S6. Regarding the processing mechanism of S5 output, abnormal interactive interference within a preset amplitude range is introduced into the input for testing. Versions with parameter fluctuations not exceeding the threshold are selected to form the final runnable version.

[0075] S7. Input the real-time state sequence into the processing mechanism of S6 to output behavioral instructions and drive the role, while dynamically adjusting the input sequence through feedback correction strategy;

[0076] It covers the entire process of data acquisition, feature extraction, model optimization, behavior verification, interpretability analysis, stability verification, and online feedback. It solves the problems of disconnect between prediction and execution and lack of closed-loop control in traditional methods. It effectively suppresses clipping and anti-physics behavior through dual physical and semantic verification, and introduces interpretability analysis and stability verification to significantly improve the system's controllability, robustness, and debuggability. It is suitable for application scenarios with high requirements for behavioral realism, such as games, simulations, and metaverse.

[0077] Example 2:

[0078] Step S1 specifically includes the following steps:

[0079] S11. Collect multimodal behavior information of the virtual character within a preset time period, including three-dimensional position change trajectory, skeletal posture data, user input interaction signals and environmental status information; skeletal posture data includes joint angles and angular velocities; interaction signals include keyboard commands, mouse commands, touch commands or voice commands; environmental status information includes terrain type, obstacle distribution and lighting conditions;

[0080] S12. Synchronize the time of each data source to align different modal data on the time axis, and unify the data format and sampling frequency to 30Hz or 60Hz; using a right-handed three-dimensional coordinate system with the system master clock as the time reference, process the numerical features and normalize them to the [0,1] interval or standardize them to a mean of 0 and a variance of 1.

[0081] S13. Complete missing data frames using linear interpolation or spline interpolation; remove outliers using threshold filtering or statistical outlier detection.

[0082] S14. Establish reference information based on expected behavior within a preset time period in the future, including movement direction vector, action category identifier, and target position coordinates; action category identifier includes walking, running, and jumping; the reference information is appended to the input data for the corresponding time period as input to S2;

[0083] By acquiring multimodal data and standardizing preprocessing, the system comprehensively integrates location, skeletal pose, user interaction, and environmental information. It eliminates jitter caused by frame rate differences based on time synchronization and unified sampling frequency, and improves data integrity by handling missing and outlier values. At the same time, it introduces reference information on future behavior to provide clear optimization goals for supervised learning, thereby improving the accuracy and stability of predictions and providing a reliable data foundation for subsequent feature extraction and decision-making.

[0084] Step S2 specifically includes the following steps:

[0085] S21. Use temporal processing structures to extract the dependencies of behavior in the time dimension; the temporal processing structures are selected from LSTM, GRU, and Transformer, which input the features of multiple consecutive frames in temporal order and output a temporal feature vector of fixed dimension.

[0086] S22. Using a multi-source information fusion method, position information, attitude information, and environmental information are mapped into feature vectors of the same dimension through independent encoders, and then mapped to a unified feature representation space through concatenation and multilayer perceptron.

[0087] S23. In the fusion process, combine the weight allocation strategy, calculate the weight coefficients of different frames or information sources through the attention mechanism and normalize them through softmax, or set fixed importance coefficients according to the task requirements using a configuration file.

[0088] S24. Semantically encode the action categories. After one-hot encoding, the discrete action categories are connected to the embedding layer and converted into continuous embedding vectors. The dimension of the embedding vectors is consistent with the dimension of the hidden layer of the backbone network. Generate feature data and input it into S3.

[0089] Using a temporal processing structure to capture long-term behavioral dependencies and adapt to complex operation sequences, multi-source independent encoding and fusion avoid intermodal interference, and attention or fixed weight allocation dynamically focuses on key frames or information sources. At the same time, discrete action categories are transformed into continuous semantic embedding vectors, which significantly enhances the temporal dependence and semantic expressive power of behavioral features, enabling the model to more accurately distinguish different action types and their potential intentions.

[0090] Step S3 specifically includes the following steps:

[0091] S31. Divide the preprocessed multimodal behavioral information into multiple processing subsets according to action type and environmental scene. Each subset independently maintains a training sample pool and evaluation index. Use a sliding window method to generate input-output pairs.

[0092] Using historical frames of a fixed length as input, the window step size supports overlapping or non-overlapping configurations, and the next future frame immediately outside the window is the prediction target. Multimodal behavior information of the following M frames is predicted from the character state of the previous N frames.

[0093] S32. In the processing, an adaptive adjustment strategy is introduced: in the case of small sample new scenarios, the parameters of the backbone network are frozen and only the output layer is fine-tuned, or the learning rate is adjusted to 0.1 to 1 times the initial value according to the real-time prediction error in online inference; at the same time, through multi-task parallel processing, the same backbone network is shared and connected to independent action classification head and position regression head respectively, and the two outputs are generated synchronously at the same time step, while predicting action category and position change.

[0094] S33. Multi-objective joint error evaluation is adopted, using cross-entropy loss as action classification loss and mean squared error loss as position regression loss. The two types of losses are added together with fixed weights to obtain the total loss.

[0095] S34. Adjust the internal parameters iteratively through backpropagation and configure an early stopping strategy during training: terminate training when the validation set loss no longer decreases within 5-10 consecutive training cycles, thus obtaining the optimized processing mechanism.

[0096] To reduce cross-domain distribution differences, subsets are divided according to action type and environmental scene. Training samples are constructed using a sliding window to adapt to historical inputs of different lengths. Adaptive adjustment and multi-task parallel processing are introduced to flexibly adjust the learning strategy in small sample or online inference scenarios. At the same time, action category and position changes are predicted simultaneously to improve the model's generalization ability, training efficiency and prediction accuracy, and avoid overfitting or information loss caused by single-task training.

[0097] Step S4 specifically includes the following steps:

[0098] S41. Based on the preliminary behavioral information output by S3, the verification process is triggered, and predefined physical rules and semantic constraints are loaded. The physical rules are expressed in the form of numerical inequalities or kinematic equations, and the semantic constraints are stored in the form of rule tables, which together constitute the verification benchmark.

[0099] S42. Perform compliance checks on the predicted behavior parameters using physical rules in the form of executable code, and remove objects that penetrate, large displacements, or unnatural joint movements.

[0100] S43. Perform logical verification based on semantic constraints in the form of a rule table. The rule table supports hot updates and does not require retraining of the processing mechanism. Remove attack actions in non-combat states or actions that conflict with scene targets.

[0101] S44. The multimodal behavior information adjusted by physical rule compliance verification and semantic constraint logic verification is used as the input of S5;

[0102] By efficiently intercepting objects, ultra-high-speed displacement, and non-physiological joint movements through physical rules in the form of executable code, and relying on hot-updated rule tables to quickly correct logical errors that conflict with the scene, such as non-combat attacks, the system ensures the legality and rationality of predicted behavior without retraining the model, significantly reducing the incidence of abnormal behavior and improving the realism of virtual character behavior and user acceptance.

[0103] Step S5 specifically includes the following steps:

[0104] S51. Add a feature attribution analysis step to capture and lock the intermediate state data of S2-S4, including time-series feature vectors, fused feature data, and behavioral parameters before and after verification.

[0105] S52. Perform reverse analysis on the intermediate state, calculate the contribution weight of the feature map or input features to the prediction result, and output it in the form of feature contribution ranking; map the high-dimensional features to human-understandable intention categories, including probing, avoidance, aggression or cooperation, and convert the numerical calculation results into behavioral motivation labels.

[0106] S53. Analyze the causal relationship between input information and predicted behavior, quantify the influence of location, posture and environmental information on the final decision, and establish a logical link from environmental perception, intention inference to behavior execution.

[0107] By introducing controlled disturbances into the input to simulate real-world anomalies such as sudden occlusion, high-frequency operational fluctuations, and transmission delays, model versions that maintain stable parameters under disturbances are selected. This prevents online behavior jitter and crashes, ensures the consistency of the final version's performance under different terminals and network environments, and enhances the engineering reliability of the system in complex interactive environments.

[0108] Step S6 specifically includes the following steps:

[0109] S61. Regarding the processing mechanism for the output of S5, a controlled disturbance is introduced into the input sequence. The disturbance intensity is controlled by a percentage and does not exceed 30% of the input amplitude.

[0110] S62. The injected disturbances include sudden occlusion, high-frequency operation fluctuations, or transmission delay noise: sudden occlusion is the random discarding of part of the joint pose data in consecutive frames; high-frequency operation fluctuations are the simulation of continuous key inputs exceeding a threshold per unit time; transmission delay noise is the introduction of random time offsets into the data frames.

[0111] S63. By repeatedly processing the perturbed input, the processing mechanism maintains stable feature extraction capability and consistent output behavior under uncertain interference; the processing parameters verified by stability testing are locked and solidified to form the final runnable version.

[0112] By introducing controlled disturbances into the input to simulate real-world anomalies such as sudden occlusion, high-frequency operational fluctuations, and transmission delays, model versions that maintain stable parameters under disturbances are selected. This prevents online behavior jitter and crashes, ensures the consistency of the final version's performance under different terminals and network environments, and enhances the engineering reliability of the system in complex interactive environments.

[0113] Step S7 specifically includes the following steps:

[0114] S71. During the operation of the virtual character, the state sequence of N frames before the current moment is obtained in real time, including position, posture and environment information, and input into the final runnable version processing mechanism of S6.

[0115] S72. The processing mechanism is based on the input state sequence and outputs the distribution of possible behaviors or specific behavior parameters for future moments, including the target position and action category probability; it converts the prediction results into animation system control commands to drive the character to perform walking, jumping, attacking, or dodging.

[0116] S73. Compare the actual motion completion rate returned by the animation system with the collision detection results and the predicted values. The deviation threshold is set according to the historical statistical distribution. When the deviation exceeds the preset threshold, the prediction is judged to be abnormal and the correction process is triggered.

[0117] S74. In the correction process, the current frame and several frames before and after it are added back to the input sequence and the input buffer is updated. The weights and composition of the input sequence are adjusted according to environmental changes.

[0118] Based on real-time prediction of future behavior using N historical frames, the system meets the requirements for low-latency interaction and directly converts the prediction results into animation control commands to achieve integrated prediction and execution. At the same time, it compares the actual action completion with the collision detection results to identify deviations, triggering a correction process and dynamically adjusting the input sequence in case of anomalies. This allows the character to adapt to environmental changes and sudden changes in player behavior, achieving continuous and stable intelligent performance.

[0119] Among them, the temporal processing structure, encoder, multilayer perceptron, action classification head and position regression head are all implemented by neural networks and run on GPU or dedicated AI acceleration chip;

[0120] Multimodal behavioral information, reference information, and intermediate feature data are all represented in tensor form and are computed and fused within the same feature space;

[0121] Inverse analysis uses Grad-CAM or SHAP values ​​to perform feature attribution analysis on the intermediate state of the model, quantifying the contribution of each input feature to the prediction result;

[0122] During online inference, the state sequence of the past N frames is acquired in real time as input, and the behavioral instructions for future moments are output. Through a feedback correction strategy, the actual action completion degree returned by the animation system and the collision detection results are compared with the predicted values, and the input sequence is dynamically adjusted according to the deviation.

[0123] By deploying GPUs or dedicated AI acceleration chips, computational efficiency is improved. End-to-end tensor representation and unified feature space operations reduce data conversion overhead. Interpretable algorithms such as Grad-CAM or SHAP are integrated to balance performance and interpretability. At the same time, online inference and feedback correction form continuous adaptive capabilities to ensure stable operation of the system in high-concurrency real-time scenarios.

[0124] Example 3:

[0125] A virtual character behavior prediction system based on neural networks, characterized in that it includes:

[0126] The data acquisition module is used to receive multimodal behavior information of virtual characters within a preset time period, perform time synchronization, formatting and cleaning processing, and generate reference information based on the preset future time period behavior targets;

[0127] The feature fusion module receives preprocessed data and reference information from the data acquisition module, calls the temporal processing structure to extract behavioral features, and outputs feature data containing temporal and semantic information through multi-source information fusion and action category semantic encoding.

[0128] The model optimization module receives the feature data output by the feature fusion module, constructs a training task by dividing the data subset and sliding window, and generates and outputs the optimized processing mechanism through joint evaluation of multi-objective errors and iterative optimization, combined with adaptive adjustment and multi-task parallel processing mechanism.

[0129] The dual verification module receives the preliminary behavioral information output by the model optimization module, performs dual verification and adjustment through physical rules in the form of executable code and semantic constraints in the form of hot update rule tables, removes behaviors that violate physical laws and non-logical behavior, and outputs the verified behavioral information.

[0130] The interpretability analysis module receives the behavioral information output by the dual verification module, performs reverse analysis on the intermediate state data in the feature extraction, fusion and verification process, quantifies the feature contribution and maps it to the intent category, and establishes and outputs the logical link from environmental perception to behavior execution.

[0131] The stability verification module receives the logical links and processing mechanisms output by the interpretability analysis module, introduces abnormal interactive interference within a preset amplitude range into the input sequence for testing, filters out versions whose parameter fluctuations do not exceed the threshold under interference conditions, and solidifies them into the final runnable version.

[0132] The online feedback module receives the final runnable version output by the stability verification module, inputs the real-time status sequence into the processing mechanism to output behavioral instructions and drive the role, and dynamically adjusts the input sequence through the feedback correction strategy.

[0133] The general execution unit is used to carry out the operation of the feature fusion module, model optimization module, double verification module and interpretability analysis module, and performs calculations on GPU or dedicated AI acceleration chip;

[0134] It provides a modular and scalable system-level implementation solution with clear responsibilities for each module, facilitating division of labor in development and maintenance. The general execution unit uniformly carries the core computing logic, supports cross-platform deployment, and has good evolvability, making it convenient to expand new functional modules such as emotion and social interaction in the future, and meeting the needs of different application stages for intelligent upgrades of virtual character behavior.

[0135] Example 4:

[0136] The data acquisition module receives multimodal behavioral information of the virtual character within a preset time period, including 3D position change trajectory, skeletal pose data, user input interaction signals, and environmental state information. This data undergoes time-synchronized formatting, cleaning, and normalization. Simultaneously, reference information is generated based on expected behavior within a preset future time period, providing a high-quality and time-consistent input foundation for subsequent processing. Next, the feature fusion module uses temporal processing structures such as LSTMGRU or Transformer to extract the temporal dependencies of behavior. Through multi-source information fusion methods, position, pose, and environmental information are encoded and mapped to a unified feature representation space. Combined with attention mechanisms or fixed weight allocation strategies and semantic encoding of action categories, feature data rich in temporal and semantic information is generated. Subsequently, the model optimization module utilizes this feature data, dividing it into subsets according to action type and environmental scene, and constructing training tasks through a sliding window. Adaptive adjustment strategies such as freezing the backbone network or dynamically adjusting the learning rate, along with multi-task parallel processing mechanisms, are employed to simultaneously predict action classification and position regression. An optimized processing mechanism is generated through multi-objective error joint evaluation and iterative optimization. The dual verification module loads physical rules in executable code form and a hot-update rule table form. The system employs several mechanisms: **Constraints:** Initial behavioral information undergoes dual verification for compliance and logic, removing unnatural joint movements that penetrate objects and abnormal behaviors that don't align with the scene's logic. The interpretability analysis module reverse-engineers intermediate data from feature extraction, fusion, and verification, quantifying the contribution weight of each input feature to the prediction result and mapping it to human-understandable intent categories, establishing a logical link from environmental perception to behavior execution. The stability verification module introduces abnormal interactive interference within a preset amplitude range into the input sequence, such as sudden occlusion, high-frequency operational fluctuations, or transmission delay noise, for testing. Stable versions that do not exceed threshold fluctuations under disturbances are selected and solidified. The final runnable version is achieved by having the online feedback module input the real-time acquired character state sequence into the processing mechanism, output future action instructions and convert them into animation system control instructions to drive the character to perform corresponding actions. At the same time, the actual action completion degree returned by the animation system and the collision detection results are compared with the predicted values. When the deviation exceeds the preset threshold, the correction process is triggered and the weight and composition of the input sequence are dynamically adjusted. The entire system is carried by a general-purpose execution unit and runs on a GPU or dedicated AI acceleration chip. All data is processed in the same feature space in the form of tensors, thereby achieving high-precision prediction, stable operation and continuous optimization of virtual character behavior.

[0137] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0138] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope of this disclosure is indicated by the following claims.

Claims

1. A method for predicting the behavior and actions of virtual characters based on neural networks, characterized in that, Includes the following steps: S1. Collect multimodal behavior information of virtual characters within a preset time period, perform time synchronization, formatting and cleaning, and generate reference information based on preset behavior targets before passing it to S2. S2. Based on the preprocessed data of S1, behavioral features are extracted using a temporal processing structure. After combining multi-source information fusion and action category semantic encoding, feature data is generated and input into S3. S3. Using the feature data from S2, a training task is constructed by dividing the data subset and using a sliding window. Combined with adaptive adjustment and multi-task parallel processing, an optimized processing mechanism is generated through joint evaluation of multi-objective errors and iterative optimization. S4. Based on the preliminary behavioral information output by S3, the results are double-checked and adjusted using physical rules in the form of executable code and semantic constraints in the form of hot-update rule tables. After removing illegal and illogical behaviors, the results are used as the input of S5. S5. Based on the behavioral information verified by S4, reverse analyze the intermediate state from S2 to S4, quantify the feature contribution and map it to the intention category, and establish a logical link from environmental perception to behavior execution. S6. Regarding the processing mechanism of S5 output, abnormal interactive interference within a preset amplitude range is introduced into the input for testing. Versions with parameter fluctuations not exceeding the threshold are selected to form the final runnable version. S7. Input the real-time state sequence into the processing mechanism of S6 to output behavioral instructions and drive the role, while dynamically adjusting the input sequence through feedback correction strategy.

2. The method for predicting virtual character behavior and actions based on neural networks according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11. Collect multimodal behavior information of the virtual character within a preset time period, including three-dimensional position change trajectory, skeletal posture data, user input interaction signals and environmental status information; the skeletal posture data includes joint angles and angular velocities; the interaction signals include keyboard commands, mouse commands, touch commands or voice commands; the environmental status information includes terrain type, obstacle distribution and lighting conditions; S12. Synchronize the time of each data source to align different modal data on the time axis, and unify the data format and sampling frequency to 30Hz or 60Hz; using a right-handed three-dimensional coordinate system with the system master clock as the time reference, process the numerical features and normalize them to the [0,1] interval or standardize them to a mean of 0 and a variance of 1. S13. Complete missing data frames using linear interpolation or spline interpolation; remove outliers using threshold filtering or statistical outlier detection. S14. Establish reference information based on expected behavior within a preset time period in the future, including movement direction vector, action category identifier, and target position coordinates; the action category identifier includes walking, running, and jumping; the reference information is appended to the input data of the corresponding time period as input to S2.

3. The method for predicting virtual character behavior and actions based on neural networks according to claim 2, characterized in that, Step S2 specifically includes the following steps: S21. Use a temporal processing structure to extract the dependencies of behavior in the time dimension; the temporal processing structure is selected from LSTM, GRU, and Transformer, and the features of multiple consecutive frames are input in time order, and the output is a temporal feature vector of fixed dimension. S22. Using a multi-source information fusion method, position information, attitude information, and environmental information are mapped into feature vectors of the same dimension through independent encoders, and then mapped to a unified feature representation space through concatenation and multilayer perceptron. S23. In the fusion process, combine the weight allocation strategy, calculate the weight coefficients of different frames or information sources through the attention mechanism and normalize them through softmax, or set fixed importance coefficients according to the task requirements using a configuration file. S24. Semantically encode the action categories. After one-hot encoding, the discrete action categories are connected to the embedding layer and converted into continuous embedding vectors. The dimension of the embedding vectors is consistent with the dimension of the hidden layer of the backbone network. Generate feature data and input it into S3.

4. The method for predicting virtual character behavior and actions based on neural networks according to claim 3, characterized in that, Step S3 specifically includes the following steps: S31. Divide the preprocessed multimodal behavioral information into multiple processing subsets according to action type and environmental scene. Each subset independently maintains a training sample pool and evaluation index. Use a sliding window method to generate input-output pairs. Using historical frames of a fixed length as input, the window step size supports overlapping or non-overlapping configurations, and the next future frame immediately outside the window is the prediction target. Multimodal behavior information of the following M frames is predicted from the character state of the previous N frames. S32. In the processing, an adaptive adjustment strategy is introduced: in the case of small sample new scenarios, the parameters of the backbone network are frozen and only the output layer is fine-tuned, or the learning rate is adjusted to 0.1 to 1 times the initial value according to the real-time prediction error in online inference; at the same time, through multi-task parallel processing, the same backbone network is shared and connected to independent action classification head and position regression head respectively, and the two outputs are generated synchronously at the same time step, while predicting action category and position change. S33. Multi-objective joint error evaluation is adopted, using cross-entropy loss as action classification loss and mean squared error loss as position regression loss. The two types of losses are added together with fixed weights to obtain the total loss. S34. Adjust the internal parameters iteratively through backpropagation and configure an early stopping strategy during training: terminate training when the validation set loss no longer decreases within 5-10 consecutive training cycles, thus obtaining the optimized processing mechanism.

5. The method for predicting virtual character behavior and actions based on neural networks according to claim 4, characterized in that, Step S4 specifically includes the following steps: S41. Based on the preliminary behavioral information output by S3, trigger the verification process and load predefined physical rules and semantic constraints; the physical rules are expressed in the form of numerical inequalities or kinematic equations, and the semantic constraints are stored in the form of rule tables, which together constitute the verification benchmark. S42. Perform compliance checks on the predicted behavior parameters using physical rules in the form of executable code, and remove objects that penetrate, large displacements, or unnatural joint movements. S43. Perform logical verification based on semantic constraints in the form of a rule table. The rule table supports hot updates and does not require retraining of the processing mechanism. Remove attack actions in non-combat states or actions that conflict with scene targets. S44. The multimodal behavior information adjusted by physical rule compliance verification and semantic constraint logic verification is used as the input of S5.

6. The method for predicting virtual character behavior based on neural networks according to claim 5, characterized in that, Step S5 specifically includes the following steps: S51. Add a feature attribution analysis step to capture and lock the intermediate state data of S2-S4, including time-series feature vectors, fused feature data, and behavioral parameters before and after verification. S52. Perform reverse analysis on the intermediate state, calculate the contribution weight of the feature map or input features to the prediction result, and output it in the form of feature contribution ranking; map the high-dimensional features to human-understandable intention categories, including probing, avoidance, aggression or cooperation, and convert the numerical calculation results into behavioral motivation labels. S53. Analyze the causal relationship between input information and predicted behavior, quantify the influence of location, posture and environmental information on the final decision, and establish a logical link from environmental perception, intention inference to behavior execution.

7. The method for predicting virtual character behavior and actions based on neural networks according to claim 6, characterized in that, Step S6 specifically includes the following steps: S61. Regarding the processing mechanism for the output of S5, a controlled disturbance is introduced into the input sequence. The disturbance intensity is controlled by a percentage and does not exceed 30% of the input amplitude. S62. The injected disturbances include sudden occlusion, high-frequency operation fluctuations, or transmission delay noise: the sudden occlusion is the random discarding of part of the joint pose data in consecutive frames; the high-frequency operation fluctuation is the simulation of continuous key input exceeding a threshold within a unit time; the transmission delay noise is the introduction of random time offsets into the data frames. S63. By repeatedly processing the perturbed input, the processing mechanism maintains stable feature extraction capability and consistent output behavior under uncertain interference; the processing parameters verified by stability testing are locked and solidified to form the final runnable version.

8. The method for predicting virtual character behavior and actions based on neural networks according to claim 7, characterized in that, Step S7 specifically includes the following steps: S71. During the operation of the virtual character, the state sequence of N frames before the current moment is obtained in real time, including position, posture and environment information, and input into the final runnable version processing mechanism of S6. S72. The processing mechanism is based on the input state sequence and outputs the distribution of possible behaviors or specific behavior parameters for future moments, including the target position and action category probability; the prediction results are converted into animation system control commands to drive the character to perform walking, jumping, attacking or dodging. S73. Compare the actual motion completion rate returned by the animation system with the collision detection results and the predicted values. The deviation threshold is set according to the historical statistical distribution. When the deviation exceeds the preset threshold, the prediction is judged to be abnormal and the correction process is triggered. S74. In the correction process, the current frame and several frames before and after it are added back to the input sequence and the input buffer is updated. The weights and composition of the input sequence are adjusted according to environmental changes.

9. The method for predicting virtual character behavior and actions based on neural networks according to claim 8, characterized in that, The timing processing structure, encoder, multilayer perceptron, action classification head and position regression head are all implemented by neural networks and run on GPU or dedicated AI acceleration chip. The multimodal behavioral information, reference information, and intermediate feature data are all represented in tensor form and are computed and fused within the same feature space; The reverse analysis uses Grad-CAM or SHAP values ​​to perform feature attribution analysis on the intermediate state of the model, quantifying the contribution of each input feature to the prediction result. During online inference, the state sequence of the past N frames is acquired in real time as input, and the behavioral instructions for future moments are output. Through a feedback correction strategy, the actual action completion degree returned by the animation system and the collision detection results are compared with the predicted values, and the input sequence is dynamically adjusted according to the deviation.

10. A virtual character behavior prediction system based on neural networks, characterized in that, include: The data acquisition module is used to receive multimodal behavior information of virtual characters within a preset time period, perform time synchronization, formatting and cleaning processing, and generate reference information based on the preset future time period behavior target; The feature fusion module is used to receive preprocessed data and reference information output by the data acquisition module, call the temporal processing structure to extract behavioral features, and output feature data containing temporal and semantic information through multi-source information fusion and action category semantic encoding. The model optimization module receives the feature data output by the feature fusion module, constructs a training task by dividing the data subset and sliding window, combines adaptive adjustment and multi-task parallel processing mechanism, generates and outputs the optimized processing mechanism through multi-objective error joint evaluation and iterative optimization. The dual verification module receives the preliminary behavioral information output by the model optimization module, performs dual verification and adjustment through physical rules in the form of executable code and semantic constraints in the form of hot update rule tables, removes behaviors that violate physical laws and non-logical behaviors, and outputs the verified behavioral information. An interpretability analysis module is used to receive behavioral information output by the dual verification module, perform reverse analysis on intermediate state data in the feature extraction, fusion and verification process, quantify feature contributions and map them to intent categories, and establish and output a logical link from environmental perception to behavior execution. The stability verification module is used to receive the logical link and processing mechanism output by the interpretability analysis module, introduce abnormal interactive interference within a preset amplitude range into the input sequence for testing, screen out the version whose parameter fluctuations do not exceed the threshold under the interference conditions, and solidify it into the final runnable version. An online feedback module is used to receive the final runnable version output by the stability verification module, input the real-time state sequence into the processing mechanism to output behavioral instructions and drive the role, and dynamically adjust the input sequence through a feedback correction strategy. A general execution unit is used to carry out the operation of the feature fusion module, model optimization module, dual verification module and interpretability analysis module, and to perform calculations on a GPU or a dedicated AI acceleration chip.

Images