Interactive systems and methods for 3D virtual digital humans

Abstract

This invention discloses a 3D virtual digital human interaction system and method, relating to the fields of human-computer interaction and artificial intelligence. It addresses the problem of traditional methods relying on preset templates, lacking natural and personalized expression, and resulting in poor interaction adaptability. The system obtains the current behavioral parameters of the virtual digital human and the user's interaction intent, calculates basic action logic, and initially matches the action model range according to the interaction type. During the interaction phase, it detects the signal strength of the input device, sets strength conditions to extract the data stream, establishes initial response logic, collects dynamic behavior, and constructs a random forest model to predict behavioral trends. It combines the prediction results with the model for adaptability classification; if the adaptability is low, it collects interaction samples through an auxiliary channel for visualization simulation, and filters suitable samples and models. Based on the simulation results, it matches the interaction scene, replaces the initial scene, and executes actions, achieving intelligent response and natural interaction of the virtual digital human.

Description

Technical Field

[0001] This invention relates to the fields of human-computer interaction and artificial intelligence technology, and more specifically, to an interaction system and method for 3D virtual digital humans. Background Technology

[0002] With the development of virtual reality (VR), augmented reality (AR) and digital twin technologies, 3D virtual digital humans, as an important carrier of human-computer interaction, are widely used in online education, virtual customer service, digital exhibitions, film and television entertainment and other fields. In existing technologies, the interactive behavior of virtual digital humans mostly relies on preset action models and static matching rules, that is, by recognizing user input signals and matching corresponding action instructions, the virtual digital human is driven to perform specific behaviors.

[0003] The existing technology has the following shortcomings:

[0004] Currently, traditional methods for virtual digital humans rely heavily on preset templates for responses, failing to generate differentiated actions in real time based on user behavior. This results in a lack of natural and personalized performance, poor interaction adaptability, and an inability to dynamically optimize action matching strategies based on historical interaction samples. Consequently, the intelligence level of virtual interaction systems and the quality of user experience are reduced. Therefore, this paper proposes an interaction system and method for 3D virtual digital humans.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an interactive system and method for 3D virtual digital humans, which solves the problems mentioned in the background art by employing dynamic behavior prediction models, input signal strength perception mechanisms, sample realization simulation methods, and scene adaptive control strategies.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a 3D virtual digital human interaction system and method, comprising the following steps:

[0008] Step S1: Obtain the current behavior parameters of the virtual digital human, collect user interaction intent information, calculate the basic action logic based on the above information, and initially match the action model range according to the interaction type;

[0009] Step S2: During the user interaction phase, detect the signal strength of the input device, set different strength conditions to detect the interactive data stream, set the initial response logic of the virtual digital human through the model range, collect its dynamic behavior, and build a random forest model to predict its dynamic behavior.

[0010] Step S3: Combine the dynamic behavior prediction value with the preliminary matched action model to classify the behavior adaptability. If the adaptability is low, collect user interaction samples through the auxiliary channel, perform visualization simulation, and screen out samples and their corresponding models within the standard adaptability range.

[0011] Step S4: Based on the matching scenario of the simulation calculation sample interaction data, infer the scenario matched by the virtual digital human interaction data stream, and perform the corresponding action after replacing the initial setting scenario.

[0012] In a preferred embodiment, in step S1, the current behavior parameters of the virtual digital human are obtained from the system, including spatial location vector, action identifier, facial expression state code and action intensity, and the current behavior parameters are combined into a current state vector.

[0013] The system collects user interaction intent information from input devices, analyzes and extracts the interaction semantics through intelligent recognition tools, obtains the interaction type, target object, and emotional tendency, and combines them into a user intent vector.

[0014] In a preferred embodiment, in step S1, the basic action logic is constructed by combining the current state vector and the user intent vector to obtain the candidate action model:

[0015] The current state vector and the user intent vector are merged as input conditions. Rule matching is performed in the preset rule base to filter out the action identifiers corresponding to the matching rule items and retrieve the candidate action model set.

[0016] Retrieve a subset of action models belonging to the current interaction type from the preset action library, filter the candidate action model set, and retain the candidate action models belonging to the current interaction type subset as the initial range of action models for matching.

[0017] In a preferred embodiment, in step S2, the signal of the user input device is detected in real time, and the strength of different input devices is weighted by preset weights to obtain the device signal strength value;

[0018] Preset a first strength threshold and a second strength threshold to classify the device signal strength into different user intent strength levels:

[0019] If the device signal strength is greater than or equal to the first strength threshold, it is judged as a high intent interaction strength;

[0020] If the device signal strength is between the first strength threshold and the second strength threshold, it is determined to be the desired interaction strength;

[0021] If the device signal strength is less than or equal to the first strength threshold, it is judged as low intent interaction strength.

[0022] In a preferred embodiment, in step S2, the interactive data stream under different user intent strength levels is detected, including the current state vector, user intent vector, and device signal strength.

[0023] The initial response logic is constructed based on the user intent intensity level and the range of candidate action models. The response action features of the current intent intensity level are matched as the initial response action model.

[0024] A multi-level conditional matching mechanism is constructed based on user intent intensity levels and candidate action models:

[0025] Condition 1: The intensity of user intent matches the intensity of action to filter candidate action models;

[0026] Condition 2: Select the candidate action model with the most coordinate points in the action path parameters;

[0027] Condition 3: The facial expression adjustment parameters match the emotional tendency, and candidate action models are selected;

[0028] The candidate action models obtained from the final screening are used as the initial response action models. The virtual digital human performs the corresponding actions and records the current state vector during the response execution process.

[0029] In a preferred embodiment, a random forest model is constructed based on the interactive data stream and the current state vector to predict the movement trends of the virtual digital human.

[0030] The interactive data stream and the current state vector are merged as samples, and non-numerical features are converted into numerical features through feature encoding. A time series sample set is generated using a time sliding window.

[0031] After performing random forest ensemble and training, the predicted behavior vectors are obtained, including:

[0032] Action identifier: Select the action identifier that appears most frequently in the action identifiers predicted by the sub-decision tree as the action identifier in the predicted behavior vector;

[0033] Action intensity: The arithmetic mean of the action intensity values ​​predicted by the corresponding sub-decision tree is used as the action intensity in the predicted behavior vector.

[0034] In a preferred embodiment, in step S3, the initial response action model is classified for adaptability based on the predicted behavior vector:

[0035] If the action identifiers are consistent, the applicability is considered high; otherwise, the applicability is considered low.

[0036] When the adaptability is determined to be high, the initial response action model is used as the standard adaptability action model;

[0037] When the compatibility is determined to be low, interaction samples are collected through an auxiliary channel, including user intent vector, current state vector and initial response action model, and environmental context information recorded during the interaction.

[0038] By receiving interaction samples through visualization simulation, a virtual interaction scene matching the information of the interaction samples is constructed, driving the virtual digital human to perform corresponding actions based on the candidate action model.

[0039] In a preferred embodiment, in step S3, dynamic behavioral response data parameters during the simulation process are collected, and the parameters are normalized and then summed to obtain a comprehensive response score.

[0040] Determine whether an interaction sample is a standard-fit interaction sample based on the overall response score:

[0041] If the overall response score is greater than the preset score threshold, it is judged as a standard adaptive action model;

[0042] If the overall response score is less than the preset score threshold, auxiliary channel sampling and visualization simulation will be performed.

[0043] In a preferred embodiment, in step S4, a matching scenario of sample interaction data is calculated based on standard adaptability samples, and the dynamic behavior response data is normalized and then merged into an interaction feature vector.

[0044] Each scene tag in the preset scene tag library corresponds to a different scene feature vector. The matching degree between the interaction feature vector and the scene feature vector is calculated by Euclidean distance.

[0045] Replace the current scene label of the virtual digital human with the scene label with the highest similarity, and complete the specific response of the action execution according to the standard adaptive action model.

[0046] The 3D virtual digital human interaction system includes an interaction intent parsing module, a dynamic behavior prediction module, an adaptability determination and simulation module, and a scene reasoning and action execution module.

[0047] The interaction intent parsing module is used to collect virtual digital human behavior parameters and user interaction intent, construct basic action logic, and match the action model range;

[0048] The dynamic behavior prediction module sets the initial response logic based on signal strength, collects dynamic behavior data, uses random forest to predict behavior, and passes it to subsequent modules for behavior adaptability evaluation.

[0049] The adaptability assessment and simulation module is used to evaluate behavioral adaptability, assist the channel in collecting samples and realizing simulations, and screen out samples and their corresponding models within the standard adaptability range.

[0050] The scene reasoning and action execution module is used for scene reasoning and replacement, and realizes action scheduling and virtual digital human execution.

[0051] The technical effects and advantages of this invention are as follows:

[0052] 1. This invention acquires the current behavioral parameters of a virtual digital human, collects user interaction intent information, calculates basic action logic based on the above information, and initially matches the action model range according to the interaction type. During the user interaction phase, it detects the signal strength of the input device, sets different strength conditions to detect the interaction data stream, sets the initial response logic of the virtual digital human through the model range, collects its dynamic behavior, and constructs a random forest model to predict its dynamic behavior. It classifies the behavior adaptability by combining the predicted dynamic behavior value with the initially matched action model. If the adaptability is low, it collects user interaction samples through an auxiliary channel, performs a visualization simulation, and filters out samples and their corresponding models within the standard adaptability range. Based on the visualization simulation, it calculates the matching scenario of the sample interaction data, infers the scenario matched by the virtual digital human's interaction data stream, and executes the corresponding action after replacing the initially set scenario. This achieves a deep understanding of user input and natural and smooth action behavior of the virtual digital human. Attached Figure Description

[0053] Figure 1 This is a flowchart of the interaction method for the 3D virtual digital human of the present invention.

[0054] Figure 2 This is a schematic diagram of the interactive system of the 3D virtual digital human of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Example 1

[0057] Please see Figure 1 The interaction method for 3D virtual digital humans, and the specific operation process are as follows:

[0058] Step S1: Obtain the current behavior parameters of the virtual digital human, collect user interaction intent information, calculate the basic action logic based on the above information, and initially match the action model range according to the interaction type;

[0059] Step S2: During the user interaction phase, detect the signal strength of the input device, set different strength conditions to detect the interactive data stream, set the initial response logic of the virtual digital human through the model range, collect its dynamic behavior, and build a random forest model to predict its dynamic behavior.

[0060] Step S3: Combine the dynamic behavior prediction value with the preliminary matched action model to classify the behavior adaptability. If the adaptability is low, collect user interaction samples through the auxiliary channel, perform visualization simulation, and screen out samples and their corresponding models within the standard adaptability range.

[0061] Step S4: Based on the matching scenario of the simulation calculation sample interaction data, infer the scenario matched by the virtual digital human interaction data stream, and perform the corresponding action after replacing the initial setting scenario.

[0062] The specific implementation is as follows:

[0063] In step S1, the current behavior parameters of the virtual digital human are obtained from the system, including spatial location vector, action identifier, facial expression state code and action intensity, and the current behavior parameters are combined into a current state vector.

[0064] The spatial position vector represents the current position coordinates of the virtual digital human in three-dimensional space, the action identifier represents the action model number currently being performed by the virtual digital human, the facial expression state code represents the set of encoded values ​​for the current state of the facial expression, and the action intensity represents the dynamic amplitude of the current action.

[0065] Collect user interaction intent information from input devices, including voice input information, image input information, touch input information, action input information, and text input information;

[0066] Intelligent recognition tools are used to analyze and extract the semantics of user interaction intent information to unify user interaction intent under different modalities, obtain their interaction type, target object, emotional tendency, and combine them into a user intent vector.

[0067] Interaction type refers to the category of user interaction behavior, target object refers to the object identifier or location information of the intended action, and emotion tendency refers to the emotional orientation of the user during interaction.

[0068] By combining the current state vector and the user intent vector, basic action logic is constructed to obtain candidate action models:

[0069] The current state vector and the user intent vector are merged and used as input conditions for rule matching. Each rule item is matched sequentially in the preset rule base. When the matching condition of a rule item is consistent with the input condition, the corresponding rule item is determined to be a matching rule item. The action identifier in the matching rule item is filtered out, and the corresponding candidate action model is retrieved and merged into a candidate action model set.

[0070] The candidate action model includes action identifier, action start and end time, action intensity, action path parameters, and facial expression adjustment parameters;

[0071] The current interaction type is obtained, and the candidate action model set is further filtered. The virtual digital human action models are classified according to different interaction types in the preset action library and divided into different action model subsets, corresponding to the semantics of different interaction types. For example, they are divided into action model subsets such as "greeting", "request", "feedback" and "command".

[0072] Filter the subset of action models corresponding to the interaction type, and select the candidate action models belonging to the subset of action models from the candidate action model set as the initial range of matching action models;

[0073] It should be noted that input devices refer to hardware devices used to collect user input, acquiring multimodal input signals from users during interaction, including voice commands, facial expressions, gestures, keyboard input, etc.; intelligent recognition tools include, but are not limited to: automatic speech recognition engines, image or video analysis frameworks, facial expression recognition tools, and device signal monitoring modules, to parse user interaction intent information. For example, the automatic speech recognition engine transcribes the voice input information, which is then input into a natural language understanding model to extract the interaction type label from the sentence. For images or video streams captured by camera devices, the image or video analysis framework extracts and calculates the coordinates of the user's skeletal key points to determine the user's action type, etc.; the preset rule library is a set of behavior decision rules predefined and stored in the system, used to determine the appropriate action logic output based on the current behavior state of the virtual digital human and the user's interaction intent; the preset action library is a set of predefined, callable structured action templates based on different interaction types. For example, if the interaction type is a greeting, the matching actions include waving, smiling, etc.

[0074] In step S2, signals from the user input device are detected in real time, including volume intensity in the voice channel, confidence level of gesture recognition in the video channel, and amplitude of hand movements in the touch channel.

[0075] Calculate the signal strength of the integrated input device signal: ,in, , , , , , These are volume intensity, posture recognition confidence, and hand movement amplitude, respectively. This refers to the device signal strength value.

[0076] A first strength threshold and a second strength threshold are preset, wherein the first strength threshold is greater than the second strength threshold. The device signal strength is divided into different user intent strength levels to achieve hierarchical recognition of user interaction intent.

[0077] If the device signal strength is greater than or equal to the first strength threshold, it is judged as a high intent interaction strength;

[0078] If the device signal strength is between the first strength threshold and the second strength threshold, it is determined to be the desired interaction strength;

[0079] If the device signal strength is less than or equal to the first strength threshold, it is judged as low intent interaction strength.

[0080] Detect the interaction data stream under different user intent strength levels, including user intent vector and device signal strength, and use the interaction data stream as input data for the random forest model;

[0081] The initial response logic is constructed based on the user intent intensity level and the range of candidate action models. The response action features of the current intent intensity level are matched as the initial response action model.

[0082] A multi-level conditional matching mechanism is constructed based on user intent intensity levels and candidate action models:

[0083] Condition 1: Match the intensity of user intent with the intensity of action to filter candidate action models;

[0084] Condition 2: Select the candidate action model with the most coordinate points in the action path parameters;

[0085] Condition 3: The facial expression adjustment parameters match the emotional tendency, and candidate action models are selected;

[0086] The candidate action models obtained from the final screening are used as the initial response action models. The virtual digital human performs the corresponding actions and records the current state vector during the response execution process.

[0087] A random forest model is constructed based on the interactive data stream and the current state vector to predict the subsequent action trends of the virtual digital human.

[0088] Data collection: The interactive data stream and the current state vector are merged as samples. Non-numerical features in the samples are mapped into numerical form through a feature encoding conversion mechanism. Samples at different times are collected through a time sliding window and merged into a sample set according to the time series.

[0089] The process of constructing a single-subject decision tree:

[0090] From all the features of the sample, m sub-features are randomly selected, where m is less than the total number of features, to form the candidate feature set of the current tree;

[0091] Iterate through each feature in the candidate feature set and calculate the split point based on the principle of minimizing the mean squared error:

[0092] For each candidate feature, traverse all split points, calculate the sum of the variances of the target variables of the left and right subsets after splitting, and select the feature with the smallest total variance and its split point as the basis for splitting the current node.

[0093] Repeat the above steps for the generated left and right child nodes, recursively constructing subtrees until the number of node samples is less than the preset sample threshold.

[0094] Random forest ensemble and training: K subsets of samples are randomly selected with replacement from the sample set to ensure the diversity of training data for each tree; the process of building single-subject decision trees is repeated for each decision tree, and candidate feature subsets are randomly selected when splitting at each node.

[0095] Prediction output: The output of a single decision tree is a multi-dimensional prediction vector, including action identifier and action intensity. The frequency of the action identifiers predicted by all decision trees is counted, and the one with the highest frequency is selected as the action identifier. The arithmetic mean of the action intensity predicted by the corresponding single decision tree is calculated as the action intensity, and it is combined with the action identifier to form a predicted behavior vector.

[0096] It should be noted that the random forest model is an ensemble learning algorithm that makes predictions by constructing multiple decision trees; the encoding conversion mechanism refers to converting non-numerical categorical features into numerical representations using encoding tools; the action identifiers are discrete labels and do not require numerical conversion, as the system directly performs frequency statistics on the action identifiers output by all decision trees; the preset sample threshold is used to determine whether to continue splitting the current node, and is set by professionals.

[0097] In step S3, the initial response action model is classified for adaptability based on the predicted behavior vector:

[0098] If the action identifier in the predicted behavior vector is consistent with the action identifier in the initial response action model, it is judged as having high fitness; otherwise, it is judged as having low fitness.

[0099] When the adaptability is determined to be high, the initial response action model is used as the standard adaptability action model;

[0100] When the compatibility is judged to be low, user interaction samples are collected through auxiliary channels, including user intent vector, current state vector and initial response action model, and environmental context information recorded during the interaction. The environmental context information includes interaction scene identifier, interaction time, user state, previous interaction behavior sequence and its results.

[0101] By realizing the interaction samples collected by the auxiliary sampling channel, a virtual interaction scene matching the interaction sample information is constructed. The virtual digital human is driven to perform corresponding actions according to the initial response action model. The virtual digital human's action posture, expression changes and action intensity are rendered in real time to generate a three-dimensional animation with continuity and naturalness.

[0102] In the completed virtual interactive scene, the initial response action model of the realization simulation engine drives the virtual digital human to perform corresponding actions. The system renders the virtual digital human's posture, facial expression changes and action intensity in real time, generating a three-dimensional animation process with continuity and naturalness.

[0103] During the action simulation process, the visualization simulation engine continuously collects dynamic behavior response data, including action trajectory similarity, facial expression and emotion matching score, spatial interaction target hit rate, behavior continuity score, and action response delay time. The dynamic behavior response data is normalized and then summed to obtain the comprehensive response score.

[0104] Determine whether an interaction sample is a standard-fit interaction sample based on the overall response score:

[0105] If the overall response score is greater than the preset score threshold, it is judged as a standard adaptive interaction sample and associated with its corresponding candidate action model;

[0106] If the overall response score is less than the preset score threshold, auxiliary channel sampling and visualization simulation will be performed.

[0107] It should be noted that the auxiliary channel refers to the additional data acquisition and sample generation pathway activated during the interaction between the virtual digital human and the user, when the system detects that the current action model library cannot meet the high adaptability requirements; the realization simulation refers to the use of 3D simulation technology to realistically reproduce the interaction process between the user and the virtual digital human in a virtual environment based on the collected user interaction samples and the current behavioral parameters of the virtual digital human, thereby realizing the visualization and dynamic verification of the interaction scenario; the preset scoring threshold is set by professionals, for example, the sum of the mean and standard deviation of the adaptability score is used as the preset scoring threshold, and the method is not unique, so it will not be elaborated here.

[0108] In step S4, based on the standard adaptive samples and the standard adaptive action model, the matching scenario of the interaction data of the simulation samples is calculated. The dynamic behavior response data of the simulation is continuously collected, normalized and then merged into an interaction feature vector.

[0109] Each set of scene tags in the preset scene tag library corresponds to a different scene feature vector, where the dimensions and meanings of each parameter are consistent with those of the interaction feature vector. The matching degree between the interaction feature vector and the scene feature vector is calculated using Euclidean distance. ,in, Let i be the i-th parameter in the interaction feature vector. Let be the i-th parameter corresponding to the j-th scene in the scene feature vector, and m be the total number of parameters. Similarity;

[0110] The system replaces the current scene label of the virtual digital human with the scene label with the highest similarity. In the updated matching scene, the system completes the specific driving of action execution based on the standard adaptive action model, including action trajectory control, facial expression encoding and action intensity adjustment.

[0111] The system achieves accurate scene matching and dynamic scene replacement based on real-world simulation samples, ensuring the scene adaptability and execution effect of virtual digital human interactive actions, and realizing an efficient and natural multimodal user interaction experience.

[0112] It should be noted that the preset scene tag library is used to establish a tag system for various interactive scenes. The scene tag corresponds to the scene feature vector, which is used to numerically describe the interactive attributes of the corresponding scene.

[0113] Example 2

[0114] Interactive systems based on 3D virtual digital humans, such as Figure 2 As shown, the interaction method for realizing 3D virtual digital human includes an interaction intent parsing module, a dynamic behavior prediction module, an adaptability determination and simulation module, and a scene reasoning and action execution module.

[0115] The interaction intent parsing module is used to collect virtual digital human behavior parameters and user interaction intent, construct basic action logic, and match the action model range;

[0116] The dynamic behavior prediction module sets the initial response logic based on signal strength, collects dynamic behavior data, uses random forest to predict behavior, and passes it to subsequent modules for behavior adaptability evaluation.

[0117] The adaptability assessment and simulation module is used to evaluate behavioral adaptability, assist the channel in collecting samples and realizing simulations, and screen out samples and their corresponding models within the standard adaptability range.

[0118] The scene reasoning and action execution module is used for scene reasoning and replacement, and realizes action scheduling and virtual digital human execution.

[0119] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0120] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0121] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0122] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0123] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0124] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0129] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A 3D virtual digital human interaction method, characterized by: Includes the following steps: Step S1: Obtain the current behavior parameters of the virtual digital human, collect user interaction intent information, calculate the basic action logic based on the above information, and initially match the action model range according to the interaction type; Step S2: During the user interaction phase, detect the signal strength of the input device, set different strength conditions to detect the interactive data stream, set the initial response logic of the virtual digital human through the model range, collect its dynamic behavior, and build a random forest model to predict its dynamic behavior. Step S3: Combine the dynamic behavior prediction value with the preliminary matched action model to classify the behavior adaptability. If the adaptability is low, collect user interaction samples through the auxiliary channel, perform visualization simulation, and screen out samples and their corresponding models within the standard adaptability range. If the adaptability is high, use the initial response action model as the standard adaptability action model. Step S4: If the adaptability is low, calculate the scene matched by the virtual digital human interaction data stream based on the matching scene of the simulation calculation sample interaction data, and perform the corresponding action after replacing the initial setting scene.

2. The interaction method for a 3D virtual digital human according to claim 1, characterized in that: In step S1, the current behavior parameters of the virtual digital human are obtained from the system, including spatial location vector, action identifier, facial expression state code and action intensity, and the current behavior parameters are combined into a current state vector. The system collects user interaction intent information from input devices, analyzes and extracts the interaction semantics through intelligent recognition tools, obtains the interaction type, target object, and emotional tendency, and combines them into a user intent vector.

3. The interaction method for a 3D virtual digital human according to claim 2, characterized in that: In step S1, the basic action logic is constructed by combining the current state vector and the user intent vector to obtain the candidate action model: The current state vector and the user intent vector are merged as input conditions. Rule matching is performed in the preset rule base to filter out the action identifiers corresponding to the matching rule items and retrieve the candidate action model set. Retrieve a subset of action models belonging to the current interaction type from the preset action library, filter the candidate action model set, and retain the candidate action models belonging to the current interaction type subset as the initial range of action models for matching.

4. The interaction method for a 3D virtual digital human according to claim 3, characterized in that: In step S2, the signal of the user input device is detected in real time, and the strength of different input devices is weighted by preset weights to obtain the device signal strength value; Preset a first strength threshold and a second strength threshold to classify the device signal strength into different user intent strength levels: If the device signal strength is greater than or equal to the first strength threshold, it is judged as a high intent interaction strength; If the device signal strength is between the first strength threshold and the second strength threshold, it is determined to be the desired interaction strength; If the device signal strength is less than or equal to the first strength threshold, it is judged as low intent interaction strength.

5. The interaction method for a 3D virtual digital human according to claim 4, characterized in that: In step S2, the interaction data stream under different user intent strength levels is detected, including the current state vector, user intent vector, and device signal strength; The initial response logic is constructed based on the user intent intensity level and the range of candidate action models. The response action features of the current intent intensity level are matched as the initial response action model. A multi-level conditional matching mechanism is constructed based on user intent intensity levels and candidate action models: Condition 1: The intensity of user intent matches the intensity of action to filter candidate action models; Condition 2: Select the candidate action model with the most coordinate points in the action path parameters; Condition 3: The facial expression adjustment parameters match the emotional tendency, and candidate action models are selected; The candidate action models obtained from the final screening are used as the initial response action models. The virtual digital human performs the corresponding actions and records the current state vector during the response execution process.

6. The interaction method for a 3D virtual digital human according to claim 5, characterized in that: A random forest model is constructed based on the interactive data stream and the current state vector to predict the movement trends of the virtual digital human. The interactive data stream and the current state vector are merged as samples, and non-numerical features are converted into numerical features through feature encoding. A time series sample set is generated using a time sliding window. After performing random forest ensemble and training, the predicted behavior vectors are obtained, including: Action identifier: Select the action identifier that appears most frequently in the action identifiers predicted by the sub-decision tree as the action identifier in the predicted behavior vector; Action intensity: The arithmetic mean of the action intensity values ​​predicted by the corresponding sub-decision tree is used as the action intensity in the predicted behavior vector.

7. The interaction method for a 3D virtual digital human according to claim 6, characterized in that: In step S3, the initial response action model is classified for adaptability based on the predicted behavior vector: If the action identifiers are consistent, the applicability is considered high; otherwise, the applicability is considered low. When the compatibility is determined to be low, interaction samples are collected through an auxiliary channel, including user intent vector, current state vector and initial response action model, and environmental context information recorded during the interaction. By receiving interaction samples through visualization simulation, a virtual interaction scene matching the information of the interaction samples is constructed, driving the virtual digital human to perform corresponding actions based on the candidate action model.

8. The interaction method for a 3D virtual digital human according to claim 7, characterized in that: In step S3, dynamic behavioral response data parameters are collected during the simulation process, and the parameters are normalized and then summed to obtain the comprehensive response score. Determine whether an interaction sample is a standard-fit interaction sample based on the overall response score: If the overall response score is greater than the preset score threshold, it is judged as a standard adaptive action model; If the overall response score is less than the preset score threshold, auxiliary channel sampling and visualization simulation will be performed.

9. The interaction method for a 3D virtual digital human according to claim 8, characterized in that: In step S4, the matching scenario of sample interaction data is calculated based on the standard adaptability sample to realize the simulation. The dynamic behavior response data is normalized and then merged into an interaction feature vector. Each scene tag in the preset scene tag library corresponds to a different scene feature vector. The matching degree between the interaction feature vector and the scene feature vector is calculated by Euclidean distance. Replace the current scene label of the virtual digital human with the scene label with the highest similarity, and complete the specific response of the action execution according to the standard adaptive action model.

10. A 3D virtual digital human interaction system, used to implement the 3D virtual digital human interaction method according to any one of claims 1-9, characterized in that: It includes an interaction intent parsing module, a dynamic behavior prediction module, an adaptability determination and simulation module, and a scene reasoning and action execution module; The interaction intent parsing module is used to collect virtual digital human behavior parameters and user interaction intent, construct basic action logic, and match the action model range; The dynamic behavior prediction module sets the initial response logic based on signal strength, collects dynamic behavior data, uses random forest to predict behavior, and passes it to subsequent modules for behavior adaptability evaluation. The adaptability assessment and simulation module is used to evaluate behavioral adaptability. If the adaptability is low, samples are collected and visualized through auxiliary channels to screen out samples and their corresponding models within the standard adaptability range. If the adaptability is high, the initial response action model is used as the standard adaptability action model. The scene reasoning and action execution module is used to perform scene reasoning and replacement when the adaptability is low, so as to realize action scheduling and virtual digital human execution.

Images