Business training scene updating method and device, computer device, and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and discloses a business training scene updating method and device in the field of financial technology, computer equipment and a storage medium. The method obtains the interaction data of voice, video and behavior modalities of a target training user in a current training scene, extracts features, obtains multi-modal features, aligns the multi-modal features in a time sequence window, obtains multi-modal alignment features, fuses the multi-modal alignment features, obtains training fusion features, scores and predicts the training fusion features through a preset evaluation model, obtains stress index scores, mastery index scores and standardization index scores, and when at least one score meets a scene adjustment condition, the current training scene is updated in terms of scene parameters, so that the target training user completes the next training in the updated training scene. The application realizes accurate evaluation of training results and linkage adjustment of training scenes, and helps to improve the training effect.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more particularly to a method, apparatus, computer equipment, and storage medium for updating business training scenarios in the field of financial technology. Background Technology

[0002] In the financial insurance industry, the professional skills of insurance agents directly determine an insurance company's sales, service, and brand image. To improve the professional skills of insurance agents, insurance companies often need to conduct business training. Against the backdrop of digital transformation in the insurance industry, online training has been widely promoted. However, existing online business training, based on task-oriented and FAQ-based methods, has several shortcomings. Firstly, the generation of business scenarios heavily relies on manually preset templates, limiting training to repetitive basic scenarios and failing to meet the dynamic generation needs of complex claims scenarios. Secondly, the depth of multimodal data fusion in the training process is insufficient; voice, operational behavior, and visual information are analyzed independently, making it impossible to accurately evaluate the professional skills of each insurance agent, and hindering reasonable scenario updates based on the linkage between training evaluation and training scenarios, thus making it difficult to guarantee the effectiveness of business training. Summary of the Invention

[0003] Therefore, it is necessary to provide a method, device, computer equipment, and storage medium for updating business training scenarios to address the aforementioned technical problems, thereby resolving the issues of business training being limited to repetitive training in basic scenarios and the lack of linkage between training evaluation and training scenarios.

[0004] A method for updating business training scenarios includes: Acquire voice interaction data, video interaction data, and behavioral interaction data of the target training users in the current training scenario, and extract features from the voice interaction data, video interaction data, and behavioral interaction data to obtain voice modal features, video modal features, and behavioral modal features; Temporal window alignment processing is performed on the speech modal features, video modal features, and behavioral modal features to obtain speech aligned features, video aligned features, and behavioral aligned features; The speech alignment features, video alignment features, and behavior alignment features are fused together to obtain the training fusion features; The training integration features are scored and predicted using a preset evaluation model to obtain stress index scores, mastery index scores, and standardization index scores. When at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, the scenario parameters of the current training scenario are updated based on the stress index score, mastery index score, and standardization index score to obtain an updated training scenario, so that the target training user can complete the next training in the updated training scenario.

[0005] A business training scenario updating device, comprising: The feature extraction module is used to acquire voice interaction data, video interaction data and behavioral interaction data of the target training user in the current training scenario, and to extract features from the voice interaction data, video interaction data and behavioral interaction data to obtain voice modal features, video modal features and behavioral modal features; The feature alignment module is used to perform temporal window alignment processing on the speech modal features, video modal features and behavioral modal features to obtain speech aligned features, video aligned features and behavioral aligned features; The feature fusion module is used to fuse the speech alignment features, video alignment features, and behavior alignment features to obtain training fusion features; The scoring prediction module is used to perform scoring prediction processing on the training integration features through a preset evaluation model to obtain stress index scores, mastery index scores, and standardization index scores. The scenario update module is used to update the scenario parameters of the current training scenario based on the stress index score, mastery index score, and standardization index score when at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, thereby obtaining an updated training scenario, so that the target training user can complete the next training in the updated training scenario.

[0006] A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor executes the computer-readable instructions to implement the above-described business training scenario update method.

[0007] A computer-readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the business training scenario update method described above.

[0008] In the above method, device, computer equipment and storage medium for updating business training scenarios, the method for updating business training scenarios obtains voice interaction data, video interaction data and behavior interaction data of a target training user in the current training scenario, extracts features therefrom, and obtains voice modality features, video modality features and behavior modality features; performs time-series window alignment processing on the voice modality features, video modality features and behavior modality features to obtain voice alignment features, video alignment features and behavior alignment features; performs fusion processing on the voice alignment features, video alignment features and behavior alignment features to obtain training fusion features; performs scoring prediction processing on the training fusion features through a preset evaluation model to obtain stress index scores, mastery index scores and normality index scores; when at least one of the stress index scores, mastery index scores and normality index scores meets the scenario adjustment condition, updates the scenario parameters of the current training scenario according to the stress index scores, mastery index scores and normality index scores to obtain an updated training scenario, so that the target training user completes the next training in the updated training scenario. The present invention deeply analyzes multi-modal data of a user's voice, video and operation behavior in the current training scenario, obtains quantitative evaluation scores in the stress dimension, mastery dimension and normality dimension, and realizes accurate evaluation of the training results of each business personnel. At the same time, the present invention can generate a dynamic adjustment strategy based on the evaluation results, realize the linkage adjustment between training evaluation and training scenarios, meet the training requirements of composite training scenarios, and help improve the business training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts.

[0010] Figure 1 is a schematic diagram of an application environment of the method for updating a business training scenario in an embodiment of the present invention; Figure 2 is a schematic flowchart of the method for updating a business training scenario in an embodiment of the present invention; Figure 3 is a schematic structural diagram of the device for updating a business training scenario in an embodiment of the present invention; Figure 4 is a schematic diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] The business training scenario update method provided in this embodiment can be applied to, for example, Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0013] The business training scenario update method of this embodiment can be applied to the financial and insurance industry. In business scenarios such as financial transactions and insurance claims, business personnel need to use online tests, simulated operations, practical exercises, and job assessments to test their mastery of theoretical knowledge such as financial regulations, product terms, and claims procedures, as well as their operational standardization in actual work. Specifically, for training in financial transactions, the client sends training interaction data from the simulated transaction system to the server, and the server assesses the business personnel's proficiency in transaction processes, risk warnings, and system operations. For training in insurance claims, the client sets up simulated claims scenarios and sends training interaction data to the server, which assesses the claims personnel's practical abilities in document review, on-site investigation, and claims calculation. The server uses big data analysis of the trainees' training interaction data to generate evaluation results and updates the content and methods of the training scenarios based on these results, such as adding practical training to address weaknesses.

[0014] In one embodiment, such as Figure 2 As shown, a method for updating business training scenarios is provided, which is applied in... Figure 2 The server shown includes the following steps S10-S50.

[0015] S10. Obtain the voice interaction data, video interaction data and behavioral interaction data of the target training user in the current training scenario, and extract features from the voice interaction data, video interaction data and behavioral interaction data to obtain voice modal features, video modal features and behavioral modal features.

[0016] Understandably, users can input voice, video, and behavioral interaction data through the client's display interface. The client generates voice interaction data, video interaction data, and behavioral interaction data based on the user's input and sends these data to the server. Target training users refer to specific users undergoing business training in the current training scenario. The current training scenario refers to the specific contextual content that provides users with an immersive learning experience by simulating or recreating a real business environment based on specific training objectives. Interaction data is the recorded data generated when users interact with the simulation system and other training users in the training scenario. Voice interaction data is the recording of the user's voice signals during training; video interaction data is the recording of the user's facial video during training; and behavioral interaction data is the recording of the user's operational procedures during training. Voice modal features are a series of feature parameters reflecting specific attributes of voice; video modal features are a series of feature parameters reflecting facial expression attributes; and behavioral modal features are a series of feature parameters reflecting behavioral logical attributes.

[0017] In one specific embodiment, the client synchronously collects interactive data in three modalities: voice, video, and behavior, based on a heterogeneous sensor array. Specifically, an 8-microphone linear array is used to collect voice interaction data at a sampling frequency of 16kHz. An RGB-D camera is used to collect video interaction data with Kinect v4, a resolution of 640×480@30fps, and a depth accuracy of ±2mm@2m. Behavioral interaction data is obtained by recording operation events (clicks / swipes / inputs) using the touchscreen SDK, with a timestamp accuracy of 1ms. This embodiment constructs a full-stack heterogeneous data acquisition system. Through the coordinated deployment of multi-microphone array directional enhancement technology and RGB-D camera facial detection, it overcomes the limitations of traditional single-modal acquisition, achieving synchronous capture of the training user's voice, facial micro-expressions, and operational behaviors. A high-precision synchronization mechanism ensures timeline consistency.

[0018] In one embodiment, step S10, namely, extracting features from the voice interaction data, video interaction data, and behavioral interaction data to obtain voice modal features, video modal features, and behavioral modal features, includes: S101. The voice interaction data is processed by feature analysis using a preset voice analysis model to obtain audio features and emotion features, and voice modal features are generated based on the audio features and emotion features; the audio features are obtained by inputting the voice interaction data into the audio feature extraction tool in the preset voice analysis model, and the emotion features are obtained by inputting the audio features into the sentiment analysis network in the preset voice analysis model. S102. Perform facial analysis processing on the video interaction data using a facial analysis tool to obtain facial features, and determine the facial features as video modal features; S103. Perform feature analysis on the behavioral interaction data to obtain speech professionalism features, operation pattern features and operation efficiency features, and generate voice modal features based on the speech professionalism features, operation pattern features and operation efficiency features.

[0019] Understandably, the preset speech analysis model is a pre-trained neural network model that reflects specific attributes of speech from speech interaction data, including an audio feature extraction tool and a sentiment analysis network. The server first inputs the speech interaction data into the audio feature extraction tool within the preset speech analysis model to obtain audio features, then inputs the audio features into the sentiment analysis network within the preset speech analysis model to obtain emotion features. Finally, the set of audio features and emotion features is determined as the speech modal features. Specifically, the audio feature extraction tool in the preset speech analysis model is OpenSMILE 2.3, used to extract speech fundamental frequency and energy features, and the sentiment analysis network in the preset speech analysis model is the Wav2Vec2 model, used to achieve emotion type recognition.

[0020] A facial analysis tool is a pre-built application used to extract facial expression attributes from video interaction data. Specifically, the facial analysis tool is OpenFace 2.2. The server uses the facial analysis tool to perform facial analysis on the video interaction data, extract facial motion units, obtain facial features (68 facial key points + pupil detection), and determine the facial features as video modal features.

[0021] The server performs feature analysis on the behavioral interaction data to obtain features of speech professionalism, operational patterns, and operational efficiency. The set of these features is defined as the voice modal features. Speech professionalism features are quantitative values ​​extracted from the behavioral interaction data to characterize the user's level of language professionalism in a specific scenario. Operational patterns are quantitative values ​​extracted from the behavioral interaction data to characterize the user's habitual level of adjacent operational steps in a specific scenario. Operational efficiency features are quantitative values ​​extracted from the behavioral interaction data to characterize the efficiency of the user's operational path in a specific scenario.

[0022] In addition, to address the issue of missing modalities in data acquisition, a mean vector imputation mechanism is designed. Robust compensation is performed by pre-calculating the statistical mean vector of each modality training set, ensuring that the system can still maintain basic functions when some sensors malfunction.

[0023] This embodiment employs different methods for feature extraction based on the different modal characteristics of voice interaction data, video interaction data, and behavioral interaction data, ensuring the accuracy of voice modal features, video modal features, and behavioral modal features.

[0024] In one embodiment, the behavioral interaction data includes dialogue interaction data, operation jump interaction data, and operation path interaction data; step S103, namely, performing feature analysis processing on the behavioral interaction data to obtain speech professionalism features, operation pattern features, and operation efficiency features, includes: S1031. Perform word frequency analysis on the dialogue interaction data to obtain the professionalism characteristics of the speech; S1032. Perform transition matrix analysis on the operation jump interaction data to obtain operation pattern characteristics; S1033. Perform flowchart analysis on the operation path interaction data to obtain the actual operation path and the theoretical shortest path, and determine the operation efficiency characteristics based on the actual operation path and the theoretical shortest path.

[0025] Understandably, behavioral interaction data includes dialogue interaction data, operation jump interaction data, and operation path interaction data. Dialogue interaction data is a record of the dialogue content between users during the training operation process; operation jump interaction data is a record of the jump between adjacent steps during the training operation process; and operation path interaction data is a record of the complete path of the user during the training operation process.

[0026] In one specific embodiment, the server combines TF-IDF weighted vocabulary enrichment to quantify the professionalism of the dialogue, thereby obtaining the professionalism feature of the dialogue; it models the state transition probability of the operation sequence through Markov chain transition matrix to obtain the operation pattern feature; and it calculates the ratio of the actual operation path to the theoretical shortest path as the operation efficiency feature through flowchart (DAG) path analysis.

[0027] Specifically, the server employs a method similar to TF-IDF (Typical Frequency-Inverse Document Frequency) to statistically analyze the frequency and repetition of professional terms used by training users, quantifying professionalism. For example, the frequent appearance of "car insurance claims" without repetition of other terms indicates more professional terminology. TF stands for Term Frequency, measuring the frequency of a word's occurrence in the current text; IDF stands for Inverse Document Frequency, measuring the rarity of a word in the entire corpus. A higher TF-IDF value for a word indicates a higher probability of its professionalism. The server models the state transition probabilities of operation sequences using Markov chain transition matrices. By statistically analyzing the transition probabilities between two different operation steps (e.g., the probability of transitioning from "viewing information" to "submitting a report"), the relationships between 12 predefined operation types are organized into a 12x12 transition probability table for easy lookup during subsequent analysis of operational patterns. The server also visualizes the actual operation paths of training users as flowcharts, automatically calculating the theoretically optimal path length and operational efficiency characteristics. For example, if a user actually performs 15 steps during training, but the system shows that the optimal steps are only 9, then the efficiency index η = 15 / 9 ≈ 1.67. The closer the value is to 1, the more efficient it is.

[0028] This embodiment obtains the professionalism characteristics of the dialogue interaction data through word frequency analysis, which can accurately grasp the user's level of professional terminology use during training. This embodiment obtains operational pattern characteristics through transition matrix analysis of operation jump interaction data, which helps to discover potential patterns and habits of user operation behavior. This embodiment determines operational efficiency characteristics by analyzing the flowchart of operation path interaction data and based on actual operation paths and theoretical shortest paths, which can intuitively quantify the user's operational efficiency during training. The professionalism characteristics of the dialogue, operational pattern characteristics, and operational efficiency characteristics provide specific data support for discovering potential training bottlenecks, which helps to optimize and improve training scenarios in a targeted manner.

[0029] S20. Perform temporal window alignment processing on the speech modal features, video modal features and behavioral modal features to obtain speech alignment features, video alignment features and behavioral alignment features.

[0030] Understandably, the voice interaction data, video interaction data, and behavioral interaction data collected by the client contain timestamp information. Therefore, voice modal features, video modal features, and behavioral modal features also contain time-series parameters. In multimodal feature engineering, the time-series window is the configuration information for processing time-series parameters, used to extract time-dependent features from the original sequence. Voice-aligned features, video-aligned features, and behavioral-aligned features refer to the features of voice modal features, video modal features, and behavioral modal features after time synchronization processing.

[0031] In one embodiment, step S20, namely performing temporal window alignment processing on the speech modal features, video modal features, and behavioral modal features to obtain speech alignment features, video alignment features, and behavioral alignment features, includes: S201. The speech modal features, video modal features and behavioral modal features are subjected to temporal window alignment processing using a dynamic time warping algorithm to obtain initial speech alignment features, initial video alignment features and initial behavioral alignment features; S202. Perform similarity analysis on the initial speech alignment features, initial video alignment features and initial behavior alignment features to obtain numerical similarity and trend similarity, and determine the temporal window deviation degree based on the numerical similarity and trend similarity. S203. When the time window deviation is less than a preset deviation threshold, the initial speech alignment feature, initial video alignment feature, and initial behavior alignment feature are determined as speech alignment feature, video alignment feature, and behavior alignment feature, respectively.

[0032] Understandably, initial speech alignment features, initial video alignment features, and initial behavior alignment features refer to the features of speech modality features, video modality features, and behavior modality features after time synchronization processing and before deviation verification. Temporal window deviation is a value used to characterize the initial alignment features of multimodal systems within a specific time window, quantifying the degree of deviation between different time series. The preset deviation threshold is a pre-set critical value for determining whether the degree of deviation between different time series can be ignored; it can be set to a default value or adjusted as needed.

[0033] In one specific embodiment, firstly, the server uses a Dynamic Time Warping (DTW) algorithm to automatically "stretch" or "compress" data streams of different modalities to obtain initially aligned multimodal features. For example, when the video action is delayed by half a second compared to the audio, the DTW algorithm intelligently identifies the timing of the "raising hand" action corresponding to the "saying hello" action. Then, the server evaluates sequence similarity by calculating both numerical and trend similarity, obtaining numerical similarity and trend similarity, and weights these two similarities to determine the temporal window deviation. Numerical similarity is measured by calculating the Euclidean distance between the values ​​at two time points, accounting for 70% of the weight. Trend similarity is measured by calculating cosine similarity to determine whether the direction of change is consistent, accounting for 30% of the weight. Next, when the temporal window deviation is less than a preset deviation threshold, the server determines the initial alignment features of the multimodal features as the alignment features. For example, a "time filter" is set as the preset deviation threshold, allowing only a maximum of 15% temporal window deviation (e.g., a 1.5-second deviation is allowed for a 10-second video). For example, a "tolerance red line" can be set as a preset deviation threshold. Deviations exceeding 3 frames (0.1 seconds) are discarded to avoid amplifying errors.

[0034] This embodiment employs a dynamic time warping algorithm to align different modal features along the time axis through flexible matching, achieving a more precise correspondence between modal features in the time dimension. The temporal window deviation, determined based on numerical similarity and trend similarity, more accurately reflects the degree of deviation in temporal alignment between different modal features. In business training scenarios, efficient alignment of multimodal features is achieved. High-quality voice alignment features, video alignment features, and behavior alignment features can provide more accurate and effective structured data support for subsequent multimodal feature fusion analysis.

[0035] S30. The speech alignment features, video alignment features, and behavior alignment features are fused to obtain training fusion features.

[0036] Understandably, data from different modalities possess different characteristics and advantages. Speech data can provide rich semantic and emotional information, video data can provide intuitive visual information, and behavioral data can reflect actual actions and interactions. The server uses a pre-set multimodal fusion model to fuse speech alignment features, video alignment features, and behavioral alignment features to obtain training fusion features. The pre-set multimodal fusion model is a pre-trained deep learning model used to integrate multi-source heterogeneous data features from speech, video, and behavior to construct a unified feature representation. The training fusion feature refers to the comprehensive representation vector output by the multimodal fusion model, used to form a global characterization of the training user's state.

[0037] In one embodiment, step S30, namely, fusing the speech alignment features, video alignment features, and behavior alignment features to obtain training fusion features, includes: S301. The speech alignment features, video alignment features, and behavior alignment features are dimensionality reduced by the kernel function layer in the preset multimodal fusion model to obtain speech dimensionality reduction features, video dimensionality reduction features, and behavior dimensionality reduction features; S302. The speech dimensionality reduction features, video dimensionality reduction features and behavior dimensionality reduction features are processed by the bidirectional long short-term memory network in the preset multimodal fusion model to obtain speech context features, video context features and behavior context features; S303. The speech context features, video context features, and behavioral context features are spliced ​​together by the multimodal fusion layer in the preset multimodal fusion model to obtain the training fusion features.

[0038] Understandably, the pre-defined multimodal fusion model includes a kernel layer, a bidirectional long short-term memory network (LSTM), and a multimodal fusion layer. First, the server uses an RBF kernel (γ=0.1) to perform nonlinear dimensionality reduction on the high-dimensional heterogeneous features, obtaining speech, video, and behavior dimensionality-reduced features. This dimensionality reduction maps the speech, video, and behavior features to a unified latent space, retaining 128 principal components, effectively eliminating redundant information while preserving key features. Next, the server uses a bidirectional long short-term memory network (LSTM) to perform contextual modeling on the dimensionality-reduced temporal features, adding "time tags" to the processed data. Its 256-unit hidden layer structure captures cross-modal temporal dependencies, yielding speech context features, video context features, and behavior context features. Finally, in the feature fusion stage, the server uses the multimodal fusion layer to dynamically allocate modal weights based on the importance of XGBoost features (e.g., speech 0.4, vision 0.3, behavior 0.3), concatenating the speech context features, video context features, and behavior context features to obtain the trained fused features.

[0039] This embodiment achieves efficient fusion of multimodal features in business training scenarios through dimensionality reduction, capturing sequence context dependencies, and splicing. It integrates information from different modalities to form a more comprehensive and richer training fusion feature, giving full play to the complementary advantages of different modal data and improving the expressive power of features.

[0040] In one embodiment, the multimodal fusion layer includes a spatiotemporal branch network and a temporal branch network; in step S303, the process of concatenating the speech context features, video context features, and behavioral context features through the multimodal fusion layer in the preset multimodal fusion model to obtain training fusion features includes: S3031. The video context features and behavioral context features are correlated and analyzed by the spatiotemporal branch network in the preset multimodal fusion model to obtain the spatiotemporal attention weights; S3032. The speech context features and behavioral context features are correlated and analyzed by the temporal branch network in the preset multimodal fusion model to obtain the temporal attention weight; S3033. Based on the spatiotemporal attention weight and the temporal attention weight, the speech context features, video context features and behavioral context features are weighted and concatenated to obtain training fusion features.

[0041] Understandably, the multimodal fusion layer in the pre-defined multimodal fusion model includes a spatiotemporal branch network and a temporal branch network. The spatiotemporal branch network refers to a pre-trained neural network used to analyze the correlation between spatial and temporal dimensions of information, while the temporal branch network refers to a pre-trained neural network used to analyze the characteristics and changing patterns of data over time. Specifically, the spatiotemporal branch network is a 3D-CNN network with a kernel size of 3×3×3 and the number of channels [32, 64, 128]. The temporal branch network is a Transformer network, including 8-head attention and a 512-dimensional FFN. The spatiotemporal attention weights are weight values ​​used to measure the degree of correlation between the video modality and the behavioral modality, while the temporal attention weights are weight values ​​used to measure the degree of correlation between the speech modality and the behavioral modality.

[0042] The server uses a spatiotemporal branching network to perform correlation analysis on video context features and behavioral context features to obtain spatiotemporal attention weights. This allows for precise identification of which video features are more relevant to behavioral features at different spatiotemporal locations, capturing the temporal evolution and spatial correlation of actions. For example, the sequence of actions from "viewing the policy" to "clicking the calculator" can reflect temporal continuity, while the correspondence between hand movements at the same time point and the screen gaze area can reflect spatial correlation.

[0043] The server analyzes speech and behavioral context features through a temporal branching network to obtain temporal attention weights, which can better capture the synchronicity and causal relationships between speech and behavior in the temporal dimension. Based on the Transformer architecture, global dependency modeling is performed on speech and behavioral modalities. An interaction matrix between visual queries and speech / behavioral key-value pairs is constructed through a cross-modal attention mechanism. Attention weights are calculated using Softmax (e.g., Softmax = (QK^T / √d)⊗Value), achieving fine-grained feature alignment between modalities. Specifically, key information (such as technical terms like "deductible" and "no deductible") is extracted from speech, and important nodes (such as repeatedly verifying policy terms) are identified from the operational path.

[0044] The server-side introduces an attention-weighted concatenation strategy during the feature fusion stage and employs L2 regularization to constrain model complexity. The L2 regularization formula is (λ=0.01) + Dropout (0.5). The server obtains initial modality weights (e.g., speech 0.4, vision 0.3, behavior 0.3) based on the importance of XGBoost features. These initial modality weights are then adjusted according to spatiotemporal and temporal attention weights to enhance the contribution of key modalities to the final representation. After weighted concatenation, the training fusion features are obtained. In insurance scenarios, the correlation between "facial visual actions - speech expression - decision path" can be dynamically captured. For example, when a training user views an insurance policy (visual query), if the speech mentions "deductible" (speech key) and the operation path shows repeated verification (behavioral value), the attention weights will enhance the correlation strength of these three modalities, thereby forming a cross-modal fusion feature representation in the latent space.

[0045] This embodiment can dynamically allocate the importance of different modal features through spatiotemporal attention weights and temporal attention weights. During the weighted splicing process, the fused features can be more focused on key information, reducing the impact of redundancy and noise, thereby improving the representational ability and discriminative power of the fused features.

[0046] S40. The training integration features are scored and predicted using a preset evaluation model to obtain stress index score, mastery index score and standardization index score.

[0047] Understandably, the pre-trained evaluation model is a neural network model used to convert training fusion features into quantitative evaluation scores, achieving a comprehensive quantitative analysis of the training user's stress, mastery, and standardization abilities. The stress score is a quantitative value representing the level of stress experienced by the user during training; the mastery score is a quantitative value representing the degree of mastery achieved by the user during training; and the standardization score is a quantitative value representing the user's performance in adhering to standardized procedures during training. The network architecture of the pre-trained evaluation model consists of a shared layer and three task heads, with a weighted combination strategy for the loss function. The shared layer is a multilayer perceptron (MLP) with dimensions (256→128→64), used to extract cross-modal training fusion features from a unified latent space. The three task heads include a stress assessment head, a mastery classification head, and a standardization scoring head. The total loss function is expressed as S = 0.4×L_regression + 0.5×L_classification + 0.1×L_regularization. The stress assessment head uses the ReLU activation function to construct a continuous regression output of 0-100 points, simulating the psychological load of trainees in high-pressure scenarios such as explaining insurance terms. The mastery classification head converts scores into probability scores using Softmax and implements a three-class classification (good mastery / basic mastery / no mastery, based on decision path matching degree defined by a knowledge graph). The prescriptive scoring head uses the Sigmoid function combined with linear transformation to generate a score in the [0,1] interval, used to assess the compliance of operational procedures, such as the compliance of the claims investigation steps. In the insurance training scenario, the pre-set evaluation model can simultaneously identify the user's stress peaks during simulated auto insurance claims training (such as increased heart rate variability when encountering complex damage assessment scenarios), quantify their mastery of deductible calculation rules, and assess the prescriptiveness of the damage assessment process (such as whether the photo shooting order meets standards). In this embodiment, the evaluation model adopts a shared-dedicated hierarchical architecture, constructing a gradient-coordinated backpropagation mechanism through joint optimization of stress assessment, mastery classification, and operational prescriptiveness scoring.

[0048] S50. When at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, the scenario parameters of the current training scenario are updated according to the stress index score, mastery index score, and standardization index score to obtain an updated training scenario, so that the target training user can complete the next training in the updated training scenario.

[0049] Understandably, the server determines whether the stress indicator score, mastery indicator score, and standardization indicator score meet the scenario adjustment conditions. If at least one of the conditions is met, the scenario parameters of the current training scenario need to be updated to obtain an updated training scenario, so that the target training users can complete the next training session in the updated scenario. The scenario adjustment conditions are pre-set rules used to determine whether to adjust the scenario parameters. Updating the training scenario refers to the result of adjusting the scenario parameters of the current training scenario.

[0050] This embodiment acquires and extracts features from the voice, video, and behavioral interaction data of the target training user in the current training scenario, obtaining voice modal features, video modal features, and behavioral modal features. It then performs temporal window alignment processing on these features to obtain voice-aligned features, video-aligned features, and behavioral-aligned features. Finally, it fuses these features to obtain training fusion features. A preset evaluation model is used to predict the scores of these training fusion features, resulting in stress index scores, mastery index scores, and standardization index scores. When at least one of these scores meets the scenario adjustment conditions, the scenario parameters of the current training scenario are updated based on these scores, resulting in an updated training scenario. This allows the target training user to complete the next training session in the updated scenario. This embodiment deeply integrates and analyzes multimodal data of the user's voice, video, and operational behavior in the current training scenario to obtain quantitative evaluation scores for the stress, mastery, and standardization dimensions, enabling accurate evaluation of the training results for each business personnel. Meanwhile, this embodiment can generate dynamic adjustment strategies based on the evaluation results, realize the linkage adjustment between training evaluation and training scenario, meet the training needs of complex training scenarios, and help improve the effectiveness of business training.

[0051] In one embodiment, step S50, i.e., when at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, updates the scenario parameters of the current training scenario based on the stress index score, mastery index score, and standardization index score to obtain an updated training scenario, includes: S501. When the pressure index score reaches the preset pressure score threshold, it is determined that the scenario adjustment condition is met, and the pressure adjustment parameter corresponding to the preset pressure score threshold is obtained. S502. When the mastery index score reaches the preset mastery score threshold, determine that the scenario adjustment condition is met, and obtain the mastery adjustment parameter corresponding to the preset mastery score threshold. S503. When the normative index score reaches the preset normative score threshold, determine that the scenario adjustment conditions are met, and obtain the normative adjustment parameters corresponding to the preset normative score threshold. S504. Perform a reward function calculation on the stress index score, mastery index score and standardization index score to obtain a reward score value, and determine that the scenario adjustment condition is met when the reward score value reaches a preset reward score threshold, and obtain the reward adjustment parameter corresponding to the preset reward score threshold. S505. Based on the pressure adjustment parameters, mastery adjustment parameters, standardization adjustment parameters, and reward adjustment parameters, update the scenario parameters of the current training scenario to obtain an updated training scenario.

[0052] Understandably, the preset stress score threshold is a pre-set critical value used to measure whether to adjust scene parameters from a stress perspective. The stress adjustment parameter refers to the parameter change value that triggers scene parameter adjustment from a stress perspective. The preset mastery score threshold is a pre-set critical value used to measure whether to adjust scene parameters from a mastery perspective. The mastery adjustment parameter refers to the parameter change value that triggers scene parameter adjustment from a mastery perspective. The preset normativity score threshold is a pre-set critical value used to measure whether to adjust scene parameters from a normativity perspective. The normativity adjustment parameter refers to the parameter change value that triggers scene parameter adjustment from a normativity perspective. The reward score is a quantitative value that measures the comprehensive balance of the three dimensions: stress, mastery, and normativity. The pre-set reward function is a weighted sum of the mastery index score, normativity index score, and stress index score, specifically expressed as R = 0.5 × mastery index score + 0.3 × normativity index score - 0.2 × stress index score. This can integrate three optimization objectives to guide scene adjustment strategies to achieve a balance between improving skills and maintaining psychological comfort. The preset reward scoring threshold is a pre-set critical value used to measure whether to adjust scene parameters from a reward perspective. The reward adjustment parameter refers to the parameter change value that triggers scene parameter adjustment from a reward perspective. The preset stress scoring threshold, preset mastery scoring threshold, preset standardization scoring threshold, and preset reward scoring threshold can be set to default values ​​or adjusted as needed.

[0053] In one specific embodiment, a dynamic adjustment strategy is generated based on the scores in the evaluation results. The server employs a Proximal Policy Optimization (PPO) algorithm, and the policy network MLP is based on a 256→128→action dimension. The policy network maps the evaluation results to specific scenario adjustment actions. The scenario adjustment actions include two dimensions of control: a difficulty adjustment layer and a script recommendation layer. The difficulty adjustment layer supports three levels of elastic adjustment [-0.2, 0, +0.2] (corresponding to reducing, maintaining, and increasing task complexity, such as adjusting the damage identification difficulty in car insurance claims). The script recommendation layer generates Top 3 interaction strategies through knowledge graph retrieval (such as recommending stress-reducing scripts like "handle vulnerable parts first" when the stress index score is >75). In the insurance claims adjuster training system, it can respond to changes in the status of training users in real time. For example, if the preset standardization score threshold is 0.6, when it detects that the training user's operation standardization index score is <0.6 twice consecutively, the server obtains the mastery adjustment parameter corresponding to the preset standardization score threshold, automatically reduces the task difficulty by 0.2, and recommends the operation guidance prompt "shoot a panoramic view first, then a close-up." For example, if the mastery index score stagnates for more than 3 rounds of training, the knowledge graph-driven error tracing mechanism is triggered, and related cases are pushed to strengthen weak links.

[0054] In one embodiment, a closed-loop control system based on knowledge graph-driven scene retrieval and real-time loading enables the adaptive evolution of training scenarios. Specifically, knowledge graph-driven intelligent generation technology constructs personalized training environments, dynamically generating training scenarios tailored to trainees' abilities. The knowledge graph employs a three-level ontology architecture, including accident types (e.g., rear-end collisions, rollovers, water wading), liability determination rules (primary, secondary, and full liability), and script templates (e.g., a damage assessment communication script library) as core nodes. A five-level difficulty mapping (L1-L5, with quantitative dimensions including the number of liability determination elements, the density of accident scene interference items, and the level of script compliance requirements) is established through the HAS_COMPLEXITY relationship. The scene generation engine uses a Cypher query optimization strategy, and the generated scene templates are rendered in real-time using Unity3D hot-loading technology (employing AssetBundle block loading and asynchronous decompression technology, with an average loading time of 427ms±63ms). It supports dynamically replacing accident environment parameters (e.g., visibility coefficient in rainy / snowy weather, collision angle error range) and compliance script trigger thresholds. During actual training, the system responds in real-time to changes in training evaluation status, with a preset stress score threshold of 70. When a stress index > 70 is detected, the corresponding stress adjustment parameter automatically reduces the scenario complexity and activates the script prompt pop-up window. The preset mastery score threshold is when the difference between two consecutive mastery score indicators is greater than 0. When the mastery score indicator continuously improves, the corresponding mastery adjustment parameter dynamically adds multi-factor liability determination scenarios (such as a composite case of "secondary collision + wading + exceeding the insurance limit").

[0055] In another embodiment, a real-time interactive enhancement system is built based on the Transformer architecture to generate personalized dialogue suggestions. The Transformer architecture adopts an encoder-decoder structure, where the encoder uses a bidirectional GRU to encode features of multimodal input sequences (speech-to-text transcription, operation timing features, and physiological indicator change rates), and the decoder integrates an attention-enhanced GRU to focus on key interaction nodes (such as clause explanation bottlenecks and operational hesitation periods) through dynamic attention weight allocation. The optimization strategy includes the following three aspects: beam search retains Top-K candidate sequences during decoding to balance generation quality and computational cost; model pruning retains 80% of key parameters, combined with INT8 quantization technology to achieve the inference latency requirement (latency <200ms); and a dynamic filtering mechanism excludes mastered dialogue templates based on the mastery output of the multi-task evaluation module. For example, the preset mastery score threshold is 0.8. When the mastery index score of the training user is ≥0.8, the corresponding mastery adjustment parameter is to filter basic dialogue such as "explanation of absolute deductible" and prioritize the generation of improvement suggestions (such as recommending advanced dialogue such as "calculation of special circumstances liability superposition"). For example, if the preset standardization score threshold is 0.8, when it is detected that the standardization score of a trained user is <0.7 twice in a water-related insurance claim scenario, the corresponding standardization adjustment parameter is to automatically trigger the decoder to generate guiding words including "first confirm the extent of engine damage" and provide voice prompts through AR glasses.

[0056] In another embodiment, trainees need to wear AR glasses for training interaction. Through multi-channel collaboration of adaptive transparency adjustment of the AR glasses, particle effect pressure visualization, and low-latency voice feedback, real-time response capabilities are achieved on edge computing devices, providing a scalable intelligent adjustment solution for the insurance training field. Dynamic feedback is delivered through multiple channels, including visual feedback, voice feedback, and anomaly handling. Visual feedback uses the Unity3D particle system to achieve pressure visualization feedback, with a preset pressure score threshold of 70. When a pressure index > 70 is detected, the corresponding pressure adjustment parameter triggers a red turbulent particle effect with a speed coefficient of 1.5× accompanied by a frequency-modulated sound effect. The Hololens 2 AR glasses achieve spatial awareness interaction through a floating speech box with adjustable transparency (α = 0.3~0.8). For example, the transparency is dynamically adjusted based on the trainee's current normative indicator score; the lower the score, the higher the α value to enhance visual cues. The voice feedback channel integrates Google CloudTTS service, supporting dynamic matching of multiple voice timbres, with a preset pressure score threshold of 85 and a preset normative score threshold of 85. When the detected stress index > 85, the corresponding stress adjustment parameter switches to a low-frequency, stable tone; when the normative index score < 0.5, a high-frequency prompt tone is enabled. An anomaly handling mechanism employs a three-tiered feedback priority strategy. When computing power is limited or network fluctuations occur, it automatically downgrades to a multimodal fusion mode that retains the core channels (prioritizing AR dialogue boxes + key voice prompts, switching particle effects to low-polygon rendering). Dynamic load balancing of the feedback channels is achieved through the Unity3D Job System. In the training system, based on a multi-channel feedback latency of < 200ms (meeting the critical threshold for VR / AR interaction), the accuracy of stress visualization recognition is significantly improved.

[0057] This embodiment acquires scores for various indicators in real time and compares them with preset thresholds to promptly determine whether scenario adjustment conditions are met and obtains corresponding adjustment parameters. It can dynamically update scenario parameters based on the real-time performance of trainees during training, establishing a real-time feedback mechanism. This mechanism makes the training process more flexible, allowing for timely adjustments to training strategies and scenario settings, improving the timeliness and relevance of training, and enabling trainees to achieve comprehensive improvement and better adapt to the needs of actual business scenarios.

[0058] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0059] In one embodiment, a business training scenario update device is provided, which corresponds one-to-one with the business training scenario update method in the above embodiments. For example... Figure 3As shown, the business training scenario update device includes a feature extraction module 10, a feature alignment module 20, a feature fusion module 30, a scoring prediction module 40, and a scenario update module 50. Detailed descriptions of each functional module are as follows: The feature extraction module 10 is used to acquire voice interaction data, video interaction data and behavioral interaction data of the target training user in the current training scenario, and to extract features from the voice interaction data, video interaction data and behavioral interaction data to obtain voice modal features, video modal features and behavioral modal features; The feature alignment module 20 is used to perform temporal window alignment processing on the speech modal features, video modal features and behavioral modal features to obtain speech aligned features, video aligned features and behavioral aligned features; Feature fusion module 30 is used to fuse the speech alignment features, video alignment features and behavior alignment features to obtain training fusion features; The scoring prediction module 40 is used to perform scoring prediction processing on the training integration features through a preset evaluation model to obtain stress index scores, mastery index scores and standardization index scores. The scenario update module 50 is used to update the scenario parameters of the current training scenario based on the stress index score, mastery index score and standardization index score when at least one of the stress index score, mastery index score and standardization index score meets the scenario adjustment conditions, so as to obtain an updated training scenario, and enable the target training user to complete the next training in the updated training scenario.

[0060] In one embodiment, the feature extraction module 10 includes: The speech analysis unit is used to perform feature analysis processing on the speech interaction data through a preset speech analysis model to obtain audio features and emotion features, and generate speech modal features based on the audio features and emotion features; the audio features are obtained by inputting the speech interaction data into the audio feature extraction tool in the preset speech analysis model, and the emotion features are obtained by inputting the audio features into the sentiment analysis network in the preset speech analysis model. The facial analysis unit is used to perform facial analysis processing on the video interaction data using a facial analysis tool to obtain facial features and determine the facial features as video modal features. The behavior analysis unit is used to perform feature analysis processing on the behavior interaction data to obtain speech professionalism features, operation pattern features and operation efficiency features, and generate voice modal features based on the speech professionalism features, operation pattern features and operation efficiency features.

[0061] In one embodiment, the feature extraction module 10 further includes: The word frequency analysis unit is used to perform word frequency analysis on the dialogue interaction data to obtain the professionalism characteristics of the speech. The transition matrix analysis unit is used to perform transition matrix analysis on the operation jump interaction data to obtain operation pattern characteristics. The flowchart analysis unit is used to perform flowchart analysis on the operation path interaction data to obtain the actual operation path and the theoretical shortest path, and to determine the operation efficiency characteristics based on the actual operation path and the theoretical shortest path.

[0062] In one embodiment, the feature alignment module 20 includes: The temporal window alignment unit is used to perform temporal window alignment processing on the speech modal features, video modal features and behavioral modal features through a dynamic time warping algorithm to obtain initial speech alignment features, initial video alignment features and initial behavioral alignment features; The deviation determination unit is used to perform similarity analysis on the initial speech alignment features, initial video alignment features and initial behavior alignment features to obtain numerical similarity and trend similarity, and to determine the temporal window deviation based on the numerical similarity and trend similarity. The deviation comparison unit is used to determine the initial speech alignment feature, initial video alignment feature and initial behavior alignment feature as speech alignment feature, video alignment feature and behavior alignment feature when the deviation of the time window is less than a preset deviation threshold.

[0063] In one embodiment, the feature fusion module 30 includes: The dimensionality reduction processing unit is used to perform dimensionality reduction processing on the speech alignment features, video alignment features and behavior alignment features through the kernel function layer in the preset multimodal fusion model to obtain speech dimensionality reduction features, video dimensionality reduction features and behavior dimensionality reduction features; The context processing unit is used to perform context processing on the speech dimensionality reduction features, video dimensionality reduction features and behavior dimensionality reduction features through the bidirectional long short-term memory network in the preset multimodal fusion model to obtain speech context features, video context features and behavior context features; The splicing processing unit is used to splice the speech context features, video context features and behavioral context features through the multimodal fusion layer in the preset multimodal fusion model to obtain training fusion features.

[0064] In one embodiment, the feature fusion module 30 further includes: The spatiotemporal correlation analysis unit is used to perform correlation analysis on the video context features and behavioral context features through the spatiotemporal branch network in the preset multimodal fusion model to obtain spatiotemporal attention weights; The temporal correlation analysis unit is used to perform correlation analysis on the speech context features and behavioral context features through the temporal branch network in the preset multimodal fusion model to obtain temporal attention weights; The weighted splicing processing unit is used to perform weighted splicing processing on the speech context features, video context features and behavioral context features according to the spatiotemporal attention weight and the temporal attention weight to obtain training fusion features.

[0065] In one embodiment, the scene update module 50 includes: The pressure adjustment parameter determination unit is used to determine that the scenario adjustment conditions are met when the pressure index score reaches a preset pressure score threshold, and to obtain the pressure adjustment parameter corresponding to the preset pressure score threshold. The mastery adjustment parameter determination unit is used to determine that the scenario adjustment conditions are met when the mastery index score reaches the preset mastery score threshold, and to obtain the mastery adjustment parameter corresponding to the preset mastery score threshold. The normative adjustment parameter determination unit is used to determine whether the scenario adjustment conditions are met when the normative index score reaches a preset normative score threshold, and to obtain the normative adjustment parameters corresponding to the preset normative score threshold. The reward adjustment parameter determination unit is used to perform reward function calculation on the stress index score, mastery index score and standardization index score to obtain the reward score value, and determine that the scenario adjustment condition is met when the reward score value reaches the preset reward score threshold, and obtain the reward adjustment parameter corresponding to the preset reward score threshold; The scenario parameter update unit is used to update the scenario parameters of the current training scenario based on the pressure adjustment parameters, mastery adjustment parameters, standardization adjustment parameters, and reward adjustment parameters, so as to obtain an updated training scenario.

[0066] Specific limitations regarding the business training scenario update device can be found in the limitations of the business training scenario update method described above, and will not be repeated here. Each module in the aforementioned business training scenario update device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0067] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a readable storage medium and internal memory. The readable storage medium stores an operating system, computer-readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database stores data related to the business training scenario update method. The network interface communicates with external terminals via a network connection. When the computer-readable instructions are executed by the processor, a business training scenario update method is implemented. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0068] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor performs the following steps when executing the computer-readable instructions: Acquire voice interaction data, video interaction data, and behavioral interaction data of the target training users in the current training scenario, and extract features from the voice interaction data, video interaction data, and behavioral interaction data to obtain voice modal features, video modal features, and behavioral modal features; Temporal window alignment processing is performed on the speech modal features, video modal features, and behavioral modal features to obtain speech aligned features, video aligned features, and behavioral aligned features; The speech alignment features, video alignment features, and behavior alignment features are fused together to obtain the training fusion features; The training integration features are scored and predicted using a preset evaluation model to obtain stress index scores, mastery index scores, and standardization index scores. When at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, the scenario parameters of the current training scenario are updated based on the stress index score, mastery index score, and standardization index score to obtain an updated training scenario, so that the target training user can complete the next training in the updated training scenario.

[0069] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media stores computer-readable instructions, which, when executed by one or more processors, perform the following steps: Acquire voice interaction data, video interaction data, and behavioral interaction data of the target training users in the current training scenario, and extract features from the voice interaction data, video interaction data, and behavioral interaction data to obtain voice modal features, video modal features, and behavioral modal features; Temporal window alignment processing is performed on the speech modal features, video modal features, and behavioral modal features to obtain speech aligned features, video aligned features, and behavioral aligned features; The speech alignment features, video alignment features, and behavior alignment features are fused together to obtain the training fusion features; The training integration features are scored and predicted using a preset evaluation model to obtain stress index scores, mastery index scores, and standardization index scores. When at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, the scenario parameters of the current training scenario are updated based on the stress index score, mastery index score, and standardization index score to obtain an updated training scenario, so that the target training user can complete the next training in the updated training scenario.

[0070] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0071] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0072] The software tools or components not belonging to this company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for updating business training scenarios, characterized in that, include: Acquire voice interaction data, video interaction data, and behavioral interaction data of the target training users in the current training scenario, and extract features from the voice interaction data, video interaction data, and behavioral interaction data to obtain voice modal features, video modal features, and behavioral modal features; Temporal window alignment processing is performed on the speech modal features, video modal features, and behavioral modal features to obtain speech aligned features, video aligned features, and behavioral aligned features; The speech alignment features, video alignment features, and behavior alignment features are fused together to obtain the training fusion features; The training integration features are scored and predicted using a preset evaluation model to obtain stress index scores, mastery index scores, and standardization index scores. When at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, the scenario parameters of the current training scenario are updated based on the stress index score, mastery index score, and standardization index score to obtain an updated training scenario, so that the target training user can complete the next training in the updated training scenario.

2. The business training scenario update method as described in claim 1, characterized in that, The step of extracting features from the voice interaction data, video interaction data, and behavioral interaction data to obtain voice modal features, video modal features, and behavioral modal features includes: The voice interaction data is processed by a preset voice analysis model to obtain audio features and emotion features, and voice modal features are generated based on the audio features and emotion features. The audio features are obtained by inputting the voice interaction data into the audio feature extraction tool in the preset voice analysis model, and the emotion features are obtained by inputting the audio features into the sentiment analysis network in the preset voice analysis model. The video interaction data is processed by facial analysis tools to obtain facial features, and the facial features are identified as video modal features. The behavioral interaction data is subjected to feature analysis to obtain speech professionalism features, operational pattern features, and operational efficiency features, and speech modal features are generated based on the speech professionalism features, operational pattern features, and operational efficiency features.

3. The business training scenario update method as described in claim 2, characterized in that, The behavioral interaction data includes dialogue interaction data, operation jump interaction data, and operation path interaction data; The process of performing feature analysis on the behavioral interaction data yields features related to the professionalism of the speech, operational patterns, and operational efficiency, including: Word frequency analysis was performed on the dialogue interaction data to obtain the professionalism characteristics of the speech. Transition matrix analysis is performed on the operation jump interaction data to obtain operation pattern characteristics; The operation path interaction data is analyzed using a flowchart to obtain the actual operation path and the theoretical shortest path, and the operation efficiency characteristics are determined based on the actual operation path and the theoretical shortest path.

4. The business training scenario update method as described in claim 1, characterized in that, The step of performing temporal window alignment processing on the speech modal features, video modal features, and behavioral modal features to obtain speech-aligned features, video-aligned features, and behavioral-aligned features includes: The speech modal features, video modal features, and behavioral modal features are subjected to temporal window alignment processing using a dynamic time warping algorithm to obtain initial speech alignment features, initial video alignment features, and initial behavioral alignment features. Similarity analysis is performed on the initial speech alignment features, initial video alignment features, and initial behavior alignment features to obtain numerical similarity and trend similarity, and the temporal window deviation is determined based on the numerical similarity and trend similarity. When the time window deviation is less than a preset deviation threshold, the initial speech alignment feature, initial video alignment feature, and initial behavior alignment feature are determined as speech alignment feature, video alignment feature, and behavior alignment feature, respectively.

5. The business training scenario update method as described in claim 1, characterized in that, The process of fusing the speech alignment features, video alignment features, and behavior alignment features to obtain training fusion features includes: The speech alignment features, video alignment features, and behavior alignment features are dimensionality reduced by using the kernel function layer in the preset multimodal fusion model to obtain speech dimensionality reduction features, video dimensionality reduction features, and behavior dimensionality reduction features. The speech dimensionality reduction features, video dimensionality reduction features, and behavior dimensionality reduction features are subjected to contextual processing through a bidirectional long short-term memory network in a preset multimodal fusion model to obtain speech contextual features, video contextual features, and behavior contextual features. The speech context features, video context features, and behavioral context features are concatenated by the multimodal fusion layer in the preset multimodal fusion model to obtain the training fusion features.

6. The business training scenario update method as described in claim 5, characterized in that, The multimodal fusion layer includes a spatiotemporal branch network and a temporal branch network; The training fusion features are obtained by concatenating the speech context features, video context features, and behavioral context features through a multimodal fusion layer in a preset multimodal fusion model, including: The video context features and behavioral context features are correlated and analyzed by a spatiotemporal branch network in a pre-defined multimodal fusion model to obtain spatiotemporal attention weights; The speech context features and behavioral context features are correlated and analyzed by a temporal branch network in a pre-defined multimodal fusion model to obtain temporal attention weights; Based on the spatiotemporal attention weights and the temporal attention weights, the speech context features, video context features, and behavioral context features are weighted and concatenated to obtain the training fusion features.

7. The business training scenario update method as described in claim 1, characterized in that, When at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, the scenario parameters of the current training scenario are updated based on the stress index score, mastery index score, and standardization index score to obtain an updated training scenario, including: When the pressure index score reaches a preset pressure score threshold, it is determined that the scenario adjustment conditions are met, and the pressure adjustment parameters corresponding to the preset pressure score threshold are obtained. When the mastery score reaches the preset mastery score threshold, it is determined that the scenario adjustment condition is met, and the mastery adjustment parameter corresponding to the preset mastery score threshold is obtained. When the normative index score reaches the preset normative score threshold, it is determined that the scenario adjustment condition is met, and the normative adjustment parameter corresponding to the preset normative score threshold is obtained; A reward function is performed on the stress index score, mastery index score, and standardization index score to obtain a reward score value. When the reward score value reaches a preset reward score threshold, it is determined that the scenario adjustment condition is met, and the reward adjustment parameter corresponding to the preset reward score threshold is obtained. Based on the pressure adjustment parameters, mastery adjustment parameters, standardization adjustment parameters, and reward adjustment parameters, the scenario parameters of the current training scenario are updated to obtain the updated training scenario.

8. A business training scenario updating device, characterized in that, include: The feature extraction module is used to acquire voice interaction data, video interaction data and behavioral interaction data of the target training user in the current training scenario, and to extract features from the voice interaction data, video interaction data and behavioral interaction data to obtain voice modal features, video modal features and behavioral modal features; The feature alignment module is used to perform temporal window alignment processing on the speech modal features, video modal features and behavioral modal features to obtain speech aligned features, video aligned features and behavioral aligned features; The feature fusion module is used to fuse the speech alignment features, video alignment features, and behavior alignment features to obtain training fusion features; The scoring prediction module is used to perform scoring prediction processing on the training integration features through a preset evaluation model to obtain stress index scores, mastery index scores, and standardization index scores. The scenario update module is used to update the scenario parameters of the current training scenario based on the stress index score, mastery index score, and standardization index score when at least one of the stress index score, mastery index score, and standardization index score meets the scenario adjustment conditions, thereby obtaining an updated training scenario, so that the target training user can complete the next training in the updated training scenario.

9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the business training scenario update method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors perform the business training scenario update method as described in any one of claims 1 to 7.

Images