Multi-dimensional assessment method and system for virtual workplace based on deep reinforcement learning

Abstract

This invention provides a multi-dimensional virtual workplace assessment method and system based on deep reinforcement learning, which relates to the field of human resources assessment technology. The method involves collecting behavioral data from users in a virtual workplace environment, constructing a state transition probability matrix, generating interactive decision sequences based on deep reinforcement learning, and then constructing a task execution plan. Finally, based on checkpoint data from the user's actual execution process, feature fusion is used to calculate a professional competency score. This invention achieves an objective and quantitative assessment of workplace competence, improving the accuracy and fairness of assessments.

Description

Technical Field

[0001] The present invention relates to human resource assessment technology, and in particular to a virtual workplace multi-dimensional assessment method and system based on deep reinforcement learning. Background Art

[0002] As enterprises deepen their digital transformation, virtual workplace environments are becoming increasingly important platforms for employee training and assessment. Traditional professional competency assessment methods rely primarily on manual observation and empirical judgment, making it difficult to accurately quantify employee performance in complex task scenarios. Furthermore, assessment results are often subjective and biased.

[0003] Existing virtual workplace assessment systems typically use fixed metrics, making it difficult to dynamically adjust assessment strategies based on employee behavioral characteristics and competency development. Furthermore, due to a lack of in-depth analysis and modeling of employee decision-making behavior, it is difficult to accurately assess an employee's adaptability and potential in different task scenarios.

[0004] Therefore, there is an urgent need for a multi-dimensional assessment method for the virtual workplace based on deep reinforcement learning. By collecting and analyzing user behavior data to build a state transition model, and combining task scenarios to dynamically generate evaluation strategies, accurate assessment and development guidance of employees' professional capabilities can be achieved. Summary of the Invention

[0005] The embodiments of the present invention provide a virtual workplace multi-dimensional assessment method and system based on deep reinforcement learning, which can solve the problems in the existing technology.

[0006] A first aspect of an embodiment of the present invention provides a multi-dimensional assessment method for a virtual workplace based on deep reinforcement learning, comprising:

[0007] Collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, construct a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user's behavior pattern based on the state transition probability matrix, and generate a user behavior feature vector;

[0008] A state space representation is constructed based on the user behavior feature vector. The state transition function is calculated based on the task scenario state information. The reward function is set according to the task completion and execution efficiency. The policy iteration method of deep reinforcement learning is used to generate the interactive decision sequence.

[0009] Build a task execution plan graph based on the interactive decision sequence, and generate a task execution plan with checkpoints based on the task execution plan graph;

[0010] When the user is executing the task execution plan, the execution status data of each checkpoint is collected, and the task completion trajectory curve is calculated based on the execution status data. The user's decision choices at each checkpoint are extracted in combination with the task execution planning diagram, and the decision choices are fused with the user behavior feature vector to generate an ability dimension vector. The professional competency score is calculated based on the ability dimension vector, and a professional competency assessment report is generated.

[0011] In an optional embodiment,

[0012] Collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, build a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user behavior pattern based on the state transition probability matrix, and generate the user behavior feature vector including:

[0013] Collecting user behavior data in a virtual workplace environment, the behavior data including operation sequence data, task switching data, problem solving data, and collaboration pattern data;

[0014] Calculating a behavior complexity index for the behavior data, dynamically determining a time window size based on the behavior complexity index, extracting statistical features and trend features of the behavior data within the time window, and combining them to generate a behavior state vector;

[0015] Calculating a time series correlation for the behavior state vector, determining a behavior transition threshold according to the time series correlation, and performing density clustering on the behavior state vector based on the behavior transition threshold to generate a behavior state sequence;

[0016] The frequency of transitions between states is calculated according to the behavioral state sequence, a state transition probability matrix is ​​constructed, and the state transition probability matrix is ​​subjected to matrix decomposition to obtain a first transition probability submatrix and a second transfer probability submatrix; the first transition probability submatrix and the second transition probability submatrix are subjected to eigenvalue decomposition respectively to obtain an eigenvalue set, the behavioral entropy value is calculated according to the maximum eigenvalue in the eigenvalue set, and the behavioral stability feature is calculated according to the remaining eigenvalues, and the behavioral entropy value and the behavioral stability feature are feature fused to generate a user behavior feature vector.

[0017] In an optional embodiment,

[0018] Calculating a time series correlation for the behavior state vector, determining a behavior transition threshold according to the time series correlation, and performing density clustering on the behavior state vector based on the behavior transition threshold to generate a behavior state sequence includes:

[0019] Obtaining a behavior state vector, calculating a vector space distance and a time interval distance between adjacent behavior state vectors, combining the vector space distance and the time interval distance to construct a time series correlation function, and calculating a time series correlation degree of the behavior state vector using the time series correlation function;

[0020] Perform statistical analysis on the time series correlation to obtain the distribution characteristics of the time series correlation, generate a behavior jump threshold based on the distribution characteristics, determine the behavior jump point based on the time series correlation and the behavior jump threshold, and calculate the local density of the behavior state vector at the behavior jump point;

[0021] Density clustering is performed on the behavior state vector based on the local density to obtain a behavior state clustering result, and the behavior state clustering result is arranged and combined in time sequence to generate a behavior state sequence.

[0022] In an optional embodiment,

[0023] The state space representation is constructed based on the user behavior feature vector, the state transition function is calculated based on the task scenario state information, the reward function is set according to the task completion and execution efficiency, and the interactive decision sequence is generated using the policy iteration method of deep reinforcement learning, including:

[0024] Acquiring task scenario status information, wherein the task scenario status information includes task difficulty information, resource status information, and environmental constraint information;

[0025] Constructing a state space representation based on the user behavior feature vector and the task scenario state information, calculating the probability of transition between states based on the state space representation, and constructing a state transfer function based on the importance weight of the historical state sequence, wherein the state transfer function includes the state evolution law and the state transition constraint;

[0026] Monitor the task execution process in real time, calculate the task completion index based on the task target completion rate, task quality score, and task timeliness, and calculate the execution efficiency index based on the computing resource utilization rate, storage resource occupancy rate, and operation time utilization rate, and construct a reward function by weightedly combining the task completion index and execution efficiency index;

[0027] The state space representation is input into a pre-trained deep reinforcement learning model, the state value is calculated based on the state transfer function, and the strategy is iteratively optimized according to the reward function and the state value to generate an interactive decision sequence.

[0028] In an optional embodiment,

[0029] Inputting the state space representation into a pre-trained deep reinforcement learning model, calculating the state value based on the state transition function, performing iterative strategy optimization based on the reward function and the state value, and generating an interactive decision sequence includes:

[0030] Acquire historical interaction data, extract state space representation, action sequence, and reward value from the historical interaction data, construct a training sample set, pre-train a deep reinforcement learning model based on the training sample set, and obtain policy network parameters and value network parameters;

[0031] Inputting the current state space representation into the policy network, generating action selection probabilities based on the policy network parameters, sampling the current state based on the action selection probabilities, and obtaining a candidate action sequence;

[0032] Inputting the state space representation and the candidate action sequence into a value network, calculating the state action value based on the value network parameters, calculating the state transition probability according to the state transition function, and combining the state action value and the state transition probability to obtain the expected cumulative value;

[0033] Calculate the immediate reward of the candidate action sequence according to the reward function, and combine the immediate reward with the expected cumulative value to generate an action evaluation value;

[0034] A cyclic unit is used to encode the historical state sequence to generate time series correlation features, which are combined with action evaluation values ​​to calculate the policy update gradient. The policy network parameters are iteratively optimized based on the policy update gradient, and an interactive decision sequence is generated according to the optimized policy network parameters and action evaluation values.

[0035] In an optional embodiment,

[0036] Constructing a task execution plan graph based on the interactive decision sequence and generating a task execution plan with checkpoints based on the task execution plan graph include:

[0037] Extracting state information and decision actions from the interactive decision sequence, calculating the state transition frequency of the state information under the action of the decision action, and calculating the state transition probability based on the state transition frequency;

[0038] Constructing nodes of a task execution planning graph based on the state information, constructing edges of the task execution planning graph based on the state transition probability, calculating the set of subsequent nodes that can be reached by each node and the transition cost, counting the number of times each state node is visited and the number of times each decision action is selected in the interactive decision sequence, and calculating the access weight of the node and the selection weight of the decision action in combination with the state transition probability;

[0039] Constructing a path evaluation function based on the access weight, selection weight, and transfer cost, solving an optimal execution path in a task execution planning graph using the path evaluation function, calculating the access weight change rate and state transition probability change rate of each node on the optimal execution path, and taking the weighted sum of the access weight change rate and the state transition probability change rate as the node importance;

[0040] A node with the local maximum importance is selected on the optimal execution path as a checkpoint, and the state information, optional decision actions and access weights of the nodes corresponding to the checkpoints are extracted to generate checkpoint configuration information. The optimal execution path, checkpoint configuration information and the state transition probabilities between the nodes are combined to generate a task execution plan.

[0041] In an optional embodiment,

[0042] Collect the execution status data of each checkpoint, calculate the task completion trajectory curve based on the execution status data, and extract the user's decision choices at each checkpoint in combination with the task execution plan diagram. Fuse the decision choices with the user behavior feature vector to generate the ability dimension vector. Calculate the professional competency score based on the ability dimension vector and generate a professional competency assessment report including:

[0043] Collecting execution status data of checkpoints during the user's execution of the task execution plan, the execution status data includes current state characteristics, selected decision actions, execution results and timestamp information;

[0044] Calculating the difference between the state feature in the execution state data and the preset state of the task execution plan map to obtain a state deviation, and calculating the difference between the decision action in the execution state data and the recommended action in the task execution plan map to obtain an action deviation;

[0045] Calculating a checkpoint completion score based on a weighted sum of the state deviation and the action deviation, calculating an execution efficiency score based on a change in execution results between adjacent timestamps, and weightedly combining the completion score and the execution efficiency score to obtain a checkpoint score;

[0046] The score of each checkpoint and the corresponding timestamp are interpolated and fitted to generate the task completion trajectory curve. Combined with the preset status and recommended actions recorded in the task execution plan, the user's decision choices at each checkpoint are extracted.

[0047] The decision-making choice and the user behavior feature vector are feature-fused in the corresponding dimensions to generate an ability dimension vector, the normalized score of the feature value of each dimension in the ability dimension vector is calculated, the normalized score is used as the professional competency score, and a professional competency assessment report is generated.

[0048] A second aspect of an embodiment of the present invention provides a virtual workplace multi-dimensional assessment system based on deep reinforcement learning, comprising:

[0049] The first unit is used to collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, construct a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user's behavior pattern based on the state transition probability matrix, and generate a user behavior feature vector;

[0050] The second unit is used to construct a state space representation based on the user behavior feature vector, calculate the state transition function based on the task scenario state information, set the reward function based on the task completion and execution efficiency, and use the policy iteration method of deep reinforcement learning to generate the interactive decision sequence;

[0051] The third unit is used to construct a task execution planning graph according to the interactive decision sequence, and generate a task execution plan with checkpoints based on the task execution planning graph;

[0052] The third unit is used to collect the execution status data of each checkpoint during the user's execution of the task execution plan, calculate the task completion trajectory curve based on the execution status data, and extract the user's decision choices at each checkpoint in combination with the task execution planning diagram, and fuse the decision choices with the user behavior feature vector to generate an ability dimension vector, calculate the professional competency score based on the ability dimension vector, and generate a professional competency assessment report.

[0053] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0054] processor;

[0055] a memory for storing processor-executable instructions;

[0056] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0057] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0058] In this embodiment, by collecting user behavior data in a virtual workplace environment and extracting time series features, and combining deep reinforcement learning technology to construct a decision sequence, it is possible to objectively and comprehensively evaluate the user's performance in the workplace environment, avoid the subjective bias in traditional assessment methods, and improve the scientificity and fairness of the assessment. Through a task execution plan with checkpoints, refined monitoring of each key node of the user's task execution process is achieved. By calculating the task completion trajectory curve, the user's execution ability and adaptability can be dynamically reflected, providing a richer and more reliable data basis for professional ability assessment. The user's decision choices are fused with the behavioral feature vector to generate a multi-dimensional ability dimension vector, making the assessment results more three-dimensional and comprehensive, and being able to accurately identify the user's strengths and weaknesses in different professional ability dimensions, providing targeted guidance and suggestions for the user's career development, and providing a scientific basis for corporate talent selection and training. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flowchart of a multi-dimensional assessment method for a virtual workplace based on deep reinforcement learning according to an embodiment of the present invention;

[0060] Figure 2 Schematic diagram of the convergence curve of the reward value of deep reinforcement learning strategy iteration;

[0061] Figure 3 A schematic diagram of the mission execution plan. DETAILED DESCRIPTION

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0063] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0064] Figure 1 This is a flow chart of a multi-dimensional assessment method for a virtual workplace based on deep reinforcement learning according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0065] Collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, construct a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user's behavior pattern based on the state transition probability matrix, and generate a user behavior feature vector;

[0066] A state space representation is constructed based on the user behavior feature vector. The state transition function is calculated based on the task scenario state information. The reward function is set according to the task completion and execution efficiency. The policy iteration method of deep reinforcement learning is used to generate the interactive decision sequence.

[0067] Build a task execution plan graph based on the interactive decision sequence, and generate a task execution plan with checkpoints based on the task execution plan graph;

[0068] When the user is executing the task execution plan, the execution status data of each checkpoint is collected, and the task completion trajectory curve is calculated based on the execution status data. The user's decision choices at each checkpoint are extracted in combination with the task execution planning diagram, and the decision choices are fused with the user behavior feature vector to generate an ability dimension vector. The professional competency score is calculated based on the ability dimension vector, and a professional competency assessment report is generated.

[0069] In an optional embodiment, user behavior data in a virtual workplace environment is collected, and time series features are extracted. The user's behavior state sequence at different task stages is calculated, and a state transition probability matrix is ​​constructed based on the behavior state sequence. The stability characteristics of the user behavior pattern are analyzed according to the state transition probability matrix. The generation of the user behavior feature vector includes:

[0070] Collecting user behavior data in a virtual workplace environment, the behavior data including operation sequence data, task switching data, problem solving data, and collaboration pattern data;

[0071] Calculating a behavior complexity index for the behavior data, dynamically determining a time window size based on the behavior complexity index, extracting statistical features and trend features of the behavior data within the time window, and combining them to generate a behavior state vector;

[0072] Calculating a time series correlation for the behavior state vector, determining a behavior transition threshold according to the time series correlation, and performing density clustering on the behavior state vector based on the behavior transition threshold to generate a behavior state sequence;

[0073] The frequency of transitions between states is calculated according to the behavioral state sequence, a state transition probability matrix is ​​constructed, and the state transition probability matrix is ​​subjected to matrix decomposition to obtain a first transition probability submatrix and a second transfer probability submatrix; the first transition probability submatrix and the second transition probability submatrix are subjected to eigenvalue decomposition respectively to obtain an eigenvalue set, the behavioral entropy value is calculated according to the maximum eigenvalue in the eigenvalue set, and the behavioral stability feature is calculated according to the remaining eigenvalues, and the behavioral entropy value and the behavioral stability feature are feature fused to generate a user behavior feature vector.

[0074] For example, when collecting user behavior data in a virtual workplace environment, the user's operation sequence data, task switching data, problem-solving data, and collaboration mode data are recorded. Operation sequence data includes basic user operations such as clicking, dragging, and typing, and their timestamps; task switching data records the frequency and switching mode of users switching between different tasks; problem-solving data includes the path users choose to solve problems and the time consumed; and collaboration mode data records the user's interaction with other virtual characters or real users. For example, when a user processes a document task in a virtual office environment, an operation sequence such as "open document A - read for 10 minutes - modify content - send a message to colleague B - switch to task C" will be recorded.

[0075] A behavioral complexity index is calculated for the collected behavioral data to dynamically determine the time window size. This index is derived from a comprehensive calculation of the diversity of operation types, the rate of change in operation frequency, the frequency of task switching, and the complexity of collaborative interactions. In specific implementation, if a user's operation types frequently change within a certain period, the complexity is assessed as high, and the time window is narrowed to 30 seconds accordingly. If the operation type is single and stable, the complexity is assessed as low, and the time window is expanded to 120 seconds. Within the specified time window, statistical and trend features of the behavioral data are extracted. Statistical features include operation frequency, task duration, and switching frequency; trend features include operation acceleration and efficiency change trends. For example, in a virtual meeting scenario, the number of user speeches within a 60-second window is extracted as 5, with an average speech duration of 20 seconds and a question-asking frequency of 3 times every 10 minutes. These statistical values ​​and their changing trends together constitute the behavioral state vector.

[0076] The generated behavior state vectors are then evaluated for temporal correlation, analyzing the similarity of behavior states within adjacent time windows. Temporal correlation is determined by calculating the distance between adjacent state vectors, which in turn determines the behavior transition threshold. For example, for a virtual collaboration task, analysis reveals that the distance between state vectors is concentrated in the range of 0.1-0.3, but occasionally jumps above 0.7. In this case, the behavior transition threshold can be set to 0.5. Based on this threshold, the behavior state vectors are density-clustered, grouping state vectors with similar distances into the same category, thereby generating a behavior state sequence. For example, for a user completing a virtual project management task, their behavior state sequence might be represented as a temporal sequence of discrete states, such as "planning state - execution state - coordination state - review state - execution state - planning state."

[0077] Based on the generated behavioral state sequence, the transition frequencies between states are calculated to construct a state transition probability matrix. Each element in this matrix represents the probability of transitioning from one state to another. For example, in a four-state scenario, in the constructed 4×4 matrix, the element M(1, 2) = 0.35 indicates a 35% probability of transitioning from state 1 to state 2. The state transition probability matrix is ​​subjected to matrix decomposition to obtain the first and second transition probability submatrices. The first submatrix reflects the state transition characteristics of the user in stable working mode, while the second submatrix reflects the state transition characteristics under abnormal or stressful conditions.

[0078] Perform eigenvalue decomposition on each of the two transition probability submatrices to obtain a set of eigenvalues. For the first submatrix, the possible eigenvalue set is {0.92, 0.45, 0.28, 0.15}; for the second submatrix, the possible eigenvalue set is {0.75, 0.62, 0.41, 0.22}. Behavioral entropy is calculated based on the largest eigenvalues ​​in the set (e.g., 0.92 and 0.75). This value indicates the predictability of user behavior. Lower entropy values ​​indicate more stable user behavior patterns, such as 0.25, which indicates highly predictable behavior patterns. Higher entropy values, such as 0.85, indicate greater randomness in user behavior. Behavioral stability features, including state transition stability, state persistence, and adaptability, are calculated based on the remaining eigenvalues ​​(e.g., {0.45, 0.28, 0.15} and {0.62, 0.41, 0.22}).

[0079] The behavioral entropy and behavioral stability features are fused to generate a user behavior feature vector. This feature vector can be expressed as [behavioral entropy, state transition stability, state persistence, adaptability index], for example, [0.35, 0.78, 0.62, 0.45]. This feature vector can be used to assess the stability of user behavior patterns in virtual workplace environments and can be further applied to professional competency assessment, personalized training program design, or team collaboration optimization.

[0080] This technical solution enables in-depth modeling and stability analysis of user behavior in virtual workplace environments, accurately characterizing their behavioral characteristics during multitasking, problem-solving, and collaboration. By dynamically adjusting the time window to extract key behavioral features and combining the calculation of the state transition probability matrix with behavioral entropy, a quantitative assessment of the degree of change and stability of behavioral patterns can be achieved. This helps distinguish differences in cognitive load, task focus, and behavioral consistency among different users, providing support for personalized training, behavioral risk warnings, and intelligent decision-making.

[0081] In an optional embodiment, calculating a time series correlation for the behavior state vector, determining a behavior transition threshold according to the time series correlation, and performing density clustering on the behavior state vector based on the behavior transition threshold to generate a behavior state sequence includes:

[0082] Obtaining a behavior state vector, calculating a vector space distance and a time interval distance between adjacent behavior state vectors, combining the vector space distance and the time interval distance to construct a time series correlation function, and calculating a time series correlation degree of the behavior state vector using the time series correlation function;

[0083] Perform statistical analysis on the time series correlation to obtain the distribution characteristics of the time series correlation, generate a behavior jump threshold based on the distribution characteristics, determine the behavior jump point based on the time series correlation and the behavior jump threshold, and calculate the local density of the behavior state vector at the behavior jump point;

[0084] Density clustering is performed on the behavior state vector based on the local density to obtain a behavior state clustering result, and the behavior state clustering result is arranged and combined in time sequence to generate a behavior state sequence.

[0085] Exemplarily, the vector space distance and time interval distance of adjacent behavior state vectors are calculated. The vector space distance represents the degree of difference between two behavior state vectors in the feature space and can be calculated using the Euclidean distance. For example, if two behavior state vectors are [0.8, 0.6, 0.9, 0.7] and [0.7, 0.6, 0.8, 0.5], respectively, their Euclidean distance is 0.22. The time interval distance represents the difference between the time points at which two behavior state vectors were collected and can be measured in minutes or hours. For example, if the collection times of two behavior state vectors are 10:00 and 10:15, respectively, the time interval is 15 minutes.

[0086] The vector space distance and time interval distance are combined to construct a temporal correlation function. This function measures the temporal correlation between behavioral state vectors, taking into account both the vector space distance and the time interval. A function can be designed that comprehensively considers both factors. For example, when the vector space distance is small and the time interval is short, the temporal correlation is high; when the vector space distance is large or the time interval is long, the temporal correlation is low. In practice, the vector space distance can be divided by the time interval to obtain a ratio; a smaller ratio indicates a higher temporal correlation. For example, if the vector space distance is 0.22 and the time interval is 15 minutes, the ratio is 0.0147, indicating that the two behavioral states have a high temporal correlation.

[0087] The temporal correlation function is used to calculate the temporal correlation of the behavior state vectors. For all pairs of adjacent behavior state vectors, their temporal correlation is calculated to form a temporal correlation sequence. For example, if the system collects behavior state vectors every 15 minutes during a workday, for a total of 32 vectors, 31 temporal correlation values ​​will be obtained.

[0088] Perform statistical analysis on the time series correlation to obtain its distribution characteristics. Distribution characteristics can include statistics such as mean, variance, and quantiles. For example, for the 31 time series correlation values ​​above, the calculated mean is 0.02, the standard deviation is 0.015, the 25th percentile is 0.01, and the 75th percentile is 0.03. These distribution characteristics reflect the regularity of behavioral state changes.

[0089] Generate a behavior transition threshold based on the distribution characteristics. This threshold is used to determine whether a significant change in behavior has occurred. An appropriate threshold can be selected based on the distribution characteristics of the time series correlation. For example, the 75th percentile of the time series correlation, 0.03, can be selected as the behavior transition threshold. This means that a behavior transition is considered to have occurred when the time series correlation between adjacent behavior state vectors is greater than 0.03.

[0090] The behavior jump point is determined based on the temporal correlation and the behavior jump threshold. The behavior jump point refers to the time point at which the behavior state changes significantly. For example, if the temporal correlation between the 5th and 6th behavior state vectors is 0.035, which exceeds the behavior jump threshold of 0.03, the time point corresponding to the 6th behavior state vector is marked as a behavior jump point. The local density of the behavior state vector is calculated at the behavior jump point. The local density indicates the degree of clustering of the behavior state vectors in a certain area in the behavior state vector space. The local density can be estimated by calculating the distance between a behavior state vector and its surrounding behavior state vectors. For example, for the 6th behavior state vector, calculate its average distance from the 3 behavior state vectors before and after it. The smaller the distance, the higher the local density. Assume that the local density of the 6th behavior state vector is calculated to be 0.8.

[0091] Density clustering is performed on behavioral state vectors based on local density. Density clustering is a density-based clustering algorithm that groups contiguous regions with similar density into clusters. The system can use density clustering algorithms such as DBSCAN or OPTICS to perform clustering based on the local density of behavioral state vectors. For example, for 32 behavioral state vectors, density clustering may yield five clusters, each corresponding to a different behavioral state type. The behavioral state clustering results are arranged and combined in time sequence to generate a behavioral state sequence. A behavioral state sequence refers to the evolution of an employee's behavioral state over a period of time. For example, if 32 behavioral state vectors are clustered into five categories, the resulting behavioral state sequence might be "Category 1 (vectors 1-5) -> Category 2 (vectors 6-12) -> Category 3 (vectors 13-18) -> Category 4 (vectors 19-25) -> Category 5 (vectors 26-32)." This sequence reflects the evolution of an employee's behavioral state over different time periods.

[0092] In practical applications, this method can be used to analyze and assess employee behavior in a virtual workplace environment. For example, for a salesperson, the system can collect their behavioral state vectors throughout the day, including multiple dimensions such as the number of customer communications, product demonstrations, and transaction amounts. By generating a behavioral state sequence using this method, the salesperson's different work states can be identified, such as the preparation phase, communication phase, and transaction phase, allowing for targeted evaluation and assessment.

[0093] Based on the above technical solution, it is possible to achieve fine-grained identification and dynamic segmentation of user behavior states in the temporal dimension, thereby effectively capturing the mutation points and stable patterns of behavior states. By introducing a temporal correlation function, the content similarity and temporal continuity of behavior states are comprehensively considered to improve the accuracy of jump detection. Clustering based on the local density of jump points can avoid the misjudgment of abnormal behavior or stage transitions by traditional temporal segmentation methods, and generate state sequences that are more consistent with the actual evolution of behavior. This provides a clearly structured and hierarchical data foundation for subsequent behavior modeling, stage identification, and behavior prediction, and enhances the system's ability to perceive complex behavior patterns.

[0094] In an optional embodiment, a state space representation is constructed based on the user behavior feature vector, a state transition function is calculated based on the task scenario state information, a reward function is set according to the task completion and execution efficiency, and a policy iteration method of deep reinforcement learning is used to generate an interactive decision sequence, including:

[0095] Acquiring task scenario status information, wherein the task scenario status information includes task difficulty information, resource status information, and environmental constraint information;

[0096] Constructing a state space representation based on the user behavior feature vector and the task scenario state information, calculating the probability of transition between states based on the state space representation, and constructing a state transfer function based on the importance weight of the historical state sequence, wherein the state transfer function includes the state evolution law and the state transition constraint;

[0097] Monitor the task execution process in real time, calculate the task completion index based on the task target completion rate, task quality score, and task timeliness, and calculate the execution efficiency index based on the computing resource utilization rate, storage resource occupancy rate, and operation time utilization rate, and construct a reward function by weightedly combining the task completion index and execution efficiency index;

[0098] The state space representation is input into a pre-trained deep reinforcement learning model, the state value is calculated based on the state transfer function, and the strategy is iteratively optimized according to the reward function and the state value to generate an interactive decision sequence.

[0099] In this embodiment, during the task scenario status information acquisition phase, task difficulty information, resource status information, and environmental constraint information are collected. Task difficulty information includes a task complexity score (1-10 points), required skill level (beginner / intermediate / advanced), and estimated completion time (hours); resource status information includes the percentage of available computing resources (0%-100%), storage space remaining (GB), and bandwidth utilization (%); and environmental constraint information includes the maximum allowed runtime (hours), power consumption limit (watts), and noise control requirements (decibels). For example, the task scenario status information for a data processing task might be: task complexity score 8 points, intermediate skill level required, estimated completion time 4 hours; available computing resources 75%, storage space remaining 500GB, bandwidth utilization 30%; maximum allowed runtime 6 hours, power consumption limit 100W, and noise control requirement no more than 40 decibels.

[0100] During the state space representation construction phase, the user behavior feature vector is integrated with task scenario state information. The user behavior feature vector includes dimensions such as operation frequency, interaction mode, and preferences. Specifically, operation frequency records the number of user operations per minute; interaction mode includes types and proportions of mouse clicks, keyboard input, and touchscreen operations; and preferences include interface layout, response speed, and use of accessibility features. The system uses feature embedding methods to convert this heterogeneous data into a unified vector representation with 128 dimensions. The probability of transitions between states is calculated by analyzing historical interaction data. The probability of transitioning from state i to state j is equal to the number of direct transitions from state i to state j in the historical record divided by the total number of transitions from state i. The importance weight of the historical state sequence is calculated using a time-decay method, with more recent states being given higher weights. For example, states within the last hour receive a weight of 1.0, states within 1-3 hours receive a weight of 0.8, states within 3-12 hours receive a weight of 0.5, states within 12-24 hours receive a weight of 0.3, and states older than 24 hours receive a weight of 0.1. The construction of the state transfer function combines the state migration probability and importance weight, and introduces state migration constraints, such as resource utilization cannot exceed 95% and response time cannot exceed 200 milliseconds.

[0101] During the task execution monitoring phase, task completion and execution efficiency metrics are calculated in real time. The task completion metric is calculated based on the following parameters: task target completion rate (number of completed subtasks / total number of subtasks × 100%), task quality score (1-10, assessed using pre-set quality checkpoints), and task timeliness (actual time / estimated time × 100%). The execution efficiency metric is calculated based on the following parameters: computing resource utilization (actual CPU / GPU percentage used), storage resource utilization (used storage space / allocated storage space × 100%), and operation time utilization (effective operation time / total operation time × 100%). The reward function is constructed using a weighted combination, with a weight of 0.6 for the task completion metric and a weight of 0.4 for the execution efficiency metric. For example, when the task target completion rate is 85%, the task quality score is 7 points, and the task timeliness is 90%, the task completion index is 0.85×0.4+7 / 10×0.3+(1-0.9)×0.3=0.64; when the computing resource utilization rate is 70%, the storage resource occupancy rate is 60%, and the operation time utilization rate is 80%, the execution efficiency index is 0.7×0.4+0.6×0.3+0.8×0.3=0.7; the final reward value is 0.64×0.6+0.7×0.4=0.664.

[0102] During the interactive decision sequence generation phase, a deep reinforcement learning model is used for policy iteration. This model consists of a value network and a policy network. The value network estimates state value, while the policy network generates action probability distributions. The value network consists of three fully connected layers, with 256, 128, and 64 nodes per layer, respectively, using the ReLU activation function. The policy network consists of four fully connected layers, with 256, 128, and 128 nodes per layer, respectively, and the action space dimension. The first three layers use the ReLU activation function, and the final layer uses the Softmax function to output action probabilities. Policy iteration optimization involves the following steps: inputting the current state into the value network to obtain a state value estimate; inputting the current state into the policy network to obtain a probability distribution of possible actions; sampling and selecting actions based on the probability distribution; executing actions and observing environmental feedback to obtain rewards and next states; and updating the value and policy network parameters based on the rewards and next states. Network parameter updates are performed using the Adam optimizer with a learning rate of 0.0003. After 10,000 epochs of training, the model can generate interactive decision sequences that are tailored to user behavior and task scenario requirements. For example, for a document editing task, the generated interactive decision sequence includes: opening the most recently edited related document first (improving relevance), recommending applicable document templates (reducing operation time), providing intelligent completion during user input (improving efficiency), prompting to save at the appropriate time (avoiding data loss), and recommending relevant reference materials based on the user's editing content (improving quality).

[0103] In this embodiment, a dynamic state space is constructed by integrating user behavior characteristics with task scenario state information. Combined with a deep reinforcement learning policy iteration method, this approach effectively improves decision-making efficiency and personalized adaptability in complex interaction scenarios. Existing technologies often formulate interaction strategies based on preset rules or static behavior models, failing to flexibly adapt to individual user differences and the dynamic changes in the task environment. This can easily lead to irrational resource allocation, deviation from target task execution, and a degraded user experience. This solution, centered around user behavior feature vectors, constructs a state space based on task difficulty, resource status, and environmental constraints, enhancing the comprehensiveness and pertinence of state representation. Furthermore, by incorporating weights from historical state sequences into the state transition function, it enables reasonable modeling of state evolution trends. In terms of reward function design, it integrates task completion and execution efficiency, breaking away from the traditional reinforcement learning evaluation model based solely on outcome feedback and better reflecting the multidimensional performance of task execution. Ultimately, through the policy iteration optimization method of deep reinforcement learning, the system can dynamically adjust interaction strategies, achieving adaptive optimization under different user and task scenarios, effectively improving the intelligence level and decision-making quality of the interactive system.

[0104] Figure 2Schematic diagram of the reward value convergence curve for deep reinforcement learning policy iterations. The main curve shows the trend of the reward function value as the number of iterations increases. As training progresses, the reward value gradually increases and eventually converges to a stable value. The experiment used a deep reinforcement learning architecture that combines a value network and a policy network. The value network (3 fully connected layers with 256, 128, and 64 nodes, respectively) is used to evaluate state values, and the policy network (4 fully connected layers with 256, 128, and 128 nodes, respectively, and the action space dimension) is used to generate the action probability distribution. The Adam optimizer was used during training, with a learning rate of 0.0003.

[0105] In this experiment, a state space representation was constructed by integrating the user behavior feature vector (128 dimensions) with task scenario state information (including task difficulty, resource status, and environmental constraints). The state evolution pattern was modeled using a state transition function. The reward function design comprehensively considered the task completion index (weight 0.6) and the execution efficiency index (weight 0.4). The figure shows three key stages: the initial learning stage (the first 400 iterations), the starting point of policy convergence (approximately 600 iterations), and the stable convergence stage (after 850 iterations). The final model achieved a reward value of 0.664 in the test task, with a task completion index of 0.64 and an execution efficiency index of 0.70, demonstrating good task execution capabilities and resource utilization efficiency.

[0106] In an optional embodiment, inputting the state space representation into a pre-trained deep reinforcement learning model, calculating the state value based on the state transition function, performing policy iterative optimization based on the reward function and the state value, and generating an interactive decision sequence includes:

[0107] Acquire historical interaction data, extract state space representation, action sequence, and reward value from the historical interaction data, construct a training sample set, pre-train a deep reinforcement learning model based on the training sample set, and obtain policy network parameters and value network parameters;

[0108] Inputting the current state space representation into the policy network, generating action selection probabilities based on the policy network parameters, sampling the current state based on the action selection probabilities, and obtaining a candidate action sequence;

[0109] Inputting the state space representation and the candidate action sequence into a value network, calculating the state action value based on the value network parameters, calculating the state transition probability according to the state transition function, and combining the state action value and the state transition probability to obtain the expected cumulative value;

[0110] Calculate the immediate reward of the candidate action sequence according to the reward function, and combine the immediate reward with the expected cumulative value to generate an action evaluation value;

[0111] A cyclic unit is used to encode the historical state sequence to generate time series correlation features, which are combined with action evaluation values ​​to calculate the policy update gradient. The policy network parameters are iteratively optimized based on the policy update gradient, and an interactive decision sequence is generated according to the optimized policy network parameters and action evaluation values.

[0112] For example, historical interaction data is first acquired. This data is derived from the user's historical interactions with the system. For example, in an intelligent recommendation system, historical interaction data may include records of user behavior such as browsing, clicking, and purchasing items. From this historical interaction data, a state space representation, action sequence, and reward value are extracted. The state space representation can be a user feature vector, such as [0.8, 0.2, 0.5, 0.7], representing the user's interests and preferences; the action sequence can be a list of recommended item IDs, such as [1001, 1024, 1056]; and the reward value can be feedback from the user's click or purchase behavior, such as assigning a value of 1 to a click and 0 to a non-click. This extracted data is then constructed into a training sample set, with each sample containing a state representation, an action performed, and a reward received. Based on this constructed training sample set, the system pre-trains the deep reinforcement learning model to obtain initial policy network parameters and value network parameters. The policy network parameters can be expressed as a set of weight values, such as [-0.2, 0.5, 0.3, -0.1]; the value network parameters can also be expressed as another set of weight values, such as [0.4, -0.3, 0.6, 0.2].

[0113] When an interactive decision needs to be made for the current user, the current state space representation is first input into the policy network. For example, the current user state is represented as [0.7, 0.3, 0.6, 0.4], which represents the user's current interests, preferences, and behavioral characteristics. Based on the input state representation and the pre-trained policy network parameters, the policy network generates selection probabilities for each possible action. Assuming the system has three possible actions, the policy network outputs selection probabilities of [0.2, 0.5, 0.3], indicating a probability of selecting action 1 of 0.2, a probability of selecting action 2 of 0.5, and a probability of selecting action 3 of 0.3. Based on these action selection probabilities, sampling is performed, and the resulting candidate action sequence is [2, 3, 2, 1], indicating that action 2, action 3, action 2, and action 1 are selected as candidate recommendation sequences.

[0114] The state space representation and candidate action sequences are input into the value network. State-action values ​​are calculated based on the pre-trained value network parameters. For example, for the action sequence [2, 3, 2, 1], the value network calculates state-action values ​​of [3.2, 2.8, 3.0, 2.5], representing the expected long-term value of executing these actions. State transition probabilities are calculated based on the state transition function, which describes the probability distribution of the system transitioning to a new state after executing an action in the current state. For example, after executing action 2, the system has a 0.7 probability of transitioning to state A and a 0.3 probability of transitioning to state B. The state-action values ​​and state transition probabilities are combined to obtain the expected cumulative value. For example, the calculated expected cumulative value is [2.8, 2.5, 2.7, 2.2], representing the expected cumulative value of each action after taking state transitions into account.

[0115] The immediate reward for each candidate action sequence is calculated based on a reward function. A reward function defines the immediate feedback obtained from performing an action under a specific state. For example, if a user clicks a recommended item, the system receives a reward of 1; if the user purchases the recommended item, the system receives a reward of 5; and if the user ignores the recommendation, the system receives a reward of 0. Assume that for the candidate action sequence [2, 3, 2, 1], the calculated immediate rewards are [1, 0, 1, 5], respectively. The immediate reward is combined with the expected cumulative value to generate an action evaluation value. For example, the immediate reward and expected cumulative value can be weighted and summed, with weights of 0.3 and 0.7, to obtain an action evaluation value of [2.26, 1.75, 2.19, 3.04], representing the comprehensive evaluation of the value of performing each action.

[0116] Recurrent units are used to encode historical state sequences and generate temporal correlation features. Historical state sequences record the user's state changes during past interactions. Recurrent units (such as long short-term memory (LSTM) networks) can capture the temporal dependencies within state sequences. Assume that the temporal correlation features encoded by the system are [0.6, 0.4, 0.7, 0.3], representing the temporal pattern of the user's historical behavior. The temporal correlation features are combined with the action evaluation values ​​to calculate the policy update gradient. For example, the temporal correlation features can be used as weights and weighted with the action evaluation values ​​to obtain adjusted action evaluation values ​​of [1.356, 0.7, 1.533, 0.912]. Based on these adjusted action evaluation values, the update gradients of the policy network parameters are calculated and iteratively optimized. The optimized policy network parameters may become [-0.15, 0.55, 0.28, -0.08], indicating that a better decision-making policy has been learned. Finally, based on the optimized policy network parameters and action evaluation values, the final interactive decision sequence, such as [2, 1, 3, 2], is generated as the recommendation decision for the current user.

[0117] In this embodiment, a dynamic state space is constructed by integrating user behavior characteristics with task scenario state information. This is combined with a deep reinforcement learning policy iteration method to effectively improve decision-making efficiency and personalized adaptability in complex interactive scenarios. Existing technologies often formulate interaction strategies based on preset rules or static behavior models, failing to flexibly adjust to individual user differences and the dynamic changes in the task environment. This can easily lead to irrational resource allocation, deviation from target task execution, and a degraded user experience. This solution, centered around user behavior feature vectors, constructs a state space based on task difficulty, resource status, and environmental constraints, enhancing the comprehensiveness and pertinence of state representation. Furthermore, by incorporating weights from historical state sequences into the state transition function, it enables reasonable modeling of state evolution trends. In terms of reward function design, it integrates task completion and execution efficiency, breaking away from the traditional reinforcement learning evaluation model based solely on outcome feedback and better reflecting the multidimensional performance of task execution. Ultimately, through the policy iteration optimization method of deep reinforcement learning, the system can dynamically adjust interaction strategies, achieving adaptive optimization under different user and task scenarios, effectively improving the intelligence level and decision-making quality of the interactive system.

[0118] In an optional embodiment, constructing a task execution plan graph according to the interactive decision sequence, and generating a task execution plan with checkpoints based on the task execution plan graph includes:

[0119] Extracting state information and decision actions from the interactive decision sequence, calculating the state transition frequency of the state information under the action of the decision action, and calculating the state transition probability based on the state transition frequency;

[0120] Constructing nodes of a task execution planning graph based on the state information, constructing edges of the task execution planning graph based on the state transition probability, calculating the set of subsequent nodes that can be reached by each node and the transition cost, counting the number of times each state node is visited and the number of times each decision action is selected in the interactive decision sequence, and calculating the access weight of the node and the selection weight of the decision action in combination with the state transition probability;

[0121] Constructing a path evaluation function based on the access weight, selection weight, and transfer cost, solving an optimal execution path in a task execution planning graph using the path evaluation function, calculating the access weight change rate and state transition probability change rate of each node on the optimal execution path, and taking the weighted sum of the access weight change rate and the state transition probability change rate as the node importance;

[0122] A node with the local maximum importance is selected on the optimal execution path as a checkpoint, and the state information, optional decision actions and access weights of the nodes corresponding to the checkpoints are extracted to generate checkpoint configuration information. The optimal execution path, checkpoint configuration information and the state transition probabilities between the nodes are combined to generate a task execution plan.

[0123] In this embodiment, state information and decision actions are extracted from the interactive decision sequence. State information refers to the work status of an employee at a specific point in time in a virtual workplace environment, and may include characteristic values ​​of multiple dimensions such as current task progress, resource usage, and work efficiency indicators. For example, for a project manager, the state information may include project completion, team collaboration index, resource allocation rationality, etc. Decision actions refer to the actions taken by employees in a specific state, such as assigning tasks, adjusting resources, and holding meetings. The interactive decision sequence is a sequence composed of a series of state information and decision actions in chronological order, which records the state changes and decision-making process of employees during task execution.

[0124] Calculate the state transition frequency of state information under the influence of decision actions. State transition frequency indicates the number of times a state transitions from one state to another through a decision action in an interactive decision sequence. For example, if a project manager makes the "Add Resources" decision in the "Project Delayed" state, the state might transition to "Project Schedule Restored" five times and to "Project Still Delayed" only two times.

[0125] The state transition probability is calculated based on the state transition frequency. The state transition probability represents the probability of transitioning to each possible state after a decision action is taken in a certain state. For example, if a project manager makes the "add resources" decision in the "project delayed" state, the probability of transitioning to "project progress restored" is 5 / 7, and the probability of transitioning to "project still delayed" is 2 / 7. The state transition probability reflects the effectiveness of the decision action and the uncertainty of the state transition.

[0126] Nodes in the task execution plan graph are constructed based on state information. A task execution plan graph is a directed graph structure that represents the state transitions during task execution. Each node in the graph corresponds to a state, such as "project initiation," "requirements analysis," or "design implementation." All different states that occur in the interactive decision sequence are considered nodes in the plan graph.

[0127] The edges of the task execution plan graph are constructed based on state transition probabilities. Edges in the plan graph represent transitions from one state node to another, and the weights on the edges are the corresponding state transition probabilities. For example, the weight of the edge from the "Project Delayed" node to the "Project Schedule Restored" node is 5 / 7. This approach constructs a complete task execution plan graph, reflecting the overall structure and probability distribution of state transitions. The set of reachable subsequent nodes and the transition cost are calculated for each node. The set of subsequent nodes refers to all nodes reachable from the current node via a single transition. The transition cost is the "cost" required to move from the current node to a subsequent node. It can be calculated as the inverse of the state transition probability; the higher the transition probability, the lower the transition cost. For example, the transition cost from the "Project Delayed" node to the "Project Schedule Restored" node is 7 / 5, while the transition cost to the "Project Still Delayed" node is 7 / 2.

[0128] Count the number of times each state node is visited and the number of times each decision action is selected in the interactive decision sequence. Node visit counts indicate the total number of times a state is reached in the interactive decision sequence. Decision action selection counts indicate the total number of times a decision action is selected in the interactive decision sequence. For example, the "Project Delay" state might be visited 10 times, while the "Add Resources" decision action might be selected 7 times.

[0129] The state transition probabilities are combined to calculate the node's access weight and the decision action's selection weight. The access weight reflects the importance of the state node in the task execution process and can be calculated by dividing the number of node visits by the total number of state transitions. The decision action's selection weight reflects its importance in the task execution process and can be calculated by dividing the number of decision action selections by the total number of decisions. For example, if there are 100 state transitions in the interactive decision sequence and the "Project Delay" state is visited 10 times, its access weight is 0.1.

[0130] A path evaluation function is constructed based on access weights, selection weights, and transfer costs. This function evaluates the merits of different paths within a task execution plan graph, comprehensively considering the access weights of each node on the path, the selection weights of each decision action, and the transfer costs of each transfer. For example, a function can be designed such that the greater the sum of the access weights of the nodes on the path, the greater the sum of the selection weights of the decision actions, and the smaller the sum of the transfer costs, the higher the path evaluation value.

[0131] The path evaluation function is used to determine the optimal execution path within the task execution plan graph. The optimal execution path is the path with the highest path evaluation function value among all possible paths from the start node to the target node within the task execution plan graph. Dynamic programming or graph search algorithms, such as the Dijkstra algorithm or the A* algorithm, can be used to determine the optimal execution path within the task execution plan graph. For example, for a project management task, the optimal execution path might be "project initiation → requirements analysis → design implementation → testing and verification → project delivery."

[0132] Calculate the access weight change rate and state transition probability change rate for each node on the optimal execution path. The access weight change rate indicates the degree of change in the access weights of adjacent nodes and can be calculated by dividing the difference in the access weights of adjacent nodes by the access weight of the previous node. The state transition probability change rate indicates the degree of change in the state transition probability between adjacent transitions and can be calculated by dividing the difference in the state transition probability of adjacent transitions by the state transition probability of the previous transition.

[0133] The weighted sum of the access weight change rate and the state transition probability change rate is used as the node importance. Node importance reflects the criticality of a node in the optimal execution path. Highly important nodes are often key points or turning points in task execution. For example, if a node has a high access weight change rate and a high state transition probability change rate, then the node is highly important and may represent a critical stage in task execution.

[0134] On the optimal execution path, select the node with the highest local importance as a checkpoint. A checkpoint is a critical node that requires special attention and evaluation during task execution. By comparing the importance of adjacent nodes, the node with the highest local importance is selected as the checkpoint. For example, in the optimal execution path of "Project Initiation → Requirements Analysis → Design Implementation → Test and Verification → Project Delivery", if the "Design Implementation" node has the highest local importance, then it is designated as the checkpoint.

[0135] The checkpoint configuration information is generated by extracting the status information, optional decision actions, and access weights of the corresponding checkpoint node. This information includes a description of the checkpoint's status, available decision actions, and corresponding weights, guiding decision-making and evaluation during task execution. For example, for a "Design Implementation" checkpoint, its configuration information might include status information such as "Design Document Completion" and "Technical Difficulty Resolution," as well as optional decision actions such as "Design Plan Adjustment" and "Add Technical Resources."

[0136] The optimal execution path, checkpoint configuration information, and state transition probabilities between nodes are combined to generate a task execution plan. A task execution plan is a complete task execution guidance document that includes the optimal path from task initiation to completion, key checkpoints, and the probabilities of each state transition. For example, the execution plan for a project management task might include the optimal execution path "Project Initiation → Requirements Analysis → Design Implementation → Test and Verification → Project Delivery", the configuration information for the "Design Implementation" checkpoint, and the probabilities of each state transition, such as a probability of 0.9 for the transition from "Requirements Analysis" to "Design Implementation".

[0137] During the multi-dimensional assessment process in the virtual workplace, this task execution plan can be used to guide employee execution and serve as a benchmark for assessment. It can compare the employee's actual execution path with the optimal one, assess their performance at checkpoints, and analyze the consistency of their decisions with recommended ones, enabling comprehensive and objective multi-dimensional assessments. Furthermore, based on deep reinforcement learning algorithms, the task execution plan and optimal execution path can be continuously optimized, improving the accuracy and guidance of assessments.

[0138] This technical solution enables structured representation of interactive decision-making processes and dynamic calibration of key nodes, thereby improving the controllability and robustness of task execution plans. Existing technologies often rely on static flowcharts or predefined paths, lacking a clear understanding of actual user interactions and making it difficult to identify key control points in complex tasks. This results in weak fault tolerance and delayed feedback during task execution. This solution constructs a task execution plan by extracting state transition probabilities and behavior weights, comprehensively integrating users' path preferences and decision-making habits in actual operations. It then introduces a path evaluation function to comprehensively evaluate multiple possible paths, ensuring that the selected path strikes a balance between efficiency and stability. By calculating the rate of change of access weights and transition probabilities, it identifies fluctuating regions in the task flow and selects representative, important nodes as checkpoints, effectively enhancing the task plan's responsiveness to emergencies and process monitoring capabilities. The resulting task execution plan not only reflects the optimal decision path for user behavior but also clarifies the relationship between key control nodes and state transitions, providing solid data support for dynamic task management and process optimization.

[0139] Figure 3 This diagram shows an optimal execution path (connected by solid arrows) for task execution planning: starting with "Project Initiation" (weight 0.15), passing through "Requirements Analysis" (weight 0.18), reaching the checkpoint "Design Implementation" (weight 0.25), then entering "Testing and Verification" (weight 0.22), and finally completing "Project Delivery" (weight 0.20). The "Design Implementation" node is identified as a critical checkpoint based on its local maximum importance.

[0140] Two alternative paths (connected by dotted arrows) are also shown: one is from "Project Initiation" to "Scheme Review" (weight 0.12) and then to "Design Implementation"; the other is from "Design Implementation" to "Bug Fixing" (weight 0.15) and then to "Project Delivery." The state transition probabilities show that in the "Project Initiation" state, there is a 0.75 probability of directly entering "Requirements Analysis" and a 0.25 probability of transitioning to "Scheme Review." In the "Design Implementation" state, there is a 0.68 probability of directly entering "Testing and Verification" and a 0.32 probability of requiring "Bug Fixing."

[0141] This task execution planning diagram quantifies the feasibility of different execution paths by analyzing the frequency and probability of state transitions in the interactive decision sequence, and identifies key control nodes, providing data support and guidance for task execution.

[0142] In an optional embodiment, the execution status data of each checkpoint is collected, and the task completion trajectory curve is calculated based on the execution status data. The user's decision choices at each checkpoint are extracted in combination with the task execution plan diagram. The decision choices are fused with the user behavior feature vector to generate a capability dimension vector. The professional competency score is calculated based on the capability dimension vector, and the generated professional competency assessment report includes:

[0143] Collecting execution status data of checkpoints during the user's execution of the task execution plan, the execution status data includes current state characteristics, selected decision actions, execution results and timestamp information;

[0144] Calculating the difference between the state feature in the execution state data and the preset state of the task execution plan map to obtain a state deviation, and calculating the difference between the decision action in the execution state data and the recommended action in the task execution plan map to obtain an action deviation;

[0145] Calculating a checkpoint completion score based on a weighted sum of the state deviation and the action deviation, calculating an execution efficiency score based on a change in execution results between adjacent timestamps, and weightedly combining the completion score and the execution efficiency score to obtain a checkpoint score;

[0146] The score of each checkpoint and the corresponding timestamp are interpolated and fitted to generate the task completion trajectory curve. Combined with the preset status and recommended actions recorded in the task execution plan, the user's decision choices at each checkpoint are extracted.

[0147] The decision-making choice and the user behavior feature vector are feature-fused in the corresponding dimensions to generate an ability dimension vector, the normalized score of the feature value of each dimension in the ability dimension vector is calculated, the normalized score is used as the professional competency score, and a professional competency assessment report is generated.

[0148] For example, execution status data is first collected at checkpoints during the user's task execution plan. Execution status data is a comprehensive record of the user's work status, decision-making behavior, and execution results at each checkpoint during task execution. Execution status data consists of four key components: current state characteristics, selected decision actions, execution results, and timestamp information. Current state characteristics provide a multi-dimensional description of the user's work status at that checkpoint. For example, for a product manager, these characteristics might include the completeness of the product requirements document, the depth of user research, and the completeness of the prototype design. Selected decision actions refer to the specific actions the user has chosen in the current state, such as "optimize product requirements" or "increase user research samples." Execution results are quantitative or qualitative evaluations of the effects of executing the decision actions, such as "product requirements document completeness increased by 20%." Timestamp information records the exact time of state collection for subsequent time series analysis.

[0149] The state deviation is calculated by calculating the difference between the state characteristics in the execution status data and the preset state in the task execution plan. The task execution plan presets the ideal state characteristics for each checkpoint, representing the working state that the user should achieve at that checkpoint. The state deviation represents the gap between the user's actual state and the ideal state. It can be obtained by calculating the difference between the state characteristic vector and the preset state vector. For example, if the product manager's actual completion value of the product requirements document at the requirements analysis checkpoint is 75%, while the preset value is 90%, the state deviation is -15%. The smaller the state deviation, the closer the user's working state is to the ideal state.

[0150] Action deviation is calculated by calculating the difference between the decision action in the execution status data and the action recommended by the task execution plan. The task execution plan recommends the optimal decision action for each checkpoint. These recommended actions are learned from a large amount of historical data using a deep reinforcement learning model. Action deviation indicates the degree of discrepancy between the decision action actually selected by the user and the recommended action. In actual implementation, decision actions can be represented as multidimensional vectors, with each dimension corresponding to the execution intensity of a basic action type. Action deviation can be calculated by calculating the difference between the decision action vector and the recommended action vector. For example, if the recommended action is "Increase user research samples and optimize the requirements document," but the user only performs "Optimize the requirements document," there is action deviation. The smaller the action deviation, the closer the user's decision is to the optimal strategy.

[0151] The checkpoint completion score is calculated based on the weighted sum of state deviation and action deviation. The completion score reflects the degree to which the user's work status and decision-making behavior at the checkpoint align with the ideal state. During the calculation process, different weights can be assigned to state deviation and action deviation based on different task types and checkpoint characteristics. Typically, state deviation is weighted more highly because work status directly reflects task completion. For example, a weight of 0.7 for state deviation and 0.3 for action deviation can be set, and the completion score is derived through this weighted combination. A higher completion score indicates better performance at the checkpoint.

[0152] The execution efficiency score is calculated based on the change in execution results between adjacent timestamps. This score reflects the speed and efficiency with which a user progresses tasks between checkpoints. The calculation compares the change in execution results between adjacent timestamps and evaluates the time interval. For example, if a user increases the completion level of a product requirements document from 50% to 80% within two hours, the execution efficiency score is calculated based on this change and the time interval. A higher execution efficiency score indicates a more efficient user.

[0153] The checkpoint score is calculated by weighting the completion score and the execution efficiency score. The checkpoint score is a quantitative evaluation of the user's overall performance at the checkpoint, taking into account multiple factors such as work status, decision-making behavior, and execution efficiency. In practice, the weighting of the completion score and execution efficiency score can be adjusted based on different positions and task characteristics. For example, R&D engineers may prioritize execution efficiency, so the execution efficiency score can be given a higher weight; while product managers may prioritize work quality, so the completion score can be given a higher weight.

[0154] The scores and corresponding timestamps of each checkpoint are interpolated and fitted to generate a task completion trajectory curve. This curve, with time on the horizontal axis and the checkpoint score on the vertical axis, reflects the changing trend of the user's performance throughout the task. An interpolation fitting algorithm is used to connect the discrete checkpoint scores into a smooth curve. Common interpolation methods include linear interpolation and spline interpolation, and the appropriate interpolation method can be selected based on the data characteristics. The task completion trajectory curve can intuitively demonstrate the user's work rhythm and performance fluctuations, helping to identify the user's strengths and areas for improvement.

[0155] By combining the preset states and recommended actions recorded in the task execution plan, we can extract the user's decision choices at each checkpoint. Decision choices refer to the specific actions taken by the user at each checkpoint and are an important basis for assessing their decision-making and problem-solving abilities. By comparing user choices with the recommended actions in the task execution plan, we can analyze their decision-making patterns and preferences. For example, we might find that a user tends to choose conservative decisions and is less likely to take innovative ones. This decision choice information will be used in subsequent ability dimension analysis.

[0156] Decision choices are fused with the user behavior feature vector in corresponding dimensions to generate a capability dimension vector. The user behavior feature vector is a multi-dimensional description of user behavior patterns in the virtual workplace, formed through long-term observation and recording. It encompasses communication style, collaboration patterns, learning ability, stress tolerance, and other aspects. Feature fusion integrates decision choice information with the user behavior feature vector to form a more comprehensive capability dimension vector. During the fusion process, decision choices are mapped to corresponding capability dimensions and weightedly combined with the corresponding dimensions in the user behavior feature vector. For example, if a user chooses to "increase resource investment" when facing the risk of project delays, this decision is mapped to dimensions such as "resource management capability" and "risk response capability" and fused with the corresponding dimensions in the user behavior feature vector.

[0157] Calculate the normalized score of the eigenvalues ​​for each dimension in the ability dimension vector and use this normalized score as the professional competency score. Normalization is the process of converting eigenvalues ​​of different dimensions and ranges to a unified standard to ensure comparability across all dimensions. Common normalization methods include min-max normalization and Z-score normalization. Through normalization, the eigenvalues ​​for each dimension in the ability dimension vector are converted into a professional competency score between 0 and 100. For example, a user's "communication and coordination skills" score might be 85, "professional skills" might be 92, and "innovative thinking" might be 78. These scores reflect the user's performance level in each ability dimension.

[0158] A professional competency assessment report is generated based on the professional competency score. This report is a comprehensive assessment of the user's professional competency, including analysis of competency dimensions, identification of strengths, and suggestions for improvement. The report displays the user's task completion trajectory, analyzes performance characteristics at different checkpoints, and highlights the distribution of the user's professional competency score across various dimensions. Based on a deep reinforcement learning model, the report compares the user's competency profile with different positions and provides career development recommendations.

[0159] In practical application, this method can provide a scientific basis for talent assessment and development within companies. Through task execution and multi-dimensional assessment within a virtual workplace environment, companies can gain a comprehensive understanding of their employees' capabilities, characteristics, and development potential, providing precise guidance for talent development, job matching, and career planning. Furthermore, through professional competency assessment reports, employees can clearly identify their strengths and weaknesses, improve key competencies in a targeted manner, and achieve their career development goals.

[0160] In this embodiment, by collecting status data during task execution and comparing and analyzing it with the planning map, the user's task completion status can be evaluated in real time. By calculating the state deviation and action deviation, the gap between the user's execution effect and the expected goal can be accurately measured. Combined with the weighted calculation of the completion score and the execution efficiency score, the user's performance level during task execution is fully reflected. By interpolating and fitting the checkpoint scores, a continuous task completion trajectory curve is generated, which intuitively shows the user's execution process and ability improvement trend. By integrating the decision-making choices with the behavioral feature vectors, and generating the ability dimension vectors on the corresponding dimensions, a multi-dimensional quantitative evaluation of the user's professional ability is achieved. Finally, a standardized competency score is obtained through normalization processing, which provides an objective basis for talent assessment and job matching. This solution can not only dynamically track and evaluate the user's task execution status, but also deeply analyze the user's decision-making behavior and ability characteristics, providing data support for enterprise talent development.

[0161] A second aspect of an embodiment of the present invention provides a virtual workplace multi-dimensional assessment system based on deep reinforcement learning, the system comprising:

[0162] The first unit is used to collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, construct a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user's behavior pattern based on the state transition probability matrix, and generate a user behavior feature vector;

[0163] The second unit is used to construct a state space representation based on the user behavior feature vector, calculate the state transition function based on the task scenario state information, set the reward function based on the task completion and execution efficiency, and use the policy iteration method of deep reinforcement learning to generate the interactive decision sequence;

[0164] The third unit is used to construct a task execution planning graph according to the interactive decision sequence, and generate a task execution plan with checkpoints based on the task execution planning graph;

[0165] The fourth unit is used to collect the execution status data of each checkpoint during the user's execution of the task execution plan, calculate the task completion trajectory curve based on the execution status data, and extract the user's decision choices at each checkpoint in combination with the task execution planning diagram, and fuse the decision choices with the user behavior feature vector to generate an ability dimension vector, calculate the professional competency score based on the ability dimension vector, and generate a professional competency assessment report.

[0166] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0167] processor;

[0168] a memory for storing processor-executable instructions;

[0169] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0170] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0171] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-dimensional assessment method for virtual workplace based on deep reinforcement learning, characterized by: include: Collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, construct a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user's behavior pattern based on the state transition probability matrix, and generate a user behavior feature vector; A state space representation is constructed based on the user behavior feature vector. The state transition function is calculated based on the task scenario state information. The reward function is set according to the task completion and execution efficiency. The policy iteration method of deep reinforcement learning is used to generate the interactive decision sequence. Build a task execution plan graph based on the interactive decision sequence, and generate a task execution plan with checkpoints based on the task execution plan graph; During the user's execution of the task plan, the execution status data of each checkpoint is collected. The task completion trajectory curve is calculated based on the execution status data. The user's decision choices at each checkpoint are extracted in combination with the task execution plan diagram. The decision choices are then fused with the user's behavior feature vector to generate a capability dimension vector. The professional competency score is calculated based on the capability dimension vector to generate a professional competency assessment report. Constructing a task execution plan graph based on the interactive decision sequence and generating a task execution plan with checkpoints based on the task execution plan graph include: Extracting state information and decision actions from the interactive decision sequence, calculating the state transition frequency of the state information under the action of the decision action, and calculating the state transition probability based on the state transition frequency; Constructing nodes of a task execution planning graph based on the state information, constructing edges of the task execution planning graph based on the state transition probability, calculating the set of subsequent nodes that can be reached by each node and the transition cost, counting the number of times each state node is visited and the number of times each decision action is selected in the interactive decision sequence, and calculating the access weight of the node and the selection weight of the decision action in combination with the state transition probability; Constructing a path evaluation function based on the access weight, selection weight, and transfer cost, solving an optimal execution path in a task execution planning graph using the path evaluation function, calculating the access weight change rate and state transition probability change rate of each node on the optimal execution path, and taking the weighted sum of the access weight change rate and the state transition probability change rate as the node importance; A node with the local maximum importance is selected on the optimal execution path as a checkpoint, and the state information, optional decision actions and access weights of the nodes corresponding to the checkpoints are extracted to generate checkpoint configuration information. The optimal execution path, checkpoint configuration information and the state transition probabilities between the nodes are combined to generate a task execution plan.

2. The method according to claim 1, characterized in that Collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, build a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user behavior pattern based on the state transition probability matrix, and generate the user behavior feature vector including: Collecting user behavior data in a virtual workplace environment, the behavior data including operation sequence data, task switching data, problem solving data, and collaboration pattern data; Calculating a behavior complexity index for the behavior data, dynamically determining a time window size based on the behavior complexity index, extracting statistical features and trend features of the behavior data within the time window, and combining them to generate a behavior state vector; Calculating a time series correlation for the behavior state vector, determining a behavior transition threshold according to the time series correlation, and performing density clustering on the behavior state vector based on the behavior transition threshold to generate a behavior state sequence; The frequency of transitions between states is calculated according to the behavioral state sequence, a state transition probability matrix is ​​constructed, and the state transition probability matrix is ​​subjected to matrix decomposition to obtain a first transition probability submatrix and a second transfer probability submatrix; the first transition probability submatrix and the second transition probability submatrix are subjected to eigenvalue decomposition respectively to obtain an eigenvalue set, the behavioral entropy value is calculated according to the maximum eigenvalue in the eigenvalue set, and the behavioral stability feature is calculated according to the remaining eigenvalues, and the behavioral entropy value and the behavioral stability feature are feature fused to generate a user behavior feature vector.

3. The method according to claim 2, characterized in that Calculating a time series correlation for the behavior state vector, determining a behavior transition threshold according to the time series correlation, and performing density clustering on the behavior state vector based on the behavior transition threshold to generate a behavior state sequence includes: Obtaining a behavior state vector, calculating a vector space distance and a time interval distance between adjacent behavior state vectors, combining the vector space distance and the time interval distance to construct a time series correlation function, and calculating a time series correlation degree of the behavior state vector using the time series correlation function; Perform statistical analysis on the time series correlation to obtain the distribution characteristics of the time series correlation, generate a behavior jump threshold based on the distribution characteristics, determine the behavior jump point based on the time series correlation and the behavior jump threshold, and calculate the local density of the behavior state vector at the behavior jump point; Density clustering is performed on the behavior state vector based on the local density to obtain a behavior state clustering result, and the behavior state clustering result is arranged and combined in time sequence to generate a behavior state sequence.

4. The method according to claim 1, wherein The state space representation is constructed based on the user behavior feature vector, the state transition function is calculated based on the task scenario state information, the reward function is set according to the task completion and execution efficiency, and the interactive decision sequence is generated using the policy iteration method of deep reinforcement learning, including: Acquiring task scenario status information, wherein the task scenario status information includes task difficulty information, resource status information, and environmental constraint information; Constructing a state space representation based on the user behavior feature vector and the task scenario state information, calculating the probability of transition between states based on the state space representation, and constructing a state transfer function based on the importance weight of the historical state sequence, wherein the state transfer function includes the state evolution law and the state transition constraint; Monitor the task execution process in real time, calculate the task completion index based on the task target completion rate, task quality score, and task timeliness, and calculate the execution efficiency index based on the computing resource utilization rate, storage resource occupancy rate, and operation time utilization rate, and construct a reward function by weightedly combining the task completion index and execution efficiency index; The state space representation is input into a pre-trained deep reinforcement learning model, the state value is calculated based on the state transfer function, and the strategy is iteratively optimized according to the reward function and the state value to generate an interactive decision sequence.

5. The method according to claim 4, characterized in that Inputting the state space representation into a pre-trained deep reinforcement learning model, calculating the state value based on the state transition function, performing iterative strategy optimization based on the reward function and the state value, and generating an interactive decision sequence includes: Acquire historical interaction data, extract state space representation, action sequence, and reward value from the historical interaction data, construct a training sample set, pre-train a deep reinforcement learning model based on the training sample set, and obtain policy network parameters and value network parameters; Inputting the current state space representation into the policy network, generating action selection probabilities based on the policy network parameters, sampling the current state based on the action selection probabilities, and obtaining a candidate action sequence; Inputting the state space representation and the candidate action sequence into a value network, calculating the state action value based on the value network parameters, calculating the state transition probability according to the state transition function, and combining the state action value and the state transition probability to obtain the expected cumulative value; Calculate the immediate reward of the candidate action sequence according to the reward function, and combine the immediate reward with the expected cumulative value to generate an action evaluation value; A cyclic unit is used to encode the historical state sequence to generate time series correlation features, which are combined with action evaluation values ​​to calculate the policy update gradient. The policy network parameters are iteratively optimized based on the policy update gradient, and an interactive decision sequence is generated according to the optimized policy network parameters and action evaluation values.

6. The method according to claim 1, characterized in that Collect the execution status data of each checkpoint, calculate the task completion trajectory curve based on the execution status data, and extract the user's decision choices at each checkpoint in combination with the task execution plan diagram. Fuse the decision choices with the user behavior feature vector to generate the ability dimension vector. Calculate the professional competency score based on the ability dimension vector and generate a professional competency assessment report including: Collecting execution status data of checkpoints during the user's execution of the task execution plan, the execution status data includes current state characteristics, selected decision actions, execution results and timestamp information; Calculating the difference between the state feature in the execution state data and the preset state of the task execution plan map to obtain a state deviation, and calculating the difference between the decision action in the execution state data and the recommended action in the task execution plan map to obtain an action deviation; Calculating a checkpoint completion score based on a weighted sum of the state deviation and the action deviation, calculating an execution efficiency score based on a change in execution results between adjacent timestamps, and weightedly combining the completion score and the execution efficiency score to obtain a checkpoint score; The score of each checkpoint and the corresponding timestamp are interpolated and fitted to generate the task completion trajectory curve. Combined with the preset status and recommended actions recorded in the task execution plan, the user's decision choices at each checkpoint are extracted. The decision-making choice and the user behavior feature vector are feature-fused in the corresponding dimensions to generate an ability dimension vector, the normalized score of the feature value of each dimension in the ability dimension vector is calculated, the normalized score is used as the professional competency score, and a professional competency assessment report is generated.

7. A virtual workplace multi-dimensional assessment system based on deep reinforcement learning, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is used to collect user behavior data in the virtual workplace environment, extract time series features, calculate the user's behavior state sequence at different task stages, construct a state transition probability matrix based on the behavior state sequence, analyze the stability characteristics of the user's behavior pattern based on the state transition probability matrix, and generate a user behavior feature vector; The second unit is used to construct a state space representation based on the user behavior feature vector, calculate the state transition function based on the task scenario state information, set the reward function based on the task completion and execution efficiency, and use the policy iteration method of deep reinforcement learning to generate the interactive decision sequence; The third unit is used to construct a task execution planning graph according to the interactive decision sequence, and generate a task execution plan with checkpoints based on the task execution planning graph; The fourth unit is used to collect the execution status data of each checkpoint during the user's execution of the task execution plan, calculate the task completion trajectory curve based on the execution status data, and extract the user's decision choices at each checkpoint in combination with the task execution planning diagram, and fuse the decision choices with the user behavior feature vector to generate an ability dimension vector, calculate the professional competency score based on the ability dimension vector, and generate a professional competency assessment report.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images