A work station intelligent agent task execution value evaluation and visualization method and device based on value point quantification, electronic equipment and storage medium

Abstract

The application provides a work station intelligent agent task execution value evaluation and visualization method and device based on value point quantification, electronic equipment and storage medium, belonging to the technical field of data processing. It aims to solve the technical problem that the existing technology lacks effective identification, quantification and visualization of the business value created by the work station intelligent agent. It includes: value point identification of the task execution log of the work station intelligent agent, obtaining at least one value point, and the type of the value point belongs to a plurality of preset value point types; for each identified value point, quantitatively score according to a plurality of business value dimensions; based on the value point type and the business value score of each value point, determine the comprehensive value score of the work station intelligent agent; according to the comprehensive value score, generate and display a visualization component. The application can realize the transformation from process indicators to the measurement of actual business value, more accurately evaluate the business contribution of the work station intelligent agent, and realize intuitive display through the visualization component.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, electronic device and storage medium for evaluating and visualizing the task execution value of a workstation intelligent agent based on value point quantification. Background Technology

[0002] In enterprise-level artificial intelligence (AI) agent platforms, workstation agents serve as execution units, undertaking various automated tasks. With the development of large-scale modeling technology, effectively measuring and demonstrating the business value created by these workstation agents for enterprises has become a core requirement for the operation and management of agent platforms.

[0003] Current value assessment methods primarily focus on data asset quality evaluation and data storage value scoring. For example, some solutions emphasize quality attributes such as the integrity and accuracy of data assets, or assess data storage priorities, evaluating static data or storage strategies. Current value assessment methods lack a business value perspective, failing to automatically identify and quantify the value created by AI agents performing tasks, and also lacking visualization of value, making it difficult for managers to accurately measure and optimize the business contributions of AI agents. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, electronic device and storage medium for evaluating and visualizing the task execution value of intelligent agents at workstations based on value point quantification, so as to solve the technical problem of lacking effective identification, quantification and visualization of the business value created by intelligent agents in the prior art.

[0005] To achieve the above objectives, this application provides a method for evaluating and visualizing the task execution value of a workstation intelligent agent based on value point quantification. The method includes: identifying value points in the task execution logs of the workstation intelligent agent to obtain at least one value point, wherein the value point type of the at least one value point belongs to a preset set of multiple value point types; quantifying and scoring each identified value point according to multiple business value dimensions to obtain a business value score for each business value dimension; determining a comprehensive value score for the workstation intelligent agent based on the value point type of each value point and the business value score for each business value dimension; and generating and displaying a visualization component based on the comprehensive value score.

[0006] Optionally, the step of identifying value points in the task execution log of the workstation agent to obtain at least one value point includes: extracting keywords from the target text included in the execution log; in response to a successful match between the extracted keywords and a keyword library, determining at least one value point based on the keyword matching result; wherein the target text includes one or more of action description text and output result text, and the keyword library includes keywords corresponding to the preset multiple value point types; and / or, extracting temporal behavioral features from the execution events included in the execution log; in response to a successful match between the temporal behavioral features and a behavioral feature template, determining at least one value point based on the feature matching result; wherein the behavioral feature template includes behavioral features of events of the preset multiple value point types; and / or, determining the influence range of the output results included in the execution log; in response to the influence range satisfying the value condition, determining at least one value point based on the influence range.

[0007] Optionally, generating and displaying the visualization component based on the comprehensive value score includes: generating and displaying the visualization component based on the comprehensive value score and the analysis results, wherein the analysis results are obtained by performing multi-dimensional analysis on the comprehensive value score, and the analysis dimensions of the comprehensive value score include one or more of value distribution, trend prediction, and anomaly attribution.

[0008] Optionally, the preset multiple value point types include at least one of decision-making value type, insight value type, cost-saving value type, time-saving value type, and accuracy improvement value type.

[0009] Optionally, the multiple business value dimensions include at least one of the following: scope of influence, frequency of occurrence, reusability, innovation, and business relevance.

[0010] Optionally, determining the comprehensive value score of the workstation agent based on the value point type of each value point and the business value score of each business value dimension includes: for each value point, determining the weighted sum of the business value dimensions according to the business value score of each business value dimension and the weight of each business value dimension; and determining the comprehensive value score of the workstation agent according to the intensity coefficient corresponding to the value point type of each value point and the weighted sum of the business value dimensions.

[0011] Optionally, the weight of each business value dimension is dynamically determined based on the self-awareness context of the workstation agent, which is used to characterize the workstation agent's meta-cognition of its own state, capabilities, and boundaries.

[0012] Optionally, the method further includes: in response to detecting an abnormal change in the overall value score or the business value score, and the abnormal change meeting an abnormal condition, performing an attribution operation to determine the root cause of the abnormal change.

[0013] Optionally, the attribution operation includes: determining the abnormal location and change information of the abnormal change, wherein the abnormal location includes the abnormal time period and the identifier of the workstation agent where the abnormal change occurred, and the change information includes the value point type and magnitude of the abnormal value point; obtaining task execution information related to the abnormal change based on the abnormal location; and performing multi-dimensional analysis on the task execution information to determine the root cause; wherein the multi-dimensional analysis includes one or more of the following dimensions: task relationship diagram task structure change dimension, data source availability change dimension, model version switching dimension, prompt word configuration change dimension, and service level target constraint change dimension.

[0014] Optionally, the method further includes generating an anomaly report based on the root cause.

[0015] Optionally, the visualization components include one or more of the following: a value heatmap, a value trend chart, an attribution radar chart, a value ranking, and an anomaly marker; the value heatmap is a two-dimensional matrix with the time axis as the horizontal axis and the value point type as the vertical axis, and the color intensity in the value heatmap is used to represent the density of the comprehensive value score of each value point type in each time period; the value trend chart is used to show the trend of the comprehensive value score of each value point type over time; the attribution radar chart is used to show the score distribution of the workstation agent in multiple business value dimensions; the value ranking is used to show the ranking order of the workstation agent in terms of the comprehensive value score; and the anomaly marker is used to identify the comprehensive value score that has changed abnormally.

[0016] Optionally, the anomaly identifier is also used to display an anomaly report of the comprehensive value score that has undergone abnormal changes after being triggered.

[0017] This application also provides a workstation intelligent agent task execution value assessment and visualization device based on value point quantification. The device includes: an identification unit configured to identify value points in the task execution logs of the workstation intelligent agent to obtain at least one value point, wherein the value point type of the at least one value point belongs to a preset plurality of value point types; a first evaluation unit configured to quantify and score each identified value point according to multiple business value dimensions to obtain a business value score for each business value dimension; a second evaluation unit configured to determine a comprehensive value score of the workstation intelligent agent based on the value point type of each value point and the business value score of each business value dimension; and a visualization unit configured to generate and display a visualization component based on the comprehensive value score.

[0018] This application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any of the preceding claims.

[0019] This application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform any of the methods described above.

[0020] Compared with the prior art, this application has the following beneficial effects: 1. The technical solution provided in this application, by identifying, quantifying and scoring the value points of the task execution logs of the workstation intelligent agent and determining the comprehensive value score, realizes the transformation from focusing on process indicators such as success rate and response time to measuring actual business value, and can more accurately evaluate the business contribution of the workstation intelligent agent.

[0021] 2. This application determines the comprehensive value score by quantitatively scoring multiple business value dimensions and combining the intensity coefficients corresponding to the value point types. This achieves refined attribution and differentiated quantification of the sources of value contribution, making the value assessment results more accurate and interpretable.

[0022] 3. By generating and displaying visualization components, this application can intuitively and panoramically show the value creation of intelligent agents at workstations, making it easier for managers to quickly grasp and analyze the situation, and improving the observability of AI platform operation and management. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the value assessment and visualization method according to an embodiment of this application.

[0025] Figure 2 This is a flowchart illustrating the anomaly attribution method according to an embodiment of this application.

[0026] Figure 3 This is a schematic diagram of the structure of a value assessment and visualization device according to an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for descriptive purposes only and is not intended to limit the scope of this application.

[0029] Before providing a further detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0030] (1) A workstation-based office space is a mapping of a job position in an enterprise into a digital system. It is a persistent organizational unit that corresponds to a job position (rather than a specific person) in the enterprise. A workstation-based office space is a "container" that contains all the configurations, data, and workstations for that job position.

[0031] (2) Workstation: A persistent task definition unit containing a Mission Relationship Graph (MRG). The Mission Relationship Graph is used to define the mission, inputs and outputs, skill requirements, etc. of the workstation. The workstation exists before the workstation agent and is defined through the MRG editor.

[0032] (3) Mission Relationship Graph (MRG): A formal task representation language based on directed graphs. MRG includes: Mission Node Start (MNS), Mission Node (MN), Mission Edge (ME), and Operational Mission Node (OMN). Among them, MNS: The most basic and core description of user needs, and the starting point and root node of task deduction. MNS summarizes the entire task in natural language and is used for retrieval and matching of workstations / Agents. MN: Intermediate results or sub-goals in the deduction process, the execution results are not visible to the user. ME: Directed edge, connecting the previous node (MNS or MN) and the next node (MN or OMN), representing the logical reasoning. ME is not a node and does not generate independent code units. OMN: A special subclass of MN, representing the stage or final demand that the user expects to obtain (user-oriented output), and is the necessary and sufficient condition for triggering Agent execution and matching. The code template of OMN inherits the execution logic of MN and adds an extension layer such as output formatting and persistence.

[0033] (4) Workstation Agent: In this application, also referred to as Workstation Agent, or simply Agent, refers to a specific execution instance bound to a specific workstation in the enterprise-level AI agent collaborative operating system. When a workstation needs to be put into operation (responding to a demand), the system creates an independent Agent instance for that workstation and binds it. The Workstation Agent is the smallest unit for executing automated tasks within the workstation-based office space. The Workstation Agent is only responsible for the tasks defined for that workstation and transmits necessary data (Token consumption, cost, output results, etc.) according to the requirements of the development framework. The task context of the Workstation Agent is consistent with the MRG definition of the bound workstation.

[0034] (5) Mission Context: The mission description of the Agent, expressed in MRG format. The Mission Context can be initiated by the user through LUI or triggered passively.

[0035] (6) Ego Context: This refers to the dynamic cognition of a workstation agent regarding its own state, capabilities, boundaries, organizational environment, and strategic goals. It defines the "worldview" of the workstation agent and can be dynamically generated and continuously evolved. Ego Context inherits the TrustLink authorization topology of its assigned workstation office and retrieves data from the DTV data stream and file system to form the cognitive background of that workstation office (including organizational goals, historical performance, strategic document knowledge, etc.) to provide background information and constraints during decision-making (such as determining weights). Ego Context provides constraints and context for decision-making, rather than being an object of active "alignment".

[0036] (7) Trust Link: This is an authorization link between offices based on parent-child relationships, represented by directed edges (upper level → lower level). Trust Link defines data visibility, command issuance permissions, and peer isolation rules to determine the legal path of data flow.

[0037] (8) Data Value Flow (DTV): This is an abbreviation for Data flow, Trust link, and Value point, i.e., the data-trust-value triple. DTV describes how execution data (such as token consumption, task status, and output results) and value data (such as cost and value score) generated by workstation agents flow and converge across organizational levels based on the Trust Link topology. Among them, Data flow refers to the execution data (Token consumption, task status, quality score, output results, etc.) transparently transmitted by each workstation agent.

[0038] (9) Value point: This refers to the value and cost generated by each workstation agent. A value point refers to the event or result that can be identified as beneficial to the business when the workstation agent performs its tasks.

[0039] In enterprise-level AI agent platforms, there are numerous workstation agents performing various automated tasks. However, existing technologies generally lack effective means to measure and intuitively demonstrate their contribution from a business value perspective, making it difficult to measure the value created by workstation agents for the business. This application aims to address this technical pain point by constructing a complete "value identification-quantification-display" technical closed loop to achieve accurate assessment and dynamic monitoring of the business value of workstation agents.

[0040] Please see Figure 1This application provides a flowchart of a method for evaluating and visualizing the task execution value of a workstation intelligent agent based on value point quantification, including steps S110-S140: Step S110: Identify value points in the task execution logs of the workstation intelligent agent to obtain at least one value point.

[0041] The task execution log of the workstation agent can be a DTV execution log. The task execution log records the process of executing the task.

[0042] At least one value point belongs to one of several preset value point types. These preset value point types can be determined based on the business value classification types created by the workstation agent. Value point types can be defined based on business needs.

[0043] As an example, the pre-defined value point types may include at least one of the following: decision-making value type, insight value type, cost-saving value type, time-saving value type, and accuracy improvement value type. Specifically, the decision-making value type reflects how the workstation agent's output directly or indirectly influences the formulation or adjustment of business decisions. The insight value type reflects how the workstation agent discovers data patterns, trends, or anomalies that were previously unnoticed by humans. The cost-saving value type reflects how the workstation agent's execution replaces manual or other paid operations. The time-saving value type reflects how the workstation agent's execution significantly shortens task completion time. The accuracy improvement value type reflects how the workstation agent's execution improves the accuracy or precision of task results.

[0044] This classification method systematically defines the main business value forms that workstation agents may create, enabling managers to clearly understand the value creation structure of workstation agents, such as determining whether their main contribution comes from cost savings or decision support, thereby providing a basis for subsequent resource optimization and capacity allocation.

[0045] Step S110 transforms the complex and unstructured execution records of the workstation agent into quantifiable and analyzable value events with clear business implications. Unlike methods that only focus on process metrics such as task success or failure, this step delves into the semantics and impact of task execution. By proactively identifying value points, it addresses the limitation of existing technologies in measuring the specific business contributions of AI agents. This step maps previously vague AI outputs to clear business value categories, laying the foundation for subsequent refined evaluation.

[0046] In one possible implementation, the process of identifying value points from the task execution logs of a workstation agent can employ various methods. For example, one method, keyword recognition, involves extracting keywords from the target text (such as action description text or output result text) in the execution log. When the extracted keywords successfully match a pre-defined keyword library, one or more value points can be determined based on the matching results. The keyword library pre-stores keywords (such as "suggestion," "strategy," "discovery," "trend," etc.) corresponding to various value point types (such as "decision value," "insight value," etc.). This method, based on Natural Language Processing (NLP), effectively captures value signals from text descriptions. Furthermore, another method, feature recognition, extracts temporal behavioral features from the execution events in the execution log, such as resource consumption patterns, response latency trends, and fluctuations in task completion quality scores. When these temporal behavioral features match pre-defined behavioral feature templates, value points can also be determined. For example, a behavioral pattern where token consumption is significantly lower than the equivalent manual cost can be identified as "cost-saving value." Another identification method, namely the scope of influence identification method, can determine the scope of influence of the output results in the execution log. The scope of influence, for example, is the path of the output results within the organization. When this scope of influence meets specific value conditions, the value point can be determined accordingly. These value conditions can be flexibly set based on business needs. As an example, value conditions include the output influencing the upper-level decision chain, meaning it is adopted by higher-level decision-makers. Another example is triggering an Ego Context state update.

[0047] By employing one or more of the aforementioned identification methods, value points can be identified from different perspectives, significantly improving the accuracy and coverage of value point identification. Combining multiple identification methods can reduce the omission of value points in the identification process.

[0048] As an example, this application provides the definitions and suitable identification methods for the above five value point types, as shown in Table 1.

[0049] Table 1 Step S120: For each identified value point, quantify and score it according to multiple business value dimensions to obtain the business value score for each business value dimension.

[0050] The business value dimension is a dimension for evaluating value points from the perspective of business value. The business value dimension can be defined based on business needs.

[0051] For example, multiple business value dimensions can include at least one of the following: scope of influence, frequency of occurrence, reusability, innovation, and business relevance. The scope of influence dimension measures the range of influence of a value point. The frequency of occurrence dimension measures the number of times this type of value point occurs within the evaluation period. The reusability dimension assesses whether the output of a value point can be reused by other workstation agents. The innovation dimension measures the degree of innovation of the value point's output compared to the historical baseline. The business relevance dimension assesses the closeness of the connection between the value point and business objectives (such as core corporate strategic objectives). The establishment of business value dimensions provides specific and actionable scoring criteria for quantifying the value of value points.

[0052] As an example, this application provides the scoring criteria and data sources for the above five business value dimensions, as shown in Table 2.

[0053] Table 2 The purpose of step S120 is to establish a structured value attribution quantification model, transforming qualitative value points into quantitative scores. Step S120 effectively assesses business value by introducing a dimension that directly reflects business contribution, namely the business value dimension. By scoring the same value point from multiple orthogonal business perspectives, it can more comprehensively and objectively reflect its intrinsic value composition, avoiding the one-sidedness of a single-dimensional assessment.

[0054] Step S130: Determine the comprehensive value score of the workstation agent based on the value point type of each value point and the business value score of each business value dimension.

[0055] It should be noted that the type of value point has a certain impact on the value of the value point. The comprehensive value score of the workstation agent is determined by combining both the value point type and business value.

[0056] Step S130 aims to aggregate the multi-dimensional, scattered scores into a single indicator that represents the overall value contribution of the workstation agent, taking into account the differentiated importance of different value types and business dimensions. By constructing a comprehensive scoring model, the multi-dimensional evaluation results are transformed into a final score that is easy to understand and compare, making subsequent ranking and trend analysis possible.

[0057] Furthermore, in one possible implementation, the specific process of determining the comprehensive value score in step S130 can be as follows: First, for each value point, calculate the weighted sum of the business value dimensions based on the business value score of each business value dimension and the weight set for that dimension. Then, adjust the weighted sum according to the intensity coefficient corresponding to the value point type of that value point, and finally obtain the comprehensive value score of the workstation agent.

[0058] As an example, the formula for calculating the comprehensive value score of a workstation agent is as follows: V=α×(w1·S1+ w2·S2+ w3·S3+ w4·S4+ w5·S5) (1) Where V represents the overall value score, α is the intensity coefficient corresponding to the value point type, S1 to S5 correspond to the scores of the five business value dimensions, and w1 to w5 correspond to the weights of the five business value dimensions. The values ​​of S1 to S5 can range from 1 to 10.

[0059] The design of the intensity coefficient reflects the inherent differences in importance among different value types. For example, the "decision value type," which directly affects strategy, may be assigned a higher intensity coefficient than the "time-saving value type." This calculation method of "intensity coefficient × dimensional weighted sum" enables refined attribution and differentiated quantification of value, solving the deficiency of simple scoring models in existing technologies that cannot reflect differences in value importance.

[0060] As an example, this application provides the values ​​of the intensity coefficients for the above five value point types, as shown in Table 3.

[0061] Table 3 In one possible implementation, the sum of the weights of the five business value dimensions is 1. This application does not limit the specific values ​​of the business value dimensions. For example, the weight of each business value dimension can be a fixed value, such as w1 = w2 = w3 = w4 = w5 = 0.2. Alternatively, each business value dimension can be dynamically determined based on the self-awareness context of the workstation agent. The self-awareness context is used to characterize the workstation agent's meta-cognition of its own state, capabilities, and boundaries. For example, when the workstation agent's self-awareness context indicates that its current core task (i.e., strategic goal) is to explore new businesses, the weight of the "innovation" dimension may be dynamically increased; while when its task is to stably support existing businesses, the weights of "reusability" and "frequency of occurrence" may be higher. Utilizing this dynamic weighting mechanism, it is possible to adapt to different lifecycle stages and strategic environments in which the workstation agent is located, achieving intelligent and contextualized evaluation criteria, and further improving the accuracy and rationality of the evaluation results.

[0062] Step S140: Generate and display the visualization component based on the comprehensive value score.

[0063] A visualization component is a reusable module in front-end development that transforms data into a graphical interface, used to intuitively display data relationships, trends, and distributions. This application does not limit the specific type or generation method of the visualization component. For example, a visualization component can be various types of images or tables.

[0064] As an example, visual components can be displayed in the Office administration panel.

[0065] The purpose of step S140 is to transform abstract evaluation data into an intuitive and easy-to-understand graphical interface, enabling managers to grasp the overall value creation landscape of all workstation agents within their jurisdiction at a glance. This step effectively solves the problem of limited visualization dimensions and inability to effectively understand business value by generating specialized value-reflecting visualization components, providing strong data support for management decisions.

[0066] Furthermore, in one possible implementation, the process of generating the visualization component in step S140 includes: firstly, performing a multi-dimensional analysis on the comprehensive value score to obtain the analysis results; and then generating and displaying the visualization component based on the comprehensive value score and the analysis results. The analysis dimensions may include one or more of value distribution, trend prediction, and anomaly attribution. Thus, the visualization component displays not only the comprehensive value score but also the analysis results, greatly enhancing the information content and decision support capabilities of the displayed content.

[0067] For example, through value distribution analysis, we can statistically analyze the proportion, mean, and standard deviation of various value points. We can also summarize upwards by office level (based on the Trust Link authorization tree) and identify the structural characteristics of value creation. For example, the value of this office mainly comes from the cost savings provided by the workstation agent.

[0068] Trend forecasting analysis can predict future value trends. For example, based on historical comprehensive value scores, time series models can be used to predict future trends in comprehensive value scores. These time series models can be pre-trained AI models capable of predicting future trends in the comprehensive value scores of value points based on historical data. Furthermore, the prediction results generated by time series models can be written into strategic objectives within the Ego Context, providing input for "strategic objective achievement probability assessment."

[0069] Anomaly attribution analysis can automatically analyze the reasons for abnormal changes in the overall value score or the business value score. Specifically, in response to the detection of abnormal changes in the overall value score or the business value score that meet preset anomaly conditions, an attribution operation is automatically performed to determine the root cause of the abnormal change.

[0070] Among them, the preset abnormal conditions may be that the comprehensive value score or business value score drops by more than a preset threshold during the evaluation period (for example, a drop of more than 30%), or that the comprehensive value score or business value score shows a downward trend for multiple consecutive periods, or that the comprehensive value score or business value score deviates from the historical average by more than a preset threshold.

[0071] For example, using a sliding window-based statistical analysis method, the comprehensive value score for the current evaluation period is compared with the historical sliding window mean. If the score exceeds N times the standard deviation, it is marked as an anomaly. N is greater than 1.

[0072] Anomaly attribution analysis can solve the problems of difficulty in tracing value fluctuations and reliance on manual problem investigation. It can automate the process from "discovering problems" to "locating causes," effectively reducing management costs and helping to form a complete management loop.

[0073] Please see Figure 2 In one possible implementation, the process of performing the attribution operation includes steps S210-S240.

[0074] Step S210: Determine the location of the abnormal change and the change information.

[0075] The anomaly location can include the specific time period in which the anomaly occurred and the unique identifier of the workstation agent where the anomaly occurred. The change information can include the value point type of the anomaly's value point and the magnitude of the anomaly.

[0076] Step S220: Based on the determined abnormal location, automatically obtain and backtrack the task execution information related to the abnormal change.

[0077] For example, task execution information includes all task execution logs, inference chains, input and output data of the agent at that workstation during the specified time period. Specifically, task execution information may include: all task execution records of the agent at that workstation during the specified time period (including MRG task identifier, execution action sequence, inference chain, output results, quality score, time consumed, and token consumption).

[0078] Step S230: Perform multi-dimensional analysis on the acquired task execution information to determine the root cause.

[0079] The analytical dimensions here can be very rich, including: whether the task structure of the Task Relationship Graph (MRG) has changed, whether the availability of the data source it relies on has changed, whether the underlying AI model version has been switched, whether the prompt configuration has been modified, and whether the Service Level Objective (SLO) constraints have changed. By comparing the differences in these dimensions between task execution information during normal periods and task execution information during abnormal periods, and by using causal reasoning algorithms, the root cause can be efficiently located. Through this structured, multi-dimensional causal analysis, deep-seated problems such as "the insight value of a certain workstation agent dropped sharply this week because its access to the external knowledge base failed" can be automatically discovered. Relevant information on the root cause can be automatically pushed to the corresponding job manager, enabling them to respond and handle the issue promptly.

[0080] In one possible implementation, after determining the root cause, the method may further perform step S240: generating an anomaly report based on the root cause.

[0081] Anomaly reports can include a detailed description of the root cause, an assessment of the impact of the anomaly, and targeted improvement recommendations. Anomaly reports can be automatically pushed to the relevant responsible personnel for timely response and handling.

[0082] Furthermore, the aforementioned anomaly marker can not only be used to mark abnormal changes, but also serve as an interactive trigger. For example, when a user clicks or triggers the anomaly marker on the visual interface, the anomaly report generated in step S240 related to the anomaly can be directly displayed. This seamlessly links anomaly discovery, attribution analysis results, and anomaly reports, greatly improving user experience and anomaly handling efficiency, and achieving a complete closed-loop feedback loop of "value assessment - anomaly discovery - cause identification - strategy adjustment - effect verification".

[0083] In one optional implementation, the visualization components may include one or more of the following: a value heatmap, a value trend chart, an attribution radar chart, a value ranking, and anomaly markers. The value heatmap can be a two-dimensional matrix with a time axis on the horizontal axis and a value point type axis on the vertical axis. The color intensity or fill style at different locations in the value heatmap represents the overall value score density of each value point type over different time periods. Furthermore, the value heatmap may also support filtering by office, workstation, or workstation agent.

[0084] A value trend chart can be used to display the changing trend of the comprehensive value score of each value point type over time. As an example, the value trend chart can be linked with a cost trend chart to form a dual-axis comparison view of "cost-value". Additionally, a value trend forecast line can be overlaid on the value trend chart. The value trend forecast line can be generated based on a time series model.

[0085] Attribution radar charts can be used to visually display the score distribution of a single workstation agent across multiple business value dimensions (such as scope of influence D1, frequency of occurrence D2, reusability D3, innovation D4, and business relevance D5). Attribution radar charts can also be overlaid and compared with the average score or historical score performance of agents in similar workstations.

[0086] Value ranking can be used to display the order of different workstation agents based on their overall value score. For example, the office, workstation, and workstation agent can be ranked according to their overall value score, resulting in value rankings across different dimensions. Furthermore, value ranking can be combined with cost ranking to generate a "value-cost" efficiency ranking. The efficiency score is obtained by subtracting the cost ranking from the value ranking.

[0087] Anomaly indicators can be used to mark abnormal changes in the overall value score on charts (such as value heatmaps).

[0088] Based on the above diverse visualization components, the comprehensive value score can be interpreted from different perspectives, which helps to form a more comprehensive value monitoring dashboard.

[0089] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments. It should be noted that the following embodiments are merely specific implementations of this application and are not intended to limit the scope of protection of this application.

[0090] In a specific application scenario, suppose a workstation agent is deployed at a "sales analysis workstation" in a company's sales department. Its task is to assist in formulating sales strategies. The task execution logs of this workstation agent are continuously collected. In one execution, the log records that the workstation agent's actions were "querying historical customer purchase data, comparing competitor price trends, and generating sales strategy suggestions." The execution result was "generating a report containing three action suggestions," and this report was automatically distributed to the sales director and five regional managers. The method in this application first identifies the value points of the logs. Using keyword recognition and Natural Language Processing (NLP) technology to analyze the log text, preset decision-related keywords such as "strategy" and "suggestion" were matched. Simultaneously, an impact scope identification method was used to track the report's transmission to the management level. Based on these identification signals, the value points of the "decision value" type are determined.

[0091] Next, the value points of the "decision value" type identified above are quantitatively scored. The scoring process is based on multiple preset business value dimensions. For example: (1) Scope of impact (D1): Since the report affected 1 director and 5 managers, a total of 6 people, the score is set at 7 points (out of 10). (2) Frequency of occurrence (D2): This type of task is performed 2-3 times a day, which is a high frequency, and the score is set at 8 points. (3) Reusability (D3): The analysis template of the report can be reused, but the data analyzed each time needs to be updated in real time. The reusability is moderate, and the score is set at 6 points. (4) Innovation (D4): The strategy recommendations generated this time are a certain improvement compared to the past, but they are not disruptive innovations, and the score is set at 5 points. (5) Business relevance (D5): The sales strategy is directly related to the company's core sales performance targets. The relevance is high, and the score is set at 8 points.

[0092] After obtaining the scores for each dimension, the system begins to calculate the comprehensive value score. The calculation is performed according to formula (1). In this embodiment, the intensity coefficient α_D of the "decision value" type in formula (1) is set to 1.5 to reflect its high inherent importance. Assuming the weight of each business value dimension is the default value of 0.2, the comprehensive value score for this value point is: V = 1.5 × (0.2 × 7 + 0.2 × 8 + 0.2 × 6 + 0.2 × 5 + 0.2 × 8) = 1.5 × (1.4 + 1.6 + 1.2 + 1.0 + 1.6) = 1.5 × 6.8 = 10.2 points. This comprehensive value score will be recorded and can be used for subsequent analysis and display.

[0093] In another application scenario, continuous monitoring detected a 40% drop in the "Insight Value" score of a "Customer Service Knowledge Q&A Workstation" agent within the week, from a historical average of 8.5 to 5.1. This significant deviation triggered the anomaly attribution process. First, the anomaly was identified as occurring between Wednesday and Friday, with the "Customer Service Knowledge Q&A Workstation" as the subject of the anomaly, and a value deviation of -40%. Next, the task execution logs of this workstation agent during this period were automatically reviewed. It was found that the frequency of keywords related to insight value, such as "discovery," "anomaly," and "new pattern," in the output results plummeted from an average of 15 times per day to an average of 3 times per day. Simultaneously, the quality score of its internal "deep analysis" subtask also dropped from 8 to 4. To investigate the root cause, further multi-dimensional causal analysis was conducted. Log comparison revealed that on Wednesday, the external knowledge base relied upon by this workstation agent was inaccessible due to system maintenance. Additionally, its underlying inference model version was temporarily rolled back from the more performant version 4 to version 3.5. Based on this, it was determined that the root cause was the combined effect of "external knowledge base unavailability" and "model version downgrade." Finally, an anomaly report was automatically generated, detailing the above analysis process and conclusions, and proposing improvement measures such as "immediately restoring knowledge base access and upgrading the model version back to version 4 as soon as possible." This anomaly report was then pushed to the head of the customer service department.

[0094] Furthermore, it can be integrated with cost accounting systems for joint value-cost analysis. For example, in the aforementioned anomaly attribution scenario, the cost system shows that the token (model resource) consumption of the workstation's AI agent did not decrease; instead, it slightly increased due to retries caused by the model's reduced capabilities. Simultaneously, its value score dropped significantly. This change can be quantified using the efficiency score formula: Value Ranking - Cost Ranking = Efficiency Score.

[0095] In this scenario, its "value-cost" efficiency score dropped from the normal 1.2 to 0.7, representing a 42% deterioration in efficiency. This joint analysis provides managers with a more comprehensive perspective for decision-making.

[0096] Ultimately, all evaluation and analysis results are displayed on a visual dashboard. For example, in the sales department management panel of an e-commerce platform, managers can see the value ranking of each workstation agent, such as the "Sales Strategy Recommendation Agent" ranking first with a comprehensive value score of 58.3. Managers can also click on a specific workstation agent, such as the "Competitor Monitoring Agent," to view its attribution radar chart. If the radar chart shows that its score on the "Innovation" D4 dimension is only 3 points, far below the average level, but as high as 9 points on the "Frequency" D2 dimension, this reveals that although the workstation agent executes frequently, its innovation contribution is low. Based on this, managers can propose improvements to optimize its analysis logic to uncover deeper insights. By combining value ranking and cost ranking, a value-cost efficiency ranking can also be generated to help managers identify those "high-cost, low-value" workstation agent configurations that need optimization, and those "low-cost, high-value" configurations that are worth promoting.

[0097] Please see Figure 3 This application embodiment also provides a workstation intelligent agent task execution value assessment and visualization device based on value point quantification. This device can be a value assessment and visualization system. The device 300 may include: an identification unit 310, a first evaluation unit 320, a second evaluation unit 330, and a visualization unit 340. The identification unit 310 is configured to perform the value point identification step in the aforementioned method, that is, to identify value points in the task execution log of the workstation intelligent agent to obtain at least one value point, wherein the value point type of the at least one value point belongs to a preset set of multiple value point types. The first evaluation unit 320 is configured to perform the quantification scoring step in the aforementioned method, that is, to perform quantification scoring on each identified value point according to multiple business value dimensions to obtain a business value score for each business value dimension. The second evaluation unit 330 is configured to perform the step of determining a comprehensive value score in the aforementioned method, that is, to determine the comprehensive value score of the workstation intelligent agent based on the value point type of each value point and the business value score of each business value dimension. The visualization unit 340 is configured to perform the visualization display step in the aforementioned method, that is, to generate and display a visualization component based on the comprehensive value score. The specific working methods of each unit of the device in this application embodiment have been described in detail in the foregoing method embodiments, and will not be repeated here.

[0098] This application also provides an electronic device, which may be a server, personal computer (PC), portable computer, smartphone, etc. The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor. When executed by the at least one processor, the instructions enable the at least one processor to perform the workstation intelligent agent task execution value assessment and visualization method based on value point quantification as described in any of the foregoing embodiments.

[0099] This application also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the workstation intelligent agent task execution value assessment and visualization method based on value point quantification described in any of the foregoing embodiments. The non-transitory computer-readable storage medium may be a read-only memory (ROM), random access memory (RAM), solid-state drive (SSD), optical disk, or other media capable of storing computer programs.

[0100] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for evaluating and visualizing the task execution value of a workstation intelligent agent based on value point quantification, characterized in that, The method includes: Value point identification is performed on the task execution log of the workstation intelligent agent to obtain at least one value point, and the value point type of the at least one value point belongs to a preset multiple value point types; For each of the identified value points, a quantitative score is performed based on multiple business value dimensions to obtain a business value score for each business value dimension. Based on the value point type of each value point and the business value score of each business value dimension, the comprehensive value score of the workstation intelligent agent is determined. Based on the comprehensive value score, a visualization component is generated and displayed.

2. The method according to claim 1, characterized in that, The process of identifying value points from the task execution logs of the workstation intelligent agent yields at least one value point, including: Keyword extraction is performed on the target text included in the execution log. In response to a successful match between the extracted keywords and the keyword library, at least one value point is determined based on the keyword matching result. The target text includes one or more of the action description text and the output result text. The keyword library includes keywords corresponding to the preset multiple value point types. And / or, Extract temporal behavioral features from the execution events included in the execution log; in response to a successful match between the temporal behavioral features and the behavioral feature template, determine at least one value point based on the feature matching result; the behavioral feature template includes behavioral features of events of the preset multiple value point types. And / or, Determine the scope of influence of the output results included in the execution log, and in response to the scope of influence satisfying a value condition, determine at least one value point based on the scope of influence.

3. The method according to claim 1, characterized in that, The step of generating and displaying a visualization component based on the comprehensive value score includes: Based on the comprehensive value score and analysis results, a visualization component is generated and displayed. The analysis results are obtained by performing a multi-dimensional analysis on the comprehensive value score. The analysis dimensions of the comprehensive value score include one or more of the following: value distribution, trend prediction, and anomaly attribution.

4. The method according to claim 1, characterized in that, The preset multiple value point types include at least one of decision-making value type, insight value type, cost-saving value type, time-saving value type, and accuracy improvement value type.

5. The method according to claim 1, characterized in that, The multiple business value dimensions include at least one of the following: scope of influence, frequency of occurrence, reusability, innovation, and business relevance.

6. The method according to claim 1, characterized in that, The determination of the comprehensive value score of the workstation agent based on the value point type of each value point and the business value score of each business value dimension includes: for each value point, determining the weighted sum of the business value dimensions according to the business value score of each business value dimension and the weight of each business value dimension; and determining the comprehensive value score of the workstation agent according to the intensity coefficient corresponding to the value point type of each value point and the weighted sum of the business value dimensions.

7. The method according to claim 6, characterized in that, The weight of each business value dimension is dynamically determined based on the self-awareness context of the workstation agent, which is used to characterize the workstation agent's meta-cognition of its own state, capabilities, and boundaries.

8. The method according to claim 1, characterized in that, The method further includes: In response to the detection of an abnormal change in the overall value score or the business value score, and the abnormal change meets the abnormal conditions, an attribution operation is performed to determine the root cause of the abnormal change.

9. The method according to claim 8, characterized in that, The attribution operation includes: The abnormal location and change information of the abnormal change are determined. The abnormal location includes the abnormal time period and the identifier of the workstation intelligent agent where the abnormal change occurred. The change information includes the value point type and abnormal magnitude of the abnormal value point. Based on the location of the anomaly, obtain the task execution information related to the anomaly. The task execution information is analyzed from multiple dimensions to determine the root cause; the multi-dimensional analysis includes one or more of the following dimensions: task relationship diagram task structure change dimension, data source availability change dimension, model version switch dimension, prompt word configuration change dimension, and service level target constraint change dimension.

10. The method according to claim 8, characterized in that, The method further includes: An anomaly report is generated based on the aforementioned root cause.

11. The method according to any one of claims 1-10, characterized in that, The visualization components include one or more of the following: a value heatmap, a value trend chart, an attribution radar chart, a value ranking, and an anomaly marker. The value heatmap is a two-dimensional matrix with the time axis as the horizontal axis and the value point type as the vertical axis. The color intensity in the value heatmap represents the density of the comprehensive value score for each value point type in each time period. The value trend chart displays the trend of the comprehensive value score of each value point type over time. The attribution radar chart displays the score distribution of the workstation agent across multiple business value dimensions. The value ranking displays the order in which the workstation agent ranks in terms of the comprehensive value score. The anomaly marker identifies comprehensive value scores that show abnormal changes.

12. The method according to claim 11, characterized in that, The anomaly identifier is also used to display an anomaly report of the comprehensive value score that has undergone abnormal changes when it is triggered.

13. A device for evaluating and visualizing the task execution value of a workstation intelligent agent based on value point quantification, characterized in that, The device includes: The identification unit is configured to identify value points in the task execution log of the workstation intelligent agent to obtain at least one value point, wherein the value point type of the at least one value point belongs to a preset plurality of value point types. The first evaluation unit is configured to quantify and score each of the identified value points according to multiple business value dimensions to obtain a business value score for each business value dimension. The second evaluation unit is configured to determine the comprehensive value score of the workstation agent based on the value point type of each value point and the business value score of each business value dimension. The visualization unit is configured to generate and display visualization components based on the comprehensive value score.

14. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

Images