Computational graph dynamic topology reconstruction method, agent training method and related devices

Abstract

The invention provides a computational graph dynamic topology reconstruction method, an agent training method and a related device, and relates to the field of enterprise-level process automation. The method comprises the following steps that: electronic equipment counts an abnormal index of processing a historical task by a digital employee, notifies a user to take over manually when the abnormal index exceeds a preset alarm threshold value, and synchronously records actual operation steps after the user takes over the task; and then, according to the obtained operation record, updating the current standard operation flow chart, and generating a new standard operation flow chart adaptive to the new rule. In the process, the real business feedback is directly converted into the new flow chart without depending on manual code writing or offline retraining of the model, so that the digital employees can autonomously learn rule changes and quickly recover the execution capability from one-time clicking or one-time takeover of the user, and business interruption caused by flow change is effectively avoided.

Description

Technical Field

[0001] This application relates to the field of enterprise-level process automation, and more specifically, to a method for dynamic topology reconstruction of computational graphs, an agent training method, and related devices. Background Technology

[0002] Large-scale model-based intelligent agents can act as digital employees, simulating human actions. For example, in vertical business scenarios such as auto finance or government approvals, digital employees can interact with the front-end user interface like real customer service representatives, clicking buttons, filling out forms, and calling back-end interfaces to autonomously complete end-to-end business processes such as vehicle release from lien, loan approval, and document submission. Furthermore, these digital employees no longer rely on manually written fixed rules; instead, after training in specific business scenarios, they can dynamically generate action decisions, thus possessing a certain degree of generalization and task transfer capabilities.

[0003] In practice, it has been found that when current digital employees encounter changes in business standard operating procedures (SOPs), traditional models generally cannot effectively utilize non-gradient implicit feedback signals. These non-gradient implicit feedback signals specifically manifest as front-end UI interactions such as clicks, keyboard inputs, and mouse hovers during manual takeover. These signals themselves do not carry gradient information and do not constitute the labels required for standard supervised learning. Therefore, traditional models typically treat user takeover as a simple failure sample and discard it directly, or they must rely on manual post-processing of these interactions before feeding the labeled data into the offline training process. This entire process often takes a considerable amount of time to complete the model update, thus completely failing to achieve real-time adaptive effects. Once the business SOP is adjusted, for example, by temporarily tightening financial risk control rules or changing the sequence of processes on an industrial production line, the success rate of digital employees will plummet and will be difficult to recover autonomously for a considerable period of time, ultimately causing actual business disruption. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides a computation graph dynamic topology reconstruction method, an agent training method, and related apparatus. This method can directly transform real business feedback into new flowcharts without relying on manual coding or offline model retraining. This enables digital employees to autonomously learn rule changes from a single user click or takeover, quickly restore execution capabilities, and effectively avoid business interruptions caused by process changes.

[0005] In a first aspect, this application provides a method for dynamic topology reconstruction of a computational graph, the method further comprising: Statistical data on abnormal indicators in employees' handling of historical tasks; If the abnormal indicator exceeds the set alarm threshold, the user is notified to take over manually, and the operation records after the user takes over are obtained. The current standard operating procedure flowchart is updated based on the operation record to obtain a new standard operating procedure flowchart.

[0006] Secondly, this application provides an agent training method, the method comprising: Obtain the actual operation sequence of the model to be trained performing the target task; Based on the actual operation sequence and the current reference operation sequence of the target task, reward and penalty information is obtained for the actual operation sequence, wherein the reward and penalty information includes a first penalty item for redundant operations in the actual operation sequence; The model to be trained is updated based on the reward and punishment information to obtain an agent model capable of completing the target task.

[0007] Thirdly, this application provides a computational graph dynamic topology reconstruction apparatus, the apparatus further comprising: The model detection module is used to statistically analyze abnormal indicators in the processing of historical tasks by digital employees. The process update module is used to notify the user to take over manually if the abnormal indicator is greater than the set alarm threshold, and to obtain the operation record after the user takes over. The process update module is also used to update the current standard operating procedure flowchart based on the operation record to obtain a new standard operating procedure flowchart.

[0008] Fourthly, this application provides an intelligent agent training device, the device comprising: The forward inference module is used to obtain the actual operation sequence of the model to be trained in performing the target task. The model update module is used to obtain reward and penalty information for the actual operation sequence based on the actual operation sequence and the current reference operation sequence of the target task, wherein the reward and penalty information includes a first penalty item for redundant operations in the actual operation sequence; The model update module is further configured to update the model to be trained based on the reward and punishment information, so as to obtain a digital employee capable of completing the target task.

[0009] Fifthly, this application provides an electronic device, which includes a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the computation graph dynamic topology reconstruction method or the agent training method.

[0010] Compared with the prior art, this application has the following beneficial effects: The computational graph dynamic topology reconstruction method, agent training method, and related apparatus provided in this application involve an electronic device that statistically analyzes abnormal indicators of digital employees handling historical tasks. When these abnormal indicators exceed a preset alarm threshold, the device notifies the user to take over manually and simultaneously records the actual operational steps taken after the user takes over. Subsequently, based on the acquired operation records, the current standard operating procedure flowchart is updated to generate a new standard operating procedure flowchart adapted to the new rules. In this process, there is no need to rely on manually writing code or retraining the model offline; instead, real business feedback is directly transformed into a new flowchart. This allows digital employees to autonomously learn rule changes from a single user click or takeover, quickly restore execution capabilities, and effectively avoid business interruptions caused by process changes. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating the dynamic topology reconstruction method for computational graphs provided in this application embodiment; Figure 2 This is a schematic diagram illustrating the effect of the backup flowchart provided in the embodiments of this application; Figure 3 This is a diagram illustrating the actual effect of the computational graph dynamic topology reconstruction method provided in the embodiments of this application. Figure 4 This is a schematic diagram illustrating the working principle of the operation mask provided in the embodiments of this application; Figure 5 The implementation code for the operation mask provided in the embodiments of this application; Figure 6 This is a comparison diagram of actual video memory usage provided in the embodiments of this application; Figure 7 A flowchart illustrating the agent training method provided in this application embodiment; Figure 8 A comparison diagram of the training effects of the agent training method provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the computational graph dynamic topology reconstruction device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of the intelligent agent training device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application (hereinafter referred to as "the embodiments") clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0014] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0015] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0016] In the description of this application, it should be noted that the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0017] Based on the above statement, as introduced in the background technology, when current digital employees encounter changes in business standard operating procedure flowcharts, traditional models generally cannot effectively utilize non-gradient implicit feedback signals, causing the execution success rate of digital employees to plummet and be difficult to recover autonomously for a considerable period of time, ultimately resulting in actual business interruption.

[0018] Further research revealed that the relevant technologies failed to capture the business modification intentions implied by manual takeover in a timely manner, nor could they transform such intentions into effective learning signals that drive model evolution. Consequently, they could not form a continuous evolutionary capability from manual intervention to flowchart updates, and thus could not achieve on-the-job learning in the production environment.

[0019] It should be noted that the defects in the solutions in the prior art are the result of practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of this application in the following text should be regarded as contributions to this application in the process of invention and creation, and should not be understood as technical content known to those skilled in the art.

[0020] Based on the discovery of the above-mentioned technical problems, this embodiment provides a method for dynamic topology reconstruction of computational graphs, such as... Figure 1 The method includes: S1A is an anomaly indicator for employees' handling of historical tasks.

[0021] S2A: If the abnormal indicator exceeds the set alarm threshold, the user will be notified to take over manually, and the operation records after the user takes over will be obtained.

[0022] S3A updates the current standard operating procedure flowchart based on the operation record to obtain a new standard operating procedure flowchart.

[0023] This can be understood as follows: the server analyzes anomaly metrics in the handling of historical tasks by digital employees. When these metrics exceed a preset alarm threshold, it notifies the user to initiate manual takeover and simultaneously records the actual steps taken after the user takes over. Subsequently, based on the acquired operation records, the current standard operating procedure (SOP) flowchart is updated, generating a new SOP flowchart adapted to the new rules. This process eliminates the need for manually writing code or retraining models offline; instead, it directly transforms real business feedback into a new flowchart. This allows digital employees to autonomously learn rule changes from a single user click or takeover, quickly restoring execution capabilities and effectively preventing business interruptions caused by process changes.

[0024] For the computation graph dynamic topology reconstruction method provided in this embodiment, the electronic device implementing the method can be any device capable of providing sufficient computing power, in addition to a server. Examples include mobile terminals, tablet computers, laptop computers, and desktop computers. The server can be a single server or a group of servers. The server group can be centralized or distributed (e.g., the server can be a distributed system). In some embodiments, the server can be local or remote relative to the user terminal. In some embodiments, the server can be implemented on a cloud platform; as an example only, the cloud platform can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, multi-cloud, or any combination thereof. In some embodiments, the server can be implemented on an electronic device having one or more components.

[0025] To make the solution provided in this embodiment clearer, a server is used as an example below, and in conjunction with... Figure 1 Each step of the method is described in detail. However, it should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical contextual relationships may be reversed in order or implemented simultaneously. Furthermore, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowchart, or remove one or more operations from the flowchart. Figure 1 As shown, the method includes: S1A is an anomaly indicator for employees' handling of historical tasks.

[0026] In practical applications, the server continuously receives task execution logs from the online inference process during operation and analyzes the operation sequence, node status, resource consumption and final results contained in the logs to effectively distinguish between occasional jitter and real anomalies. When a certain type of abnormal indicator continues to exceed the set alarm threshold, the server triggers the subsequent manual takeover and flowchart update mechanism.

[0027] For example, the server recognizes the phenomenon that the automation success rate drops precipitously to 0% the moment the new rule takes effect, causing the old standard operating procedure flowchart to become invalid. Based on this, it immediately triggers the abnormal takeover mechanism to smoothly transfer control to human customer service.

[0028] It should be noted that the standard operating procedure flowchart here refers to a vertical domain business rule knowledge base that is structured in the form of a Directed Acyclic Graph (DAG). For example, in the auto finance mortgage release process, logical constraints such as "login is required before submission" and "materials must be uploaded before verification" are defined. The current execution status includes the description information of the current business node, the binary bitmap representation of the completed business nodes, and contextual information such as system resource availability.

[0029] Therefore, after step S1A, the computation graph dynamic topology reconstruction method further includes: S2A: If the abnormal indicator exceeds the set alarm threshold, the user will be notified to take over manually, and the operation records after the user takes over will be obtained.

[0030] This embodiment can be understood as follows: when an abnormal indicator exceeds a set alarm threshold, the server notifies the user to take over manually and obtains the user's operation records after taking over.

[0031] In practical applications, when the server identifies abnormal indicators that exceed the set alarm threshold, it does not only indicate that a single task has failed, but rather reflects that the model has exhibited a high failure rate in several consecutive rounds of tasks. For example, in the scenario of auto finance mortgage release, if the frequency of the operation of "submitting an application without uploading a video" increases by 5 times within 10 minutes, or the overall task success rate drops precipitously from 98% to 0% within 5 business cases, the server will determine that the current standard operating procedure flowchart can no longer cover the current compliance requirements.

[0032] In this embodiment, the server does not terminate the service at this time, but pushes a lightweight takeover prompt to the current operation interface and simultaneously hands over control to human customer service. After triggering the takeover, the server captures all interactive actions performed by human customer service in real time, including unstructured UI operations such as clicking buttons, filling out forms, and switching pages.

[0033] It should be noted that related technologies lack the ability to directly convert non-gradient heterogeneous UI operations (e.g., Click / Input) into computable tensors. In this embodiment, however, the server allocates a heterogeneous event buffer in memory and uses a hash mapping table... Click events in the interactive interface are mapped to one-hot encoded tensors called by the backend via the corresponding Document Object Model (DOM) tree, forming an operation record after user intervention. This record not only contains the action type and sequence, but also implicitly includes information such as timestamps, context state, and resource overhead. In this way, real human intervention is transformed into a structured operation record.

[0034] Based on the above explanation regarding operation logs, the following will continue with... Figure 1 Step S3A will be explained as follows: S3A updates the current standard operating procedure flowchart based on the operation record to obtain a new standard operating procedure flowchart.

[0035] It should be noted that directly modifying the current standard operating procedure (SOP) flowchart based on user-managed operation records may introduce loops due to misidentification of operation records, or the new SOP flowchart itself may contain loops, leading to deadlocks or infinite loops in subsequent operations by digital employees after deployment. Furthermore, existing technologies lack a security verification mechanism for graph structure changes, failing to ensure that the new SOP flowchart still meets the business execution prerequisite of a directed acyclic graph. More seriously, hot updates without verifying topology validity can cause production incidents in high-reliability scenarios such as finance and government. Therefore, this embodiment provides the following optional implementation methods for step S3A: S3A-1 determines the added or deleted business nodes based on the operation records.

[0036] As an optional implementation, the server can compare and analyze the operation records with the current standard operating procedure flowchart to determine the added or deleted business nodes.

[0037] This embodiment can be understood as follows: by performing a structured comparison between the operation records after the user takes over and the current standard operating procedure flowchart, analyzing the differences in the timing logic of their respective operations, it automatically infers the business nodes that need to be added or deleted.

[0038] In practical applications, when the server executes this step, it can first process the operation record into a structured tensor in the form of a one-hot encoded sequence, where each tensor element corresponds to a standardized action semantic, and all actions are arranged in the order of actual execution time; and then convert each action in the operation record into a node identifier in the standard operation flowchart, thereby achieving semantic alignment between heterogeneous operation signals and flowcharts.

[0039] Based on this, the server does not simply count the new action types that appear in the operation records, but instead matches the action sequence with the longest common subsequence of all legal paths in the current standard operation flowchart to identify the nodes that are not covered by the original diagram (e.g., starting action, interruption jump point or unexpected termination point) as new or deleted business nodes.

[0040] For example, if the only valid path in the current standard operating procedure flowchart is "Login → Upload Materials → Submit Application", but the user's operation record after taking over shows "Login → Upload Video → Submit Application", then the server can determine that "Upload Video" is a new business node that has not yet been defined in the current standard operating procedure flowchart, and its insertion position should be after the "Login" node and before the "Submit Application" node.

[0041] For example, if the operation log frequently shows instances of skipping the "upload materials" step and directly proceeding to the "verification" step, the server can determine that the "upload materials" node was reasonably deleted.

[0042] Furthermore, the server can optionally invoke the large model to perform joint semantic analysis on operation records and the current standard operating procedure flowchart, understanding the differences between operation intent and process logic at the natural language description level, and thus identifying newly added or deleted business nodes. In this way, the large model enhances the adaptability to complex business change scenarios.

[0043] S3A-2: Obtain the backup flowchart of the current standard operating procedure flowchart, and update the backup flowchart according to the added or deleted business nodes to obtain a new backup flowchart.

[0044] S3A-2, if the new backup flowchart has no loop path, then the new backup flowchart will be used as the new standard operation flowchart.

[0045] This embodiment can be understood as a graph update mechanism that builds and verifies sandbox copies on the server side, thereby resolving the risk of system deadlock and business interruption that may be caused when dynamically modifying standard operating procedure flowcharts.

[0046] In this embodiment, the standard operating procedure flowchart must always maintain a directed acyclic graph structure; otherwise, fatal errors such as circular calls, unreachable states, or infinite waiting may occur during execution. Furthermore, new rules reflected in manual takeover operations, such as adding a "video verification" node to the vehicle release process and forcibly inserting it between "login" and "submit," can easily introduce loops due to edge reconnection or improper node insertion if directly applied to the online standard operating procedure flowchart. If the server lacks strict topology security guarantees, the entire service will fall into an unrecoverable deadlock state, which is unacceptable in high-reliability scenarios such as finance and government.

[0047] In view of this, this embodiment also provides a loop detection mechanism for the standard operating procedure flowchart, which is used to detect whether there is a loop in the new backup flowchart in the sandbox environment before step S3A-2.

[0048] For this loop detection mechanism, the server can count the number of waiting nodes for each business node in the new backup flowchart, where the number of waiting nodes for each business node is equal to the number of directed edges pointing to that business node; all business nodes with a waiting node count of 0 are added to the waiting queue as nodes to be accessed; target nodes are sequentially removed from the waiting queue and marked as accessed, and the waiting node count of the successor nodes of the target node is decremented by 1; if there is a successor node with a waiting node count of 0, it is added to the waiting queue as a node to be accessed; the process returns to the step of sequentially removing target nodes from the waiting queue and marking them as accessed, until the waiting queue is empty; if the total number of accessed business nodes is equal to the total number of nodes in the new backup flowchart, then the new backup flowchart is determined to have no loop path. Otherwise, the new backup flowchart is determined to have a loop.

[0049] To facilitate understanding of the loop detection mechanism in this embodiment, a simple example is provided below. However, it should be understood that the standard operating procedure diagram for actual business operations is far more complex than this example: Taking the vehicle release process in auto finance as an example, and assuming the vehicle release process is as follows: Log in to the system → Upload ID card → Upload vehicle registration certificate → Submit release application → Generate release certificate.

[0050] The vehicle release process follows a standard, one-way execution path with no return route, and is the current legal process benchmark. When a user takes over manually, the vehicle release process, executed multiple times, is as follows: Log in to the system → Upload video verification materials → Upload ID card → Upload vehicle registration certificate → Submit release application → Generate release certificate.

[0051] like Figure 2 As shown, after analysis, the server generates a new backup flowchart. It should be noted that this is a simplified version of the actual vehicle release process for illustrative purposes; the actual process is far more complex than the current example.

[0052] The new backup flowchart adds a "Upload Video Verification Materials" node and attempts to insert it after "Login to System" and before "Upload ID Card". However, if two mutually referential edges, "Upload Video Verification Materials → Upload ID Card" and "Upload ID Card → Upload Video Verification Materials", are mistakenly added at the same time, the new backup flowchart will form a loop path containing the two nodes.

[0053] See also Figure 2 The server first counts the number of waiting nodes for each business node, which is the number of directed edges pointing to that node. Among them, the number of waiting nodes for "Login System" is 0, and the number of waiting nodes for "Upload Video Verification Materials" node is pointed to by both "Login System" and "Upload ID Card", so its number of waiting nodes is 2; the number of waiting nodes for the remaining nodes is 1.

[0054] Furthermore, the server only adds nodes with a waiting node count of 0 (such as "Login System") to the queue of nodes to be accessed. After removing "Login System" and marking it as accessed, the waiting node count of its successor nodes (such as "Upload Video Verification Materials") is reduced by 1. However, because the waiting node count of "Upload ID Card" is never 0 (maintained by a loop edge), it cannot be added to the queue, and its subsequent nodes cannot be fully triggered. Therefore, when the queue of nodes to be accessed is empty, the server finds that the total number of accessed nodes is less than the total number of nodes in the new backup flowchart, which means that there may be a loop, causing mutual locking between them, and the waiting node count cannot be reduced to 0. At this time, the update process is stopped to avoid deploying the graph structure with the risk of logical deadlock to the production environment.

[0055] Thus, only after the backup flowchart passes the acyclic verification will the server use a hot-swap of memory pointers to change the standard operating procedure flowchart referenced by the online inference into a new backup flowchart.

[0056] like Figure 3 The experimental results of the auto finance business shown in the figure illustrate the change in success rate as cases (hereinafter collectively referred to as tasks) are continuously processed. Among them, when processing the 10th task, a new rule was forcibly injected: "Video verification materials must be uploaded before a release of mortgage application can be submitted."

[0057] The moment the new rules took effect, the old standard operating procedure flowcharts became invalid, and the automation success rate plummeted to 0%. The "abnormal takeover" mechanism was immediately triggered, smoothly transferring control to human customer service.

[0058] After human intervention, a rapid learning phase begins, with the curve exhibiting a steep upward trend (the middle section of an S-shaped curve). During this period, the server silently records the "video upload" operations of human customer service representatives and converts them into spectral variation signals. A new business model was identified through the accumulation of only about 40 samples.

[0059] After the 60th task was completed, the system entered a stable convergence period. The server completed the verification and hot update of the new standard operating procedure flowchart, and the automation success rate rebounded to over 98%. It should be noted that during these 60 tasks, not only the same type of task was executed, but other types of tasks may also be executed; therefore, the success rate will increase slowly.

[0060] In this way, the business adaptability of "learning by doing" is realized, and the standard operation flowchart change cycle that requires 3-5 days in the traditional code development mode is compressed into an adaptation that can be completed in a very short time, thereby improving business stability.

[0061] The above embodiments describe a dynamic update method for the standard operating procedure flowchart. However, it should be understood that while digital employees are built based on pre-trained agent models, in practice, it has been found that traditional model inference methods require significant GPU memory to support inference when handling digital employee tasks.

[0062] For example, every visible button, input box, or clickable area in the user interface corresponds to a potential operation to be encoded. Before the server uses the agent model for inference, all these operations need to be uniformly converted into numerical codes that the model can recognize and incorporated into the context for computation. As the complexity of business processes increases, the number of interface elements also increases accordingly. For example, a vehicle release page may simultaneously display a login button, multiple material upload portals, verification switches, and submission controls. This results in hundreds or even thousands of candidate operations. At this point, traditional inference methods need to embed representations of all operations, which not only causes an exponential increase in data scale but also significantly increases the server's memory usage and computational load. Consequently, it becomes impossible to deploy the model on relatively low-performance graphics cards.

[0063] The study further revealed that in vehicle release operations, users are not allowed to submit applications if they are not logged in; however, traditional digital employees still include this illegal action in the action space for forward propagation to calculate the full probability. However, this calculation neither produces valid business results nor consumes GPU memory and computing power, requiring a high-performance graphics card to run.

[0064] In view of this, this embodiment further reduces the consumption of computing resources during agent model inference. Therefore, the dynamic topology reconstruction method of the computation graph also includes: S4A obtains the current execution status of the task and observation information about the task environment.

[0065] The execution status includes a description of the current business node.

[0066] It should be understood that if the server directly takes screenshots of the interactive interface and uses a large multimodal model for recognition, it needs to load a visual encoder, a text decoder, and a cross-modal alignment module simultaneously. Each inference requires processing millions of pixel-level image tensors, resulting in a large consumption of GPU memory per observation and a significant increase in inference time. It should be noted that this method not only requires the server to be equipped with a high-end GPU (e.g., A100), but also causes the GPU memory to be continuously filled due to the frequent model calls during the current task, making it impossible to process multiple tasks concurrently.

[0067] Therefore, as an optional implementation, the task environment is the current interactive interface, which includes multiple interactive controls. The server maps the unique identifier of each interactive control to a one-hot encoding and uses this as observation information for the current task environment. It should be noted that the interactive controls here refer to functional units in the interactive interface that have user-triggered capabilities, including but not limited to buttons, input boxes, drop-down options, checkboxes, etc.

[0068] This can be understood as the server parsing the DOM structure of the current interactive interface into structured observation information, taking all interactive controls that may be manipulated by the user as basic units, assigning them unique identifiers, and mapping them to unique tensors of fixed dimensions; the tensor dimension is equal to the total number of predefined interactive controls, and each dimension corresponds to the enabled state of a control.

[0069] In practical applications, the current document object model is traversed to identify all interactive control nodes, and their unique identifiers are used to look up and map them to a preset hash mapping table. The index position is determined in the memory; and it is mapped to a vector in the Compute Unified Device Architecture (CUDA) memory. During the mapping process, for controls that are currently operable, the corresponding dimension is set to 1.0, and the rest are set to 0, forming a sparse one-hot encoded tensor.

[0070] In this way, the dynamically changing structured information in the document object model (e.g., whether a button is enabled, whether an input box is editable, and whether a drop-down menu is expanded) can be transformed into a mathematical representation that the neural network can directly understand. This allows the agent model to accurately perceive which buttons are clickable, which input boxes are fillable, and which options are selectable on the current screen without performing multimodal recognition.

[0071] In addition, the execution status includes descriptions of the current business node and descriptions of completed business nodes.

[0072] Specifically, the server can represent the execution state of an agent performing a target task as a structured binary state vector, which consists of two parts. One part is the descriptive information of the current business node, used to identify the specific stage of the task's progress. For example, in a vehicle release process, a standard operating procedure flowchart describes the entire release process, and based on this flowchart, it clarifies which node is currently in the "login, upload materials, submit application" process. The other part is the descriptive information of completed business nodes, expressed in the form of a binary bitmap, such as a sequence. This indicates that steps 1, 2, and 5 have been completed, while steps 3 and 4 have not yet been executed.

[0073] Thus, this embodiment does not require screenshots of the user interface. Instead, it uses one-hot encoding to compress the data, ensuring that the model can perceive the context of the entire business.

[0074] Based on the description of the current business node in the above embodiments, the dynamic topology reconstruction method of the computation graph further includes: S5A generates an operation mask based on the description information of the current business node and the current standard operating procedure flowchart.

[0075] The operation mask is used to identify legal and illegal operations in the candidate operation space under the current business node.

[0076] In practical applications, the server obtains information about the specific stage of the current task. For example, in the auto finance vehicle release process, the current business node is "upload vehicle registration certificate," which clearly indicates that the process is in the stage where specific documents need to be submitted. The server further retrieves the current standard operating procedure flowchart, which, in the form of a directed graph, clearly defines the execution constraints between each business node. For example, it stipulates that the "upload vehicle registration certificate" node can only be started after "login to the system," and its subsequent steps must connect to "submit release application," and it is not allowed to directly jump to "generate release certificate" or return to "upload ID card."

[0077] Based on the two inputs above, the server performs compliance checks on all candidate operations that may occur in the current task (such as "clicking the upload button, dragging in a file, skipping this step, switching document types," etc.). Operations that meet the preconditions, subsequent paths, and business rules specified in the standard operating procedure flowchart for the current business node are marked as legal operations (set to 1); operations that violate the connection relationships or business logic constraints in the flowchart are marked as illegal operations (set to an infinitesimal minimum value of -1e9).

[0078] Based on the above description of the operation mask, this computation graph dynamic topology reconstruction method also includes: S6A processes the description and observation information of the current business node through an intelligent agent model, and determines the predicted operation to be executed from the candidate operation space.

[0079] In this context, the operation mask prunes the candidate operation space during the inference process of the agent model, and prevents the data corresponding to the candidate operations that are identified as illegal operations from participating in the computation of the agent model.

[0080] This embodiment can be understood as follows: the agent model processes the description and observation information of the current business node to determine the predicted operation to be executed from the candidate operation space. In this process, the operation mask is used to prune the candidate operation space and ensure that the data corresponding to the candidate operation that is marked as an illegal operation does not participate in the operation of the agent model.

[0081] In practical applications, the server does not load all the numerical data corresponding to the candidate operations into the GPU computing unit for complete calculation. Instead, when a bit in the operation mask is 0 (i.e., indicating that the candidate operation is illegal), the server skips the loading and operation of the data corresponding to that bit, so that it does not enter the physical computing unit such as the TensorCore at all. Although the entire operation space is still logically represented as a large-sized tensor structure, only a small part of the legal operation data corresponding to the value of 1 in the mask actually participates in the mathematical operation.

[0082] In this way, the server uses operation masks for sparsity processing, actively identifying and excluding all data blocks corresponding to mask values ​​of 0 before computation, and only processing the valid data segments pointed to by mask values ​​of 1, thereby significantly reducing the number of floating-point operations and memory bandwidth usage.

[0083] Furthermore, based on this operation mask, the agent model in this embodiment does not blindly explore all possible operations when performing tasks, but is always subject to the hard constraints of business rules. Specifically, after the output layer of the agent model and before the Softmax activation function, the original Logits tensor is directly acted upon, cutting off the computation of illegal actions at the physical memory level.

[0084] like Figure 4 As shown, the tensor fusion layer synchronously receives two inputs from the dual-processing: one is the original action space tensor output by the agent model, and the other is a binary state bitmap generated in real time by a finite state machine (FSM). This state bitmap is mapped from the standard operation flowchart of the target task and accurately identifies the legality of each candidate operation in the current state.

[0085] See also Figure 4 The server first processes the received current execution state and observed information using the agent model to obtain the original action space tensor output by the model's output layer. This original action space tensor represents the original probability distribution of multiple candidate operations without any rule filtering, and its numerical form is a set of unnormalized real numbers (i.e., Logits) with a dimension of . ,in The total number of all candidate operations.

[0086] The server further converts this binary state bitmap into an operation mask. Its size is consistent with the original tensor of the action space. In the operation mask, for all operation index positions judged as illegal, the corresponding element is assigned the value of negative infinity (…). The remaining valid positions are assigned a value of zero; by adding elements one by one, this mask tensor is superimposed on the original action space tensor to obtain the pruned action space tensor, which is mathematically expressed as:

[0087] In the formula, Represents the original tensor of the action space. This represents the pruned action space tensor. Based on the pruned action space tensor, the server further applies a Softmax function to the pruned tensor to generate the final action probability distribution:

[0088]

[0089] In the formula, since After the Softmax operation, the corresponding probability approaches zero. Therefore, illegal actions are effectively set to zero in the probability distribution. Consequently, when performing the Softmax operation, it is not necessary to adjust the zeroed probability distribution. Perform exponentiation .

[0090] See Figure 5 The provided implementation code shows that the mask generation and tensor pruning process is not completed on the CPU and then copied to the GPU, but is executed entirely in the GPU memory. The mask operation used is based on the "torch.where" function to implement branchless logic, which can avoid GPU pipeline interruption due to conditional jumps. At the same time, the entire process is encapsulated according to a custom VRAM-aware mask layer, so that the pruning behavior is strictly embedded in the forward propagation link of the neural network.

[0091] See Figure 6 The experimental results shown demonstrate that, in a typical inference scenario with a batch size of 128, this invention successfully reduced GPU memory usage from 24.5GB to 12.2GB—a saving of 50.2%—by excluding data corresponding to candidate operations identified as illegal from the agent model's computation and by implementing pruning operations at the Logit layer. Even when the batch size increased to 256, causing a GPU memory overflow and service crash in the baseline method, this embodiment remained stable (23.8GB). Furthermore, the 12.2GB GPU memory requirement allows this large model task to be successfully deployed on a low-cost NVIDIA Tesla T4 (16GB) graphics card, eliminating the need for an expensive A100 (40GB / 80GB) graphics card.

[0092] The agent model in this embodiment can infer the next operation to be performed based on the current task execution state and observation information of the task environment, and this operation is then executed by a digital employee based on the agent model. It should be understood that this agent model requires targeted training of a training model, which can be a readily available open-source large model. Research has found that current large-model-based digital employees generally only focus on whether the task is ultimately completed, without conducting a refined evaluation of the specific operational paths upon which task completion depends.

[0093] For example, currently, this reward / penalty information is typically used as immediate feedback to the objective function, invoking mature reinforcement learning optimizers such as Proximal Policy Optimization (PPO) to continuously calculate the policy gradient in the asynchronous evolutionary domain, and adjusting the model weight parameters accordingly, thereby updating the model through continuous trial and error. In other words, the current training method gives a positive reward as long as the action sequence eventually achieves the goal, but ignores whether there are a large number of redundant, inefficient, or even illegal operation steps in the sequence.

[0094] For example, in the vehicle release process, a digital employee might repeatedly perform the login operation three times, then attempt to call the identity verification interface that is not yet ready, and then repeatedly refresh the material upload page. Although these actions do not directly cause the task to fail, they all participate in the forward propagation and gradient calculation of the neural network, continuously consuming GPU memory and computation cycles. This indiscriminate full-scale exploration mechanism results in a long model training cycle and requires a large amount of resources.

[0095] Based on the discovery of the above-mentioned technical problems, this embodiment provides a method for training an intelligent agent. For example... Figure 7 As shown, the method includes: S1B: Obtain the actual operation sequence of the model to be trained to perform the target task.

[0096] S2B obtains reward and punishment information for the actual operation sequence based on the actual operation sequence and the current reference operation sequence of the target task.

[0097] The reward and punishment information includes the first penalty for redundant operations in the actual operation sequence.

[0098] S3B updates the model to be trained based on reward and punishment information to obtain an agent model that can complete the target task.

[0099] In this way, by comparing the actual operation sequence with the reference operation sequence, redundant operations are identified, and a first penalty term proportional to the performance cost is generated accordingly, thereby guiding the model not only to pursue the correctness of the result, but also to pursue the efficiency of the path.

[0100] To make the solution provided in this embodiment clearer, a server is used as an example below, and in conjunction with... Figure 7 The steps of this method are explained in detail. Therefore, please refer to [link to previous text]. Figure 7 The method includes: S1B: Obtain the actual operation sequence of the model to be trained to perform the target task.

[0101] The target tasks of the bank can be vehicle release from lien, or tasks in enterprise-level process automation scenarios such as loan approval, document submission, social security application, tax declaration, business registration, housing provident fund withdrawal, real estate registration, medical insurance reimbursement, or government affairs consultation.

[0102] In this embodiment, the model to be trained continuously outputs a series of individual operations based on the current business state vector and external observation inputs. These operations are arranged in chronological order to form a complete behavioral trajectory. This can be understood as the model outputting only one operation at a time, corresponding to a specific business operation executable in the current state, such as calling an Application Programming Interface (API), clicking a page button, or submitting form data. After the model outputs this operation, the digital employee based on the model immediately executes the operation and feeds back a new business state vector and external observation inputs. The model then outputs the next operation based on this new input, and so on, ultimately forming an operation sequence from start to finish.

[0103] Based on the above description of the actual operation sequence, please refer to [link to previous document]. Figure 7 Next, further investigation Figure 7 Step S2B will be explained as follows: S2B obtains reward and punishment information for the actual operation sequence based on the actual operation sequence and the current reference operation sequence of the target task.

[0104] The reward and punishment information includes the first penalty for redundant operations in the actual operation sequence.

[0105] Research has found that if servers simply deduct a fixed score from redundant operations, they cannot distinguish the actual impact of a low-overhead redundancy (e.g., a local cache read) on system resources from a high-overhead redundancy (e.g., a cross-datacenter API write operation). Therefore, the first penalty is proportional to the performance overhead of the redundant operation.

[0106] This embodiment can be understood as follows: by imposing heavier penalties on redundant operations with higher performance overhead, the model is guided to prioritize operation sequences with lower IO costs, lower network latency, and less GPU memory usage among multiple legitimate paths, thereby suppressing resource waste.

[0107] The study also found that simply penalizing redundant operations is insufficient to guarantee correct task completion. If the server does not simultaneously identify and penalize missing operations, the model may oversimplify the process to avoid high overhead, for example, skipping the necessary step of "uploading identity verification," resulting in the task appearing complete but actually being invalid. Traditional methods often treat task success or failure as a binary judgment, failing to distinguish whether task failure is due to excessive operations or insufficient operations. Therefore, this embodiment provides the following optional implementation of step S2B: S2B-1, based on the execution results of the actual operation sequence, obtains the task completion instruction item; S2B-2 compares the actual operation sequence with the reference operation sequence to obtain redundant and missing operations; S2B-3, the first penalty term is obtained based on the performance overhead factor of redundant operations; S2B-4, the second penalty term is obtained based on the performance overhead factor of the missing operation; S2B-5, based on the task completion instructions, the first penalty item, and the second penalty item, obtains reward and punishment information.

[0108] This embodiment can be understood as follows: by constructing a multi-dimensional reward function that takes into account task completion, operational integrity, and system resource consumption, the server, when training the agent model, not only needs to pay attention to whether the task is completed, but also to whether key steps are missed and whether high-cost actions are performed.

[0109] Therefore, the task completion indicator in the reward function represents whether the actual operation sequence has achieved the final result of the target task; the first penalty term quantifies the system overhead caused by redundant operations in the actual operation sequence; and the second penalty term quantifies the risk of business logic disruption caused by missing operations in the actual operation sequence. These three together constitute the overall reward and penalty information. Its mathematical expression is:

[0110] In the formula, Indicates reward and punishment information; This indicates the task completion indicator, with a value of 1 (success) or 0 (failure). Indicates the reference operation sequence, This represents the actual sequence of operations, specifically the tensor-quantized representation of the reference sequence defined in the standard operating procedure flowchart for the target task. This represents the performance overhead factor corresponding to redundant operations and missing operations, Used to calculate the differences between two sequences in terms of operation type and execution order.

[0111] Regarding this expression, it should also be understood that Specifically, it can be the tensor-quantized representation of the actual operation sequence generated by the model to be trained. It can be the tensor-quantized representation of the reference operation sequence defined in the standard operating procedure flowchart corresponding to the target task; Specifically, it could be an edit distance metric based on tensor norm approximation; This is a system overhead weight matrix, where each element is assigned a different value based on the type of operation. For example, write operations (such as database inserts and remote API calls) are given higher weights. Read operations (such as local cache queries and DOM node reads) are given lower weight. This ensures that the second penalty for a missing write operation is significantly higher than that for a missing read operation, and the first penalty for a redundant write operation is also much higher than that for a redundant read operation.

[0112] In practical applications, when the server runs this method, it first compares the actual operation sequence with the reference operation sequence item by item to identify redundant operations (operations that are not in the reference sequence but appear in the actual sequence) and missing operations (operations that exist in the reference sequence but are not executed in the actual sequence). Based on their respective operation types, it looks up the corresponding performance overhead factors in a table and multiplies them by their respective weights to obtain the first penalty term and the second penalty term. Finally, it subtracts the sum of the two penalties from the task completion indicator to obtain the final reward and penalty information. The server uses this reward and penalty information to perform reinforcement learning training on the model to be trained, training the model into an intelligent agent model that meets the requirements.

[0113] See Figure 8 The experimental results shown demonstrate the trend of average reward values ​​for the method of this invention and the benchmark method (Standard PPO without Masking) over 3000 iteration cycles.

[0114] It is easy to see that during the exploration phase (0-500 Epochs), the baseline method requires random exploration in the entire action space (about 5000 API combinations), resulting in extremely slow reward growth (0.1->0.15). However, this embodiment benefits from the operation masking mechanism, which directly blocks about 84% of invalid and illegal actions, enabling it to quickly focus on valid paths and rapidly increase the reward value to 0.6.

[0115] The method reaches a stable convergence state around 2400 epochs (reward value > 0.99), while the baseline method has a reward value of only 0.65 at 3000 epochs and is still in a period of fluctuating upward movement. Calculations show that the convergence speed of this embodiment is improved by about 6.25 times.

[0116] Furthermore, the smoothing of the curve in the original instance demonstrates that the generated reward signal has low noise characteristics, effectively avoiding the training oscillation problem commonly found in traditional reinforcement learning from human feedback (RLHF).

[0117] In this way, the model will neither skip necessary steps in pursuit of low overhead nor introduce high-cost redundancy by blindly covering all steps during the optimization process.

[0118] Based on the above explanation of reward and punishment information, we will continue with... Figure 1 Step S3B will be explained as follows: S3B updates the model to be trained based on reward and punishment information to obtain an agent model that can complete the target task.

[0119] Thus, by comparing the actual operation sequence with the reference operation sequence, redundant operations are identified, and a first penalty term proportional to the performance cost is generated accordingly. This guides the model to not only pursue correct results but also efficient paths. Furthermore, this agent training method simultaneously introduces a second penalty term for missing operations in the reference operation sequence into the reward and punishment information, assigning it differentiated based on the performance cost factor of the missing operations, preventing the model from skipping critical steps to reduce overhead.

[0120] Based on the same inventive concept as the computational graph dynamic topology reconstruction method provided in this embodiment, this embodiment also provides a computational graph dynamic topology reconstruction apparatus. This apparatus includes at least one software functional module that can be stored in a memory or embedded in an electronic device. A processor in the electronic device executes the executable module stored in the memory. For example, the software functional module and computer program included in this apparatus. Please refer to... Figure 9 Functionally, the device may include: Model detection module 11A is used to collect statistics on abnormal indicators of digital employees' handling of historical tasks; The process update module 12A is used to notify the user to take over manually if the abnormal indicator is greater than the set alarm threshold, and to obtain the operation record after the user takes over. The process update module 12A is also used to update the current standard operating procedure flowchart based on the operation record to obtain a new standard operating procedure flowchart.

[0121] In this embodiment, the model detection module 11A is used to implement Figure 1 In step S1A, the process update module 12A is used to implement... Figure 1 Therefore, for detailed descriptions of the above modules, please refer to the specific implementation methods of the corresponding steps.

[0122] Optionally, the process update module 12A updates the current standard operating procedure flowchart based on the operation record to obtain a new standard operating procedure flowchart in the following ways: Based on the operation logs, determine the new or deleted business nodes; Obtain the backup flowchart of the current standard operating procedure flowchart, and update the backup flowchart according to the added or deleted business nodes to obtain the new backup flowchart; If the new backup flowchart has no loop path, then the new backup flowchart will be used as the new standard operating procedure flowchart.

[0123] Optionally, the process update module 12A determines the method for adding or deleting business nodes based on the operation records, including: By comparing and analyzing the operation records with the current standard operating procedure flowchart, the business nodes that have been added or deleted can be identified.

[0124] Optionally, before adopting the new backup flowchart as the new standard operating procedure flowchart, the process update module 12A is also used for: The number of waiting nodes for each business node in the new backup process diagram is calculated, where the number of waiting nodes for each business node is equal to the number of directed edges pointing to that business node. All business nodes with a waiting node count of 0 are added to the waiting queue as nodes to be accessed. The target node is retrieved from the queue of nodes to be visited in sequence and marked as visited. The number of waiting nodes for the successor node of the target node is decremented by 1. If there is a successor node whose number of waiting nodes is reduced to 0, it is added to the queue of nodes to be visited. Return to the steps of sequentially retrieving target nodes from the queue to be visited and marking them as visited, until the queue to be visited is empty; If the total number of accessed business nodes is equal to the total number of nodes in the new backup flowchart, then the new backup flowchart is determined to have no loop paths.

[0125] Optionally, the digital employee is built based on an intelligent agent model. The device also includes a user interaction module for obtaining the execution status of the current task and observation information of the task environment, wherein the execution status includes descriptive information of the current business node. Based on the description information of the current business node and the current standard operating procedure flowchart, an operation mask is generated. The operation mask is used to identify the legal and illegal operations in the candidate operation space under the current business node. The agent model processes the description and observation information of the current business node to determine the predicted operation to be executed from the candidate operation space. The operation mask prunes the candidate operation space during the inference process of the agent model and prevents the data corresponding to the candidate operation marked as illegal from participating in the calculation of the agent model.

[0126] Based on the same inventive concept as the agent training method provided in this embodiment, this embodiment also provides an agent training device, which includes at least one software functional module that can be stored in a memory or embedded in an electronic device. The processor in the electronic device is used to execute the executable module stored in the memory. For example, the software functional module and computer program included in this device. Please refer to... Figure 10 Functionally, the device may include: The forward inference module 11B is used to obtain the actual operation sequence of the model to be trained performing the target task. The model update module 12B is used to obtain reward and penalty information for the actual operation sequence based on the actual operation sequence and the current reference operation sequence of the target task. The reward and penalty information includes a first penalty term for redundant operations in the actual operation sequence. The model update module 12B is also used to update the model to be trained based on reward and punishment information in order to obtain an agent model that can complete the target task.

[0127] In this embodiment, the forward inference module 11B is used to implement Figure 7 In step S1B, the model update module 12B is used to implement... Figure 7 Therefore, for a detailed description of each of the above modules, please refer to the specific implementation of the corresponding steps.

[0128] Optionally, the first penalty term is proportional to the performance overhead of redundant operations.

[0129] Optionally, the reward and punishment information may also include task completion instructions and a second penalty for missing operations in the reference operation sequence; The model update module 12B obtains reward and penalty information for the actual operation sequence based on the actual operation sequence and the reference operation sequence required to complete the target task in the following ways: Based on the execution results of the actual operation sequence, task completion indicators are obtained; The actual operation sequence is compared with the reference operation sequence to obtain redundant and missing operations. The first penalty term is obtained based on the performance overhead factor of redundant operations; The second penalty term is derived based on the performance overhead factor of the missing operation; Based on the task completion instructions, the first penalty, and the second penalty, you will receive reward and punishment information.

[0130] Optionally, the relationship between the task completion indicator, the first penalty, the second penalty, and the reward / punishment information is as follows:

[0131] In the formula, Indicates reward and punishment information. Indicates task completion indicator. Indicates the reference operation sequence, Indicates the actual operation sequence. This represents the performance overhead factor corresponding to redundant operations and missing operations, respectively.

[0132] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0133] It should also be understood that if the above embodiments are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0134] Therefore, this embodiment also provides a storage medium, which is a computer-readable storage medium. This storage medium stores a computer program, which, when executed by a processor, implements the agent training method or the computation graph dynamic topology reconstruction method provided in this embodiment. The storage medium can be any medium capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0135] This embodiment provides an electronic device for implementing an agent training method or a computation graph dynamic topology reconstruction method. For example... Figure 11 As shown, the electronic device may include a processor 22 and a memory 21. The memory 21 stores a computer program, and the processor reads and executes the computer program corresponding to the above embodiments in the memory 21 to implement the agent training method or the computational graph dynamic topology reconstruction method provided in this embodiment.

[0136] See also Figure 11 The electronic device also includes a communication unit 23. The memory 21, processor 22 and communication unit 23 are electrically connected to each other directly or indirectly through system bus 24 to realize data transmission or interaction.

[0137] The memory 21 can be an information recording device based on any electronic, magnetic, optical, or other physical principles, used to record execution instructions, data, etc. In some embodiments, the memory 21 can be, but is not limited to, volatile memory, non-volatile memory, memory drive, etc.

[0138] In some embodiments, the volatile memory may be random access memory (RAM); in some embodiments, the non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, etc.; in some embodiments, the storage drive may be a disk drive, solid-state drive, any type of storage disk (such as optical disc, DVD, etc.), or similar storage media, or a combination thereof.

[0139] The communication unit 23 is used to send and receive data over a network. In some embodiments, the network may include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof. In some embodiments, the network may include one or more network access points. For example, the network may include wired or wireless network access points, such as base stations and / or network switching nodes, through which one or more components of the service request processing system can connect to the network to exchange data and / or information.

[0140] The processor 22 may be an integrated circuit chip with signal processing capabilities, and may include one or more processing cores (e.g., a single-core processor or a multi-core processor). By way of example only, the processor described above may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), an Application Specific Instruction-set Processor (ASIP), a Graphics Processing Unit (GPU), a Physics Processing Unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller unit, a Reduced Instruction Set Computing (RISC) computer, or a microprocessor, or any combination thereof.

[0141] Understandable. Figure 11The structure shown is for illustrative purposes only. Electronic devices may also have more advanced features. Figure 11 Showing more or fewer components, or having with Figure 11 The different configurations shown. Figure 11 The components shown can be implemented using hardware, software, or a combination thereof.

[0142] It should be understood that the apparatus and methods disclosed in the above embodiments can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0143] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for dynamic topology reconstruction of a computational graph, characterized in that, The method further includes: Statistical data on abnormal indicators in employees' handling of historical tasks; If the abnormal indicator exceeds the set alarm threshold, the user is notified to take over manually, and the operation records after the user takes over are obtained. The current standard operating procedure flowchart is updated based on the operation record to obtain a new standard operating procedure flowchart.

2. The computational graph dynamic topology reconstruction method according to claim 1, characterized in that, The current standard operating procedure flowchart is updated based on the operation records to obtain a new standard operating procedure flowchart, including: Based on the operation records, determine the new or deleted business nodes; Obtain a backup flowchart of the current standard operation flowchart, and update the backup flowchart according to the added or deleted business nodes to obtain a new backup flowchart; If the new backup flowchart has no loop path, then the new backup flowchart will be used as the new standard operation flowchart.

3. The computational graph dynamic topology reconstruction method according to claim 2, characterized in that, Based on the operation records, determine the added or deleted business nodes, including: The operation records are compared and analyzed with the current standard operating procedure flowchart to determine the newly added or deleted business nodes.

4. The computational graph dynamic topology reconstruction method according to claim 2, characterized in that, Before using the new backup flowchart as the new standard operating procedure flowchart, the method further includes: The number of waiting nodes for each business node in the new backup flowchart is calculated, wherein the number of waiting nodes for each business node is equal to the number of directed edges pointing to that business node. All business nodes with a waiting node count of 0 are added to the waiting queue as nodes to be accessed. Target nodes are sequentially retrieved from the queue to be accessed and marked as accessed. The number of waiting nodes for the successor nodes of the target node is decremented by 1. If there is a successor node whose number of waiting nodes is reduced to 0, it is added to the queue to be accessed as a node to be accessed. Return to the step of sequentially retrieving target nodes from the queue to be accessed and marking them as accessed, until the queue to be accessed is empty; If the total number of accessed business nodes is equal to the total number of nodes in the new backup flowchart, then the new backup flowchart is determined to have no loop paths.

5. The computational graph dynamic topology reconstruction method according to claim 1, characterized in that, The digital employee is constructed based on an intelligent agent model, and the method further includes: Obtain the execution status of the current task and the observation information of the task environment, wherein the execution status includes the description information of the current business node; Based on the description information of the current business node and the current standard operation flowchart, an operation mask is generated, wherein the operation mask is used to identify legal and illegal operations in the candidate operation space under the current business node; The agent model processes the description information of the current business node and the observation information to determine the predicted operation to be executed from the candidate operation space. The operation mask prunes the candidate operation space during the inference process of the agent model and prevents the data corresponding to the candidate operations marked as illegal operations from participating in the operation of the agent model.

6. A method for training an intelligent agent, characterized in that, The method includes: Obtain the actual operation sequence of the model to be trained performing the target task; Based on the actual operation sequence and the current reference operation sequence of the target task, reward and penalty information is obtained for the actual operation sequence, wherein the reward and penalty information includes a first penalty item for redundant operations in the actual operation sequence; The model to be trained is updated based on the reward and punishment information to obtain an agent model capable of completing the target task.

7. The agent training method according to claim 6, characterized in that, The first penalty item is proportional to the performance overhead of the redundant operation, and the reward and penalty information also includes a task completion indication item and a second penalty item for missing operations in the reference operation sequence; Based on the actual operation sequence and the reference operation sequence required to complete the target task, reward and penalty information for the actual operation sequence is obtained, including: Based on the execution results of the actual operation sequence, the task completion indication item is obtained; The actual operation sequence is compared with the reference operation sequence to obtain the redundant operations and missing operations. The first penalty term is obtained based on the performance overhead factor of the redundant operation; The second penalty term is obtained based on the performance overhead factor of the missing operation; The reward and punishment information is obtained based on the task completion instruction, the first penalty, and the second penalty. The relationship between the task completion instruction, the first penalty item, the second penalty item, and the reward / punishment information is as follows: In the formula, This indicates the reward and punishment information. This indicates the task completion indicator. This represents the reference operation sequence. This represents the actual operation sequence. This represents the performance overhead factor corresponding to the redundant operation and the missing operation, respectively. Used to calculate the differences between two sequences in terms of operation type and execution order.

8. A computational graph dynamic topology reconstruction device, characterized in that, The device further includes: The model detection module is used to statistically analyze abnormal indicators in the processing of historical tasks by digital employees. The process update module is used to notify the user to take over manually if the abnormal indicator is greater than the set alarm threshold, and to obtain the operation record after the user takes over. The process update module is also used to update the current standard operating procedure flowchart based on the operation record to obtain a new standard operating procedure flowchart.

9. A smart agent training device, characterized in that, The device includes: The forward inference module is used to obtain the actual operation sequence of the model to be trained in performing the target task. The model update module is used to obtain reward and penalty information for the actual operation sequence based on the actual operation sequence and the current reference operation sequence of the target task, wherein the reward and penalty information includes a first penalty item for redundant operations in the actual operation sequence; The model update module is further configured to update the model to be trained based on the reward and punishment information, so as to obtain a digital employee capable of completing the target task.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing a computer program, which, when executed by the processor, implements the computational graph dynamic topology reconstruction method according to any one of claims 1-5 or the agent training method according to any one of claims 6-7.

Images