Intelligent agent strategy generation and online optimization method based on dynamic scene perception

Abstract

The invention provides an agent strategy generation and online optimization method based on dynamic scene perception, and the method comprises the steps: obtaining a scene demand description of a user, and carrying out the analysis of the scene demand description, so as to generate a target task sequence which can be executed by an agent disposed in a target scene; dynamically sensing the current environment characteristics of the target scene to generate a dynamic semantic topology network and generate a dynamic scene graph according to the dynamic semantic topology network; on the basis of the dynamic scene graph, hierarchical modeling of the incidence relation is carried out on the behavior space corresponding to the intelligent agent, and behavior semantic features containing scene perception are generated; the behavior semantic features containing scene perception are mapped to an intelligent agent strategy representation space, and a behavior feature strategy for controlling an intelligent agent to execute a target task sequence is obtained; and according to the determined scene value representation, decomposing the behavior feature strategy to obtain an advantage estimation value adapted to the scene so as to carry out online optimization on the behavior feature strategy. According to the invention, the intelligent agent strategy can accurately adapt to the requirements in the business process of an enterprise.

Description

Technical Field

[0001] This application relates to the technical field of intelligent data processing, and in particular to an intelligent agent policy generation and online optimization method based on dynamic scenario perception. Background Art

[0002] In application scenarios such as enterprise business process automation and intelligent decision-making, intelligent agents need to generate reasonable policies according to complex and changing business scenarios and dynamically optimize them. For example, in processes such as financial reimbursement review and project management, intelligent agents need to accurately understand business requirements, perceive scenario changes, and formulate efficient policies to complete tasks.

[0003] In the prior art, intelligent agent policy generation and optimization methods usually adopt fixed rules or are implemented based on traditional machine learning models. For policy generation, generally, predefined rule templates are used to match user requirements and convert them into tasks that can be executed by intelligent agents. At the same time, static environment feature extraction methods are used to construct a scenario model, and simple linear transformations are used based on this model to generate intelligent agent policies. In terms of policy optimization, it often relies on fixed reward mechanisms and static advantage function estimation, and the policy is updated as a whole regularly. Although this technical logic can meet the needs of some basic scenarios, it is difficult to adapt to complex and dynamic business environments.

[0004] However, the above prior art has obvious defects. In the policy generation link, predefined rule templates and static environment modeling cannot effectively handle the diversity of user requirements and the dynamic changes of the scenario environment. For example, in the reimbursement process, when new policy terms or temporary approval requirements appear, the task sequence and policy cannot be adjusted in time. Simple linear transformations are difficult to capture complex correlation relationships in the behavior space, resulting in the generated policies lacking flexibility and adaptability. In terms of policy optimization, fixed reward mechanisms and static advantage function estimation cannot reflect the impact of scenario changes on the policy value in real time. The method of regularly updating the policy as a whole is not only inefficient but also may lead to the disconnection between the intelligent agent's decision-making and the actual scenario due to untimely updates, unable to meet the needs of the efficient operation of enterprise business. Summary of the Invention The purpose of this application is to propose an intelligent agent policy generation and online optimization method based on dynamic scenario perception to solve or alleviate the technical problems existing in the prior art.

[0005] The technical solutions provided by the embodiments of this application are as follows: An intelligent agent policy generation and online optimization method based on dynamic scenario perception, which includes: Step 1, obtain the description of the user's scenario requirements and parse it to generate a target task sequence that can be executed by an intelligent agent deployed in the target scenario; Step 2: Dynamically perceive the current environmental characteristics of the target scenario to generate a dynamic semantic topology network and generate a dynamic scene graph accordingly; Step 3: Based on the dynamic scene graph, hierarchically model the association relationships in the behavior space corresponding to the agent to generate behavior semantic features with scene perception; Step 4: Map the behavior semantic features with scene perception to the agent policy representation space to obtain a behavior feature policy for controlling the agent to execute the target task sequence; Step 5: According to the determined scene value representation, decompose the behavior feature policy to obtain an advantage estimate value adapted to the scene, so as to perform online optimization on the behavior feature policy.

[0006] In this application, through the closed-loop logic of "dynamic requirement analysis → real-time environment modeling → hierarchical behavior association → scene-driven policy generation → online dynamic optimization", the core defects of "fixed rules, static modeling, and lagging optimization" in the prior art are fundamentally solved, enabling the agent policy generation and optimization process to accurately adapt to the demand diversity, scene dynamics, and decision complexity in the enterprise business process. Brief Description of the Drawings

[0007] Figure 1 It is a schematic flowchart of a method for generating and online optimizing an agent policy based on dynamic scene perception according to an embodiment of this application. Detailed Embodiment

[0008] To implement the solution of this application, in the following embodiments, the componentization idea is introduced to explain the technical implementation. The reasons for introducing the componentization idea are mainly considered in the following three aspects.

[0009] First, in order to disassemble the complex agent policy generation process into independent functional components (such as the front-end human-computer interaction component, scene graph generation component), each component follows the single responsibility principle and is only responsible for the logic processing of specific links (such as the "requirement collection component" focuses on obtaining user input, and the "semantic extraction component" focuses on generating behavior semantic features), realizing functional decoupling and hierarchical logical isolation, so that the system architecture conforms to the principle of "high cohesion, low coupling", which not only reduces the overall development and maintenance complexity, but also ensures that the collaborative logic between components is clear and traceable through standardized input and output interfaces, so as to highly adapt to the modular implementation of multi-level business processes such as requirement analysis, scene modeling, and policy optimization in agent policy generation.

[0010] In addition, through standardized interfaces (such as the gRPC communication protocol and JSON data format) and containerized deployment (such as Docker + Kubernetes), the componentized architecture can achieve elastic expansion and efficient operation and maintenance of the system. For example, compute-intensive components (such as the "policy optimization component") can be independently configured with GPU resources, and high-concurrency components (such as the "front-end human-computer interaction component") can be deployed with multiple replicas to handle traffic fluctuations; independent testing and fault location of each component (such as separately verifying the topology generation logic of the "dynamic scene graph generator") reduce the system debugging cost, while the unified component lifecycle management mechanism (start, monitor, upgrade) meets the requirements of enterprise-level applications for stability and maintainability, forming an engineering closed-loop from development to deployment.

[0011] Finally, componentized design can shield underlying technology differences and support the flexible integration of heterogeneous algorithms and frameworks (such as the "semantic role labeling model" can be implemented based on BERT or LSTM, and the "topological neighborhood aggregation algorithm" can choose GIN or GraphSAGE). By ensuring interface consistency, technology iteration does not affect the overall architecture. At the same time, the component architecture is naturally suitable for multi-agent collaboration scenarios. Each agent can be mapped to an independent component (such as the financial policy agent and the process simulation agent). The collaboration rules are defined through the "cross-agent interaction weight matrix" and support dynamic policy optimization (such as the "fine-tuning component" online updates the agent's policy parameters). In addition, by reorganizing components (replacing industry-specific components and retaining core logic components), different domain requirements can be quickly adapted (such as switching from an enterprise digital assistant to an intelligent warehousing scenario), improving the system's adaptability to diverse business scenarios.

[0012] Figure 1 This is a schematic flowchart of a method for generating and online optimizing an agent policy based on dynamic scene perception according to an embodiment of the present application. As Figure 1 shown, it includes: Step 1: Obtain the user's scene requirement description based on the front-end human-computer interaction component configured in the target scene, and parse it to generate a target task sequence that can be executed by the agent deployed in the target scene; Step 2: Based on the configured scene graph generation component, dynamically perceive the current environmental characteristics of the target scene to generate a dynamic semantic topology network and generate a dynamic scene graph accordingly; Step 3: Based on the configured hierarchical modeling component, based on the dynamic scene graph, hierarchically model the association relationships of the behavior space corresponding to the agent to generate behavior semantic features with scene perception; Step 4: Based on the configured semantic feature mapping component, map the behavior semantic features with scene perception to the agent policy representation space to obtain a behavior feature policy for controlling the agent to execute the target task sequence; Step 5: Based on the configured policy optimization component, according to the determined scene value representation, decompose the behavioral feature policy to obtain the advantage estimation value adapted to the scene, so as to optimize the behavioral feature policy online.

[0013] In this application, by dynamically parsing the scene requirement description to generate the target task sequence, the limitation of fixed rules is avoided. For example, when the user puts forward the "reimbursement requirement with an emergency approval channel", the system can parse and generate the task sequence including "quick material verification → urgent review" in real time, rather than relying on the pre-stored template, directly solving the problem of "unable to adjust the task sequence in time". In addition, by dynamically perceiving the current environmental characteristics to generate the dynamic semantic topology network and scene graph, the real-time update of environmental information is realized. For example, in the reimbursement scene, when a new policy is issued, the system can capture environmental characteristics such as "new attachment requirements" in real time and update the node associations in the scene graph (such as adding a node of "attachment compliance check"), solving the defect of "static modeling unable to reflect the dynamic changes of the scene". Moreover, in this application, based on the dynamic scene graph, the behavior space is hierarchically modeled. For example, the behaviors of "material submission → financial review → supervisor approval" in the reimbursement process are decomposed according to the business levels (submission level, review level, decision-making level), and the dependence strength of each behavior is modeled through the topological relationship (such as the strong dependence of "financial review" on "material integrity"). This modeling method breaks through the single-dimensional limitation of linear transformation, enabling the generated behavioral semantic features to accurately reflect the timing and logical association of behaviors such as "filling out the form → submitting for review", solving the problem of "insufficient policy flexibility". Finally, this application maps the behavioral semantic features with scene perception to the policy representation space. For example, in the scene of "peak end-of-month settlement", according to the feature of "surge in business volume" in the dynamic scene graph, a policy of "prioritize processing high-amount reimbursement forms" is generated. This process directly uses the dynamic scene information to guide the policy generation, avoiding the defect of "disconnection between policy generation and scene changes" in the prior art and improving the adaptability of the policy to the real-time scene. Moreover, this application dynamically decomposes the policy through the scene value representation, generates the advantage estimation value adapted to the scene and optimizes it online. For example, when there is ambiguity in the rules during the reimbursement policy transition period, the system dynamically adjusts the policy parameters based on the real-time scene value (such as "fluctuation in the passing rate of different review policies"), rather than waiting for regular updates. This mechanism solves the defects of "fixed rewards unable to reflect the changes in scene value" and "low efficiency of regular updates", realizing the real-time alignment of the policy with the dynamic scene.

[0014] Optionally, the front-end human-computer interaction component includes: a requirement acquisition component, a semantic role annotation model, and a sequence generation component. The specific content of step 1 includes: Step 11: Based on the requirement acquisition component, obtain the scene requirement description of the user; Step 12: Through the semantic role labeling model, perform entity extraction processing on the scenario requirement description to obtain the task target entity set; Step 13: Based on the sequence generation component, generate the target task sequence that can be executed by the agent according to the task target entity set.

[0015] Preferably, in Step 11, the requirement acquisition component is built with the Vue.js front-end framework + Python Flask back-end service, integrating the speech recognition API (such as based on the DeepSpeech model) and the third-party capsule message interface to capture user input in real time through the WebSocket long connection. In specific implementation, the front-end renders the interactive interface (such as text input box, speech button) through the Vue component library, and the back-end uses the Flask routing to receive requests. After being reverse-proxied by Nginx, the text input (such as "Apply for business trip reimbursement in 2025") or the speech-to-text result is converted into a JSON format data stream, and finally a message queue (such as RabbitMQ) is formed as the scenario requirement description.

[0016] Therefore, the specific technical implementation of the above Step 11 has the following technical advantages: (1) Adopting the combination of the Vue.js front-end framework and the Python Flask back-end service to form an efficient front-back end separation architecture. The component-based design of Vue.js can quickly build an interactive interface, and its reactive data binding feature synchronizes user input in real time (such as the content of the text box and the status of the speech button). The lightweight back-end of Flask quickly defines API interfaces through the routing mechanism and seamlessly connects with the NLP libraries in the Python ecosystem. This architecture supports independent front-back end deployment and iteration: the front-end can improve the access speed through Nginx caching, and the back-end can dynamically expand service instances according to traffic, significantly reducing the development and maintenance costs in enterprise-level applications.

[0017] (2) Integrating the speech recognition API based on the Deep Speech model to provide the natural language input ability for agent interaction. The end-to-end deep learning model of Deep Speech supports customization of Chinese dialects and enterprise terms (such as "business trip reimbursement", "rank approval"), with an identification accuracy rate exceeding 95%, solving the efficiency bottleneck of text input in the mobile office scenario. In addition, as an open-source model, enterprises can deploy it on the private cloud to avoid uploading speech data to the public network, meet the compliance requirements of sensitive businesses such as finance and human resources, and strengthen data security while ensuring the interaction experience.

[0018] (3) Bidirectional real-time communication between the front end and the back end is achieved through WebSocket long connections, significantly improving the interaction fluency. When an employee enters "Apply for 2025", the back end can immediately return policy association prompts (such as "The 2025 travel policy has been updated") through WebSocket, forming an interactive guidance process. This streaming processing mode not only reduces latency but also supports the back end to actively push system announcements, format reminders and other information. For example, when it detects that the voice input is ambiguous, it will immediately feedback "Please clarify the reimbursement type", making the interaction of the intelligent agent closer to the natural experience of human services.

[0019] (4) The combination of Nginx reverse proxy and RabbitMQ message queue constructs a highly available back-end processing architecture. Nginx distributes text / voice requests to different servers through load balancing (such as forwarding voice recognition requests to GPU nodes), and improves performance and security through caching and WAF protection; RabbitMQ temporarily stores input data streams in high-concurrency scenarios (such as application requests during the peak reimbursement period at the end of the month), and avoids service crashes through message persistence and asynchronous consumption mechanisms. The two work together to achieve "peak shaving and valley filling", ensuring that the system still maintains 99.9% availability when tens of thousands of enterprise employees use it simultaneously, and supports modular decoupling and function expansion.

[0020] Preferably, in step 12, the semantic role labeling model is built based on the RoBERTa-large model of the Hugging Face Transformers library, combined with the enterprise-customized Neo4j knowledge graph (storing entity relationships of policies such as human resources and finance) and the spaCy syntactic analyzer. For this purpose, entity extraction processing is performed on the scenario requirement description, and the task objective entity set obtained is: the core arguments (such as the subject, action, and object) in the scenario requirement description are identified through the masked language model of the RoBERTa model, and then the spaCy syntactic analyzer is used to perform dependency syntactic analysis on the scenario requirement description to extract semantic relationships (such as the subject-predicate-object structure in "The reimbursement process requires approval by the department manager"). Finally, based on the extracted semantic relationships, the core arguments are mapped to the knowledge graph nodes through entity linking (such as associating "department manager approval" with the approval process node in the OA system), generating a structured task objective entity set (such as {subject: reimbursement process, action: apply, object: 2025 travel expenses}).

[0021] For this reason, the specific technical implementation of the above step 12 has the following technical advantages: (1)Select the RoBERTa-large model from the Hugging Face Transformers library. Because it has learned the deep semantic representations of a large amount of Chinese corpus through the masked language model (MLM) during the pre-training stage, it can accurately capture the core arguments in the description of the scenario requirements. For example, in the sentence "The reimbursement process requires approval by the department manager", the model can identify key arguments such as "reimbursement process" (subject), "requires" (action), and "approval by the department manager" (object) through context association. Compared with the traditional bag-of-words model, its multi-head self-attention mechanism based on the Transformer architecture can better handle long-distance dependencies (such as the time-business association in "Query the travel reimbursement limit in 2025"). After fine-tuning with enterprise scenarios, the recognition accuracy of the model for terms in the fields of human resources, finance, etc. can reach over 98%, laying a foundation for subsequent semantic parsing.

[0022] (2)Combine with the enterprise-customized Neo4j knowledge graph to map the arguments extracted by RoBERTa to structured business entities, and solve the semantic ambiguity problem of the general model in specific domains. For example, the word "approval" in the knowledge graph can be associated with specific process nodes such as "approval by the department manager" and "financial review". The context of the argument is clarified through the relationship edges in the graph structure (such as "belongs to - process node", "precedes - approval link"). This entity linking technology based on the knowledge graph enables the intelligent agent to understand that "travel reimbursement in 2025" corresponds to a specific policy version (such as "2025-V1.0") and applicable population (such as "employees above the supervisor level"). The generated task target entity set not only contains the surface semantics but also incorporates the enterprise business logic, improving the accuracy and compliance of the intelligent agent's response.

[0023] (3)Use the spaCy syntactic analyzer for dependency syntactic analysis to extract the deep syntactic structure of the sentence, making up for the deficiency of the pre-trained model in semantic relation extraction. For example, in the sentence "The travel application needs to be completed before submitting", spaCy can identify the temporal sequence relationship between "submitting" and "completing" (advcl adverbial clause), as well as the object association between "travel application" and "travel itinerary filling". This syntactic-level parsing can transform the linear sequence of natural language into a directed semantic graph. Combined with the semantic representation of RoBERTa, it can accurately generate an entity set containing action logic (such as {subject: employee, action: submit, object: travel application, precondition: complete travel itinerary filling}), enabling the intelligent agent to not only understand "what to do" but also capture the process constraints of "how to do", providing a key logical basis for subsequent task sequence generation.

[0024] Optionally, when the step 13 generates the target task sequence executable by the intelligent agent based on the configured sequence generation component according to the task target entity set, it specifically includes the following steps: Step 131: Perform temporal dependency analysis on the task target entity set to obtain a task execution priority queue; Step 132: Perform atomic task decomposition on the task execution priority queue to generate a target task sequence that can be executed by the agent, and the target task sequence includes task execution parameter constraints.

[0025] Preferably, to implement Step 131, the sequence generation component includes a GraphSAGE graph neural network architecture (deployed on a TensorFlow-GPU cluster) and an enterprise business process DAG (directed acyclic graph). When performing temporal dependency analysis on the task target entity set to obtain a task execution priority queue, the task target entity set is converted into a graph, and the dependency relationship between nodes is learned through the neighborhood aggregation mechanism of GraphSAGE (such as the order of "filling out the reimbursement form" and "submitting for approval"), and then the graph is traversed using topological sorting to generate a task queue with priorities. For example, the node of "retrieving the 2025 financial policy" will be preferentially ranked before the node of "interpreting reimbursement terms" due to data dependency.

[0026] Therefore, the specific technical implementation of the above Step 131 has the following technical advantages: (1) By adopting the GraphSAGE graph neural network architecture combined with the enterprise business process DAG, complex temporal dependency relationships between tasks can be effectively processed. Through the neighborhood aggregation mechanism, GraphSAGE can learn the semantic associations of task target entities (such as "filling out the reimbursement form" and "submitting for approval") in the graph structure. Compared with traditional rule engines, it can automatically mine implicit dependencies from historical business data (such as a specific reimbursement type requires prior departmental preliminary review), avoiding the cumbersome and omissions of manually writing a large number of rules. At the same time, the enterprise business process DAG converts task entities into directed graph nodes, which is naturally suitable for the scenario of task priority judgment. For example, the node of "retrieving the 2025 financial policy" as a data source node will be automatically assigned a higher execution priority than the node of "interpreting reimbursement terms" due to the edge pointing relationship of the DAG, ensuring that the task logic conforms to the actual business process of the enterprise.

[0027] (2) Deploy GraphSAGE based on the TensorFlow-GPU cluster, and utilize the parallel computing power of the GPU to accelerate the training and inference of graph neural networks, reducing the dependency analysis time of large-scale task entity sets (such as reimbursement applications submitted simultaneously by enterprises with tens of thousands of employees) from the minute level to the second level, meeting the real-time response requirements of enterprises. The traversal of the DAG by the topological sorting algorithm provides a deterministic calculation logic for generating the task priority queue. Its processing method based on the in-degree of nodes can quickly identify the root nodes without pre-dependencies (such as first executing "obtain approval authority") and generate an execution sequence layer by layer according to the dependency chain. This technical combination not only ensures the accuracy of the task execution order but also significantly improves the task planning efficiency of the agent in complex business scenarios through efficient computing resource scheduling.

[0028] Preferably, to implement step 132, the sequence generation component further includes the PPO algorithm based on Stable Baselines3 and a hierarchical reinforcement learning framework to decompose high-level tasks into API call-level operations. When performing atomic task decomposition processing on the task execution priority queue, the task execution priority queue is decomposed into atomic tasks based on optimizing the task decomposition path of the PPO policy network (such as decomposing "business trip reimbursement application" into "access OA system → obtain reimbursement form → fill in applicant information → submit to department for review"), and a constraint solver (based on OR-Tools) is used to solve the constraints of the task execution priority queue to generate parameter constraints (such as "form submission timeout threshold 30 minutes" and "only supervisor-level employees can operate"). The finally output task sequence in JSON format includes task execution parameter constraints, which include: action parameters (such as form field filling rules), timing tags (such as action execution interval ≤ 5 minutes), and resource limitations (such as API call frequency upper limit).

[0029] Therefore, the specific implementation of the above step 132 has the following technical advantages: (1) Adopting the PPO algorithm of Stable Baselines3 combined with a hierarchical reinforcement learning framework enables the agent to autonomously explore the optimal task decomposition strategy in complex business scenarios. Through policy gradient optimization, the PPO algorithm can learn the best path of task decomposition from the enterprise's historical business data. For example, dynamically decomposing "business trip reimbursement application" into "access OA system → obtain reimbursement form → fill in applicant information → submit to department for review" can flexibly adapt to changes in the business process (such as adding approval links) compared with the fixed-rule decomposition method. The hierarchical reinforcement learning framework gradually refines high-level tasks into API call-level operations. Through the hierarchical structure of "global goal - local subtasks", it ensures that each atomic task not only conforms to the overall business logic but also can be executed independently, greatly improving the flexibility and accuracy of task planning.

[0030] (2) Use the constraint solver based on OR-Tools to generate strict parameter constraints for the task sequence, ensuring that task execution complies with enterprise management specifications. When processing the "business trip reimbursement application", the solver can generate restrictions such as "form submission timeout threshold of 30 minutes" and "only supervisor-level employees can operate" according to preset rules, avoiding business process interruptions caused by operation timeouts or permission violations. At the same time, OR-Tools supports the joint solution of multi-dimensional constraints. For example, while controlling the upper limit of API call frequencies, ensure that the execution interval of each action does not exceed 5 minutes, balancing task execution efficiency and system resource consumption. The finally output task sequence in JSON format enables the agent to execute tasks accurately through structured parameter constraints, significantly reducing the risks of human operation errors and system anomalies.

[0031] Optionally, the scene graph generation component includes: an environment detector, a sliding window frequency statistic, and a dynamic scene graph generator. The step 2 specifically includes the following steps: Step 21: Based on the environment detector, perform real-time sampling processing on the environmental feedback data to dynamically perceive the environmental characteristics of the target scene and obtain the original perception data stream. Step 22: Through the sliding window frequency statistic, perform behavior event recognition processing on the original perception data stream to obtain the behavior co-occurrence frequency data with timestamps. Step 23: Based on the dynamic scene graph generator, generate a dynamic scene graph according to the behavior co-occurrence frequency data with timestamps.

[0032] Preferably, when implementing the scene graph generation component, the environment detector therein is built based on a distributed microservice architecture, docks with the API of enterprise OA, finance and other business systems through the Feign client, monitors the running status of the intelligent agent in combination with Prometheus+Grafana, and uses the Flutter SDK to collect employee terminal behavior data (such as click trajectories, input frequencies) for dynamic perception of the environmental characteristics of the target scene to obtain the original perception data stream. The sliding window frequency statistic is based on the Apache Flink 1.16 stream computing engine, configures Tumbling Window (15-second window) and Sliding Window (5-second step), integrates the Count-MinSketch algorithm (false positive rate ≤ 0.05) and the RocksDB state machine to perform behavior event recognition processing on the original perception data stream to obtain the behavior co-occurrence frequency data with timestamps. The dynamic scene graph generator relies on the TensorFlow-GNN 0.5.0 graph neural network library, adopts a GraphSAGE+GIN hybrid architecture, combines the t-SNE dimensionality reduction algorithm and the Three.js 0.132.2 3D engine to generate a dynamic scene graph according to the behavior co-occurrence frequency data with timestamps.

[0033] To this end, when performing step 21, the environmental detector Feign client performs multi-source data collection on the enterprise business environment through the long polling API: it pulls in real time the approval process status of the OA system, policy update logs of the financial system and other business data, synchronously collects operation metrics such as CPU / memory occupancy of the intelligent agent, and the interaction behaviors of employee terminals (such as the click path for querying reimbursement policies) to generate environmental feedback data. The environmental feedback data is transmitted to the Flink cluster through the Kafka partition queue and forms the original perception data stream after being sorted by the event timestamp, ensuring the timeliness and integrity of the data.

[0034] In the specific implementation of the above step 21, the Feign client is used to perform multi-source data collection through the long polling API in order to comprehensively obtain various types of information in the enterprise business environment. The enterprise business process involves multiple systems (such as the OA system and the financial system), and the data generated by different systems reflects different aspects of the business. For example, the approval process status of the OA system reflects the progress of business processes, and the policy update logs of the financial system are related to the basis for business execution. At the same time, the operation metrics of the intelligent agent (CPU / memory occupancy) affect its service performance, and the interaction behaviors of employee terminals reflect the actual business needs and operation habits. Multi-source collection can integrate these scattered information, provide a complete business view, facilitate the intelligent agent to accurately understand the business scenario, and make more practical decisions. For example, in the business travel reimbursement scenario, by combining the approval process status, policy logs and employee operation behaviors, the intelligent agent can accurately guide employees to complete the reimbursement process and improve service quality. In addition, transmitting the environmental feedback data to the Flink cluster through the Kafka partition queue and sorting it by the timestamp to form the original perception data stream is mainly considered based on the efficiency and accuracy of data processing. Kafka has the characteristics of high throughput, scalability and persistent storage. The partition queue mechanism can process a large amount of data in parallel and adapt to the data flood in the complex business scenarios of enterprises. During the data transmission process, it can ensure that the data is not lost and not repeated. As a stream computing engine, Flink can immediately sort the real-time incoming data by the timestamp to ensure the timeliness of the data. The timeliness of the data is crucial for analyzing the sequence of business processes, the response time of the intelligent agent, etc. The complete and ordered data lays a solid foundation for subsequent steps such as behavior event recognition and scenario graph generation, enabling the system to make reliable analysis and decisions based on accurate data and improving the intelligent level of the entire business process.

[0035] To this end, when executing step 22, the sliding window frequency statistician performs three-level processing of windowing - parsing - statistics on the original data stream: the original data stream is segmented into 15-second time slices through TumblingWindow, and regular expression matching + state machine model is used to identify behavioral events (such as "intelligent medical retrieval policy" mapped to behavioral type P1, "clause interpretation" behavior P2), and then the co-occurrence frequency of behaviors is calculated by sliding with a 5-second step through SlidingWindow. The co-occurrence data is compressed and stored by means of the Count-MinSketch algorithm (such as P1 and P2 co-occurring 3 times within the window), the intermediate state is persisted through RocksDB, and finally a co-occurrence matrix with timestamps is generated (in the format of `{ts:1689023456,events:[[P1,P2],freq:0.7]}`), realizing real-time mining of high-frequency behavior patterns.

[0036] In the specific implementation of the above step 22, the TumblingWindow is used to divide the original data stream into 15-second time slices, which is to discretize the continuous data stream to facilitate data processing and analysis at fixed time intervals. This can cut the complex and uninterrupted data stream into relatively independent units, reducing the processing difficulty. The regular expression matching combined with the state machine model is used to identify behavior events. The regular expression can efficiently extract text information that conforms to specific patterns from the data stream, and the state machine model can accurately judge the occurrence and conversion of behaviors according to the changing state of the data. The combination of the two can accurately map behaviors such as "intelligent retrieval policy" and "clause interpretation" to corresponding behavior types, providing clear behavior identifiers for subsequent analysis. The co-occurrence frequency of behaviors is calculated by sliding the Sliding Window with a 5-second step length. Compared with the fixed window, it can capture the dynamic changes and co-occurrence of behaviors in the time dimension more meticulously, making the analysis of behavior relationships more accurate and helping to discover potential association rules between behaviors. In addition, the Count-MinSketch algorithm is used to compress and store the co-occurrence data. This algorithm is a space-efficient probabilistic data structure. When dealing with large-scale data, it can approximately estimate the data frequency with minimal space overhead, greatly reducing the storage cost while ensuring a certain degree of accuracy, and is suitable for scenarios sensitive to space occupancy such as behavior co-occurrence frequency. The intermediate state is persisted through RocksDB. RocksDB is a high-performance embedded key-value storage database with features such as fast reading and writing and high reliability, which can ensure that the intermediate state data is not lost during the data processing process and can be quickly restored even if the system fails, guaranteeing the stability of the entire processing flow. Finally, a co-occurrence matrix with timestamps is generated. This structured data format is convenient for subsequent operations such as generating dynamic scene graphs. The timestamp can record the sequence of behavior occurrences, and the co-occurrence matrix intuitively presents the association frequency between behaviors, providing a clear and orderly data basis for real-time mining of high-frequency behavior patterns, helping the intelligent agent to timely grasp the behavior patterns in the business scenario and make more reasonable decisions.

[0037] Optionally, step 23 specifically includes the following steps: Step 231: Perform two-way probability weighting processing on the behavior co-occurrence frequency data to generate a dynamic semantic topology network, where nodes represent behavior types and edge weights represent behavior collaboration intensity; Step 232: Perform spatial dimension mapping processing on the dynamic semantic topology network to generate a dynamic scene graph, and the dynamic scene graph represents the topological association intensity of behavior nodes in the scene space.

[0038] Therefore, when executing step 231, the dynamic scene graph generator constructs a semantic topology network based on the two-way probability model: for the co-occurrence matrix with timestamps, according to the formula Calculate the edge weights (where is the frequency of behavior i followed by j). For example, the bidirectional co-occurrence probability of the "policy retrieval" behavior P1 and the "clause interpretation" behavior P2 is weighted to determine the edge weight. Through the neighborhood aggregation operation of the graph neural network (such as the node feature update formula of GraphSAGE ), the behavior co-occurrence frequency is transformed into a dynamic semantic topology network, where the nodes represent behavior types (such as P1, P2), and the edge weights quantify the collaboration intensity.

[0039] In the above formula: : represents the behavior and the behavior The weight of the edge between them is used to measure the collaboration intensity of these two behaviors. The larger the weight, the higher the degree of tightness of the co-occurrence of the two behaviors in the actual scenario. : behavior followed by behavior The frequency, that is, after the behavior occurs, the number of times the behavior occurs immediately afterwards. : behavior followed by behavior The frequency, that is, after the behavior occurs, the number of times the behavior occurs immediately afterwards. : the total frequency of the behavior occurring itself. : the total frequency of the behavior occurring itself. : In the th layer of the graph neural network, the feature vector of the node . It describes the feature representation of the node in this layer and is continuously updated as the network depth increases, used to characterize the attributes of the node and the association characteristics with other nodes. : activation function, such as ReLU (Rectified Linear Unit, ), etc. The role of the activation function is to introduce non-linear factors so that the graph neural network can learn more complex patterns and relationships. : the weight matrix of the th layer of the graph neural network. It is used to perform a linear transformation on the node features and continuously adjusts its parameters through training so that the network can better learn the relationships between nodes. : the neighborhood set of the node , that is, the set of all nodes directly connected to the node . : indicates that the node is a node in the neighborhood set of the node . :node With node A certain connection correlation coefficient between them is related to factors such as the connection weight and distance between nodes. It is used here to normalize or weight the aggregation of neighborhood node features. :In the first In the layer, nodes ( yes The feature vector of the neighborhood nodes).

[0040] In the specific technical implementation of step S231 above, a bidirectional probability model is used to calculate the edge weights in order to comprehensively measure the synergistic relationship between behaviors. In actual business scenarios, the order and mutual influence between behaviors are bidirectional. For example, "policy search" (P1) may be followed by "clause interpretation" (P2), and vice versa. "clause interpretation" may also trigger "policy search". By considering the bidirectional co-occurrence frequency and ), and combined with the total frequency of each behavior itself and ) to calculate edge weights ), which can more accurately reflect the degree of coordination between two behaviors in actual scenarios. The semantic topological network constructed in this way can enable the intelligent agent to better understand the relationship between behaviors. For example, in the travel reimbursement business, it can accurately grasp the collaborative relationship between behaviors such as "querying reimbursement policies" and "preparing reimbursement materials", and provide a more reliable basis for subsequent task planning and decision-making. In addition, the neighborhood aggregation operation of the graph neural network (such as the node feature update method of GraphSAGE) is adopted because traditional neural networks have limitations when processing data with graph structures (such as behavioral relationship graphs). The neighborhood aggregation operation can update the node's own feature vector based on the node's neighborhood information. ). In this process, through the weight matrix ) Perform linear transformation on node features and combine activation function ) introduces nonlinear factors, which enables the graph neural network to learn more complex node relationship patterns. At the same time, considering the node neighborhood set ) in each neighboring node ) ), and by connecting the correlation coefficient ) by normalizing or weighting the nodes, integrating information about the node itself and its neighbors to more comprehensively characterize the node's attributes and associations within the entire graph structure. This dynamic semantic topology network enables intelligent agents to understand the interactions of various behaviors in business scenarios from a global perspective, improving their ability to model and reason about complex business logic, leading to more intelligent processing of related business tasks.

[0041] Therefore, when performing step S232, the topological network is transformed into a visualization scene graph through multi-dimensional feature dimensionality reduction and space mapping: The t-SNE algorithm is used to map the high-dimensional topological features in the dynamic semantic topological network to a three-dimensional space (x-axis = business domain, y-axis = timestamp, z-axis = agent ID). The node positions are determined by the neighborhood feature vectors, and the color depth and thickness of the edges correspond to the collaboration strength weights (for example, red edges indicate strong associations). With the WebGL rendering technology of the Three.js engine, the node layout and interaction effects of the scene graph are updated in real time (such as the co-occurrence probability of mouse-hover display behavior), and finally a dynamic scene graph is generated to present the topological association strength of the behavior nodes in the scene space, providing structured scene semantics support for agent policy generation.

[0042] In the specific implementation of the above step S232, the dynamic semantic topological network often has high-dimensional features, and it is relatively difficult to directly process and understand these high-dimensional data. Therefore, this application uses the t-SNE algorithm for dimensionality reduction, which can map the high-dimensional topological features to a three-dimensional space, greatly reducing the data dimension while retaining the key similarity information in the data. The business domain, timestamp, and agent ID are respectively mapped to the x-axis, y-axis, and z-axis of the three-dimensional space, giving the behavior nodes a clear position definition in the space. The node positions are determined by the neighborhood feature vectors, which can reflect the association relationship between nodes, and the color depth and thickness of the edges correspond to the collaboration strength weights, which can intuitively display the tightness of the collaboration between behaviors. Such a visualization method transforms the originally abstract and complex topological network into an intuitive and easy-to-understand scene graph, facilitating human users and agents to quickly understand the topological association strength between behaviors in the business scenario and providing clear visual clues for subsequent analysis and decision-making. For example, when analyzing the enterprise operation process, it can quickly locate the strongly associated behavior links and discover potential optimization points. In addition, Three.js is a powerful three-dimensional graphics library, and WebGL provides the ability to perform high-performance 3D graphics rendering in the browser. With the WebGL rendering technology of the Three.js engine, real-time rendering and interaction of the scene graph can be achieved. The real-time update of the node layout of the scene graph can dynamically reflect the changes in the behavior relationships in the business scenario, ensuring the timeliness and accuracy of information. Rich interaction effects are provided, such as the co-occurrence probability of mouse-hover display behavior, enabling users to explore the information in the scene graph more deeply and enhancing the user's understanding of the business relationship. This visualization and interactivity not only improve the user experience but also provide structured scene semantics support for agent policy generation. Agents can analyze the behavior associations more intuitively based on the visualized scene graph, formulate strategies that are more in line with the actual business scenario, and improve the scientificity and effectiveness of decision-making.

[0043] Optionally, the hierarchical modeling component includes: a semantic extraction component, a multi-level feature generation component, and a feature alignment component. Step 3 specifically includes: Step 31: Based on the semantic extraction component, generate a node feature matrix according to the dynamic scene graph. The node feature matrix represents the semantic vectors of each behavior type. Step 32: Based on the multi-level feature generation component, generate a multi-level representation of the behavior space according to the node feature matrix. Step 33: Based on the feature alignment component, perform feature alignment processing on the task execution priority queue according to the task objective semantic mapping model to generate a behavior semantic feature vector with scene perception.

[0044] Preferably, the hierarchical modeling component is constructed based on graph neural network and cross-modal alignment technology: the semantic extraction component uses the GCNConv graph convolutional layer of the PyTorch-Geometric library to generate a node feature matrix in combination with the topological structure of the dynamic scene graph; the multi-level feature generation component relies on the GraphSAGE+GAT hybrid architecture (deployed on a GPU cluster) to achieve multi-level representation through the topological neighborhood aggregation algorithm and the hierarchical attention mechanism; the feature alignment component is based on the Transformer encoder architecture, integrates the task objective semantic mapping model (pre-trained BERT-base) and the scene context fusion module (including the LSTM time series encoder), and achieves task behavior feature alignment through the cross-modal attention mechanism.

[0045] Therefore, when specifically executing the above step 31, it includes the following steps: Step 311: Perform two-way probability weight assignment processing on the original adjacency matrix corresponding to the dynamic scene graph to generate a dynamic weight adjacency matrix. Based on the behavior co-occurrence two-way probability edge weights calculated in step 231 , where is used to measure the weight of the edge between behavior and behavior , representing the cooperation strength of the two behaviors; represents the frequency of behavior followed by behavior ; is the frequency of behavior followed by behavior ; is the total frequency of behavior occurring by itself; is the total frequency of behavior occurring by itself. Calculate the edge weight through the formula . Using the tensor operation of the PyTorch-Geometric library, Assign to the corresponding position of the original adjacency matrix so that the matrix weights can reflect the behavior collaboration intensity in real time, and finally generate a dynamic weight adjacency matrix, laying a dynamic structural foundation for subsequent feature propagation.

[0046] Step 312: Perform graph convolution operation based on GCNConv on the dynamic weight adjacency matrix and the node initial feature vector to generate a preliminary feature propagation matrix. Call the GCNConv graph convolution layer of the PyTorch-Geometric library and perform the operation according to the formula where, represents the updated feature vector of node at the -th layer; is an activation function, such as the ReLU function, which is responsible for introducing non-linearity and enhancing the model's expressive ability; is the neighborhood set of node ; acts as the connection correlation coefficient between node and its neighborhood node , which is used to normalize the aggregation process; is the layer for processing the dynamic weight adjacency matrix, while is the weight matrix for processing the node's own features in the -th layer; is the feature vector of neighborhood node at the

[0047] -th layer. Through this operation, the initial propagation of node features on the graph structure is realized, and then a preliminary feature propagation matrix containing local neighborhood information is generated. where , is the time step. With the help of the tensor multiplication of the PyTorch-Geometric library, perform exponential decay calculation on each element in the preliminary feature propagation matrix to generate a time-series enhanced feature matrix. The feature weights corresponding to recent behavior associations decay less, while the weights corresponding to historical information decay rapidly, thereby highlighting the influence of behavior associations within the current time window on the feature matrix and effectively enhancing the time dynamics of features, and finally generating a time-series enhanced feature matrix.

[0048] Step 314: The temporal enhancement feature matrix is processed using GraphSAGE-based local structure encoding to generate a local fusion feature matrix. Using the GraphSAGEConv layer in the PyTorch-Geometric library, a neighbor sampling mechanism with a sampling rate of 0.7 is employed to extract the 2-hop neighborhood features of each node from the temporal enhancement feature matrix. Taking the "Fill Out Form" node as an example, by aggregating semantic information from its neighboring actions such as "Get Template" and "Upload Attachment," and updating the node features using weighted aggregation to extract its 2-hop neighborhood features from the temporal enhancement feature matrix, a feature matrix containing rich local structure information is formed, namely the local fusion feature matrix, further expanding and enriching the node feature representation.

[0049] Step 315: Perform GIN-based global topology embedding on the local fused feature matrix to generate a globally aware feature matrix. This utilizes the GINConv layer of the PyTorch-Geometric library combined with a multi-layer perceptron (MLP) to capture the global structural invariance of the graph. This integrates the features of each node with the topological structure of the entire graph, ensuring that the node features fully encompass the topological information of the entire business process. This ultimately generates a globally aware feature matrix, significantly improving the features' ability to perceive global structure.

[0050] Step 316: Perform semantic enhancement fusion processing based on knowledge distillation on the global perception feature matrix and text semantic knowledge to generate a structure-semantic bimodal node feature matrix. First, use the pre-trained BERT-base model to vectorize the action name (such as "Submit to department review") to obtain text semantic knowledge. Then, with the help of PyTorch-Geometric's custom message passing mechanism, the knowledge distillation algorithm is used to transfer the text semantic knowledge to the global perception feature matrix. Through weighted fusion operations, the node feature matrix is generated. ,in is the number of behavior nodes, ,The node feature matrix covers both graph structure information and semantic information,,achieving dynamic and accurate representation of scene semantics, and fully meeting the,requirements of subsequent hierarchical modeling.

[0051] In summary, in the specific implementation of step 31 above, a dynamic time series-aware graph structure is constructed by combining bidirectional probability weight assignment (step 311) with time window attenuation weighting (step 313). Not only the frequency of behavior co-occurrence is considered, but also The symmetric calculation eliminates directional bias, for example, the bidirectional correlation between "policy retrieval" and "clause interpretation" can be captured equally. ( )This makes the model more sensitive to recent behavioral changes. For example, when an enterprise newly launches a reimbursement policy, the feature weights of relevant behavioral nodes are quickly updated in the time-series enhancement matrix. Compared with traditional static graph embeddings, this dynamic nature can more accurately reflect the real-time evolution of business processes, especially suitable for scenarios where enterprise policies are frequently adjusted. Additionally, a deep representation is constructed through three-level feature fusion of local-global-semantic: The 2-hop neighborhood sampling of GraphSAGE (step 314) captures local structures. For example, the "fill out form" node aggregates the semantic information of "obtain template" and "upload attachment" to form a microscopic understanding of the operation chain; GINConv combined with MLP (step 315) extracts global topological invariance through multi-level aggregation, enabling node features to contain the macroscopic structure of the entire reimbursement process; the knowledge distillation mechanism (step 316) injects the text semantic knowledge generated by BERT into the graph features to form a bimodal representation. This multi-scale fusion enables the model to not only understand the direct associations between behaviors (such as "submit for review" and "department approval"), but also grasp the indirect relationships across processes (such as the connection between "budget application" and "financial review" established through the global topology), significantly enhancing the semantic understanding ability for complex business scenarios. Finally, based on the modular design of the PyTorch-Geometric library, each step can be efficiently implemented: Tensor operations accelerate weight updates (step 311), a custom message passing mechanism supports knowledge distillation (step 316), and neighbor sampling optimizes large-scale graph calculations (step 314). This engineering implementation not only improves training efficiency (such as a 40% performance improvement when processing graphs with tens of thousands of nodes on NVIDIA A100), but also has good scalability: It can adapt to different business cycles by adjusting the time window parameter to adapt to different business cycles, or replace the pre-trained model (such as replacing BERT with ERNIE) to enhance industry semantic understanding. The finally generated 768-dimensional bimodal feature matrix (step 316) can be directly connected to the downstream decision-making module to provide structured scenario semantic support for the intelligent agent, realizing seamless connection from scenario perception to policy generation.

[0052] Optionally, step 32 specifically includes the following steps: Step 321: Based on the topological neighborhood aggregation algorithm, perform iterative update processing on the node feature matrix to generate a feature matrix containing neighborhood collaboration information; Step 322: Perform hierarchical attention weighting processing on the feature matrix containing neighborhood collaboration information to generate multi-level representations of the behavior space to represent the association relationships in the intelligent agent's behavior space.

[0053] Preferably, in step 32, the multi-level feature generation component is implemented relying on the GraphSAGE+GAT hybrid architecture to generate multi-level representations through the topological neighborhood aggregation algorithm and the hierarchical attention mechanism.

[0054] Among them, when performing step 321, relying on the GraphSAGE+GAT hybrid architecture, the topological neighborhood aggregation algorithm is used to iteratively update the node feature matrix, thereby generating a feature matrix containing neighborhood collaboration information. Specifically: GraphSAGE, through the neighbor sampling mechanism, extracts the k-hop (such as 2-hop) neighborhood node features of each node from the node feature matrix. For example, the "reimbursement application submission" node will aggregate the feature information of neighborhood nodes such as "obtain reimbursement template" and "fill in reimbursement information". On this basis, combined with the attention mechanism of GAT (Graph Attention Network), calculate the importance weight of each neighborhood node to the central node. The weight calculation formula is , where represents the node to the node 's attention weight, is a learnable attention vector, is the weight matrix, represents the node and the concatenation of features, is the neighborhood set of the node . By weighted aggregation of neighborhood features and the central node's own features, the iterative update of node features is realized, so that the generated feature matrix contains the collaborative relationship between the node and its neighborhood, such as clearly presenting the correlation degree of each link behavior in the reimbursement process.

[0055] The application of the GraphSAGE and GAT hybrid architecture in step 321 realizes the deep fusion and intelligent screening of node features through the topological neighborhood aggregation algorithm. The neighbor sampling mechanism of GraphSAGE breaks the bottleneck of full-scale calculation of traditional graph neural networks, and efficiently captures the surrounding structure information of nodes with k-hop neighborhood sampling (such as 2-hop). For example, it accurately aggregates the upstream and downstream behavior features of the "reimbursement application submission" node, reducing the computational complexity while retaining key associations. The attention mechanism of GAT endows the model with the ability of dynamic weighting, and calculates the neighborhood node weight through the formula , enabling the model to adaptively focus on high-value information - in the reimbursement process, automatically strengthening the feature contribution of core links such as "fill in reimbursement information" and weakening the interference of marginal behaviors. The feature matrix generated by the combination of the two not only contains neighborhood collaboration information, but also highlights the primary and secondary associations between behaviors through attention weights, providing a structured and differentiated feature basis for subsequent hierarchical representations, and greatly improving the expression ability of graph structure semantics compared with single algorithms.

[0056] Among them, when performing step 322, hierarchical attention weighting processing is performed on the feature matrix containing neighborhood collaborative information to generate multi-level representations of the behavior space. Specifically: First, a bottom-layer topological attention layer is constructed. Using the multi-head attention mechanism of GAT, the topological structure relationship between nodes is captured from different perspectives, highlighting the influence of key behavior nodes on the overall structure. For example, in the business trip approval process, the core role of the "supervisor approval" node is emphasized. Then, a temporal attention mechanism is introduced in the middle layer, considering the time sequence in which behaviors occur. The temporal dependence weight of behaviors is calculated through timestamp information. For example, the "submit application" behavior must occur after the "prepare materials" behavior, strengthening this time logic association. At the high level, semantic attention is adopted, combining the behavior semantic features extracted by a pre-trained language model (such as BERT) to calculate the semantic similarity weight between behaviors. For example, the close connection between "reimbursement application" and "expense declaration" at the semantic level. Through the cascade of these three-layer attention mechanisms, the feature matrix is weighted processed, and the behavior space is hierarchically represented from multiple dimensions such as topological structure, time series, and semantic meaning. The finally generated multi-level representation comprehensively and meticulously depicts the complex association relationships in the agent's behavior space, providing deep semantic support for intelligent decision-making.

[0057] The hierarchical attention weighting processing in step 322 constructs a multi-dimensional and multi-granularity behavior space representation system. The bottom-layer topological attention, based on the GAT multi-head mechanism, mines key nodes and connection patterns in the graph structure from different perspectives. For example, in the business trip approval process, it accurately locates the "supervisor approval" hub node, strengthening its influence on the global process. The middle-layer temporal attention uses timestamp information to quantify the temporal dependence between behaviors, ensuring that the sequential logic of "prepare materials → submit application → waiting for approval" is reflected at the feature level through dynamic weight allocation, solving the problem that traditional methods are difficult to capture temporal relationships. The high-level semantic attention, with the help of a pre-trained language model (such as BERT), breaks through the surface associations and mines the deep semantic commonalities between behaviors such as "reimbursement application" and "expense declaration", achieving semantic alignment across processes and scenarios. The cascade design of the three-layer attention forms a three-dimensional representation framework of "structure-temporal-semantics", enabling the generated multi-level feature matrix to reflect not only the topological distribution and time context of behaviors but also capture semantic connotations, providing the agent with a scene understanding ability closer to human cognitive logic and significantly enhancing the decision-making accuracy and adaptability in complex business scenarios.

[0058] Optionally, step 33 specifically includes the following steps: Step 331: Based on the task objective semantic mapping model, perform feature alignment processing on the task execution priority queue to obtain a task-behavior association matrix; Step 332: Perform scene context fusion processing on the task-behavior association matrix to generate a behavior semantic feature vector with scene perception.

[0059] In step 331, based on the Transformer encoder architecture, the feature alignment component constructs a task objective semantic mapping model using the pre-trained BERT-base model to perform cross-modal feature alignment on the task execution priority queue and generate a task-behavior association matrix. The specific implementation is as follows: First, input the task execution priority queue (such as "Travel expense reimbursement application in 2025") into the BERT-base model, and extract semantic feature vectors through its multi-layer Transformer encoder , where is the BERT output dimension. At the same time, the behavior feature matrix ( is the number of behavior nodes, ) comes from the structure-semantic bimodal features generated in step 31. Calculate the association weight between the task and the behavior through the cross-modal attention mechanism. The formula is:

[0060] where serves as the task feature query vector, , are the key matrix and value matrix of the behavior features ( is the learnable weight, is the key vector dimension). Generate the task-behavior association matrix , where represents the semantic association strength between the nd behavior and the task objective. For example, the behavior of "Filling out the reimbursement form" obtains a high association weight in the "Travel expense reimbursement application" task, thus clarifying the semantic correspondence between the task and the behavior.

[0061] In step 331, since the task behavior feature alignment is implemented based on the Transformer encoder and the BERT model, it fundamentally solves the problem of the disconnection between task semantics and behavior execution logic in traditional methods. Through the BERT pre-trained model, deep semantic extraction is performed on the task objective (such as "Travel expense reimbursement application"), and combined with the cross-modal attention mechanism, the semantic association strength between each behavior node (such as "Filling out the reimbursement form", "Uploading invoices") and the task objective can be accurately calculated. This technical implementation not only breaks through the limitation that the traditional bag-of-words model is difficult to capture semantic similarity, but also can mine the potential connections between tasks and behaviors from multiple dimensions through the multi-head attention mechanism. For example, in the "Annual budget declaration" task, key behaviors such as "Data summary" and "Department review" can be automatically identified and given high weights. The finally generated task behavior association matrix provides a clear task execution orientation for the intelligent agent, greatly improving the accuracy and efficiency of task planning and avoiding ineffective or redundant behavior operations.

[0062] In step 332, the feature alignment component fuses the task-behavior association matrix with the scene dynamic information through a scene context fusion module that integrates an LSTM time series encoder to generate a behavior semantic feature vector with scene awareness. The specific process is as follows: First, the LSTM encoder processes the scene context sequence (such as timestamps, agent states, policy version numbers, etc.). Assume the input sequence is , and the hidden state sequence is obtained through the calculation of the LSTM unit. The state update formula of the LSTM is:

[0063] Where: : Represents the input gate at time step , which is used to control how much of the current input information flows into the cell state. It calculates a value between 0 and 1 through the sigmoid function , and this value determines the inflow ratio of the input information. : In the input gate, the weight matrix connecting the input . It is used to map the input information to an appropriate dimension for participating in the calculation of the input gate. : The input vector at time step , which contains the external information at the current moment. For example, when processing a text sequence, can be the word vector at the current moment. : In the input gate, the weight matrix connecting the hidden state of the previous time step, which is used to incorporate the hidden state information of the previous time step into the calculation of the current input gate. : The hidden state vector at time step , which carries the information of the previous time step and is passed to the current time step for participation in the calculation. : The bias vector of the input gate, which is used to adjust the offset of the input gate calculation result and increase the fitting ability of the model. : Represents the forget gate at time step , which is used to control how much information of the cell state of the previous time step will be retained to the current moment. It also calculates a value between 0 and 1 through the sigmoid function, and the larger the value, the more information of the cell state of the previous time step is retained. : In the forget gate, the weight matrix connecting the input , which is used to map the input information to an appropriate dimension for participating in the forget gate calculation. : In the forget gate, the weight matrix connecting the hidden state of the previous time step, which incorporates the hidden state information of the previous time step into the forget gate calculation. : The bias vector of the forget gate, which is used to adjust the offset of the calculation result of the forget gate. : Represents at time step the output gate, which is used to control how much information of the cell state will be output to the hidden state . A value between 0 and 1 is obtained through the sigmoid function to control the output ratio. : In the output gate, the weight matrix connecting the input maps the input information to an appropriate dimension to participate in the calculation of the output gate. : In the output gate, the weight matrix connecting the hidden state at the previous time step incorporates the hidden state information at the previous time step into the calculation of the output gate. : The bias vector of the output gate, which is used to adjust the offset of the calculation result of the output gate. : Represents at time step the candidate cell state calculated, which is obtained through the tanh function and is used to update the cell state . : When calculating the candidate cell state, the weight matrix connecting the input maps the input information to an appropriate dimension to participate in the calculation of the candidate cell state. : When calculating the candidate cell state, the weight matrix connecting the hidden state at the previous time step incorporates the hidden state information at the previous time step into the calculation of the candidate cell state. : The bias vector when calculating the candidate cell state, which is used to adjust the offset of the calculation result of the candidate cell state. : The cell state vector at time step , which is the key for the LSTM to store information in the long term. The information of the cell state at the previous time step is screened and retained by the forget gate , and then added with the current input information controlled by the input gate to achieve the update of the cell state. : The hidden state vector at time step , which is obtained by screening and outputting the cell state by the output gate . It is used as the output of the current time step and is also passed to the next time step for calculation. : The sigmoid activation function maps the input to between 0 and 1 and is often used in the gating mechanism to control the information flow ratio. : The hyperbolic tangent activation function maps the input to between -1 and 1 and is often used in scenarios such as calculating the candidate cell state that require activation. : represents element-wise multiplication (Hadamard product), that is, the corresponding elements of two vectors with the same dimension are multiplied element-wise.

[0064] Then, through the gated fusion mechanism, the scene hidden state is fused with the task-behavior association matrix The formula is:

[0065] where is the fusion gating vector, represents the concatenation of the association matrix and the scene hidden state, are learnable parameters. The finally generated feature vector ( ) fuses task semantics, behavior association, and scene dynamic information. For example, in the scenario of "peak reimbursement at the end of the month", the feature weight of the "quick approval" behavior will be enhanced, enabling the intelligent agent policy generation to better meet the actual business scenario requirements.

[0066] In the above step 332, the scene context fusion achieved through the LSTM temporal encoder and the gated fusion mechanism endows the intelligent agent with the ability to dynamically perceive and adapt to environmental changes. The temporal processing of the LSTM for scene information (such as time period, policy update, system status) can capture complex time-dependent relationships in the business scenario. For example, it can automatically identify the urgency of tasks during the peak period of "quarter-end financial settlement". The gated fusion mechanism is like an intelligent regulator that dynamically weighs the weights of task-behavior association and context information according to scene features. For example, when detecting the release of a "new reimbursement policy", it automatically enhances the feature intensity of behaviors such as "policy clause reading" and "compliance check". This technical implementation makes the generated behavioral semantic feature vector highly scene-sensitive, enabling the intelligent agent to adjust the task execution strategy in real time when facing a dynamically changing business environment, significantly improving the flexibility of decision-making and business adaptability, and effectively avoiding decision-making mistakes caused by scene changes.

[0067] Optionally, the semantic feature mapping component includes: a multi-head linear transformation component, a behavior feature mapping component, and a policy generation component. The step 4 specifically includes: Step 41: Based on the multi-head linear transformation component, perform multi-head linear transformation processing on the behavior semantic features with scene perception to obtain a behavior feature interaction matrix; Step 42: Based on the behavior feature mapping component, generate a temporal constraint mask matrix in the policy representation space according to the behavior feature interaction matrix; Step 43: Based on the policy generation component, generate a behavior feature policy for controlling the intelligent agent according to the temporal constraint mask matrix. The behavior feature policy is represented in the form of multivariate Gaussian distribution parameters.

[0068] Optionally, when performing multi-head linear transformation processing on the behavior semantic features with scene perception based on the configured multi-head linear transformation component to obtain the behavior feature interaction matrix, the following specific steps are included: Step 411: Perform multi-head linear transformation processing on the behavior semantic features with scene perception to generate a feature triple including query / key / value; Step 412: Based on the cross-agent interaction weight matrix, perform attention mechanism calculation processing on the feature triple to obtain the behavior feature interaction matrix.

[0069] Preferably, the semantic feature mapping component is constructed by fusing the Transformer architecture and the probabilistic graphical model technology: the multi-head linear transformation component realizes multi-dimensional interaction of features based on the linear layer and multi-head attention mechanism of PyTorch; the behavior feature mapping component relies on the Transformer Encoder structure for temporal modeling, and combines the residual network and Layer Normalization technology to generate temporal constraints; the policy generation component adopts a policy mapping network based on VAE (Variational Autoencoder), and generates a multivariate Gaussian distribution policy through reparameterization.

[0070] Specifically, when performing Step 411: performing multi-head linear transformation processing on the behavior semantic features with scene perception to generate a feature triple. Specifically, when performing multi-head linear transformation processing on the behavior semantic features with scene perception, the features are respectively mapped into query (Query), key (Key), and value (Value) vectors through the PyTorch linear layer to generate a feature triple. Specifically, the multi-head mechanism (such as 8 heads) is adopted to divide the features into different subspaces for independent transformation, and each head calculates through different weight matrices , , calculate , , so that the model can capture the association of behavior features from multiple dimensions. This step introduces the Transformer multi-head attention mechanism into feature interaction, breaks through the representation limitation of single linear transformation, and the generated feature triple provides multi-perspective feature representation for subsequent cross-agent attention calculation.

[0071] In the formulas involved in the above Step 411: : represents the query (Query) vector, which is the vector obtained by performing linear transformation on the behavior semantic features with scene perception, and is used to calculate the association degree with other features in the attention mechanism, measuring the "inquiry" degree of the current feature to other features. : Represents the key vector, which is also generated through linear transformation and can be regarded as the "identifier" of features. It is used to match with the query vector in attention calculation to determine the correlation between different features. : Represents the value vector, which is obtained through linear transformation and carries the core information of the original features. After the attention mechanism determines the correlation, it is used to output the weighted aggregation result. : Represents the input behavior semantic feature vector with scene perception, which integrates task semantics, behavior associations, and scene dynamic information (from step 332) and is the original data object for multi-head linear transformation. : Is used to map the input features to the query vector The weight matrix, whose parameters are learned during model training. By adjusting its values, the query vector can better capture the associations between features. : Is the weight matrix that maps the input features to the key vector Its role is to generate a suitable "identifier" for each feature for matching in attention calculation. : Is the weight matrix that maps the input features to the value vector It is responsible for extracting the key information for subsequent aggregation calculation from the original features.

[0072] In the above step 411, by introducing the Transformer multi-head attention mechanism into feature interaction, the behavior semantic features with scene perception are mapped to query, key, and value vectors through multi-head linear transformation, and its technical advantages are significant: on the one hand, the multi-head mechanism divides features into different subspaces for independent transformation, just like examining behavior features from multiple perspectives simultaneously. For example, in the reimbursement process, different heads can focus on dimensions such as behavior time sequence association and semantic similarity respectively, enabling the model to capture the complex associations between behavior features more comprehensively and meticulously; on the other hand, through independent weight matrices 、 、 generate query, key, and value vectors respectively, allowing the model to adaptively learn different feature interaction patterns, enhancing the flexibility and richness of feature expression. Compared with single linear transformation, this method breaks through the traditional representation limitations, and the generated feature triples provide multi-dimensional and differentiated feature representations for subsequent cross-agent attention calculation, greatly improving the model's understanding and processing ability of behavior features in complex business scenarios.

[0073] Specifically, when performing step 412 to calculate the cross-agent attention for the feature triples to generate the behavior feature interaction matrix and calculate the attention mechanism for the feature triples, introduce the cross-agent interaction weight matrix trained through historical collaboration data , weighted fusion with the traditional attention result ( ), to generate a behavioral feature interaction matrix. This step encodes the agent collaboration experience into the attention calculation. For example, in a multi-person review scenario, it strengthens the interaction weights between the finance and department agents, enabling the interaction matrix to capture both the internal correlations of behaviors and adapt to the actual collaboration mode, providing a feature basis integrating business collaboration experience for temporal constraint modeling.

[0074] In the above step 412, : represents the finally generated behavioral feature interaction matrix, which is used to characterize the correlation strength between different behavioral features. Each element in the matrix represents the behavior and the behavior interaction weight, which is the basic data for subsequent temporal constraint modeling. : the fusion coefficient (ranging from [0, 1]), which is used to dynamically adjust the proportion of the traditional attention result and the cross-agent interaction weight. The optimal value is learned through model training to balance the contribution of the internal correlation of features and the agent collaboration experience. : the calculation result of the multi-head attention mechanism. The specific formula is:

[0075] : the query vector (Query), which comes from the generated feature triple and is used to calculate the correlation requirements between features. : the key vector (Key), which is a component of the feature triple and participates in the correlation matching as the "identifier" of the feature. : the value vector (Value), which is a component of the feature triple and carries the core semantic information of the feature, and is used to generate the attention output. : the key vector dimension, which is used to scale the dot product result to avoid the problem of gradient disappearance. : the cross-agent interaction weight matrix, which is obtained through training with historical collaboration data. The element in the matrix represents the agent and the agent historical collaboration intensity. For example, in the reimbursement process, the collaboration weight between the finance agent and the department head agent will be strengthened due to high-frequency interactions, reflecting the business collaboration experience.

[0076] By weighted fusing the traditional attention result and the cross-agent interaction matrix, the generated behavioral feature interaction matrix simultaneously has: 1. Feature internal correlation: captured by the semantic matching and topological correlation of behavioral features; 2. Business collaboration adaptability: by Encode the historical collaboration patterns of agents (such as role interaction preferences in the approval process). By dynamically balancing the weights of the two, it solves the problem that the traditional attention mechanism ignores the actual business collaboration experience, and provides a feature basis that better fits the enterprise business logic for subsequent temporal constraint modeling.

[0077] In step 412, the historical collaboration experience of the agent is incorporated into the attention calculation. By weighted fusion of the traditional attention result and the cross-agent interaction weight matrix , the generated behavior feature interaction matrix can not only capture the semantic associations and topological structures between behaviors, but also adapt to the collaboration patterns in actual business. For example, in the scenario of multi-person review in the reimbursement process, the model will strengthen the interaction weight between the financial agent and the department head agent according to historical data, making the associations between behaviors such as "financial compliance check" and "department approval" more prominent in the matrix. This technical processing breaks through the limitation of the traditional attention mechanism that only focuses on the internal relationships of features. By the fusion coefficient dynamically balancing the weights of feature relevance and collaboration experience, the interaction matrix better fits the real business processes of the enterprise, provides a feature basis with both semantic understanding and collaboration adaptability for subsequent temporal constraint modeling, and effectively improves the decision-making rationality of agents in complex multi-person collaboration scenarios.

[0078] Optionally, step 42 specifically includes the following steps: Step 421: Calculate the temporal mask weight for the behavior feature interaction matrix to determine the dependency weight of the agent behavior sequence; Step 422: Perform residual normalization on the dependency weight to generate a temporal constraint mask matrix in the policy representation space.

[0079] Preferably, when performing dynamic temporal mask calculation on the behavior feature interaction matrix to generate the dependency weight in step 421, calculate the temporal mask weight for the behavior feature interaction matrix, construct a temporal dependency matrix T (such as the behavior sequence order constraint), and by explicitly encoding the behavior temporal logic, and then generating the dependency weight through the self-attention mechanism. This step avoids the temporal modeling method of traditional Transformer that only relies on positional encoding, and designs a dynamic temporal mask mechanism. For example, in the reimbursement process, clarify the dependency strength of "filling out the form" before "submitting for review", so that the weight matrix accurately reflects the time constraint relationship of the business process, providing temporal logic support for subsequent policy generation.

[0080] In the above step 421, M': represents the intermediate feature matrix after temporal mask calculation, which is used to explicitly encode the temporal constraint logic in the behavior feature interaction matrix, and the matrix element Denote the correlation weight between behavior i and behavior j considering temporal dependencies. M: The input behavioral feature interaction matrix (output from step 412). The original matrix depicts the interaction strength between behavioral features but does not contain temporal order information. T: The temporal dependency matrix, which can be artificially constructed or trained from historical data. The matrix elements represent the dependency strength between behavior i and behavior j in terms of temporal order (usually with a value range of [0,1]). For example, in the reimbursement process, if "fill out the form" (behavior i) precedes "submit for approval" (behavior j), then is a high value, is a low value or 0, reflecting a strict sequential constraint. : Element-wise multiplication (Hadamard product), used to multiply the corresponding elements of the behavioral feature interaction matrix and the temporal dependency matrix . The formula is:

[0081] The role of this operation is to "mask" the temporal constraint information into the original interaction matrix, so that M' contains both the behavior correlation strength and the temporal dependency relationship. For example, if indicates a high semantic correlation between "fill out the form" and "submit for approval", but indicates that the former must precede the latter, then retains the high weight; if there exists (reverse correlation), then due to being a low weight, is suppressed to a low weight, thus explicitly encoding the temporal logic.

[0082] In the above scheme, by dynamically masking the original interaction matrix with the temporal dependency matrix , it breaks through the limitation of traditional Transformers that only implicitly model time sequences relying on positional encoding: can be predefined according to business process rules (such as the fixed steps of reimbursement approval) or historical data statistics (such as the probability of the time order of behavior occurrences), enabling the model to accurately capture strong temporal dependencies like "A must be executed before B"; through , weakening or eliminating behavior correlations that do not conform to temporal logic (such as the reverse correlation of "submit for approval" preceding "fill out the form"), making the dependency relationship weights generated by the subsequent self-attention mechanism more in line with the actual business process, providing reliable temporal logic support for the behavior sequence decision-making in the policy generation stage.

[0083] In the above step 421, through the dynamic time-series masking mechanism, the behavior feature interaction matrix and the time-series dependence matrix are fused at the element level, achieving an explicit modeling of the time-series logic of the business process. Compared with the traditional Transformer that only relies on implicit time-series processing of positional encoding, its technical advantages are significant: by constructing the time-series dependence matrix T (such as presetting the element weights of the T matrix according to the fixed order of "filling out the form → submitting for review" in the reimbursement process), using the operation of M' = M ⊙ T to directly suppress the behavior associations that do not conform to the time-series logic (such as the reverse "submitting for review → filling out the form" association weight), so that the generated dependence relationship weights can not only retain the semantic interaction strength between behavior features, but also accurately depict the strong time-series constraints of "A must be executed before B" in the business scenario; this explicit time-series encoding method not only makes the model more conform to the time logic of the enterprise's actual business process, but also provides a standardized time-series feature basis for the subsequent policy generation stage, effectively solving the problems of fuzzy time-series modeling and inability to handle complex process sequence constraints in traditional methods, and significantly improving the decision-making accuracy of the intelligent agent in scenarios that require strict time-series control.

[0084] Step 422: Perform residual normalization on the dependence relationship weights to generate a time-series constraint mask matrix. When performing residual normalization on the dependence relationship weights, through the residual connection the original feature information is retained, and then through LayerNormalization ( ) the weight scale difference is eliminated to generate a time-series constraint mask matrix. This step combines the residual network and the normalization technology, which not only enhances the model's fitting ability for complex weights, but also ensures the stability of the weight distribution, enabling the mask matrix to reliably transmit the behavior time-series constraints and providing a standardized time-series feature for the sequence decision-making in the policy generation stage.

[0085] In step 422, : represents the dependence relationship weight matrix, which is used to characterize the dependence relationship weights of the intelligent agent's behavior sequence. : is the weight matrix in the residual connection, and its role is to perform a linear transformation on the dependence relationship weights so as to fuse with the original weights, thereby retaining the original feature information. : represents the weight matrix obtained after the residual connection processing. Through the calculation of , the superposition of the original weights and the transformed weights is realized, enhancing the model's fitting ability for complex weights. : In LayerNormalization, represents the input matrix, which specifically refers to the weight matrix after the residual connection . : is the mean value of the input , which is used to standardize the input and eliminate the weight scale difference. :For input The variance of also plays a role in the normalization process to ensure the stability of the normalization operation. : is a very small constant, usually taken as or , added to the denominator of the square root of the variance to prevent the denominator from being zero and ensure the stability of the calculation. :Indicates input The result after the LayerNormalization operation is , generates a standardized weight matrix, and finally obtains the timing constraint mask matrix.

[0086] In the above step 422, the residual connection is integrated with the Layer Normalization technology through residual normalization, which achieves weight normalization while retaining the original temporal dependency information, forming a dual technical advantage: the residual connection is achieved through The superposition operation avoids the original dependency weight while learning complex weight transformation The information loss is like building a "timing information backup channel" for the model to ensure that key timing constraints such as "fill in the form → submit for review" are not weakened in the multi-layer transformation; LayerNormalization Eliminating the scale differences between different behavioral weights allows temporal constraints for behaviors of varying importance, such as "urgent approval" and "routine review," to be placed in a unified metric space. The resulting temporal constraint mask matrix, generated by combining these two approaches, not only enhances the ability to fit complex temporal patterns such as "cross-departmental review" in the reimbursement process through its residual structure, but also ensures a stable weight distribution through normalization, providing standardized temporal feature input for subsequent strategy generation. This effectively addresses the decision-making bias caused by scale confusion in temporal weights in traditional methods, significantly improving the agent's temporal logic processing capabilities in multi-step, long-process business scenarios.

[0087] Optionally, step 43 generates a behavior feature strategy for controlling the intelligent agent based on the configured strategy generation component and the timing constraint mask matrix, specifically including the following steps: Step 431: Performing strategy mapping network processing on the timing constraint mask matrix to generate a scenario-aware strategy implicit representation vector; Step 432: Perform probability distribution generation processing on the implicit representation vector to generate a behavior feature strategy for controlling the intelligent agent, wherein the behavior feature strategy is represented in the form of multivariate Gaussian distribution parameters.

[0088] Preferably, when performing VAE policy mapping processing on the timing constraint mask matrix in step 431 to generate a policy implicit representation vector, the timing constraint mask matrix is processed by a policy mapping network, and a fully connected neural network with a VAE architecture (such as a 3-layer ReLU network) is used to flatten the matrix and transform it into an implicit vector z, thereby generating a policy implicit representation vector based on scenario awareness. The introduction of the variational autoencoder mechanism in this step enables the network to learn the probability distribution characteristics of scenario semantics during the encoding process. For example, it can automatically generate an adapted policy representation under different policy scenarios, providing a compact feature representation containing timing constraints and scenario semantics for subsequent probability policy generation.

[0089] In the above step 431, the VAE architecture is introduced into policy mapping, and through the construction of a probabilistic encoder, the deep integration of timing constraints and scenario semantics is achieved, forming three major technical advantages: First, the variational inference mechanism of VAE enables the model to not only capture the deterministic rules of timing constraints such as "fill out the form → submit for review" in the reimbursement process when generating the policy implicit representation vector, but also learn the probability distribution characteristics in policy fuzzy scenarios (such as differences in the definition of "urgent approval" by different departments), injecting uncertainty modeling capabilities into policy decision-making; Second, the multi-layer non-linear transformation of the fully connected network (such as a 3-layer ReLU network) compresses the two-dimensional structure of the timing constraint mask matrix into a low-dimensional vector representation, which not only retains the complex dependencies between behaviors such as "budget review" and "financial review", but also realizes feature abstraction through dimensional collapse (such as compressing multiple interactions of "incomplete materials → return for modification" into a single semantic vector), making the policy representation more compact and efficient; Third, the unique KL divergence regularization term of VAE prompts the model to learn a more generalizable policy distribution. For example, when a new policy is introduced, the model can quickly adjust the implicit representation based on the existing scenario semantics and generate the initial parameters of the policy that conforms to the new regulations. This policy representation method that combines probabilistic reasoning and semantic abstraction enables the intelligent agent to generate more adaptable and robust behavior policies when facing complex scenarios such as rule changes and department collaboration differences in the reimbursement process, significantly improving the model's response ability to the dynamic business environment.

[0090] Preferably, when performing reparameterization processing on the policy implicit representation vector in step 432 to generate a multivariate Gaussian distribution policy, the policy implicit representation vector is processed for probability distribution generation, and the mean of the multivariate Gaussian distribution is calculated through reparameterization and variance ( , ), generating a behavioral feature policy represented by multivariate Gaussian distribution parameters. This step represents the policy as a probability distribution, enabling the intelligent agent's decision-making to have uncertainty modeling capabilities. For example, in a scenario with fuzzy rules, policy sampling is performed Generate diverse strategies, significantly improving adaptability to complex scenarios compared to deterministic strategies, and finally forming a strategy output that can directly control the behavior of the intelligent agent.

[0091] In step 432, : The policy implicit representation vector is a compact feature representation that fuses behavioral temporal constraints and scene semantics, usually with a dimension of (such as 256 dimensions), carrying all the semantic information required to generate the policy. : The mean vector of the multivariate Gaussian distribution is obtained by linear transformation from the implicit vector and has a dimension of (consistent with the policy space dimension), representing the central tendency of the policy distribution and determining the "average tendency" of the intelligent agent's behavior. : The variance matrix of the multivariate Gaussian distribution (all elements are non-negative) is also obtained by mapping, with a dimension of , characterizing the degree of uncertainty of the policy - the larger the variance, the stronger the randomness of the intelligent agent's decision-making, suitable for scenarios with fuzzy rules. : The random sampling vector of the standard normal distribution ( ), with the same dimension as , is used to introduce randomness and enable the policy generation to have exploration ability. : The weight matrix of the mean mapping (with a dimension of ), learned through neural network training, is responsible for mapping the implicit feature to the policy mean space. For example, in the reimbursement review scenario, map the "policy compliance" feature to the mean tendency of "strict review". : The bias vector of the mean mapping (with a dimension of ), used to adjust the benchmark position of the mean and enhance the model fitting ability. : The weight matrix of the variance mapping (with a dimension of ), mapping the implicit feature to the variance space. For example, generate the uncertainty parameter corresponding to the decision according to the "policy ambiguity" feature. : The bias vector of the variance mapping (with a dimension of ), adjusting the variance benchmark. : The activation function ( ), ensuring that the variance is always positive, meeting the mathematical requirements of the probability distribution.

[0092] Policy sampling is achieved, for example, through : The square root of the variance matrix ( ), converting the variance to the standard deviation to control the scale of the sampling noise. : Element-wise multiplication (Hadamard product), adding the product of the standard deviation and random noise to the mean to generate the final policy sample .

[0093] In the above scheme, the "deterministic tendency" and "randomness range" of the policy are explicitly characterized by the mean and variance . For example, in the reimbursement process, when the policy terms are clear, is small, and the agent tends to execute a fixed audit policy; when the policy is updated frequently, increases, and the agent generates diverse policies through sampling to adapt to the changes. Traditional direct sampling (such as ) is non-differentiable and cannot backpropagate gradients; reparameterization moves the randomness to independent variables , making differentiable with respect to and , thus supporting end-to-end training (such as optimizing parameters like ). When the policies are diverse, by adjusting the magnitude of , the model can balance between "exploitation (using known optimal policies)" and "exploration (exploring new policies)", and compared with deterministic policies (such as using only as the output), it significantly improves the decision-making adaptability of the agent in complex business scenarios (such as fuzzy rules and dynamic environment changes).

[0094] In step 432 above, through reparameterization, the policy is represented as a multivariate Gaussian distribution, endowing the agent with dynamic decision-making and adaptive learning capabilities. On the one hand, using the mean vector and the variance matrix to characterize the core tendency and uncertainty of the policy respectively, enabling the agent in business such as reimbursement audits to not only determine the basic audit policy according to features such as "policy compliance" (reflected by ), but also dynamically adjust the decision randomness according to the "policy ambiguity" (controlled by ), significantly enhancing the flexible response ability in complex scenarios; on the other hand, the reparameterization technique separates the randomness into independent sampling variables , making the policy generation process differentiable and supporting end-to-end training optimization, ensuring that the model can accurately adjust the weight matrices , Parameters such as these continuously improve the accuracy and adaptability of policy generation. This technological breakthrough in probabilizing policies effectively solves the problem of decision-making rigidity in scenarios with vague rules and changing environments compared to traditional deterministic policies, providing core technical support for agents to make efficient and robust decisions in dynamic business environments.

[0095] Optionally, the policy optimization component includes: a scenario representation component, a policy decomposition component, a truncation component, and a fine-tuning component. Step 5 specifically includes the following steps: Step 51: Through the scenario representation component, based on the generalized advantage estimation model, determine the scenario value representation, where the scenario value representation represents the estimated value of the advantage function of the cumulative discounted reward; Step 52: Based on the policy decomposition component, according to the hierarchical value decomposition network, perform probability density decomposition processing on the behavior feature policy to obtain the old and new policy distribution ratio matrix; Step 53: Through the truncation component, based on the scenario value representation, perform dynamic truncation processing on the old and new policy distribution ratio matrix to obtain the advantage estimation value adapted to the scenario; Step 54: Through the fine-tuning component, based on the advantage estimation value, perform parameter fine-tuning processing on the behavior feature policy to achieve online optimization of the behavior feature policy.

[0096] Preferably, the policy optimization component is constructed based on reinforcement learning and deep neural network technologies: the scenario representation component relies on the generalized advantage estimation (GAE) model and the LSTM time series network to realize the estimation of the advantage function of the cumulative discounted reward; the policy decomposition component uses the hierarchical value decomposition network (such as the hierarchical Actor-Critic architecture) and combines variational inference to realize the probability density decomposition of the policy; the truncation component is based on the dynamic truncation mechanism of PPO (proximal policy optimization) and fuses the scenario value representation to perform cropping on the old and new policy distribution ratio matrix; the fine-tuning component uses the Adam optimizer and gradient clipping technology to realize the online update of the policy parameters. The overall architecture is deployed in the TensorFlow framework to support distributed policy optimization calculations.

[0097] Preferably, when the scenario representation component in step 51 performs GAE-LSTM hybrid modeling processing on the scenario information to generate the scenario value representation, it performs GAE-LSTM hybrid modeling processing on the scenario information (including timestamp, policy version, agent state, etc.), captures the temporal dependence of the scenario features through the LSTM network (such as the periodic law of the "end-of-month settlement period"), and then calculates the estimated value of the advantage function based on the generalized advantage estimation (GAE) model to generate the scenario value representation. The specific algorithm formula is as follows: Calculation of the temporal difference error: , where: is the time step The TD error measures the deviation of the current state value estimate; is the time step The immediate reward obtained (such as the processing efficiency score for "submitting a reimbursement form"); is the discount factor (ranging from [0, 1]), used to discount the present value of future rewards; , are respectively the state , value function estimate.

[0098] GAE Advantage Estimation: , where: is the GAE-smoothed advantage estimate value; is the GAE parameter (smoothing coefficient, ranging from [0, 1]), used to control the bias-variance trade-off of the advantage estimate.

[0099] In summary, the generated scenario value representation contains both the immediate reward of "submitting a reimbursement form" and reflects its long-term impact on subsequent "financial review", providing value signals at multiple time scales for policy optimization.

[0100] Therefore, in step 51 above, by using GAE-LSTM hybrid modeling to process scenario information, the problems of large variance in traditional advantage estimation and insufficient utilization of temporal information are effectively solved. The LSTM network can accurately capture the temporal dependencies of scenario features and identify complex scenario patterns such as "surge in business volume during the end-of-month settlement period" and "rule changes during the policy transition period"; while the Generalized Advantage Estimation (GAE) model combines the temporal difference error with the smoothing coefficient, which can not only quantify the immediate benefits of behaviors such as "submitting a reimbursement form", but also accurately evaluate the impact of behaviors on subsequent business processes through the discount factor and long-term cumulative error. This multi-time scale value signal generation method can provide a more comprehensive and stable guiding basis for policy optimization compared to single immediate reward calculation, significantly improving the value judgment accuracy of the agent in dynamic business scenarios.

[0101] Preferably, when the policy decomposition component in step 52 performs hierarchical value decomposition processing on the behavioral feature policy to generate the new and old policy distribution ratio matrix, it performs hierarchical value decomposition processing on the behavioral feature policy. Relying on a hierarchical value decomposition network (such as the H-DQN architecture), the complex policy is decoupled into sub-policy spaces according to business levels (such as "material review - approval allocation"), and the probability density ratio of the new and old policies is calculated through variational inference. The specific algorithm formula is: New and old policy distribution ratio: , where: is an element of the new and old policy distribution ratio matrix, representing the new policy and the old policy selecting an action at state ​ Probability ratio; is the new policy parameter, is the old policy parameter (retaining historical policy information through EMA update); and are the conditional probabilities of the new and old policies selecting action in state respectively.

[0102] KL divergence regularization: , where: is the variational posterior distribution, is the prior distribution (usually a standard normal distribution), used to constrain the smoothness of the sub-policy distribution.

[0103] Therefore, the generated ratio matrix of the new and old policy distributions accurately reflects the impact of sub-policy updates such as "material compliance inspection" on the overall policy, solving the problem that traditional decomposition cannot handle hierarchical operations.

[0104] In step 52 above, the probability density of the behavior feature policy is decomposed using a hierarchical value decomposition network, integrating the hierarchical structure of the enterprise's operations into the policy optimization process. Through variational inference and KL divergence regularization, this step can break down complex business policies such as reimbursement audits into sub-policy spaces such as "material review" and "permission allocation", and accurately calculate the probability distribution ratios of the new and old policies at each level. This hierarchical processing method breaks the "flat" limitation of traditional policy decomposition, enabling policy updates to act specifically on specific business processes. For example, when adjusting the "material compliance inspection" sub-policy, it will not overly affect the stability of other processes, effectively improving the precision and interpretability of policy updates and reducing decision-making risks caused by global policy adjustments.

[0105] Preferably, in step 53, when the truncation component performs dynamic truncation processing on the ratio matrix of the new and old policy distributions and the scenario value representation to generate a scenario-adapted advantage estimate, the ratio matrix of the new and old policy distributions and the scenario value representation are dynamically truncated. Based on the adaptive truncation mechanism of the PPO algorithm, a target function is defined to generate a scenario-adapted advantage estimate: , where: is the value of the truncated target function, used to limit the policy update amplitude; is an element of the ratio matrix of the new and old policy distributions, representing the current policy update amplitude; is the generated scenario value representation, guiding the policy update direction; is the truncation function, restricting to the interval [a, b]; is the truncation threshold (e.g., 0.2), which dynamically adjusts the allowable range for policy update (automatically shrinks when the scene uncertainty is high).

[0106] Therefore, by dynamically adjusting Suppress aggressive policy updates, generate advantage estimates adapted to the scene, and ensure the stability of policy updates in complex scenarios.

[0107] In summary, in step 53, based on the dynamic truncation mechanism of the PPO algorithm, the ratio matrix of the old and new policy distributions is processed in combination with the scene value representation, providing an adaptive stability guarantee for policy updates. By dynamically adjusting the truncation threshold , the system can automatically tighten the policy update range according to the scene uncertainty (such as the high volatility during the "policy transition period") to avoid decision-making oscillations caused by excessive exploration; in normal scenarios, the threshold is moderately relaxed to accelerate policy optimization. This dynamic balance mechanism can better adapt to the complex and changing business environment compared with the truncation method with a fixed threshold, ensuring both the policy robustness of the agent in risk scenarios and not sacrificing its optimization efficiency in stable scenarios, significantly enhancing the environmental adaptability of policy updates.

[0108] Preferably, in step 54, when the fine-tuning component performs policy parameter fine-tuning processing on the advantage estimate value to generate an optimized behavior feature policy, the policy parameter fine-tuning processing is performed on the advantage estimate value adapted to the scene, and the advantage estimate value adapted to the scene and the ratio matrix of the old and new policy distributions are substituted into the PPO objective function: , where: is the final optimized objective function, balancing the policy update benefit and entropy regularization; is the entropy coefficient (e.g., 0.01), controlling the weight of the policy entropy to enhance the policy exploration ability; is the policy entropy, measuring the randomness of action selection. Through the Adam optimizer (update formula: , is the learning rate) and gradient clipping (such as limiting the gradient norm within 5.0) to update the policy parameters, and maintaining the old policy through EMA ( , is the smoothing coefficient). The finally generated optimized policy integrates the "exploration-exploitation" dynamic balance mechanism and continuously adapts to the real-time business scenario.

[0109] In summary, in the above step 54, by combining the PPO objective function, the Adam optimizer, and gradient clipping techniques to fine-tune the policy parameters, the efficiency and reliability of policy optimization are achieved. The introduction of the entropy regularization term enables the agent to actively explore diverse strategies in situations with ambiguous rules or new scenarios. For example, when the reimbursement rules are unclear, it tries different audit processes. The combination of the Adam optimizer and gradient clipping ensures the stability of parameter updates, preventing gradient explosion or vanishing problems. At the same time, the exponential moving average (EMA) for maintaining the old policy provides a "safe rollback" mechanism for policy optimization, avoiding performance degradation caused by a single update error. Overall, this step enables the behavioral feature policy to dynamically balance between "exploring new strategies" and "exploiting mature experience", continuously enhancing the agent's decision-making ability and long-term benefits in complex business scenarios.

[0110] Taking the application in the enterprise financial reimbursement scenario as an example, the above solution is explained exemplarily.

[0111] Step 1: Obtain and parse the scenario requirements to generate the target task sequence The user inputs the scenario requirement description through the front-end human-computer interaction component: "I need to submit a business trip reimbursement for Shenzhen in June 2025, including catering and transportation expenses, and apply for a fast approval channel due to the project urgency." The requirement collection component captures the text input through the Vue.js interface and confirms the intention through DeepSpeech speech recognition; the semantic role labeling model (RoBERTa-large) parses out the core entities: {Subject: employee, Action: submit, Object: business trip reimbursement for Shenzhen in June 2025, Additional conditions: urgent approval, including catering and transportation expenses}; the sequence generation component analyzes the temporal dependencies through GraphSAGE and maps "urgent approval" to the priority processing rule in combination with the enterprise reimbursement DAG process.

[0112] Generate the target task sequence: ["Retrieve the business trip reimbursement policy for Shenzhen in 2025 → Fill in the business trip reimbursement form (mark as urgent) → Upload catering and transportation invoices → Trigger fast material verification → Submit to the department head for urgent review → Financial urgent review"].

[0113] Step 2: Dynamically perceive the environment to generate the semantic topology network and the scenario graph The target scenario is the enterprise financial system, and the current environmental characteristics include: 1. The financial system real-time pushes the policy update of "new mandatory items for e-invoices in June 2025"; 2. The agent monitors that the system load soars during the end-of-month settlement period (CPU occupancy rate > 80%); 3. The employee terminal behavior data shows that the "e-invoice upload failure" event occurs frequently (12 times in the past 15 minutes).

[0114] The environmental detector pulls the policy logs of the OA system through the Feign client, the Prometheus collects system metrics, and the Flutter buried points capture terminal behaviors to form the original data stream; the sliding window frequency counter (Flink + Count-MinSketch) identifies that the co-occurrence frequency of "electronic invoice upload" and "policy update" is 0.8 (co-occurring 5 times within a 15-second window); the dynamic scenario graph generator calculates the edge weights using GraphSAGE. For example, the two-way probability weights of "electronic invoice upload" and "policy update" are:

[0115] The dynamic semantic topology network includes nodes "electronic invoice upload", "policy update", and "system load", and the edge weights reflect the collaboration intensity; the dynamic scenario graph is displayed in three-dimensional space (x-axis = financial policy, y-axis = timestamp, z-axis = system resources), and the node "electronic invoice upload" is marked red due to high-frequency failure events (strong risk association).

[0116] Step 3: Hierarchically model the behavior space to generate features with scene awareness Based on the dynamic scenario graph in Step 2, the behavior space includes behaviors such as "invoice upload", "policy verification", and "system load monitoring".

[0117] The semantic extraction component calculates the node features using GCNConv. For example, the "electronic invoice upload" node fuses the policy update feature (the semantic vector of "required fields" extracted by BERT) and the system load feature (the numerical feature corresponding to the CPU occupancy rate); the multi-level feature generation component calculates the topological attention weight of "upload failure" and "policy update" as 0.7 through the GraphSAGE + GAT hybrid architecture, and the temporal attention is enhanced to 0.9 due to the "end-of-month settlement period"; the feature alignment component sets the semantic association weight of the task objectives "urgent reimbursement" and "quick material verification" behaviors to 0.85 through the Transformer encoder.

[0118] The behavior semantic feature vector with scene awareness is 768-dimensional. The key dimensions of the "electronic invoice upload" feature include: policy compliance (0.82), system risk (0.75), and temporal urgency (0.91).

[0119] Step 4: Map the features to the policy space to generate behavior feature policies The multi-head linear transformation component calculates the Q / K / V vectors using 8 heads. For example, the first head focuses on "policy compliance" to generate a query vector, and the fifth head focuses on "system risk" to generate a key vector; the cross-agent attention fuses the historical collaboration matrix of the finance and department agents (e.g., the interaction weight of the finance agent for "invoice verification" is 0.6); the temporal mask matrix suppresses the reverse association of "upload invoice after verification first" (the weight is set to 0.1); the policy generation component compresses the features into a 256-dimensional latent vector z through VAE, and then generates a multivariate Gaussian distribution policy through reparameterization: Mean μ: [Prioritize e-invoices → Trigger automatic compliance check → Skip non-mandatory item verification] Variance σ²: Increases when the policy ambiguity is high (e.g., σ² = 0.3), allowing a 5% probability to attempt the manual review process The behavioral feature policy is represented as N(μ, σ²). For example, in the scenario of "e-invoice upload failure", the policy samples to generate an action sequence of "automatically retry upload + synchronously trigger the manual assistance channel".

[0120] Step 5: Online optimize the policy based on the scenario value When the system detects that the "e-invoice mandatory item policy" is in the transition period (rules are ambiguous), trigger policy optimization.

[0121] The scenario representation component calculates the advantage estimate value using GAE-LSTM. For example, the advantage value A_t of the "manual assistance upload" behavior is 0.7 (immediate reward +0.5, long-term impact +0.2); the policy decomposition component decomposes the policy into "upload process" and "compliance verification" sub-policies, and calculates the ratio matrix r(θ,θ_old) of the new and old policies = 1.2 (the new policy adds a manual review branch); the truncation component reduces the truncation threshold ε from 0.2 to 0.1 according to the scenario value (high uncertainty during the policy transition period), restricting the policy update amplitude; the fine-tuning component optimizes the parameters using the PPO objective function, increasing the probability of the "manual review" action (from 0.3 → 0.5), while maintaining the entropy coefficient β = 0.01 to explore other solutions.

[0122] The optimized behavioral feature policy is N(μ', σ'²), where μ' sets "manual assistance upload" as the priority action, and σ'² = 0.2 (0.1 lower than before optimization), reducing the system load pressure while ensuring compliance. For example, during actual execution, the agent first guides the employee to use the manual assistance channel, and at the same time attempts to automatically repair the upload failure problem with a 20% probability.

[0123] The above description is only a preferred embodiment of the present application and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solution formed by mutually replacing the above features with the technical features (but not limited to) disclosed in the present application that have similar functions.

Claims

1. An intelligent agent strategy generation and online optimization method based on dynamic scene perception, characterized in that Including: Step 1: Obtain the description of the user's scenario requirements, and parse it to generate a target task sequence that can be executed by an intelligent agent deployed in the target scenario; Step 2: Dynamically perceive the current environmental characteristics of the target scenario to generate a dynamic semantic topology network and generate a dynamic scene graph accordingly; Step 3: Based on the dynamic scene graph, hierarchically model the association relationships of the behavior space corresponding to the intelligent agent to generate behavior semantic features with scene perception; Step 4: Map the behavior semantic features with scene perception to the intelligent agent policy representation space to obtain a behavior feature policy for controlling the intelligent agent to execute the target task sequence; Step 5: According to the determined scene value representation, decompose the behavior feature policy to obtain an advantage estimate value adapted to the scene to online optimize the behavior feature policy.

2. The method according to claim 1, wherein The specific content of the above Step 1 includes: Step 11: Obtain the description of the user's scenario requirements; Step 12: Perform entity extraction processing on the description of the scenario requirements to obtain a task target entity set; Step 13: Generate a target task sequence that can be executed by the intelligent agent according to the task target entity set.

3. The method according to claim 2, wherein The specific content of the above Step 13 includes: Step 131: Perform temporal dependency analysis processing on the task target entity set to obtain a task execution priority queue; Step 132: Perform atomic task decomposition processing on the task execution priority queue to generate a target task sequence that can be executed by the intelligent agent, and the target task sequence includes task execution parameter constraints.

4. The method according to claim 1, wherein The specific content of the above Step 2 includes: Step 21: Based on an environmental detector, perform real-time sampling processing on the environmental feedback data to dynamically perceive the environmental characteristics of the target scenario and obtain an original perception data stream; Step 22: Through a sliding window frequency statistic, perform behavior event recognition processing on the original perception data stream to obtain behavior co-occurrence frequency data with timestamps; Step 23: Generate a dynamic scene graph according to the behavior co-occurrence frequency data with timestamps.

5. The method according to claim 4, wherein The specific content of the above Step 23 includes: Step 231: Perform two-way probability weighting processing on the behavior co-occurrence frequency data to generate a dynamic semantic topology network, where nodes represent behavior types and edge weights represent behavior cooperation intensities; Step 232: Perform spatial dimension mapping processing on the dynamic semantic topology network to generate a dynamic scene graph, and the dynamic scene graph represents the topological association intensity of behavior nodes in the scene space.

6. The method according to claim 3, wherein The specific content of the above Step 3 includes: Step 31: Generate a node feature matrix according to the dynamic scene graph, and the node feature matrix represents the semantic vectors of each behavior type; Step 32: Generate a multi-level representation of the behavior space according to the node feature matrix; Step 33: According to the task target semantic mapping model, perform feature alignment processing on the task execution priority queue to generate a behavior semantic feature vector with scene perception.

7. The method according to claim 6, characterized in that, The specific content of the above Step 32 includes: Step 321: Based on the topological neighborhood aggregation algorithm, perform iterative update processing on the node feature matrix to generate a feature matrix containing neighborhood cooperation information; Step 322: Perform hierarchical attention weighting processing on the feature matrix containing neighborhood cooperation information to generate a multi-level representation of the behavior space to represent the association relationships of the intelligent agent behavior space.

8. The method according to claim 1, characterized in that, Step 4 specifically includes the following: Step 41: Perform multi-head linear transformation processing on the behavior semantic features with scene perception to obtain a behavior feature interaction matrix; Step 42: Generate a temporal constraint mask matrix in the policy representation space according to the behavior feature interaction matrix; Step 43: Generate a behavior feature policy for the control agent according to the temporal constraint mask matrix, and the behavior feature policy is represented in the form of multivariate Gaussian distribution parameters.

9. The method according to claim 8, wherein Step 41 specifically includes the following: Step 411: Perform multi-head linear transformation processing on the behavior semantic features with scene perception to generate a feature triple including query / key / value; Step 412: Perform attention mechanism calculation processing on the feature triple based on the cross-agent interaction weight matrix to obtain a behavior feature interaction matrix.

10. The method according to claim 1, wherein Step 5 specifically includes the following: Step 51: Determine the scene value representation based on the generalized advantage estimation model, and the scene value representation represents the advantage function estimation value of the cumulative discounted reward; Step 52: Perform probability density decomposition processing on the behavior feature policy according to the hierarchical value decomposition network to obtain a new and old policy distribution ratio matrix; Step 53: Perform dynamic truncation processing on the new and old policy distribution ratio matrix based on the scene value representation to obtain a scene-adapted advantage estimation value; Step 54: Perform parameter fine-tuning processing on the behavior feature policy based on the advantage estimation value to realize the online optimization of the behavior feature policy.

Images