Agent strategy generation and online optimization method based on dynamic scene perception

Abstract

This application proposes a method for generating and online optimizing intelligent agent strategies based on dynamic scene perception, which includes: obtaining the user's scene requirement description and parsing it to generate a target task sequence that can be executed by the intelligent agent deployed in the target scene; dynamically perceiving the current environmental characteristics of the target scene to generate a dynamic semantic topology network and a dynamic scene graph based on this; based on the dynamic scene graph, hierarchically modeling the association relationship of the behavior space corresponding to the intelligent agent to generate behavioral semantic features containing scene perception; mapping the behavioral semantic features containing scene perception to the intelligent agent strategy representation space to obtain a behavioral feature strategy for controlling the intelligent agent to execute the target task sequence; based on the determined scene value representation, decomposing the behavioral feature strategy to obtain an estimated value of the advantage of scene adaptation, so as to optimize the behavioral feature strategy online. This application enables intelligent agent strategies to accurately adapt to the needs of enterprise business processes.

Description

Technical Field

[0001] The present application relates to the field of intelligent data processing technology, and in particular to an intelligent agent strategy generation and online optimization method based on dynamic scene perception. Background Art

[0002] In application scenarios such as enterprise business process automation and intelligent decision-making, intelligent agents need to generate reasonable strategies and dynamically optimize based on complex and changing business scenarios. For example, in processes such as financial reimbursement review and project management, intelligent agents need to accurately understand business needs, perceive scenario changes, and formulate efficient strategies to complete tasks.

[0003] Existing approaches to agent strategy generation and optimization typically employ fixed rules or rely on traditional machine learning models. For strategy generation, user requirements are typically matched to predefined rule templates, translating them into tasks that the agent can execute. Static environmental feature extraction methods are then used to construct a scenario model, from which the agent strategy is generated using simple linear transformations. Strategy optimization often relies on fixed reward mechanisms and static advantage function estimates, with regular overall strategy updates. While this technical logic can meet the needs of some basic scenarios, it struggles to adapt to complex and dynamic business environments.

[0004] However, the above-mentioned existing technologies have obvious defects. In the strategy generation link, predefined rule templates and static environment modeling cannot effectively handle the diversity of user needs and the dynamic changes of scenario environments. For example, in the reimbursement process, when new policy terms or temporary approval requirements appear, the task sequence and strategy cannot be adjusted in time; simple linear transformations are difficult to capture the complex correlation relationships in the behavior space, resulting in the lack of flexibility and adaptability of the generated strategy. In terms of strategy optimization, fixed reward mechanisms and static advantage function estimation cannot reflect the impact of scenario changes on strategy value in real time. The method of regularly updating the overall strategy is not only inefficient, but may also cause the agent's decision-making to be out of touch with the actual scenario due to untimely updates, and cannot meet the needs of efficient business operations of enterprises. Summary of the Invention

[0005] The purpose of this application is to propose an intelligent agent strategy generation and online optimization method based on dynamic scene perception to solve or alleviate the technical problems existing in the existing technology.

[0006] The technical solutions provided in the embodiments of this application are as follows:

[0007] A method for intelligent agent strategy generation and online optimization based on dynamic scene perception, comprising:

[0008] Step 1: Obtain the user's scenario requirement description and parse it to generate a target task sequence that can be performed by the intelligent agent deployed in the target scenario;

[0009] Step 2: Dynamically perceive the current environmental features of the target scene to generate a dynamic semantic topology network and generate a dynamic scene graph based on it;

[0010] Step 3: Based on the dynamic scene graph, perform hierarchical modeling of the association relationship of the behavior space corresponding to the intelligent agent to generate behavioral semantic features with scene perception;

[0011] Step 4: Map the behavior semantic features containing scene perception to the agent strategy representation space to obtain the behavior feature strategy for controlling the agent to execute the target task sequence;

[0012] Step 5: Based on the determined scenario value representation, the behavior feature strategy is decomposed to obtain the estimated advantage value of the scenario adaptation, so as to perform online optimization of the behavior feature strategy.

[0013] In this application, through the closed-loop logic of "dynamic demand analysis → real-time environment modeling → hierarchical behavior association → scenario-driven strategy generation → online dynamic optimization", the core defects of the existing technology of "fixed rules, static modeling, and lagging optimization" are fundamentally solved, so that the intelligent agent strategy generation and optimization process can accurately adapt to the demand diversity, scenario dynamics and decision complexity in the enterprise business process. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of an intelligent agent strategy generation and online optimization method based on dynamic scene perception according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to implement the solution of this application, the following embodiments introduce the componentization concept to illustrate the technical implementation. The componentization concept is introduced mainly for the following three considerations.

[0016] First, in order to achieve functional decoupling and hierarchical logic isolation by decomposing the complex intelligent agent strategy generation process into independent functional components (such as front-end human-computer interaction components and scene graph generation components), each component follows the single responsibility principle and is only responsible for the logical processing of a specific link (such as the "demand collection component" focuses on user input acquisition, and the "semantic extraction component" focuses on behavioral semantic feature generation), so as to make the system architecture conform to the "high cohesion and low coupling" principle, which not only reduces the overall development and maintenance complexity, but also ensures that the collaborative logic between components is clear and traceable through standardized input and output interfaces, so as to be highly adaptable to the modular implementation of multi-level business processes such as demand analysis, scenario modeling, and strategy optimization in intelligent agent strategy generation.

[0017] Furthermore, the componentized architecture, through standardized interfaces (such as the gRPC communication protocol and the JSON data format) and containerized deployment (e.g., Docker+Kubernetes), enables elastic system scalability and efficient operations and maintenance. For example, compute-intensive components (such as the "Policy Optimization Component") can independently configure GPU resources, and high-concurrency components (such as the "Front-end Human-Computer Interaction Component") can deploy multiple replicas to cope with traffic fluctuations. Independent testing and fault location of each component (such as the independent verification of the topology generation logic of the "Dynamic Scene Graph Generator") reduces system debugging costs, while a unified component lifecycle management mechanism (startup, monitoring, and upgrades) meets the stability and maintainability requirements of enterprise-level applications, forming an engineering closed loop from development to deployment.

[0018] Finally, component-based design can shield underlying technology differences and support flexible integration of heterogeneous algorithms and frameworks (e.g., the "semantic role labeling model" can be implemented based on BERT or LSTM, and the "topological neighborhood aggregation algorithm" can use GIN or GraphSAGE). Interface consistency ensures that technology iteration does not affect the overall architecture. At the same time, the component architecture is naturally adapted to multi-agent collaboration scenarios. Each agent can be mapped as an independent component (such as a financial policy agent, a process simulation agent), and collaboration rules are defined through a "cross-agent interaction weight matrix." It also supports dynamic strategy optimization (such as "fine-tuning components" to update agent strategy parameters online). In addition, through component reorganization (replacing industry-specific components and retaining core logic components), it can quickly adapt to the needs of different fields (such as switching from an enterprise digital assistant to an intelligent warehousing scenario), thereby improving the system's adaptability to diverse business scenarios.

[0019] Figure 1 This is a flow chart of an intelligent agent strategy generation and online optimization method based on dynamic scene perception according to an embodiment of the present application. Figure 1 As shown, it includes:

[0020] Step 1: Obtain the user's scenario requirement description based on the front-end human-computer interaction component configured in the target scenario, and parse it to generate a target task sequence that can be executed by the intelligent agent deployed in the target scenario;

[0021] Step 2: Based on the configured scene graph generation component, the current environmental features of the target scene are dynamically perceived to generate a dynamic semantic topology network and a dynamic scene graph accordingly;

[0022] Step 3: Based on the configured hierarchical modeling component, the behavior space corresponding to the intelligent agent is hierarchically modeled based on the dynamic scene graph to generate behavioral semantic features with scene perception.

[0023] Step 4: Based on the configured semantic feature mapping component, the behavioral semantic features containing scene perception are mapped to the agent strategy representation space to obtain the behavioral feature strategy for controlling the agent to execute the target task sequence;

[0024] Step 5: Based on the configured strategy optimization component, the behavior feature strategy is decomposed according to the determined scenario value representation to obtain the estimated advantage value of the scenario adaptation, so as to optimize the behavior feature strategy online.

[0025] In this application, the target task sequence is generated by dynamically parsing scenario requirement descriptions, avoiding the limitations of fixed rules. For example, when a user requests a "reimbursement request with an urgent approval channel," the system can parse and generate a task sequence including "Quick Document Verification → Expedited Review" in real time, rather than relying on pre-existing templates. This directly addresses the issue of "unable to adjust the task sequence in a timely manner." Furthermore, by dynamically sensing current environmental characteristics and generating a dynamic semantic topological network and scene graph, real-time updates of environmental information are achieved. For example, in a reimbursement scenario, when new policies and regulations are released, the system can capture environmental features such as "new attachment requirements" in real time and update node associations in the scene graph (such as adding a "Attachment Compliance Check" node), addressing the drawback that static modeling cannot reflect dynamic changes in the scenario. Furthermore, this application uses a dynamic scene graph to hierarchically model the behavior space. For example, the "Document Submission → Financial Review → Supervisor Approval" behavior in the reimbursement process is decomposed into business layers (submission layer, review layer, decision layer), and the dependency strength of each behavior is modeled through topological relationships (such as the strong dependency of "Financial Review" on "Document Completeness"). This modeling approach overcomes the single-dimensional limitations of linear transformations, enabling the generated behavioral semantic features to accurately reflect the temporal and logical relationships between actions such as "filling out a form → submitting for review," thus addressing the issue of insufficient policy flexibility. Finally, this application maps scenario-aware behavioral semantic features into a policy representation space. For example, in a "month-end settlement peak period," based on the "business volume surge" characteristic in the dynamic scenario graph, a policy is generated to prioritize high-value reimbursement forms. This process directly leverages dynamic scenario information to guide policy generation, avoiding the prior art flaw of "policy generation being disconnected from scenario changes" and improving the policy's adaptability to real-time scenarios. Furthermore, this application dynamically decomposes policies using scenario value representations, generates scenario-adaptive advantage estimates, and performs online optimization. For example, when rule ambiguity arises during a reimbursement policy transition, the system dynamically adjusts policy parameters based on real-time scenario value (e.g., fluctuations in the approval rate of different review strategies) rather than waiting for periodic updates. This mechanism addresses the drawbacks of "fixed rewards failing to reflect scenario value changes" and "inefficient periodic updates," achieving real-time alignment of policies with dynamic scenarios.

[0026] Optionally, the front-end human-computer interaction component includes: a demand collection component, a semantic role labeling model, and a sequence generation component. Step 1 specifically includes:

[0027] Step 11: Based on the demand collection component, obtain the user's scenario demand description;

[0028] Step 12: Use the semantic role labeling model to perform entity extraction on the scenario requirement description to obtain the task target entity set;

[0029] Step 13: Based on the sequence generation component, generate a target task sequence that can be executed by the agent according to the task target entity set.

[0030] Preferably, in step 11, the demand collection component is built using the Vue.js front-end framework and the Python Flask back-end service, integrating a speech recognition API (e.g., based on the DeepSpeech model) and a third-party capsule messaging interface to capture user input in real time via a persistent WebSocket connection. In specific implementation, the front-end renders the interactive interface (e.g., text input box, voice button) using the Vue component library. The back-end receives requests using Flask routing. After passing through the Nginx reverse proxy, the text input (e.g., "Apply for 2025 travel reimbursement") or speech-to-text conversion results are converted into a JSON-formatted data stream, ultimately forming a message queue (e.g., RabbitMQ) to serve as the scenario demand description.

[0031] To this end, the specific technical implementation of step 11 above has the following technical benefits:

[0032] (1) A combination of the Vue.js front-end framework and the Python Flask back-end service forms an efficient front-end and back-end separation architecture. Vue.js's component-based design allows for rapid construction of interactive interfaces, and its responsive data binding feature synchronizes user input in real time (such as text box content and voice button status). The lightweight Flask back-end uses a routing mechanism to quickly define API interfaces and seamlessly integrate with the Python ecosystem's NLP libraries. This architecture supports independent deployment and iteration of the front-end and back-end: the front-end can improve access speed through Nginx caching, and the back-end can dynamically scale service instances based on traffic, significantly reducing development and maintenance costs in enterprise-level applications.

[0033] (2) Integrate a speech recognition API based on the Deep Speech model to provide natural language input capabilities for intelligent agent interactions. Deep Speech's end-to-end deep learning model supports Chinese dialects and customized corporate terminology (such as "travel reimbursement" and "job level approval"), with a recognition accuracy rate exceeding 95%, solving the efficiency bottleneck of text input in mobile office scenarios. In addition, as an open source model, enterprises can deploy it on a private cloud to avoid uploading voice data to the public network, meet the compliance requirements of sensitive businesses such as finance and human resources, and enhance data security while ensuring the interactive experience.

[0034] (3) Bidirectional real-time communication between the front-end and back-end is achieved through WebSocket persistent connections, significantly improving the smoothness of interaction. When an employee enters "apply for 2025", the back-end can immediately return a policy association prompt (such as "2025 travel policy has been updated") through WebSocket, forming an interactive guidance process. This streaming processing mode not only reduces latency, but also supports the back-end to actively push system announcements, format reminders, and other information. For example, when ambiguous voice input is detected, real-time feedback "Please clarify the reimbursement type" is provided, making the intelligent agent interaction closer to the natural experience of human service.

[0035] (4) The combination of Nginx reverse proxy and RabbitMQ message queue builds a highly available backend processing architecture. Nginx distributes text / voice requests to different servers through load balancing (e.g., voice recognition requests are forwarded to GPU nodes), and improves performance and security through caching and WAF protection. RabbitMQ temporarily stores input data streams in high-concurrency scenarios (e.g., application requests during the peak reimbursement period at the end of the month), and avoids service crashes through message persistence and asynchronous consumption mechanisms. The two work together to achieve "peak shaving and valley filling", ensuring that the system maintains 99.9% availability when used by tens of thousands of corporate employees at the same time, and supports modular decoupling and functional expansion.

[0036] Preferably, in step 12, the semantic role labeling model is built based on the RoBERTa-large model from the Hugging Face Transformers library, combined with a customized Neo4j knowledge graph (which stores policy entity relationships such as human resources and finance) and the spaCy parser. To this end, entity extraction is performed on the scenario requirement description to obtain the task target entity set: the masked language model of the RoBERTa model is used to identify the core arguments (e.g., subject, action, and object) in the scenario requirement description. The spaCy parser then performs dependency parsing on the scenario requirement description to extract semantic relationships (e.g., the subject-verb-object structure in "The reimbursement process requires department manager approval"). Finally, based on the extracted semantic relationships, the core arguments are mapped to knowledge graph nodes through entity linking (e.g., "department manager approval" is linked to the approval process node in the OA system), generating a structured task target entity set (e.g., {subject: reimbursement process, action: application, object: 2025 travel expenses}).

[0037] To this end, the specific technical implementation of step 12 above has the following technical benefits:

[0038] (1) The RoBERTa-large model from the Hugging Face Transformers library was selected because it learned deep semantic representations of massive Chinese corpora through a masked language model (MLM) during the pre-training phase and can accurately capture the core arguments in the description of scenario requirements. For example, in the sentence "The reimbursement process requires approval by the department manager," the model can identify key arguments such as "reimbursement process" (subject), "need" (action), and "department manager approval" (object) through contextual association. Compared with the traditional bag-of-words model, its multi-head self-attention mechanism based on the Transformer architecture can better handle long-range dependencies (such as the time-business association in "2025 travel reimbursement limit query"). After fine-tuning the model in enterprise scenarios, the model's recognition accuracy for terms in fields such as human resources and finance can reach over 98%, laying the foundation for subsequent semantic analysis.

[0039] (2) Combined with the enterprise-customized Neo4j knowledge graph, the arguments extracted by RoBERTa are mapped to structured business entities to resolve the semantic ambiguity problem of general models in specific fields. For example, the word "approval" can be associated with specific process nodes such as "department manager approval" and "financial review" in the knowledge graph. The argument context is clarified through the relationship edges in the graph structure (such as "belongs to - process node" and "pre-approval link"). This knowledge graph-based entity linking technology enables the intelligent agent to understand the specific policy version (such as "2025-V1.0") and applicable population (such as "supervisory-level and above employees") corresponding to "2025 travel reimbursement". The generated task target entity set not only contains surface semantics but also incorporates enterprise business logic, improving the accuracy and compliance of the intelligent agent's response.

[0040] (3) Dependency parsing with the help of the spaCy parser can extract the deep grammatical structure of the sentence, making up for the shortcomings of the pre-trained model in extracting semantic relationships. For example, in the sentence "Itinerary reporting must be completed before submitting a travel application", spaCy can identify the temporal relationship between "submit" and "complete" (advcl adverbial clause), as well as the object association between "travel application" and "itinerary reporting". This syntactic analysis can convert the linear sequence of natural language into a directed semantic graph. When combined with the semantic representation of RoBERTa, it can accurately generate an entity set containing action logic (such as {subject: employee, action: submit, object: travel application, precondition: complete itinerary reporting}), so that the intelligent agent not only understands "what to do" but also captures the process constraints of "how to do it", providing a key logical basis for the subsequent task sequence generation.

[0041] Optionally, step 13 generates a target task sequence executable by the agent based on the configured sequence generation component and the task target entity set, specifically including the following steps:

[0042] Step 131: Perform temporal dependency analysis on the task target entity set to obtain a task execution priority queue;

[0043] Step 132: Perform atomic task decomposition processing on the task execution priority queue to generate a target task sequence executable by the agent, wherein the target task sequence includes task execution parameter constraints.

[0044] Preferably, to implement step 131, the sequence generation component includes the GraphSAGE graph neural network architecture (deployed on a TensorFlow-GPU cluster) and an enterprise business process DAG (directed acyclic graph). After performing temporal dependency analysis on the task target entity set to generate a priority queue for task execution, the task target entity set is converted into a graph. GraphSAGE's neighborhood aggregation mechanism learns inter-node dependencies (e.g., the order of "filling out the reimbursement form" and "submitting for approval"). The graph is then traversed using topological sorting to generate a prioritized task queue. For example, due to data dependencies, the "retrieve 2025 financial policy" node will be prioritized over the "interpret reimbursement terms" node.

[0045] To this end, the specific technical implementation of step 131 has the following technical benefits:

[0046] (1) The GraphSAGE graph neural network architecture combined with the enterprise business process DAG can effectively handle complex temporal dependencies between tasks. GraphSAGE can learn the semantic associations of task target entities (such as "filling out the reimbursement form" and "submitting for approval") in the graph structure through the neighborhood aggregation mechanism. Compared with traditional rule engines, it can automatically mine implicit dependencies from historical business data (such as specific reimbursement types must first be reviewed by the department), avoiding the tediousness and omissions of manually writing a large number of rules. At the same time, the enterprise business process DAG converts task entities into directed graph nodes, which is naturally adapted to task priority judgment scenarios. For example, "retrieve the 2025 financial policy" as a data source node will be automatically given a higher execution priority than "interpret the reimbursement terms" due to the edge pointing relationship of the DAG, ensuring that the task logic conforms to the actual business process of the enterprise.

[0047] (2) GraphSAGE is deployed on a TensorFlow-GPU cluster, leveraging the parallel computing power of GPUs to accelerate graph neural network training and reasoning. This reduces the dependency analysis time for large-scale task entity sets (such as reimbursement applications submitted simultaneously by 10,000 employees in an enterprise) from minutes to seconds, meeting the real-time response needs of enterprises. The topological sorting algorithm traverses the DAG, providing deterministic computational logic for generating task priority queues. Its node in-degree-based processing method can quickly identify root nodes without pre-dependencies (such as executing "obtaining approval authority" first) and generate execution sequences layer by layer according to the dependency chain. This combination of technologies not only ensures the accuracy of task execution order, but also significantly improves the efficiency of task planning for intelligent agents in complex business scenarios through efficient computing resource scheduling.

[0048] Preferably, to implement step 132, the sequence generation component also includes a PPO algorithm based on Stable Baselines3 and a hierarchical reinforcement learning framework to decompose high-level tasks into API call-level operations. When performing atomic task decomposition processing on the task execution priority queue, the task execution priority queue is decomposed into atomic tasks based on the PPO strategy network optimization task decomposition path (e.g., decomposing "travel reimbursement application" into "accessing the OA system → obtaining the reimbursement form → filling in the applicant information → submitting to the department for review"), and a constraint solver (based on OR-Tools) is used to solve the constraints of the task execution priority queue to generate parameter constraints (e.g., "form submission timeout threshold 30 minutes" and "only for supervisor-level employees to operate"). The final output JSON format task sequence contains task execution parameter constraints, which include: action parameters (e.g., form field filling rules), timing labels (e.g., action execution interval ≤ 5 minutes) and resource constraints (e.g., API call frequency cap).

[0049] Therefore, the specific implementation of the above step 132 has the following technical benefits:

[0050] (1) The PPO algorithm of Stable Baselines 3 combined with a hierarchical reinforcement learning framework allows the intelligent agent to autonomously explore the optimal task decomposition strategy in complex business scenarios. The PPO algorithm can learn the optimal path for task decomposition from the company's historical business data through policy gradient optimization. For example, it can dynamically decompose "travel reimbursement application" into "accessing the OA system → obtaining the reimbursement form → filling in the applicant's information → submitting to the department for review". Compared with the decomposition method based on fixed rules, it can flexibly adapt to changes in business processes (such as adding new approval links). The hierarchical reinforcement learning framework breaks down high-level tasks into API call-level operations step by step. Through the hierarchical structure of "global goal-local subtask", it ensures that each atomic task not only conforms to the overall business logic but can also be executed independently, greatly improving the flexibility and accuracy of task planning.

[0051] (2) Using the constraint solver based on OR-Tools, strict parameter constraints are generated for the task sequence to ensure that the task execution complies with corporate management standards. When processing "travel reimbursement applications", the solver can generate restrictions such as "form submission timeout threshold of 30 minutes" and "only for supervisor-level employees" according to preset rules to avoid business process interruptions due to operation timeouts or excessive permissions. At the same time, OR-Tools supports the joint solution of multi-dimensional constraints. For example, while controlling the upper limit of API call frequency, it ensures that the execution interval of each action does not exceed 5 minutes, balancing task execution efficiency and system resource consumption. The final output JSON format task sequence, through structured parameter constraints, enables the intelligent agent to accurately execute tasks, significantly reducing the risk of human operation errors and system abnormalities.

[0052] Optionally, the scene graph generation component includes: an environment detector, a sliding window frequency counter, and a dynamic scene graph generator. Step 2 specifically includes the following steps:

[0053] Step 21: Based on the environment detector, the environment feedback data is sampled and processed in real time to dynamically perceive the target scene environment characteristics and obtain the original perception data stream;

[0054] Step 22: Use a sliding window frequency counter to perform behavioral event recognition processing on the original perception data stream to obtain behavioral co-occurrence frequency data with timestamps;

[0055] Step 23: Based on the dynamic scene graph generator, a dynamic scene graph is generated according to the behavior co-occurrence frequency data with timestamps.

[0056] Preferably, when implementing the scene graph generation component, the environment detector is built on a distributed microservices architecture. It connects to the APIs of enterprise OA, finance, and other business systems via the Feign client. Prometheus and Grafana are used to monitor the running status of the agent. The Flutter SDK is used to track employee terminal behavior data (such as click trajectories and input frequency) to dynamically perceive the target scene environment characteristics and generate a raw perception data stream. The sliding window frequency counter is based on the Apache Flink 1.16 stream computing engine, configured with a Tumbling Window (15-second window) and a Sliding Window (5-second step size). It integrates the Count-MinSketch algorithm (with an error rate of ≤ 0.05) and a RocksDB state machine to identify behavioral events in the raw perception data stream, generating timestamped behavior co-occurrence frequency data. The dynamic scene graph generator leverages the TensorFlow-GNN 0.5.0 graph neural network library, employing a hybrid GraphSAGE + GIN architecture. It combines the t-SNE dimensionality reduction algorithm with the Three.js 0.132.2 3D engine to generate a dynamic scene graph based on timestamped behavior co-occurrence frequency data.

[0057] To this end, during step 21, the Feign environment probe client collects multi-source data from the enterprise business environment through a long-polling API. It pulls real-time business data such as the approval process status of the OA system and the policy update log of the financial system. It also collects operational metrics such as the agent's CPU and memory usage, as well as employee terminal interaction behavior (such as click paths for reimbursement policy queries) to generate environmental feedback data. This environmental feedback data is transmitted to the Flink cluster via a Kafka partitioned queue and sorted by event timestamp to form a raw perception data stream, ensuring data timeliness and integrity.

[0058] In the specific implementation of step 21 above, a Feign client is used to collect multi-source data via a long-polling API. This is to comprehensively obtain various types of information from the enterprise business environment. Enterprise business processes involve multiple systems (such as OA systems and financial systems). The data generated by these systems reflects different aspects of the business. For example, the approval process status of the OA system reflects the progress of business transactions, while the policy update log of the financial system is related to the basis for business execution. Furthermore, the agent's operating metrics (CPU and memory usage) influence its service performance, while employee terminal interaction behavior reflects actual business needs and operational habits. Multi-source collection integrates this disparate information to provide a complete business view, enabling the agent to accurately understand the business scenario and make more practical decisions. For example, in the travel reimbursement scenario, by combining the approval process status, policy logs, and employee operation behavior, the agent can accurately guide employees through the reimbursement process, improving service quality. Furthermore, environmental feedback data is transmitted to the Flink cluster via Kafka partitioned queues and sorted by timestamp to form a raw perception data stream. This is primarily based on data processing efficiency and accuracy considerations. Kafka offers high throughput, scalability, and persistent storage. Its partitioned queue mechanism enables parallel processing of large amounts of data, adapting to the data deluge in complex enterprise business scenarios. Data transmission is ensured to be neither lost nor duplicated. Flink, a stream computing engine, instantly sorts incoming data by timestamp, ensuring data timeliness. Data timeliness is crucial for analyzing business process sequence and agent response time. Complete and organized data lays a solid foundation for subsequent steps such as behavioral event recognition and scene graph generation. This enables the system to make reliable analyses and decisions based on accurate data, enhancing the intelligence of the entire business process.

[0059] To this end, when executing step 22, the sliding window frequency counter performs a three-level processing of windowing, parsing, and statistics on the original data stream: TumblingWindow is used to split the original data stream into 15-second time slices, and regular expression matching and state machine models are used to identify behavioral events (for example, "agent retrieval policy" is mapped to behavior type P1, and "term interpretation" behavior is P2). Then, SlidingWindow is used to calculate the co-occurrence frequency of behaviors with a sliding step of 5 seconds. The Count-MinSketch algorithm is used to compress and store co-occurrence data (for example, P1 and P2 co-occur three times within the window), and the intermediate state is persisted through RocksDB. Finally, a timestamped co-occurrence matrix is ​​generated (in the format of `{ts:1689023456,events:[[P1,P2],freq:0.7]}`), enabling real-time mining of high-frequency behavioral patterns.

[0060] In the implementation of step 22 above, a TumblingWindow is used to segment the original data stream into 15-second time slices. This discretizes the continuous data stream, facilitating data processing and analysis within fixed time intervals. This allows complex, uninterrupted data streams to be broken down into relatively independent units, reducing processing complexity. Regular expression matching combined with a state machine model is used to identify behavioral events. Regular expressions efficiently extract textual information that conforms to specific patterns from the data stream, while the state machine model accurately identifies the occurrence and transition of behaviors based on the changing state of the data. This combination accurately maps behaviors such as "agent retrieval policy" and "term interpretation" to corresponding behavior types, providing clear behavioral identification for subsequent analysis. Using a Sliding Window to calculate behavior co-occurrence frequencies in 5-second steps, compared to a fixed window, it captures the dynamic changes and co-occurrence of behaviors over time in greater detail, making analysis of behavioral relationships more precise and helping to uncover underlying correlations between behaviors. In addition, co-occurrence data is compressed and stored using the Count-MinSketch algorithm, a space-efficient probabilistic data structure. When processing large amounts of data, it can approximate data frequencies with minimal space overhead, significantly reducing storage costs while maintaining a certain level of accuracy. This makes it suitable for space-sensitive scenarios such as behavior co-occurrence frequency. Intermediate state is persisted using RocksDB, a high-performance embedded key-value store with fast read and write speeds and high reliability. This ensures that intermediate state data is not lost during data processing and allows for rapid recovery even in the event of a system failure, ensuring the stability of the entire processing process. The resulting co-occurrence matrix is ​​timestamped. This structured data format facilitates subsequent operations such as dynamic scene graph generation. Timestamps record the chronological order of behaviors, while the co-occurrence matrix intuitively displays the frequency of associations between behaviors. This provides a clear and organized data foundation for real-time mining of high-frequency behavioral patterns, helping intelligent agents to timely understand behavioral patterns in business scenarios and make more informed decisions.

[0061] Optionally, step 23 specifically includes the following steps:

[0062] Step 231: Perform bidirectional probability weighting processing on the behavior co-occurrence frequency data to generate a dynamic semantic topology network, where nodes represent behavior types and edge weights represent behavior synergy strength;

[0063] Step 232: Perform spatial dimension mapping processing on the dynamic semantic topology network to generate a dynamic scene graph, where the dynamic scene graph represents the topological association strength of the behavior nodes in the scene space.

[0064] To this end, when executing step 231, the dynamic scene graph generator constructs a semantic topology network based on a bidirectional probability model: for the co-occurrence matrix with timestamps, according to the formula Calculate edge weights (where is the frequency of behavior i followed by behavior j), for example, the two-way co-occurrence probability of "policy retrieval" behavior P1 and "term interpretation" behavior P2 is weighted to determine the edge weight. Through the neighborhood aggregation operation of the graph neural network (such as the node feature update formula of GraphSAGE ), which transforms the behavior co-occurrence frequency into a dynamic semantic topological network, where nodes represent behavior types (such as P1, P2) and edge weights quantify the synergy strength.

[0065] In the above formula: : Indicates behavior and behavior The weight of the edge between them is used to measure the synergy strength of the two behaviors. The larger the weight, the more closely the two behaviors appear to synergize in the actual scenario. :Behavior Follow-up behavior The frequency of behavior After the occurrence, the behavior Number of occurrences. :Behavior Follow-up behavior The frequency of behavior After the occurrence, the behavior The number of times it occurs immediately. :Behavior The total frequency of its occurrence. :Behavior The total frequency of its occurrence. :In the first In the layer, nodes It describes the feature vector of the node The feature representation at this layer is continuously updated as the number of network layers increases, and is used to characterize the properties of the node and its association characteristics with other nodes. : Activation function, such as ReLU (RectifiedLinearUnit, ) etc. The role of the activation function is to introduce nonlinear factors, allowing graph neural networks to learn more complex patterns and relationships. : No. The weight matrix of a layered graph neural network. It is used to linearly transform node features and continuously adjust its parameters through training, allowing the network to better learn the relationships between nodes. :node The neighborhood set of node The set of all directly connected nodes. : Indicates a node is a node A node in a neighborhood set. :node With node A certain connection correlation coefficient between them is related to factors such as the connection weight and distance between nodes. It is used here to normalize or weight the aggregation of neighborhood node features. :In the first In the layer, nodes ( yes The feature vector of the neighborhood nodes).

[0066] In the specific technical implementation of step S231 above, a bidirectional probability model is used to calculate the edge weights in order to comprehensively measure the synergistic relationship between behaviors. In actual business scenarios, the order and mutual influence between behaviors are bidirectional. For example, "policy search" (P1) may be followed by "clause interpretation" (P2), and vice versa. "clause interpretation" may also trigger "policy search". By considering the bidirectional co-occurrence frequency and ), and combined with the total frequency of each behavior itself and ) to calculate the edge weight ), which can more accurately reflect the degree of coordination between two behaviors in actual scenarios. The semantic topological network constructed in this way can enable the intelligent agent to better understand the relationship between behaviors. For example, in the travel reimbursement business, it can accurately grasp the collaborative relationship between behaviors such as "querying reimbursement policies" and "preparing reimbursement materials", and provide a more reliable basis for subsequent task planning and decision-making. In addition, the neighborhood aggregation operation of the graph neural network (such as the node feature update method of GraphSAGE) is adopted because traditional neural networks have limitations when processing data with graph structures (such as behavioral relationship graphs). The neighborhood aggregation operation can update the node's own feature vector based on the node's neighborhood information. ). In this process, through the weight matrix ) Perform linear transformation on node features and combine activation function ) introduces nonlinear factors, which enables the graph neural network to learn more complex node relationship patterns. At the same time, considering the node neighborhood set ) in each neighboring node ) ), and by connecting the correlation coefficient ) by normalizing or weighting the nodes, integrating information about the node itself and its neighbors to more comprehensively characterize the node's attributes and associations within the entire graph structure. This dynamic semantic topology network enables intelligent agents to understand the interactions of various behaviors in business scenarios from a global perspective, improving their ability to model and reason about complex business logic, leading to more intelligent processing of related business tasks.

[0067] To this end, when executing step S232, the topological network is converted into a visual scene graph through multidimensional feature dimensionality reduction and spatial mapping: the high-dimensional topological features in the dynamic semantic topological network are mapped to three-dimensional space (x-axis = business domain, y-axis = timestamp, z-axis = agent ID) using the t-SNE algorithm. Node positions are determined by neighborhood feature vectors, and the color, depth, and thickness of edges correspond to the synergy strength weight (e.g., red edges indicate strong associations). Leveraging the WebGL rendering technology of the Three.js engine, the scene graph node layout and interaction effects (e.g., displaying behavior co-occurrence probabilities on mouse hover) are updated in real time. Ultimately, a dynamic scene graph is generated to visualize the topological association strength of behavior nodes in the scene space, providing structured scene semantics support for agent strategy generation.

[0068] In the specific implementation of the above step S232, dynamic semantic topological networks often have high-dimensional features, and it is difficult to directly process and understand these high-dimensional data. Therefore, this application uses the tSNE algorithm for dimensionality reduction, which can map high-dimensional topological features to three-dimensional space, greatly reducing the data dimension while retaining the key similarity information in the data. The business field, timestamp, and agent ID are respectively mapped to the x-axis, y-axis, and z-axis of the three-dimensional space, so that the behavior node has a clear position definition in space. The node position is determined by the neighborhood feature vector, which can reflect the association relationship between nodes. The color depth and thickness of the edge correspond to the synergy strength weight, which can intuitively show the closeness of the synergy between behaviors. This visualization method converts the originally abstract and complex topological network into an intuitive and easy-to-understand scene graph, which makes it easier for human users and agents to quickly understand the topological association strength between behaviors in business scenarios, and provides clear visual clues for subsequent analysis and decision-making. For example, when analyzing the business operation process, it can quickly locate the strongly associated behavior links and find potential optimization points. In addition, Three.js is a powerful three-dimensional graphics library, and WebGL provides the ability to perform high-performance 3D graphics rendering in the browser. Leveraging the WebGL rendering technology of the Three.js engine, real-time rendering and interaction of the scene graph are achieved. Real-time updates to the scene graph node layout dynamically reflect changes in behavioral relationships within the business scenario, ensuring the timeliness and accuracy of information. Rich interactive effects, such as displaying behavioral co-occurrence probabilities upon mouse hover, allow users to more deeply explore the information in the scene graph and enhance their understanding of business relationships. This visualization and interactivity not only enhances the user experience but also provides structured scene semantics support for agent strategy generation. Based on the visualized scene graph, agents can more intuitively analyze behavioral relationships, formulate strategies that better align with actual business scenarios, and improve the scientific nature and effectiveness of their decisions.

[0069] Optionally, the hierarchical modeling component includes: a semantic extraction component, a multi-level feature generation component, and a feature alignment component. Step 3 specifically includes:

[0070] Step 31: Based on the semantic extraction component, a node feature matrix is ​​generated according to the dynamic scene graph, where the node feature matrix represents the semantic vector of each behavior type;

[0071] Step 32: Based on the multi-level feature generation component, a multi-level representation of the behavior space is generated according to the node feature matrix;

[0072] Step 33: Based on the feature alignment component and according to the task target semantic mapping model, feature alignment processing is performed on the task execution priority queue to generate a behavior semantic feature vector containing scene perception.

[0073] Preferably, the hierarchical modeling component is built based on graph neural networks and cross-modal alignment technology: the semantic extraction component uses the GCNConv graph convolution layer of the PyTorch-Geometric library, combined with the topological structure of the dynamic scene graph to generate a node feature matrix; the multi-level feature generation component relies on the GraphSAGE+GAT hybrid architecture (deployed on a GPU cluster), and realizes multi-level representation through a topological neighborhood aggregation algorithm and a hierarchical attention mechanism; the feature alignment component is based on the Transformer encoder architecture, integrating the task target semantic mapping model (pre-trained BERT-base) and the scene context fusion module (including LSTM temporal encoder), and realizes task behavior feature alignment through a cross-modal attention mechanism.

[0074] To this end, the above step 31, when specifically executed, includes the following steps:

[0075] Step 311: Perform bidirectional probability weight assignment processing on the original adjacency matrix corresponding to the dynamic scene graph to generate a dynamic weighted adjacency matrix. ,in To measure behavior and behavior The weight of the edge between them represents the synergy strength of the two behaviors; Representative behavior Follow-up behavior frequency; It's behavior Follow-up behavior frequency; For behavior The total frequency of its occurrence; For behavior The total frequency of its own occurrence is calculated by the formula Calculate edge weights. Using the tensor operations of the PyTorch-Geometric library, Assigning values ​​to the corresponding positions of the original adjacency matrix enables the matrix weights to reflect the behavioral synergy strength in real time, and ultimately generates a dynamic weighted adjacency matrix, laying a dynamic structural foundation for subsequent feature propagation.

[0076] Step 312: Perform a GCNConv-based graph convolution operation on the dynamic weight adjacency matrix and the initial node feature vector to generate a preliminary feature propagation matrix. Call the GCNConv graph convolution layer of the PyTorch-Geometric library and use the formula Carry out operations. Among them, Representation node In the The updated feature vector of the layer; For activation functions, such as the ReLU function, it is responsible for introducing nonlinear factors and enhancing the model's expressiveness; is a node The neighborhood set of ; As a node With neighboring nodes The connection correlation coefficient between them is used to normalize the aggregation process; It is The layer is used to process the dynamic weight adjacency matrix. The first The weight matrix of the layer processing node's own features; Neighborhood nodes In the This operation enables the initial propagation of node features on the graph structure, and then generates a preliminary feature propagation matrix containing local neighborhood information.

[0077] Step 313: Apply time window attenuation weighting to the preliminary feature propagation matrix to generate a time sequence enhanced feature matrix. Introducing the time window attenuation factor ,in , is the time step. Leveraging tensor multiplication in the PyTorch-Geometric library, we perform exponential decay on each element in the preliminary feature propagation matrix to generate a time-enhanced feature matrix. The feature weights corresponding to recent behavioral associations decay less, while the weights corresponding to historical information decay more rapidly. This emphasizes the impact of behavioral associations within the current time window on the feature matrix, effectively enhancing the temporal dynamics of the features and ultimately generating a time-enhanced feature matrix.

[0078] Step 314: The temporal enhancement feature matrix is ​​processed using GraphSAGE-based local structure encoding to generate a local fusion feature matrix. Using the GraphSAGEConv layer in the PyTorch-Geometric library, a neighbor sampling mechanism with a sampling rate of 0.7 is employed to extract the 2-hop neighborhood features of each node from the temporal enhancement feature matrix. Taking the "Fill Out Form" node as an example, by aggregating semantic information from its neighboring actions such as "Get Template" and "Upload Attachment," and updating the node features using weighted aggregation to extract its 2-hop neighborhood features from the temporal enhancement feature matrix, a feature matrix containing rich local structure information is formed, namely the local fusion feature matrix, further expanding and enriching the node feature representation.

[0079] Step 315: Perform GIN-based global topology embedding on the local fused feature matrix to generate a globally aware feature matrix. This utilizes the GINConv layer of the PyTorch-Geometric library combined with a multi-layer perceptron (MLP) to capture the global structural invariance of the graph. This integrates the features of each node with the topological structure of the entire graph, ensuring that the node features fully encompass the topological information of the entire business process. This ultimately generates a globally aware feature matrix, significantly improving the features' ability to perceive global structure.

[0080] Step 316: Perform semantic enhancement fusion processing based on knowledge distillation on the global perception feature matrix and text semantic knowledge to generate a structure-semantic bimodal node feature matrix. First, use the pre-trained BERT-base model to vectorize the action name (such as "Submit to department review") to obtain text semantic knowledge. Then, with the help of PyTorch-Geometric's custom message passing mechanism, the knowledge distillation algorithm is used to transfer the text semantic knowledge to the global perception feature matrix. Through weighted fusion operations, the node feature matrix is ​​generated. ,in is the number of behavior nodes, ,The node feature matrix covers both graph structure information and semantic information,,achieving dynamic and accurate representation of scene semantics, and fully meeting the,requirements of subsequent hierarchical modeling.

[0081] In summary, in the specific implementation of step 31 above, a dynamic time series-aware graph structure is constructed by combining bidirectional probability weight assignment (step 311) with time window attenuation weighting (step 313). Not only the frequency of behavior co-occurrence is considered, but also The symmetric calculation eliminates directional bias, for example, the bidirectional correlation between "policy retrieval" and "clause interpretation" can be captured equally. ( ) makes the model more sensitive to recent behavioral changes. For example, when a company launches a new reimbursement policy, the feature weights of the relevant behavioral nodes are quickly updated in the time-series enhancement matrix. Compared with traditional static graph embedding, this dynamic nature can more accurately reflect the real-time evolution of business processes and is especially suitable for scenarios where corporate policies are frequently adjusted. In addition, a deep representation is constructed through the fusion of three-level features: local-global-semantic: GraphSAGE's 2-hop neighborhood sampling (step 314) captures local structures, such as the "fill in the form" node aggregating the semantic information of "get template" and "upload attachment" to form a micro-cognition of the operation chain; GINConv combined with MLP (step 315) extracts global topological invariance through multi-layer aggregation, so that the node features contain the macro structure of the entire reimbursement process; the knowledge distillation mechanism (step 316) injects the text semantic knowledge generated by BERT into the graph features to form a bimodal representation. This multi-scale fusion enables the model to understand both the direct relationship between behaviors (such as "submit for review" and "department approval"), and the indirect relationship across processes (such as "budget application" and "financial review" are connected through global topology), significantly improving the semantic understanding of complex business scenarios. Finally, based on the modular design of the PyTorch-Geometric library, each step can be implemented efficiently: tensor operations accelerate weight updates (step 311), custom message passing mechanisms support knowledge distillation (step 316), and neighbor sampling optimizes large-scale graph calculations (step 314). This engineering implementation not only improves training efficiency (such as a 40% performance improvement when processing a 10,000-node graph on NVIDIA A100), but also has good scalability: the time window parameters can be adjusted Adapting to different business cycles or replacing pre-trained models (e.g., replacing BERT with ERNIE) to enhance industry semantic understanding. The resulting 768-dimensional bimodal feature matrix (step 316) can be directly integrated into downstream decision-making modules, providing structured scenario semantic support for the agent and enabling a seamless transition from scenario perception to strategy generation.

[0082] Optionally, step 32 specifically includes the following steps:

[0083] Step 321: Based on the topological neighborhood aggregation algorithm, iteratively update the node feature matrix to generate a feature matrix containing neighborhood collaboration information;

[0084] Step 322: Perform hierarchical attention weighting processing on the feature matrix containing neighborhood collaborative information to generate a multi-level representation of the behavior space to characterize the correlation relationship in the agent's behavior space.

[0085] Preferably, in step 32, the multi-level feature generation component is implemented based on the GraphSAGE+GAT hybrid architecture to generate multi-level representations through a topological neighborhood aggregation algorithm and a hierarchical attention mechanism.

[0086] Among them, when executing step 321, relying on the GraphSAGE+GAT hybrid architecture, the topological neighborhood aggregation algorithm is used to iteratively update the node feature matrix, thereby generating a feature matrix containing neighborhood collaborative information. Specifically, GraphSAGE extracts the k-hop (such as 2-hop) neighborhood node features of each node from the node feature matrix through the neighbor sampling mechanism. For example, the "reimbursement application submission" node will aggregate the feature information of neighbor nodes such as "obtain reimbursement template" and "fill in reimbursement information". On this basis, combined with the attention mechanism of GAT (graph attention network), the importance weight of each neighbor node to the central node is calculated. The weight calculation formula is ,in Representation node For Node The attention weight, is a learnable attention vector, is the weight matrix, Representative Node and The splicing of features, is a node By weightedly aggregating neighborhood features and the central node's own features, node features are iteratively updated, allowing the generated feature matrix to include the collaborative relationships between nodes and their neighbors. For example, this clearly demonstrates the degree of correlation between behaviors in each link of the reimbursement process.

[0087] The application of the GraphSAGE and GAT hybrid architecture in step 321 achieves deep fusion and intelligent screening of node features through the topological neighborhood aggregation algorithm. GraphSAGE's neighbor sampling mechanism breaks the bottleneck of traditional graph neural network full-scale computation, and efficiently captures node surrounding structural information with k-hop neighborhood sampling (such as 2-hop). For example, it accurately aggregates the "reimbursement application submission" node and its upstream and downstream behavioral features, reducing computational complexity while retaining key associations. The GAT attention mechanism gives the model dynamic weighting capabilities, through the formula Calculate neighborhood node weights , enabling the model to adaptively focus on high-value information—in the reimbursement process, automatically strengthening the feature contributions of core steps like "filling in reimbursement information" while weakening the interference of marginal behaviors. The combined feature matrix generated by the two not only contains neighborhood collaboration information but also highlights the primary and secondary relationships between behaviors through attention weighting, providing a structured and differentiated feature foundation for subsequent hierarchical representations. Compared to a single algorithm, this significantly improves the expressive power of graph structure semantics.

[0088] During step 322, a hierarchical attention weighting process is applied to the feature matrix containing neighborhood collaboration information to generate a multi-level representation of the behavior space. Specifically, a bottom-level topological attention layer is constructed, utilizing GAT's multi-head attention mechanism to capture the topological relationships between nodes from different perspectives, highlighting the impact of key behavior nodes on the overall structure. For example, in the travel approval process, the core role of the "supervisor approval" node is emphasized. A temporal attention mechanism is then introduced at the middle level, considering the chronological order of behaviors. The temporal dependency weights of behaviors are calculated using timestamp information. For example, the "submit application" behavior must occur after the "prepare materials" behavior, reinforcing this temporal logical connection. Semantic attention is then applied at the upper level, combining behavioral semantic features extracted by pre-trained language models (such as BERT) to calculate semantic similarity weights between behaviors. For example, the close semantic connection between "reimbursement application" and "expense declaration" is emphasized. Through the cascade of these three layers of attention mechanisms, the feature matrix is ​​weighted and the behavior space is hierarchically represented from multiple dimensions such as topological structure, time series, and semantic meaning. The final multi-level representation comprehensively and meticulously depicts the complex correlation relationships in the behavior space of the intelligent agent, providing deep semantic support for intelligent decision-making.

[0089] The hierarchical attention weighting process in step 322 constructs a multi-dimensional, multi-granular behavior space representation system. The bottom-level topological attention, based on the GAT multi-head mechanism, explores key nodes and connection patterns within the graph structure from different perspectives. For example, in the travel approval process, it precisely locates the pivotal node "Supervisor Approval" and strengthens its influence on the overall process. The middle-level temporal attention utilizes timestamp information to quantify the sequential dependencies of behaviors. Through dynamic weight assignment, it ensures that the sequential logic of "prepare materials → submit application → await review" is reflected at the feature level, addressing the difficulty of traditional methods in capturing temporal relationships. The high-level semantic attention, leveraging pre-trained language models (such as BERT), breaks through surface-level associations and explores deep semantic commonalities between behaviors such as "reimbursement application" and "expense reporting," achieving semantic alignment across processes and scenarios. The cascaded design of the three attention layers forms a three-dimensional representation framework of "structural temporal semantics." The resulting multi-level feature matrix not only reflects the topological distribution and temporal context of behaviors, but also captures semantic connotations. This provides the agent with scenario understanding capabilities that are closer to human cognitive logic, significantly enhancing decision-making accuracy and adaptability in complex business scenarios.

[0090] Optionally, step 33 specifically includes the following steps:

[0091] Step 331: Based on the task target semantic mapping model, feature alignment processing is performed on the task execution priority queue to obtain a task-behavior association matrix;

[0092] Step 332: Perform scene context fusion processing on the task-behavior association matrix to generate a scene-aware behavior semantic feature vector.

[0093] In step 331, the feature alignment component uses the pre-trained BERT-base model based on the Transformer encoder architecture to build a task target semantic mapping model, aligns the task execution priority queue across modal features, and generates a task-behavior association matrix. The specific implementation is as follows: First, the task execution priority queue (such as "2025 travel reimbursement application") is input into the BERT-base model, and the semantic feature vector is extracted through its multi-layer Transformer encoder. ,in is the BERT output dimension. At the same time, the behavioral feature matrix ( is the number of behavior nodes, ) comes from the structural-semantic bimodal features generated in step 31. The association weight between the task and the behavior is calculated through the cross-modal attention mechanism, and the formula is:

[0094]

[0095] in As the task feature query vector, 、 is the key matrix and value matrix of the behavioral features ( are learnable weights, is the key vector dimension). This mechanism generates the task-behavior association matrix ,in Indicates the The semantic association strength between each behavior and the task goal is determined. For example, the behavior of "filling in the reimbursement form" obtains a high association weight in the task of "applying for travel reimbursement", thereby clarifying the semantic correspondence between the task and the behavior.

[0096] In step 331, task-behavior feature alignment based on the Transformer encoder and the BERT model fundamentally resolves the disconnect between task semantics and behavior execution logic in traditional approaches. The BERT pre-trained model performs deep semantic extraction of task objectives (e.g., "travel reimbursement application"). Combined with a cross-modal attention mechanism, this approach accurately calculates the semantic correlation strength between each behavior node (e.g., "fill out the reimbursement form" or "upload an invoice") and the task objective. This technical implementation not only overcomes the limitations of traditional bag-of-words models in capturing semantic similarity, but also utilizes a multi-head attention mechanism to explore potential connections between tasks and behaviors across multiple dimensions. For example, in the "annual budget submission" task, key behaviors such as "data aggregation" and "department review" are automatically identified and assigned high weights. The resulting task-behavior correlation matrix provides the agent with a clear task execution guide, significantly improving the accuracy and efficiency of task planning and avoiding ineffective or redundant behavior operations.

[0097] In step 332, the feature alignment component integrates the scene context fusion module of the LSTM temporal encoder to fuse the task-behavior association matrix with the scene dynamic information to generate a behavior semantic feature vector containing scene perception. The specific process is as follows: First, the LSTM encoder is used to process the scene context sequence (such as timestamp, agent state, policy version number, etc.). Assume that the input sequence is , the hidden state sequence is calculated by LSTM unit , where the state update formula of LSTM is:

[0098]

[0099] in: : Indicates the time step The input gate is used to control how much of the current input information will flow into the cell state. It uses the sigmoid function The calculation results in a value between 0 and 1, which determines the inflow ratio of the input information. : In the input gate, connect the input The weight matrix is ​​used to map the input information to the appropriate dimension so that it can participate in the calculation of the input gate. : time step The input vector contains the external information at the current moment. For example, when processing text sequences, It can be the word vector at the current moment. : In the input gate, connect the hidden state of the previous moment The weight matrix is ​​used to integrate the hidden state information of the previous moment into the calculation of the current input gate. : time step The hidden state vector of , which carries the information of the previous time step and is passed to the current time step for calculation. : The bias vector of the input gate is used to adjust the offset of the input gate calculation result to increase the fitting ability of the model. : Indicates the time step The forget gate is used to control the cell state at the previous moment How much information will be retained until the current moment. This is also calculated using the sigmoid function to obtain a value between 0 and 1. A larger value indicates that more information about the cell state at the previous moment is retained. : In the forget gate, connect the input The weight matrix is ​​used to map the input information to the appropriate dimension to participate in the forget gate calculation. : In the forget gate, connect to the hidden state of the previous moment The weight matrix of is used to integrate the hidden state information of the previous moment into the forget gate calculation. : Bias vector of the forget gate, used to adjust the offset of the forget gate calculation result. : Indicates the time step Output Gate, used to control the cell state How much information is output to the hidden state? The sigmoid function is used to calculate a value between 0 and 1 to control the output ratio. : In the output gate, connect the input The weight matrix maps the input information to the appropriate dimension to participate in the output gate calculation. : In the output gate, connect the hidden state of the previous moment The weight matrix of is used to integrate the hidden state information of the previous moment into the output gate calculation. : The bias vector of the output gate, used to adjust the offset of the output gate calculation result. : Indicates the time step The calculated candidate cell state (Candidate Cell State), which is calculated by the tanh function and used to update the cell state . : When calculating candidate cell states, connect the input The weight matrix maps the input information to the appropriate dimension to participate in the candidate cell state calculation. : When calculating the candidate cell state, connect the hidden state of the previous moment The weight matrix of is used to integrate the hidden state information of the previous moment into the candidate cell state calculation. : Bias vector when calculating candidate cell states, used to adjust the offset of the candidate cell state calculation results. : time step The cell state vector is the key to LSTM’s ability to store information for a long time. The cell state at the previous moment The information is filtered and retained, and then the input gate Current input information of the control , to update the cell state. : time step The hidden state vector is composed of the output gate Cell status The output is obtained by filtering and used as the output of the current time step, and is also passed to the next moment for calculation. : The sigmoid activation function maps the input to between 0 and 1 and is often used in gating mechanisms to control the flow ratio of information. : Hyperbolic tangent activation function, which maps the input to between -1 and 1, is often used in scenarios that require activation, such as calculating candidate cell states. : Represents element-wise multiplication (Hadamard product), that is, the corresponding elements of two vectors of the same dimension are multiplied.

[0100] Then, the scene hidden state is merged into Task-Behavior Correlation Matrix Fusion, the formula is:

[0101]

[0102] in is the fusion gate vector, represents the concatenation of the association matrix and the hidden state of the scene, is a learnable parameter. The final generated feature vector ( ) integrates task semantics, behavioral associations, and scenario dynamic information. For example, in the "month-end reimbursement peak" scenario, the feature weight of the "quick approval" behavior will be enhanced, making the intelligent agent strategy generation more in line with actual business scenario needs.

[0103] In step 332, the scenario-context fusion achieved through the LSTM temporal encoder and the gated fusion mechanism enables the agent to dynamically perceive and adapt to environmental changes. The LSTM's temporal processing of scenario information (such as time periods, policy updates, and system status) enables it to capture complex temporal dependencies within business scenarios. For example, it can automatically identify the urgency of tasks during peak periods like "quarter-end financial close." The gated fusion mechanism acts as an intelligent regulator, dynamically weighing the relevance of task-behavior relationships with contextual information based on scenario characteristics. For example, when detecting the release of a new reimbursement policy, it automatically enhances the feature strength of behaviors such as "reading policy terms" and "compliance checking." This technical implementation makes the generated behavioral semantic feature vector highly context-sensitive, enabling the agent to adjust task execution strategies in real time in the face of dynamic and changing business environments. This significantly improves decision-making flexibility and business adaptability, effectively preventing decision errors caused by scenario changes.

[0104] Optionally, the semantic feature mapping component includes: a multi-head linear transformation component, a behavioral feature mapping component, and a strategy generation component. Step 4 specifically includes:

[0105] Step 41: Based on the multi-head linear transformation component, perform multi-head linear transformation processing on the behavior semantic features containing scene perception to obtain a behavior feature interaction matrix;

[0106] Step 42: Based on the behavior feature mapping component and the behavior feature interaction matrix, a temporal constraint mask matrix in the strategy representation space is generated;

[0107] Step 43: Based on the strategy generation component, a behavior feature strategy for controlling the intelligent agent is generated according to the timing constraint mask matrix. The behavior feature strategy is expressed in the form of multivariate Gaussian distribution parameters.

[0108] Optionally, step 41 performs multi-head linear transformation processing on the behavior semantic features containing scene perception based on the configured multi-head linear transformation component to obtain the behavior feature interaction matrix, specifically including the following steps:

[0109] Step 411: Perform multi-head linear transformation on the behavior semantic features containing scene perception to generate feature triples including query / key / value;

[0110] Step 412: Based on the cross-agent interaction weight matrix, perform attention mechanism calculation processing on the feature triples to obtain the behavior feature interaction matrix.

[0111] Preferably, the semantic feature mapping component is constructed by integrating the Transformer architecture and probabilistic graphical model technology: the multi-head linear transformation component is based on PyTorch's linear layer and multi-head attention mechanism to achieve multi-dimensional interaction of features; the behavioral feature mapping component relies on the Transformer Encoder structure of time series modeling, combined with the residual network and Layer Normalization technology to generate time series constraints; the strategy generation component adopts a strategy mapping network based on VAE (variational autoencoder) to generate multivariate Gaussian distribution strategies through reparameterization.

[0112] Specifically, when executing step 411: performing multi-head linear transformation processing on the behavior semantic features containing scene perception to generate feature triplets, specifically, when performing multi-head linear transformation processing on the behavior semantic features containing scene perception, the features are mapped into query (Query), key (Key), and value (Value) vectors through the PyTorch linear layer to generate feature triplets, and a multi-head mechanism (such as 8 heads) is used to divide the features into different subspaces for independent transformation, and each head is transformed through different weight matrices. 、 、 calculate 、 、 , enabling the model to capture behavioral feature associations from multiple dimensions. This step introduces the Transformer multi-head attention mechanism into feature interaction, breaking through the representation limitations of a single linear transformation. The generated feature triplets provide multi-perspective feature representations for subsequent cross-agent attention calculations.

[0113] In the formula involved in the above step 411: : Represents the query vector, which is a vector obtained by linearly transforming the behavioral semantic features containing scene perception. It is used to calculate the degree of association with other features in the attention mechanism and measure the degree of "inquiry" of the current feature to other features. : represents the key vector, which is also generated by linear transformation and can be regarded as the "identifier" of the feature. It is used to match the query vector in the attention calculation to determine the correlation between different features. : Represents the value vector, which is obtained through linear transformation and carries the core information of the original features. After the attention mechanism determines the relevance, it is used to output the weighted aggregation result. : Represents the input scene-aware behavioral semantic feature vector, which integrates task semantics, behavioral association and scene dynamic information (from step 332) and is the original data object for multi-head linear transformation. :It is used to input features Mapping to query vector The weight matrix of this matrix is ​​learned during the model training process. By adjusting its value, the query vector can better capture the correlation between features. :Is the input feature Mapping to key vector The weight matrix is ​​used to generate a suitable “identity” for each feature so that it can be matched in the attention calculation. :Is the input feature Mapping to a value vector The weight matrix is ​​responsible for extracting key information from the original features for subsequent aggregation calculations.

[0114] In the above step 411, by introducing the Transformer multi-head attention mechanism into feature interaction, the behavioral semantic features containing scene perception are mapped into query, key, and value vectors through multi-head linear transformation, which has significant technical advantages: on the one hand, the multi-head mechanism divides the features into different subspaces and transforms them independently, just like examining the behavioral features from multiple perspectives at the same time. For example, in the reimbursement process, different heads can focus on dimensions such as behavioral temporal association and semantic similarity, enabling the model to capture the complex associations between behavioral features more comprehensively and meticulously; on the other hand, through independent weight matrices 、 、 Generating query, key, and value vectors separately allows the model to adaptively learn different feature interaction patterns, enhancing the flexibility and richness of feature expression. Compared to single linear transformations, this method breaks the limitations of traditional representations. The generated feature triplets provide multi-dimensional and differentiated feature representations for subsequent cross-agent attention calculations, greatly improving the model's ability to understand and process behavioral characteristics in complex business scenarios.

[0115] Specifically, when executing step 412 to perform cross-agent attention calculation on the feature triples to generate the behavior feature interaction matrix, the cross-agent interaction weight matrix trained by historical collaboration data is introduced. , and weighted fusion with traditional attention results ( ), generating a behavioral feature interaction matrix. This step encodes the agent's collaborative experience into the attention calculation. For example, in a multi-person review scenario, the interaction weights between the finance and department agents are strengthened. This enables the interaction matrix to capture the inherent correlations between behaviors while adapting to actual collaboration patterns, providing a feature foundation for integrating business collaboration experience into temporal constraint modeling.

[0116] In the above step 412, : represents the final generated behavioral feature interaction matrix, which is used to characterize the correlation strength between different behavioral features. Each element in the matrix Representative behavior and behavior The interaction weight is the basic data for subsequent timing constraint modeling. : Fusion coefficient (value range is [0,1]), which is used to dynamically adjust the proportion of traditional attention results and cross-agent interaction weights, learn the optimal value through model training, and balance the contribution of feature intrinsic correlation and agent collaboration experience. : The calculation result of the multi-head attention mechanism, the specific formula is:

[0117] : Query vector (Query), which comes from the generated feature triples and is used to calculate the association requirements between features. : Key vector (Key), a component of the feature triplet, participates in association matching as the "identifier" of the feature. : Value vector (Value), a component of the feature triplet, carries the core semantic information of the feature and is used to generate attention output. : key vector The dimension is used to scale the dot product result to avoid the gradient disappearance problem. : The cross-agent interaction weight matrix is ​​obtained through training of historical collaboration data. The elements in the matrix Representing an agent With the agent For example, in the reimbursement process, the collaboration weight between the financial agent and the department head agent is It will be strengthened by high-frequency interactions and reflect business collaboration experience.

[0118] By weighted fusion of traditional attention results and cross-agent interaction matrix, the generated behavior feature interaction matrix At the same time: 1. Intrinsic correlation of features: Capturing semantic matching and topological association of behavioral features; 2. Business collaboration adaptability: Encodes the historical collaboration patterns of agents (e.g., role interaction preferences in an approval process). Dynamically balancing the weights of the two solves the problem of traditional attention mechanisms ignoring actual business collaboration experience, and provides a feature foundation that is more in line with the enterprise business logic for subsequent timing constraint modeling.

[0119] In step 412, the historical collaborative experience of the agent is integrated into the attention calculation, and the traditional attention results are weighted fused with the cross-agent interaction weight matrix. , so that the generated behavioral feature interaction matrix can not only capture the semantic associations and topological structures between behaviors, but also adapt to the collaboration model in actual business. For example, in the multi-person review scenario of the reimbursement process, the model will strengthen the interaction weights between the financial agent and the department head agent based on historical data, making the associations between behaviors such as "financial compliance inspection" and "department approval" more prominent in the matrix. This technical processing breaks through the limitation of the traditional attention mechanism that only focuses on the intrinsic relationship of features, and through the fusion coefficient Dynamically balancing the weights of feature relevance and collaborative experience makes the interaction matrix more aligned with the enterprise's real business processes, providing a feature foundation with both semantic understanding and collaborative adaptability for subsequent temporal constraint modeling, effectively improving the rationality of intelligent agents' decisions in complex multi-person collaborative scenarios.

[0120] Optionally, step 42 specifically includes the following steps:

[0121] Step 421: Perform temporal mask weight calculation on the behavior feature interaction matrix to determine the dependency weight of the agent behavior sequence;

[0122] Step 422: Perform residual normalization processing on the dependency weights to generate a temporal constraint mask matrix in the policy representation space.

[0123] Preferably, in step 421, when the dynamic temporal mask calculation process is performed on the behavior feature interaction matrix to generate dependency weights, the temporal mask weight calculation is performed on the behavior feature interaction matrix to construct a temporal dependency matrix T (such as behavior sequence constraints). Explicitly encode the temporal logic of behaviors, then generate dependency weights through a self-attention mechanism. This step avoids the traditional Transformer temporal modeling approach that relies solely on positional encoding. Instead, a dynamic temporal masking mechanism is designed. For example, in the reimbursement process, the dependency strength of "filling out the form" precedes "submitting for review" is clearly defined. This allows the weight matrix to accurately reflect the time constraints of the business process, providing temporal logic support for subsequent policy generation.

[0124] In the above step 421, M' represents the intermediate feature matrix after the timing mask calculation, which is used to explicitly encode the timing constraint logic in the behavior feature interaction matrix. The matrix elements = represents the association weight between behavior i and behavior j after considering temporal dependency. M: The input behavioral feature interaction matrix (output from step 412). The original matrix describes the interaction strength between behavioral features but does not contain temporal order information. T: Temporal dependency matrix, constructed manually or trained through historical data. The matrix elements Indicates the time-order dependency strength between behavior i and behavior j (usually in the range of [0,1]). For example, in the reimbursement process, "filling out the form" (behavior i) precedes "submitting for review" (behavior j), then is a high value, A low value or 0 reflects a strict order constraint. : Element-wise multiplication (Hadamard product) is used to transform the behavioral feature interaction matrix and the timing dependency matrix Multiply the elements at corresponding positions. The formula is:

[0125]

[0126] The purpose of this operation is to "mask" the timing constraint information into the original interaction matrix, so that M' contains both the behavior association strength and the timing dependency. It means that "filling in the form" and "submitting for review" have a high semantic correlation, but means that the former must precede the latter, then Keep high weight; if there is (reverse association), then For low weight, are suppressed to low weight, thereby explicitly encoding the sequential logic.

[0127] In the above scheme, through the timing dependency matrix The original interaction matrix Dynamic masking breaks through the limitation of traditional Transformer that only relies on position encoding to implicitly model timing: It can be pre-defined based on business process rules (such as fixed steps for reimbursement approval) or historical data statistics (such as the time sequence probability of behavior occurrence), so that the model can accurately capture the strong temporal dependency of "A must be executed before B". , weakening or eliminating behavioral associations that do not conform to temporal logic (such as the reverse association that "submit for review" precedes "fill out the form"), so that the dependency weights generated by the subsequent self-attention mechanism are more in line with the actual business process, providing reliable temporal logic support for behavioral sequence decisions in the strategy generation stage.

[0128] In step 421, the behavioral feature interaction matrix and the temporal dependency matrix are fused element-wise via a dynamic temporal masking mechanism, achieving a breakthrough in explicitly modeling the temporal logic of business processes. Compared to the implicit temporal processing of traditional Transformers that relies solely on positional encoding, this method offers significant technical advantages: by constructing the temporal dependency matrix T (e.g., presetting the weights of T matrix elements based on the fixed order of "filling out a form → submitting for review" in the reimbursement process), and utilizing the operation M' = M⊙T to directly suppress behavioral associations that do not conform to temporal logic (e.g., the reverse "submitting for review → filling out a form" association weight), the generated dependency weights preserve the semantic interaction strength between behavioral features while accurately capturing the strong temporal constraints of "A must be executed before B" in business scenarios. This explicit temporal encoding method not only makes the model more consistent with the temporal logic of the enterprise's actual business processes but also provides a standardized temporal feature foundation for the subsequent policy generation stage. This effectively addresses the issues of ambiguous temporal modeling and the inability to handle complex process sequence constraints in traditional methods, significantly improving the decision-making accuracy of the intelligent agent in scenarios requiring strict temporal control.

[0129] Step 422: Perform residual normalization on the dependency weights to generate a timing constraint mask matrix. When performing residual normalization on the dependency weights, the residual connection Retain the original feature information and then pass LayerNormalization ( ) eliminates weight scale differences and generates a temporal constraint mask matrix. This step combines residual networks with normalization techniques to enhance the model's ability to fit complex weights while ensuring a stable weight distribution. This allows the mask matrix to reliably convey behavioral temporal constraints, providing standardized temporal features for sequential decision-making during the policy generation phase.

[0130] In step 422, : Represents the dependency weight matrix, which is used to characterize the dependency weights of the agent's behavior sequence. : It is the weight matrix in the residual connection, which is used to linearly transform the dependency weights so that they can be integrated with the original weights to retain the original feature information. : Represents the weight matrix obtained after residual connection processing, through The calculation of realizes the superposition of original weights and transformed weights, and enhances the model's ability to fit complex weights. : In LayerNormalization, Represents the input matrix, specifically the weight matrix after the residual connection . : is input The mean of is used to standardize the input and eliminate the difference in weight scale. :For input The variance of also plays a role in the normalization process to ensure the stability of the normalization operation. : is a very small constant, usually taken as or , added to the denominator of the square root of the variance to prevent the denominator from being zero and ensure the stability of the calculation. :Indicates input The result after the LayerNormalization operation is , generates a standardized weight matrix, and finally obtains the timing constraint mask matrix.

[0131] In the above step 422, the residual connection is integrated with the Layer Normalization technology through residual normalization, which achieves weight normalization while retaining the original temporal dependency information, forming a dual technical advantage: the residual connection is achieved through The superposition operation avoids the original dependency weight while learning complex weight transformation The information loss is like building a "timing information backup channel" for the model to ensure that key timing constraints such as "fill in the form → submit for review" are not weakened in the multi-layer transformation; LayerNormalization Eliminating the scale differences between different behavioral weights allows temporal constraints for behaviors of varying importance, such as "urgent approval" and "routine review," to be placed in a unified metric space. The resulting temporal constraint mask matrix, generated by combining these two approaches, not only enhances the ability to fit complex temporal patterns such as "cross-departmental review" in the reimbursement process through its residual structure, but also ensures a stable weight distribution through normalization, providing standardized temporal feature input for subsequent strategy generation. This effectively addresses the decision-making bias caused by scale confusion in temporal weights in traditional methods, significantly improving the agent's temporal logic processing capabilities in multi-step, long-process business scenarios.

[0132] Optionally, step 43 generates a behavior feature strategy for controlling the intelligent agent based on the configured strategy generation component and the timing constraint mask matrix, specifically including the following steps:

[0133] Step 431: Performing strategy mapping network processing on the timing constraint mask matrix to generate a scenario-aware strategy implicit representation vector;

[0134] Step 432: Perform probability distribution generation processing on the implicit representation vector to generate a behavior feature strategy for controlling the intelligent agent, wherein the behavior feature strategy is represented in the form of multivariate Gaussian distribution parameters.

[0135] Preferably, in step 431, when performing VAE policy mapping on the timing constraint mask matrix to generate the policy implicit representation vector, the timing constraint mask matrix is ​​processed using a policy mapping network. A fully connected neural network (e.g., a three-layer ReLU network) with a VAE architecture is used to flatten the matrix and transform it into an implicit vector z, thereby generating a scenario-aware policy implicit representation vector. This step introduces a variational autoencoder mechanism, enabling the network to learn the probabilistic distribution characteristics of scenario semantics during the encoding process. For example, it can automatically generate adaptive policy representations under different policy scenarios, providing a compact feature representation that incorporates both timing constraints and scenario semantics for subsequent probabilistic policy generation.

[0136] In the above step 431, the VAE architecture is introduced into the strategy mapping, and the deep integration of temporal constraints and scene semantics is achieved by constructing a probabilistic encoder, forming three technical advantages: First, the variational inference mechanism of VAE enables the model to capture the deterministic rules of temporal constraints such as "filling out the form → submitting for review" in the reimbursement process when generating the implicit representation vector of the strategy, and also learn the probability distribution characteristics under the policy ambiguity scenario (such as the different definitions of "urgent approval" by different departments), injecting uncertainty modeling capabilities into the strategy decision-making; second, the multi-layer nonlinear transformation of the fully connected network (such as 3 layers) The ReLU network compresses the two-dimensional structure of the temporal constraint mask matrix into a low-dimensional vector representation, preserving the complex dependencies between behaviors such as "budget review" and "financial review" while also achieving feature abstraction through dimensionality collapse (for example, compressing multiple interactions such as "incomplete materials → return for revision" into a single semantic vector), making the policy representation more compact and efficient. Third, the KL divergence regularization term unique to VAE enables the model to learn a more generalized policy distribution. For example, when new policies are introduced, the model can quickly adjust the implicit representation based on the existing scenario semantics to generate policy initialization parameters that comply with the new regulations. This policy representation method that combines probabilistic reasoning and semantic abstraction enables the intelligent agent to generate more adaptive and robust behavioral strategies when faced with complex scenarios such as rule changes in the reimbursement process and differences in departmental collaboration, significantly improving the model's responsiveness to dynamic business environments.

[0137] Preferably, in step 432, when the implicit representation vector of the strategy is reparameterized to generate a multivariate Gaussian distribution strategy, the implicit representation vector of the strategy is subjected to probability distribution generation processing, and the mean of the multivariate Gaussian distribution is calculated by reparameterization. and variance ( , ), generate behavioral feature strategies represented by multivariate Gaussian distribution parameters. This step represents the strategy as a probability distribution, which enables the agent to have uncertainty modeling capabilities, such as through strategy sampling in rule fuzzy scenarios. Generate diverse strategies, which significantly improve adaptability to complex scenarios compared to deterministic strategies, and ultimately form policy outputs that can directly control the behavior of intelligent agents.

[0138] In step 432, :The implicit representation vector of the strategy is a compact feature representation that combines the behavior timing constraints and scene semantics. The dimension is usually (e.g. 256 dimensions), carrying all the semantic information required for generating strategies. : The mean vector of the multivariate Gaussian distribution is transformed from the implicit vector Mapping is obtained, the dimension is (consistent with the dimension of the strategy space), represents the central tendency of the strategy distribution and determines the "average tendency" of the agent's behavior. : The variance matrix of the multivariate Gaussian distribution (all elements are non-negative), also given by Mapping is obtained, the dimension is , characterizes the degree of uncertainty of the strategy - the larger the variance, the more random the agent's decision-making is, and is suitable for scenarios with fuzzy rules. : Random sampling vector from standard normal distribution, dimension is Consistency is used to introduce randomness and enable strategy generation to have exploration capabilities. : Weight matrix of mean mapping (dimension is ), which is responsible for learning implicit features through neural network training. Mapping to the policy mean space. For example, in the reimbursement review scenario, the "policy compliance" feature is mapped to the mean tendency of "strict review". : bias vector of mean mapping (dimension is ), which is used to adjust the reference position of the mean and enhance the model fitting ability. : The weight matrix of the variance map (dimension is ), mapping implicit features to variance space, for example, generating uncertainty parameters for corresponding decisions based on the “policy ambiguity” feature. : bias vector of variance mapping (dimension is ), adjust the variance benchmark. : activation function ( ), ensuring the variance It is always a positive number, satisfying the mathematical requirements of probability distribution.

[0139] Strategy sampling such as through accomplish, : Square root of the variance matrix ( ), converting the variance into standard deviation to control the scale of sampling noise. : Element-wise multiplication (Hadamard product) to combine the standard deviation with the random noise Multiply and add to the mean Generate the final strategy sample .

[0140] In the above scheme, the mean and variance Explicitly characterize the "deterministic tendency" and "randomness range" of the policy. For example, in the reimbursement process, when the policy terms are clear, When is smaller, the agent tends to execute fixed audit policies; when the policy is updated frequently, Increase, the agent generates diversified strategies through sampling to adapt to changes. Traditional direct sampling (such as ) is not differentiable and cannot backpropagate gradients; reparameterization moves randomness to the independent variables ,make right and Differentiable, thus supporting end-to-end training (e.g., through gradient descent optimization When the strategies are diverse, by adjusting The size of the model can balance between "exploitation (using the known optimal strategy)" and "exploration (exploring new strategies)", which is more effective than deterministic strategies (such as using only As output), it significantly improves the decision-making adaptability of intelligent agents in complex business scenarios (such as fuzzy rules and dynamic changes in the environment).

[0141] In the above step 432, the strategy is represented as a multivariate Gaussian distribution through reparameterization, giving the agent dynamic decision-making and adaptive learning capabilities. On the one hand, using the mean vector and variance matrix The core tendency and uncertainty of the strategy are respectively characterized, so that the intelligent agent can determine the basic audit strategy (based on the characteristics of "policy compliance") in the business of reimbursement review. It can also dynamically adjust the randomness of decision-making according to the “policy ambiguity” (by control), significantly enhancing the ability to respond flexibly in complex scenarios; on the other hand, the reparameterization technique separates randomness into independent sampling variables , making the strategy generation process differentiable, supporting end-to-end training optimization, and ensuring that the model can accurately adjust the weight matrix based on historical collaboration data and business feedback 、 Parameters such as , continuously improve the accuracy and adaptability of strategy generation. This technological breakthrough of probabilizing strategies effectively solves the problem of rigid decision-making in scenarios with ambiguous rules and changing environments, compared to traditional deterministic strategies. It provides core technical support for intelligent agents to achieve efficient and robust decision-making in dynamic business environments.

[0142] Optionally, the policy optimization component includes: a scenario representation component, a policy decomposition component, a truncation component, and a fine-tuning component. Step 5 specifically includes the following steps:

[0143] Step 51: Determine a scenario value representation based on a generalized advantage estimation model using a scenario representation component, wherein the scenario value representation represents an estimate of an advantage function of a cumulative discounted reward;

[0144] Step 52: Based on the strategy decomposition component and the hierarchical value decomposition network, the behavior feature strategy is subjected to probability density decomposition to obtain a new and old strategy distribution ratio matrix;

[0145] Step 53: Dynamically truncate the new and old strategy distribution ratio matrix based on the scenario value representation through the truncation component to obtain the advantage estimation value of the scenario adaptation;

[0146] Step 54: Fine-tune the parameters of the behavior feature strategy based on the advantage estimation value through the fine-tuning component to achieve online optimization of the behavior feature strategy.

[0147] The policy optimization component is preferably built based on reinforcement learning and deep neural network technologies. The scenario representation component relies on the generalized advantage estimation (GAE) model and LSTM temporal network to estimate the advantage function of cumulative discounted rewards. The policy decomposition component uses a hierarchical value decomposition network (such as a hierarchical actor-critic architecture) combined with variational inference to decompose policy probability density. The truncation component uses a dynamic truncation mechanism based on PPO (Proximal Policy Optimization) and integrates scenario value representation to trim the new and old policy distribution ratio matrices. The fine-tuning component uses the Adam optimizer and gradient clipping technology to achieve online updates of policy parameters. The entire architecture is deployed in the TensorFlow framework, supporting distributed policy optimization computation.

[0148] Preferably, in step 51, when the scenario representation component performs GAE-LSTM hybrid modeling on the scenario information to generate the scenario value representation, the scenario information (including timestamps, policy versions, agent states, etc.) is subjected to GAE-LSTM hybrid modeling. The temporal dependencies of the scenario features (such as the periodicity of the "month-end settlement period") are captured through the LSTM network, and then the advantage function estimate is calculated based on the generalized advantage estimation (GAE) model to generate the scenario value representation. The specific algorithm formula is as follows:

[0149] Time difference error calculation: ,in: is the time step TD error, which measures the deviation of the current state value estimate; is the time step Instant rewards received (e.g., efficiency rating for “submitting expense claims”); is the discount factor (value range [0,1]), used to discount the present value of future rewards; 、 Status 、 The estimated value of the value function.

[0150] GAE advantage estimation: ,in: is the advantage estimate after GAE smoothing; is a GAE parameter (smoothing coefficient, range [0,1]), used to control the bias-variance trade-off of the advantage estimate.

[0151] In summary, the generated scenario value representation not only includes the immediate reward of "submitting the reimbursement form", but also reflects its long-term impact on the subsequent "financial review", providing multi-time-scale value signals for strategy optimization.

[0152] To this end, in step 51 above, scenario information is processed through GAE-LSTM hybrid modeling, effectively addressing the high variance and insufficient utilization of temporal information inherent in traditional advantage estimation. The LSTM network's capture of the temporal dependencies of scenario features enables accurate identification of complex scenario patterns, such as "business volume surges during the month-end settlement period" and "rule changes during the policy transition period." The generalized advantage estimation (GAE) model, combining temporal difference error with a smoothing coefficient, not only quantifies the immediate benefits of actions like "submitting an expense report" but also accurately assesses their impact on subsequent business processes through discount factors and long-term cumulative error. This multi-timescale approach to generating value signals provides a more comprehensive and stable basis for policy optimization than a single instantaneous reward calculation, significantly improving the accuracy of the agent's value judgments in dynamic business scenarios.

[0153] Preferably, in step 52, the strategy decomposition component performs a hierarchical value decomposition process on the behavior feature strategy to generate a new and old strategy distribution ratio matrix. The behavior feature strategy is subjected to a hierarchical value decomposition process. The complex strategy is decoupled into sub-strategy spaces according to the business level (such as "material review-approval allocation") by relying on a hierarchical value decomposition network (such as the H-DQN architecture). The probability density ratio of the new and old strategies is calculated through variational inference. The specific algorithm formula is:

[0154] Distribution ratio of new and old strategies: ,in: is the element of the new and old strategy distribution ratio matrix, representing the new strategy With the old strategy In state Next select action The probability ratio of are the new strategy parameters, For old strategy parameters (preserve historical strategy information through EMA updates); 、 The new and old strategies are in state Next select action The conditional probability of .

[0155] KL divergence regularization: ,in: is the variational posterior distribution, is a prior distribution (usually a standard normal distribution) used to constrain the smoothness of the sub-strategy distribution.

[0156] To this end, the generated new and old strategy distribution ratio matrix accurately reflects the impact of sub-strategy updates such as "material compliance check" on the overall strategy, solving the problem that traditional decomposition cannot handle hierarchical business.

[0157] In the above step 52, the hierarchical value decomposition network is used to perform probability density decomposition on the behavioral feature strategy, and the hierarchical structure of the enterprise business is integrated into the strategy optimization process. Through variational inference and KL divergence regularization, this step can decompose complex business strategies such as reimbursement review into sub-strategy spaces such as "material review" and "authority allocation", and accurately calculate the probability distribution ratio of new and old strategies at each level. This hierarchical processing method breaks the "flat" limitation of traditional strategy decomposition, allowing strategy updates to act on specific business links in a targeted manner. For example, when adjusting the "material compliance check" sub-strategy, it will not excessively affect the stability of other links, effectively improving the precision and explainability of strategy updates and reducing the decision-making risks caused by global strategy adjustments.

[0158] Preferably, in step 53, when the truncation component dynamically truncates the new and old strategy distribution ratio matrix and the scenario value representation to generate the scenario-adapted advantage estimate, the new and old strategy distribution ratio matrix and the scenario value representation are dynamically truncate, and based on the adaptive truncation mechanism of the PPO algorithm, an objective function is defined to generate the scenario-adapted advantage estimate:

[0159] ,in: is the truncated objective function value, which is used to limit the strategy update amplitude; is the element of the new and old strategy distribution ratio matrix, representing the current strategy update amplitude; To generate scenario value representation and guide strategy update direction; is the truncation function, Restricted to the interval [a,b]; A cutoff threshold (e.g., 0.2) is used to dynamically adjust the allowed range of policy updates (automatically narrowing it when the scene uncertainty is high).

[0160] To this end, by dynamically adjusting Suppress aggressive policy updates, generate scenario-adaptive advantage estimates, and ensure the stability of policy updates in complex scenarios.

[0161] In summary, in step 53, the dynamic truncation mechanism based on the PPO algorithm is combined with the scene value representation to process the new and old strategy distribution ratio matrix, providing adaptive stability guarantee for strategy update. By dynamically adjusting the truncation threshold The system automatically tightens the policy update range based on scenario uncertainty (such as high volatility during policy transitions) to avoid decision-making oscillations caused by overexploration. In conventional scenarios, it moderately relaxes the threshold to accelerate policy optimization. This dynamic balancing mechanism, compared to fixed threshold truncation, better adapts to complex and changing business environments. It ensures the robustness of the agent's strategy in risky scenarios without sacrificing optimization efficiency in stable scenarios, significantly enhancing the environmental adaptability of policy updates.

[0162] Preferably, in step 54, when the fine-tuning component performs policy parameter fine-tuning on the advantage estimate to generate the optimized behavioral feature strategy, the policy parameter fine-tuning is performed on the scenario-adapted advantage estimate, and the scenario-adapted advantage estimate and the new and old strategy distribution ratio matrix are substituted into the PPO objective function:

[0163] ,in: To ultimately optimize the objective function, balance the policy update benefit and entropy regularization; is the entropy coefficient (such as 0.01), and the control strategy entropy The weight of ,enhancing the strategy exploration capability; The policy entropy is used to measure the randomness of action selection. , is the learning rate) and gradient clipping (such as limiting the gradient norm to 5.0), and updates the policy parameters through EMA ( , The resulting optimized strategy incorporates a dynamic "exploration-exploitation" balancing mechanism to continuously adapt to real-time business scenarios.

[0164] In summary, in step 54 above, by fine-tuning the policy parameters through a combination of the PPO objective function, the Adam optimizer, and gradient clipping, efficient and reliable policy optimization is achieved. The introduction of the entropy regularization term enables the agent to proactively explore diverse strategies in ambiguous rules or novel scenarios, such as trying different review processes when reimbursement rules are unclear. The combination of the Adam optimizer and gradient clipping ensures the stability of parameter updates, preventing exploding or vanishing gradients. Furthermore, the exponential moving average (EMA) maintains the old policy, providing a "safe rollback" mechanism for policy optimization, preventing performance degradation caused by a single update error. Overall, this step enables the behavioral feature strategy to dynamically balance exploring new strategies with leveraging proven experience, continuously improving the agent's decision-making capabilities and long-term benefits in complex business scenarios.

[0165] The following is an example of the application of the above solution in the corporate financial reimbursement scenario.

[0166] Step 1: Obtain and analyze scenario requirements and generate target task sequences

[0167] The user enters a scenario description through the front-end human-computer interaction component: "I need to submit a travel reimbursement claim for June 2025 in Shenzhen, including food and transportation expenses, and I'm applying for a fast-track approval channel due to an urgent project."

[0168] The demand collection component captures text input through the Vue.js interface and confirms intent through DeepSpeech voice recognition; the semantic role annotation model (RoBERTa-large) parses the core entities: {Subject: Employee, Action: Submit, Object: Travel Reimbursement in Shenzhen, June 2025, Additional Conditions: Emergency Approval, Including Food and Transportation Expenses}; The sequence generation component analyzes temporal dependencies through GraphSAGE and, combined with the company's reimbursement DAG process, maps "Emergency Approval" as a priority processing rule.

[0169] Generate the target task sequence: ["Retrieve the 2025 Shenzhen travel reimbursement policy → Fill out the travel reimbursement form (mark urgent) → Upload catering and transportation invoices → Trigger rapid material verification → Submit to department head for expedited review → Finance urgently review"].

[0170] Step 2: Dynamically perceive the environment and generate semantic topology networks and scene graphs

[0171] The target scenario is an enterprise financial system. The current environment features include:

[0172] 1. The financial system will push the policy update of "New mandatory items for electronic invoices in June 2025" in real time;

[0173] 2. The system load during the month-end settlement period (CPU usage > 80%) is monitored by the agent.

[0174] 3. Employee terminal behavior data shows frequent "electronic invoice upload failure" incidents (12 times in the past 15 minutes).

[0175] The environmental detector pulls OA system policy logs through the Feign client, Prometheus collects system metrics, and Flutter tracking captures terminal behavior, forming a raw data stream. The sliding window frequency counter (Flink + Count-MinSketch) identifies that the co-occurrence frequency of "electronic invoice upload" and "policy update" is 0.8 (5 co-occurrences within a 15-second window). The dynamic scene graph generator uses GraphSAGE to calculate edge weights. For example, the bidirectional probability weight of "electronic invoice upload" and "policy update" is:

[0176]

[0177] The dynamic semantic topology network includes the nodes "electronic invoice upload", "policy update", and "system load", and the edge weights reflect the intensity of collaboration; the dynamic scenario graph is displayed in three-dimensional space (x-axis = financial policy, y-axis = timestamp, z-axis = system resources), and the node "electronic invoice upload" is marked red (strong risk association) due to high frequency of failure events.

[0178] Step 3: Hierarchically model the behavior space and generate scene-aware features

[0179] Based on the dynamic scenario graph in step 2, the behavior space includes behaviors such as "invoice upload", "policy verification", and "system load monitoring".

[0180] The semantic extraction component uses GCNConv to calculate node features. For example, the "electronic invoice upload" node integrates policy update features (the "required items" semantic vector extracted by BERT) and system load features (numerical features corresponding to CPU usage). The multi-level feature generation component uses the GraphSAGE+GAT hybrid architecture to calculate the topological attention weight of "upload failure" and "policy update" as 0.7, and the temporal attention is enhanced to 0.9 due to the "month-end settlement period". The feature alignment component uses the Transformer encoder to set the semantic association weight of the task objectives "urgent reimbursement" and "quick material verification" to 0.85.

[0181] The behavioral semantic feature vector with scenario perception is 768-dimensional, among which the key dimensions of the "electronic invoice upload" feature include: policy compliance (0.82), system risk (0.75), and time urgency (0.91).

[0182] Step 4: Map features to strategy space and generate behavioral feature strategies

[0183] The multi-head linear transformation component uses eight heads to calculate the Q / K / V vectors. For example, the first head focuses on "policy compliance" to generate the query vector, and the fifth head focuses on "system risk" to generate the key vector. Cross-agent attention integrates the historical collaboration matrix between the finance and department agents (for example, the interaction weight of the finance agent for "invoice verification" is 0.6). The temporal mask matrix suppresses the reverse association of "review before uploading invoices" (with a weight of 0.1). The strategy generation component compresses the features into a 256-dimensional latent vector z through VAE, and then reparameterizes it to generate a multivariate Gaussian distribution strategy:

[0184] Mean μ: [Prioritize electronic invoice processing → Trigger automatic compliance checks → Skip non-mandatory field checks]

[0185] Variance σ²: Increases when policy ambiguity is high (e.g., σ² = 0.3), allowing a 5% probability of attempting a manual review process

[0186] The behavioral feature strategy is represented as N(μ, σ²). For example, in the scenario of "electronic invoice upload failure", the strategy sampling generates the action sequence of "automatically retry uploading + synchronously triggering the manual auxiliary channel".

[0187] Step 5: Optimize the strategy online based on the scenario value

[0188] When the system detects that the "E-invoice Mandatory Items Policy" is in a transition period (the rules are vague), it triggers policy optimization.

[0189] The scenario representation component uses GAE-LSTM to calculate the advantage estimate, for example, the advantage value of the "manually assisted uploading" behavior is A_t=0.7 (immediate reward +0.5, long-term impact +0.2); the policy decomposition component decomposes the policy into "upload process" and "compliance verification" sub-policies, and calculates the new and old policy ratio matrix r(θ,θ_old)=1.2 (the new policy adds a manual review branch); the truncation component reduces the truncation threshold ε from 0.2 to 0.1 according to the scenario value (high uncertainty during the policy transition period) to limit the policy update range; the fine-tuning component uses the PPO objective function to optimize the parameters, increase the probability of the "manual review" action (from 0.3→0.5), while maintaining the entropy coefficient β=0.01 to explore other solutions.

[0190] The optimized behavioral feature strategy is N(μ', σ'²), where μ' sets "manual-assisted upload" as the priority action and σ'²=0.2 (a decrease of 0.1 compared to the pre-optimization setting). This reduces system load while ensuring compliance. For example, during actual execution, the agent prioritizes guiding employees to use the manual-assisted channel and attempts to automatically repair upload failures with a 20% probability.

[0191] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to technical solutions formed by a specific combination of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for agent strategy generation and online optimization based on dynamic scene perception, characterized in that: include: Step 1: Obtain the user's scenario requirement description and parse it to generate a target task sequence that can be performed by the intelligent agent deployed in the target scenario; Step 2, which includes: Step 21: Based on the environment detector, the environment feedback data is sampled and processed in real time to dynamically perceive the target scene environment characteristics and obtain the original perception data stream; Step 22: Use a sliding window frequency counter to perform behavioral event recognition processing on the original perception data stream to obtain behavioral co-occurrence frequency data with timestamps; Step 23 specifically includes: Step 231: Perform bidirectional probability weighting processing on the behavior co-occurrence frequency data to generate a dynamic semantic topology network, where nodes represent behavior types and edge weights represent behavior synergy strength; Step 232: Perform spatial dimension mapping processing on the dynamic semantic topology network to generate a dynamic scene graph, wherein the dynamic scene graph represents the topological association strength of the behavior nodes in the scene space; Step 3: Based on the dynamic scene graph, perform hierarchical modeling of the association relationship of the behavior space corresponding to the intelligent agent to generate behavioral semantic features with scene perception; Step 4: Map the behavior semantic features containing scene perception to the agent strategy representation space to obtain the behavior feature strategy for controlling the agent to execute the target task sequence; Step 5: Based on the determined scenario value representation, the behavior feature strategy is decomposed to obtain the estimated advantage value of the scenario adaptation, so as to perform online optimization of the behavior feature strategy.

2. The method according to claim 1, characterized in that The step 1 specifically includes: Step 11: Obtain the user's scenario requirement description; Step 12: Perform entity extraction on the scenario requirement description to obtain the task target entity set; Step 13: Generate a target task sequence that can be executed by the agent based on the task target entity set.

3. The method according to claim 2, characterized in that The step 13 specifically includes: Step 131: Perform temporal dependency analysis on the task target entity set to obtain a task execution priority queue; Step 132: Perform atomic task decomposition processing on the task execution priority queue to generate a target task sequence executable by the agent, wherein the target task sequence includes task execution parameter constraints.

4. The method according to claim 3, characterized in that The step 3 specifically includes: Step 31: Generate a node feature matrix based on the dynamic scene graph, where the node feature matrix represents the semantic vector of each behavior type; Step 32: Generate a multi-level representation of the behavior space based on the node feature matrix; Step 33: Based on the task target semantic mapping model, feature alignment processing is performed on the task execution priority queue to generate a behavior semantic feature vector containing scene perception.

5. The method according to claim 4, characterized in that The step 32 specifically includes: Step 321: Based on the topological neighborhood aggregation algorithm, iteratively update the node feature matrix to generate a feature matrix containing neighborhood collaboration information; Step 322: Perform hierarchical attention weighting processing on the feature matrix containing neighborhood collaborative information to generate a multi-level representation of the behavior space to characterize the correlation relationship in the agent's behavior space.

6. The method according to claim 1, characterized in that The step 4 specifically includes: Step 41: Perform multi-head linear transformation on the behavior semantic features containing scene perception to obtain a behavior feature interaction matrix; Step 42: Generate a temporal constraint mask matrix in the strategy representation space based on the behavior feature interaction matrix; Step 43: Generate a behavior feature strategy for controlling the intelligent agent according to the timing constraint mask matrix, wherein the behavior feature strategy is expressed in the form of multivariate Gaussian distribution parameters.

7. The method according to claim 6, characterized in that The step 41 specifically includes: Step 411: Perform multi-head linear transformation on the behavior semantic features containing scene perception to generate feature triples including query / key / value; Step 412: Based on the cross-agent interaction weight matrix, perform attention mechanism calculation processing on the feature triples to obtain the behavior feature interaction matrix.

8. The method according to claim 1, characterized in that The step 5 specifically includes: Step 51: Determine a scenario value representation based on a generalized advantage estimation model, wherein the scenario value representation represents an estimated value of an advantage function of a cumulative discounted reward; Step 52: Perform probability density decomposition on the behavior feature strategy based on the hierarchical value decomposition network to obtain a new and old strategy distribution ratio matrix; Step 53: Based on the scenario value representation, dynamically truncate the new and old strategy distribution ratio matrix to obtain an estimated advantage value for scenario adaptation; Step 54: Based on the advantage estimate, fine-tune the parameters of the behavior feature strategy to achieve online optimization of the behavior feature strategy.

Images