Intelligent agent control method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of intelligent decision making, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses an agent control method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining multi-source heterogeneous information of a target agent, and constructing a feature vector according to the multi-source heterogeneous information; feature mapping is carried out on the feature vectors to obtain mapping features, and the expert weight of each expert network in a pre-constructed expert network library is calculated according to the mapping features; selecting a target expert network set according to the expert weight, and calculating agent trajectory data of the mapping features based on each target expert network in the target expert network set; performing network parameter updating on the target expert network according to the agent trajectory data to obtain an updated expert network; and generating an intelligent control strategy of the target agent based on the updated expert network.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology, and in particular to an intelligent agent control method, device, electronic device, and storage medium. Background Technology

[0002] In the current field of intelligent agents, mainstream products and research results mostly adopt a single-policy network architecture, that is, one intelligent agent corresponds to a fixed policy network to handle various task scenarios. Although some improved solutions for complex tasks have introduced multi-policy branch designs, they only switch policies through simple task type matching and do not achieve the dynamism and adaptability of policy selection.

[0003] In the field of large language models, hybrid expert architectures have been applied, but their core objective is to improve the parameter scale and computational efficiency of language models. Their application scenarios are limited to natural language processing tasks and have not been transferred to the policy control domain of intelligent agents. Among existing intelligent agent training techniques, the PPO (Proximal Policy Optimization) algorithm is widely used, but it is based on a full parameter update mode and does not incorporate sparse activation mechanisms to achieve precise allocation of computational resources. This leads to the following defects and shortcomings in existing technologies: 1. Poor task adaptability: Single-policy networks need to take into account multiple task scenarios, which leads to insufficient fitting accuracy of the model for specific tasks. In scenarios with large differences in task types, decision bias is prone to occur. For example, the same agent may not be able to achieve optimal performance in both "path planning" and "interaction and collaboration" tasks at the same time.

[0004] 2. Low training efficiency: Traditional multi-policy agents adopt a full parameter update mode. Regardless of whether the task requires it, the parameters of all policy branches will participate in the training, resulting in a large waste of computing resources and a significant extension of the training cycle. Especially in agent collaboration scenarios, the computing cost burden is heavy.

[0005] 3. Excessive resource consumption: During operation, the full-policy activation mode requires loading all policy network parameters, which places stringent requirements on the storage and computing power of hardware devices, making it difficult to adapt to resource-constrained scenarios such as edge computing devices.

[0006] 4. Insufficient scalability: When adding new task types, the original policy network needs to be reconstructed or new network layers need to be superimposed, which can easily damage the original stability of the model and cannot achieve modular addition or deletion of expert policies.

[0007] In the healthcare field, intelligent agents can integrate local drug resistance data and patient contraindications to generate personalized treatment plans (such as combination therapy of β-lactams and macrolides); and formulate overall strategies, such as continuous monitoring of vital signs and multidisciplinary collaboration. In the fintech field, intelligent agents can be used to monitor enterprise operating data in real time for pre-loan credit granting and post-loan management.

[0008] Therefore, how to deeply integrate hybrid expert architecture with agent policy network to improve the accuracy of agent control has become an urgent problem to be solved. Summary of the Invention

[0009] This invention provides an intelligent agent control method, device, electronic device, and storage medium to solve the problem of low accuracy in intelligent agent control under complex scenarios, thereby improving the precision of intelligent agent control.

[0010] Firstly, an intelligent agent control method is provided, including: Obtain multi-source heterogeneous information of the target intelligent agent, and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information; The feature vector is subjected to feature mapping to obtain the mapped features, and the expert weights of each expert network in the pre-constructed expert network library are calculated based on the mapped features. A set of target expert networks is selected from the expert network library based on the expert weights, and agent trajectory data of the mapping features is calculated based on each target expert network in the set of target expert networks; The network parameters of each target expert network are updated based on the trajectory data of the intelligent agent to obtain an updated expert network. The intelligent control strategy for the target agent is generated based on the updated expert network.

[0011] Secondly, an intelligent agent control device is provided, comprising: The feature vector construction module is used to obtain multi-source heterogeneous information of the target intelligent agent and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information; The expert weight calculation module is used to perform feature mapping on the feature vector to obtain the mapped features, and calculate the expert weight of each expert network in the pre-built expert network library based on the mapped features. The agent trajectory data calculation module is used to select a set of target expert networks from the expert network library according to the expert weights, and calculate the agent trajectory data of the mapping features based on each target expert network in the set of target expert networks; The network parameter update module is used to update the network parameters of each target expert network according to the intelligent agent trajectory data to obtain an updated expert network. The intelligent control strategy generation module is used to generate an intelligent control strategy for the target intelligent agent based on the updated expert network.

[0012] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned intelligent agent control method.

[0013] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the aforementioned intelligent agent control method.

[0014] In the aforementioned intelligent agent control method, device, electronic device, and storage medium, the feature vector of the target intelligent agent is constructed by multi-source heterogeneous information. This allows for the capture of feature information of the target intelligent agent from multiple information sources, enriching the semantic information of the feature vector. Calculating expert weights based on the feature vectors reflects the adaptability of each expert network to the current task of the target intelligent agent, effectively avoiding resource waste caused by the simultaneous activation of multiple expert networks and improving computational efficiency. Based on the expert weights, the target expert network set is selected, and intelligent agent trajectory data is calculated. The network parameters of the target expert network are updated based on the intelligent agent trajectory data to obtain an updated expert network. Furthermore, an intelligent control strategy is generated based on the updated expert network, enabling the output of more suitable decision results in various task scenarios, thus achieving precise control of the target intelligent agent. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of an application environment for an intelligent agent control method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an intelligent agent control method according to an embodiment of the present invention; Figure 3 yes Figure 2 A flowchart illustrating a specific implementation method of step S1; Figure 4 yes Figure 2 A flowchart illustrating a specific implementation method of step S3; Figure 5 This is a schematic diagram of a smart agent control device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 7This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The intelligent agent control method provided in this embodiment of the invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can obtain multi-source heterogeneous information about the target intelligent agent, construct a feature vector of the target intelligent agent based on the multi-source heterogeneous information, perform feature mapping on the feature vector to obtain mapped features, calculate the expert weight of each expert network in a pre-built expert network library based on the mapped features, select a set of target expert networks from the expert network library based on the expert weights, calculate the intelligent agent trajectory data of the mapped features based on each target expert network in the target expert network set, update the network parameters of each target expert network based on the intelligent agent trajectory data to obtain an updated expert network, and generate an intelligent control strategy for the target intelligent agent based on the updated expert network. This invention provides an intelligent agent control device that updates the network parameters of a target expert network using intelligent agent trajectory data to obtain an updated expert network. This allows for updating only the target expert network parameters, reducing invalid parameter calculations. By generating an intelligent control strategy through the updated expert network, it can output decision results that better meet the needs in various task scenarios, achieving precise control of the target intelligent agent. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0019] Please see Figure 2 As shown, Figure 2 A flowchart illustrating an intelligent agent control method provided in an embodiment of the present invention includes the following steps: S1. Obtain multi-source heterogeneous information of the target intelligent agent, and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information.

[0020] In this embodiment of the invention, the target intelligent agent refers to an autonomous system capable of perceiving the environment, reasoning and making decisions based on internal states or models, and executing actions to achieve a goal. For example, in the field of autonomous driving, the target intelligent agent can be a decision control module applied to an autonomous driving system; in the field of industrial intelligent control, the target intelligent agent can be used in scenarios such as multi-equipment collaborative control and intelligent quality inspection on production lines, such as an intelligent system in an electronic product assembly line; in the field of game artificial intelligence, the target intelligent agent can be an NPC design system applied to massively multiplayer online games (MMOs) or strategy games; in the field of intelligent logistics, the target intelligent agent can be a decision control system used for equipment such as warehouse robots and delivery drones.

[0021] Specifically, in the healthcare field, intelligent agents can directly take over or intervene in clinical processes. For example, after the previous shift nurse voice-records the patient's status, the intelligent agent automatically generates a structured handover form and synchronizes it to the infusion pump and monitor parameter setting interface. If it is detected that the incoming nurse has not confirmed the dosage of critical high-risk drugs, the infusion pump is temporarily locked by the intelligent agent until the voice verification is completed. Before surgery, the agent automatically calls up the anemia / coagulation / autologous blood collection assessment model → generates a blood preparation plan → directly invokes the blood use request form and writes it into the inventory system. During surgery, the agent monitors the amount of bleeding and vital signs in real time → dynamically corrects the optimal hemoglobin target → if the predicted deficit is >1 U, it automatically adds a blood matching instruction to the blood bank. If the actual Hb value is more than 10% lower than the intelligent agent's predicted value within 24 hours after surgery, it triggers a "review-learning" closed loop and automatically updates the prediction coefficient for the next similar surgery.

[0022] In the fintech field, intelligent agents can drive transactions, risk control, and operations. For example, they can monitor client portfolio deviations, market volatility, and macroeconomic events in real time: when any client portfolio deviates from its strategic allocation by ≥3% and the expected loss is >1%, the agent automatically generates rebalancing instructions (including fund subscriptions and redemptions, and applications for quick redemption quotas for money market funds). After one-click confirmation in the client's app, the system directly connects to the agent and the exchange to complete the transaction. It performs 24 / 7 real-time streaming processing of over 200 dimensions of features, including transactions, public opinion, MAC addresses, and device fingerprints. When the anomaly score is >0.82 and involves funds >500,000 RMB, the agent immediately calls the "account freeze" API, simultaneously generating a risk report and pushing it to compliance personnel. If compliance does not reject the report within 5 minutes, it automatically reports to regulators and initiates post-loan / post-investment processes.

[0023] Furthermore, multi-source heterogeneous information includes heterogeneous information from multiple sources, such as environmental state information of the target intelligent agent (e.g., spatial coordinates, obstacle distribution, interactive object behavior, etc.), task objective instructions (e.g., "reach the designated location" "complete the transport of items"), and system constraint conditions (e.g., energy consumption limit, computing power limit, etc.). Feature encoding is performed on multi-source heterogeneous information to obtain feature vectors that represent the key information of the target intelligent agent.

[0024] Specifically, see Figure 3 As shown, constructing the feature vector of the target agent based on the multi-source heterogeneous information includes: S31. Embed and encode the heterogeneous information of each information source in the multi-source heterogeneous information to generate encoding features; S32. Perform multi-head self-attention calculation on the encoded features to obtain attention features; S33. Perform a fully connected computation on the attention features to obtain the information source features corresponding to each information source; S34. Perform feature fusion on the information source features to form the feature vector of the target intelligent agent.

[0025] In this embodiment of the invention, the information sources in the multi-source heterogeneous information include environmental state information, task target instructions, and system constraints. The environmental state information, task target instructions, and system constraints are embedded and encoded respectively to obtain encoded features.

[0026] Among them, based on the pre-built Transformer encoder structure, the correlation between input information can be captured through the self-attention mechanism to obtain the information source features corresponding to each information source.

[0027] Specifically, the Transformer encoder structure includes word embedding + position encoding, and then uses multiple stacked multi-head self-attention mechanisms and position-wise feed-forward networks to extract features for each information source.

[0028] In this process, word embedding maps the heterogeneous information from each information source into a continuous high-dimensional vector, and then performs positional encoding through sine and cosine coding to obtain the encoded features.

[0029] Furthermore, attention computation involves performing a linear transformation on the encoded features to obtain query, key, and value matrices. The similarity between each query matrix and all key matrices is calculated to obtain an attention score. This score is then converted into weights of 0 to 1 using softmax, with the sum of these weights being 1. The normalized weights are then used to perform a weighted summation on the value vector V matrix to obtain the single-head attention feature for each feature value of the encoded features. Multi-head self-attention computation involves concatenating multiple single-head attention features and integrating them using a linear transformation matrix to obtain the final attention feature.

[0030] In this embodiment of the invention, the feature fusion of the information source features involves unifying the feature dimensions of the information source features and then performing a weighted summation. For example, a feature weight is set for each information source, and the feature weight is multiplied by the corresponding information source feature, followed by vector concatenation to obtain the feature vector of the target agent. The dimension of the feature vector can be d_model.

[0031] In this embodiment of the invention, the feature vector of the target intelligent agent is constructed by multi-source heterogeneous information, which can capture the feature information of the target intelligent agent from multiple information sources and enrich the semantic information of the feature vector.

[0032] S2. Perform feature mapping on the feature vector to obtain the mapped features, and calculate the expert weights of each expert network in the pre-built expert network library based on the mapped features.

[0033] In this embodiment of the invention, feature mapping is the process of mapping feature vectors from their original feature space to a gated decision feature space through linear or nonlinear transformations. Gated decision is a decision-making method based on a "gating mechanism," using a gated network in the gated decision layer as the core of the architecture. This network receives the feature vector s output from the input parsing layer and dynamically decides which expert network the target agent should activate in the current task scenario.

[0034] Specifically, the step of performing feature mapping on the feature vector to obtain mapped features includes: Obtain the projection weight matrix and bias vector of the preset gated network; The feature vector is multiplied by the projection weight matrix to obtain the projection vector; The projection vector and the bias vector are concatenated to generate the mapping feature.

[0035] Gated networks are neural networks or computational architectures with a gating mechanism as their core component. Essentially, they use learnable or rule-defined "gates" to dynamically control the transmission, filtering, weighting, or forgetting of information within the network. This allows the network to adapt to complex data characteristics (such as temporal dependencies, noise interference, and nonlinear relationships), ultimately improving the model's expressive power, robustness, and task adaptability.

[0036] For example, a gated network can have an input layer to receive mapped features; a gate layer to calculate gate weights through a gate function and dynamically filter input information; a feature processing layer to perform linear / nonlinear transformations (such as convolution, fully connected layers, and iterative computation) on the filtered information to extract higher-order features; and an output layer to output the results of the feature processing layer.

[0037] Furthermore, in the gated network, the projection weight matrix and the bias vector are the core learnable parameters of the gated layer and the feature processing layer. Therefore, the feature vector can be mapped through the projection weight matrix and the bias vector of the gated network to obtain the mapped features required by the input of the composite expert network.

[0038] In this embodiment of the invention, the feature vector can be mapped using the following formula to obtain the mapped features:

[0039] in, Representing the eigenvector The corresponding mapping features, Represents the projection weight matrix. This represents the bias vector.

[0040] Furthermore, each expert network in the expert network library consists of multiple structurally independent and functionally specialized expert policy heads. Each expert network is specifically optimized for a particular task type (such as path planning expert, obstacle avoidance expert, interactive collaboration expert, etc.). After receiving the activation command from the gating network, it performs policy calculation and outputs the decision result, i.e., the intelligent control policy.

[0041] Specifically, the step of calculating the expert weights of each expert network in the pre-built expert network library based on the mapping features includes: The expert weight vector and expert bias term of each expert network in the expert network library are calculated based on a preset gated network. Based on the expert weight vector and the expert bias term, feature mapping is performed on the mapped features to obtain the second mapped features; The activation operation is performed on the second mapping feature to obtain the expert weights of each expert network.

[0042] In this embodiment of the invention, the expert network has its own weight matrix and bias vector, which are responsible for processing the mapping features of the input. The gating network sets expert weight vectors and expert bias terms for each expert network. The expert weight vectors and expert bias terms can be updated through agent trajectory data until an intelligent control strategy is obtained.

[0043] Furthermore, based on the above-described step of performing feature mapping on the feature vector to obtain mapped features, feature mapping can be performed on the mapped features based on the expert weight vector and the expert bias term to obtain second mapped features, and then based on... The activation function performs activation operations to obtain the expert weights of the expert network.

[0044] Specifically, the expert weights of each expert network in the pre-built expert network library can be calculated using the following formula:

[0045] in, The first in the expert network database Expert weights in an expert network This represents the activation function. This indicates that the gating network is for the first The expert weight vector of an expert network. Represents mapping features, This indicates that the gating network is for the first Expert bias of an expert network.

[0046] In this embodiment of the invention, expert weights can reflect the degree of adaptation of each expert network to the current task of the target agent, effectively avoiding resource waste caused by the simultaneous activation of multiple expert networks and improving computational efficiency.

[0047] S3. Select a set of target expert networks from the expert network library according to the expert weights, and calculate the agent trajectory data of the mapping features based on each target expert network in the set of target expert networks.

[0048] In this embodiment of the invention, to avoid resource waste caused by the simultaneous activation of multiple experts, an expert weight threshold τ is set (which can be dynamically adjusted according to the task complexity, with a value range of 0-1), and only experts whose weights satisfy the specified conditions are activated. An expert network with a weight ≥τ is used; if all expert weights are lower than the expert weight, only the expert network with the largest expert weight is activated to ensure decision continuity.

[0049] In detail, the target expert network set generates activation signals through a gating network, receives mapping features through activation signals, and uses these features to generate expert networks for intelligent strategies. By activating the expert network, the optimal expert network can be matched, avoiding the problem of "excessive generalization but insufficient specificity" of a single strategy, and enabling the target intelligent agent to output decision results that better meet the needs in various task scenarios.

[0050] In this embodiment of the invention, the agent trajectory data is the mapping features of each target expert network from the input - target expert network set. - Decision action a - Reward signal r - Five-tuple of the next state

[0051] Furthermore, by selecting a target set of expert networks from the expert network library, the gating network can automatically select a suboptimal expert network to complete the task when some expert networks are abnormal, avoiding system collapse caused by the failure of a single strategy and effectively enhancing the adaptability of the target intelligent agent in complex and uncertain environments.

[0052] Specifically, see Figure 4 As shown, the process of calculating the agent trajectory data based on the mapping features of each target expert network in the target expert network set includes: S41. Based on each target expert network in the target expert network set, perform action probability prediction on the mapped features to obtain the prediction probability; S42. Determine the decision action corresponding to the mapping feature based on the predicted probability; S43. Calculate a reward signal based on the decision action according to the preset action reward rules; S44. Generate agent trajectory data corresponding to the target agent based on the decision action and the reward signal.

[0053] In this embodiment of the invention, each expert network adopts a standardized structure of "input interface - core computation - output interface". The input interface uniformly receives activation signals from the gating network and mapping features from the input parsing layer. The core computation layer employs a customized network structure based on its task type (e.g., graph neural networks for path planning experts and recurrent neural networks for interactive collaboration experts). The output interface uniformly outputs the action probability distribution for different decision actions, ensuring seamless integration of different expert networks into the system. When adding a new task, only a new expert module conforming to the interface specification needs to be developed, and the gating network needs to be fine-tuned to incorporate the new expert's decision logic; no changes to the original architecture are required.

[0054] Furthermore, the next state in the agent's trajectory data is the new feature vector generated by the target agent after the decision action is applied. The reward signal is the core feedback signal for measuring the "reasonableness of gating weight allocation" and the "effectiveness of expert decision-making." Through the reward signal, the gating network learns to select the correct expert network in different task scenarios, and the expert network calculates more accurate decision actions.

[0055] The preset action reward rules define "correct actions" and "incorrect actions" (such as the target agent taking emergency avoidance as a "correct action"), set a fixed reward value (reward for correct actions) and a penalty value (penalty for incorrect actions), and directly assign a reward signal based on the result of the decision action.

[0056] S4. Update the network parameters of each target expert network according to the trajectory data of the intelligent agent to obtain the updated expert network.

[0057] In this embodiment of the invention, network parameter updating involves updating the parameters of the target expert network, such as scientific parameters like weight matrices and bias terms, as well as the parameters of the gating network optimized by the reward signal in the trajectory data, such as scientific parameters like weight matrices and bias terms. This ensures that the decision logic of the gating network matches the performance improvement of the expert network, thereby enhancing the accuracy of expert selection.

[0058] Specifically, the step of updating the network parameters of each target expert network based on the agent trajectory data to obtain an updated expert network includes: Calculate the objective function parameters based on the agent's trajectory data; Construct an optimization objective function based on the objective function parameters; The parameters of the target expert network are updated based on the optimization objective function to obtain the updated expert network.

[0059] In this embodiment of the invention, the objective function parameter is the ratio of the predicted probability of the current decision action to the predicted probability of the old decision action in the agent's trajectory data. An optimization function is constructed based on the predicted probability ratio to obtain the optimized objective function.

[0060] Specifically, the optimization objective function is expressed as:

[0061] in, This represents the function value of the objective function. Indicates the first Objective function parameters of a target expert network This represents the network parameters of the target expert network. This indicates the expectation for the update time step. This represents the pre-defined advantage function. This indicates a cropping operation. This indicates the preset clipping parameters.

[0062] In this embodiment of the invention, the trimming parameter is typically set to 0.2.

[0063] In another optional embodiment of the present invention, an optimization objective function is constructed based on the objective function of the Proximal Policy Optimization (PPO) algorithm. The Proximal Policy Optimization algorithm maximizes policy benefits while constraining the magnitude of policy updates ("proximal" constraint) to avoid parameter update collapse caused by excessively large update steps, thereby obtaining a more accurate update expert network.

[0064] For example, suppose the target agent is in state st: the advantage function for choosing action a1 is 2, the probability of choosing action a1 with old parameters is 0.2; the initial probability of choosing action a1 with new parameters is 0.3, then the objective function parameter is 0.3 / 0.2 = 1.5; let... =0.2, clip range is [0.8, 1.2], objective function parameter = 1.5 is clipped to 1.2; the optimized objective function value is min(1.5×2, 1.2×2) = min(3, 2.4) = 2.4; optimizing the objective function will update the new parameter in the direction of increasing this value, but will not exceed 0.2×1.2 = 0.24, thus avoiding excessive update magnitude; if the probability of selecting a1 as the new parameter is 0.22 (objective function parameter = 1.1, within the clip range), then the optimized objective function value is 1.1×2 = 2.2, which will be retained, and the parameter can be updated normally.

[0065] Preferably, the parameters are updated iteratively using the gradient ascent method until the cumulative reward of the policy (such as the total reward of the agent) no longer increases, thus obtaining the final updated expert network.

[0066] In this embodiment of the invention, the sparse activation mechanism through network parameter updates can reduce resource consumption from both training and operation dimensions. During training, only the target expert network parameters are updated to reduce the calculation of invalid parameters; during operation, only the parameters of the gating network and the activation experts are loaded, reducing hardware storage and computing power requirements, making the target intelligent agent adaptable to resource-constrained scenarios such as edge computing devices.

[0067] S5. Generate the intelligent control strategy for the target agent based on the updated expert network.

[0068] In this embodiment of the invention, the intelligent control strategy is the decision-making action of the target intelligent agent in the next stage. The intelligent control strategy can control the execution action of the target intelligent agent, thereby achieving precise control of the target intelligent agent.

[0069] Specifically, the intelligent control strategy for generating the target agent based on the updated expert network includes: The updated expert network is used to perform feature mapping on the feature vector to obtain the target mapping features; Decision action prediction is performed on the target mapping features to obtain the prediction result; The intelligent control strategy for generating the target intelligent agent is determined based on the prediction results.

[0070] In this embodiment of the invention, feature mapping maps the feature vector to the feature dimension of the updated expert network by updating the weight vector and bias term. This allows for the direct calculation of the prediction probability of different decision actions, thus obtaining the prediction result.

[0071] For example, if the target intelligent agent is in the field of autonomous driving, a dedicated expert network is designed for different road condition scenarios such as "high-speed cruising", "urban congestion", "emergency avoidance", and "parking-car". The gating network activates the optimal expert based on environmental data input from sensors such as LiDAR and cameras, thereby improving the safety and reliability of decision-making in complex road conditions.

[0072] In the field of industrial intelligent control, target intelligent agents are used in scenarios such as multi-equipment collaborative control and intelligent quality inspection on production lines. For example, in an electronic product assembly line, "equipment scheduling experts," "fault diagnosis experts," and "quality inspection experts" can be set up. The gating network activates the corresponding experts based on the real-time status of the production line (such as equipment load and product inspection results) to achieve intelligent and efficient management and control of the production process.

[0073] In the field of game artificial intelligence, the target intelligent agent is applied to the design of NPCs in massively multiplayer online games (MMOs) or strategy games. The NPCs are equipped with "combat strategy experts", "quest guidance experts" and "social interaction experts". The gating network dynamically switches expert strategies according to the game scenario (such as players launching attacks or triggering quest plots), so that the NPC behavior is more in line with the needs of the scenario and enhances the immersion and interactivity of the game.

[0074] In the field of intelligent logistics, the target intelligent agent is used for decision-making and control of equipment such as warehouse robots and delivery drones. Warehouse robots can be equipped with "path planning experts," "cargo recognition experts," and "loading and unloading collaboration experts." The gating network activates the corresponding experts based on the warehouse environment (such as shelf layout and cargo type) to improve warehouse operation efficiency. Delivery drones can activate dedicated experts for scenarios such as "high-altitude cruising," "low-altitude obstacle avoidance," and "precision landing" to ensure a stable and reliable delivery process.

[0075] In the healthcare field, target intelligent agents can automatically generate patient profiles and accurately push medication reminders, side effect management solutions, and repurchase suggestions. In the fintech field, each target intelligent agent outputs a confidence score, and the final decision is formed through dynamic weight allocation. For example, when a black swan event occurs in the market, the weight of the risk control intelligent agent automatically increases to 70%, the threshold for triggering forced liquidation logic decreases by 30%, and a decision tree visualization report is generated for each transaction, showing the contribution of each intelligent agent.

[0076] Therefore, by updating the expert network to generate intelligent control strategies, the target intelligent agent can output decision results that better meet the needs in various task scenarios, thus achieving precise control of the target intelligent agent.

[0077] As can be seen, in the above scheme, constructing the feature vector of the target intelligent agent through multi-source heterogeneous information can capture the feature information of the target intelligent agent from multiple information sources, enriching the semantic information of the feature vector; calculating expert weights based on feature vectors can reflect the degree of adaptation of each expert network to the current task of the target intelligent agent, effectively avoiding the waste of resources caused by the simultaneous activation of multiple expert networks and improving computational efficiency; calculating the intelligent agent trajectory data from the selected target expert network set based on expert weights can update the network parameters of the target expert network based on the intelligent agent trajectory data, obtaining an updated expert network, and then generating an intelligent control strategy based on the updated expert network. This can output more demanding decision results in various task scenarios, achieving precise control of the target intelligent agent.

[0078] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0079] In one embodiment, an intelligent agent control device is provided, which corresponds one-to-one with the intelligent agent control method described in the above embodiments. For example... Figure 5 As shown, the intelligent agent control device 100 includes a feature vector construction module 101, an expert weight calculation module 102, an intelligent agent trajectory data calculation module 103, a network parameter update module 104, and an intelligent control strategy generation module 105. Detailed descriptions of each functional module are as follows: The feature vector construction module 101 is used to acquire multi-source heterogeneous information of the target intelligent agent and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information. The expert weight calculation module 102 is used to perform feature mapping on the feature vector to obtain the mapped features, and calculate the expert weight of each expert network in the pre-built expert network library based on the mapped features. The agent trajectory data calculation module 103 is used to select a set of target expert networks from the expert network library according to the expert weights, and calculate the agent trajectory data of the mapping features based on each target expert network in the set of target expert networks; The network parameter update module 104 is used to update the network parameters of each target expert network according to the intelligent agent trajectory data to obtain an updated expert network. The intelligent control strategy generation module 105 is used to generate an intelligent control strategy for the target intelligent agent based on the updated expert network.

[0080] In one embodiment, the feature vector construction module 101 constructs a feature vector of the target agent based on the multi-source heterogeneous information, for the following purposes: The heterogeneous information of each information source in the multi-source heterogeneous information is embedded and encoded to generate encoded features; Multi-head self-attention computation is performed on the encoded features to obtain attention features; Fully connected computation is performed on the attention features to form information source features corresponding to each information source; The features of the information source are fused to obtain the feature vector of the target intelligent agent.

[0081] In one embodiment, the expert weight calculation module 102 performs feature mapping on the feature vector to obtain mapped features, which are used for: Obtain the projection weight matrix and bias vector of the preset gated network; The feature vector is multiplied by the projection weight matrix to obtain the projection vector; The projection vector and the bias vector are concatenated to generate the mapping feature.

[0082] In one embodiment, the expert weight calculation module 102 calculates the expert weights of each expert network in the pre-built expert network library based on the mapping features, for the following purposes: The expert weight vector and expert bias term of each expert network in the expert network library are calculated based on a preset gated network. Based on the expert weight vector and the expert bias term, feature mapping is performed on the mapped features to obtain the second mapped features; The activation operation is performed on the second mapping feature to obtain the expert weights of each expert network.

[0083] In one embodiment, the agent trajectory data calculation module 103 calculates agent trajectory data based on the mapping features of each target expert network in the target expert network set, for the following purposes: Based on each target expert network in the target expert network set, the action probability of the mapped features is predicted to obtain the predicted probability. The decision action corresponding to the mapping feature is determined based on the predicted probability; Calculate a reward signal based on the decision-making action according to a preset action reward rule; Based on the decision-making action and the reward signal, generate the trajectory data of the target agent.

[0084] In one embodiment, the network parameter update module 104 updates the network parameters of each target expert network based on the agent trajectory data to obtain an updated expert network, for the following purposes: Calculate the objective function parameters based on the agent's trajectory data; Construct an optimization objective function based on the objective function parameters; The parameters of the target expert network are updated based on the optimization objective function to obtain the updated expert network.

[0085] In one embodiment, the intelligent control strategy generation module 105 generates an intelligent control strategy for the target agent based on the updated expert network, including: The updated expert network is used to perform feature mapping on the feature vector to obtain the target mapping features; Decision action prediction is performed on the target mapping features to obtain the prediction result; The intelligent control strategy for generating the target intelligent agent is determined based on the prediction results.

[0086] This invention provides an intelligent agent control device that constructs feature vectors of a target intelligent agent through multi-source heterogeneous information. This allows for the capture of feature information from multiple information sources, enriching the semantic information of the feature vectors. Expert weights are calculated based on the feature vectors, reflecting the suitability of each expert network for the current task of the target intelligent agent. This effectively avoids resource waste caused by the simultaneous activation of multiple expert networks and improves computational efficiency. Based on the expert weights, the device selects a set of target expert networks and calculates the intelligent agent's trajectory data. The device then updates the network parameters of the target expert networks based on this trajectory data, resulting in an updated expert network. Furthermore, an intelligent control strategy is generated based on the updated expert network, enabling the device to output more tailored decision results in various task scenarios and achieve precise control of the target intelligent agent.

[0087] For specific limitations regarding an intelligent agent control device, please refer to the limitations regarding an intelligent agent control method described above, which will not be repeated here. Each module in the aforementioned intelligent agent control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0088] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side intelligent agent control method.

[0089] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a client-side intelligent agent control method. In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain multi-source heterogeneous information of the target intelligent agent, and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information; The feature vector is subjected to feature mapping to obtain the mapped features, and the expert weights of each expert network in the pre-constructed expert network library are calculated based on the mapped features. A set of target expert networks is selected from the expert network library based on the expert weights, and agent trajectory data of the mapping features is calculated based on each target expert network in the set of target expert networks; The network parameters of each target expert network are updated based on the trajectory data of the intelligent agent to obtain an updated expert network. The intelligent control strategy for the target agent is generated based on the updated expert network.

[0090] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain multi-source heterogeneous information of the target intelligent agent, and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information; The feature vector is subjected to feature mapping to obtain the mapped features, and the expert weights of each expert network in the pre-constructed expert network library are calculated based on the mapped features. A set of target expert networks is selected from the expert network library based on the expert weights, and agent trajectory data of the mapping features is calculated based on each target expert network in the set of target expert networks; The network parameters of each target expert network are updated based on the trajectory data of the intelligent agent to obtain an updated expert network. The intelligent control strategy for the target agent is generated based on the updated expert network.

[0091] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0093] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0094] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

[0095] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for controlling an intelligent agent, characterized in that, include: Obtain multi-source heterogeneous information of the target intelligent agent, and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information; The feature vector is subjected to feature mapping to obtain the mapped features, and the expert weights of each expert network in the pre-constructed expert network library are calculated based on the mapped features. A set of target expert networks is selected from the expert network library based on the expert weights, and agent trajectory data of the mapping features is calculated based on each target expert network in the set of target expert networks; The network parameters of each target expert network are updated based on the trajectory data of the intelligent agent to obtain an updated expert network. The intelligent control strategy for the target agent is generated based on the updated expert network.

2. The intelligent agent control method as described in claim 1, characterized in that, The step of constructing the feature vector of the target intelligent agent based on the multi-source heterogeneous information includes: The heterogeneous information of each information source in the multi-source heterogeneous information is embedded and encoded to generate encoded features; Multi-head self-attention computation is performed on the encoded features to obtain attention features; Perform fully connected computation on the attention features to obtain the information source features corresponding to each information source; The information source features are fused to form the feature vector of the target intelligent agent.

3. The intelligent agent control method as described in claim 1, characterized in that, The step of performing feature mapping on the feature vector to obtain mapped features includes: Obtain the projection weight matrix and bias vector of the preset gated network; The feature vector is multiplied by the projection weight matrix to obtain the projection vector; The projection vector and the bias vector are concatenated to generate the mapping feature.

4. The intelligent agent control method as described in claim 1, characterized in that, The step of calculating the expert weights of each expert network in the pre-built expert network library based on the mapping features includes: The expert weight vector and expert bias term of each expert network in the expert network library are calculated based on a preset gated network. Based on the expert weight vector and the expert bias term, feature mapping is performed on the mapped features to obtain the second mapped features; The activation operation is performed on the second mapping feature to obtain the expert weights of each expert network.

5. The intelligent agent control method as described in claim 1, characterized in that, The process of calculating the agent trajectory data based on the mapped features of each target expert network in the target expert network set includes: Based on each target expert network in the target expert network set, the action probability of the mapped features is predicted to obtain the predicted probability. The decision action corresponding to the mapping feature is determined based on the predicted probability; Calculate a reward signal based on the decision-making action according to a preset action reward rule; Based on the decision-making action and the reward signal, the trajectory data of the target agent is generated.

6. The intelligent agent control method as described in claim 1, characterized in that, The step of updating the network parameters of each target expert network based on the agent trajectory data to obtain an updated expert network includes: Calculate the objective function parameters based on the agent's trajectory data; Construct an optimization objective function based on the objective function parameters; The parameters of the target expert network are updated based on the optimization objective function to obtain the updated expert network.

7. The intelligent agent control method as described in claim 1, characterized in that, The intelligent control strategy for generating the target agent based on the updated expert network includes: The updated expert network is used to perform feature mapping on the feature vector to obtain the target mapping features; Decision action prediction is performed on the target mapping features to obtain the prediction result; The intelligent control strategy for the target agent is generated based on the prediction results.

8. An intelligent agent control device, characterized in that, include: The feature vector construction module is used to obtain multi-source heterogeneous information of the target intelligent agent and construct the feature vector of the target intelligent agent based on the multi-source heterogeneous information; The expert weight calculation module is used to perform feature mapping on the feature vector to obtain the mapped features, and calculate the expert weight of each expert network in the pre-built expert network library based on the mapped features. The agent trajectory data calculation module is used to select a set of target expert networks from the expert network library according to the expert weights, and calculate the agent trajectory data of the mapping features based on each target expert network in the set of target expert networks; The network parameter update module is used to update the network parameters of each target expert network according to the intelligent agent trajectory data to obtain an updated expert network. The intelligent control strategy generation module is used to generate an intelligent control strategy for the target intelligent agent based on the updated expert network.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent agent control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent agent control method as described in any one of claims 1 to 7.

Images