Resource allocation method and device of intelligent agent, equipment and storage medium

Abstract

The invention relates to an agent resource allocation method and device, equipment and a storage medium. The method comprises the steps of collecting current task environment data related to a task demand from a surrounding environment where an intelligent agent is located according to the task demand of each task needing to be executed by the intelligent agent; for each task, performing feature extraction on the current task environment data by adopting a feature extraction network corresponding to the task to obtain a current task feature vector of the task; and mapping the feature vector of the current task by adopting a vector mapping network corresponding to the task to obtain the current priority of the task, and according to the current priority of each task and the resource condition of the intelligent agent, carrying out resource allocation on each task. By adopting the method, the intelligent agent determines the task priority according to the feature extraction network and the vector mapping network of each task, and performs resource allocation according to the task priority, complex network communication is not needed, and the resource allocation efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a resource allocation method, apparatus, device, and storage medium for an intelligent agent. Background Technology

[0002] In today's era of rapid technological development, intelligent agents, with their unique advantages, are gradually becoming one of the core research directions in the field of artificial intelligence, demonstrating enormous application potential in many areas.

[0003] Typically, intelligent agents face situations where multiple tasks need to be performed simultaneously. For example, a transport robot may need to perform navigation, obstacle avoidance, and transport tasks concurrently. Since the computing resources available to the agent are limited, resource allocation is necessary for different tasks. Currently, to avoid consuming the agent's computing resources, a resource allocation strategy is determined in the cloud and then sent to the agent. The agent can then directly allocate resources according to the strategy, reducing the computing resources required to determine the allocation strategy.

[0004] However, in practical applications of intelligent agents, millisecond-level response time has become a key indicator of their effective operation. The methods described above require complex network communication between the agent and the cloud to transmit data, which, even in high-speed network environments, struggles to meet millisecond-level real-time response requirements, resulting in low resource allocation efficiency. Summary of the Invention

[0005] Therefore, it is necessary to provide a resource allocation method, apparatus, device, and storage medium for intelligent agents that can improve the resource allocation efficiency of intelligent agents, addressing the aforementioned technical problems.

[0006] Firstly, this application provides a resource allocation method for an intelligent agent, including:

[0007] Based on the task requirements of each task that the agent needs to perform, collect current task environment data related to the task requirements from the surrounding environment of the agent;

[0008] For each task, a feature extraction network corresponding to the task is used to extract features from the current task environment data to obtain the current task feature vector; and,

[0009] The current priority of the task is obtained by mapping the feature vector of the current task using the vector mapping network corresponding to the task.

[0010] Resource allocation is performed for each task based on its current priority and the resource status of the agent.

[0011] In one embodiment, the step of allocating resources to each task based on the current priority of each task and the resource status of the agent includes: determining a resource allocation strategy based on the current priority of each task and the resource status of the agent; and allocating resources to each task according to the resource allocation strategy.

[0012] In one embodiment, the resource allocation for each of the tasks includes any of the following:

[0013] According to the current priority of each task in descending order, allocate the minimum resources required to execute each task; determine the resource allocation ratio for each task according to its current priority, and allocate resources to each task according to the resource allocation ratio; construct an objective function based on the current priority of each task, and solve the objective function with the resource status as a constraint to obtain the resource allocation amount for each task, and allocate resources to each task according to the resource allocation amount.

[0014] In one embodiment, the feature extraction network corresponding to each task is trained in the following manner: for each task, the initial extraction network corresponding to the task is used to extract features from the sample task environment data to obtain the sample feature vector of the task; based on the similarity between the sample feature vectors of different tasks, the parameters of each initial extraction network are adjusted to obtain the feature extraction network corresponding to each task.

[0015] In one embodiment, adjusting the parameters of each initial extraction network based on the similarity between sample feature vectors of different tasks includes: determining the similarity between each sample feature vector and other sample feature vectors; constructing a regularization loss function based on the determined similarity; and adjusting the parameters of each initial extraction network based on the regularization loss function.

[0016] In one embodiment, the method further includes: for each task, determining an instantaneous reward signal based on the execution status of the task and the amount of resources allocated to the task; and optimizing the feature extraction network and vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority.

[0017] In one embodiment, optimizing the feature extraction network and vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority includes: determining a first parameter gradient of the feature extraction network corresponding to the task and a second parameter gradient of the vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority; adjusting the parameters of the feature extraction network based on the first parameter gradient and the learning rate of the feature extraction network; and adjusting the parameters of the vector mapping network based on the second parameter gradient and the learning rate of the feature extraction network.

[0018] Secondly, this application also provides a resource allocation device for an intelligent agent, comprising:

[0019] The data acquisition module is used to collect current task environment data related to the task requirements of each task that the agent needs to perform from the surrounding environment of the agent.

[0020] The extraction module is configured to, for each task, use a feature extraction network corresponding to the task to extract features from the current task environment data to obtain the current task feature vector; and,

[0021] The mapping module is used to map the current task feature vector using the vector mapping network corresponding to the task to obtain the current priority of the task.

[0022] The allocation module is used to allocate resources to each task based on the current priority of each task and the resource status of the agent.

[0023] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the various method embodiments provided in the first aspect above.

[0024] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the various method embodiments provided in the first aspect above.

[0025] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the various method embodiments provided in the first aspect above.

[0026] The aforementioned resource allocation method, apparatus, device, and storage medium for intelligent agents first collect current task environment data related to the task requirements from the surrounding environment of the intelligent agent. Second, a feature extraction network corresponding to the task is used to extract features from the current task environment data to obtain the current task feature vector. Then, a vector mapping network corresponding to the task is used to map the current task feature vector to obtain the current priority of the task. Finally, resources are allocated to each task based on its current priority and the resource status of the intelligent agent. In this method, the intelligent agent directly determines the current priority of each task based on its corresponding feature extraction network and vector mapping network, and then allocates resources accordingly. This eliminates the need for complex network communication with the cloud, reducing communication overhead and improving resource allocation efficiency. Furthermore, since each task uses an independent feature extraction network and vector mapping network, when adding or deleting tasks in the intelligent agent, only the feature extraction network and vector mapping network for the corresponding task need to be added or deleted, without retraining the feature extraction networks and vector mapping networks for other tasks. This enables "plug-and-play" task management, improving the flexibility and real-time performance of the intelligent agent. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is an application environment diagram of a resource allocation method for intelligent agents in one embodiment;

[0029] Figure 2 This is a flowchart illustrating a resource allocation method for an intelligent agent in one embodiment;

[0030] Figure 3 This is a flowchart illustrating the resource allocation steps in one embodiment;

[0031] Figure 4 This is a flowchart illustrating the steps of training a feature extraction network in one embodiment;

[0032] Figure 5 This is a flowchart illustrating the parameter adjustment steps in one embodiment;

[0033] Figure 6 This is a flowchart illustrating the network optimization steps in one embodiment;

[0034] Figure 7 This is a flowchart illustrating the resource allocation method for an intelligent agent in another embodiment;

[0035] Figure 8 This is a structural block diagram of a resource allocation device for an intelligent agent in one embodiment;

[0036] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0038] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0039] In today's era of rapid technological development, intelligent agents, with their unique advantages, are gradually becoming one of the core research directions in the field of artificial intelligence, demonstrating enormous application potential in many areas.

[0040] Typically, intelligent agents face situations where multiple tasks need to be performed simultaneously. For example, a transport robot may need to perform navigation, obstacle avoidance, and transport tasks concurrently. Since the computing resources available to the agent are limited, resource allocation is necessary for different tasks. Currently, to avoid consuming the agent's computing resources, a resource allocation strategy is determined in the cloud and then sent to the agent. The agent can then directly allocate resources according to the strategy, reducing the computing resources required to determine the allocation strategy.

[0041] However, in practical applications of intelligent agents, millisecond-level response time has become a key indicator of their effective operation. The methods described above require complex network communication between the agent and the cloud to transmit data, which, even in high-speed network environments, struggles to meet millisecond-level real-time response requirements, resulting in low resource allocation efficiency.

[0042] In view of this, a resource allocation method, apparatus, device, and storage medium for intelligent agents are proposed. First, based on task requirements, current task environment data related to the task requirements is collected from the surrounding environment of the intelligent agent. Second, a feature extraction network corresponding to the task is used to extract features from the current task environment data to obtain the current task feature vector. Then, a vector mapping network corresponding to the task is used to map the current task feature vector to obtain the current priority of the task. Finally, resources are allocated to each task based on its current priority and the resource status of the intelligent agent. In this method, the intelligent agent directly determines the current priority of each task based on its corresponding feature extraction network and vector mapping network, and thus allocates resources according to the current priority. Resource allocation can be achieved without complex network communication with the cloud, reducing communication overhead and improving resource allocation efficiency. Furthermore, since each task uses an independent feature extraction network and vector mapping network, when adding or deleting tasks in the intelligent agent, only the feature extraction network and vector mapping network for the corresponding task need to be added or deleted, without retraining the feature extraction networks and vector mapping networks for other tasks. This enables "plug-and-play" tasks, improving the flexibility and real-time performance of the intelligent agent.

[0043] The resource allocation method for intelligent agents provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is a multi-agent system (MAS) composed of multiple agents. Each agent in the MAS can allocate resources using the resource allocation method described in this embodiment. Taking agent 102 as an example, based on the task requirements of each task to be executed by the agent, current task environment data related to the task requirements is collected from the surrounding environment of the agent. For each task, a feature extraction network corresponding to the task is used to extract features from the current task environment data to obtain the current task feature vector. A vector mapping network corresponding to the task is used to map the current task feature vector to obtain the current priority of the task. Based on the current priority of each task and the resource status of the agent, resources are allocated to each task. The agents communicate with each other through a network. Agents can be, but are not limited to, various drones, robots, etc.

[0044] In one exemplary embodiment, such as Figure 2 As shown, a resource allocation method for an intelligent agent is provided, which can be applied to... Figure 1 Taking agent 102 as an example, the explanation includes:

[0045] S201, based on the task requirements of each task that the agent needs to perform, collect current task environment data related to the task requirements from the surrounding environment of the agent.

[0046] The tasks that the agent needs to perform vary depending on the type of agent and the scenario. Examples include navigation, obstacle avoidance, search and rescue, resource delivery, and image acquisition, which will not be listed here. Task requirements indicate the data needed to perform the corresponding task. For example, for navigation, the requirements might include the agent's own position, the starting point, and the ending point; for obstacle avoidance, the requirements might include the agent's own position and the positions of obstacles in its surrounding environment. Current task environment data refers to the environmental data acquired to perform the current task. It's understood that even for the same task, the collected environmental data will differ depending on the environment in which the agent performs the task. Therefore, it's referred to as current task environment data. Current task environment data is structured observation data containing task and environmental information. Environmental information may include obstacle positions, types, and speeds, while task-related information may include target detection boxes and semantic labels.

[0047] Optionally, the current mission environment data can be collected by multiple heterogeneous acquisition devices. These heterogeneous sensors can include cameras, radar, and other acquisition devices. Different acquisition devices exhibit different acquisition characteristics due to differences in physical properties. For example, cameras acquire continuous image sequences through optical imaging, while radar generates 3D point cloud data through laser scanning. Since the sampling frequency, triggering mechanism, and data transmission latency of each acquisition device differ, time synchronization processing is required to align the data collected by different acquisition devices to a unified time reference.

[0048] Specifically, time synchronization processing can employ a combined hardware and software strategy: at the hardware level, a Precision Time Protocol (PTP) is used to synchronize the time of different sensors; at the software level, timestamp alignment algorithms (such as time offset estimation based on cross-correlation) are used to interpolate and correct asynchronous data. Differential noise suppression is implemented based on the noise characteristics of different sensors: image data acquired by cameras uses a combination of bilateral filtering and non-local mean filtering to preserve edge details and suppress Gaussian or salt-and-pepper noise; radar point cloud data uses statistical outlier filtering and radius filtering combined with spatial-temporal context information to remove outliers or environmental interference points. In the coordinate calibration stage, extrinsic parameters (rotation matrix and translation vector) of each sensor relative to a reference coordinate system (such as a checkerboard or reflective target) are first obtained using calibration equipment (such as a checkerboard or reflective target), and coordinate transformation is performed on the acquired data; for minor changes in sensor installation positions in dynamic scenes, online self-calibration algorithms (such as extrinsic parameter optimization based on feature matching) are used, employing the Iterative Closest Point (ICP) algorithm to iteratively optimize the registration error between point cloud and image features, achieving real-time coordinate calibration. Finally, the processed multi-source data is organized into structured observation data according to a predefined format, which is the current task environment data. The predefined format can be JavaScript Object Notation (JSON) format, Protocol Buffers format, etc.

[0049] S202, for each task, the feature extraction network corresponding to the task is used to extract features from the current task environment data to obtain the current task feature vector; and the vector mapping network corresponding to the task is used to map the current task feature vector to obtain the current priority of the task.

[0050] Each task corresponds to an independent feature extraction network and a vector mapping network. The feature extraction network can be a fully connected layer, a lightweight neural network, a multilayer perceptron (MLP), etc., and the vector mapping network can also be a fully connected layer, a lightweight neural network, an MLP, etc., without specific limitations. The current task feature vector can be a d (positive integer) dimensional vector, and the current priority can be a scalar. The current priority of a task indicates its importance in resource allocation; tasks with higher priority should be allocated the resources they need first. Since each feature extraction network is deployed independently for a task, updates to the feature extraction network only affect the parameters corresponding to the current task, avoiding interference with other tasks and supporting plug-and-play functionality. The feature extraction network design considers real-time requirements, and its computational complexity is linearly related to the scale of the input data in the current task environment.

[0051] Optionally, the vector mapping module employs a single-layer differentiable transformation structure to convert the d-dimensional current task feature vector output by the feature extraction network into a scalar current priority. The vector mapping module is implemented through linear operations, where priority = weight vector − task vector + bias parameter. The single-layer structure ensures the differentiability of the computation process and supports dynamic adjustment of the weight vector and bias parameter through gradient descent. Since it only involves a single-layer linear transformation, its computational complexity is linearly related to the dimension d of the current task feature vector, significantly reducing computational overhead. In terms of communication, because the current priority is a single scalar value, it greatly reduces the amount of data transmitted compared to transmitting high-dimensional vectors.

[0052] S203: Allocate resources for each task based on its current priority and the agent's resource status.

[0053] The resource status of the intelligent agent can include real-time resource information such as remaining computing resources and communication bandwidth utilization. During resource allocation, tasks with higher current priority are allocated resources first. For example, when resources are sufficient, resources are allocated to each task in descending order of current priority. When resources are scarce, higher priority tasks are allocated resources first, while lower priority tasks receive less or no resources. It should be noted that the resource allocation method can be set according to actual needs and is not limited to the methods listed above.

[0054] In the above method, firstly, based on task requirements, current task environment data related to the task requirements is collected from the surrounding environment of the agent. Secondly, a feature extraction network corresponding to the task is used to extract features from the current task environment data to obtain the current task feature vector. Then, a vector mapping network corresponding to the task is used to map the current task feature vector to obtain the current task priority. Finally, resources are allocated to each task based on its current priority and the agent's resource status. In this method, the agent directly determines the current priority of each task based on its corresponding feature extraction network and vector mapping network, and then allocates resources accordingly. This eliminates the need for complex network communication with the cloud, reducing communication overhead and improving resource allocation efficiency. Furthermore, since each task uses an independent feature extraction network and vector mapping network, when adding or deleting tasks in the agent, only the corresponding task's feature extraction network and vector mapping network need to be added or deleted, without retraining the feature extraction networks and vector mapping networks for other tasks. This enables "plug-and-play" task management, improving the agent's flexibility and real-time performance.

[0055] In one embodiment, such as Figure 3As shown, the resource allocation steps in S203 are further detailed, including:

[0056] S301 determines the resource allocation strategy based on the current priority of each task and the resource status of the agent.

[0057] The resource allocation strategy refers to the method of allocating resources for each task. For example, the resource allocation strategy can be to allocate resources to each task sequentially according to its current priority from high to low (Resource Allocation Strategy 1), or to calculate and allocate resources to each task according to its priority (Resource Allocation Strategy 2). These are not listed here. A correspondence between the agent's resource status and the resource allocation strategy can be pre-established, and then the corresponding resource allocation strategy can be found based on the resource status. For example, if the agent's resources are greater than the resource threshold, Resource Allocation Strategy 1 is used; if the agent's resources are not greater than the resource threshold, Resource Allocation Strategy 2 is used.

[0058] S302, allocate resources to each task according to the resource allocation strategy.

[0059] Specifically, resources are allocated to each task according to the established resource allocation strategy.

[0060] In the above embodiments, different resource allocation strategies are dynamically selected based on the current priority of the task and the resource status of the agent to allocate resources, thereby achieving reasonable resource allocation, improving resource utilization, and ensuring the efficient operation of the agent.

[0061] In one embodiment, the method for allocating resources for each task in S302 includes:

[0062] 1) Allocate the minimum resources required to execute each task in descending order of their current priority.

[0063] This approach, known as the greedy selection strategy, is typically used when there are significant differences in the priorities of different tasks within an agent. While it doesn't guarantee that every task will be allocated resources, it prioritizes the highest-priority task and allocates the minimum resource units necessary to meet its basic requirements. Then, it allocates the minimum resources required for each task in descending order of priority. For example, if the agent has 10 units of remaining computing resources, and the tasks are prioritized as follows (task 1, task 3, and task 2), task 1 requires a minimum of 5 resources, so task 1 is allocated 5 units of resources. Task 3 requires a minimum of 3 resources, so task 3 is allocated 3 units of resources. Task 2 requires a minimum of 3 resources, but only 2 units of resources remain; therefore, only 2 units of resources can be allocated to task 2.

[0064] 2) Determine the resource allocation ratio for each task according to its current priority, and allocate resources to each task according to the resource allocation ratio.

[0065] This method, known as the proportional allocation strategy, is typically used in multi-task parallel scenarios and when the current priorities of the tasks are not significantly different. Under this strategy, each task can be allocated resources. Specifically, since the current priority is a scalar (represented by a specific number), the sum of all current priorities can be calculated. Then, the proportion of each task's current priority to the sum can be calculated as the resource allocation ratio. Finally, resources are allocated to each task according to the resource allocation ratio. For example, if task 1 has a current priority of 4, task 2 has a current priority of 3, and task 3 has a current priority of 3, then task 1's resource allocation ratio is 40%, task 2's is 30%, and task 3's is 30%.

[0066] 3) Construct an objective function based on the current priority of each task, and solve the objective function with resource status as a constraint to obtain the resource allocation for each task. Then, allocate resources for each task according to the resource allocation for each task.

[0067] This approach, known as a constrained optimization strategy, optionally incorporates constraints such as total resource quantity (i.e., resource status) and real-time constraints on task execution (e.g., task processing latency requirements). The optimization objective is to maximize overall task execution efficiency or minimize resource idleness. Mathematical programming methods are used to solve the objective function, yielding the resource allocation for each task. Under this strategy, some tasks may not be allocated resources. Constrained optimization strategies can be employed when resources are limited or when real-time constraints exist. For example, if the real-time constraint requires that the processing latency of task 1 be less than a latency threshold, then the processing latency of task 1 can be added as a constraint.

[0068] In the above embodiments, various resource allocation strategies can be flexibly set according to different resource scenarios, which can achieve reasonable allocation of resources, improve resource utilization, and ensure the efficient operation of intelligent agents.

[0069] In one embodiment, such as Figure 4 As shown, the feature extraction networks for each task are trained in the following way:

[0070] S401, for each task, the initial extraction network corresponding to the task is used to extract features from the sample task environment data to obtain the sample feature vector of the task.

[0071] The initial extraction network for the task is iteratively trained to obtain the feature extraction network. In other words, the initial extraction network and the feature extraction network for the task have the same network structure but different parameters. The sample task environment data and the current task environment data are essentially collected from the environment according to the task requirements; the sample task environment data is pre-constructed for network training. The sample feature vector is a d-dimensional vector.

[0072] S402, based on the similarity between the sample feature vectors of different tasks, adjust the parameters of each initial extraction network to obtain the feature extraction network corresponding to each task.

[0073] Specifically, during the training of the initial extraction network, to avoid semantic interference between the feature vectors of samples from different tasks, orthogonal regularization constraints are introduced to minimize the similarity between the feature vectors of different samples, and the parameters of each initial extraction network are adjusted.

[0074] In the above embodiments, by introducing orthogonal regularization constraints during the training process of the feature extraction network, semantic interference between different tasks can be avoided, the accuracy of the feature vectors output by the feature extraction network can be improved, and the accuracy of priority can be improved, thereby achieving reasonable allocation of resources.

[0075] In one embodiment, such as Figure 5 As shown, the steps for adjusting the parameters in S402 are further detailed, including:

[0076] S501, for each sample feature vector, determine the similarity between the sample feature vector and other sample feature vectors.

[0077] The similarity between a sample feature vector and other sample feature vectors can be determined using methods such as cosine similarity or vector inner product, without specific limitations here. Determining the similarity between each sample feature vector and other sample feature vectors essentially involves calculating the pairwise similarity for all sample feature vectors. For example, for sample feature vectors 1-3, it is necessary to determine the similarity between sample feature vector 1 and sample feature vector 2, sample feature vector 1 and sample feature vector 3, and sample feature vector 2 and sample feature vector 3.

[0078] S502, construct a regularization loss function based on the determined similarity; adjust the parameters of each initial extraction network according to the regularization loss function.

[0079] Optionally, the sum of each similarity score can be used as the regularization loss function. The regularization loss function and the feature extraction loss function are then summed according to their weights to obtain the total loss function. Parameters are then adjusted based on this total loss function. In other words, during the initial training of the extraction network, parameter adjustments are not solely based on the regularization loss function. Instead, a regularization term is added to the original network loss to force the feature vectors generated by feature extraction networks for different tasks to maintain orthogonality in the semantic space, thereby enhancing the semantic isolation capability of the tasks.

[0080] In one embodiment, a task vector matrix is ​​formed by the feature vectors of each sample, and each sample feature vector is a column vector of the task vector matrix. The regularization loss function is determined based on the Frobenius Norm (F-norm) of the difference between the product of the task vector matrix and the transpose of the task vector matrix and the identity matrix.

[0081] In the above embodiments, a regularization loss function is constructed based on the similarity between feature vectors of different samples to adjust the parameters of the initial extraction network. This can avoid semantic interference between feature extraction networks of different tasks, improve the accuracy of feature vectors output by the feature extraction network, thereby improving the accuracy of priority and achieving reasonable allocation of resources.

[0082] In one embodiment, such as Figure 6 As shown, after allocating resources for each task, network optimization can be performed based on the task execution status, including:

[0083] S601 determines an instantaneous reward signal for each task based on the task's execution status and the amount of resources allocated to the task.

[0084] The instantaneous reward signal can be determined based on the task's execution status, the amount of resources allocated to the task, or a combination of both. Specifically, the task's execution status and the allocated resources determine whether the instantaneous reward signal is positive or negative. It's understood that after allocating resources to tasks, it's expected that each task can utilize the allocated resources to complete its task objective. Therefore, optionally, if the task's execution status indicates that the task objective has been completed, the instantaneous reward signal is positive; conversely, if the task's execution status indicates that the task objective has not been completed, or only partially completed, the instantaneous reward signal is negative. Correspondingly, during task resource allocation, it's expected that the amount of resources allocated to each task is the minimum resources sufficient to support the completion of the task, thus achieving full utilization of resources. Optionally, the resource utilization efficiency of a task can be determined based on the amount of resources allocated to the task and the amount of resources used to execute the task. An instantaneous reward signal can be determined based on the resource utilization efficiency. When the resource utilization efficiency is greater than the efficiency threshold, the instantaneous reward signal is positive; when the resource utilization efficiency is not greater than the efficiency threshold, the instantaneous reward signal is negative. The efficiency threshold can be set based on experience, multiple trials, and actual needs, and is not specifically limited here.

[0085] S602 optimizes the feature extraction network and vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority.

[0086] Specifically, a loss function can be constructed based on the instantaneous reward signal and the current priority, and the feature extraction network and vector mapping network can be optimized based on the loss function. The optimization in S602 can be understood as online optimization (updating) during the use of the feature extraction network and vector mapping network, rather than retraining the entire network. The optimized feature extraction network and vector mapping network affect the generation of the current task feature vector and the calculation of the current priority in subsequent tasks, thereby changing the basis for resource allocation and forming a closed-loop process of dynamic optimization. The entire process can be completed locally by the agent, ensuring millisecond-level response capability and plug-and-play characteristics, and ensuring rapid adaptation to real-time task requirements.

[0087] In the above embodiments, the feature extraction network and vector mapping network are optimized online based on the task execution status and the amount of resources allocated to the task. This can improve the adaptability of the feature extraction network and vector mapping network to the real-time task requirements and make resource allocation more reasonable.

[0088] In one embodiment, the steps of optimizing the feature extraction network and the vector mapping network in S602 are further refined, including:

[0089] Based on the instantaneous reward signal and the current priority, determine the first parameter gradient of the feature extraction network corresponding to the task, and the second parameter gradient of the vector mapping network corresponding to the task; adjust the parameters of the feature extraction network according to the first parameter gradient and the learning rate of the feature extraction network; adjust the parameters of the vector mapping network according to the second parameter gradient and the learning rate of the feature extraction network.

[0090] The gradients of the first and second parameters can be calculated and updated in real time using stochastic gradient descent or the Adaptive Moment Estimation Optimizer. In addition, for the feature extraction network, the gradient of the regularization term can also be determined based on the instantaneous reward signal and the current priority, and then backpropagated to the parameters of the feature extraction network through the chain rule to ensure that the orthogonal constraints remain effective during the online optimization process.

[0091] Optionally, the first parameter θ of the feature extraction network is updated using gradient descent, with the gradient calculated based on the instantaneous reward signal. Similarly, the second parameter φ of the vector mapping network is updated based on the gradient determined by the instantaneous reward signal, using a chain rule for backpropagation. Both the updates of the first and second parameters are performed locally at the agent's edge nodes, avoiding communication overhead with the cloud or other remote nodes. To achieve low latency, a lightweight gradient calculation framework is adopted, retaining only necessary parameter gradient caches and computation paths. Furthermore, locally stored historical task data can be used to optimize gradient estimation efficiency. The update process shares computational resources with the real-time task. By time-sharing the CPU / GPU cores, it is ensured that the update operation does not significantly encroach on the computational resources of the real-time task, with the total latency strictly limited to within 10 milliseconds.

[0092] In the above embodiments, the feature extraction network and vector mapping network are optimized online based on the task execution status and the amount of resources allocated to the task. This can improve the adaptability of the feature extraction network and vector mapping network to the real-time task requirements and make resource allocation more reasonable.

[0093] Based on the above embodiments, in one embodiment, this application provides a resource allocation method for intelligent agents that can be applied to a multi-task real-time scheduling scenario for mobile robots to allocate resources for the mobile robot. Taking the application of the intelligent agent resource allocation method to a resource allocation system in a mobile robot as an example, it includes the following modules:

[0094] The perception module is equipped with a red-green-blue tri-color camera and one radar in the forward field of view of the mobile robot (which can be a wheeled mobile robot), and two short-range radars in the lateral obstacle avoidance area. A hardware-triggered synchronization mechanism is used to ensure that the timestamp error of the data collected by each sensor is less than 1 mm. In the preprocessing stage, statistical filtering is applied to the radar point cloud data to remove noise, and median filtering is used to remove salt-and-pepper noise from the camera images. All data are unified to the mobile robot's body coordinate system (with the chassis center as the origin, the forward x-axis and the lateral y-axis), and finally outputs a 128-dimensional observation vector, which includes obstacle coordinates (32-dimensional), target point coordinates (2-dimensional), task urgency (1-dimensional), and other information.

[0095] The task vector encoding module, currently handling resource allocation for navigation and obstacle avoidance tasks, consists of feature extraction networks for each task. Each feature extraction network is a 2-layer MLP, with a 34-dimensional input and a 16-dimensional output (d). The vector mapping network has a 32-dimensional input and a 16-dimensional output (d). The parameters θ1 of the feature extraction network and θ2 of the vector mapping network are stored independently. The navigation task's feature extraction network encodes vector 1 as v1(t); the obstacle avoidance task's current environment data is a 32-dimensional vector 2, containing side radar data and local obstacle information, which is encoded as v2(t) by the obstacle avoidance network. An orthogonal regularization loss term L is added during the training phase of the feature extraction network. orth This constraint is used to ensure that the absolute value of the inner product of v1(t) and v2(t) is ≤0.2. During feature extraction, the current task environment data for the navigation task is a 34-dimensional vector, containing the target point coordinates and global obstacle information.

[0096] The priority calculation module consists of a vector mapping network for the navigation task and a vector mapping network for the obstacle avoidance task. Both vector mapping networks use a single-layer linear transformation to calculate v. i (t) is mapped to the scalar p. i (t)=w i· v i (t), where the weight vector w1 of the vector mapping network for the navigation task is initialized as a vector negatively correlated with the target distance, and the weight w2 of the obstacle avoidance task is initialized as a vector negatively correlated with the obstacle distance.

[0097] The behavior scheduling module, also known as the coordinator, dynamically selects and executes a resource allocation strategy suitable for the current scenario based on real-time perceived resource status and task current priorities. Specifically, when a sudden obstacle is detected (the distance between the obstacle and the agent is less than a distance threshold), a greedy selection strategy is triggered, prioritizing the allocation of 90% of computing resources to the obstacle avoidance task. In multi-target tracking (e.g., tracking two targets simultaneously), resources are allocated to the navigation and obstacle avoidance tasks in a 3:2 ratio. When the battery level is below a power threshold, a constraint optimization strategy is adopted. The objective function maximizes the product of task completion and current priority, with the constraint that computing power does not exceed a computing power threshold, resulting in the resources allocated to the navigation and obstacle avoidance tasks. The distance threshold, power threshold, and computing power threshold can all be set based on experience, multiple trials, and actual needs, and are not specifically limited here. After obtaining the current priorities of the navigation and obstacle avoidance tasks, the behavior scheduling module executes the resource allocation strategy and issues action commands to the mobile robot. The mobile robot executes actions according to the action commands and generates instantaneous reward signals based on task execution and resource allocation, which are then fed back to the online update module.

[0098] The online update module addresses the real-time multi-task processing needs of edge computing scenarios. It forms a closed-loop feedback loop with the behavior scheduling module, using a local gradient update mechanism to dynamically optimize the parameters of the feature extraction network and the vector mapping network online. When the navigation task is completed, the instantaneous reward signal r = +5; when encountering obstacles, r = -10. The mean squared error loss L(r,p) is used. i )=(rp i )², for θ i With a learning rate of 0.001 and w i Gradient updates are performed with a learning rate of 0.005. In actual testing, all calculations are completed within a single time step (100ms), with a communication latency of less than 2 milliseconds.

[0099] Based on the above embodiments, in one embodiment, this application provides a resource allocation method for intelligent agents that can be applied to a heterogeneous multi-agent collaborative scenario of UAV-ground robot, wherein...

[0100] Perception Module: An inertial navigation system and altimeter are added to the UAV, and an Ultra-Wideband (UWB) positioning module is added to the ground robot, expanding the observation vector to 256 dimensions (including cross-agent information such as UAV altitude and relative position). Time synchronization adopts Network Time Protocol (NTP), with a synchronization error of less than 5 milliseconds.

[0101] Task Vector Encoding Module: A new feature extraction module for collaborative tasks between UAVs and ground robots (such as joint search and rescue) has been added. The input includes the target coordinates within the UAV's field of view (from the camera) and the local obstacle information of the ground robot (from the radar). The output is a 32-dimensional task vector v3(t). During the training phase, the absolute value of the inner product of v3(t) with v1(t) and v2(t) is constrained to be ≤0.3.

[0102] Priority calculation module: Introduces collaborative weight coefficient γ, and updates the priority calculation formula to p. i (t)=γ·w i ·v i (t)+(1-γ)·Σp j (t) (j is the agent associated with the collaborative task), where, for the UAV, the ground robot is the agent associated with the collaborative task, and for the ground robot, the UAV is the agent associated with the collaborative task. That is, for the collaborative task of each agent, its priority is the weighted sum of the current priority output by the vector mapping network of the collaborative task and the current priority output by the agent associated with the collaborative task for the collaborative task. This priority calculation method can enhance the collaboration between tasks.

[0103] Behavior scheduling module: Added cross-device scheduling strategy. When the remaining battery power of the drone is less than 30%, some of the drone's perception tasks can be offloaded to the ground robot (communication bandwidth constraint: communication bandwidth ≤ 10 megabits per second). The optimal offloading scheme is solved by a mixed integer linear programming model (the objective function is to minimize the total delay, and the constraints are computing power, bandwidth and battery power).

[0104] Online update module: The instantaneous reward signal r is expanded into a distributed form (r 总 =0.6r 无人机 +0.4r 地面机器人 ), where r 无人机 The target detection success rate, r 地面机器人 This improves path planning efficiency. A federated learning framework is used to update the parameters of the feature extraction network and the vector mapping network, synchronizing parameters every 5 time steps to avoid single points of failure.

[0105] Based on the above embodiments, in an exemplary embodiment, such as Figure 7 As shown, the resource allocation method for intelligent agents in this application embodiment may include the following steps:

[0106] S701: Based on the task requirements of each task that the agent needs to perform, collect current task environment data related to the task requirements from the surrounding environment of the agent.

[0107] S702: For each task, the feature extraction network corresponding to the task is used to extract features from the current task environment data to obtain the current task feature vector; and the vector mapping network corresponding to the task is used to map the current task feature vector to obtain the current priority of the task.

[0108] S703: Determine the resource allocation strategy based on the current priority of each task and the resource status of the agent.

[0109] S704: According to the resource allocation strategy, allocate the minimum resources required to execute each task in descending order of their current priority; or, determine the resource allocation ratio for each task according to its current priority, and allocate resources to each task according to that ratio; or, construct an objective function based on the current priority of each task, and solve the objective function with resource constraints to obtain the resource allocation amount for each task, and allocate resources to each task according to that allocation amount.

[0110] S705: For each task, determine the instantaneous reward signal based on the task's execution status and the amount of resources allocated to the task.

[0111] S706: Based on the instantaneous reward signal and the current priority, determine the first parameter gradient of the feature extraction network corresponding to the task, and the second parameter gradient of the vector mapping network corresponding to the task.

[0112] S707: Adjust the parameters of the feature extraction network based on the gradient of the first parameter and the learning rate of the feature extraction network.

[0113] S708: Adjust the parameters of the vector mapping network based on the gradient of the second parameter and the learning rate of the feature extraction network.

[0114] The specific implementation methods of S701-S708 are the same as those in the above method embodiments, and will not be repeated here.

[0115] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0116] Based on the same inventive concept, this application also provides a resource allocation device for implementing the resource allocation method for intelligent agents described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the resource allocation device for intelligent agents provided below can be found in the limitations of the resource allocation method for intelligent agents described above, and will not be repeated here.

[0117] In one exemplary embodiment, such as Figure 8 As shown, a resource allocation device for an intelligent agent is provided, comprising: a collection module 801, an extraction module 802, a mapping module 803, and an allocation module 804, wherein:

[0118] The acquisition module 801 is used to collect current task environment data related to the task requirements from the surrounding environment of the agent according to the task requirements of each task to be performed by the agent.

[0119] Extraction module 802 is used to extract features from the current task environment data using a feature extraction network corresponding to the task for each task, thereby obtaining the current task feature vector; and,

[0120] The mapping module 803 is used to map the current task feature vector using the vector mapping network corresponding to the task to obtain the current priority of the task.

[0121] The allocation module 804 is used to allocate resources to each task based on the current priority of each task and the resource status of the agent.

[0122] In one embodiment, the allocation module 804 includes:

[0123] The determining unit is used to determine the resource allocation strategy based on the current priority of each task and the resource status of the agent;

[0124] The resource allocation unit is used to allocate resources to each task according to the resource allocation strategy.

[0125] In one embodiment, the resource allocation unit is configured to perform any of the following:

[0126] Based on the current priority of each task from high to low, allocate the minimum resources required to execute each task in order; determine the resource allocation ratio for each task according to its current priority, and allocate resources to each task according to the resource allocation ratio; construct an objective function based on the current priority of each task, and solve the objective function with resource status as a constraint to obtain the resource allocation amount for each task, and allocate resources to each task according to the resource allocation amount for each task.

[0127] In one embodiment, the device further includes a training module 805, comprising:

[0128] The sample extraction unit is used to extract features from the sample task environment data using the initial extraction network corresponding to the task for each task, so as to obtain the sample feature vector of the task.

[0129] The parameter adjustment unit is used to adjust the parameters of each initial extraction network based on the similarity between the sample feature vectors of different tasks, so as to obtain the feature extraction network corresponding to each task.

[0130] In one embodiment, the parameter adjustment unit is specifically used to: determine the similarity between each sample feature vector and other sample feature vectors; construct a regularization loss function based on the determined similarity; and adjust the parameters of each initial extraction network based on the regularization loss function.

[0131] In one embodiment, the device further includes an optimization module 806, comprising:

[0132] The reward unit includes determining an instantaneous reward signal for each task based on the task's performance and the amount of resources allocated to the task.

[0133] The network optimization unit includes optimizing the feature extraction network and vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority.

[0134] In one embodiment, the network optimization unit is specifically configured to: determine the first parameter gradient of the feature extraction network corresponding to the task and the second parameter gradient of the vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority; adjust the parameters of the feature extraction network based on the first parameter gradient and the learning rate of the feature extraction network; and adjust the parameters of the vector mapping network based on the second parameter gradient and the learning rate of the feature extraction network.

[0135] Each network in the resource allocation device of the aforementioned intelligent agent can be implemented entirely or partially through software, hardware, or a combination thereof. These networks can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can invoke and execute the operations corresponding to each network.

[0136] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores current task environment data, task priorities, etc. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a resource allocation method for an intelligent agent.

[0137] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0138] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to: collect current task environment data related to the task requirements from the surrounding environment of the agent, based on the task requirements of each task to be performed by the agent; for each task, extract features from the current task environment data using a task-specific feature extraction network to obtain a current task feature vector; map the current task feature vector using a task-specific vector mapping network to obtain the current priority of the task; and allocate resources for each task based on the current priority of each task and the resource status of the agent.

[0139] In one embodiment, resource allocation is performed for each task based on its current priority and the resource status of the agent, including: determining a resource allocation strategy based on the current priority of each task and the resource status of the agent; and allocating resources for each task according to the resource allocation strategy.

[0140] In one embodiment, resource allocation for each task includes any of the following:

[0141] Based on the current priority of each task from high to low, allocate the minimum resources required to execute each task in order; determine the resource allocation ratio for each task according to its current priority, and allocate resources to each task according to the resource allocation ratio; construct an objective function based on the current priority of each task, and solve the objective function with resource status as a constraint to obtain the resource allocation amount for each task, and allocate resources to each task according to the resource allocation amount for each task.

[0142] In one embodiment, the feature extraction network corresponding to each task is trained in the following manner: for each task, the initial extraction network corresponding to the task is used to extract features from the sample task environment data to obtain the sample feature vector of the task; according to the similarity between the sample feature vectors of different tasks, the parameters of each initial extraction network are adjusted to obtain the feature extraction network corresponding to each task.

[0143] In one embodiment, the parameters of each initial extraction network are adjusted based on the similarity between sample feature vectors of different tasks, including: determining the similarity between each sample feature vector and other sample feature vectors; constructing a regularization loss function based on the determined similarity; and adjusting the parameters of each initial extraction network based on the regularization loss function.

[0144] In one embodiment, the method further includes: for each task, determining an instantaneous reward signal based on the task's execution status and the amount of resources allocated to the task; and optimizing the feature extraction network and vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority.

[0145] In one embodiment, the feature extraction network and vector mapping network corresponding to the task are optimized based on the instantaneous reward signal and the current priority, including: determining the first parameter gradient of the feature extraction network and the second parameter gradient of the vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority; adjusting the parameters of the feature extraction network based on the first parameter gradient and the learning rate of the feature extraction network; and adjusting the parameters of the vector mapping network based on the second parameter gradient and the learning rate of the feature extraction network.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following: based on the task requirements of each task to be performed by the agent, collecting current task environment data related to the task requirements from the surrounding environment of the agent; for each task, using a task-specific feature extraction network to extract features from the current task environment data to obtain a current task feature vector; and using a task-specific vector mapping network to map the current task feature vector to obtain the current priority of the task; and allocating resources for each task based on the current priority of each task and the resource status of the agent.

[0147] In one embodiment, resource allocation is performed for each task based on its current priority and the resource status of the agent, including: determining a resource allocation strategy based on the current priority of each task and the resource status of the agent; and allocating resources for each task according to the resource allocation strategy.

[0148] In one embodiment, resource allocation for each task includes any of the following:

[0149] Based on the current priority of each task from high to low, allocate the minimum resources required to execute each task in order; determine the resource allocation ratio for each task according to its current priority, and allocate resources to each task according to the resource allocation ratio; construct an objective function based on the current priority of each task, and solve the objective function with resource status as a constraint to obtain the resource allocation amount for each task, and allocate resources to each task according to the resource allocation amount for each task.

[0150] In one embodiment, the feature extraction network corresponding to each task is trained in the following manner: for each task, the initial extraction network corresponding to the task is used to extract features from the sample task environment data to obtain the sample feature vector of the task; according to the similarity between the sample feature vectors of different tasks, the parameters of each initial extraction network are adjusted to obtain the feature extraction network corresponding to each task.

[0151] In one embodiment, the parameters of each initial extraction network are adjusted based on the similarity between sample feature vectors of different tasks, including: determining the similarity between each sample feature vector and other sample feature vectors; constructing a regularization loss function based on the determined similarity; and adjusting the parameters of each initial extraction network based on the regularization loss function.

[0152] In one embodiment, the method further includes: for each task, determining an instantaneous reward signal based on the task's execution status and the amount of resources allocated to the task; and optimizing the feature extraction network and vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority.

[0153] In one embodiment, the feature extraction network and vector mapping network corresponding to the task are optimized based on the instantaneous reward signal and the current priority, including: determining the first parameter gradient of the feature extraction network and the second parameter gradient of the vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority; adjusting the parameters of the feature extraction network based on the first parameter gradient and the learning rate of the feature extraction network; and adjusting the parameters of the vector mapping network based on the second parameter gradient and the learning rate of the feature extraction network.

[0154] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements: collecting current task environment data related to the task requirements from the surrounding environment of the agent, based on the task requirements of each task to be performed by the agent; for each task, extracting features from the current task environment data using a task-specific feature extraction network to obtain a current task feature vector; mapping the current task feature vector using a task-specific vector mapping network to obtain the current priority of the task; and allocating resources for each task based on the current priority of each task and the resource status of the agent.

[0155] In one embodiment, resource allocation is performed for each task based on its current priority and the resource status of the agent, including: determining a resource allocation strategy based on the current priority of each task and the resource status of the agent; and allocating resources for each task according to the resource allocation strategy.

[0156] In one embodiment, resource allocation for each task includes any of the following:

[0157] Based on the current priority of each task from high to low, allocate the minimum resources required to execute each task in order; determine the resource allocation ratio for each task according to its current priority, and allocate resources to each task according to the resource allocation ratio; construct an objective function based on the current priority of each task, and solve the objective function with resource status as a constraint to obtain the resource allocation amount for each task, and allocate resources to each task according to the resource allocation amount for each task.

[0158] In one embodiment, the feature extraction network corresponding to each task is trained in the following manner: for each task, the initial extraction network corresponding to the task is used to extract features from the sample task environment data to obtain the sample feature vector of the task; according to the similarity between the sample feature vectors of different tasks, the parameters of each initial extraction network are adjusted to obtain the feature extraction network corresponding to each task.

[0159] In one embodiment, the parameters of each initial extraction network are adjusted based on the similarity between sample feature vectors of different tasks, including: determining the similarity between each sample feature vector and other sample feature vectors; constructing a regularization loss function based on the determined similarity; and adjusting the parameters of each initial extraction network based on the regularization loss function.

[0160] In one embodiment, the method further includes: for each task, determining an instantaneous reward signal based on the task's execution status and the amount of resources allocated to the task; and optimizing the feature extraction network and vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority.

[0161] In one embodiment, the feature extraction network and vector mapping network corresponding to the task are optimized based on the instantaneous reward signal and the current priority, including: determining the first parameter gradient of the feature extraction network and the second parameter gradient of the vector mapping network corresponding to the task based on the instantaneous reward signal and the current priority; adjusting the parameters of the feature extraction network based on the first parameter gradient and the learning rate of the feature extraction network; and adjusting the parameters of the vector mapping network based on the second parameter gradient and the learning rate of the feature extraction network.

[0162] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0163] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0165] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for resource allocation of an agent, the method comprising: The method comprises: collecting current task environment data related to task requirements of each task required to be performed by the agent from a surrounding environment in which the agent is located; performing feature extraction on the current task environment data by using a feature extraction network corresponding to each task to obtain a current task feature vector of the task; and mapping the current task feature vector by using a vector mapping network corresponding to the task to obtain a current priority of the task; allocating resources to each task according to the current priority of each task and a resource status of the agent.

2. The method of claim 1, wherein, The resource allocation to each task according to the current priority of each task and the resource status of the agent comprises: determining a resource allocation strategy according to the current priority of each task and the resource status of the agent; allocating resources to each task according to the resource allocation strategy.

3. The method of claim 2, wherein, The resource allocation to each task comprises any one of the following: allocating the minimum resources required to perform each task in a descending order of the current priority of each task; determining a resource allocation proportion of each task according to the current priority of each task, and allocating resources to each task according to the resource allocation proportion of each task; constructing an objective function according to the current priority of each task, and solving the objective function with the resource status as a constraint to obtain a resource allocation amount of each task, and allocating resources to each task according to the resource allocation amount of each task.

4. The method of claim 1, wherein, The feature extraction network corresponding to each task is trained in the following manner: performing feature extraction on sample task environment data by using an initial extraction network corresponding to each task to obtain a sample feature vector of the task; adjusting parameters of each initial extraction network according to the similarity between sample feature vectors of different tasks to obtain a feature extraction network corresponding to each task.

5. The method of claim 4, wherein, The parameter adjustment of each initial extraction network according to the similarity between sample feature vectors of different tasks comprises: determining the similarity between each sample feature vector and other sample feature vectors; constructing a regularization term loss function according to the determined similarity; adjusting parameters of each initial extraction network according to the regularization term loss function.

6. The method of claim 1, wherein, The method further comprises: determining an instantaneous reward signal according to the execution of each task and the resource amount allocated to the task; optimizing the feature extraction network and the vector mapping network corresponding to each task according to the instantaneous reward signal and the current priority.

7. The method of claim 6, wherein, The optimization of the feature extraction network and the vector mapping network corresponding to each task according to the instantaneous reward signal and the current priority comprises: According to the instantaneous reward signal and the current priority, a first parameter gradient of a feature extraction network corresponding to the task and a second parameter gradient of a vector mapping network corresponding to the task are determined; According to the first parameter gradient and a learning rate of the feature extraction network, the feature extraction network is adjusted in parameters; According to the second parameter gradient and a learning rate of the feature extraction network, the vector mapping network is adjusted in parameters.

8. A resource allocation device for an intelligent agent, characterized in that, The device comprises: A collection module configured to collect, from a surrounding environment in which an agent is located, current task environment data related to a task requirement of each task required to be executed by the agent; An extraction module configured to, for each task, extract features of the current task environment data by using a feature extraction network corresponding to the task, to obtain a current task feature vector of the task; and A mapping module configured to map the current task feature vector by using a vector mapping network corresponding to the task, to obtain a current priority of the task; An allocation module configured to allocate resources to each task according to the current priority of each task and a resource status of the agent. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images