A method and device for generating a heterogeneous group collaborative sensing task allocation scheme

Abstract

The present invention relates to a method and device for generating a heterogeneous group collaborative perception task allocation scheme. The method comprises: modeling a task scenario, a heterogeneous perception subject and a collaborative perception process; wherein the heterogeneous perception subject comprises: an unmanned intelligent terminal and a human worker; extracting features of the global system state and the local observation of the perception subject to obtain the feasible action of the perception subject, and using an intelligent agent evaluation network to evaluate the value of the feasible action of the perception subject according to the local observation of the perception subject, and then using the overall Q network to select actions according to the global system state using a specific strategy, thereby evaluating the value of each action selected by all perception subjects, calculating the loss function and using the gradient descent method to update the parameters of the intelligent agent evaluation network and the overall Q network, thereby updating the network parameters, and obtaining a trained neural network, thereby obtaining a heterogeneous group collaborative perception task allocation scheme.

Description

Technical Field

[0001] The present application relates to the field of multi-agent collaboration technology, and in particular to a method and device for generating a heterogeneous group collaborative perception task allocation scheme. Background Art

[0002] The problem of rapid perception and search in emergency rescue scenarios is an important issue in the field of emergency management and group intelligence perception. Traditional perception methods for emergency rescue scenarios mainly rely on manpower and simple equipment, with a single perception subject and low search and rescue efficiency. In recent years, unmanned intelligent terminals represented by drones and unmanned ground platforms have gradually matured and can assist humans in performing perception tasks. The method of relying on multiple perception subjects such as humans, drones and unmanned ground platforms to collaborate in performing perception tasks is called heterogeneous group collaborative perception.

[0003] The earliest studies focused on the collaborative perception of heterogeneous human participants. Some studies designed hierarchical trees and time series queues to achieve the matching of online large-scale tasks with heterogeneous multi-skilled participants. Some studies optimized task allocation from two stages by constructing a hybrid framework called "HyTasker". In the offline stage, opportunistic perception workers are selected to complete perception tasks in daily life. In the online stage, participatory workers are selected, who need to make special movements to complete tasks that opportunistic workers have not completed. In addition, some studies have focused on the allocation problem of a mixture of mobile participants and static participants under budget constraints, and proposed a task allocation method based on reverse auctions to solve the problem by selecting the winning bid round by round, and proposed an online algorithm based on Shapley value to ensure task completion.

[0004] As research progresses, some studies focus on using perception vehicles and humans to collaborate in performing perception tasks. Some studies focus on the time-aware task allocation problem for heterogeneous mobile users in mobile crowd perception. In view of the differences in moving speed and task execution time between humans and perception vehicles, a pedestrian task allocation algorithm based on Kuhn-Munkres and a perception vehicle task allocation algorithm based on a greedy algorithm are proposed respectively. Another study proposes a framework for human-vehicle collaborative perception under the framework of sparse mobile crowd perception. By comprehensively considering factors such as the perception ability of the vehicle, the movement trajectory of the mobile user, and the task execution ability, effective collaborative task allocation for mobile users and perception vehicles is achieved.

[0005] The problem to be solved by this application is to provide an efficient task allocation solution for heterogeneous group collaborative perception in emergency rescue scenarios. Summary of the invention

[0006] Based on this, it is necessary to provide a method and device for generating a heterogeneous group collaborative sensing task allocation solution to address the above technical issues.

[0007] A method for generating a heterogeneous group collaborative sensing task allocation scheme, the method comprising:

[0008] Modeling of task scenarios, heterogeneous perception subjects and collaborative perception processes; heterogeneous perception subjects include: unmanned intelligent terminals and human workers; modeling of the collaborative perception process of heterogeneous perception subjects includes: mathematically representing the global system state of the task scenario and the local observation of a single perception subject, and setting the action space and reward return of the perception subject;

[0009] An agent evaluation network is used to evaluate the value of the perceptual subject's feasible actions based on the local observation of the perceptual subject. Then, the overall Q network is used to evaluate the value of the actions selected by all perceptual subjects based on the global system state. The loss function is calculated and the gradient descent method is used to update the parameters of the agent evaluation network and the overall Q network to obtain a trained neural network, thereby obtaining a heterogeneous group collaborative perception task allocation scheme; the neural network includes an agent evaluation network and an overall Q network.

[0010] A device for generating a heterogeneous group collaborative sensing task allocation scheme, the device comprising:

[0011] A modeling module is used to model the task scenario, heterogeneous perception subjects and collaborative perception process; the heterogeneous perception subjects include: unmanned intelligent terminals and human workers; modeling the collaborative perception process of heterogeneous perception subjects includes: mathematically representing the global system state of the task scenario and the local observation of a single perception subject, and setting the action space and reward return of the perception subject;

[0012] The task allocation module is used to use the intelligent agent evaluation network to evaluate the value of the perceptual subject's feasible actions based on the local observation of the perceptual subject, and then use the overall Q network to evaluate the value of the actions selected by all perceptual subjects according to the global system state, calculate the loss function and use the gradient descent method to update the parameters of the intelligent agent evaluation network and the overall Q network to obtain a trained neural network, thereby obtaining a heterogeneous group collaborative perception task allocation plan; the neural network includes the intelligent agent evaluation network and the overall Q network.

[0013] In the above-mentioned method and device for generating a heterogeneous group collaborative perception task allocation scheme, a collaborative perception scene model and a perception subject model are first established, and the collaboration mode of the heterogeneous perception subjects is designed. Then, the existing heterogeneous group collaborative perception task allocation algorithm is improved, and an environmental information processing module and a perception subject decision module are introduced, wherein the environmental information processing module is used to extract features of the local observation of the perception subject and the global system state. The perception subject decision module corresponds to the intelligent agent evaluation network, which is used to evaluate the value of the feasible actions of the perception subject based on the local observation of the perception subject, thereby completing the action selection. Then, all selected actions are evaluated according to the global system state, the loss function is calculated, and the gradient descent method is used to update the parameters of the intelligent agent evaluation network and the overall Q network to obtain a trained neural network, and finally a heterogeneous group collaborative perception task allocation scheme for emergency rescue scenarios is formed. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic diagram of a flow chart of a method for generating a heterogeneous group collaborative sensing task allocation scheme in one embodiment;

[0015] Figure 2 Schematic diagram of a task scene grid in one embodiment; wherein (a) is obstacle modeling, and (b) is task point modeling;

[0016] Figure 3 Schematic diagram of the state space of the system in one embodiment; wherein (a) is the obstacle position, (b) is the task point position, (c) is the perception subject position, and (d) is the task attributes, task time, and task execution subject information corresponding to all perception tasks;

[0017] Figure 4 Schematic diagram of local observation of a perception subject in an embodiment; wherein (a) is the location of the task point, (b) is the location of the perception subject itself, (c) is the movable range of the perception subject at the current moment, (d) is the power information of the drone, and (e) is the number of the perception subject;

[0018] Figure 5 A schematic diagram of a heterogeneous group collaborative sensing task allocation framework in one embodiment;

[0019] Figure 6 A schematic diagram of a QTRAN-plus hybrid network design in one embodiment;

[0020] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] In one embodiment, Figure 1 As shown, a method for generating a heterogeneous group collaborative sensing task allocation scheme is provided, comprising the following steps:

[0023] Step 102: Modeling the task scenario, heterogeneous sensing subjects and collaborative sensing process.

[0024] Among them, heterogeneous perception subjects include: unmanned intelligent terminals and human workers.

[0025] Modeling the collaborative perception process of heterogeneous perception agents includes: mathematically representing the global system state of the task scenario and the local observations of a single perception agent, and setting the action space and reward returns of the perception agent.

[0026] Step 104, use the agent evaluation network to evaluate the value of the perceptual subject's feasible actions based on the local observation of the perceptual subject, and then use the overall Q network to evaluate the value of the actions selected by all perceptual subjects based on the global system state, calculate the loss function and use the gradient descent method to update the parameters of the agent evaluation network and the overall Q network to obtain a trained neural network, thereby obtaining a heterogeneous group collaborative perception task allocation solution.

[0027] Among them, one type of perception subject shares one agent evaluation network. Therefore, if the unmanned intelligent terminal only includes drones, then this solution has a total of two agent evaluation networks. If the unmanned intelligent terminal includes drones and unmanned ground platforms, then this solution has a total of three agent evaluation networks, and so on.

[0028] The aforementioned neural network includes an agent evaluation network and an overall Q network. In fact, this solution also uses an environmental information processing module to extract features from the local observations of the perceptual subject and the global system state, which can be implemented using a convolutional neural network. In this case, the aforementioned neural network includes an agent evaluation network, an overall Q network, and a convolutional neural network. The above is an example and is not specifically limited here.

[0029] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0030] In one embodiment, modeling a task scenario includes:

[0031] Divide the area of ​​the task scene into several grids to obtain the corresponding two-dimensional area set ; Each element in the two-dimensional region set represents a region grid, Represents the first task in the scenario area grids, each of which can be an obstacle or a task point. Figure 2 As shown in the figure, each area grid has an obstacle mark and a task mark. The obstacle mark is used to indicate whether there is an obstacle, and the task mark is used to indicate whether there is a task. , Represents a regional grid If there is an obstacle, then ;otherwise . Represents a regional grid The task ID of the task, if there is a task, then ;otherwise .

[0032] Assume the number of perception tasks is , then the task set can be expressed as For the Tasks , suppose there is perception subtasks, then the set of perception subtasks is ,in Indicates Tasks No. Subtasks . Indicates the subtask type, that is, which type of perception subject needs to perform it. Indicates the subtask location. Indicates the subtask execution time, satisfying , that is, the execution time of a subtask is a multiple of a perception cycle. As time goes by, the number of tasks that have not been executed will gradually decrease. The number of unexecuted tasks at the moment is , then at the initial moment we have .

[0033] Define the number of perceptual entities as , mainly including human workers, drones and unmanned ground platforms. These three types of perception subjects have different attributes and characteristics, and each undertakes different tasks in the process of performing perception tasks. In this application, in order to facilitate the description of the perception process, the entire perception process is referred to as Divide into Perception cycle , the duration of each sensing cycle is , satisfying the following formula. The sensing cycle is the minimum unit of task allocation.

[0034] ;

[0035] In one embodiment, the unmanned intelligent terminal includes a drone and an unmanned ground platform; modeling of heterogeneous perception subjects includes:

[0036] Assume the number of human workers is , build a collection of human workers ;in, Indicates Human workers. Represents human workers exist The position at the moment. Represents human workers The movable range within one sensing cycle, i.e. During the perception task execution, the set of routes for human workers is ,in Represents human workers The route is composed of human workers The position at every moment.

[0037] Assume the number of drones is , then the set of drones is:

[0038] ;

[0039] in, Indicates A drone. Indicates drone exist The position at the moment. Indicates drone At the moment remaining power. Indicates drone The amount of power consumed by movement in one sensing cycle. Represents drone The movable range within a sensing cycle. During the sensing task execution, the drone route set is ,in Indicates drone The route, whose elements are drones The position at every moment.

[0040] Assume the number of unmanned ground platforms is , then the set of unmanned platforms is:

[0041] ;

[0042] in, Indicates Unmanned platform. Indicates unmanned platform exist The position at the moment. Represents the unmanned platform The movable range within one sensing cycle. Represents the unmanned platform The detection range of the drone within a sensing cycle, i.e. the unmanned platform The range of the drone’s battery power and location information can be detected. During the perception task execution, the unmanned platform route set is ,in Indicates unmanned platform The route consists of unmanned platforms The position at every moment.

[0043] In one embodiment, the collaboration sensing process is modeled, including:

[0044] Due to the limited communication conditions after the urban earthquake disaster, the perception of heterogeneous subjects is partially observable, so a seven-tuple Dec-POMDP is used To model the collaborative perception process of heterogeneous subjects. is the system state space set; is the number of perceiving subjects; It is the set of possible actions of the perceiving subject; is the probability of transition between system states; It is the immediate reward that the next state system gets after the perceptual subject takes a certain action; is the discount factor; is the set of observations of the perceptual subject. The following is a detailed introduction to the system state space and the perceptual subject observations.

[0045] In this application, The status of the system at the moment It can be expressed by the following formula:

[0046]

[0047] in, refers to The location distribution of obstacles in the environment at all times; refers to The location distribution of task points at all times; refers to The remaining execution time of the task point at a certain moment; It refers to the type of task, that is, what type of perceptual subject needs to perform the task; refers to The matching situation between the task point at the moment and the perception subject, that is, which perception subject is currently executing the task at the moment; refers to Always be aware of the location distribution of the subject. Figure 3 The state space diagram of the system is shown graphically. Figure 3 Different grayscales in (c) represent different types of perception subjects, where the squares with “A” represent drones, the squares with “W” represent human workers, and the squares with “G” represent unmanned ground platforms.

[0048] In this application, Always aware of the subject Observed value for:

[0049] ;

[0050] in, It is the location distribution, execution time, and attribute category information of all perception tasks. The task information is informed to the perception subject before the perception task begins, which is used for the perception subject to clarify the goal and direction of the perception task execution. For the perceiving subject The location at the moment; For the perceiving subject The range of movement at any time; The power information of the drone, including Power at all times The power consumed by movement during a sensing cycle ; It is the unique hot encoding of the perceptual subject, which contains the category information of the perceptual subject. Figure 4 The image shows the local observation diagram of the perceiving subject, where Figure 4 The light grey squares in (b) represent obstacle areas. Figure 4The light grey squares in (c) represent obstacle areas, and the dark grey squares represent areas that the perceiving subject can move to at the next moment.

[0051] In one embodiment, the unmanned platform and the drone adopt a "hard collaboration" energy replenishment mode. If the unmanned platform detects that a drone cannot move due to insufficient power, the unmanned platform will replace the power for the drone at the next moment, as shown in the following formula:

[0052] ;

[0053] The following constraints apply to the change in the power of the drone: If the unmanned platform replaces the battery of the drone, the battery of the drone becomes 1. If the unmanned platform does not replace the battery of the drone, and the battery of the drone is less than the power consumed by the next sensing cycle, the drone will not move and the power will remain unchanged. If the battery of the drone can support the movement of the next sensing cycle, the drone moves and the power is reduced accordingly. These three situations are expressed by the following formulas:

[0054] ;

[0055] in, Indicates Drone in The amount of electricity at the moment, Indicates The amount of electricity consumed by a drone during a sensing process, Respectively represent The first drone and Unmanned ground platform The position at the moment.

[0056] In one embodiment, feature extraction is performed on the global system state and the local observation of the perception subject to obtain the feasible actions of the perception subject, and the value of the actions is evaluated. Then, a specific strategy is used to select actions, thereby evaluating the actions selected by each perception subject, thereby updating the network parameters and obtaining a trained neural network, including:

[0057] The convolutional neural network is used to extract features of the global system state and the local observation of the perception subject to obtain global system features and local observation features;

[0058] The local observation features, the action and hidden state of the perceptual subject at the last moment are input into the agent evaluation network to obtain the value of all actions and update the hidden state. After filtering out the inactionable actions, the actionable actions are selected according to the ε-greedy strategy to obtain the action value of the corresponding perceptual subject.

[0059] The action values, hidden states and global system characteristics of all perceptual subjects are input into a hybrid network composed of an overall Q network and a difference network to obtain a joint action value, a transformed action value and a difference; wherein the action values ​​of all perceptual subjects are summed up to obtain the transformed action value, and the joint action value and the difference are obtained according to the hidden states of all perceptual subjects and the global system characteristics, wherein the difference network includes a convolutional neural network.

[0060] The loss function is calculated based on the joint action value, the transformed action value and the difference, and the gradient descent method is used to update the neural network parameters. At this time, the neural network includes the agent evaluation network and the hybrid network.

[0061] During the training process of a deep neural network, if the value function network is used directly to calculate the target value, the calculated target value will also change frequently due to the continuous update of the parameters of the value function network during the training process. This frequent change may cause the training process to be unstable, so that the algorithm cannot converge. In addition, directly using the value function to calculate the target value will also lead to the bootstrapping problem (using the current learned knowledge to update itself during the learning process). This update method may lead to the accumulation and propagation of errors. Therefore, this application designs a target network for each value function network for updating the value function network parameters.

[0062] The target network and the value function network have the same network structure but different network parameters. The value function network is used to calculate the action value in the current state, while the target network is used to calculate the action value in the future state. During the training process, when the action value needs to be calculated, the algorithm will use the target network to calculate it. In this way, even if the parameters of the value network are continuously updated, the target value can remain stable for a period of time. After a certain period of time, the algorithm will directly copy the parameters in the value function network to the target network to update the target network.

[0063] As shown below, the pseudo code of the heterogeneous group learning-aware task allocation algorithm is provided:

[0064] Algorithm: QTRAN-plus heterogeneous group collaborative sensing task allocation algorithm

[0065] Input: environment information, simulation duration, learning rate;

[0066] Output: spatial convolutional network cnnSpace, agent evaluation network evalAgent, overall Network evalTotal, difference network V;

[0067] 1. Initialize cnnSpace, evalAgent, evalTotal, V, parameter update frequency , Experience Pool ;

[0068] 2. Copy the network parameters of evalAgent, evalTotal and cnnSpace to tgtAgent, tgtTotal and tgtcnnSpace respectively;

[0069] 3. ;

[0070] 4. While cnnSpace, evalAgent, evalTotal and V network did not converge;

[0071] 5.for simulation time from start to end do;

[0072] 6. Initialize the experience pool single game storage (No. Bureau, Moment, agents);

[0073] 7. Obtain the global state information of the agent , local state information and reach ;

[0074] 8. ;

[0075] 9. Combine and Get the next action of the agent ;

[0076] 10. Based on Update environmental information;

[0077] 11. Get reward value ;

[0078] 12. Get the agent's global state information , local state information and reach ;

[0079] 13.if simulation time is terminated then;

[0080] 14. ;

[0081] 15.else;

[0082] 16. ;

[0083] 17.end if;

[0084] 18. , , , , add to the experience pool middle;

[0085] 19.end for;

[0086] 20. Add to experience pool middle;

[0087] 21. Randomly extract training data from; / / The following steps are based on training data;

[0088] twenty two. ;

[0089] twenty three. ;

[0090] 24. Loss Function ;

[0091] 25. ;

[0092] 26. Evaluate;

[0093] 27. ;

[0094] 28. ;

[0095] 29.Based on Update evalAgent, evalTotal, V and cnnspace;

[0096] 30. ;

[0097] 31. if then;

[0098] 32. Copy the network parameters of evalAgent, evalQtotal and cnnspace to tgtAgent, tgtQtotal and tgtcnnSpace respectively;

[0099] 33.end if;

[0100] 34.end while;

[0101] In the pseudo code, the spatial convolution network cnnspace, the agent evaluation network evalagent, the overall Q network evaltotal and the difference network V are all value function networks. Tgtagent, tgttotal and tgtcnnSpace are the target networks of the agent evaluation network evalagent, the overall Q network evaltotal and the spatial convolution network cnnSpace respectively, and the difference network has no target network. P is the frequency of parameter update. Every P training times, the parameters in the value function network will be copied to its corresponding target network. The input of the algorithm is environmental information and perception conditions, and the output is a trained neural network. The algorithm as a whole can be divided into three parts. The first part is initialization, which mainly initializes the parameters of various neural networks and experience pools, and aligns the value network and evaluation network; the second part is information acquisition and decision execution. The algorithm processes the information obtained from the environment and executes corresponding decisions to perform collaborative perception task allocation; the third part is parameter training and network update. The algorithm randomly selects data for training and updates network parameters through loss functions to continuously improve the value of perception subject decisions.

[0102] After obtaining the trained neural network, the neural network parameters are input into the perception subject, and the autonomous collaborative perception of heterogeneous perception subjects in emergency search and rescue scenarios can be realized without the need for scheduling by the central hybrid network, thereby obtaining a heterogeneous group collaborative perception task allocation scheme for emergency rescue scenarios. The framework is as follows: Figure 5 As shown in Figure 2, the collaborative control of multiple sensing entities is effectively realized, where the QTRAN-plus hybrid network structure is as follows: Figure 6 shown.

[0103] In one embodiment, a device for generating a heterogeneous group collaborative sensing task allocation scheme is provided, comprising: a modeling module and a task allocation module, wherein:

[0104] A modeling module is used to model the task scenario, heterogeneous perception subjects and collaborative perception process; the heterogeneous perception subjects include: unmanned intelligent terminals and human workers; modeling the collaborative perception process of heterogeneous perception subjects includes: mathematically representing the global system state of the task scenario and the local observation of a single perception subject, and setting the action space and reward return of the perception subject;

[0105] The task allocation module is used to use the intelligent agent evaluation network to evaluate the value of the perceptual subject's feasible actions based on the local observation of the perceptual subject, and then use the overall Q network to evaluate the value of the actions selected by all perceptual subjects according to the global system state, calculate the loss function and use the gradient descent method to update the parameters of the intelligent agent evaluation network and the overall Q network to obtain a trained neural network, thereby obtaining a heterogeneous group collaborative perception task allocation solution.

[0106] For the specific limitations on the device for generating a heterogeneous group collaborative perception task allocation scheme, please refer to the limitations on the method for generating a heterogeneous group collaborative perception task allocation scheme above, which will not be repeated here. Each module in the above-mentioned device for generating a heterogeneous group collaborative perception task allocation scheme can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0107] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store task scenario information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating a heterogeneous group collaborative perception task allocation scheme is implemented.

[0108] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0111] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0112] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for generating a heterogeneous group collaborative sensing task allocation scheme, characterized in that: The method comprises: Modeling of task scenarios, heterogeneous perception subjects and collaborative perception processes; heterogeneous perception subjects include: unmanned intelligent terminals and human workers; modeling of the collaborative perception process of heterogeneous perception subjects includes: mathematically representing the global system state of the task scenario and the local observation of a single perception subject, and setting the action space and reward return of the perception subject; An agent evaluation network is used to evaluate the value of the perceptual subject's feasible actions according to the local observation of the perceptual subject, and then the overall Q network is used to evaluate the value of the actions selected by all perceptual subjects according to the global system state, and the loss function is calculated and the parameters of the agent evaluation network and the overall Q network are updated by the gradient descent method to obtain a trained neural network, thereby obtaining a heterogeneous group collaborative perception task allocation scheme; the neural network includes an agent evaluation network and an overall Q network; The convolutional neural network is used to extract features of the global system state and the local observation of the perception subject to obtain global system features and local observation features; The local observation features, the action and hidden state of the perceptual subject at the last moment are input into the agent evaluation network to obtain the value of all actions and update the hidden state. After filtering out the inactionable actions, the actionable actions are selected according to the ε-greedy strategy to obtain the action value of the corresponding perceptual subject. The action values, hidden states and global system characteristics of all perceptual subjects are input into a hybrid network composed of an overall Q network and a difference network to obtain a joint action value, a transformed action value and a difference; wherein the action values ​​of all perceptual subjects are summed to obtain the transformed action value, and the joint action value and the difference are obtained according to the hidden states of all perceptual subjects and the global system characteristics; wherein the difference network includes the convolutional neural network; The loss function is calculated based on the joint action value, the transformed action value and the difference, and the neural network parameters are updated using the gradient descent method; the neural network includes an agent evaluation network and the hybrid network.

2. The method according to claim 1, characterized in that Modeling of mission scenarios, including: The area of ​​the task scene is divided into several grids to obtain the corresponding two-dimensional area set; each element in the two-dimensional area set represents an area grid; each area grid has an obstacle identifier and a task identifier; the obstacle identifier is used to indicate whether there is an obstacle, and the task identifier is used to indicate whether there is a task; each task contains several perception subtasks; the attributes of each perception subtask include: 1) subtask type, that is, which type of perception subject needs to perform and complete it; 2) subtask location; 3) subtask execution time, which is a multiple of a perception cycle.

3. The method according to claim 1, characterized in that The unmanned intelligent terminal includes an unmanned aerial vehicle and an unmanned ground platform; Modeling of heterogeneous sensing agents, including: Construct a set of human workers; the attributes of each human worker include: The position at a moment and the range of movement within a perception cycle; Construct a collection of drones; the properties of each drone include: The current position and remaining power, the power consumed by moving within a sensing cycle, and the movable range; Construct a collection of unmanned ground platforms; the attributes of each unmanned ground platform include: The position at any moment, the movable range within a sensing cycle, and the drone detection range; the drone detection range refers to the range within which the unmanned ground platform can detect the drone's power and location information.

4. The method according to claim 1, characterized in that: Model the collaborative sensing process, including: The collaborative perception process of heterogeneous subjects is modeled based on a distributed partially objective Markov decision process model, and a seven-tuple is obtained. ;in is the system state space set; is the number of perceiving subjects; It is the set of possible actions of the perceiving subject; is the probability of transition between system states; It is the immediate reward that the next state system gets after the perceptual subject takes a certain action; is the discount factor; is the set of observations of the perceiving subject; The status of the system at the moment It can be expressed by the following formula: ； in, refers to The location distribution of obstacles at all times; refers to The location distribution of task points at all times; refers to The remaining execution time of the task point at a certain moment; Refers to the task type, refers to The matching situation between the task point at the moment and the perception subject, that is, which perception subject is currently executing the task at the moment; refers to Always be aware of the location distribution of the subject; Always aware of the subject Observed value for: ； in, The location distribution, execution time, and attribute category information of all perception tasks; For the perceiving subject The location at the moment; For the perceiving subject The movable range at the moment; The power information of the drone, including Power at all times The power consumed by movement within a sensing cycle ; It is the unique hot encoding of the perceptual subject, which contains the category information of the perceptual subject.

5. The method according to claim 3, characterized in that: The unmanned ground platform and the drone adopt a hard-cooperative energy replenishment mode, that is, if the unmanned platform detects that a drone cannot move due to insufficient power, the unmanned platform will replace the power for the drone at the next moment; The following constraints apply to the change in the power of the drone: if the unmanned platform replaces the battery of the drone, the power of the drone becomes 1; if the unmanned platform does not replace the battery of the drone, and the power of the drone is less than the power required for the next sensing cycle, the drone will not move and the power remains unchanged; if the power of the drone can support the movement of the next sensing cycle, the drone moves and the power decreases accordingly; these three situations are expressed by the following formulas: ； in, Indicates Drone in The amount of electricity at the moment, Indicates The amount of electricity consumed by a drone during a sensing process, , Respectively represent The first drone and Unmanned ground platform The position at the moment.

6. A device for generating a heterogeneous group collaborative sensing task allocation scheme, characterized in that: The device comprises: A modeling module is used to model the task scenario, heterogeneous perception subjects and collaborative perception process; the heterogeneous perception subjects include: unmanned intelligent terminals and human workers; modeling the collaborative perception process of heterogeneous perception subjects includes: mathematically representing the global system state of the task scenario and the local observation of a single perception subject, and setting the action space and reward return of the perception subject; The task allocation module is used to use the agent evaluation network to evaluate the value of the perceptual subject's feasible actions according to the local observation of the perceptual subject, and then use the overall Q network to evaluate the value of the actions selected by all perceptual subjects according to the global system state, calculate the loss function and use the gradient descent method to update the parameters of the agent evaluation network and the overall Q network to obtain a trained neural network, thereby obtaining a heterogeneous group collaborative perception task allocation scheme; the neural network includes the agent evaluation network and the overall Q network; the global system state and the local observation of the perceptual subject are extracted through the convolutional neural network to obtain the global system features and local observation features; the local observation features of the perceptual subject, the action and hidden state of the previous moment are input into the agent evaluation network to obtain the value of all actions and update hidden state, after filtering out inactionable actions, selecting actionable actions according to the ε-greedy strategy to obtain the action value of the corresponding perception subject; inputting the action values, hidden states and global system characteristics of all perception subjects into a hybrid network composed of an overall Q network and a difference network to obtain a joint action value, a transformed action value and a difference; wherein the action values ​​of all perception subjects are summed up to obtain the transformed action value, and the joint action value and the difference are obtained according to the hidden states of all perception subjects and the global system characteristics; wherein the difference network includes the convolutional neural network; calculating the loss function according to the joint action value, the transformed action value and the difference, and updating the neural network parameters by the gradient descent method; the neural network includes an agent evaluation network and the hybrid network.

Images