A method and system for task allocation in collaborative measurement of aircraft skin using multiple heterogeneous robots

Abstract

This invention relates to the field of task allocation, specifically to a method and system for task allocation in collaborative measurement of aircraft skin by multiple heterogeneous robots. The method includes: 1. Defining the environment and agents; 2. Assigning initial states to multiple heterogeneous robots; 3. The multiple heterogeneous robots solve for the optimal action based on action selection strategies and linear programming models, then calculate the comprehensive global reward, update the experience replay pool after information sharing, construct a training dataset using the updated experience replay pool, and then train the main network and target network using the training dataset; 4. Transferring the trained main network and target network to the actual aircraft skin measurement scenario, using the final output of the trained main network as the task allocation strategy. This invention solves the problem of low task allocation efficiency in collaborative measurement of aircraft skin data by multiple heterogeneous robots, realizes information sharing among robots, dynamically adapts to scene changes, and improves the operational efficiency of multi-heterogeneous robot teams.

Description

Technical Field

[0001] This invention relates to the field of task allocation technology for multi-heterogeneous robots, and in particular to a task allocation method and system for collaborative measurement of aircraft skin by multi-heterogeneous robots. Background Technology

[0002] Aircraft skin is a critical component of aircraft structure. Data on its surface contours, shape errors, and thickness can reflect localized depressions, deformation, seam gaps, and misalignments, allowing for the timely detection of potential structural damage and fatigue issues. Therefore, aircraft surface data is crucial for ensuring the safe, efficient, and reliable operation of aircraft, requiring efficient and precise measurement technologies. However, relying solely on a single type of robot (such as a flight measurement robot or a mobile measurement robot) is insufficient to comprehensively cover all areas of the aircraft skin. Therefore, collaborative operation of multiple heterogeneous robots (flight measurement robots and mobile measurement robots) with appropriate task allocation is necessary to improve inspection efficiency and accuracy.

[0003] Different types of robots have their own advantages and limitations, requiring task allocation to optimize measurement coverage. Flying measurement robots can quickly scan large areas such as the top of the fuselage and wings, but struggle to enter confined spaces or perform high-precision inspections close to the skin. Mobile measurement robots, on the other hand, can get close to the fuselage and use sensors such as LiDAR and ultrasonic sensors to perform detailed measurements on critical parts such as rivets and seams, but their range of motion is limited, making it difficult to cover high places and complex curved surfaces. Furthermore, without proper task allocation, multiple robots may repeatedly measure the same area, leading to a waste of time and computing resources. Therefore, task allocation is necessary to rationally plan the measurement range of each robot and improve overall measurement efficiency.

[0004] Meanwhile, existing technologies have limitations in addressing the low efficiency of task allocation in collaborative measurement of aircraft skin data by multiple heterogeneous robots. Traditional reinforcement learning-based algorithms (such as Q-Learning) rely on iterative optimization of task allocation strategies by constructing state-action value tables. However, since the state space expands exponentially with the number of robots and task complexity, traditional algorithms struggle to handle real-time task allocation requirements in large-scale, multi-task scenarios. Heuristic search-based task allocation algorithms (such as genetic algorithms) simulate natural evolution to iteratively optimize task allocation schemes, achieving suboptimal solutions within a certain timeframe. However, heuristic algorithms typically require extensive parameter tuning and are poorly adaptable to complex dynamic environments. Static optimization algorithms based on deep learning (such as deep neural network classifiers) depend on large amounts of offline training data for model learning. While they can adapt to specific task allocation patterns during training, they lack adaptability to dynamic task changes and struggle to handle real-time measurement task allocation scenarios.

[0005] However, when the traditional DQN (Deep Q_Learning) algorithm is applied to the task of multi-heterogeneous robot collaborative measurement of aircraft surface data, it still faces the following challenges: On the one hand, the efficiency of task allocation is easily limited by robot communication and multi-objective optimization, and a single reward mechanism is difficult to fully consider key factors such as priority, cooperation and distance; on the other hand, the traditional DQN algorithm is insufficient in terms of convergence speed and global search capability, and may get stuck in local optima in dynamic environments. Summary of the Invention

[0006] This invention provides a task allocation method and system for collaborative measurement of aircraft skin by multiple heterogeneous robots, in order to solve the technical problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0008] This invention provides a task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots, comprising the following steps:

[0009] S1. Define the task allocation scenario, the state space and actions of each heterogeneous robot, and initialize the main network and target network corresponding to each heterogeneous robot;

[0010] S2. Assign an initial state to each heterogeneous robot in the state space;

[0011] S3. All heterogeneous robots are conditioned on CE equilibrium and the optimal action is solved according to the action selection strategy and linear programming model. Then the comprehensive global reward of the optimal action is calculated. The comprehensive global reward, state and action are stored in the experience replay pool after information sharing among multiple heterogeneous robots. The updated experience replay pool is used to build a training dataset. Then the training dataset is used to train the main network and the target network.

[0012] S4. Transfer the trained main network and target network to the actual aircraft skin measurement scenario, and use the final output of the trained main network as the task allocation strategy.

[0013] Furthermore, step S1 specifically includes the following steps:

[0014] S11. Define a task allocation scenario for collaborative measurement of aircraft skin by multiple heterogeneous robots. The task allocation scenario includes three heterogeneous robots: one flight measurement robot and two mobile measurement robots.

[0015] S12. Define the state space for each heterogeneous robot. Robot position Task distribution ,in , Indicates the task location; Represents the basic priority of tasks; state space That is, the set of spatial locations where each heterogeneous robot begins its action;

[0016] S13, Define the actions of the flight measurement robot Actions of the flight measurement robot For measurement, or for moving horizontally north, west, south, or east, or vertically upward or downward;

[0017] Define the actions of two mobile measurement robots , The actions of the two mobile measuring robots , All of these are measurements, or any one of the following: the sensor moves forward or backward, or the sensor is raised or lowered.

[0018] S14. Select and initialize the main network for each heterogeneous robot. and target network ;

[0019] S15. Define the learning parameters, including the discount factor. Learning rate Exploration rate Then initialize the experience replay pool D for each heterogeneous robot.

[0020] Furthermore, step S2 specifically includes the following steps:

[0021] S21. First, determine the state space of each heterogeneous robot. Scope;

[0022] S22, in the state space An initial state is assigned to each corresponding heterogeneous robot. This ensures that the initial state meets the constraints of the measurement task and the limitations of the aircraft skin scenario.

[0023] Furthermore, step S3 specifically includes the following steps:

[0024] S31. Based on the information interaction mechanism, multiple heterogeneous robots conduct data interaction to achieve information sharing;

[0025] S32. All heterogeneous robots select the optimal action based on a CE equilibrium and an action selection strategy. CE equilibrium means that there exists a "coordinator" that recommends actions for all heterogeneous robots, such that when other heterogeneous robots follow the recommended actions, any single heterogeneous robot deviating from the recommended actions will not gain a higher expected benefit. If the optimal joint action is selected... If the optimal action of all heterogeneous robots is selected as random action, then proceed to S33.

[0026] S33. Using the three established probabilistic conditions as constraints, the constraints form a linear programming model to maximize the performance of all heterogeneous robots in the current state. Using the Q-value as the objective function, the optimal joint action of multiple heterogeneous robots is solved. Q-value is the metric for evaluating the value of an action; then, based on the optimal combined action... Obtain the optimal action for each heterogeneous robot;

[0027] S34. Each heterogeneous robot executes the optimal action and dynamically calculates the comprehensive global reward corresponding to the completion of the action based on three key factors: dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward.

[0028] S35. Obtain the next state based on the overall global reward. It then determines whether the aircraft skin measurement task is complete; if so, it updates the current state. Next state ,action The overall global rewards are stored in the corresponding experience replay pool D to update the experience replay pool D and enter S36; otherwise, the task allocation strategy is output directly.

[0029] S36. Randomly sample B training samples from the experience replay pool D, and assign an index to each of the B training samples.

[0030] S37. Select one training sample from B training samples and input it into the target network. Calculate the Q value of the optimal action of each target network in the next state according to the Bellman equation.

[0031] S38. For each heterogeneous robot, calculate the mean squared error loss (MSE) of the heterogeneous robot using the Q value of the corresponding optimal action and the output of the current main network, and repeat steps S36 to S38 until the set number of iterations is reached. Update the main network parameters by minimizing the mean squared error loss (MSE). At the same time, in each iteration, the initialized counter is incremented by 1. When the preset interval step number N is reached, the parameters of the main network are completely copied to the corresponding target network so that the parameters of the target network are completely consistent with the current main network.

[0032] Furthermore, the action selection strategy in S32 is the optimal joint action. The method of selecting actions randomly is expressed as follows:

[0033] ;

[0034] in, This indicates the action selected by the action selection strategy; Indicates the probability of choosing the current action;

[0035] In the above formula, the exploration rate The following relationship must be satisfied:

[0036] ;

[0037] Where k represents the kth time step; The attenuation rate; To maximize the probability of exploration; denoted as the minimum exploration probability; e represents the natural constant.

[0038] Furthermore, step S33 specifically includes the following steps:

[0039] S331, through the state space of each heterogeneous robot Constructing a joint state space , ,in, These represent the state spaces of one flying measurement robot and two mobile measurement robots, respectively.

[0040] Actions of each heterogeneous robot Construct the corresponding individual action space Then, a joint motion space is constructed based on the individual motion space of each heterogeneous robot. , ,in These represent the individual motion spaces of the flight measurement robot and the two mobile measurement robots, respectively.

[0041] Establish a transition probability function for each heterogeneous robot transition probability function This refers to heterogeneous robots taking action. From the state to state The probability function;

[0042] S332. Establish three probabilistic conditions as constraints for the heterogeneous robot. The three probabilistic conditions are as follows:

[0043] The first probabilistic condition: The sum of the probabilities of all actions of each heterogeneous robot i is 1, specifically in the form of:

[0044] ;

[0045] in, Select an action for heterogeneous robot i at time step k. The probability; m represents the total number of heterogeneous robots; In mathematics, it is a universal quantifier, meaning "for all";

[0046] The second probabilistic condition: the probability of each heterogeneous robot i choosing an action must be non-negative and conform to the probability distribution of actions. Specifically, conforming to the probability distribution of actions means conforming to the seven possible actions of the flying measurement robot and the five possible actions of the mobile measurement robot. The seven possible actions are: the flying measurement robot performs measurement, moves north, west, south, or east in the lateral direction, or moves up or down; the five possible actions are: the mobile measurement robot performs measurement, moves forward, moves backward, and raises or lowers the sensor.

[0047] The specific form of the second probabilistic condition is:

[0048] ;

[0049] The third probabilistic condition: Guarantee that the current action of each heterogeneous robot i is superior to other actions, specifically in the form of:

[0050] ;

[0051] in, For the remaining actions of other heterogeneous robots; Indicates another candidate action for the i-th robot; Indicates the current state If the i-th heterogeneous robot chooses The reward for the i-th heterogeneous robot remains unchanged, while the reward for the other heterogeneous robots remains unchanged.

[0052] S333, to maximize the current state of all heterogeneous robots. The Q-value is used as the objective function, specifically in the form of:

[0053] ;

[0054] in, Indicates the state of heterogeneous robot i. Execute the optimal joint action Q-value; optimal joint action Let represent the joint action of three heterogeneous robots at time step k, and , These represent the actions of one flying measurement robot and two mobile measurement robots at time step k, respectively.

[0055] S334, Combine the state space Joint Action Space and transition probability function As input to the main network of each heterogeneous robot, the main network solves for the optimal joint action by simultaneously solving the constraints and the objective function. Then, based on the optimal joint action... Obtain the optimal action for each heterogeneous robot.

[0056] Furthermore, step S34 specifically includes the following steps:

[0057] S341. Each heterogeneous robot executes its optimal action, triggering an update of the environmental state.

[0058] S342. Calculate the dynamic region priority reward of heterogeneous robot i, using the following formula:

[0059] ;

[0060] in, Indicates dynamic region priority reward; This represents the base priority value of the nth task region; This represents the distance between the position of heterogeneous robot i and the task area n; This indicates the duration during which task region n remains unprocessed. Indicates the time decay coefficient; This indicates the avoidance of division by zero error when the distance is zero, i.e., division by zero error one.

[0061] S343. Calculate the distance penalty reward for heterogeneous robot i, using the following formula:

[0062] ;

[0063] in, Indicates distance as a penalty or reward; This represents the efficiency coefficient of the i-th heterogeneous robot in solving the task;

[0064] S344. Calculate the collaborative reward of heterogeneous robot i in a multi-heterogeneous robot group. The calculation formula is as follows:

[0065] ;

[0066] in, This indicates a collaborative reward among a group of heterogeneous robots. The synergy factor represents the collaboration between heterogeneous robots i and j. This represents the collaboration efficiency between heterogeneous robots i and j in the current task; This represents the distance between heterogeneous robot i and heterogeneous robot j at time step k; This indicates the avoidance of division by zero error when the distance is zero, i.e., division by zero error two;

[0067] S345. Based on the relative sharing of the dynamic region priority reward, distance penalty reward, and multi-heterogeneous robot group collaborative reward of heterogeneous robot i, normalization calculation is performed to obtain the weight factors of the three rewards of heterogeneous robot i, and the calculation formula is as follows:

[0068] ;

[0069] ;

[0070] ;

[0071] in, , , Let represent the weight factors of the dynamic region priority reward, distance penalty reward, and multi-heterogeneous robot group collaborative reward, respectively, and satisfy the following relationship:

[0072] ;

[0073] S346. Based on the dynamic region priority reward, distance penalty reward, multi-heterogeneous robot group cooperation reward, and their respective weight factors, dynamically calculate the comprehensive global reward of heterogeneous robot i. The calculation formula is as follows:

[0074] ;

[0075] in, This represents the overall global reward of heterogeneous robot i, that is, the overall global reward corresponding to the completion of the action of heterogeneous robot i;

[0076] S347, repeat S342 to S346 until the combined global reward of all heterogeneous robots is obtained.

[0077] Furthermore, the formula for calculating the Q value of the optimal action in S37 is:

[0078] ;

[0079] in, Indicates the first The Q-value of the best action in each training sample; This represents the comprehensive global reward of heterogeneous robot i at time step k; This represents the maximum Q-value predicted by the target network for the next state; This indicates that the aircraft skin measurement mission has been completed; This indicates that the aircraft skin measurement mission has not been completed.

[0080] Furthermore, the specific formula for calculating the mean squared error loss (MSE) in S38 is as follows:

[0081] ;

[0082] in, This indicates the current output of the main network; Indicates the first The current state in each training sample; Indicates the first Actions in each training sample.

[0083] In another aspect, the present invention provides a task allocation system for collaborative measurement of aircraft skin by multiple heterogeneous robots, comprising multiple heterogeneous robots, which are configured or execute the above task allocation method.

[0084] The beneficial effects of this invention are:

[0085] 1. This invention provides a task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots, which enables information sharing among the multiple heterogeneous robots and improves the globality of task allocation;

[0086] This invention introduces an information exchange mechanism, enabling multiple heterogeneous robots to share their status information and network parameters in real time, thus overcoming the limitations of single-robot task decision-making. Through information sharing, the multi-heterogeneous robot team can comprehensively grasp the overall task status, thereby optimizing task allocation schemes, avoiding task conflicts and duplications in measuring aircraft skin data, and improving overall task execution efficiency.

[0087] 2. This invention optimizes the allocation of multi-objective tasks by integrating and comprehensively considering global rewards;

[0088] By introducing a fusion of global rewards, this invention incorporates three factors—dynamic regional priority rewards for tasks, distance penalty rewards for robots, and collaborative rewards for multi-heterogeneous robot groups—into the action selection within the task allocation strategy. This mechanism not only improves the response efficiency of high-priority tasks but also reduces energy consumption and time costs associated with multi-heterogeneous robot allocation. Furthermore, it encourages collaboration among multi-heterogeneous robots, further enhancing the global optimization effect of task allocation.

[0089] 3. This invention accelerates the convergence speed of task allocation;

[0090] Traditional reinforcement learning methods often suffer from slow convergence when dealing with large-scale heterogeneous robot teams due to the excessively large state space. This invention improves upon the DQN method by introducing an action selection strategy, balancing exploration and utilization among multiple heterogeneous robots. It not only enables the robots to perform tasks using optimal actions but also allows them to explore the task area through random actions. For complex components such as aircraft skin, this exploration strategy can help multiple heterogeneous robots discover potential critical areas.

[0091] 4. This invention utilizes CE equalization to take into account the differences in capabilities and functions among multiple heterogeneous robots (such as sensor type, coverage, measurement accuracy, etc.) during task allocation. It uses a linear programming model to help find the stable behavior of each heterogeneous robot, thereby improving the collaborative efficiency of the multi-heterogeneous robot team. Attached Figure Description

[0092] Figure 1 This is a flowchart of the task allocation method in this invention;

[0093] Figure 2 This is a flowchart of step S3 in the task allocation method of this invention;

[0094] Figure 3 This is a schematic diagram illustrating the operational principle of S3 within the task allocation method of this invention.

[0095] Figure 4 This is a schematic diagram of the task allocation system in this invention. Detailed Implementation

[0096] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0097] Reference Figure 1 This application provides a method for task allocation in the collaborative measurement of aircraft skin by multiple heterogeneous robots, including the following steps:

[0098] S1. Define the task allocation scenario, the state space and actions of each heterogeneous robot, and initialize the main network and target network corresponding to each heterogeneous robot;

[0099] S2. Assign an initial state to each heterogeneous robot in the state space;

[0100] S3. All heterogeneous robots are conditioned on CE equilibrium and the optimal action is solved according to the action selection strategy and linear programming model. Then the comprehensive global reward of the optimal action is calculated. The comprehensive global reward, state and action are stored in the experience replay pool after information sharing among multiple heterogeneous robots. The updated experience replay pool is used to build a training dataset. Then the training dataset is used to train the main network and the target network.

[0101] S4. Transfer the trained main network and target network to the actual aircraft skin measurement scenario, and use the final output of the trained main network. As a task allocation strategy.

[0102] This invention addresses the problem of low task allocation efficiency in scenarios involving collaborative measurement of aircraft skin data by multiple heterogeneous robots. It enables information sharing among these robots, thereby improving the operational efficiency of the multi-heterogeneous robot team. Based on the DQN method, this invention introduces CE (correlated Equilibrium), action selection strategies, and comprehensive global rewards to achieve efficient task allocation in environments where multiple heterogeneous robots collaboratively measure aircraft skin surface data.

[0103] In some embodiments, S1 specifically includes the following steps:

[0104] S11. Define a task allocation scenario for collaborative measurement of aircraft skin by multiple heterogeneous robots. The task allocation scenario includes three heterogeneous robots: one flight measurement robot and two mobile measurement robots.

[0105] S12. Define the state space for each heterogeneous robot. Robot position Task distribution ,in , Indicates the task location; Represents the basic priority of tasks; state space That is, the set of spatial positions where each heterogeneous robot starts its action; each heterogeneous robot has its own corresponding state space. The state space does not store the state, but only provides a mathematical description of the robot's state. The actual storage structure is the experience replay pool. The experience replay pool D is used to store transition samples and does not store the initial state separately. The initial state is generated through environmental initialization and is implicit in the first transition sample.

[0106] S13, Define the actions of the flight measurement robot Actions of the flight measurement robot For measurement, or for moving horizontally north, west, south, or east, or vertically upward or downward;

[0107] Define the actions of two mobile measurement robots , The actions of the two mobile measuring robots , All of these are measurements, or any one of the following: the sensor moves forward or backward, or the sensor is raised or lowered.

[0108] S14. Select and initialize the main network for each heterogeneous robot. and target network The specific expression is as follows:

[0109] ; ;

[0110] S15. Define the learning parameters, including the discount factor. Learning rate Exploration rate Among them, discount factor The importance of controlling future rewards; learning rate Used to adjust the step size for each parameter update; exploration rate This is used to achieve a balance between control exploration strategies (random actions) and utilization (selecting the optimal joint action); then, the experience replay pool D for each heterogeneous robot is initialized. The experience replay pool D is used to store state transition samples of the interactions, each sample can be represented as... .

[0111] In some embodiments, S2 specifically includes the following steps:

[0112] S21. First, determine the state space of each heterogeneous robot. Scope;

[0113] S22, in the state space An initial state is assigned to each corresponding heterogeneous robot. , represented as The coordinates in parentheses represent the initial position coordinates; ensure that the initial state satisfies the measurement task constraints and the aircraft skin scene limitations.

[0114] In some embodiments, refer to Figure 2 and Figure 3 S3 specifically includes the following steps:

[0115] S31. Based on the information interaction mechanism, multiple heterogeneous robots conduct data interaction to achieve information sharing;

[0116] First, the heterogeneous robot i (the transmitter) broadcasts its current state. The network parameters of the current main network and target network The system transmits the currently executed action to other heterogeneous robots within communication range. Then, the other heterogeneous robots (receivers) update their state information regarding heterogeneous robot i. and The data from the interactions stored in each heterogeneous robot is represented as follows: (Stored in the recipient's experience revisit pool) ;

[0117] This invention introduces an information exchange mechanism, enabling multiple heterogeneous robots to share their status information and network parameters in real time, thus overcoming the limitations of single-robot task decision-making. Through information sharing, the multi-heterogeneous robot team can comprehensively grasp the overall task status, thereby optimizing task allocation schemes, avoiding task conflicts and duplications in measuring aircraft skin data, and improving overall task execution efficiency.

[0118] S32. All heterogeneous robots are subject to CE (correlated Equilibrium) equilibrium and make optimal action selection based on action selection strategy. CE equilibrium means that there is a "coordinator" that recommends actions for all heterogeneous robots, so that when other heterogeneous robots follow the recommended actions, no single heterogeneous robot will gain higher expected benefits if it deviates from the recommended actions. In the task allocation of multiple heterogeneous robots, each robot may have different capabilities, efficiencies, etc. CE equilibrium takes these differences into account to make task allocation more efficient.

[0119] If selected as the optimal joint action If the process proceeds to S33, and random joint action is selected, then the optimal action of all heterogeneous robots is identified as random action. Random action is the exploration strategy. The exploration strategy enables heterogeneous robots to perform random actions in the environment based on the initial state, while recording the environmental feedback in the initial state to provide reference data for subsequent optimization. It also enables the exploration of unknown areas.

[0120] Optimal joint action for multiple heterogeneous robots under high probability conditions In the process of measuring aircraft skin, in most cases, multiple heterogeneous robots adopt joint actions to maximize measurement efficiency, and CE equalization is used to balance these joint actions. Evaluation and calculation are performed to ensure that task allocation efficiency is maximized; random actions are adopted for multiple heterogeneous robots under low probability conditions. When a heterogeneous robot enters a new skin measurement area for the first time, the multiple heterogeneous robots use certain exploratory behaviors to obtain new aircraft skin measurement data.

[0121] This invention introduces an action selection strategy that balances exploration and utilization by multiple heterogeneous robots. It can not only perform tasks using optimal actions, but also explore the task area through random actions. For complex components such as aircraft skin, the exploration strategy can also help multiple heterogeneous robots discover potential critical areas.

[0122] Furthermore, this invention utilizes CE equalization to take into account the differences in capabilities and functions among multiple heterogeneous robots (such as sensor type, coverage, measurement accuracy, etc.) during task allocation, and uses a linear programming model to help find the stable behavior of each heterogeneous robot, thereby improving the collaborative efficiency of the multi-heterogeneous robot team.

[0123] S33. Using the three established probabilistic conditions as constraints, the constraints form a linear programming model to maximize the performance of all heterogeneous robots in the current state. Using the Q-value as the objective function, the optimal joint action of multiple heterogeneous robots is solved. Q-value is the metric for evaluating the value of an action; then, based on the optimal combined action... Obtain the optimal action for each heterogeneous robot;

[0124] S34. Each heterogeneous robot executes the optimal action and dynamically calculates the comprehensive global reward corresponding to the completion of the action based on three key factors: dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward.

[0125] This invention introduces a fusion-based global reward system, incorporating three factors—dynamic region priority reward, robot distance penalty reward, and multi-heterogeneous robot group collaboration reward—into the action selection within the task allocation strategy. This mechanism not only improves the response efficiency of high-priority tasks but also reduces the energy consumption of multiple heterogeneous robots and the time cost of task allocation. Furthermore, it encourages collaboration among heterogeneous robots, further enhancing the global optimization effect of task allocation.

[0126] S35. Obtain the next state based on the overall global reward. It then determines whether the aircraft skin measurement task is complete. If so (done=True), it updates the current state. Next state ,action The overall global rewards are stored in the corresponding experience replay pool D, which is the sample. Store it in the experience replay pool D to update the experience replay pool D, and then enter S36; otherwise (done=False), directly output the task allocation strategy.

[0127] S36. Randomly sample B training samples from the experience replay pool D, and assign indices to each of the B training samples; the mini-batch samples are represented as follows:

[0128] ;

[0129] Simultaneously, the sampled data includes interaction data of multiple heterogeneous robots in different states, ensuring the diversity of training data;

[0130] S37. Select one training sample from B training samples and input it into the target network. Calculate the Q value of the optimal action of each target network in the next state according to the Bellman equation.

[0131] S38. For each heterogeneous robot, calculate the mean squared error loss (MSE) of that robot using the Q-value of the corresponding optimal action and the output of the current main network. Repeat steps S36 to S38 until the set number of iterations is reached. Update the main network parameters by minimizing the MSE. The representation is as follows:

[0132] ;

[0133] in, The learning rate is used to control the step size for parameter updates. ; This represents a discount factor used to control the importance of future rewards; Indicates the learning rate of the target network;

[0134] Meanwhile, in each loop, the initialized counter is incremented by 1. When the preset interval number of steps N (preferably 10,000 steps) is reached, the parameters of the main network are completely copied to the corresponding target network, so that the parameters of the target network are completely consistent with the current main network.

[0135] In some embodiments, the action selection strategy in S32 is the optimal joint action. The method of selecting actions randomly is expressed as follows:

[0136] ;

[0137] in, This indicates the action selected by the action selection strategy; Indicates the probability of choosing the current action;

[0138] In the above formula, the exploration rate The following relationship must be satisfied:

[0139] ;

[0140] Where k represents the k-th time step; The attenuation rate is preferred; the preferred attenuation rate is 0.0001. To maximize the exploration probability, the maximum exploration probability is set to 1. The minimum exploration probability is preferably 0.01; e represents the natural constant.

[0141] In some embodiments, refer to Figure 2 and Figure 3 S33 specifically includes the following steps:

[0142] S331, through the state space of each heterogeneous robot Constructing a joint state space , ,in, These represent the state spaces of one flying measurement robot and two mobile measurement robots, respectively.

[0143] Actions of each heterogeneous robot Construct the corresponding individual action space Then, a joint motion space is constructed based on the individual motion space of each heterogeneous robot. , ,in These represent the individual motion spaces of the flight measurement robot and the two mobile measurement robots, respectively.

[0144] Establish a transition probability function for each heterogeneous robot transition probability function This refers to heterogeneous robots taking action. From the state to state The probability function;

[0145] S332. Establish three probabilistic conditions as constraints for the heterogeneous robot. The three probabilistic conditions are as follows:

[0146] The first probabilistic condition: The sum of the probabilities of all actions of each heterogeneous robot i is 1, specifically in the form of:

[0147] ;

[0148] in, Select an action for heterogeneous robot i at time step k. The probability; m represents the total number of heterogeneous robots; In mathematics, it is a universal quantifier, meaning "for all";

[0149] The second probabilistic condition: the probability of each heterogeneous robot i choosing an action must be non-negative and conform to the probability distribution of actions. Specifically, conforming to the probability distribution of actions means conforming to the seven possible actions of the flying measurement robot and the five possible actions of the mobile measurement robot. The seven possible actions are: the flying measurement robot performs measurement, moves north, west, south, or east in the lateral direction, or moves up or down; the five possible actions are: the mobile measurement robot performs measurement, moves forward, moves backward, and raises or lowers the sensor.

[0150] The specific form of the second probabilistic condition is:

[0151] ;

[0152] The third probabilistic condition: Guarantee that the current action of each heterogeneous robot i is superior to other actions, specifically in the form of:

[0153] ;

[0154] in, For the remaining actions of other heterogeneous robots; Let represent another candidate action for the i-th heterogeneous robot, indicating that the heterogeneous robot may try to deviate from another action; Indicates the current state If the i-th heterogeneous robot chooses The reward for the i-th robot remains unchanged, while the reward for the other robots remains the same.

[0155] S333, to maximize the current state of all heterogeneous robots. The Q-value is used as the objective function, specifically in the form of:

[0156] ;

[0157] in, Indicates the state of heterogeneous robot i. Execute the optimal joint action Q-value; optimal joint action Let represent the joint action of three heterogeneous robots at time step k, and , These represent the actions of one flying measurement robot and two mobile measurement robots at time step k, respectively.

[0158] S334, Combine the state space Joint Action Space and transition probability function As input to the main network of each heterogeneous robot, the main network solves for the optimal joint action by simultaneously solving the constraints and the objective function. Then, based on the optimal joint action... Obtain the optimal action for each heterogeneous robot.

[0159] In some embodiments, refer to Figure 2 and Figure 3 S34 specifically includes the following steps:

[0160] S341. Each heterogeneous robot executes its optimal action, triggering an update of the environmental state.

[0161] S342. Calculate the dynamic region priority reward of heterogeneous robot i, using the following formula:

[0162] ;

[0163] in, Indicates dynamic region priority reward; This represents the base priority value of the nth task area (preset based on the aircraft skin measurement scenario). This represents the distance between the position of heterogeneous robot i and the task area n; This indicates the duration during which task region n remains unprocessed. This represents the time decay coefficient (the sensitivity of task area priority to changes over time; set to 200, meaning the rate of change of task area priority over time is moderate). This indicates the avoidance of division by zero error when the distance is zero (taken as 10). -6 ), that is, a division-to-zero error of one;

[0164] In the above formula, The specific calculation formula is as follows:

[0165] ;

[0166] in, , , Let x, y, and z represent the x, y, and z coordinates of heterogeneous robot i at time step k, respectively; , , Let x, y, and z represent the x, y, and z coordinates of the heterogeneous robot n at time step k, respectively;

[0167] S343. Calculate the distance penalty reward for heterogeneous robot i, using the following formula:

[0168] ;

[0169] in, Indicates distance as a penalty or reward; This represents the efficiency coefficient of the i-th heterogeneous robot in solving the task;

[0170] S344. Calculate the collaborative reward of heterogeneous robot i in a multi-heterogeneous robot group. The calculation formula is as follows:

[0171] ;

[0172] in, This indicates a collaborative reward among a group of heterogeneous robots. Let i represent the collaboration factor between heterogeneous robots i and j; preferably, the collaboration factor between the flight measurement robot and the mobile measurement robot is: , The aerial surveying robot can first scan a large area, while the mobile surveying robot can perform detailed local measurements; the cooperation factor between the two mobile surveying robots is: The two mobile measuring robots can cover different angles of each other, but their measuring capabilities are similar.

[0173] This represents the collaboration efficiency (such as task completion rate and communication quality) of heterogeneous robots i and j in the current task. This represents the distance between heterogeneous robot i and heterogeneous robot j at time step k; This indicates the avoidance of division by zero error when the distance is zero (taken as 10). -6 That is, the division-to-zero error is two.

[0174] S345. Based on the relative sharing of the dynamic region priority reward, distance penalty reward, and multi-heterogeneous robot group collaborative reward of heterogeneous robot i, normalization calculation is performed to obtain the weight factors of the three rewards of heterogeneous robot i, and the calculation formula is as follows:

[0175] ;

[0176] ;

[0177] ;

[0178] in, , , Let represent the weight factors of the dynamic region priority reward, distance penalty reward, and multi-heterogeneous robot group collaborative reward, respectively, and satisfy the following relationship:

[0179] ;

[0180] S346. Based on the dynamic region priority reward, distance penalty reward, multi-heterogeneous robot group cooperation reward, and their respective weight factors, dynamically calculate the comprehensive global reward of heterogeneous robot i. The calculation formula is as follows:

[0181] ;

[0182] in, This represents the overall global reward of heterogeneous robot i, that is, the overall global reward corresponding to the completion of the action of heterogeneous robot i;

[0183] S347, repeat S342 to S346 until the combined global reward of all heterogeneous robots is obtained.

[0184] In some embodiments, the formula for calculating the Q value of the optimal action in S37 is:

[0185] ;

[0186] in, Indicates the first The Q-value of the best action in each training sample; This represents the comprehensive global reward of heterogeneous robot i at time step k; This represents the maximum Q-value predicted by the target network for the next state; This indicates that the aircraft skin measurement mission has been completed; This indicates that the aircraft skin measurement mission has not been completed.

[0187] In some embodiments, the formula for calculating the mean squared error loss (MSE) in step S38 is as follows:

[0188] ;

[0189] in, This indicates the current output of the main network; Indicates the first The current state in each training sample; Indicates the first Actions in each training sample.

[0190] Reference Figure 4 In another aspect, the present invention provides a task allocation system for collaborative measurement of aircraft skin by multiple heterogeneous robots, including multiple heterogeneous robots, which are configured or execute the above task allocation method.

[0191] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots, characterized in that, Includes the following steps: S1. Define the task allocation scenario, the state space and actions of each heterogeneous robot, and initialize the main network and target network corresponding to each heterogeneous robot; S2. Assign an initial state to each heterogeneous robot in the state space; S3. All heterogeneous robots are conditioned on CE equilibrium and the optimal action is solved according to the action selection strategy and linear programming model. Then the comprehensive global reward of the optimal action is calculated. The comprehensive global reward, state and action are stored in the experience replay pool after information sharing among multiple heterogeneous robots. The updated experience replay pool is used to build a training dataset. Then the training dataset is used to train the main network and the target network. S4. Transfer the trained main network and target network to the actual aircraft skin measurement scenario, and use the final output of the trained main network as the task allocation strategy. The overall global reward includes dynamic region priority reward for heterogeneous robots, distance penalty reward, and multi-heterogeneous robot group collaboration reward; S3 specifically includes the following steps: S31. Based on the information interaction mechanism, multiple heterogeneous robots conduct data interaction to achieve information sharing; S32. All heterogeneous robots select the optimal action based on a Coordinator-Coordinator (CE) equilibrium and an action selection strategy. CE equilibrium means that there exists a "coordinator" that recommends actions for all heterogeneous robots, such that when other heterogeneous robots follow the recommended actions, any single heterogeneous robot deviating from the recommended actions will not gain a higher expected benefit. If the optimal joint action is selected... If the optimal action of all heterogeneous robots is selected as random action, then proceed to S33. S33. Using the three established probabilistic conditions as constraints, the constraints form a linear programming model to maximize the performance of all heterogeneous robots in the current state. Using the Q-value as the objective function, the optimal joint action of multiple heterogeneous robots is solved. Q-value is the metric for evaluating the value of an action; then, based on the optimal combined action... Obtain the optimal action for each heterogeneous robot; S34. Each heterogeneous robot executes the optimal action and dynamically calculates the comprehensive global reward corresponding to the completion of the action based on three key factors: dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward. S35. Obtain the next state based on the overall global reward. It then determines whether the aircraft skin measurement task is complete; if so, it updates the current state. Next state ,action The overall global rewards are stored in the corresponding experience replay pool D to update the experience replay pool D and enter S36; otherwise, the task allocation strategy is output directly. S36. Randomly sample B training samples from the experience replay pool D, and assign an index to each of the B training samples. S37. Select one training sample from B training samples and input it into the target network. Calculate the Q value of the optimal action of each target network in the next state according to the Bellman equation. S38. For each heterogeneous robot, calculate the mean squared error loss (MSE) of the heterogeneous robot using the Q value of the corresponding optimal action and the output of the current main network, and repeat steps S36 to S38 until the set number of iterations is reached. Update the main network parameters by minimizing the mean squared error loss (MSE). At the same time, in each iteration, the initialized counter is incremented by 1. When the preset interval step number N is reached, the parameters of the main network are completely copied to the corresponding target network so that the parameters of the target network are completely consistent with the current main network.

2. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 1, characterized in that, S1 specifically includes the following steps: S11. Define a task allocation scenario for collaborative measurement of aircraft skin by multiple heterogeneous robots. The task allocation scenario includes three heterogeneous robots: one flight measurement robot and two mobile measurement robots. S12. Define the state space for each heterogeneous robot. Robot position Task distribution ,in , Indicates the task location; Represents the basic priority of tasks; state space That is, the set of spatial locations where each heterogeneous robot begins its action; S13, Define the actions of the flight measurement robot Actions of the flight measurement robot For measurement, or moving horizontally north, west, south, or east, or vertically upward or downward; Define the actions of two mobile measurement robots , The actions of the two mobile measuring robots , All of these involve measurement, or the sensor moving forward or backward, or the sensor moving upward or downward. S14. Select and initialize the main network for each heterogeneous robot. and target network ; S15. Define the learning parameters, including the discount factor. Learning rate Exploration rate Then initialize the experience replay pool D for each heterogeneous robot.

3. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 2, characterized in that, S2 specifically includes the following steps: S21. First, determine the state space of each heterogeneous robot. Scope; S22, in the state space An initial state is assigned to each corresponding heterogeneous robot. This ensures that the initial state meets the constraints of the measurement task and the limitations of the aircraft skin scenario.

4. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 3, characterized in that, The action selection strategy in S32 is the optimal joint action. The method of selecting actions randomly is expressed as follows: ； in, This indicates the action selected by the action selection strategy; Indicates the probability of choosing the current action; In the above formula, the exploration rate The following relationship must be satisfied: ； Where k represents the kth time step; The attenuation rate; To maximize the probability of exploration; denoted as the minimum exploration probability; e represents the natural constant.

5. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 4, characterized in that, S33 specifically includes the following steps: S331, through the state space of each heterogeneous robot Constructing a joint state space , ,in, , , These represent the state spaces of one flying measurement robot and two mobile measurement robots, respectively. Actions of each heterogeneous robot Construct the corresponding individual action space Then, a joint motion space is constructed based on the individual motion space of each heterogeneous robot. , ,in , , These represent the individual motion spaces of the flight measurement robot and the two mobile measurement robots, respectively. Establish a transition probability function for each heterogeneous robot transition probability function This refers to heterogeneous robots taking action. From the state to state The probability function; S332. Establish three probabilistic conditions as constraints for the heterogeneous robot. The three probabilistic conditions are as follows: The first probabilistic condition: The sum of the probabilities of all actions of each heterogeneous robot i is 1, specifically in the form of: ； in, Select an action for heterogeneous robot i at time step k. The probability; m represents the total number of heterogeneous robots; In mathematics, it is a universal quantifier, meaning "for all"; The second probabilistic condition: the probability of each heterogeneous robot i choosing an action must be non-negative and conform to the probability distribution of actions. Specifically, conforming to the probability distribution of actions means conforming to the seven possible actions of the flight measurement robot and the five possible actions of the mobile measurement robot. The seven possible actions are that the flight measurement robot performs measurement, or moves north, west, south, or east in the lateral direction, or moves up or down in the vertical direction; the five possible actions are that the mobile measurement robot performs measurement, or the sensor moves forward or backward, or the sensor moves up or down. The specific form of the second probabilistic condition is: ； The third probabilistic condition: Guarantee that the current action of each heterogeneous robot i is superior to other actions, specifically in the form of: ； in, For the remaining actions of other heterogeneous robots; Indicates another candidate action for the i-th robot; Indicates the current state If the i-th heterogeneous robot chooses The reward for the i-th heterogeneous robot remains unchanged, while the reward for the other heterogeneous robots remains unchanged. S333, to maximize the current state of all heterogeneous robots. The Q-value is used as the objective function, specifically in the form of: ； in, Indicates the state of heterogeneous robot i. Execute the optimal joint action Q-value; optimal joint action Let represent the joint action of three heterogeneous robots at time step k, and , , , These represent the actions of one flying measurement robot and two mobile measurement robots at time step k, respectively. S334, Combine the state space Joint Action Space and transition probability function As input to the main network of each heterogeneous robot, the main network solves for the optimal joint action by simultaneously solving the constraints and the objective function. Then, based on the optimal joint action... Obtain the optimal action for each heterogeneous robot.

6. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 5, characterized in that, S34 specifically includes the following steps: S341. Each heterogeneous robot executes its optimal action, triggering an update of the environmental state. S342. Calculate the dynamic region priority reward of heterogeneous robot i, using the following formula: ； in, Indicates dynamic region priority reward; This represents the base priority value of the nth task region; This represents the distance between the position of heterogeneous robot i and the task area n; This indicates the duration during which task region n remains unprocessed. Indicates the time decay coefficient; This indicates the avoidance of division by zero error when the distance is zero, i.e., division by zero error one. S343. Calculate the distance penalty reward for heterogeneous robot i, using the following formula: ； in, Indicates distance as a penalty or reward; This represents the efficiency coefficient of the i-th heterogeneous robot in solving the task; S344. Calculate the collaborative reward of heterogeneous robot i in a multi-heterogeneous robot group. The calculation formula is as follows: ； in, This indicates a collaborative reward among a group of heterogeneous robots. The cooperative factor representing heterogeneous robots i and j; This represents the collaboration efficiency of heterogeneous robots i and j in the current task; This represents the distance between heterogeneous robots i and j at time step k; This indicates the avoidance of division by zero error when the distance is zero, i.e., division by zero error two; S345. Based on the relative sharing of the dynamic region priority reward, distance penalty reward, and multi-heterogeneous robot group collaborative reward of heterogeneous robot i, normalization calculation is performed to obtain the weight factors of the three rewards of heterogeneous robot i, and the calculation formula is as follows: ； ； ； in, , , Let represent the weight factors of the dynamic region priority reward, distance penalty reward, and multi-heterogeneous robot group collaborative reward, respectively, and satisfy the following relationship: ； S346. Based on the dynamic region priority reward, distance penalty reward, multi-heterogeneous robot group cooperation reward, and their respective weight factors, dynamically calculate the comprehensive global reward of heterogeneous robot i. The calculation formula is as follows: ； in, This represents the overall global reward of heterogeneous robot i, that is, the overall global reward corresponding to the completion of the action of heterogeneous robot i; S347, repeat S342 to S346 until the combined global reward of all heterogeneous robots is obtained.

7. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 6, characterized in that, The formula for calculating the Q value of the optimal action in S37 is: ； in, Indicates the first The Q-value of the best action in each training sample; This represents the comprehensive global reward of heterogeneous robot i at time step k; This represents the maximum Q-value predicted by the target network for the next state; This indicates that the aircraft skin measurement mission has been completed; This indicates that the aircraft skin measurement mission has not been completed.

8. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 7, characterized in that, The specific formula for calculating the mean squared error loss (MSE) in S38 is as follows: ； in, This indicates the current output of the main network; Indicates the first The current state in each training sample; Indicates the first Actions in each training sample.

9. A task allocation system for collaborative measurement of aircraft skin by multiple heterogeneous robots, characterized in that, It includes multiple heterogeneous robots, which are configured to perform the task allocation method according to any one of claims 1 to 8.

Images