Task allocation method and system for multi-heterogeneous robot cooperative measurement of aircraft skin

Abstract

The invention relates to the field of task allocation, in particular to a task allocation method and system for multi-heterogeneous robot cooperative measurement of aircraft skin, and the method comprises the steps: 1, defining an environment and an intelligent agent; 2, distributing an initial state for the multi-heterogeneous robot; 3, solving an optimal action by the multi-heterogeneous robot according to an action selection strategy and a linear programming model, then calculating a comprehensive global reward, updating the experience playback pool after information sharing, constructing a training data set by utilizing the updated experience playback pool, and then training a main network and a target network by utilizing the training data set; and 4, migrating the trained main network and the target network to an actual aircraft skin measurement scene, and taking the final output of the trained main network as a task allocation strategy. According to the method, the problem of low task allocation efficiency in a scene that multiple heterogeneous robots cooperatively measure aircraft skin data is solved, information sharing among the robots is realized, scene change is dynamically adapted, and the operation efficiency of a multi-heterogeneous robot team is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-heterogeneous robot task allocation, and in particular to a task allocation method and system for collaborative multi-heterogeneous robot measurement of aircraft skin. Background Art

[0002] Aircraft skin is a critical component of the aircraft structure. Data such as its surface profile, form error, and thickness can reveal localized dents, deformation, seam gaps, and misalignment, enabling timely detection of potential structural damage and fatigue issues. Therefore, aircraft surface data is crucial for ensuring safe, efficient, and reliable aircraft operation, requiring efficient and accurate measurement technology. However, relying solely on a single type of robot (such as a flying measurement robot or a mobile measurement robot alone) is insufficient to comprehensively cover all areas of an aircraft skin. Therefore, the collaborative operation of multiple heterogeneous robots (flying measurement robots and mobile measurement robots) and the rational allocation of tasks are required to improve inspection efficiency and accuracy.

[0003] Different types of robots have their own advantages and limitations, requiring task allocation to optimize measurement coverage. Flying measurement robots can quickly scan large areas such as the top of the fuselage and wings, but they struggle to enter narrow spaces or get close to the skin for high-precision inspections. Mobile measurement robots can get close to the fuselage and use sensors such as lidar and ultrasound to perform detailed measurements of key areas such as rivets and joints, but their range of motion is limited, making it difficult to cover high altitudes and complex curved surfaces. At the same time, without proper task allocation, multiple robots may repeatedly measure the same area, resulting in a waste of time and computing resources. Therefore, task allocation is necessary to rationally plan the measurement range of each robot and improve overall measurement efficiency.

[0004] At the same time, existing algorithms have limitations in addressing the low efficiency of task allocation in scenarios involving collaborative measurement of aircraft skin data by multiple heterogeneous robots. Traditional reinforcement learning-based algorithms (such as Q-Learning) rely on iteratively optimizing task allocation strategies by constructing state-action value tables. However, because the state space expands exponentially with the number of robots and task complexity, traditional algorithms struggle to address the real-time task allocation requirements of large-scale, multi-task scenarios. Task allocation algorithms based on heuristic search (such as genetic algorithms) iteratively optimize task allocation schemes by simulating natural evolutionary processes. While they can achieve suboptimal solutions within a certain timeframe, heuristic algorithms typically require extensive parameter tuning and have poor adaptability to complex dynamic environments. Static optimization algorithms based on deep learning (such as deep neural network classifiers) rely on extensive offline training data for model learning. While they can adapt to specific task allocation patterns during the training phase, they lack adaptability to dynamic task changes and struggle to address the real-time nature of measurement task allocation.

[0005] However, the traditional DQN (Deep Q_Learning) algorithm still faces the following challenges when applied to the task of collaboratively measuring aircraft surface data by multiple heterogeneous robots: on the one hand, the efficiency of task allocation is easily limited by robot communication and multi-objective optimization, and a single reward mechanism cannot fully consider key factors such as priority, collaboration, and distance; on the other hand, the traditional DQN algorithm has shortcomings in convergence speed and global search capabilities, and may fall into local optimal solutions in dynamic environments. Summary of the Invention

[0006] The present invention provides a task allocation method and system for collaboratively measuring aircraft skin by multiple heterogeneous robots, so as to solve the technical problems mentioned in the background technology.

[0007] To achieve the above object, the technical solution of the present invention is achieved as follows: The present invention provides a task allocation method for collaboratively measuring aircraft skin by multiple heterogeneous robots, comprising the following steps: S1. Define the task allocation scenario, the state space and actions of each heterogeneous robot, and initialize the main network and target network corresponding to each heterogeneous robot; S2, assigning an initial state to each heterogeneous robot in the state space; S3. All heterogeneous robots use CE equilibrium as a condition and solve the optimal action based on the action selection strategy and linear programming model. Then, they calculate the comprehensive global reward of the optimal action. The comprehensive global reward, state, and action are stored in the experience replay pool after the multi-heterogeneous robot information is shared. The updated experience replay pool is used to construct a training dataset, and then the training dataset is used to train the main network and the target network. S4. Migrate the trained main network and target network to the actual aircraft skin measurement scenario, and use the final output of the trained main network as the task allocation strategy.

[0008] Furthermore, the S1 specifically includes the following steps: S11. Define a task allocation scenario for collaborative measurement of aircraft skin by multiple heterogeneous robots. The task allocation scenario includes three heterogeneous robots: one flying measurement robot and two mobile measurement robots. S12. Define the state space of each heterogeneous robot , robot position , task distribution ,in , Indicates the task location; Represents the task-based priority; state space That is, the set of spatial positions where each heterogeneous robot starts its action; S13. Define the actions of the flying measurement robot ; Action of the flying measurement robot To measure or move in any of the following directions: north, west, south, east, or vertically upward or downward; Define the actions of two mobile measurement robots 、 ; Movement of two mobile measurement robots 、 All are measurements, or the sensor moves forward, backward, or raises or lowers the sensor; S14. Select and initialize the main network for each multi-heterogeneous robot and target network ; S15. Define learning parameters, including discount factors , learning rate , exploration rate ; Then initialize the experience replay pool D of each heterogeneous robot.

[0009] Furthermore, the S2 specifically includes the following steps: S21. First, determine the state space of each heterogeneous robot scope; S22, in the state space Assign an initial state to each corresponding heterogeneous robot , ensuring that the initial state satisfies the measurement task constraints and aircraft skin scenario restrictions.

[0010] Furthermore, the S3 specifically includes the following steps: S31. Based on the information interaction mechanism, multiple heterogeneous robots interact with each other to achieve information sharing; S32. All heterogeneous robots take CE equilibrium as a condition and make optimal action selection according to the action selection strategy. CE equilibrium means that there is a "coordinator" who recommends actions for all heterogeneous robots, so that when other heterogeneous robots follow the recommended actions, any heterogeneous robot that deviates from the recommended action alone will not obtain higher expected benefits. If the optimal joint action is selected , then enter S33. If the random joint action is selected, the optimal action of all heterogeneous robots is considered to be a random action, and enter S34; S33, using the three established probabilistic conditions as constraints, the constraints are linear programming models, to maximize the current state of all heterogeneous robots The Q value under is used as the objective function to solve the optimal joint action of multiple heterogeneous robots ; Q value is the index value for evaluating the value of action; then according to the optimal joint action Obtain the optimal action for each heterogeneous robot; S34. Each heterogeneous robot executes the optimal action and dynamically calculates the comprehensive global reward corresponding to the action execution based on three key factors: dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward; S35. Get the next state based on the comprehensive global reward , and judge whether the aircraft skin measurement task is completed. If so, the current state , next state ,action , the comprehensive global reward is stored in the corresponding experience replay pool D to achieve the update of the experience replay pool D and enter S36, otherwise, the task allocation strategy is directly output; S36. Randomly sample B training samples from the experience replay pool D and assign indexes to the B training samples respectively; S37. Select one training sample from the B training samples and input it into the target network. Calculate the Q value of the optimal action of each target network in the next state evaluation according to the Bellman equation. S38. For each heterogeneous robot, the mean square error loss MSE of the heterogeneous robot is calculated using the Q value of the corresponding optimal action and the output of the current main network, and S36 to S38 are looped until the set number of loops is reached, and the main network parameters are updated by minimizing the mean square error loss MSE; at the same time, in each loop, the initialized counter is increased by 1, and when the preset interval step number N is reached, the parameters of the main network are completely copied to the corresponding target network, so that the parameters of the corresponding target network are completely consistent with the current main network.

[0011] Furthermore, the action selection strategy in S32 is the optimal joint action and the randomly selected action selection method, expressed as: ; in, represents the action selected by the action selection strategy; Indicates the selection probability of the current action; In the above formula, the exploration rate The following relationship is satisfied: ; Where k represents the kth time step; is the decay rate; is the maximum exploration probability; is the minimum exploration probability; e represents a natural constant.

[0012] Furthermore, the S33 specifically includes the following steps: S331, through the state space of each heterogeneous robot Constructing a joint state space , ,in, Represent the state spaces of a flying measurement robot and two mobile measurement robots respectively; The actions of each heterogeneous robot Construct the corresponding individual action space , and then construct the joint action space based on the individual action space of each heterogeneous robot , ,in These are the individual action spaces of the flying measurement robot and the two mobile measurement robots; Establish a transition probability function for each heterogeneous robot , transition probability function Refers to heterogeneous robots taking action , from the state To status The probability function of S332. Establish three probabilistic conditions for heterogeneous robots as constraints. The three probabilistic conditions are as follows: The first probabilistic condition is that the sum of all action probabilities of each heterogeneous robot i is 1. The specific form is: ; in, Select an action for heterogeneous robot i at time step k The probability of ; m represents the total number of heterogeneous robots; It is a universal quantifier in mathematics, meaning "for all"; The second probabilistic condition is that the probability of each heterogeneous robot i choosing an action must be non-negative and conform to the probability distribution of the actions. The probability distribution of the actions specifically refers to the seven possible actions of the flying measurement robot and the five possible actions of the mobile measurement robot. The seven possible actions are: the flying measurement robot taking a measurement; moving laterally north, west, south, or east; or moving up or down; and the five possible actions are: the mobile measurement robot taking a measurement; moving forward or backward; and raising or lowering the sensor. The specific form of the second probabilistic condition is: ; The third probabilistic condition: ensure that the current action of each heterogeneous robot i is better than other actions. The specific form is: ; in, The remaining actions of other heterogeneous robots; represents another candidate action of the i-th robot; Indicates the current state , if the i-th heterogeneous robot chooses , the other heterogeneous robots remain unchanged, the reward of the i-th heterogeneous robot; S333, to maximize the current state of all heterogeneous robots The Q value under is used as the objective function, and the specific form is: ; in, Represents heterogeneous robot i in state Perform optimal joint actions Q value; optimal joint action represents the joint action of the three heterogeneous robots at time step k, and , They represent the actions of a flying measurement robot and two mobile measurement robots at time step k respectively; S334, the joint state space , joint action space and the transition probability function As the input of each heterogeneous robot's main network, the main network solves the optimal joint action by combining constraints and objective functions. ; Then according to the optimal joint action Obtain the optimal action for each heterogeneous robot.

[0013] Furthermore, the S34 specifically includes the following steps: S341. Each heterogeneous robot executes the optimal action obtained by itself, triggering the update of the environment state; S342. Calculate the dynamic regional priority reward of heterogeneous robot i. The calculation formula is: ; in, represents the dynamic regional priority reward; Indicates the basic priority value of the nth task area; represents the distance between the position of heterogeneous robot i and the task area n; Indicates the duration that task area n is not processed; represents the time attenuation coefficient; It means to avoid the division error when the distance is zero, that is, the division error is one; S343. Calculate the distance penalty reward of heterogeneous robot i. The calculation formula is: ; in, Indicates distance penalty reward; represents the efficiency coefficient of the i-th heterogeneous robot in solving the task; S344. Calculate the multi-heterogeneous robot group collaboration reward of heterogeneous robot i. The calculation formula is: ; in, Represents the collaborative reward of multiple heterogeneous robots; represents the synergy factor between heterogeneous robots i and j; represents the collaboration efficiency of heterogeneous robots i and j in the current task; represents the distance between heterogeneous robots i and j at time step k; It means avoiding the division error of distance zero, that is, the division error of two; S345. Based on the relative sharing degree of the dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward of heterogeneous robot i, a normalized calculation is performed to obtain the weight factors of the three rewards of heterogeneous robot i. The calculation formula is: ; ; ; in, 、 、 They represent the weight factors of dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward, and satisfy the following relationship: ; S346. Based on the dynamic regional priority reward, distance penalty reward, multi-heterogeneous robot group collaboration reward, and their respective weight factors, the comprehensive global reward of heterogeneous robot i is dynamically calculated. The calculation formula is: ; in, represents the comprehensive global reward of heterogeneous robot i, that is, the comprehensive global reward corresponding to the execution of the action of heterogeneous robot i; S347. Loop S342 to S346 until the comprehensive global reward of all heterogeneous robots is obtained.

[0014] Furthermore, the calculation formula of the Q value of the optimal action in S37 is: ; in, Indicates the The Q value of the optimal action in the training samples; represents the comprehensive global reward of heterogeneous robot i at time step k; Indicates the maximum Q value of the next state predicted by the target network; Indicates that the aircraft skin measurement task is completed; Indicates that the aircraft skin measurement task is not completed.

[0015] Furthermore, the calculation formula of the mean square error loss MSE in S38 is as follows: ; in, Represents the output of the current main network; Indicates the The current state in the training samples; Indicates the The actions in the training samples.

[0016] On the other hand, the present invention also provides a task allocation system for collaboratively measuring aircraft skin by multiple heterogeneous robots, including multiple heterogeneous robots, which are configured or execute the above task allocation method.

[0017] Beneficial effects of the present invention: 1. The present invention provides a task allocation method for collaboratively measuring aircraft skins using multiple heterogeneous robots. The multiple heterogeneous robots can share information, improving the overall efficiency of task allocation. This invention introduces an information exchange mechanism that enables multiple heterogeneous robots to share their status information and network parameters in real time, overcoming the limitations of single-robot task decision-making. Through information sharing, the multi-robot team can fully understand the global status of the task, thereby optimizing task allocation, avoiding task conflicts and duplication when measuring aircraft skin data, and improving overall task execution efficiency.

[0018] 2. The present invention optimizes multi-objective task allocation by integrating comprehensive global rewards; By introducing a fused, integrated global reward, this paper incorporates three factors into the action selection within the task allocation strategy: the task's dynamic regional priority reward, the robot's distance penalty reward, and the multi-heterogeneous robot group collaboration reward. This mechanism not only improves the response efficiency of high-priority tasks, but also reduces the energy consumption of multi-heterogeneous robots and the time cost of task allocation. It also encourages collaboration among heterogeneous robots, further enhancing the global optimization effect of task allocation.

[0019] 3. The present invention accelerates the convergence speed of task allocation; Traditional reinforcement learning methods often suffer from slow convergence when used with large, heterogeneous robot teams due to the large state space. This paper improves on the DQN approach by introducing an action selection strategy that strikes a balance between exploration and exploitation for multiple heterogeneous robots. This approach not only allows for optimal execution of tasks using the most optimal actions, but also allows for exploration of the task area through randomized actions. For complex components like aircraft skins, this exploration strategy can help multiple heterogeneous robots discover potential key areas.

[0020] 4. The present invention utilizes CE balance to take into account the capability and functional differences (such as sensor type, coverage, measurement accuracy, etc.) among heterogeneous robots in task allocation, and uses a linear programming model to help find the stable behavior of each heterogeneous robot, thereby improving the collaborative efficiency of the heterogeneous robot team. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of the task allocation method of the present invention; Figure 2 This is a flowchart of S3 in the task allocation method of the present invention; Figure 3 This is a diagram showing the operating principle of S3 in the task allocation method of the present invention; Figure 4 It is a structural diagram of the task allocation system in the present invention. DETAILED DESCRIPTION

[0022] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The accompanying drawings illustrate preferred embodiments of the present invention. However, the present invention may be implemented in many other forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present disclosure.

[0023] Reference Figure 1 The embodiment of the present application provides a task allocation method for collaboratively measuring aircraft skin by multiple heterogeneous robots, comprising the following steps: S1. Define the task allocation scenario, the state space and actions of each heterogeneous robot, and initialize the main network and target network corresponding to each heterogeneous robot; S2, assigning an initial state to each heterogeneous robot in the state space; S3. All heterogeneous robots use CE equilibrium as a condition and solve the optimal action based on the action selection strategy and linear programming model. Then, they calculate the comprehensive global reward of the optimal action. The comprehensive global reward, state, and action are stored in the experience replay pool after the multi-heterogeneous robot information is shared. The updated experience replay pool is used to construct a training dataset, and then the training dataset is used to train the main network and the target network. S4. Migrate the trained main network and target network to the actual aircraft skin measurement scene, and use the trained main network to output the final As a task allocation strategy.

[0024] The present invention proposes a solution to the problem of low task allocation efficiency in the scenario of multi-heterogeneous robots collaboratively measuring aircraft skin data, realizes information sharing among multi-heterogeneous robots, and improves the working efficiency of the multi-heterogeneous robot team. Based on the DQN method, the present invention introduces CE equilibrium (correlated equilibrium), action selection strategy, and comprehensive global reward to achieve efficient task allocation in the environment of multi-heterogeneous robots collaboratively measuring aircraft skin surface data.

[0025] In some embodiments, the step S1 specifically includes the following steps: S11. Define a task allocation scenario for collaborative measurement of aircraft skin by multiple heterogeneous robots. The task allocation scenario includes three heterogeneous robots: one flying measurement robot and two mobile measurement robots. S12. Define the state space of each heterogeneous robot , robot position , task distribution ,in , Indicates the task location; Represents the task-based priority; state space That is, the spatial location set of each heterogeneous robot's starting action; each heterogeneous robot has its own corresponding state space, which does not store the state but is only a mathematical description of the robot's state. The actual storage structure is the experience replay pool. The experience replay pool D is used to store transfer samples and does not store the initial state separately; the initial state is generated by environment initialization and is implicit in the first transfer sample; S13. Define the actions of the flying measurement robot ; Action of the flying measurement robot To measure or move in any of the following directions: north, west, south, east, or vertically upward or downward; Define the actions of two mobile measurement robots 、 ; Movement of two mobile measurement robots 、 All are measurements, or the sensor moves forward, backward, or raises or lowers the sensor; S14. Select and initialize the main network for each multi-heterogeneous robot and target network ; The specific expression is as follows: ; ; S15. Define learning parameters, including discount factors , learning rate , exploration rate ; Among them, the discount factor Used to control the importance of future rewards; learning rate Used to adjust the step size of each parameter update; exploration rate It is used to achieve a balance between the control exploration strategy (random action) and the utilization (selection of the optimal joint action); then the experience replay pool D of each heterogeneous robot is initialized. The experience replay pool D is used to store the state transition samples of the interaction, and each sample can be expressed as .

[0026] In some embodiments, the step S2 specifically includes the following steps: S21. First, determine the state space of each heterogeneous robot scope; S22, in the state space Assign an initial state to each corresponding heterogeneous robot , expressed as , the position coordinates in brackets are the initial state; ensure that the initial state meets the measurement task constraints and aircraft skin scene restrictions.

[0027] In some embodiments, reference Figure 2 and Figure 3 , the S3 specifically includes the following steps: S31. Based on the information interaction mechanism, multiple heterogeneous robots interact with each other to achieve information sharing; First, heterogeneous robot i (transmitter) broadcasts its current state , the network parameters of the current main network and target network and the currently executing action to other heterogeneous robots within the communication range. Then, other heterogeneous robots (receivers) update their state information about heterogeneous robot i. and The data stored in the receiver's experience return pool is represented as follows: ; This invention introduces an information exchange mechanism that enables multiple heterogeneous robots to share their status information and network parameters in real time, overcoming the limitations of single-robot task decision-making. Through information sharing, the multi-robot team can fully understand the global status of the task, thereby optimizing task allocation, avoiding task conflicts and duplication when measuring aircraft skin data, and improving overall task execution efficiency.

[0028] S32. All heterogeneous robots select the optimal action based on the action selection strategy, subject to the CE (correlated equilibrium) equilibrium. CE equilibrium means that a "coordinator" recommends actions for all heterogeneous robots, ensuring that when all other heterogeneous robots follow the recommended actions, any heterogeneous robot that deviates from the recommended action will not achieve a higher expected return. In task allocation among multiple heterogeneous robots, each robot may have different capabilities, efficiency, and other factors. CE equilibrium accounts for these differences, making task allocation more efficient. If the optimal joint action is selected , then enter S33. If random joint action is selected, the optimal action of all heterogeneous robots is identified as random action. Random action is the exploration strategy. The exploration strategy enables heterogeneous robots to perform random actions in the environment according to the initial state, and record the environmental feedback in the initial state to provide reference data for subsequent optimization, and at the same time realize the exploration of unknown areas.

[0029] Optimal joint actions for multiple heterogeneous robots under high probability conditions In the process of measuring aircraft skin, in most cases, multiple heterogeneous robots take joint actions that can maximize the measurement efficiency. Evaluation calculations are performed to ensure maximum efficiency in task allocation. Random actions are adopted for multi-heterogeneous robots under low-probability conditions. When the heterogeneous robots first enter a new skin measurement area, they use certain exploration behaviors to obtain new aircraft skin measurement data.

[0030] The present invention introduces an action selection strategy to strike a balance between exploration and utilization of multi-heterogeneous robots. It can not only use the optimal action to perform tasks, but also explore the task area through random actions. For complex components such as aircraft skins, the exploration strategy can also help multi-heterogeneous robots discover potential key areas.

[0031] In addition, the present invention utilizes CE balance to take into account the capability and functional differences (such as sensor type, coverage range, measurement accuracy, etc.) among multiple heterogeneous robots in task allocation, and uses a linear programming model to help find the stable behavior of each heterogeneous robot, thereby improving the collaborative efficiency of the multi-heterogeneous robot team.

[0032] S33, using the three established probabilistic conditions as constraints, the constraints are linear programming models, to maximize the current state of all heterogeneous robots The Q value under is used as the objective function to solve the optimal joint action of multiple heterogeneous robots ; Q value is the index value for evaluating the value of action; then according to the optimal joint action Obtain the optimal action for each heterogeneous robot; S34. Each heterogeneous robot executes the optimal action and dynamically calculates the comprehensive global reward corresponding to the action execution based on three key factors: dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward; This paper introduces a fusion of comprehensive global rewards. It incorporates three factors: the dynamic regional priority reward for tasks, the robot's distance penalty reward, and the multi-heterogeneous robot group collaboration reward, into the action selection process within the task allocation strategy. This mechanism not only improves the response efficiency of high-priority tasks, but also reduces the energy consumption of multi-heterogeneous robots and the time cost of task allocation. It also encourages collaboration among heterogeneous robots, further enhancing the global optimization of task allocation.

[0033] S35. Get the next state based on the comprehensive global reward , and judge whether the aircraft skin measurement task is completed. If it is (done=True), the current state , next state ,action , the comprehensive global reward is deposited into the corresponding experience replay pool D, which will Store it in the experience replay pool D to update the experience replay pool D and enter S36. Otherwise (done=False), directly output the task allocation strategy; S36. Randomly sample B training samples from the experience replay pool D and assign indexes to the B training samples respectively; the mini-batch samples are represented as: ; The samples sampled simultaneously include interaction data of multiple heterogeneous robots in different states, ensuring the diversity of training data; S37. Select one training sample from the B training samples and input it into the target network. Calculate the Q value of the optimal action of each target network in the next state evaluation according to the Bellman equation. S38. For each heterogeneous robot, the mean square error loss (MSE) of the heterogeneous robot is calculated using the Q value of the corresponding optimal action and the output of the current main network. S36 to S38 are repeated until the set number of cycles is reached. The main network parameters are updated by minimizing the mean square error loss (MSE). The expression is: ; in, Represents the learning rate, which is used to control the step size of parameter update. ; represents the discount factor used to control the importance of future rewards; Represents the learning rate of the target network; At the same time, in each cycle, the initialized counter increases by 1. When the preset interval step number N (preferably 10,000 steps) is reached, the parameters of the main network are completely copied to the corresponding target network, so that the parameters of the corresponding target network are completely consistent with the current main network.

[0034] In some embodiments, the action selection strategy in S32 is the optimal joint action and the randomly selected action selection method, expressed as: ; in, Indicates the action selected by the action selection policy; Indicates the selection probability of the current action; In the above formula, the exploration rate The following relationship is satisfied: ; Where k represents the kth time step; is the attenuation rate; the attenuation rate is preferably 0.0001; is the maximum exploration probability, and the maximum exploration probability is set to 1; is the minimum exploration probability, which is preferably 0.01; e represents a natural constant.

[0035] In some embodiments, reference Figure 2 and Figure 3 , the S33 specifically includes the following steps: S331, through the state space of each heterogeneous robot Constructing a joint state space , ,in, Represent the state spaces of a flying measurement robot and two mobile measurement robots respectively; The actions of each heterogeneous robot Construct the corresponding individual action space , and then construct the joint action space based on the individual action space of each heterogeneous robot , ,in These are the individual action spaces of the flying measurement robot and the two mobile measurement robots; Establish a transition probability function for each heterogeneous robot , transition probability function Refers to heterogeneous robots taking action , from the state To status The probability function of S332. Establish three probabilistic conditions for heterogeneous robots as constraints. The three probabilistic conditions are as follows: The first probabilistic condition is that the sum of all action probabilities of each heterogeneous robot i is 1. The specific form is: ; in, Select an action for heterogeneous robot i at time step k The probability of ; m represents the total number of heterogeneous robots; It is a universal quantifier in mathematics, meaning "for all"; The second probabilistic condition is that the probability of each heterogeneous robot i choosing an action must be non-negative and conform to the probability distribution of the actions. The probability distribution of the actions specifically refers to the seven possible actions of the flying measurement robot and the five possible actions of the mobile measurement robot. The seven possible actions are: the flying measurement robot taking a measurement; moving laterally north, west, south, or east; or moving up or down; and the five possible actions are: the mobile measurement robot taking a measurement; moving forward or backward; and raising or lowering the sensor. The specific form of the second probabilistic condition is: ; The third probabilistic condition: ensure that the current action of each heterogeneous robot i is better than other actions. The specific form is: ; in, The remaining actions of other heterogeneous robots; represents another candidate action of the i-th heterogeneous robot, indicating that the heterogeneous robot may try to deviate from another action; Indicates the current state , if the i-th heterogeneous robot chooses , the other robots remain unchanged, the reward of the i-th robot; S333, to maximize the current state of all heterogeneous robots The Q value under is used as the objective function, and the specific form is: ; in, Represents heterogeneous robot i in state Perform optimal joint actions Q value; optimal joint action represents the joint action of the three heterogeneous robots at time step k, and , They represent the actions of a flying measurement robot and two mobile measurement robots at time step k respectively; S334, the joint state space , joint action space and the transition probability function As the input of each heterogeneous robot's main network, the main network solves the optimal joint action by combining constraints and objective functions. ; Then according to the optimal joint action Obtain the optimal action for each heterogeneous robot.

[0036] In some embodiments, reference Figure 2 and Figure 3 , the S34 specifically includes the following steps: S341. Each heterogeneous robot executes the optimal action obtained by itself, triggering the update of the environment state; S342. Calculate the dynamic regional priority reward of heterogeneous robot i. The calculation formula is: ; in, represents the dynamic regional priority reward; Indicates the basic priority value of the nth mission area (preset based on the aircraft skin measurement scenario); represents the distance between the position of heterogeneous robot i and the task area n; Indicates the duration that task area n is not processed; represents the time decay coefficient (the sensitivity of the task area priority to change over time, set to 200, that is, the rate of change of the task area priority over time is moderate); Indicates avoiding the zero division error when the distance is zero (take 10 -6 ), that is, the division by zero error is one; In the above formula, The calculation formula is as follows: ; in, 、 、 They represent the x-, y-, and z-axis coordinates of heterogeneous robot i at time step k; 、 、 They represent the x-, y-, and z-axis coordinates of the heterogeneous robot n at time step k; S343. Calculate the distance penalty reward of heterogeneous robot i. The calculation formula is: ; in, Indicates distance penalty reward; represents the efficiency coefficient of the i-th heterogeneous robot in solving the task; S344. Calculate the multi-heterogeneous robot group collaboration reward of heterogeneous robot i. The calculation formula is: ; in, Represents the collaborative reward of multiple heterogeneous robots; represents the synergy factor between heterogeneous robots i and j; preferably, the synergy factor between the flying measurement robot and the mobile measurement robot is: , , the flying measurement robot can first scan a large area while the mobile measurement robot can perform local fine measurement; the synergy factor of the two mobile measurement robots is: ,The two mobile measurement robots can cover different angles from each other, but have similar ,measurement capabilities; represents the collaboration efficiency (such as task completion rate and communication quality) between heterogeneous robots i and j in the current task; represents the distance between heterogeneous robots i and j at time step k; Indicates avoiding the zero division error (take 10 -6 ), that is, the division by zero error of two; S345. Based on the relative sharing degree of the dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward of heterogeneous robot i, a normalized calculation is performed to obtain the weight factors of the three rewards of heterogeneous robot i. The calculation formula is: ; ; ; in, 、 、 They represent the weight factors of dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward, and satisfy the following relationship: ; S346. Based on the dynamic regional priority reward, distance penalty reward, multi-heterogeneous robot group collaboration reward, and their respective weight factors, the comprehensive global reward of heterogeneous robot i is dynamically calculated. The calculation formula is: ; in, represents the comprehensive global reward of heterogeneous robot i, that is, the comprehensive global reward corresponding to the execution of the action of heterogeneous robot i; S347. Loop S342 to S346 until the comprehensive global reward of all heterogeneous robots is obtained.

[0037] In some embodiments, the calculation formula for the Q value of the optimal action in S37 is: ; in, Indicates the The Q value of the optimal action in the training samples; represents the comprehensive global reward of heterogeneous robot i at time step k; Indicates the maximum Q value of the next state predicted by the target network; Indicates that the aircraft skin measurement task is completed; Indicates that the aircraft skin measurement task is not completed.

[0038] In some embodiments, the calculation formula of the mean square error loss MSE in S38 is specifically as follows: ; in, Represents the output of the current main network; Indicates the The current state in the training samples; Indicates the The actions in the training samples.

[0039] Reference Figure 4 On the other hand, the present invention also provides a task allocation system for collaborative measurement of aircraft skin by multiple heterogeneous robots, including multiple heterogeneous robots, which are configured or execute the above task allocation method.

[0040] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots, characterized in that: The steps include: S1. Define the task allocation scenario, the state space and actions of each heterogeneous robot, and initialize the main network and target network corresponding to each heterogeneous robot; S2, assigning an initial state to each heterogeneous robot in the state space; S3. All heterogeneous robots use CE equilibrium as a condition and solve the optimal action based on the action selection strategy and linear programming model. Then, they calculate the comprehensive global reward of the optimal action. The comprehensive global reward, state, and action are stored in the experience replay pool after the multi-heterogeneous robot information is shared. The updated experience replay pool is used to construct a training dataset, and then the training dataset is used to train the main network and the target network. S4. Migrate the trained main network and target network to the actual aircraft skin measurement scenario, and use the final output of the trained main network as the task allocation strategy.

2. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 1 is characterized in that: The S1 specifically includes the following steps: S11. Define a task allocation scenario for collaborative measurement of aircraft skin by multiple heterogeneous robots. The task allocation scenario includes three heterogeneous robots: one flying measurement robot and two mobile measurement robots. S12. Define the state space of each heterogeneous robot , robot position , task distribution ,in , Indicates the task location; Represents the task-based priority; state space That is, the set of spatial positions where each heterogeneous robot starts its action; S13. Define the actions of the flying measurement robot ; Action of the flying measurement robot To measure or move in any of the following directions: north, west, south, east, or vertically upward or downward; Define the actions of two mobile measurement robots 、 ; Movement of two mobile measurement robots 、 All are measurements, or the sensor moves forward, backward, or raises or lowers the sensor; S14. Select and initialize the main network for each multi-heterogeneous robot and target network ; S15. Define learning parameters, including discount factors , learning rate , exploration rate ; Then initialize the experience replay pool D of each heterogeneous robot.

3. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 2 is characterized in that: The S2 specifically includes the following steps: S21. First, determine the state space of each heterogeneous robot scope; S22, in the state space Assign an initial state to each corresponding heterogeneous robot , ensuring that the initial state satisfies the measurement task constraints and aircraft skin scenario restrictions.

4. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 3 is characterized in that: The S3 specifically includes the following steps: S31. Based on the information interaction mechanism, multiple heterogeneous robots interact with each other to achieve information sharing; S32. All heterogeneous robots take CE equilibrium as the condition and make the optimal action selection according to the action selection strategy. CE equilibrium means that there is a "coordinator" who recommends actions for all heterogeneous robots, so that when other heterogeneous robots follow the recommended actions, any heterogeneous robot that deviates from the recommended action alone will not obtain a higher expected benefit. If the optimal joint action is selected , then enter S33. If the random joint action is selected, the optimal action of all heterogeneous robots is considered to be a random action, and enter S34; S33, using the three established probabilistic conditions as constraints, the constraints are linear programming models, to maximize the current state of all heterogeneous robots The Q value under is used as the objective function to solve the optimal joint action of multiple heterogeneous robots ; Q value is the index value for evaluating the value of action; then according to the optimal joint action Obtain the optimal action for each heterogeneous robot; S34. Each heterogeneous robot executes the optimal action and dynamically calculates the comprehensive global reward corresponding to the action execution based on three key factors: dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward; S35. Get the next state based on the comprehensive global reward , and judge whether the aircraft skin measurement task is completed. If so, the current state , next state ,action , the comprehensive global reward is stored in the corresponding experience replay pool D to achieve the update of the experience replay pool D and enter S36, otherwise, the task allocation strategy is directly output; S36. Randomly sample B training samples from the experience replay pool D and assign indexes to the B training samples respectively; S37. Select one training sample from the B training samples and input it into the target network. Calculate the Q value of the optimal action of each target network in the next state evaluation according to the Bellman equation. S38. For each heterogeneous robot, the mean square error loss MSE of the heterogeneous robot is calculated using the Q value of the corresponding optimal action and the output of the current main network, and S36 to S38 are looped until the set number of loops is reached, and the main network parameters are updated by minimizing the mean square error loss MSE; at the same time, in each loop, the initialized counter is increased by 1, and when the preset interval step number N is reached, the parameters of the main network are completely copied to the corresponding target network, so that the parameters of the corresponding target network are completely consistent with the current main network.

5. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 4 is characterized in that: The action selection strategy in S32 is the optimal joint action and the randomly selected action selection method, expressed as: ； in, represents the action selected by the action selection strategy; Indicates the selection probability of the current action; In the above formula, the exploration rate The following relationship is satisfied: ； Where k represents the kth time step; is the decay rate; is the maximum exploration probability; is the minimum exploration probability; e represents a natural constant.

6. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 5 is characterized in that: The S33 specifically includes the following steps: S331, through the state space of each heterogeneous robot Constructing a joint state space , ,in, Represent the state spaces of a flying measurement robot and two mobile measurement robots respectively; The actions of each heterogeneous robot Construct the corresponding individual action space , and then construct the joint action space based on the individual action space of each heterogeneous robot , ,in These are the individual action spaces of the flying measurement robot and the two mobile measurement robots; Establish a transition probability function for each heterogeneous robot , transition probability function Refers to heterogeneous robots taking action , from the state To status The probability function of S332. Establish three probabilistic conditions for heterogeneous robots as constraints. The three probabilistic conditions are as follows: The first probabilistic condition is that the sum of all action probabilities of each heterogeneous robot i is 1. The specific form is: ； in, Select an action for heterogeneous robot i at time step k The probability of ; m represents the total number of heterogeneous robots; It is a universal quantifier in mathematics, meaning "for all"; The second probabilistic condition is that the probability of each heterogeneous robot i choosing an action must be non-negative and conform to the probability distribution of the actions. The probability distribution of the actions specifically refers to the seven possible actions of the flying measurement robot and the five possible actions of the mobile measurement robot. The seven possible actions are: the flying measurement robot taking a measurement; moving laterally north, west, south, or east; or moving up or down; and the five possible actions are: the mobile measurement robot taking a measurement; moving forward or backward; and raising or lowering the sensor. The specific form of the second probabilistic condition is: ； The third probabilistic condition: ensure that the current action of each heterogeneous robot i is better than other actions. The specific form is: ； in, The remaining actions of other heterogeneous robots; represents another candidate action of the i-th robot; Indicates the current state , if the i-th heterogeneous robot chooses , the other heterogeneous robots remain unchanged, the reward of the i-th heterogeneous robot; S333, to maximize the current state of all heterogeneous robots The Q value under is used as the objective function, and the specific form is: ； in, Represents heterogeneous robot i in state Perform optimal joint actions Q value; optimal joint action represents the joint action of the three heterogeneous robots at time step k, and , They represent the actions of a flying measurement robot and two mobile measurement robots at time step k respectively; S334, the joint state space , joint action space and the transition probability function As the input of each heterogeneous robot's main network, the main network solves the optimal joint action by combining constraints and objective functions. ; Then according to the optimal joint action Obtain the optimal action for each heterogeneous robot.

7. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 6 is characterized in that: The S34 specifically includes the following steps: S341. Each heterogeneous robot executes the optimal action obtained by itself, triggering the update of the environment state; S342. Calculate the dynamic regional priority reward of heterogeneous robot i. The calculation formula is: ； in, represents the dynamic regional priority reward; Indicates the basic priority value of the nth task area; represents the distance between the position of heterogeneous robot i and the task area n; Indicates the duration that task area n is not processed; represents the time attenuation coefficient; It means to avoid the division error when the distance is zero, that is, the division error is one; S343. Calculate the distance penalty reward of heterogeneous robot i. The calculation formula is: ； in, Indicates distance penalty reward; represents the efficiency coefficient of the i-th heterogeneous robot in solving the task; S344. Calculate the multi-heterogeneous robot group collaboration reward of heterogeneous robot i. The calculation formula is: ； in, Represents the collaborative reward of multiple heterogeneous robots; represents the synergy factor between heterogeneous robots i and j; represents the collaboration efficiency of heterogeneous robots i and j in the current task; represents the distance between heterogeneous robots i and j at time step k; It means avoiding the division error of distance zero, that is, the division error of two; S345. Based on the relative sharing degree of the dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward of heterogeneous robot i, a normalized calculation is performed to obtain the weight factors of the three rewards of heterogeneous robot i. The calculation formula is: ； ； ； in, 、 、 They represent the weight factors of dynamic area priority reward, distance penalty reward, and multi-heterogeneous robot group collaboration reward, and satisfy the following relationship: ； S346. Based on the dynamic regional priority reward, distance penalty reward, multi-heterogeneous robot group collaboration reward, and their respective weight factors, the comprehensive global reward of heterogeneous robot i is dynamically calculated. The calculation formula is: ； in, represents the comprehensive global reward of heterogeneous robot i, that is, the comprehensive global reward corresponding to the execution of the action of heterogeneous robot i; S347. Loop S342 to S346 until the comprehensive global reward of all heterogeneous robots is obtained.

8. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 7 is characterized in that: The calculation formula of the Q value of the optimal action in S37 is: ； in, Indicates the The Q value of the optimal action in the training samples; represents the comprehensive global reward of heterogeneous robot i at time step k; Indicates the maximum Q value of the next state predicted by the target network; Indicates that the aircraft skin measurement task is completed; Indicates that the aircraft skin measurement task is not completed.

9. The task allocation method for collaborative measurement of aircraft skin by multiple heterogeneous robots according to claim 8 is characterized in that: The calculation formula of the mean square error loss MSE in S38 is as follows: ； in, Represents the output of the current main network; Indicates the The current state in the training samples; Indicates the The actions in the training samples.

10. A task allocation system for collaborative measurement of aircraft skin by multiple heterogeneous robots, characterized in that: The method comprises a plurality of heterogeneous robots, and the plurality of heterogeneous robots are configured to or execute the task allocation method according to any one of claims 1 to 9.

Images