Robot adaptive formation and obstacle avoidance control method and system based on multimode information fusion

Abstract

The invention discloses a robot adaptive formation and obstacle avoidance control method and system based on multimode information fusion. The robot adaptive formation and obstacle avoidance under the conditions of no global data and lack of map information are realized. In order to overcome global information constraints, a multi-source information fusion state space representation framework is constructed, multi-type information data under local perception of a robot are integrated, and robot formation control state representation suitable for an unknown environment is formed in a unified feature space; for a complex dynamic application scene, a self-adaptive formation and obstacle avoidance control network is provided, the model enhances the environment distribution cognitive ability of the robot by combining time sequence features and planning memory weights, and a flexible and safe optimal motion decision is efficiently generated. The self-adaptive formation and obstacle avoidance control of multiple mobile robots in a dynamic obstacle environment is realized; the multi-robot system in the method does not need to acquire prior information or deploy global sensing equipment, formation generation, maintenance and recovery tasks can be flexibly realized, and the flexibility and autonomy of distributed formation control in a complex dynamic environment are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile robot formation and obstacle avoidance, and in particular relates to a robot adaptive formation and obstacle avoidance control method and system based on multi-mode information fusion. Background Art

[0002] With the continuous development of human society, people's requirements for the functions of robots are getting higher and higher. Faced with complex task scenarios, the limitations of a single robot are becoming more and more obvious. People have begun to study the coordinated movement of multiple robots. Multi-robot collaborative systems are often built on a number of relatively simple robot platforms. These platforms work together through local collaboration to achieve overall collaborative behavior and form an efficient group system. Multi-robot collaborative work can adapt to more complex environments through mutual cooperation without significantly increasing the complexity of a single robot system, and give full play to individual advantages to complete more complex tasks with good overall cooperation. Multi-robot collaborative problems can be divided into consistency problems, collaborative communication, formation problems, coordination control problems and cooperative decision-making. Among them, formation control is a topic of widespread concern in the field of multi-robot collaborative work. Formation control refers to the control problem of multiple robots in the process of jointly completing tasks, which not only maintains the geometric formation constraints between formation members, but also adapts to the constraints brought by environmental changes, and performs external obstacle avoidance and internal collision avoidance. The purpose of formation control is to guide a team of intelligent agents to reach the target position with the desired formation with the help of surrounding intelligent agents and environmental information.

[0003] At present, there are five main formation control methods: behavior-based method, virtual structure method, distributed model predictive control, graph theory-based method and pilot-follow method. The pilot-follow method is more suitable for unknown environments because of its simple structure, strong autonomy and high flexibility and scalability. According to the way of obtaining information about the environment and adjacent mobile robots, the pilot-follow formation control method is divided into global information-based, vision-based and multi-sensor fusion-based. The pilot-follow formation control method based on global information has high stability, can achieve reliable formation management, and prevent collisions between robots from causing system crashes. However, when applying such methods, a unified coordinate framework that can fully characterize the structure of the entire formation must be constructed, which must include an accurate description of the position and posture of each single mobile robot, as well as the addition of necessary auxiliary anchor points. This makes such methods costly to implement and difficult to deploy, which limits their application scenarios and lacks flexibility. The vision-based pilot-follow formation control can break away from the constraints of global information, and has the advantages of actively obtaining the position and posture information of the leader and strong autonomy in formation. However, the existing multi-robot image information formation method relies on additional special markers to achieve formation maintenance and formation management. This is because in terms of formation management, formation members usually have the same image features, and it is difficult for followers to distinguish the pilot object based entirely on visual information. They need to rely on external labels to help divide the order of members in the formation. In terms of formation maintenance, label data is usually required to enable followers to perceive the position of the pilot and calculate motion decisions to maintain the relative positions of members. In terms of formation maintenance, in complex and unknown actual environments, factors such as obstacle occlusion and environmental noise will make it difficult for visual formation methods that rely on external labels to ensure formation stability. If the spacing between formation members is unstable, collisions between aircraft are likely to occur, causing the entire formation system to collapse. Based on this, this method will perceive the environment by introducing cameras, lidars, and ultra-wideband sensors, and fuse the perceived content. Finally, through a deep reinforcement learning algorithm, formation generation, formation obstacle avoidance, and formation maintenance in obstacle scenarios are achieved by relying on the fused content.

[0004] The current formation obstacle avoidance methods are mainly divided into traditional formation obstacle avoidance methods and deep reinforcement learning-based obstacle avoidance methods. Traditional formation obstacle avoidance methods mainly build a global positioning platform to obtain the position information of formation members for global planning and estimate the status of formation members for obstacle avoidance control. Traditional formation obstacle avoidance methods need to design the control rate for the model of the intelligent body and switch the fixed formation to pass the obstacle during the obstacle avoidance movement, which will greatly reduce the formation obstacle avoidance effect in complex and unknown environments beyond the pre-designed range. The formation obstacle avoidance method based on deep reinforcement learning is to fit the corresponding formation obstacle avoidance task through a deep neural network, which can alleviate the disadvantage of difficult task modeling in traditional formation obstacle avoidance methods. This method not only needs to maintain the formation during the task execution under static obstacle conditions, but also has fault tolerance in a dynamic obstacle environment. Therefore, a formation obstacle avoidance method based on deep reinforcement learning is adopted to ensure the stability and robustness of the formation system. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion, proposes a new algorithm framework and training method based on deep reinforcement learning, and uses multiple networks to share the pressure of fitting tasks, that is, multiple control networks jointly control the movement of multiple mobile robots, aiming to solve the problems of adaptive formation and obstacle avoidance of multiple mobile robots, and has the characteristics of high robustness and strong portability.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: a robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion, comprising the following steps:

[0007] Obtain images of the follower, surrounding radar data, and distance data;

[0008] Construct and train a deep reinforcement learning network, including a formation control network and an anti-tracking network. The formation control network is used to extract features of pre-processed fused data and output formation control actions based on the features of the fused data. The anti-tracking network is used to train the anti-tracking capability of the leader.

[0009] Based on the trained deep reinforcement learning network, the motion characteristics, distance characteristics and dynamic obstacle timing characteristics of the mobile robot in the preprocessed information are extracted, and the optimal formation motion decision that can be executed by the mobile robot is output;

[0010] According to the optimal formation motion decision of the mobile robots, the formation motion control instructions of the mobile robots are determined;

[0011] Execute formation motion control instructions to control the mobile robot formation and formation obstacle avoidance.

[0012] Furthermore, obtaining the image of the follower, the surrounding environment radar data and the distance data includes: the mobile robot's onboard camera obtains the image of the follower, and performs dimensionality reduction, compression and stacking processing on the image; the laser radar obtains the surrounding environment radar data, performs fixed-step sampling on the surrounding environment radar data, and obtains a set amount of data; and the distance data obtained by the UWB sensor is median filtered.

[0013] Furthermore, the fused data is used as the input of the formation control network and the anti-tracking network. The formation control network outputs the movement direction a of the mobile robot. ot and speed change a vt ,After the mobile robot performs the action, it receives new state information and ,reward information.

[0014] Furthermore, the formation control network includes an input layer, a CNN layer, an LSTM layer, a temporal attention layer and an output layer. The LSTM layer is responsible for modeling time series data and encoding the historical observation sequence into a vector representation of a fixed length. The temporal attention mechanism is used to weight different time steps in the LSTM output sequence, and perform value function estimation and action selection on the vector representation. The back-propagation algorithm is used for optimization to maximize the cumulative reward or minimize the difference between the Q-value estimate and the target Q-value.

[0015] Furthermore, the training of the formation control network and the anti-tracking network includes: Phase 1, randomly initializing the formation control network parameters, preprocessing the data obtained by each sensor and performing fusion encoding as the state variable, scoring each action with the value function obtained by the formation control network, selecting the next action according to the ε-greedy strategy and executing it, entering a new state, and completing the training if the termination condition is met, otherwise continue to execute, and (s f (t),a(t),r(t),s f (t+1)) The quaternion is stored in the follower's experience replay pool. When the follower's experience replay pool accumulates a set amount of experience, network training is started to update parameters;

[0016] In stage 2, the anti-tracking agent is initialized with the formation control network weights at the end of stage 1. At the beginning of training, the leader's experience replay pool has no samples. When the number of stored samples exceeds the set amount, the network is trained and the experience replay pool stores (s t ,a2 t ,r2 t ,s t+1 ), where s t and t+1 As the input of the anti-tracking network, calculate Q(s t ;ω) and Q(s t+1 ;ω) and select a2 t The corresponding Q(st ,a2 t ; ω), select the corresponding action according to the maximum Q value, and finally calculate the state action value function y2(t) and the loss function Loss2(ω) of the navigator agent at the current moment, and use the Adma optimizer to optimize the network parameters;

[0017] In stage 3, the training methods and optimizers of the anti-tracking network and formation control network remain unchanged. In addition to static obstacles, dynamic obstacles are added to the training scene. The trajectory of the dynamic obstacles is a square. The center of the square trajectory is the initial position of the follower in each round of training. The initial position of the leader is outside the rectangle. The follower needs to avoid dynamic obstacles in each round to reach the target and obtain a positive reward.

[0018] Furthermore, the network training for parameter updating includes:

[0019] Randomly sample a set (s) from the experience replay pool f (t),a1(t),r1(t),s f (t+1));

[0020] The formation control network is used to calculate the state value function Q(s(t); ω) and Q(s(t+1); ω), and the corresponding action value function value and are calculated based on the actual actions performed;

[0021] Based on the maximum value action strategy, select the action with the highest score and get the maximum action value function value max a(t+1) Q(s(t+1),a1(t+1);ω);

[0022] Using the discount factor γ and the immediate reward given by the environment, the state action value function value y1(t) at the moment can be calculated. The specific calculation method is:

[0023]

[0024] The loss function Loss1(ω) of the formation control network is:

[0025] Loss1(ω)=E[(Q((t),a1(t);ω)-y1(t)) 2 ]

[0026] The Adma optimizer is used to update the network parameters. When the training score reaches 80% of the maximum training score, the leader-follower formation control network has the initial ability to detect and approach the leader in a static obstacle environment.

[0027] Furthermore, the motion characteristics of the mobile robot include the contour characteristics, shape characteristics and position change characteristics of the mobile robot; the distance characteristics of the mobile robot include the distance characteristics between the mobile robot and all mobile robots in front; the dynamic obstacle change characteristics include the contour characteristics and position change characteristics of the dynamic obstacles.

[0028] Furthermore, the formation control network outputs a state-action value function, and its corresponding formation control action a t For: a R ,a F ,a L ; Among them, a R ,a F ,a L The directions represented are turn right, go straight, and turn left. The formation control actions are as follows:

[0029] a t =[a R a F a 3L ]

[0030] Adopting the action selection strategy, the optimal action a of the formation control action is selected according to the action evaluation estimated by the state action value function t .

[0031] In the second aspect, based on the concept of the method, the present invention also provides a robot adaptive formation and obstacle avoidance control system based on multi-mode information fusion, including: a data processing module, a feature extraction module, a control instruction output module and a control module; the data processing module is used to pre-process the image data, radar data and distance data acquired by the mobile robot; the feature extraction module is used to use the trained deep reinforcement learning network to extract the motion characteristics, distance characteristics and dynamic obstacle change characteristics of the mobile robot in the fusion data, and output the optimal formation motion decision of the mobile robot; the control instruction output module is used to determine the formation motion control instructions of the mobile robot according to the optimal formation motion decision of the mobile robot; the control module is used to control the formation of the mobile robot and the formation obstacle avoidance according to the formation motion control instructions.

[0032] In a third aspect, the present invention provides a robot formation that moves in formation based on the above-mentioned robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion.

[0033] Compared with the prior art, the present invention has at least the following beneficial effects: the present invention discloses a robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion, which realizes the robot adaptive formation and obstacle avoidance under the condition of no global data and missing map information; adopts vision, lidar and UWB sensors to obtain visual data, radar data and distance data respectively, and extracts the motion characteristics, distance characteristics and dynamic obstacle change characteristics of the mobile robot according to the fused data; directly generates control instructions such as the motion direction and motion speed of the mobile robot according to the perception results of the mobile robot, so as to realize the mobile robot formation and obstacle avoidance tasks in an environment without global information; the method combines the multi-control network collaborative control based on deep reinforcement learning with the long short-term memory network and the attention mechanism, so that the mobile robot can autonomously complete the robot adaptive formation and obstacle avoidance tasks in an environment without global information without human intervention, which greatly improves the autonomy and intelligence of the mobile robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the overall system block diagram of the robot adaptive formation and obstacle avoidance control method of the present invention.

[0035] Figure 2 This is a network structure block diagram based on deep reinforcement learning proposed in the present invention.

[0036] Figure 3 It is a flowchart of the robot adaptive formation and obstacle avoidance control method of the present invention.

[0037] Figure 4 It is a training protocol algorithm block diagram of stage one of the present invention.

[0038] Figure 5 It is a block diagram of the stage 2 training protocol algorithm of the present invention.

[0039] Figure 6 This is a data composition format of an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below in conjunction with the accompanying drawings, which are intended to explain the present invention rather than to limit it.

[0041] See also Figure 1 , Figure 2 and Figure 3 The present invention provides a robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion, comprising the following steps:

[0042] Step 1: The mobile robot's onboard camera obtains the image of the follower, the lidar obtains the surrounding environment radar data, and the UWB sensor obtains the distance data, and pre-processes them respectively.

[0043] Specifically, the image of the follower obtained is an image from the first-person perspective of the follower robot, and the preprocessing of the image includes grayscale processing and cropping processing of the image; the preprocessing of the radar data is to perform fixed-step sampling on the obtained distance data to obtain a specified amount of data; the distance data may have abnormal values ​​and cause the measured values ​​to deviate from the true values, so the preprocessing of the distance data is median filtering.

[0044] The input data of the formation control network are the fused follower image, radar data, distance data and the speed information of the leader. The input data of the formation control network are the preprocessed follower view image, radar data, distance data and the speed information of the leader. The designed state space mu-law method encodes the acquired sensor information to [0,255]. The specific data composition format is referenced Figure 6 .

[0045] Step 2: construct and train a deep reinforcement learning network, wherein the deep reinforcement learning network includes a formation control network and an anti-tracking network.

[0046] The fused data is used as the input of the formation control network and the anti-tracking network. The formation control network outputs the movement direction a and ot and speed change a vt ,Finally, after the mobile robot performs the action, it receives new ,state information and reward information.

[0047] The formation control network structure combines the long short-term memory network (LSTM) and the temporal attention mechanism to design an adaptive formation and obstacle avoidance control network. The formation control network includes an input layer, a CNN layer, an LSTM layer, an attention layer, and an output layer. The neural network architecture of the LSTM and the temporal attention mechanism is used to process reinforcement learning task samples with time series properties. The formation task in a dynamic environment requires the agent to fully learn the characteristics of adjacent mobile robots and environmental information under different time series, so as to avoid obstacles in the dynamic environment and maintain the stability of the formation. In the overall network structure, the LSTM is responsible for modeling the time series data and encoding the historical observation sequence into a vector representation of a fixed length. The temporal attention mechanism is used to weight the different time steps in the LSTM output sequence and pay attention to the information related to the current task. Finally, the value function is estimated and the action is selected for the vector representation. It is optimized by the back propagation algorithm to maximize the cumulative reward or minimize the difference between the Q value estimate and the target Q value.

[0048] Specifically, the CNN layer contains two layers of convolution. The convolution kernel size of the first convolution layer (Conv1) is 3*3, the stride is 2, and there are 8 convolution kernels in total. Its input is the multi-sensor fusion perception state of 40×40×4, and the output is 20×20×8. The second convolution layer has 16 convolution kernels of size 3*3 and stride 2. The input of this layer is the output of Conv1, and the output is 10×10×16. The activation function used is ReLu, which has the following characteristics: when the input value is greater than 0, the output is equal to the input; when the input value is less than or equal to 0, the output is 0. It can introduce nonlinear characteristics into the neural network, thereby enhancing its expressive power. When the input value is less than or equal to 0, the derivative of ReLU is 0, which means that during the back propagation process, the gradient of the neuron is 0, which can achieve gradient sparsity and reduce the amount of calculation for gradient update.

[0049] The specific training process of stage 1 is as follows Figure 4 As shown in the figure, the termination condition of each round in this stage is that the follower reaches the expected distance of the leader without collision, the reward score trend changes from negative to 0, or the follower collides and the maximum number of training steps of the round is reached. First, the formation control network parameters are randomly initialized. Then, the data obtained by each sensor is preprocessed and fused and encoded as a state variable, and the value function obtained by the formation control network is used to score each action. Then, according to the ε-greedy strategy, the next action is selected and executed to enter a new state. In this process, if the termination condition is met, the training is completed, otherwise it continues to execute. At the same time, (s f (t),a(t),r(t),s f (t+1)) quaternion is stored in the follower experience replay pool. When the follower experience replay pool accumulates a certain amount of experience, network training is started to update parameters. Figure 4 The done_flag in the figure is the end mark of the round. When the agent reaches the end condition of the round, the done_flag is set to 1 regardless of whether it is successful or not, and then enters the next round.

[0050] In the training phase, each time the network model is updated, a set of (s f (t),a1(t),r1(t),s f (t+1)). Then, the formation control network is used to calculate the state value function Q(s(t); ω) and Q(s(t+1); ω), and the corresponding action value function values ​​and are calculated based on the actual actions performed. Based on the maximum value action strategy, the action with the highest score is selected to obtain the maximum action value function value max a(t+1)Q(s(t+1),a1(t+1);ω). Then, using the discount factor γ and the immediate reward given by the environment, the state action value function value y1(t) at the moment can be calculated. The specific calculation method is shown in the figure.

[0051]

[0052] Therefore, the loss function Loss1(ω) of the formation control network is:

[0053] Loss1(ω)=E[(Q((t),a1(t);ω)-y1(t)) 2 ]

[0054] Finally, the Adma optimizer is used to update the network parameters. When the training score reaches 80% of the maximum training score, it is considered through observation and statistics that the leader-follower formation control network has the initial ability to find and approach the leader in a static obstacle environment, and the formation training will enter the second stage.

[0055] The training objective of Phase 2 is to enable the follower to have the ability to detect and track the dynamic leader. At this stage, both the follower and the leader are moving autonomously. After the training in Phase 1, the follower already has the ability to detect the leader and reach its surroundings in an obstacle environment. In Phase 2, the goal is to ensure that the follower can identify the position of the leader and successfully follow the leader during its continuous movement to form a local formation. As for the leader, it is necessary to avoid and get rid of the follower's tracking. To achieve this goal, an anti-tracking network is introduced, which is specifically used to train the leader's anti-tracking ability. Its purpose is also to improve the follower's tracking ability, so that it can robustly track the leader in a complex and unknown environment and maintain the stability of the local formation. Therefore, in order to ensure that the follower can detect and keep up with the leader, the follower's maximum speed must be greater than the leader's maximum speed. The specific training process is as follows: Figure 5 shown.

[0056] First, the anti-tracking agent is initialized with the formation control network weights at the end of phase 1. The subsequent training process is similar to the training method of the agent in phase 1. At the beginning of training, there are no samples in the leader's experience replay pool. When the number of stored samples exceeds 1280, the network training begins. As training progresses, the experience replay pool stores (s t ,a2 t ,r2 t ,s t+1 ), where s t and t+1 As the input of the anti-tracking network, Q(s t ;ω) and Q(s t+1 ;ω) and select a2 tThe corresponding Q(s t ,a2 t ; ω), select the corresponding action according to the maximum Q value. Finally, calculate the state action value function y2(t) at the current moment:

[0057]

[0058] The loss function Loss2(ω) of the navigator agent is:

[0059] Loss2(ω)=E[(Q((t),a2(t);ω)-y2(t)) 2 ]

[0060] Adma optimizer is also used to optimize network parameters.

[0061] The anti-tracking network training process is different from the formation control network training process in the following ways:

[0062] (1) Action selection strategy: In the training of the anti-tracking network, the maximum value action strategy is adopted instead of the ε-greedy strategy. This choice aims to directly select the action with the highest score to optimize the sample quality and improve the training efficiency. At the same time, this also helps to avoid falling into the local optimum and ensure the stability of the trajectory.

[0063] (2) Initial parameter values: Initialized through the weighted adversarial tracking network with initial obstacle avoidance function in stage 1. This allows the leader to avoid static obstacles without having to learn again, speeding up the joint training process and helping the follower to have dynamic following capabilities.

[0064] (3) Image preprocessing: During the training process, average pooling is used to reduce the image dimension to obtain more environmental information to help the navigator make decisions. This helps improve the navigator's environmental perception ability and ability to utilize environmental resources, giving it an advantage in the game.

[0065] The training purpose of stage 3 is to add dynamic obstacles to the original static obstacle environment so that the follower can find and keep up with the leader in a complex and unknown dynamic environment without collision. The core of the training in this stage is to let the follower identify and avoid dynamic obstacles and follow its leader through the input environment information. However, it is difficult for the agent to obtain the ability to identify dynamic obstacles through network training with image information alone. Therefore, radar data is introduced to assist in distinguishing dynamic obstacles. At the same time, the classic DQN network cannot capture the temporal variation characteristics of dynamic obstacles, so the LSTM and time attention mechanism are introduced to make up for the important part of the agent's temporal feature learning. Specifically, the training method and optimizer of the anti-tracking network and formation control network in this stage remain unchanged. The difference is that in addition to static obstacles, dynamic obstacles are added to the training scene. The running trajectory of the dynamic obstacle is a square. The center of the square trajectory is the initial position of the follower's training in each round. The initial position of the leader is outside the rectangle. Therefore, the follower needs to avoid dynamic obstacles in each round to reach the target and get a positive reward.

[0066] Step 3: Extract the motion features, distance features and dynamic obstacle change features of the mobile robot in the preprocessing information based on the trained deep reinforcement learning network. The preprocessing information is the fusion data of the follower's perspective image, radar data, distance data and the leader's speed information in step 1.

[0067] The motion characteristics of the mobile robot include the contour characteristics, shape characteristics and position change characteristics of the mobile robot; the distance characteristics of the mobile robot include the distance characteristics between the mobile robot and all mobile robots in front; the dynamic obstacle change characteristics include the contour characteristics and position change characteristics of the dynamic obstacles.

[0068] Step 4: Determine the formation motion control instructions for the mobile robot's adaptive formation and obstacle avoidance based on the acquired motion characteristics, distance characteristics, and dynamic obstacle change characteristics of the mobile robot. The formation control network is used to extract the motion characteristics, distance characteristics, and dynamic obstacle change characteristics of the mobile robot after fusion information and output the formation motion control instructions of the mobile robot.

[0069] Specifically, Figure 2 As shown, the formation control network outputs the state action value function, and its corresponding formation control action a t For: a R ,a F ,a L .

[0070] Among them, a R ,a F ,a L The directions represented are turn right, go straight, and turn left. The formation control actions are as follows:

[0071] a t =[a R a F a 3L ]

[0072] An action selection strategy is adopted to select formation control actions according to the action evaluation estimated by the state-action value function, and the formation motion control instructions of the mobile robots are output.

[0073] Step 5: According to the formation motion control instructions, control the mobile robots to adaptively form a formation and avoid obstacles.

[0074] According to the information processing results of the formation control network, the surrounding environment of the mobile robot is identified and predicted. Based on the identification and prediction results, the formation motion control instructions of the mobile robot are finally output. Figure 3 shown.

[0075] The present invention also provides a robot adaptive formation and obstacle avoidance control system based on multi-mode information fusion, including a preprocessing module, a feature extraction module, a control instruction output module and a control module.

[0076] A preprocessing module, used for preprocessing image data, radar data and distance data acquired by the mobile robot;

[0077] The feature extraction module is used to extract the motion features, distance features and dynamic obstacle change features of the mobile robot in the fusion data using the trained deep reinforcement learning network, and output the optimal formation motion decision of the mobile robot;

[0078] A control command output module, used for determining a formation motion control command of the mobile robot according to an optimal formation motion decision of the mobile robot;

[0079] The control module is used to control the mobile robot formation and formation obstacle avoidance according to the formation motion control instructions.

[0080] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion, It is characterized in that The following steps are involved: Obtain images of the follower, surrounding radar data, and distance data; Construct and train a deep reinforcement learning network, including a formation control network and an anti-tracking network. The formation control network is used to extract features of pre-processed fused data and output formation control actions based on the features of the fused data. The anti-tracking network is used to train the anti-tracking capability of the leader. Based on the trained deep reinforcement learning network, the motion characteristics, distance characteristics and dynamic obstacle timing characteristics of the mobile robot in the preprocessed information are extracted, and the optimal formation motion decision that can be executed by the mobile robot is output; According to the optimal formation motion decision of the mobile robots, the formation motion control instructions of the mobile robots are determined; Execute formation motion control instructions to control the mobile robot formation and formation obstacle avoidance.

2. The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion according to claim 1, It is characterized in that Acquiring the image of the follower, the surrounding environment radar data and the distance data, including: the mobile robot's onboard camera acquires the image of the follower, and performs dimension reduction, compression and stacking processing on the image; the laser radar acquires the surrounding environment radar data, and performs fixed-step sampling on the surrounding environment radar data to acquire a set amount of data; and the distance data acquired by the UWB sensor is median filtered.

3. The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion according to claim 1, It is characterized in that The fused data is used as the input of the formation control network and the anti-tracking network. The formation control network outputs the movement direction a and ot and speed change a vt ,After the mobile robot performs the action, it receives new state information and ,reward information.

4. The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion according to claim 1, It is characterized in that The formation control network includes an input layer, a CNN layer, an LSTM layer, a temporal attention layer, and an output layer. The LSTM layer is responsible for modeling time series data and encoding the historical observation sequence into a vector representation of a fixed length. The temporal attention mechanism is used to weight different time steps in the LSTM output sequence, estimate the value function and select actions for the vector representation, and optimize it through the back-propagation algorithm to maximize the cumulative reward or minimize the difference between the Q-value estimate and the target Q-value.

5. The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion according to claim 1, It is characterized in that Training the formation control network and anti-tracking network includes: Phase 1, randomly initialize the formation control network parameters, pre-process the data obtained by each sensor and perform fusion encoding as state variables, use the value function obtained by the formation control network to score each action, select the next action according to the ε-greedy strategy and execute it, enter a new state, and complete the training if the termination condition is met, otherwise continue to execute, and (s f (t),a(t),r(t),s f (t+1)) The quaternion is stored in the follower's experience replay pool. When the follower's experience replay pool accumulates a set amount of experience, network training is started to update parameters; In stage 2, the anti-tracking agent is initialized with the formation control network weights at the end of stage 1. At the beginning of training, the leader's experience replay pool has no samples. When the number of stored samples exceeds the set amount, the network is trained and the experience replay pool stores (s t ,a2 t ,r2 t ,s t+1 ), where s t and t+1 As the input of the anti-tracking network, calculate Q(s t ;ω) and Q(s t+1 ;ω) and select a2 t The corresponding Q(s t ,a2 t ; ω), select the corresponding action according to the maximum Q value, and finally calculate the state action value function y2(t) and the loss function Loss2(ω) of the navigator agent at the current moment, and use the Adma optimizer to optimize the network parameters; In stage 3, the training methods and optimizers of the anti-tracking network and formation control network remain unchanged. In addition to static obstacles, dynamic obstacles are added to the training scene. The trajectory of the dynamic obstacles is a square. The center of the square trajectory is the initial position of the follower in each round of training. The initial position of the leader is outside the rectangle. The follower needs to avoid dynamic obstacles in each round to reach the target and obtain a positive reward.

6. The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion according to claim 1, It is characterized in that Network training to update parameters includes: Randomly sample a set (s) from the experience replay pool f (t),a1(t),r1(t),s f (t+1)); The formation control network is used to calculate the state value function Q(s(t); ω) and Q(s(t+1); ω), and the corresponding action value function value and are calculated based on the actual actions performed; Based on the maximum value action strategy, select the action with the highest score and get the maximum action value function value max a(t+1) Q(s(t+1),a1(t+1);ω); Using the discount factor γ and the immediate reward given by the environment, the state action value function value y1(t) at the moment can be calculated. The specific calculation method is: The loss function Loss1(ω) of the formation control network is: Loss1(ω)=E[(Q((t),a1(t);ω)-y1(t)) 2 ] The Adma optimizer is used to update the network parameters. When the training score reaches 80% of the maximum training score, the leader-follower formation control network has the initial ability to detect and approach the leader in a static obstacle environment.

7. The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion according to claim 1, It is characterized in that The motion characteristics of the mobile robot include the contour characteristics, shape characteristics and position change characteristics of the mobile robot; the distance characteristics of the mobile robot include the distance characteristics between the mobile robot and all mobile robots in front; the dynamic obstacle change characteristics include the contour characteristics and position change characteristics of the dynamic obstacles.

8. The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion according to claim 1, It is characterized in that The formation control network outputs the state action value function, and its corresponding formation control action a t For: a R , a F , a L ; Among them, a R , a F , a L The directions represented are turn right, go straight, and turn left. The formation control actions are as follows: a t =[a R a F a 3L ] Adopting the action selection strategy, the optimal action a of the formation control action is selected according to the action evaluation estimated by the state action value function t .

9. A robot adaptive formation and obstacle avoidance control system based on multi-mode information fusion, It is characterized in that include: Data processing module, feature extraction module, control instruction output module and control module; The data processing module is used to pre-process the image data, radar data and distance data acquired by the mobile robot; the feature extraction module is used to extract the motion characteristics, distance characteristics and dynamic obstacle change characteristics of the mobile robot in the fusion data using the trained deep reinforcement learning network, and output the optimal formation motion decision of the mobile robot; the control instruction output module is used to determine the formation motion control instructions of the mobile robot according to the optimal formation motion decision of the mobile robot; the control module is used to control the formation formation and formation obstacle avoidance of the mobile robot according to the formation motion control instructions.

10. A robot formation, It is characterized in that The robot adaptive formation and obstacle avoidance control method based on multi-mode information fusion moves in formation based on any one of claims 1-8.

Images