Ship multi-mechanical arm welding spot cooperative welding method based on QMIX reinforcement learning algorithm

Abstract

The invention belongs to the field of welding control, and particularly relates to a ship multi-mechanical-arm welding spot cooperative welding method based on the QMIX reinforcement learning algorithm. The ship multi-mechanical-arm welding spot cooperative welding method based on the QMIX reinforcement learning algorithm comprises the following steps of: a) establishing a reinforcement learning environment, and setting a welding area and an operation area in the environment; b) determining a state value and an action value of a mechanical arm; c) setting a reward value according to the statevalue, the action value and the collaborative welding and collision avoidance task; d) calculating a local action value function of each mechanical arm through a recurrent neural network according tothe state value and the action value, and performing an action selection process; e) obtaining an overall action value function of all the mechanical arms through a super network with non-negative setweight according to the action value function; and f) constructing a loss function according to the reward value in the step c) and the overall action value function network in the step e), calculating and updating the weight of the neural network according to a back propagation algorithm, and repeating the training process. The ship multi-mechanical-arm welding spot cooperative welding method based on the QMIX reinforcement learning algorithm does not depend on a system model, is simple and effective, and can realize a task of cooperative welding of welding spots in an obstacle environment.

Description

technical field [0001] The invention belongs to the field of welding control, and in particular relates to a cooperative welding method for multi-manipulator welding points of a ship based on a QMIX reinforcement learning algorithm. Background technique [0002] Among the industrial robots in various industries around the world, welding robots account for more than half. Welding robots have evolved from assembly line arm robots at the beginning, to sensor-based off-line programming welding robots, to today's widely applicable multi-sensor, intelligent welding robots with highly adaptive capabilities, their welding capabilities and automation level also improve rapidly. In a large space like a ship, welding robots can be used to weld multiple complex scene welding points, which can greatly improve welding efficiency and quality. This technology has become a hot research topic today. Putting robots into the shipbuilding industry, partially replacing or completely replacing m...

AI Technical Summary

Problems solved by technology

The above literatures have solved the problem of multi-robot cooperative welding path planning very well, but they have not dealt with the problems of interference between robots and sensor limitations.

[0005] Chen Hui analyzed and compared the advantages and disadvantages of commonly used path planning algorithms in the literature (Chen Hui, "Path Planning Research on Multi-robot Collaborative Welding of Body-in-White" [D], Shanxi: North University of China, 2019), and aimed at the traditional ant colony algorithm. Deficiencies A composite improved ant colony algorithm is proposed, which is applied to single robot and multi-robot welding path planning problems, and a solder joint allocation model and an anti-interference model between robots are established for multi-robot welding path planning problems, but this document does not include Sensor-limited situations are taken into account

Considering that ship welding is a welding process with a large space, it is often impossible to obtain global welding information due to the limitation of sensor equipment performance during robot operation, which will affect the accuracy of path planning and the effectiveness of anti-interference

Images