Composite robot real-time cooperative control method

Abstract

The invention relates to the technical field of robot cooperative control, and discloses a composite robot real-time cooperative control method, which comprises the steps of constructing a mobile robot moving state recognition model, and performing moving state recognition on a mobile robot; a mechanical arm robot motion state recognition model is constructed, and motion state recognition is conducted on the mechanical arm robot; and the background server receives the identification result and the motion parameters of the composite robot, and carries out cooperative scheduling on the composite robot to obtain a control instruction of the composite robot. The recognition model is deployed on the composite robot, the current moving state and the motion state of the composite robot are obtained through recognition, the current environment information of the composite robot is extracted for path planning of obstacle avoidance processing, and the mechanical arm action capable of completing task operation to the maximum degree is selected; cooperative control of the composite robot integrating multiple functional robots in a complex scene is achieved, and the moving efficiency of the composite robot and the operation efficiency of a mechanical arm are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot collaborative control, and in particular to a real-time collaborative control method for a composite robot. Background Art

[0002] With the rapid development of Industry 4.0 and intelligent manufacturing, smart factories, as an important development direction of the future manufacturing industry, are receiving widespread attention. Smart factories have achieved high automation, digitization and intelligence of production processes through the comprehensive application of technologies such as the Internet of Things (IoT), artificial intelligence (AI), big data, and edge computing. In smart factories, robots are an important part of the production system. They can not only undertake highly repetitive and dangerous tasks, but also meet the needs of customized and personalized production through flexible deployment and collaborative work. In recent years, composite robot technology has gradually become an important development direction of smart factory robot systems. Composite robots are usually composed of multiple types of robots, such as mobile robots (AGV / AMR), robotic arms, industrial drones, etc. They can work together to complete complex production tasks and solve functional requirements that are difficult to cover with a single robot system. For example, mobile robots can complete material handling, and robotic arms can be responsible for assembly or inspection. However, existing composite robots may conflict due to overlapping tasks or resource competition, such as multiple robots needing to occupy the same work area or tools at the same time. How to avoid composite robot conflicts and achieve composite robot collaborative control has become an important issue that needs to be solved urgently. Summary of the invention

[0003] In view of this, the present invention provides a real-time collaborative control method for a composite robot, which realizes collaborative control of the composite robot by accurately identifying the operating status of the composite robot, thereby achieving the purpose of optimizing operation.

[0004] To achieve the above object, the present invention provides a real-time collaborative control method for a composite robot, comprising the following steps:

[0005] S1: constructing a mobile robot movement state recognition model to recognize the movement state of the mobile robot, wherein the movement state categories include stationary state, translation state, rotation state and rolling state;

[0006] S2: constructing a motion state recognition model for a manipulator robot, and performing motion state recognition on the manipulator robot, wherein the motion states include static, joint rotation, joint linear motion, joint circular motion, and grasping motion;

[0007] S3: The backend server receives the recognition result and the motion parameters of the composite robot, performs collaborative scheduling on the composite robot, and obtains control instructions for the composite robot, wherein the composite robot is a product of deep integration of the functions of the mobile robot and the mechanical arm robot;

[0008] S4: Send control instructions to the composite robot to control the motion of the composite robot.

[0009] As a further improvement method of the present invention:

[0010] Optionally, constructing a mobile robot movement state recognition model in step S1 includes:

[0011] The mobile robot movement state recognition model is a network structure of a pulse neural network, including an input layer, a pulse neuron layer and an output layer. The input layer is used to receive the movement data of the mobile robot and convert the movement data into a pulse signal. The pulse neuron layer is composed of 4 layers of pulse neurons connected to each other, which are responsible for receiving the pulse signal and gradually accumulating the membrane potential. When the membrane potential reaches a preset threshold, a pulse is triggered and the membrane potential is reset. The pulse sequence composed of the triggered pulses is used as the output. The output layer receives the pulse sequence output by the pulse neurons corresponding to the four movement states, calculates the frequency of the trigger pulse in the pulse sequence as the frequency of the pulse neuron, and selects the movement state corresponding to the pulse neuron with the highest frequency for output.

[0012] The mobile data of the mobile robot includes the position coordinates, linear velocity, angular velocity, three-axis acceleration, Euler angle and environmental scanning data of the mobile robot, wherein the three-axis acceleration is the acceleration value of the mobile robot along three orthogonal directions in three-dimensional space, the Euler angle includes yaw angle, pitch angle and roll angle, which are used to describe the posture of the mobile robot, and the environmental scanning data is the point cloud data of the surrounding environment of the mobile robot obtained by laser radar scanning;

[0013] The mobile robot is used for environmental perception and autonomous movement in different environments. It is equipped with a wheeled, tracked or foot-type mobile platform and can move on flat ground, complex terrain and narrow space. The mobile robot movement state recognition model can be deployed on the mobile robot.

[0014] Optionally, using the mobile robot movement state recognition model to recognize the movement state of the mobile robot includes:

[0015] The current movement data of the mobile robot is obtained, and the movement data is used as the input of the mobile robot movement state recognition model. The mobile robot movement state recognition model outputs the movement state corresponding to the movement data as the current movement state of the mobile robot, and the current movement data of the mobile robot is added to the movement state and transmitted to the background server.

[0016] Optionally, constructing a manipulator robot motion state recognition model in step S2 includes:

[0017] The mechanical arm robot motion state recognition model is in the form of a support vector machine, which contains a hyperplane, and the hyperplane includes a high-dimensional space mapping parameter w and a bias term b, wherein the high-dimensional space mapping parameter w is the normal vector of the hyperplane, and the bias term b represents the distance between the hyperplane and the origin;

[0018] The motion state recognition model of the manipulator robot takes the motion data of the manipulator robot as input and takes the motion state of the manipulator robot as output;

[0019] The robotic arm robot is used to complete delicate operation tasks, including grasping, assembly and welding. It is equipped with a multi-degree-of-freedom robotic arm. The robotic arm joints can operate flexibly in three-dimensional space. The end is equipped with grippers and suction cups, which can perform various forms of grasping movements. The robotic arm robot motion state recognition model can be deployed on the robotic arm robot.

[0020] Optionally, using the robot arm motion state recognition model to recognize the motion state of the robot arm includes:

[0021] Acquire motion data y of the manipulator robot, wherein the motion data of the manipulator robot includes joint motion data and terminal motion data, wherein the joint motion data includes joint torque, joint angle, joint velocity and joint acceleration, and the terminal motion data includes terminal position, terminal Euler angle, terminal velocity, terminal acceleration and terminal torque;

[0022] The manipulator motion state recognition model receives motion data y and calculates the distance dis(y) between the motion data y and the hyperplane:

[0023]

[0024] in:

[0025] ||·||2 represents the L2 norm;

[0026] The five motion states of the robotic arm robot are mapped to five distance intervals, among which the 1st to 5th motion states are respectively stationary, joint rotation, joint linear motion, joint circular motion and grasping motion. The distance intervals mapped to the 1st to 5th motion states are dis1, dis2, dis3, dis4, and dis5. The distance interval in which the distance dis(y) is located is determined, and the motion state corresponding to the distance interval is used as the recognition result.

[0027] Optionally, the composite robot is composed of a mobile robot and a plurality of manipulator arm robots, and a task target to be executed is provided to each composite robot.

[0028] Optionally, in step S3, the background server performs collaborative scheduling on the composite robot to obtain control instructions for the composite robot, including:

[0029] The backend server receives the current motion parameters and the task targets to be executed of each composite robot, wherein the current motion parameters of the composite robot include the current movement state and movement data of the mobile robot in the composite robot, the current motion state and movement data of the multiple manipulator robots, and the task targets to be executed of the composite robot include the target location and the task operation;

[0030] The collaborative scheduling strategy of the composite robot is:

[0031] Constructing a composite robot collaborative scheduling model, wherein the composite robot collaborative scheduling model is divided into a mobile robot collaborative scheduling module and a manipulator robot collaborative scheduling module;

[0032] The mobile robot collaborative scheduling module includes a mobile action space and a mobile action scheduling network. The mobile action space includes the mobile actions that the mobile robot can take. The mobile action scheduling network obtains the current mobile status and mobile data of all mobile robots, extracts environmental scanning data from the mobile data, and generates a three-dimensional environmental modeling result for each mobile robot using a point cloud modeling method. Based on the three-dimensional environmental modeling result, mobile status, mobile data and target location of the mobile robot, the path of the mobile robot is optimized to obtain the shortest obstacle avoidance path to the target location, and the mobile action of the mobile robot is selected from the mobile action space so that the subsequent mobile path of the mobile robot coincides with the shortest obstacle avoidance path, and the selected mobile action is converted into a mobile control instruction;

[0033] The robot arm collaborative scheduling module includes a strategy network and a value network. The strategy network contains all the actions that the robot arm can take. The value network is used to measure the strategy values ​​of the robot arm taking different actions in the current state, and then select the action with the highest strategy value and convert it into an action control instruction. The results to be solved in the strategy network and the value network are the strategy values ​​of taking different actions in different states, and are solved based on the reward function. The reward function is the completion degree of the task operation of the composite robot after taking the action.

[0034] The control instructions include movement control instructions and action control instructions;

[0035] The state of the robotic arm robot refers to the motion state and motion data of the robotic arm robot, and the action of the robotic arm robot refers to the motion state and motion data of the robotic arm robot at the next moment.

[0036] Optionally, sending the control instruction to the composite robot in step S4 includes:

[0037] The composite robot receives movement control instructions and motion control instructions, and controls the mobile robot and the mechanical arm robot in the composite robot according to the movement control instructions and the motion control instructions.

[0038] In order to solve the above problem, the present invention provides an electronic device, the electronic device comprising:

[0039] A memory storing at least one instruction;

[0040] Communication interface, enabling electronic equipment to communicate; and

[0041] The processor executes the instructions stored in the memory to implement the above-mentioned real-time collaborative control method of the composite robot.

[0042] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned real-time collaborative control method of the composite robot.

[0043] Compared with the prior art, the present invention proposes a real-time collaborative control method for a composite robot, which has the following advantages:

[0044] First, this scheme proposes a mobile state recognition model and a model training method, combines a mobile robot and a manipulator robot into a composite robot, adopts a mobile robot mobile state recognition model to recognize the mobile state of the mobile robot, and adopts a manipulator robot motion state recognition model to recognize the motion state of the manipulator robot. The mobile robot mobile state recognition model is a pulse neural network structure, which is driven by events that trigger pulses. Therefore, most neurons are stationary most of the time, with low computational complexity and low storage resources. It can be directly deployed on the mobile robot to process data in real time and reduce dependence on cloud computing. In addition, a synaptic weight update method is proposed, which relies on the change of membrane potential for dynamic timing update, which is more suitable for the implementation scenario of continuous operation of the mobile robot. In the training process of the manipulator robot motion state recognition model, a real manipulator robot is used for simulation, and the motion states corresponding to different motion data of the manipulator robot are collected to form a training data set. A heuristic algorithm is used to simulate the predation behavior of whales in different scenarios. According to the fitness function corresponding to the current position vector, different iteration methods are selected for iteration. When the fitness function is high, the optimal position vector is used as the benchmark.

[0045] A random search is performed, and a larger iteration weight is set at the beginning of the iteration to fully search the neighborhood space. A smaller iteration weight is set in the later stage of the iteration, so that the search range gradually approaches the optimal solution and the global search capability is improved. For position vectors with lower fitness functions, focus on the optimal position vector and use a spiral iteration method to make fine adjustments to improve local search capabilities. The high-dimensional space mapping parameters and bias terms in the support vector machine are solved, and the two recognition models can be deployed on the composite robot. Edge computing is used to avoid dependence on cloud computing, avoid the transmission of mobile data and motion data, and improve the operation efficiency of the composite robot.

[0046] At the same time, this scheme proposes a collaborative control method, which transmits the recognition results and motion parameters to the background server. The background server extracts the environmental information of the composite robot from the motion parameters and plans the path for obstacle avoidance for the mobile robot, so that the mobile robot can reach the target location faster. The reinforcement learning method is used to evaluate the strategy value of different actions taken by the manipulator robot, and the action that can maximize the completion of the task operation is selected. The path planning results and the selected actions of the manipulator robot are converted into control instructions, which are sent to the composite robot for control, so as to realize the collaborative control of the composite robot integrated with multiple functional robots in complex scenes, avoid collisions of the composite robots in complex scenes, and efficiently complete the tasks to be executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic flow chart of a real-time collaborative control method for a composite robot provided in accordance with an embodiment of the present invention.

[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0050] The embodiment of the present application provides a real-time collaborative control method for a composite robot. The execution subject of the real-time collaborative control method for a composite robot includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the real-time collaborative control method for a composite robot can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0051] Reference Figure 1 , Embodiment 1 of the present invention is:

[0052] A real-time collaborative control method for a composite robot comprises the following steps:

[0053] S1: Construct a mobile robot movement state recognition model to identify the movement state of the mobile robot.

[0054] The mobile robot movement state recognition model is constructed in the step S1, including:

[0055] The mobile robot movement state recognition model is a network structure of a pulse neural network, including an input layer, a pulse neuron layer and an output layer. The input layer is used to receive the movement data of the mobile robot and convert the movement data into a pulse signal. The pulse neuron layer is composed of 4 layers of pulse neurons connected to each other, which are responsible for receiving the pulse signal and gradually accumulating the membrane potential. When the membrane potential reaches a preset threshold, a pulse is triggered and the membrane potential is reset. The pulse sequence composed of the triggered pulses is used as the output. The output layer receives the pulse sequence output by the pulse neurons corresponding to the four movement states, calculates the frequency of the trigger pulse in the pulse sequence as the frequency of the pulse neuron, and selects the movement state corresponding to the pulse neuron with the highest frequency for output.

[0056] The mobile data of the mobile robot includes the position coordinates, linear velocity, angular velocity, three-axis acceleration, Euler angle and environmental scanning data of the mobile robot, wherein the three-axis acceleration is the acceleration value of the mobile robot along three orthogonal directions in three-dimensional space, the Euler angle includes yaw angle, pitch angle and roll angle, which are used to describe the posture of the mobile robot, and the environmental scanning data is the point cloud data of the surrounding environment of the mobile robot obtained by laser radar scanning;

[0057] The mobile robot is used for environmental perception and autonomous movement in different environments, and is equipped with a wheeled, tracked or footed mobile platform, and can move on flat ground, complex terrain and narrow space;

[0058] Constructing the training loss function of the mobile robot moving state recognition model to train the model parameters of the mobile robot moving state recognition model, the constructed loss function is:

[0059]

[0060] in:

[0061] L(p||q) represents the loss function, p represents the true value of the training sample, q represents the frequency value of the training sample, Indicates that the nth training sample is the true value of the jth movement state, and converts the nth training sample into a pulse sequence corresponding to the jth movement state. Represents the frequency of the converted pulse sequence, j∈[1,4], the 1st to 4th layer pulse neurons correspond to the four moving states of static state, translation state, rotation state and rolling state respectively, n∈[1,N], N represents the number of training samples.

[0062] Specifically, one or more combined algorithms of the gradient descent algorithm, the Momentum method, the Newton iteration method and the L-BFGS method may be used to solve the loss function and obtain the model parameters of the mobile robot movement state recognition model.

[0063] Using the mobile robot movement state recognition model to recognize the movement state of the mobile robot includes:

[0064] The current movement data of the mobile robot is obtained, and the movement data is used as the input of the mobile robot movement state recognition model. The mobile robot movement state recognition model outputs the movement state corresponding to the movement data as the current movement state of the mobile robot, and the current movement data of the mobile robot is added to the movement state and transmitted to the background server.

[0065] As a preferred embodiment of the present invention, the mobile robot movement state recognition model is a pulse neural network structure, which is driven by the event of triggering pulses. Therefore, most neurons are stationary most of the time, with low computational complexity and low storage resources. It can be directly deployed on the mobile robot to process data in real time, reduce dependence on cloud computing, and propose an update method for synaptic weights, which is dynamically updated based on the change of membrane potential, and is more suitable for the implementation scenario of continuous operation of the mobile robot. The process of using the mobile robot movement state recognition model to recognize the movement state of the mobile robot is as follows:

[0066] S11: The input layer receives the mobile robot's movement data and filters the environment scanning data therein to obtain the filtered movement data x = [x1, x2, x3, x4, x5], where x1, x2, x3, x4, x5 are the 1st to 5th data in the movement data x, corresponding to the position coordinates, linear velocity, angular velocity, three-axis acceleration and Euler angle of the mobile robot in turn;

[0067] S12: Convert the filtered mobile data x into a pulse signal, and the converted pulse signal is in the form of S(t):

[0068] S(t)=δ(t-f1(x1))+δ(t-f2(x2))+δ(t-f3(x3))+δ(t-f4(x4))+δ(t-f5(x5))

[0069]

[0070] in:

[0071] t represents time information, the initial value of t is 0 and the maximum value is 20;

[0072] f i (·) represents the encoding function of the i-th data in the mobile data, k i Represents the encoding coefficient of the i-th data, f i (x i ) represents data x i Pulse coding parameters, i∈[1,5];

[0073] δ(t―f i (x i )) is in the form of a Dirac pulse, when t = f i (x i ) then the pulse signal S(t) triggers the pulse, contributing potential increment δ(t―f i (x i ))·weight i , weight i represents the potential increment weight of the i-th data;

[0074] S13: Get the synaptic weight between the j-th layer spike neuron and the spike signal at time t Among them, j∈[1,4], the 1st to 4th layer pulse neurons correspond to the four moving states of static state, translation state, rotation state and rolling state respectively;

[0075] S14: Calculate the membrane potential V of the j-th layer spiking neuron at time t+1 j (t+1):

[0076]

[0077] in:

[0078] Represents synaptic weight The updated value of

[0079] α represents the learning rate; β represents the weight adjustment coefficient, and β is set to 0.2;

[0080] exp(·) represents an exponential function with a natural constant as the base;

[0081] Δtime represents the time interval between adjacent times. Specifically, the time interval between time t+1 and time t is Δtime;

[0082] S15: If the membrane potential V j(t+1) is not lower than the preset threshold V, it means that the j-th layer spike neuron triggers a pulse at time t+1, and V j (t+1)=0, return to step S13 to iterate t until the maximum value 20 is reached;

[0083] Otherwise, let V j (t+1)=0, return to step S13 to iterate t until the maximum value 20 is reached;

[0084] S16: constructing pulse sequences of 4 layers of pulse neurons, wherein the pulse sequence of each layer of pulse neurons is the time sequence of triggering pulses of the pulse neurons in this layer, and transmitting the pulse sequence to the output layer;

[0085] S17: The output layer calculates the frequency of the trigger pulse in the pulse sequence as the frequency of the pulse neuron, and selects the movement state corresponding to the pulse neuron with the highest frequency for output.

[0086] S2: Construct a motion state recognition model for the robotic arm robot and perform motion state recognition on the robotic arm robot.

[0087] The step S2 constructs a robot motion state recognition model, including:

[0088] The mechanical arm robot motion state recognition model is in the form of a support vector machine, which contains a hyperplane, and the hyperplane includes a high-dimensional space mapping parameter w and a bias term b, wherein the high-dimensional space mapping parameter w is the normal vector of the hyperplane, and the bias term b represents the distance between the hyperplane and the origin;

[0089] The motion state recognition model of the manipulator robot takes the motion data of the manipulator robot as input and takes the motion state of the manipulator robot as output;

[0090] The robotic arm robot is used to complete delicate manipulation tasks, including grasping, assembly and welding. It is equipped with a multi-degree-of-freedom robotic arm. The robotic arm joints can operate flexibly in three-dimensional space. The end is equipped with grippers and suction cups, which can perform various forms of grasping movements.

[0091] As a preferred embodiment of the present invention, a real manipulator robot is used for simulation, and the motion states corresponding to different motion data of the manipulator robot are collected to form a training data set data. A heuristic algorithm is used to simulate the predation behavior of whales in different scenarios. According to the fitness function corresponding to the current position vector, different iteration methods are selected for iteration. When the fitness function is high, the optimal position vector is used as a reference for random search, and a larger iteration weight is set at the beginning of the iteration to fully search the neighborhood space. A smaller iteration weight is set at the end of the iteration, so that the search range gradually approaches the optimal solution, thereby improving the global search capability. For position vectors with lower fitness functions, the optimal position vector is focused on, and a spiral iteration method is used for fine adjustment to improve the local search capability. The high-dimensional space mapping parameter w and the bias term b in the support vector machine are solved. The solution process is as follows:

[0092] Initialize H groups of position vectors, where the initialized h-th group of position vectors is is the position vector The two sets of position coordinates in correspond to the high-dimensional space mapping parameters and bias terms respectively;

[0093] Set the current iteration number of the position vector to s, the initial value of s is 0, and the maximum value is Max; the position vector The result of the sth iteration is

[0094] Based on the training data set data, a fitness function with a position vector as input is constructed to obtain the fitness function value of the position vector. The position vector The fitness function value is

[0095]

[0096] in:

[0097] V m Indicates the motion state category corresponding to the mth motion data in the training data set data, where motion state categories 1-5 correspond to stationary, joint rotation, joint linear motion, joint circular motion, and grasping motion respectively;

[0098] Represented by position vector Construct a robot arm motion state recognition model, and train the motion state category obtained by the model recognition of the mth motion data in the data set data;

[0099] The H groups of position vectors obtained in the sth iteration are used as the input of the fitness function in turn, and the position vector with the lowest fitness function value is used as the optimal position vector of the sth iteration.

[0100] The position vector is iterated until the maximum number of iterations is reached, and the position vector with the lowest fitness function value after the maximum number of iterations is reached is selected, and the two sets of coordinates of the position vector are used as the high-dimensional space mapping parameter w and the bias term b respectively; the iteration formula of the position vector is:

[0101]

[0102] in:

[0103] A represents the position vector iteration method for simulating encirclement and predation. s represents the weight coefficient of the sth iteration, and τ represents the iteration control coefficient;

[0104] represents the iterative method of the position vector simulating bubble net predation, E represents a random number uniformly distributed from -1 to 1, and g represents the iterative shape control coefficient;

[0105] rand(0,1) represents a random number between 0 and 1. express With the optimal position vector The distance vector between .

[0106] Using the robot arm motion state recognition model to recognize the motion state of the robot arm includes:

[0107] Acquire motion data y of the manipulator robot, wherein the motion data of the manipulator robot includes joint motion data and terminal motion data, wherein the joint motion data includes joint torque, joint angle, joint velocity and joint acceleration, and the terminal motion data includes terminal position, terminal Euler angle, terminal velocity, terminal acceleration and terminal torque;

[0108] The manipulator motion state recognition model receives motion data y and calculates the distance dis(y) between the motion data y and the hyperplane:

[0109]

[0110] in:

[0111] ||·||2 represents the L2 norm;

[0112] The five motion states of the robotic arm robot are mapped to five distance intervals, among which the 1st to 5th motion states are respectively stationary, joint rotation, joint linear motion, joint circular motion and grasping motion. The distance intervals mapped to the 1st to 5th motion states are dis1, dis2, dis3, dis4, and dis5. The distance interval in which the distance dis(y) is located is determined, and the motion state corresponding to the distance interval is used as the recognition result.

[0113] S3: The background server receives the recognition result and the motion parameters of the composite robot, performs collaborative scheduling on the composite robot, and obtains control instructions for the composite robot, wherein the composite robot is a product of deep integration of the functions of the mobile robot and the mechanical arm robot.

[0114] The composite robot is composed of a mobile robot and a plurality of mechanical arm robots, and a task target to be executed is provided to each composite robot.

[0115] In the step S3, the backend server performs collaborative scheduling on the composite robot to obtain control instructions for the composite robot, including:

[0116] The backend server receives the current motion parameters and the task targets to be executed of each composite robot, wherein the current motion parameters of the composite robot include the current movement state and movement data of the mobile robot in the composite robot, the current motion state and movement data of the multiple manipulator robots, and the task targets to be executed of the composite robot include the target location and the task operation;

[0117] The collaborative scheduling strategy of the composite robot is:

[0118] Constructing a composite robot collaborative scheduling model, wherein the composite robot collaborative scheduling model is divided into a mobile robot collaborative scheduling module and a manipulator robot collaborative scheduling module;

[0119] The mobile robot collaborative scheduling module includes a mobile action space and a mobile action scheduling network. The mobile action space includes mobile actions that the mobile robot can take. The mobile action scheduling network obtains the current mobile status and mobile data of all mobile robots, extracts environmental scanning data from the mobile data, and generates a three-dimensional environment modeling result for each mobile robot using a point cloud modeling method. Based on the three-dimensional environment modeling result, mobile status, mobile data and target location of the mobile robot, the path of the mobile robot is optimized to obtain the shortest obstacle avoidance path to the target location, and the mobile action of the mobile robot is selected from the mobile action space so that the subsequent mobile path of the mobile robot coincides with the shortest obstacle avoidance path, and the selected mobile action is converted into a mobile control instruction; specifically, the point cloud modeling method includes a Delaunay triangulation method or a Poisson surface reconstruction method, and the path optimization is an A* algorithm;

[0120] The robot arm collaborative scheduling module includes a strategy network and a value network. The strategy network contains all the actions that the robot arm can take. The value network is used to measure the strategy values ​​of the robot arm taking different actions in the current state, and then select the action with the highest strategy value and convert it into an action control instruction. The results to be solved in the strategy network and the value network are the strategy values ​​of taking different actions in different states, and are solved based on the reward function. The reward function is the completion degree of the task operation of the composite robot after taking the action.

[0121] The control instructions include movement control instructions and action control instructions;

[0122] The state of the robotic arm robot refers to the motion state and motion data of the robotic arm robot, and the action of the robotic arm robot refers to the motion state and motion data of the robotic arm robot at the next moment.

[0123] As an embodiment of the present invention, when the target location is not reached, the robotic arm robot collaborative scheduling module does not perform collaborative scheduling, and the action state of the robotic arm robot remains stationary.

[0124] S4: Send control instructions to the composite robot to control the motion of the composite robot.

[0125] The step S4 sends the control instruction to the composite robot, including:

[0126] The composite robot receives movement control instructions and motion control instructions, and controls the mobile robot and the mechanical arm robot in the composite robot according to the movement control instructions and the motion control instructions.

[0127] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0128] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0130] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A real-time collaborative control method for a composite robot, characterized in that: The method comprises: S1: constructing a mobile robot movement state recognition model to recognize the movement state of the mobile robot, wherein the movement state categories include stationary state, translation state, rotation state and rolling state; S2: constructing a motion state recognition model for a manipulator robot, and performing motion state recognition on the manipulator robot, wherein the motion states include static, joint rotation, joint linear motion, joint circular motion, and grasping motion; S3: The backend server receives the recognition result and the motion parameters of the composite robot, performs collaborative scheduling on the composite robot, and obtains control instructions for the composite robot, wherein the composite robot is a product of deep integration of the functions of the mobile robot and the mechanical arm robot; S4: Send control instructions to the composite robot to control the motion of the composite robot.

2. A real-time collaborative control method for a composite robot as claimed in claim 1, characterized in that: The mobile robot movement state recognition model is constructed in the step S1, including: The mobile robot movement state recognition model is a network structure of a pulse neural network, including an input layer, a pulse neuron layer and an output layer. The input layer is used to receive the movement data of the mobile robot and convert the movement data into a pulse signal. The pulse neuron layer is composed of 4 layers of pulse neurons connected to each other, which are responsible for receiving the pulse signal and gradually accumulating the membrane potential. When the membrane potential reaches a preset threshold, a pulse is triggered and the membrane potential is reset. The pulse sequence composed of the triggered pulses is used as the output. The output layer receives the pulse sequence output by the pulse neurons corresponding to the four movement states, calculates the frequency of the trigger pulse in the pulse sequence as the frequency of the pulse neuron, and selects the movement state corresponding to the pulse neuron with the highest frequency for output. The mobile data of the mobile robot includes the position coordinates, linear velocity, angular velocity, three-axis acceleration, Euler angle and environmental scanning data of the mobile robot, wherein the three-axis acceleration is the acceleration value of the mobile robot along three orthogonal directions in three-dimensional space, the Euler angle includes yaw angle, pitch angle and roll angle, which are used to describe the posture of the mobile robot, and the environmental scanning data is the point cloud data of the surrounding environment of the mobile robot obtained by laser radar scanning; The mobile robot is used for environmental perception and autonomous movement in different environments, and is equipped with a wheeled, tracked or footed mobile platform.

3. A real-time collaborative control method for a composite robot as claimed in claim 2, characterized in that: Using the mobile robot movement state recognition model to recognize the movement state of the mobile robot includes: The current movement data of the mobile robot is obtained, and the movement data is used as the input of the mobile robot movement state recognition model. The mobile robot movement state recognition model outputs the movement state corresponding to the movement data as the current movement state of the mobile robot, and the current movement data of the mobile robot is added to the movement state and transmitted to the background server.

4. A real-time collaborative control method for a composite robot as claimed in claim 1, characterized in that: The step S2 constructs a robot motion state recognition model, including: The mechanical arm robot motion state recognition model is in the form of a support vector machine, which contains a hyperplane, and the hyperplane includes a high-dimensional space mapping parameter w and a bias term b, wherein the high-dimensional space mapping parameter w is the normal vector of the hyperplane, and the bias term b represents the distance between the hyperplane and the origin; The motion state recognition model of the manipulator robot takes the motion data of the manipulator robot as input and the motion state of the manipulator robot as output; The robotic arm robot is used to complete delicate manipulation tasks, including grasping, assembly and welding. It is equipped with a multi-degree-of-freedom robotic arm. The robotic arm joints can operate flexibly in three-dimensional space. The end is equipped with grippers and suction cups, which can perform various forms of grasping movements.

5. A real-time collaborative control method for a composite robot as claimed in claim 4, characterized in that: Using the robot arm motion state recognition model to recognize the motion state of the robot arm includes: Acquire motion data y of the manipulator robot, wherein the motion data of the manipulator robot includes joint motion data and terminal motion data, wherein the joint motion data includes joint torque, joint angle, joint velocity and joint acceleration, and the terminal motion data includes terminal position, terminal Euler angle, terminal velocity, terminal acceleration and terminal torque; The manipulator motion state recognition model receives motion data y and calculates the distance dis(y) between the motion data y and the hyperplane: in: ||·||2 represents the L2 norm; The five motion states of the robotic arm robot are mapped to five distance intervals, among which the 1st to 5th motion states are respectively stationary, joint rotation, joint linear motion, joint circular motion and grasping motion. The distance intervals mapped to the 1st to 5th motion states are dis1, dis2, dis3, dis4, and dis5. The distance interval in which the distance dis(y) is located is determined, and the motion state corresponding to the distance interval is used as the recognition result.

6. A real-time collaborative control method for a composite robot as claimed in claim 1, characterized in that: The composite robot is composed of a mobile robot and a plurality of mechanical arm robots, and a task target to be executed is provided to each composite robot.

7. A real-time collaborative control method for a composite robot as claimed in claim 6, characterized in that: The S3 step includes: The backend server receives the current motion parameters and the task targets to be executed of each composite robot, wherein the current motion parameters of the composite robot include the current movement state and movement data of the mobile robot in the composite robot, the current motion state and movement data of the multiple manipulator robots, and the task targets to be executed of the composite robot include the target location and the task operation; The collaborative scheduling strategy of the composite robot is: Constructing a composite robot collaborative scheduling model, wherein the composite robot collaborative scheduling model is divided into a mobile robot collaborative scheduling module and a manipulator robot collaborative scheduling module; The mobile robot collaborative scheduling module includes a mobile action space and a mobile action scheduling network. The mobile action space includes the mobile actions that the mobile robot can take. The mobile action scheduling network obtains the current mobile status and mobile data of all mobile robots, extracts environmental scanning data from the mobile data, and generates a three-dimensional environmental modeling result for each mobile robot using a point cloud modeling method. Based on the three-dimensional environmental modeling result, mobile status, mobile data and target location of the mobile robot, the path of the mobile robot is optimized to obtain the shortest obstacle avoidance path to the target location, and the mobile action of the mobile robot is selected from the mobile action space so that the subsequent mobile path of the mobile robot coincides with the shortest obstacle avoidance path, and the selected mobile action is converted into a mobile control instruction; The robot arm collaborative scheduling module includes a strategy network and a value network. The strategy network contains all actions that the robot arm can take. The value network is used to measure the strategy values ​​of different actions taken by the robot arm in the current state, and then select the action with the highest strategy value and convert it into an action control instruction. The results to be solved in the strategy network and the value network are the strategy values ​​of different actions taken in different states, and are solved based on the reward function. The reward function is the completion degree of the task operation of the composite robot after taking the action; wherein the state of the robot arm is the motion state and motion data of the robot arm, and the action of the robot arm is the motion state and motion data of the robot arm at the next moment; The control instructions include movement control instructions and action control instructions.

8. A real-time collaborative control method for a composite robot as claimed in claim 7, characterized in that: The step S4 sends the control instruction to the composite robot, including: The composite robot receives movement control instructions and motion control instructions, and controls the mobile robot and the mechanical arm robot in the composite robot according to the movement control instructions and the motion control instructions.

Images