Multi-mechanical-arm coordination control system based on machine vision and improved artificial potential field method

Abstract

The invention discloses a multi-mechanical-arm coordination control system based on machine vision and an improved artificial potential field method, and belongs to the technical field of robot control. An innovative lifting platform is adopted. The innovative lifting platform has an intelligent height adjusting function. It means that the height of the device can be automatically adjusted according to different types of trucks or platforms, and manual intervention is not needed. The function detects the height of the truck or platform by means of a sensor. Workload of operators is reduced, and loading and unloading efficiency is improved. The loading effect picture shows the application of the information identification technology, the stacking planning technology and the multi-machine cooperation technology. The automatic loading single machine conducts unstacking treatment and transportation treatment on goods, the goods are continuously conveyed to a high position through a conveying belt by means of a lifting table, and stacking planning is achieved through the grabbing technology.

Description

Technical Field

[0001] The invention relates to a multi-manipulator coordinated control system based on machine vision and improved artificial potential field method, belonging to the field of robot control technology. Background Art

[0002] With the increasing demand for industrial automation, multi-robot collaborative operation systems have been widely used in logistics, warehousing, manufacturing and other fields. However, existing technologies have the following problems:

[0003] Insufficient task allocation and coordination: Traditional systems mostly use static task allocation strategies, which are difficult to dynamically adjust according to real-time environmental changes, and can easily lead to robotic arm conflicts or low efficiency.

[0004] Limitations of obstacle avoidance algorithms: Traditional artificial potential field methods have local minimum problems and do not consider the dynamic coupling effect of the robot arm's joint angles, resulting in a high path planning failure rate.

[0005] Insufficient real-time performance: There is a high communication delay between the visual data processing and motion control layer, which affects the system response speed.

[0006] Complex inverse kinematics solutions: The collaboration of multiple robotic arms requires frequent solutions to inverse kinematics equations, which has high computational complexity and is difficult to meet real-time requirements.

[0007] To address the above problems, the present invention proposes a multi-robotic arm coordinated control system that integrates machine vision and an improved artificial potential field method. Through hierarchical architecture design, dynamic adjustment of fuzzy PID parameters and potential field function optimization, the system robustness, obstacle avoidance efficiency and coordination ability are significantly improved. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a multi-manipulator coordination control system based on machine vision and improved artificial potential field method, which can automatically adjust the height according to different types of trucks or platforms without manual intervention, thereby improving writing efficiency.

[0009] Multi-manipulator coordination control system based on machine vision and improved artificial potential field method, including:

[0010] The information acquisition layer obtains cargo images through the visual servo system and performs image preprocessing. After obtaining high-quality images, the trained cargo recognition deep learning model is used to identify the targets in the images. After completing the recognition task, the cargo data is sent to the data processing layer.

[0011] The data processing layer, after obtaining cargo information, allocates tasks to multiple robotic arms based on the cargo's location and the current working status of each robotic arm. The allocation strategy targets the cargo on the top layer. When a robotic arm obtains a task, it performs coordinated control based on the location and subsequent actions of adjacent robotic arms while tracking and picking the cargo.

[0012] The manipulator control layer includes the manipulator visual servo control, the manipulator stable grasping control, and the underlying control module. The manipulator visual servo control module obtains the task target pose in real time and calculates the manipulator control parameters through the image Jacobian matrix to achieve dynamic tracking of the moving target. When the end of the manipulator tracks the target, the manipulator stable grasping control module performs stable grasping based on the cargo pose.

[0013] The belt conveyor directly connected to the robot arm is controlled at a constant speed, while the intermediate belt conveyor is controlled using a fuzzy PID control algorithm.

[0014] Furthermore, the flow rate and belt speed of the belt conveyor monitored in real time are compared with the given ideal values, and the comparison results are input into the fuzzy PID controller. The output frequency of the belt conveyor inverter is adjusted through the fuzzy PID control algorithm, thereby controlling the motor speed and belt conveyor speed.

[0015] Furthermore, the belt speed deviation of the belt conveyor is:

[0016] e=r(t)-y(t);

[0017] Where, e is the deviation between the given reference belt speed and the real-time monitoring belt speed. The belt speed deviation change rate ec can be obtained by taking the derivative of e with respect to time t. r(t) is the reference belt speed, and y(t) is the real-time monitoring belt speed.

[0018] Belt speed deviation change rate:

[0019] ec=de / dt;

[0020] After inputting e and ec into the fuzzy controller, the fuzzy control algorithm can be used to calculate:

[0021]

[0022] Among them, K P0 , K I0 , K D0 is the initial value of the PID controller, ΔK P , ΔK I , ΔK D The adjustment amount obtained by the fuzzy control rules.

[0023] Furthermore, the gravitational field causes the robotic arm to move toward the target position, while the repulsive field causes the robotic arm to avoid obstacles, thereby achieving the goals of path planning and obstacle avoidance.

[0024] Furthermore, if the current position coordinate of the end of the robot is X, and the position coordinate of the target point is X g , the distance between the manipulator and the obstacle is d, and the radius of the repulsive field is d0, then the gravitational potential function and the repulsive potential function are defined as:

[0025]

[0026] Where k a is the gravitational constant, k r is the repulsive force constant.

[0027] Furthermore, considering the shortest distance between each joint of the robotic arm and the obstacle, the repulsive potential energy U of each link in the repulsive field is calculated. rep (i) * , the new repulsive potential energy function of the robotic arm is obtained as:

[0028]

[0029] Where k r is the repulsive force constant, d i is the distance between each joint of the robotic arm and the obstacle, and d0 is the radius of the repulsive field.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The single-machine model utilizes an innovative lift platform with intelligent height adjustment. This means it automatically adjusts to different truck or platform types without manual intervention. This feature uses sensors to detect the height of the truck or platform, reducing operator workload and improving loading and unloading efficiency. The loading rendering demonstrates the application of information recognition, palletizing planning, and multi-machine collaboration technologies. The automated loading machine depalletizes and transports the goods, continuously delivers them to a higher position via a conveyor belt using the lift platform, and implements palletizing planning using gripping technology. DETAILED DESCRIPTION

[0032] The present invention will be further described below with reference to specific embodiments. However, people familiar with the art should understand that the detailed description given here is for better explanation, and the structure of the present invention necessarily exceeds these limited embodiments. For some equivalent replacement solutions or common means, they will not be described in detail herein, but they still fall within the scope of protection of this application.

[0033] Multi-manipulator coordination control system based on machine vision and improved artificial potential field method, including:

[0034] The information acquisition layer obtains cargo images through the visual servo system and performs image preprocessing. After obtaining high-quality images, the trained cargo recognition deep learning model is used to identify the targets in the images. After completing the recognition task, the cargo data is sent to the data processing layer.

[0035] The data processing layer, after obtaining cargo information, allocates tasks to multiple robotic arms based on the cargo's location and the current working status of each robotic arm. The allocation strategy targets the cargo on the top layer. When a robotic arm obtains a task, it performs coordinated control based on the location and subsequent actions of adjacent robotic arms while tracking and picking the cargo.

[0036] The manipulator control layer includes the manipulator visual servo control, the manipulator stable grasping control, and the underlying control module. The manipulator visual servo control module obtains the task target pose in real time and calculates the manipulator control parameters through the image Jacobian matrix to achieve dynamic tracking of the moving target. When the end of the manipulator tracks the target, the manipulator stable grasping control module performs stable grasping based on the cargo pose.

[0037] The belt conveyor directly connected to the robot arm is controlled at a constant speed, while the intermediate belt conveyor is controlled using a fuzzy PID control algorithm.

[0038] Specifically, the information acquisition layer uses a visual servo system to obtain cargo images and performs image preprocessing, focusing on removing duplicate objects. Once high-quality images are obtained, a trained cargo recognition deep learning model is used to identify the objects in the images. Once the recognition task is complete, the cargo data is sent to the data processing layer.

[0039] The data processing layer is the core of the entire software system. After acquiring cargo information, the system allocates tasks across multiple robotic arms based on the cargo's location and the current operating status of each robotic arm, with the allocation strategy prioritizing the top-level cargo. Once a robotic arm receives a task, it must coordinate control based on the positions and subsequent movements of adjacent robotic arms to avoid collisions. To ensure real-time information across the entire system, industrial Ethernet communication is used for data and command transmission between the host computer and the robotic arm controllers.

[0040] The robotic arm control layer primarily includes the robotic arm visual servo control, the robotic arm stable grasping control, and the underlying control module. The robotic arm visual servo control module uses the hand-eye system to acquire the target pose in real time and calculates the robotic arm control parameters using the image Jacobian matrix to achieve dynamic tracking of the moving target. Once the robotic arm's end tracked the target, the robotic arm stable grasping control module performed a stable grasp based on the cargo's pose to ensure successful grasping. The robotic arm's underlying control primarily utilizes PID control.

[0041] The PID control method is to realize three-level transportation of belt conveyors. The belt conveyor directly connected to the depalletizing and stacking robot adopts the traditional constant speed control, while the intermediate belt conveyor is controlled by the fuzzy PID control algorithm. This not only speeds up the transportation speed of goods, but also improves the efficiency and accuracy of the collaborative operation and collaborative control of the depalletizing and stacking robots.

[0042] The real-time monitored coal flow and speed of the conveyor belt are compared with the given ideal values, and the comparison results are input into the fuzzy PID controller. The output frequency of the belt conveyor inverter is adjusted through the fuzzy PID control algorithm, thereby controlling the motor speed and belt conveyor speed to achieve high-speed and stable transportation of goods.

[0043] The core of the control system is the fuzzy PID controller. Given the deviation between the reference belt speed and the real-time monitoring belt speed as e, the belt speed deviation change rate ec can be obtained by taking the derivative of e with respect to time t. During the operation of the belt conveyor, e and ec are continuously input into the fuzzy controller, and then K is adjusted according to the input data and control rules. P , K I , K D The adjusted parameters are input into the PID controller to achieve real-time speed regulation of the controlled motor.

[0044] Among them, r(t) is the reference belt speed and y(t) is the real-time monitoring belt speed.

[0045] Belt speed deviation:

[0046] e=r(t)-y(t)

[0047] Belt speed deviation change rate:

[0048] ec=de / dt

[0049] After inputting e and ec into the fuzzy controller, the fuzzy control algorithm can be used to calculate:

[0050]

[0051] Among them, K P0 , K I0 , K D0 is the initial value of the PID controller, ΔK P , ΔKI , ΔK D The adjustment amount obtained by the fuzzy control rules.

[0052] The fuzzy subsets of the input and output of the fuzzy controller are {NB, NM, NS, ZO, PS, PM, PB}, the triangular function is selected as the membership function, the quantization domain of the input e and ec is [-6, 6], and the output ΔK P , ΔK I , ΔK D The quantization domain of is [-1, 1]. Based on the characteristics of the cargo flow during the conveyor belt operation and the adjustment rules of the PID controller, as well as the experience of on-site staff and expert knowledge, we can summarize the fuzzy control rules for adjusting the three parameters of the PID controller. The organized fuzzy control rules are shown in the following table.

[0053] Table 1 ΔK P , ΔK I , ΔK D Fuzzy control rules

[0054]

[0055] Based on the above analysis, the entire artificial potential field is a superposition of a gravitational field and a repulsive field. It determines the direction of motion of the robotic arm by calculating the resultant force. The gravitational field forces the robotic arm to move toward the target location, while the repulsive field enables it to avoid obstacles, thus achieving the goals of path planning and obstacle avoidance. This superposition of gravitational and repulsive fields can help the robotic arm autonomously navigate and operate in complex environments. It is one of the core concepts of the artificial potential field method.

[0056] If the current position coordinate of the end of the robot is x, the target point position coordinate is x g , the distance between the manipulator and the obstacle is d, and the radius of the repulsive field is d0, then the gravitational potential function and the repulsive potential function are defined as:

[0057]

[0058] Where k a is the gravitational constant, k r is the repulsive force constant.

[0059] The gravitational force and repulsive force generated by the gravitational field and the repulsive field are the negative gradients of the corresponding potential energy, that is,

[0060]

[0061] Artificial potential field methods have some drawbacks. For example, during obstacle avoidance path planning, they are prone to falling into local minima. When the robot reaches a certain position, the repulsive force of the obstacle on the robot and the attractive force of the target point on the robot cancel each other out, causing the robot to remain in its current position and unable to move further toward the target.

[0062] If the obstacle avoidance path planning for a robotic arm is performed in Cartesian space, an inverse kinematics solution is required each time a spatial point is planned and each joint is checked for collision. The inverse kinematics solution of the robotic arm is a complex nonlinear operation that requires calculating the angles of each joint based on the target position and posture of the end effector. Since the kinematic equations of the robotic arm are usually complex and nonlinear, the inverse kinematics solution has a high computational complexity. In the artificial potential field obstacle avoidance path planning algorithm, multiple inverse kinematics solutions are required to find a safe and collision-free path. Especially for dual-arm systems, the complexity is higher, more joint angles need to be solved, and collisions between the two robotic arms need to be taken into account, which increases the complexity and computational complexity of the path planning algorithm.

[0063] Based on the above problems, and considering the factors involved in the obstacle avoidance path planning process of the robotic arm, the traditional artificial potential field method is used to re-establish new gravitational potential energy functions and repulsive potential energy functions according to the target point position and the angles of each joint of the robotic arm, so that the robotic arm can complete the obstacle avoidance behavior. The movement direction of the robotic arm is determined by the total potential energy generated by each joint. When the total potential energy of the robotic arm is 0, the robotic arm stops moving.

[0064] Based on the spatial coordinates of the target point and the angles from each joint to the target point, the new gravitational potential energy function for the six-degree-of-freedom robotic arm is constructed as follows:

[0065]

[0066] Where k a is the gravitational constant, X is the current position coordinate of the end of the robotic arm, x g is the target point position coordinate, θ i is the angle of the corresponding i-th joint, θ gi is the angle between the i-th joint angle and the target point.

[0067] Considering the shortest distance between each joint of the robotic arm and the obstacle, calculate the repulsive potential energy U of each link in the repulsive field rep (i) * , the new repulsive potential energy function of the robotic arm is obtained as

[0068]

[0069] Where k ris the repulsive force constant, d i is the distance between each joint of the robotic arm and the obstacle, and d0 is the radius of the repulsive field.

[0070] The total potential energy obtained by the repulsive forces generated by all obstacles and the attractive force of the target (final position) is:

[0071] U total =U aat * +U rep *

[0072] A multi-manipulator coordinated control system based on machine vision and an improved artificial potential field method combines environmental perception with force-guided control to achieve collaboration and coordination between the manipulators. By acquiring environmental information through machine vision and using an improved artificial potential field method to guide the manipulators' movements, the system can efficiently complete collaborative tasks while taking into account obstacle avoidance and safety considerations. This approach holds broad application prospects in industrial automation and logistics.

[0073] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A multi-manipulator coordinated control system based on machine vision and improved artificial potential field method, characterized by: include: The information acquisition layer obtains cargo images through the visual servo system and performs image preprocessing. After obtaining high-quality images, the trained cargo recognition deep learning model is used to identify the targets in the images. After completing the recognition task, the cargo data is sent to the data processing layer. The data processing layer, after obtaining cargo information, allocates tasks to multiple robotic arms based on the cargo's location and the current working status of each robotic arm. The allocation strategy targets the cargo on the top layer. When a robotic arm obtains a task, it performs coordinated control based on the location and subsequent actions of adjacent robotic arms while tracking and picking the cargo. The manipulator control layer includes the manipulator visual servo control, the manipulator stable grasping control, and the underlying control module. The manipulator visual servo control module obtains the task target pose in real time and calculates the manipulator control parameters through the image Jacobian matrix to achieve dynamic tracking of the moving target. When the end of the manipulator tracks the target, the manipulator stable grasping control module performs stable grasping based on the cargo pose. The belt conveyor directly connected to the robot arm is controlled at a constant speed, while the intermediate belt conveyor is controlled using a fuzzy PID control algorithm.

2. The multi-manipulator coordinated control system based on machine vision and improved artificial potential field method according to claim 1, characterized in that: Based on the real-time monitoring of the flow rate and belt speed of the belt conveyor, they are compared with the given ideal values, and the comparison results are input into the fuzzy PID controller. The output frequency of the belt conveyor inverter is adjusted through the fuzzy PID control algorithm, thereby controlling the motor speed and belt conveyor speed.

3. The multi-manipulator coordinated control system based on machine vision and improved artificial potential field method according to claim 3, characterized in that: The belt speed deviation of the belt conveyor is: e=r(t)-y(t); Where, e is the deviation between the given reference belt speed and the real-time monitoring belt speed. The belt speed deviation change rate ec can be obtained by taking the derivative of e with respect to time t. r(t) is the reference belt speed, and y(t) is the real-time monitoring belt speed. Belt speed deviation change rate: ec=de / dt; After inputting e and ec into the fuzzy controller, the fuzzy control algorithm can be used to calculate: Among them, K P0 , K I0 , K D0 is the initial value of the PID controller, ΔK P , ΔK I , ΔK D The adjustment amount obtained by the fuzzy control rules.

4. The multi-manipulator coordinated control system based on machine vision and improved artificial potential field method according to claim 1, characterized in that: The gravitational field enables the robotic arm to move toward the target position, while the repulsive field enables the robotic arm to avoid obstacles, thereby achieving the goals of path planning and obstacle avoidance.

5. The multi-manipulator coordinated control system based on machine vision and improved artificial potential field method according to claim 4, characterized in that: If the current position coordinate of the end of the robot is X, the target point position coordinate is X g , the distance between the manipulator and the obstacle is d, and the radius of the repulsive field is d0, then the gravitational potential function and the repulsive potential function are defined as: Where k a is the gravitational constant, k r is the repulsive force constant.

6. The multi-manipulator coordinated control system based on machine vision and improved artificial potential field method according to claim 5, characterized in that: Considering the shortest distance between each joint of the robotic arm and the obstacle, calculate the repulsive potential energy U of each link in the repulsive field rep (i) * , the new repulsive potential energy function of the robotic arm is obtained as: Where k r is the repulsive force constant, d i is the distance between each joint of the robotic arm and the obstacle, and d0 is the radius of the repulsive field.

Images