Collaborative robot safety control method based on vision

Abstract

The invention provides a vision-based collaborative robot safety control method, which comprises the following control steps: S1, acquiring scene information of a collaborative robot, and acquiring a target coordinate and a pixel position based on the scene information; and S2, the specific position of an obstacle around the target and a surrounding three-dimensional scene are recorded based on a camera on the collaborative robot, and the three-dimensional scene is modeled and projected on an operation screen, so that the collaborative robot can be ensured to achieve the effects of rapid positioning, moving and grabbing through mutual cooperation of a plurality of modules; meanwhile, a potential field method is adopted for route planning, so that the collaborative robot can be prevented from colliding with an obstacle during planning movement, it is ensured that the whole collaborative robot has good environment sensing and coping capacity, and the overall efficiency in the man-machine collaborative operation process can be effectively guaranteed; and the robot can be applied to any man-machine cooperation production environment without excessive adjustment, so that the development of the field of cooperation robots is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of collaborative robots, and in particular to a vision-based collaborative robot safety control method. Background Art

[0002] Collaborative robots, as the name suggests, are robots that can work together with humans on the production line, giving full play to the efficiency of robots and the intelligence of humans. This type of robot is not only cost-effective, but also safe and convenient, and can greatly promote the development of manufacturing companies. As a new type of industrial robot, collaborative robots have removed the obstacles to human-machine collaboration and allowed robots to completely break free from the constraints of guardrails or cages. Its groundbreaking product performance and wide range of applications have opened a new era for the development of industrial robots.

[0003] In the fields of robotics, automated control, and artificial intelligence, accurate recognition and feature extraction of environmental images are crucial. With the development of technology, neural network algorithms have become the mainstream method in this field. They can extract key information from complex image data to guide the decision-making and control of robots. However, traditional neural network algorithms face some challenges when processing environmental images. First, these algorithms usually have limitations on the image processing size, and it is difficult to directly process large-size or wide-angle environmental images. At the same time, the presence of people needs to be considered in the application environment of collaborative robots. At present, most collaborative robots use force feedback-based methods to perceive unsafe contact with people. This method can effectively reduce the damage caused by human-machine collisions, but the cost of force sensors is relatively high, which leads to an increase in the cost of collaborative robots, which to a certain extent hinders the promotion of collaborative robots. At the same time, the force feedback method cannot perceive the environment, and thus cannot predict danger in advance;

[0004] To this end, this patent designs a vision-based collaborative robot safety control method to solve the above problems. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a vision-based collaborative robot safety control method, which solves the problem that the vision algorithms of existing collaborative robots usually have limitations on the image processing size, making it difficult to directly process large-size or wide-angle environmental images, and at the same time there are certain safety risks during the movement process.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: A vision-based collaborative robot safety control method, comprising the following control steps:

[0009] S1. Obtain the scene information where the collaborative robot is located, and obtain the target coordinates and pixel position based on the scene information;

[0010] S2, using the camera on the collaborative robot to record the specific location of obstacles around the target and the surrounding three-dimensional scene, and project the three-dimensional scene modeling onto the operation screen;

[0011] S3, analyzing the best moving route based on the processing system in combination with the location of the obstacle and the target position;

[0012] S4, the collaborative robot issues a grasping command, and the user chooses whether to grasp through the brain-computer interface. If the target selection is wrong, the staff presses the abort button to make the collaborative robot abort the grasping command to select other targets. After pressing the confirm button, the collaborative robot moves to the vicinity of the target object and uses visual servoing to control the robotic arm to grasp it;

[0013] S5. After the grasping is completed, the internal information module adds the final location information and determines the best moving route based on the above steps S2 and S3, so that the collaborative robot can transfer the target object to the final location;

[0014] S6. The safety detection module inside the collaborative robot determines the degree of movement of the collaborative robot. According to the relative status of the human and the robot, it is divided into three safety levels: safe, cautious, and dangerous. The three states correspond to levels 1 to 3 respectively. Different measures will be triggered in the three states: the robot in the safe state runs at full speed, the robot in the cautious state slows down, and the robot in the dangerous state stops running and replans the path to avoid the possibility of collision;

[0015] S7. Clear the single collaborative capture information and proceed to the input operation of the next instruction program.

[0016] Preferably, the camera in S2 cooperates with a depth camera to acquire a depth image of the collaborative work scene.

[0017] Preferably: in S2, it is determined whether the ambient light is within a preset range. If the ambient light is low, the collaborative robot transmits a signal to the main controller, and the main controller issues a command to turn on the fill light to avoid collision with the body due to corners that are not recognized by the camera due to low ambient light.

[0018] Preferably: in S3, the potential field method is used for route planning. The target point in the potential field generates an attractive force on the robot, and the obstacle generates a repulsive force on the robot. The synthesis of all forces determines the size and direction of the resultant force of the robot. The attraction formula of the target can be set as follows:

[0019]

[0020] Among them, r is the distance between the robot and the target point, kattr is the attraction coefficient set by yourself, and dmax is the threshold distance. When the distance is less than the threshold dmax, the attraction is proportional to the square of the distance. When the distance is greater than the threshold, the attraction is linearly related to the distance. This setting can reduce the attraction when the robot is far away from the target point to avoid overshoot;

[0021] The obstacle rejection formula can be set as follows:

[0022]

[0023] Among them, r is the distance between the robot and the nearest obstacle, krepel is the repulsion coefficient set by yourself, and drepel is the maximum distance of the repulsion. When the robot approaches the obstacle within a certain distance, the repulsion increases significantly. When it exceeds this distance, the repulsion disappears.

[0024] The force is obtained by taking the gradient of the potential field. The total force of the robot at any position is obtained by synthesizing the vectors of all attractive and repulsive forces. The direction of the total force indicates the direction of movement of the robot. The magnitude of the force can also be used to adjust the speed or acceleration of the robot. By taking the gradient of the potential field, we can obtain the force vector at each position. Therefore, this is not only an offline path planning method, the constructed potential field also constitutes the control law of the robot, and can be used as an online real-time obstacle avoidance algorithm.

[0025] Preferably: a 3D camera is provided on the collaborative robot body in S1, a monocular camera is provided at the end of the robotic arm, a main controller is provided on the collaborative robot body, the main controller is communicatively connected with the system workstation, and the system workstation is used to send scheduling instructions to the main controller to control the collaborative robot to move to a designated workstation. The main controller is used to obtain the scheduling instructions sent by the system workstation, and can control the collaborative robot to move to the designated workstation, control the 3D camera on the robot body to identify the object to be grasped, and determine whether the 3D camera recognizes the object to be grasped. If the 3D camera does not recognize the object to be grasped, determine whether the 3D camera recognizes an obstacle. If the 3D camera recognizes an obstacle, control the robotic arm to move to the back side of the obstacle, control the monocular camera on the robotic arm to calculate the posture and position data of the object to be grasped in the monocular camera coordinate system, receive the posture and position data of the object to be grasped in the monocular camera coordinate system, calculate the posture and position data of the object to be grasped in the robotic arm coordinate system according to the posture and position data of the object to be grasped in the monocular camera coordinate system, and the transformation matrix between the robotic arm and the monocular camera, and control the robotic arm to complete the grasping of the object to be grasped according to the posture and position data of the object to be grasped in the robotic arm coordinate system.

[0026] (III) Beneficial effects

[0027] The present invention provides a vision-based collaborative robot safety control method, which has the following beneficial effects:

[0028] 1. The present invention can ensure that the collaborative robot can achieve the effects of rapid positioning, movement and grasping through the mutual cooperation between multiple modules. At the same time, the potential field method is used for route planning, which can prevent the collaborative robot from colliding with obstacles when planning movement, so as to ensure that the entire collaborative robot has good environmental perception and response capabilities, and can effectively guarantee the overall efficiency of the human-machine collaborative operation process. It can be applied in any human-machine collaborative production environment without too much adjustment, thereby promoting the development of the field of collaborative robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a framework diagram of the present invention.

[0030] Figure 2 It is a schematic diagram of the overall operation of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Example:

[0033] like Figure 1-2 As shown, an embodiment of the present invention provides a vision-based collaborative robot safety control method, comprising the following control steps:

[0034] S1. Obtain the scene information where the collaborative robot is located, and obtain the target coordinates and pixel position based on the scene information;

[0035] S2, using the camera on the collaborative robot to record the specific location of obstacles around the target and the surrounding three-dimensional scene, and project the three-dimensional scene modeling onto the operation screen;

[0036] S3, analyzing the best moving route based on the processing system in combination with the location of the obstacle and the target position;

[0037] S4, the collaborative robot issues a grasping command, and the user chooses whether to grasp through the brain-computer interface. If the target selection is wrong, the staff presses the abort button to make the collaborative robot abort the grasping command to select other targets. After pressing the confirm button, the collaborative robot moves to the vicinity of the target object and uses visual servoing to control the robotic arm to grasp it;

[0038] S5. After the grasping is completed, the internal information module adds the final location information and determines the best moving route based on the above steps S2 and S3, so that the collaborative robot can transfer the target object to the final location;

[0039] S6. The safety detection module inside the collaborative robot determines the degree of movement of the collaborative robot. According to the relative status of the human and the robot, it is divided into three safety levels: safe, cautious, and dangerous. The three states correspond to levels 1 to 3 respectively. Different measures will be triggered in the three states: the robot in the safe state runs at full speed, the robot in the cautious state slows down, and the robot in the dangerous state stops running and replans the path to avoid the possibility of collision;

[0040] S7. Clear the single collaborative capture information and proceed to the input operation of the next instruction program.

[0041] The camera in S2 cooperates with the depth camera to obtain the depth image of the collaborative work scene.

[0042] S2 determines whether the ambient light is within the preset range. If the ambient light is low, the collaborative robot transmits a signal to the main controller, which issues a command to turn on the fill light to avoid collisions with the robot due to corners that are too low to be recognized by the camera.

[0043] S3 uses the potential field method for route planning. The target point in the potential field generates attraction for the robot, and obstacles generate repulsion for the robot. The synthesis of all forces determines the size and direction of the robot's combined force. The target attraction formula can be set as follows:

[0044]

[0045] Among them, r is the distance between the robot and the target point, kattr is the attraction coefficient set by yourself, and dmax is the threshold distance. When the distance is less than the threshold dmax, the attraction is proportional to the square of the distance. When the distance is greater than the threshold, the attraction is linearly related to the distance. This setting can reduce the attraction when the robot is far away from the target point to avoid overshoot;

[0046] The obstacle rejection formula can be set as follows:

[0047]

[0048] Among them, r is the distance between the robot and the nearest obstacle, krepel is the repulsion coefficient set by yourself, and drepel is the maximum distance of the repulsion. When the robot approaches the obstacle within a certain distance, the repulsion increases significantly. When it exceeds this distance, the repulsion disappears.

[0049] The force is obtained by taking the gradient of the potential field. The total force of the robot at any position is obtained by synthesizing the vectors of all attractive and repulsive forces. The direction of the total force indicates the direction of movement of the robot. The magnitude of the force can also be used to adjust the speed or acceleration of the robot. By taking the gradient of the potential field, we can obtain the force vector at each position. Therefore, this is not only an offline path planning method, the constructed potential field also constitutes the control law of the robot, and can be used as an online real-time obstacle avoidance algorithm.

[0050] In S1, a 3D camera is provided on the collaborative robot body, a monocular camera is provided at the end of the robotic arm, and a main controller is provided on the collaborative robot body. The main controller is communicatively connected with the system workstation, and the system workstation is used to send scheduling instructions to the main controller to control the collaborative robot to move to a designated workstation.

[0051] The main controller is used to obtain the scheduling instructions sent by the system workstation, and can control the collaborative robot to move to the designated workstation, control the 3D camera on the robot body to identify the object to be grasped, and determine whether the 3D camera recognizes the object to be grasped. If the 3D camera does not recognize the object to be grasped, determine whether the 3D camera recognizes an obstacle. If the 3D camera recognizes an obstacle, control the robotic arm to move to the back side of the obstacle, control the monocular camera on the robotic arm to calculate the posture and position data of the object to be grasped in the monocular camera coordinate system, receive the posture and position data of the object to be grasped in the monocular camera coordinate system, calculate the posture and position data of the object to be grasped in the robotic arm coordinate system according to the posture and position data of the object to be grasped in the monocular camera coordinate system, and the transformation matrix between the robotic arm and the monocular camera, and control the robotic arm to complete the grasping of the object to be grasped according to the posture and position data of the object to be grasped in the robotic arm coordinate system.

[0052] The robot first needs to scan the surrounding environment through the camera and build a map model of the surrounding area. In the process of building the map, the robot will use SLAM (Simultaneous Location And Mapping) technology to achieve its own real-time positioning and map update. By matching the current position of the robot with the map, the robot can accurately know its location and plan a path to the target location. The robot will use path planning algorithms (such as A, D, RRT, etc.) to calculate the optimal path based on the target position and current position. When planning the path, the robot will consider multiple factors, such as path length, obstacle distribution, number of turns, etc., to ensure the feasibility and efficiency of the path, obstacle avoidance and navigation. During the navigation process, the robot will use a variety of sensors (such as ultrasonic sensors, infrared sensors, lidar, etc.) to monitor the surrounding environment in real time to avoid collisions with obstacles. When the robot detects an obstacle, it will immediately adjust the path to bypass the obstacle and continue to move forward. At the same time, the robot will also use map information to avoid entering dead ends or getting lost, and to avoid collisions and getting lost. By integrating information from multiple sensors, the robot can perceive the surrounding environment more accurately and improve the accuracy and reliability of obstacle avoidance. For example, lidar can provide high-precision distance information, while cameras can provide rich image information. The combination of the two can achieve accurate identification and positioning of obstacles. During the navigation process, the robot will dynamically adjust the path according to real-time environmental information to ensure the feasibility and safety of the path. When the robot detects obstacles or environmental changes, it will immediately re-plan the path to avoid collisions and getting lost.

[0053] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A vision-based collaborative robot safety control method, characterized in that: The control steps include: S1. Obtain the scene information where the collaborative robot is located, and obtain the target coordinates and pixel position based on the scene information; S2, using the camera on the collaborative robot to record the specific location of obstacles around the target and the surrounding three-dimensional scene, and project the three-dimensional scene modeling onto the operation screen; S3, analyzing the best moving route based on the processing system in combination with the location of the obstacle and the target position; S4, the collaborative robot issues a grasping command, and the user chooses whether to grasp through the brain-computer interface. If the target selection is wrong, the staff presses the abort button to make the collaborative robot abort the grasping command to select other targets. After pressing the confirm button, the collaborative robot moves to the vicinity of the target object and uses visual servoing to control the robotic arm to grasp it; S5. After the grasping is completed, the internal information module adds the final location information and determines the best moving route based on the above steps S2 and S3, so that the collaborative robot can transfer the target object to the final location; S6. The safety detection module inside the collaborative robot determines the degree of movement of the collaborative robot. According to the relative status of the human and the robot, it is divided into three safety levels: safe, cautious, and dangerous. The three states correspond to levels 1 to 3 respectively. Different measures will be triggered in the three states: the robot in the safe state runs at full speed, the robot in the cautious state slows down, and the robot in the dangerous state stops running and replans the path to avoid the possibility of collision; S7. Clear the single collaborative capture information and proceed to the input operation of the next instruction program.

2. The vision-based collaborative robot safety control method according to claim 1, characterized in that: The camera in S2 cooperates with the depth camera to obtain a depth image of the collaborative work scene.

3. The vision-based collaborative robot safety control method according to claim 1, characterized in that: In S2, it is determined whether the ambient light is within a preset range. If the ambient light is low, the collaborative robot transmits a signal to the main controller, and the main controller issues a command to turn on the fill light to avoid collision with the body due to corners that are too low in ambient light and cannot be recognized by the camera.

4. The vision-based collaborative robot safety control method according to claim 1, characterized in that: In S3, the potential field method is used for route planning. The target point in the potential field generates attraction for the robot, and the obstacle generates repulsion for the robot. The synthesis of all forces determines the size and direction of the resultant force of the robot. The attraction formula of the target can be set as follows: Among them, r is the distance between the robot and the target point, kattr is the attraction coefficient set by yourself, and dmax is the threshold distance. When the distance is less than the threshold dmax, the attraction is proportional to the square of the distance. When the distance is greater than the threshold, the attraction is linearly related to the distance. This setting can reduce the attraction when the robot is far away from the target point to avoid overshoot; The obstacle rejection formula can be set as follows: Among them, r is the distance between the robot and the nearest obstacle, krepel is the repulsion coefficient set by oneself, and drepel is the maximum distance of the repulsion force. When the robot approaches the obstacle within a certain distance, the repulsion force increases significantly. When it exceeds this distance, the repulsion force disappears. The force is obtained by taking the gradient of the potential field. The total force of the robot at any position is obtained by synthesizing the vectors of all attractive and repulsive forces. The direction of the total force indicates the direction of movement of the robot. The magnitude of the force can also be used to adjust the speed or acceleration of the robot. By taking the gradient of the potential field, we can obtain the force vector at each position. Therefore, this is not only an offline path planning method, the constructed potential field also constitutes the control law of the robot, and can be used as an online real-time obstacle avoidance algorithm.

5. The vision-based collaborative robot safety control method according to claim 1, characterized in that: In the S1, a 3D camera is arranged on the collaborative robot body, a monocular camera is arranged at the end of the robotic arm, and a main controller is arranged on the collaborative robot body. The main controller is communicatively connected with the system workstation, and the system workstation is used to send scheduling instructions to the main controller to control the collaborative robot to move to a designated workstation.

6. The vision-based collaborative robot safety control method according to claim 5, characterized in that: The main controller is used to obtain the scheduling instructions sent by the system workstation to control the collaborative robot to move to the designated workstation, control the 3D camera on the robot body to identify the object to be grasped, and determine whether the 3D camera recognizes the object to be grasped. If the 3D camera does not recognize the object to be grasped, determine whether the 3D camera recognizes an obstacle. If the 3D camera recognizes an obstacle, control the robotic arm to move to the back side of the obstacle, control the monocular camera on the robotic arm to calculate the posture and position data of the object to be grasped in the monocular camera coordinate system, receive the posture and position data of the object to be grasped in the monocular camera coordinate system, calculate the posture and position data of the object to be grasped in the robotic arm coordinate system according to the posture and position data of the object to be grasped in the monocular camera coordinate system, and the transformation matrix between the robotic arm and the monocular camera, and control the robotic arm to complete the grasping of the object to be grasped according to the posture and position data of the object to be grasped in the robotic arm coordinate system.

Images