Mechanical arm automatic door opening system and method based on magnetic attraction type device

Abstract

The invention discloses a mechanical arm automatic door opening system and method based on a magnetic attraction type device, the mechanical arm automatic door opening system comprises a depth camera, a mechanical arm and the magnetic attraction type device, the magnetic attraction type device comprises a mechanical arm clamping part arranged on the mechanical arm and an electrical cabinet door fixing part arranged on an electrical cabinet door, and the depth camera is arranged at the tail end of the mechanical arm. The grabbing process of the door handle can be simplified, and the error tolerance and the track control precision tolerance of visual positioning in the mechanical arm door opening process are improved. Meanwhile, the magnetic attraction type structure is very convenient to install, and complex mechanical structure design and installation steps are not needed. In a word, the mechanical arm automatic door opening method based on the magnetic attraction type device overcomes the problems in the prior art, and a more optimized door opening solution is provided.

Description

Technical Field

[0001] The present invention relates to a technology in the field of automation equipment, specifically a mechanical arm automatic door opening system and method based on a magnetic suction device. Background Art

[0002] In tasks such as electrical maintenance, robots need to automatically open revolving doors. Existing methods rely on sensors such as cameras to locate door handles, control the robotic arm to grab the door handle, and accurately plan the trajectory and force to open the door. However, this method relies on the camera to accurately obtain the position of the door handle and control the trajectory of the robotic arm with high precision. Errors in visual positioning and trajectory control can cause door opening failures. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention proposes a system and method for automatic door opening by a robotic arm based on a magnetic device, which can simplify the process of grabbing the door handle and improve the error tolerance of visual positioning and trajectory control accuracy tolerance of the robotic arm door opening process. At the same time, the installation of this magnetic structure is also very convenient, without the need for complicated mechanical structure design and installation steps. In short, the automatic door opening method by a robotic arm based on a magnetic device overcomes the problems in the prior art and provides a more optimized door opening solution.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a mechanical arm automatic door opening system based on a magnetic device, comprising: a depth camera, a mechanical arm and a magnetic device, wherein: the magnetic device comprises a mechanical arm clamping part arranged on the mechanical arm and an electrical cabinet door fixing part arranged on the electrical cabinet door, and the depth camera is arranged at the end of the mechanical arm.

[0006] The mechanical arm clamping part comprises: a mechanical arm clamping block, a flexible coupling and a first magnet mounting seat which are connected in sequence.

[0007] The electrical cabinet door fixing part comprises: a door sticker, a second flexible coupling and a second magnet mounting seat which are connected in sequence.

[0008] The present invention relates to an automatic door opening method of a robotic arm based on a magnetic device based on the above system. After collecting images with a depth camera, the center coordinates of the magnetic device in the image are identified by deep learning technology; then the three-dimensional coordinates of the magnetic device relative to the camera are calculated, and the three-dimensional coordinates of the magnetic device relative to the base of the robotic arm are calculated through a transformation matrix; based on the obtained coordinate relationship, the movement of the robotic arm is controlled to achieve automatic and precise opening and closing of the electrical cabinet door. Technical Effects

[0009] The present invention combines the two methods of direct operation of the robotic arm and installation of a special structure, and realizes the adsorption effect of the door handle through a magnetic structure, thereby improving the fault tolerance of door handle grasping and reducing the requirements for position recognition and robotic arm control accuracy. After visual positioning, only the end of the robotic arm needs to be roughly aligned with the magnetic device. Compared with the prior art, precise alignment is not required. After the robotic arm is connected to the door, it only needs to roughly control the movement trajectory of the robotic arm, and the traction force of the flexible coupling can be used to pull the door open, instead of grasping the door handle for precise trajectory control and force control, which reduces the requirements for robotic arm control accuracy, reduces the probability of the door being stuck, and improves the success rate of door opening. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of the structure of the magnetic attraction device of the present invention;

[0011] In the figure: a robot arm clamping part 1, an electrical cabinet door fixing part 2, a robot arm clamping block 101, a flexible coupling 102, a first magnet mounting seat 103, a door sticker 201, a second flexible coupling 202, and a second magnet mounting seat 203.

[0012] Figure 2 is a flow chart of an embodiment;

[0013] Figure 3 This is an overall diagram of the embodiment system. DETAILED DESCRIPTION

[0014] like Figure 3 As shown, a robotic arm automatic door opening system based on a magnetic device involved in this embodiment includes: a depth camera, a robotic arm and a magnetic device, wherein: the depth camera and the magnetic device are respectively arranged on the end gripper of the robotic arm and the door.

[0015] like Figure 1 As shown, the magnet size of the magnetic attraction device is 2cm×2cm×2cm, the length of the flexible connector at the clamping end of the robot arm is 3cm, and the size of the flexible connector at the fixed end is 6cm.

[0016] The mechanical arm clamping part comprises: a mechanical arm clamping block 101, a flexible coupling 102 and a first magnet mounting seat 103 which are connected in sequence.

[0017] The electrical cabinet door fixing part comprises: a door sticker 201, a second flexible coupling 202 and a second magnet mounting seat 203 which are connected in sequence.

[0018] The flexible coupling comprises a plurality of annular plastic sleeves connected in series, wherein the central axes of adjacent annular plastic sleeves are perpendicular to each other to form an 8-shaped structure, so that the magnetic device has a certain rigidity and hardness and can withstand gravity without excessive bending.

[0019] The annular plastic sleeve is made of rubber material.

[0020] The mechanical arm clamping block adopts two mutually perpendicular 8-shaped structures, so that the magnetic suction device has a certain rigidity and hardness, and can withstand the action of gravity without excessive bending. At the same time, compared with rigid structures such as door handles, this flexible structure has a certain flexibility and can complete the adsorption without complete alignment.

[0021] like Figure 2 As shown, this embodiment relates to a method for automatically opening a door by a mechanical arm based on a magnetic device based on the above system.

[0022] Step 1. Use a depth camera to capture an image of the magnetic device and mark the position of the magnetic device as a training set; build and train a neural network to realize position recognition of the magnetic device; use the trained neural network to mark the rectangular box where the magnetic device is located in the captured depth image in real time, and give the pixel coordinates of the four vertices of the rectangular box.

[0023] Step 2: Calculate the three-dimensional coordinates of the magnetic device relative to the camera using the depth image and pixel coordinates, and solve the three-dimensional coordinates of the magnetic device relative to the robotic arm base using the transformation matrix, specifically including:

[0024] 2.1) Based on the coordinates of the four points of the returned rectangular frame, the average value can be taken to obtain the center pixel coordinate of the device (x p ,y p ).

[0025] 2.2) According to the center point (x p ,y p ) and the camera internal parameters, and use the imaging principle of the camera to calculate the three-dimensional coordinates (x c ,y c ,z c ).

[0026] 2.3) Use the hand-eye calibration method to obtain the transformation matrix from the camera coordinate system to the robot base coordinate system

[0027] 2.4) Calculate the position of the device in the robot base coordinate system, specifically: Where: (x p ,y p ) is the pixel coordinate of the fixed part of the magnetic device, (x c ,y c ,z c) is the three-dimensional coordinate of the fixed part of the magnetic device in the camera coordinate system, is the camera coordinate system to the robot base coordinate system, (x b ,y b ,z b ) is the three-dimensional coordinate of the fixed part of the magnetic device relative to the base.

[0028] Step 3: Control the robot arm to drive the end magnet to move according to the calculated position of the magnetic device, so that the end magnet of the robot arm moves to the front of the magnet on the door. In this way, the end magnet of the robot arm will be adsorbed and fit with the magnet on the door.

[0029] Step 4: After the two magnetic blocks are attached to each other, the robot arm starts to move and pulls open the electrical cabinet door. The trajectory of the robot arm's movement is an arc, and it moves along the arc trajectory until the cabinet door is fully opened.

[0030] After specific experiments, it was determined that the magnetic device can complete adsorption when the center positions of the two parts deviate by 4cm in the x and y directions. On the basis that the accuracy of visual positioning can satisfy the good adsorption of the magnetic device, 20 door opening experiments were carried out, with the revolving door opened 90° as the standard. The door opening success rate of the present invention reached 90%.

[0031] Compared with the existing technology, this method can improve the fault tolerance of visual positioning of grasped objects and the fault tolerance of robot arm trajectory control.

[0032] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A mechanical arm automatic door opening system based on a magnetic device, characterized in that: include: A depth camera, a mechanical arm and a magnetic device, wherein: the magnetic device comprises a mechanical arm clamping part arranged on the mechanical arm and an electrical cabinet door fixing part arranged on the electrical cabinet door, and the depth camera is arranged at the end of the mechanical arm; The mechanical arm clamping part comprises: a mechanical arm clamping block, a flexible coupling and a first magnet mounting seat connected in sequence; The electrical cabinet door fixing part comprises: a door sticker, a second flexible coupling and a second magnet mounting seat which are connected in sequence.

2. The mechanical arm automatic door opening system based on the magnetic device according to claim 1 is characterized in that: The flexible coupling comprises a plurality of annular plastic sleeves connected in series, wherein the central axes of adjacent annular plastic sleeves are perpendicular to each other to form an 8-shaped structure.

3. The mechanical arm automatic door opening system based on the magnetic device according to claim 1 is characterized in that: The mechanical arm clamping block adopts two mutually perpendicular 8-shaped structures.

4. A method for automatically opening a door with a mechanical arm based on a magnetic device based on the system of any one of claims 1 to 3, characterized in that: After capturing images with a depth camera, the center coordinates of the magnetic device in the image are identified through deep learning technology; then the three-dimensional coordinates of the magnetic device relative to the camera are calculated, and its three-dimensional coordinates relative to the base of the robotic arm are calculated through the transformation matrix; based on the obtained coordinate relationship, the movement of the robotic arm is controlled to achieve automatic and precise opening and closing of the electrical cabinet door.

5. The automatic door opening method of a mechanical arm according to claim 4 is characterized in that: include: Step 1: Use a depth camera to capture images of the magnetic device and mark the positions of the magnetic device as a training set; Construct and train a neural network to realize the position recognition of the magnetic device; use the trained neural network to mark the rectangular box where the magnetic device is located in the acquired depth image in real time, and give the pixel coordinates of the four vertices of the rectangular box; Step 2: Calculate the three-dimensional coordinates of the magnetic device relative to the camera using the depth image and pixel coordinates, and solve the three-dimensional coordinates of the magnetic device relative to the robotic arm base using the transformation matrix; Step 3, control the robot arm to drive the end magnetic block to move, and move according to the calculated position of the magnetic device, so that the end magnetic block of the robot arm moves to the front of the magnetic block on the door, so that the end magnetic block of the robot arm will be adsorbed and fit with the magnetic block on the door; Step 4: After the two magnetic blocks are attached to each other, the robotic arm starts to move and opens the electrical cabinet door. The trajectory of the robotic arm's movement is an arc, and it moves along the arc trajectory until the cabinet door is fully opened.

6. The automatic door opening method of a mechanical arm according to claim 5, characterized in that: The step 2 specifically includes: 2.1) Based on the coordinates of the four points of the returned rectangular frame, take the average value to get the center pixel coordinate of the device (x p ,y p ); 2.2) According to the center point (x p ,y p ) and the camera internal parameters, and use the imaging principle of the camera to calculate the three-dimensional coordinates (x c ,y c ,z c ); 2.3) Use the hand-eye calibration method to obtain the transformation matrix from the camera coordinate system to the robot base coordinate system 2.4) Calculate the position of the device in the robot base coordinate system, specifically: in: (x p ,y p ) is the pixel coordinate of the fixed part of the magnetic device, (x c ,y c ,z c ) is the three-dimensional coordinate of the fixed part of the magnetic device in the camera coordinate system, is the camera coordinate system to the robot base coordinate system, (x b ,y b ,z b ) is the three-dimensional coordinate of the fixed part of the magnetic device relative to the base.

Images