Dual-arm robot dexterous assembly system and method for fiber distribution box

Abstract

This invention discloses a dual-arm robotic dexterous assembly system and method for fiber optic cables to fiber distribution boxes, comprising: a micro-gripper for gripping target optical fibers, wherein a micro-camera is embedded in the micro-gripper; a robotic arm with the micro-gripper connected to its end; a binocular vision system for acquiring images within the detection field of view; and a computer for controlling the movement of the robotic arm, camera, micro-gripper, and binocular vision system. The binocular vision system performs coarse positioning of the micro-gripper, and the micro-camera is then used to precisely position the gripping position of the micro-gripper, achieving automated assembly of the optical fiber to the fiber distribution box. This invention solves the uncertainty of the flexible body position during the assembly of flexible optical fibers, replacing traditional manual assembly with a combination of online and offline planning, thus improving work efficiency and liberating productivity.

Description

Technical Field

[0001] This invention relates to the field of mechanical automation, and in particular to a dual-arm robot dexterous assembly system and method for fiber optic distribution boxes. Background Technology

[0002] With the rapid development of my country's social economy, society has placed higher demands on factory production efficiency, and production methods relying on manual labor are gradually being replaced by automated production methods. However, in the assembly process of inserting optical fibers into optical discs, manual assembly remains the mainstream production method, which greatly limits the level of automation in the optical fiber industry chain and does not conform to the modern manufacturing solutions of high efficiency and high modularity.

[0003] Currently, most factories use manual rather than automated methods to assemble fiber optic splitters. This is mainly due to the limitations of current automated assembly solutions in accurately identifying the small, flexible components of optical fibers and the insufficient precision and flexibility of the clamping devices. Therefore, designing an automated and agile assembly solution for fiber optic splitters has become a crucial link in the current optical fiber assembly industry chain.

[0004] Chinese patent application 202410465145.2 discloses an automatic fiber optic winding system, characterized by its ability to release and wind fibers from a fiber optic reel. However, its structure is complex and its versatility is limited. It can only release fibers from the fiber optic reel but cannot install fibers, making it unsuitable for tasks such as loading fibers into fiber distribution boxes.

[0005] In summary, there is currently no complete intelligent assembly scheme and system for fiber optic splitters. Therefore, designing a robotic intelligent assembly system for fiber optic splitters would help improve the level of full automation in the fiber optic industry chain, save labor costs, and increase work efficiency. Summary of the Invention

[0006] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, the purpose of this invention is to provide a dual-arm robot dexterous assembly system and method for fiber optic splitter boxes.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A dual-arm robot dexterous assembly system for fiber optic distribution boxes includes:

[0009] Miniature clamp: used to grip the target optical fiber, the miniature clamp being embedded with a miniature camera;

[0010] Robotic arm: The end of the robotic arm is connected to a miniature gripper;

[0011] Binocular vision system: used to acquire images within the detection field of view;

[0012] Computer: Used to control the movement of robotic arm, camera, miniature gripper and binocular vision system. The binocular vision system completes the coarse positioning of the miniature gripper, and then the miniature camera is used to accurately position the gripper, so as to realize the automated assembly of fiber to fiber distribution box.

[0013] Furthermore, the robotic arm includes two six-degree-of-freedom articulated robotic arms.

[0014] Furthermore, the miniature clamp includes a movable jaw and a fixed jaw, both of which are curved.

[0015] Furthermore, a miniature camera is provided at the end of the fixed jaw.

[0016] An assembly method based on the aforementioned dual-arm robot dexterous assembly system includes the following steps:

[0017] The binocular vision system acquires images from the operating table and transmits them to the computer. The computer then separates the beam splitter from the image and obtains the poses of both ends of the beam splitter.

[0018] Based on the positions of the two ends of the splitter obtained in the previous step, the two robotic arms move to one end of the splitter and drive the clamp jaws to complete the clamping of the splitter and insert the splitter into the fiber optic tray.

[0019] The binocular vision system re-acquires the image of the operating table, separates the fiber outline in the image at this time, preliminarily confirms the gripping point on the fiber, and drives the robotic arm to move to that position.

[0020] A miniature camera acquires images of the optical fiber and calculates the clamp attitude error in real time based on the optical fiber position image of the template.

[0021] Based on the clamp posture error, the robotic arm adjusts and corrects in real time to make the image from the miniature camera close to the template image, thus achieving precise positioning of the optical fiber.

[0022] The robotic arm drives the clamps to grasp the optical fiber and inserts the grasped light into the optical fiber tray slot, thus completing the grasping and insertion of the light.

[0023] Furthermore, the binocular vision system re-acquires the image of the operating table, separates the fiber outline in the image at this time, preliminarily confirms the gripping point on the fiber, and drives the robotic arm to move to that position, specifically:

[0024] The Mask_RCNN neural network is used to segment the fiber outline from the image. The clamping points on the fiber are further confirmed by the matching point algorithm. Then, the spatial coordinates of the clamping points are determined by the principle of triangulation and transformed into the coordinate system of the robotic arm.

[0025] Furthermore, the Mask_RCNN neural network is a two-stage network structure.

[0026] Furthermore, a miniature camera acquires fiber optic images, and the clamp attitude error is calculated in real time based on the fiber optic position images of the template. Specifically:

[0027] The parameters required for fiber pose are obtained by comparing the current image with a pre-captured template fiber position image.

[0028] The direction and speed of the end-effector's miniature camera are calculated based on the image Jacobian, and the clamp posture is corrected in real time to achieve precise positioning.

[0029] Furthermore, the current image is compared with a pre-captured template image to obtain the situation where the optical fiber deviates from its preposition, and further, the imaging projection image of the miniature camera is obtained when the optical fiber deviates from its preposition state is superimposed.

[0030] Based on the imaging projection diagram, the fiber axis slope, fiber profile slope, fiber center length, fiber center distance from the target center, and fiber thickness at the target position are calculated.

[0031] Based on the set target location and the physical dimensions of the optical fiber, determine the width of the optical fiber and the distance between the target location of the optical fiber and the miniature camera;

[0032] By determining the optical fiber offset angle θ1 around the Z-axis, the optical fiber offset angle θ2 around the Y-axis, and the values ​​of the optical fiber offset ΔY and ΔZ around the Y-axis and Z-axis, the attitude error of the optical fiber at the ideal position can be obtained.

[0033] Furthermore, it also includes fiber coiling action, which is planned offline.

[0034] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0035] (1) This invention uses a binocular vision system combined with a Mask_RCNN convolutional neural network to detect and segment small optical fiber targets and determine the spatial position of their gripping points. This solves the uncertainty of the position of the flexible body during the assembly of the optical fiber flexible body. It replaces the traditional manual assembly with a combination of online and offline planning, thereby improving work efficiency and liberating productivity.

[0036] (2) The clamp of this invention has a built-in miniature camera at the end for gripping thin optical fibers. Since the clamp itself has a built-in camera, it can perform fine guidance, and use a large field of view (the large field of view refers to the field of view of the binocular camera, which can include the fiber distribution box on the operating table and all optical fibers) for coarse positioning, and then perform fine positioning within a small field of view (the small field of view refers to the field of view of the miniature camera, which only includes a section of the optical fiber and a very small background). This clamp improves the accuracy of gripping small objects and reduces the difficulty of gripping optical fibers.

[0037] (3) The jaws of the clamp in this invention have a certain curvature and can be filled with flexible materials such as silicone. In this way, on the one hand, the clamping force of the jaws can be indirectly controlled by controlling the moving distance of the drive motor, and on the other hand, the flexible material filled in can envelop the single or multiple optical fibers as much as possible, thereby achieving stable clamping of the single or multiple optical fibers. Attached Figure Description

[0038] Figure 1 This is a structural schematic diagram of a dual-arm robot dexterous assembly system for an optical fiber-fiber splitter box according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the miniature clamp according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the fiber distribution box according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the structure of a convolutional neural network according to an embodiment of the present invention. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the embodiments, but the implementation of the present invention is not limited thereto.

[0043] Example

[0044] like Figure 1 As shown, a dual-arm robotic dexterity assembly system for a fiber optic distribution box includes: two six-degree-of-freedom articulated robotic arms 101 and 104, a binocular vision system 102, a telescopic and rotatable camera bracket 103, miniature grippers 105 and 107, a fiber distribution box 106, and a computer. The computer is connected to the two six-degree-of-freedom articulated robotic arms, the binocular vision system, and the two miniature grippers. The six-degree-of-freedom articulated robotic arms are selected from Yaskawa models.

[0045] Further explanation:

[0046] like Figure 2As shown, the miniature clamp is used to grip optical fibers and splitters, and achieves precise gripping through visual guidance via a miniature camera at the end.

[0047] The miniature gripper is connected to a six-degree-of-freedom articulated robotic arm via a flange. The two miniature grippers have identical structures. The miniature gripper includes a movable jaw 301 and a fixed jaw 302. The movable jaw is the main component for achieving the gripping function. Its jaw has a certain curvature and can be filled with silicone or other flexible materials. The curvature is an arc with a radius of 14mm and an angle of 35°.

[0048] The fixed jaws have a certain curvature in their contact surface, which works in conjunction with the movable jaws to achieve the gripping function. The fixed jaws are provided with an opening slot 303 for embedding a miniature camera.

[0049] Part 304 is directly connected to the linear motor via a connector, becoming a driving component that performs linear motion in a fixed z direction; Part 305 connects Part 304 and the movable jaw 301, converting the linear motion into rotational motion.

[0050] The fiber distribution box is made of plastic, and the optical fiber needs to be inserted into and coiled in the fiber distribution box.

[0051] like Figure 3 As shown, the fiber splitter box has different types of limiting slots. The optical fiber needs to be clamped and placed under the slot in one go so that the optical fiber will not fall out of the slot in its natural state, so as to achieve the purpose of fiber coiling. The optical fiber splitter is installed at position 408 of the fiber splitter box. This position has a certain elasticity, and its two ends will press the splitter tightly to achieve the purpose of clamping the splitter.

[0052] The binocular vision system 102 includes a binocular vision camera and a retractable and movable bracket 103. One end of the retractable and movable bracket is fixed to the wall, and the other end is fixed to the two cameras and their connectors and placed directly above the operating table.

[0053] The binocular vision camera consists of two industrial cameras with identical basic parameters, which are fixed to the bracket by bolts. The distance (baseline length) and field of view between the two industrial cameras can be adjusted for better observation of the workbench space.

[0054] This example also includes an equipment installation platform: a platform for mounting two robotic arms and an operating table for placing fiber distribution boxes and optical fibers, etc.

[0055] An assembly method based on the above assembly system includes:

[0056] The robotic arm is first initialized to a zero-point state (a manually defined initialization state), and then waits for the next instruction from the computer.

[0057] The binocular vision system acquires images from the operating table and transmits them to the computer. The Mask-RCNN neural network segments the beam splitter in the image and calculates the poses of the two ends of the beam splitter.

[0058] Based on the poses of the two ends of the beam splitter obtained in the previous step, the two robotic arms move to one end of the beam splitter and drive the clamp jaws to complete the clamping of the beam splitter.

[0059] After the gripping is completed, the two robotic arms, while maintaining the gripping state, move together to the top of the fiber optic disk structure, and then install the splitter into the structure.

[0060] The binocular camera re-captures images from the control panel and separates the fiber optic outline from these images. Based on a corresponding point matching algorithm, the gripping point on the fiber optic cable is preliminarily identified, and the robotic arm is driven to move to that position.

[0061] Once the clamp moves to the above position, the miniature camera at the end of the clamp will observe a section of optical fiber. The miniature camera will capture the image at this time and calculate the clamp attitude error in real time based on the image of the optical fiber position on the template.

[0062] After the clamps calculate the attitude error, the robotic arm adjusts and corrects the error in real time to make the image from the miniature camera as close as possible to the template image, thereby achieving precise positioning of the optical fiber.

[0063] Once the clamp posture is adjusted to the expected position, the robotic arm and clamp are driven to grip the optical fiber, and the robotic arm is driven to insert the gripped optical fiber into the optical fiber tray slot (structures 401-407 are all optical fiber tray slots), thus completing the gripping and insertion of the optical fiber.

[0064] Repeat the above steps multiple times until all the optical fibers are clamped and inserted into the slot, and the optical fibers are coiled on the optical disk.

[0065] To further explain, the binocular vision system acquires images from the operating table and transmits them to the computer. The Mask-RCNN neural network segments the beam splitter in the image and calculates the poses of both ends of the beam splitter. Specifically:

[0066] The image is fed into a trained convolutional neural network, which segments the beam splitters in the two images. Then, based on the triangulation principle of binocular vision, the position coordinates of the two ends of the beam splitter are extracted and transformed into the robot coordinate system, thereby driving the robotic arm to move to the designated position.

[0067] To further explain, the binocular vision system acquires images from the operating table and transmits them to the computer, segments the fiber optic outline, determines the gripping point, achieves coarse positioning of the fiber optic cable, and drives the robotic arm to move to the coarsely positioned coordinates. Specifically:

[0068] After the robot inserts the beam splitter, it needs to grip and coil the optical fiber. Gripping the fiber requires capturing images using a binocular camera. These images are then processed by a convolutional neural network to segment the fiber outline from the background. A matching point algorithm is used to further identify the points on the fiber outline that need to be gripped. Triangulation is then used to determine the spatial coordinates of these two points, which are then transformed into the robot's coordinate system, and the robot's arms are driven to move to the vicinity of these points.

[0069] Traditional vision methods typically employ edge detection, contour fitting, and template matching to locate the object being detected. These methods require the object to have obvious texture features or a large grayscale gradient with the background. However, considering the specific task objective of this patent, the optical fiber and the fiber distribution box are similar in color, and the optical fiber itself lacks texture features, making it impossible to segment it using traditional vision methods.

[0070] Both of the above neural network steps use the Mask R-CNN neural network for fiber optic instance segmentation. Training this model requires first sampling a large number of images containing optical fibers in a production environment, and then manually labeling the fibers in the images. This process must be meticulous in clearly labeling the fiber outlines. Simultaneously, because optical fibers appear elongated in images, the size of the convolutional kernels needs to be appropriately increased so that the network can identify fibers more comprehensively or over larger areas, avoiding mistaking small linear structures in the background for optical fibers.

[0071] like Figure 4 As shown, specifically, the Mask-RCNN neural network is a two-stage network structure, including:

[0072] Phase 1: Regional Proposal Network (RPN), which proposes candidate bounding boxes in Phase 1. Each proposal in this phase will then proceed to Phase 2.

[0073] The second stage: For each proposal, the feature regions of the image are extracted through convolutional layers, and the ROI regions are pooled and passed through the remaining network. The remaining network is used to implement the discriminator and generate bounding boxes. Additionally, two convolutional layers are added after ROI pooling to extract the target mask.

[0074] Appropriately increasing the size of the convolution kernel has the following effects in fiber profile segmentation:

[0075] Depending on the specific task requirements, increasing the convolution kernel size can increase the receptive field of the image, meaning that the optical fiber can be observed over a larger area, which is beneficial for identifying the optical fiber as a whole and segmenting its mask.

[0076] To avoid misidentifying fine lines in the background as optical fibers when using small convolutional kernels, we need to minimize the likelihood of these errors. Small kernels easily extract detailed texture features from objects, while large kernels cover more structural features. Optical fibers themselves don't have inherent texture; instead, they require a larger scene to be identified for higher accuracy. In this task, the prior bounding boxes and convolutional layer sizes of Mask-RCNN were adjusted to 64, 128, 256, 512, and 1024.

[0077] In this experiment, the prior bounding boxes and convolutional layer sizes of the Mask-RCNN network were adjusted as follows, and this combination of convolutional kernel sizes yielded the best results under the same sample size and epoch.

[0078] Further explanation: The miniature camera acquires the current image and provides real-time feedback on the attitude error of the miniature gripper, achieving precise positioning. Specifically:

[0079] The current image is compared with a pre-captured template image to obtain the fiber optic deviation from its pre-position. This deviation includes several scenarios: (1) the fiber optic cable deviates by an angle θ1 around the Z-axis; (2) the fiber optic cable deviates by ΔY and ΔZ around the Y-axis and Z-axis; (3) the fiber optic cable deviates by an angle θ2 around the Y-axis; and (4) the fiber optic cable is in an ideal position. These four scenarios exhibit different characteristics in the miniature camera. In reality, these three error postures do not occur individually but are superimposed. Further analysis is performed to obtain the superimposed state of fiber optic deviation from its pre-position in the miniature camera's imaging projection.

[0080] Based on the imaging projection diagram, the fiber axis slope K1, fiber profile slope K2, fiber center length L, and fiber center distance ΔL from the target center are calculated. m And the thickness L0 of the optical fiber at the target location;

[0081] Based on the set target location and the physical dimensions of the optical fiber, determine the width W of the optical fiber and the distance Z0 from the target location of the optical fiber to the miniature camera.

[0082] By determining the optical fiber offset angle θ1 around the Z-axis, the optical fiber offset angle θ2 around the Y-axis, and the values ​​of the optical fiber offset ΔY and ΔZ around the Y-axis and Z-axis, the attitude error of the optical fiber at the ideal position can be obtained.

[0083]

[0084] θ1 = arctan(k1)

[0085]

[0086] The four variables obtained from the above formula can be used to calculate the attitude error of the optical fiber at the ideal position, which can then drive the robotic arm to move, thereby compensating for the attitude error and bringing the optical fiber closer to the ideal position for subsequent clamping operations.

[0087] To further explain, since the fiber distribution box is fixed, the position of its limiting slot can be obtained in advance. The fiber coiling action after clamping the fiber can be planned offline. That is, the dual-arm robot simply holds a section of fiber through the limiting slot and then releases one end of the fiber to complete one installation. During this process, the robot's movement trajectory needs to be planned carefully to avoid collisions.

[0088] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An assembly method for a dual-arm robot dexterous assembly system for fiber optic-fiber splitter boxes, characterized in that, The dual-arm robot dexterous assembly system includes: Miniature clamp: used to grip the target optical fiber, the miniature clamp being embedded with a miniature camera; Robotic arm: The end of the robotic arm is connected to a miniature gripper; Binocular vision system: used to acquire images within the detection field of view; Computer: Used to control the movement of robotic arm, camera, miniature gripper and binocular vision system. The binocular vision system completes the coarse positioning of the miniature gripper, and then the miniature camera is used to accurately position the gripper, so as to realize the automated assembly of fiber to fiber distribution box. Assembly methods include: The binocular vision system acquires images from the operating table and transmits them to the computer. The computer then separates the beam splitter from the image and obtains the poses of both ends of the beam splitter. Based on the positions of the two ends of the splitter obtained in the previous step, the two robotic arms move to one end of the splitter and drive the clamp jaws to complete the clamping of the splitter and insert the splitter into the fiber optic tray. The binocular vision system re-acquires the image of the operating table, separates the fiber outline in the image at this time, preliminarily confirms the gripping point on the fiber, and drives the robotic arm to move to that position. A miniature camera acquires images of the optical fiber and calculates the clamp attitude error in real time based on the optical fiber position image of the template. Based on the clamp posture error, the robotic arm adjusts and corrects in real time to make the image from the miniature camera close to the template image, thus achieving precise positioning of the optical fiber. The robotic arm drives the clamps to grip the optical fiber and inserts it into the optical fiber tray slot, thus completing the gripping and insertion of the optical fiber. The binocular vision system re-acquires the image of the operating table, separates the fiber outline in the image, preliminarily confirms the gripping point on the fiber, and drives the robotic arm to move to that position. Specifically: The Mask_RCNN neural network is used to segment the fiber outline from the image. The clamping points on the fiber are further confirmed by the matching point algorithm. Then, the spatial coordinates of the clamping points are determined by the principle of triangulation and transformed into the coordinate system of the robotic arm. The Mask_RCNN neural network has a two-stage network structure; A miniature camera acquires images of the optical fiber, and the clamp attitude error is calculated in real time based on the optical fiber position image of the template. Specifically: The parameters required for fiber pose are obtained by comparing the current image with a pre-captured template fiber position image. The direction and speed of the end-effector miniature camera are calculated based on the image Jacobian, and the clamp posture is corrected in real time to achieve precise positioning. The current image is compared with the pre-captured template image to obtain the situation where the optical fiber deviates from the preposition, and further, the imaging projection image of the miniature camera when the optical fiber deviates from the preposition is superimposed. Based on the imaging projection diagram, the fiber axis slope, fiber profile slope, fiber center length, fiber center distance from the target center, and fiber thickness at the target position are calculated. Based on the set target location and the physical dimensions of the optical fiber, determine the width of the optical fiber and the distance between the target location of the optical fiber and the miniature camera; determined fiber offset about the Z axis angle, fiber offset about the Y axis angle and fiber offsets about the Y and Z axes and the value of the fiber's pose error from the ideal position.

2. The method of assembling according to claim 1, wherein, The robotic arm comprises two six-degree-of-freedom articulated robotic arms.

3. The method of assembling according to claim 1, wherein, The miniature clamp includes a movable jaw and a fixed jaw, both of which are curved.

4. The method of assembling according to claim 1, wherein, A miniature camera is mounted on a slotted end of the fixed jaw.

5. The assembly method according to claim 1, characterized in that, Its features are, It also includes fiber coiling, which is planned offline.

Images