Double-arm robot smart assembly system and method for optical fiber-fiber distribution box

Abstract

The invention discloses a double-arm robot smart assembly system and method for an optical fiber-fiber distribution box, and the system comprises a miniature clamp which is used for clamping a target optical fiber and is embedded into a miniature camera; the tail end of the mechanical arm is connected with a miniature clamp; the binocular vision system is used for acquiring an image in a detection view field; the computer is used for controlling the mechanical arm, the camera, the miniature clamp and the binocular vision system to move, coarse positioning of the miniature clamp is completed through the binocular vision system, then the miniature camera is used for fine positioning of the clamping position of the miniature clamp, and automatic assembly from the optical fiber to the fiber distribution box is achieved. According to the method, the uncertainty of the position of the flexible body in the assembly process of the optical fiber flexible body is solved, the mode of combining online planning and offline planning is used for replacing traditional manual assembly, the working efficiency is improved, and productivity is liberated.

Description

Technical Field

[0001] The present invention relates to the field of mechanical automation, and in particular to a dual-arm robot dexterous assembly system and method for an optical fiber-splitting box. Background Art

[0002] With the rapid development of my country's social economy, society has put forward higher requirements for factory production efficiency, and the production method that relies on manual work has been gradually replaced by automated production methods. However, in the assembly process of optical fiber into splitter discs, manual assembly is still the mainstream production method, which greatly limits the automation level of the optical fiber industry chain and does not meet the high-efficiency and high-modularity modern manufacturing solutions.

[0003] At present, most factories use manual rather than mechanical automation to complete the assembly of fiber-splitters. This is mainly due to the current automated assembly solutions that cannot accurately identify small flexible bodies such as optical fibers and the lack of precision and flexibility in the operation of the clamps. Therefore, how to design an automated and flexible assembly solution for fiber-splitters has become a key link in the current optical fiber assembly industry chain.

[0004] Chinese patent application 202410465145.2 proposes an automatic optical fiber winding system, which is characterized by being able to release the optical fiber in the optical fiber reel and complete the winding. However, its structure is complex and its versatility is not high. It can only release the optical fiber from the optical fiber reel but cannot install the optical fiber, and is not suitable for the task of loading the optical fiber into the fiber distribution box.

[0005] In summary, there is no complete set of fiber-splitter smart assembly solutions and systems yet. Therefore, designing a fiber-splitter robot smart assembly system will help improve the full automation level of the optical fiber industry chain, save labor costs, and improve work efficiency. Summary of the invention

[0006] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, an object of the present invention is to provide a dual-arm robot dexterous assembly system and method for an optical fiber-split box.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] A dual-arm robot dexterous assembly system for an optical fiber-splitter box, comprising:

[0009] Micro clamp: used for clamping the target optical fiber, wherein the micro clamp is embedded in the micro camera;

[0010] Robotic arm: the end of the robotic arm is connected to a micro clamp;

[0011] Binocular vision system: used to obtain images within the detection field of view;

[0012] Computer: used to control the movement of the robotic arm, camera, micro clamp and binocular vision system. The binocular vision system is used to complete the coarse positioning of the micro clamp, and then the micro camera is used to accurately locate the clamping position of the micro clamp to realize the automated assembly of optical fiber to the fiber distribution box.

[0013] Furthermore, the robotic arm includes two six-degree-of-freedom joint robotic arms.

[0014] Furthermore, the micro clamp comprises a movable jaw and a fixed jaw, and both the movable jaw and the fixed jaw have an arc.

[0015] Furthermore, a miniature camera is arranged in a slot at the end of the fixed jaw.

[0016] An assembly method based on the dual-arm robot dexterous assembly system comprises the following steps:

[0017] The binocular vision system collects the image of the operating table and transmits it to the computer. The computer separates the beam splitter in the image and obtains the position and posture of both ends of the beam splitter.

[0018] According to the positions of the two ends of the beam splitter obtained in the previous step, the two robotic arms move to one end of the beam splitter and drive the clamp jaws to complete the clamping of the beam splitter and insert the beam splitter into the optical fiber disk;

[0019] The binocular vision system collects the operating table image again, separates the fiber outline in the image at this time, preliminarily confirms the clamping point on the fiber, and drives the robot arm to move to the position;

[0020] The micro camera collects the optical fiber image and obtains the clamp posture error in real time based on the template optical fiber position image;

[0021] According to the clamp posture error, the robot arm adjusts and corrects in real time, so that the micro camera image is close to the template image, and the optical fiber is precisely positioned;

[0022] The driving mechanical arm drives the clamp to complete the clamping of the optical fiber, and inserts the clamped light into the optical fiber disk slot to complete the clamping and insertion of the light.

[0023] Furthermore, the binocular vision system collects the operating table image again, separates the fiber outline in the image at this time, preliminarily confirms the clamping point on the fiber, and drives the robot arm to move to the position, specifically:

[0024] The Mask_RCNN neural network is used to segment the fiber outline from the image, and the clamping points on the fiber are further confirmed based on the matching point algorithm. The spatial coordinates of the clamping points are then determined using the triangulation principle and converted to the robotic arm coordinate system.

[0025] Furthermore, the Mask_RCNN neural network is a two-stage network structure.

[0026] Furthermore, the micro camera collects the optical fiber image, and obtains the clamp posture error in real time based on the template optical fiber position image, specifically:

[0027] The current image is compared with the pre-shot template fiber position image to obtain the parameters required for the fiber position;

[0028] The direction and speed at which the micro camera at the end needs to move are calculated based on the image Jacobian, and the clamp posture is corrected in real time to achieve precise positioning.

[0029] Furthermore, the current image is compared with a pre-photographed template image to obtain a situation where the optical fiber deviates from the pre-positioning, and further obtain an imaging projection diagram of the micro camera when the optical fiber deviates from the pre-positioning situation and is superimposed;

[0030] According to the imaging projection diagram, the fiber axis slope, fiber profile slope, fiber center length, distance from the fiber center to the target center, and fiber thickness at the target position are calculated;

[0031] According to the set target position and the physical size of the optical fiber, the width of the optical fiber and the distance between the optical fiber target position and the micro camera are determined;

[0032] Determine the optical fiber offset angle θ1 around the Z axis, the optical fiber offset angle θ2 around the Y axis, and the values ​​of the optical fiber offset ΔY and ΔZ around the Y axis and Z axis, and obtain the attitude error of the optical fiber from the ideal position.

[0033] Furthermore, it also includes a fiber coiling action, and the fiber coiling action is planned offline.

[0034] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0035] (1) The present invention adopts a binocular vision system with a Mask_RCNN convolutional neural network to realize the detection and segmentation of small optical fiber targets and determine the spatial position of their clamping points, thereby solving the uncertainty of the position of the flexible body during the assembly of the optical fiber flexible body. It replaces the traditional manual assembly with a combination of online planning and offline planning, thereby improving work efficiency and liberating productivity.

[0036] (2) The end of the clamp of the present invention is embedded with a micro camera for clamping small optical fibers. Since the clamp itself has a camera, it can be guided in a refined manner, and the camera with a large field of view (the large field of view refers to the binocular camera angle of view, which can include the fiber distribution box on the operating table and the field of view of all optical fibers) can be used for rough positioning, and then the small field of view (the small field of view refers to the micro camera angle of view, which only includes a section of the optical fiber and a very small background) can be used for fine positioning. This clamp improves the accuracy of clamping small objects and reduces the difficulty of clamping optical fibers.

[0037] (3) The jaws of the clamp in the present invention have a certain curvature and can be filled with flexible objects such as silicone. In this way, on the one hand, the clamping force of the clamp jaws can be indirectly controlled by controlling the moving distance of the driving motor. On the other hand, the filled flexible material can envelop a single or multiple optical fibers as much as possible, thereby achieving stable clamping of a single or multiple optical fibers. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a structural schematic diagram of a dual-arm robot dexterous assembly system of an optical fiber-fiber splitter box according to an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of a micro clamp according to an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of the structure of a fiber distribution box according to an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of the structure of a convolutional neural network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below in conjunction with the examples, but the embodiments of the present invention are not limited thereto.

[0043] Example

[0044] like Figure 1 As shown, a dual-arm robot dexterous assembly system for optical fiber-split box includes: two six-degree-of-freedom joint robot arms 101, 104, a binocular vision system 102, a retractable and rotatable camera bracket 103, micro clamps 105, 107, a split box 106 and a computer. The computer is respectively connected to the two six-degree-of-freedom joint robot arms, the binocular vision system and the two micro clamps. The six-degree-of-freedom joint robot arm is a Yaskawa six-degree-of-freedom joint robot arm.

[0045] Further explanation:

[0046] like Figure 2As shown, the micro clamp is used to clamp optical fibers and splitters and achieves precise clamping through visual guidance through a micro camera at the end.

[0047] The micro clamp is connected to the six-degree-of-freedom joint robot arm through a flange. The two micro clamps have the same structure. The micro clamp includes a movable jaw 301 and a fixed jaw 302. The movable jaw is the main part for realizing the clamping function. The jaw has a certain curvature and can be filled with silicone or other flexible materials. The curvature is a circular arc with a radius of 14 mm and a curvature angle of 35°.

[0048] The fixed jaw has a contact surface with a certain curvature, and cooperates with the movable jaw to realize the clamping function. The fixed jaw is provided with an opening slot 303 for embedding the miniature camera.

[0049] Part 304 is directly connected to the linear motor through a connector, becoming a driving part, and performs linear motion in a fixed z direction; part 305 connects part 304 and the movable jaw 301, and converts the linear motion into rotational motion.

[0050] The fiber splitter box is made of plastic, and the optical fiber needs to be inserted into and coiled in the fiber splitter box.

[0051] like Figure 3 As shown, the fiber splitter box has different types of limit slots, and the light needs to be clamped and placed under the slots at one time so that the optical fiber will not fall off the slots in a natural state to achieve the purpose of fiber coiling. The optical fiber splitter is installed at position 408 of the fiber splitter box. This position has a certain degree of elasticity, and its two ends will press the splitter tightly to achieve the purpose of clamping the splitter.

[0052] The binocular vision system 102 includes a binocular vision camera and a retractable and movable bracket 103. One end of the retractable and movable bracket is fixed to the wall, and the other end is fixed to the binocular camera and its connecting parts and placed directly above the operating table.

[0053] The binocular vision camera includes two industrial cameras with the same basic parameters, which are fixed to a bracket by bolts. The distance (baseline length) and field of view angle between the two industrial cameras can be adjusted to better observe the operating table space.

[0054] This example also includes an equipment installation platform: a platform for installing two robotic arms and an operating table for placing fiber splitter boxes and optical fibers.

[0055] An assembly method based on the above assembly system comprises:

[0056] The robot arm is first initialized to the zero state (the artificially specified initialization state) and waits for the next instruction from the computer;

[0057] The binocular vision system collects the image on the operating table and transmits it to the computer. The Mask-RCNN neural network segments the spectrometer in the image and calculates the position and posture of both ends of the spectrometer.

[0058] According to the positions of the two ends of the beam splitter obtained in the previous step, the two robotic arms move to one end of the beam splitter and drive the clamp jaws to complete the clamping of the beam splitter;

[0059] After the clamping is completed, the two robotic arms maintain the holding state and move collaboratively to the top of the fiber tray structure, and then install the optical splitter into the structure;

[0060] The binocular camera captures the image on the operating table again and separates the fiber outline in the image. Based on the corresponding point matching algorithm, the clamping point on the fiber is preliminarily confirmed and the robot arm is driven to move to that position.

[0061] When the clamp moves to the above position, the micro camera at the end of the clamp will observe a section of optical fiber, collect the image at this time, and calculate the clamp posture error in real time based on the template optical fiber position image;

[0062] After the clamp obtains the posture error, the robot arm adjusts and corrects the error in real time, so that the micro camera image is as close to the template image as possible, thus achieving precise positioning of the optical fiber;

[0063] When the clamp posture is adjusted to meet the expectations, the mechanical arm and the clamp are driven to complete the clamping of the optical fiber, and the mechanical arm is driven to insert the clamped optical fiber into the optical fiber reel slot (structures 401-407 are all optical fiber reel slots), completing the clamping and insertion of the optical fiber.

[0064] The above steps are repeated several times until all the optical fibers are clamped and inserted into the card slots, and the optical fibers are coiled on the optical disk.

[0065] To further explain, the binocular vision system collects the image on the operating table and transmits it to the computer. The Mask-RCNN neural network segments the spectrometer in the image and calculates the poses of both ends of the spectrometer. Specifically:

[0066] The image is passed to the trained convolutional neural network, which will segment the spectrometers on the two images. Then, based on the triangulation principle of binocular vision, the position coordinates of the two ends of the spectrometer are extracted and converted into the robot coordinate system, thereby driving the robotic arm to move to the specified position.

[0067] To further explain, the binocular vision system collects the image of the operating table and transmits it to the computer, segments the fiber outline, determines the clamping point, realizes the rough positioning of the fiber, and drives the robotic arm to move to the rough positioning coordinates, specifically:

[0068] After the robot completes the insertion of the beam splitter, it needs to clamp and coil the optical fiber. When clamping the optical fiber, it needs to use a binocular camera to capture images, segment the optical fiber outline from the background after passing through a convolutional neural network, further confirm the points on the optical fiber outline that need to be clamped based on the matching point algorithm, and then use the triangulation principle to determine the spatial coordinates of the two points, convert them to the robot coordinate system, and drive the arms to move near the position.

[0069] Traditional visual methods generally use edge detection, contour fitting, template matching and other methods to locate the detected object. These methods require the detected object to have obvious texture features or a large grayscale gradient with the background. However, combined with the specific task objectives of this patent, the optical fiber and the fiber splitter box are similar in color, and the optical fiber itself lacks texture features, so it cannot be segmented using traditional visual methods.

[0070] The above two-step neural network both use Mask_RCNN neural network to perform instance segmentation of optical fiber. To train this model, you need to first sample a large number of pictures containing optical fibers in a production environment, and then manually mark the optical fibers in the images. In this process, you must carefully mark the outline of the optical fiber clearly. At the same time, since the optical fiber appears to be slender in the image, it is necessary to appropriately increase the size of the convolution kernel so that the network can identify the optical fiber more from the overall or large area, and avoid treating the small linear structures in the background as optical fibers.

[0071] like Figure 4 As shown, specifically, the Mask-RCNN neural network is a two-stage network structure, including:

[0072] The first stage: Region Proposal Network (RPN), which proposes candidate object bounding boxes in one stage. Each proposal in this stage will go through the second stage.

[0073] The second stage: For each proposal, the feature area of ​​the image is extracted through the convolution layer and the ROI area is pooled and passed through the remaining network. The remaining network is used to implement the discriminator and generate bounding boxes. At the same time, the network adds two convolution layers after ROI pooling to extract the target mask.

[0074] Properly increasing the size of the convolution kernel has the following effects when segmenting the fiber contour:

[0075] According to the specific task requirements, increasing the convolution kernel size can increase the receptive field of the image, that is, the optical fiber can be observed in a larger range, which is conducive to identifying the optical fiber as a whole and segmenting its mask.

[0076] Avoid the possibility of misidentifying thin linear objects in the background as optical fibers when using small convolution kernels. Because small convolution kernels can easily extract detailed texture features of objects, large convolution kernels can cover larger structural features. Optical fibers themselves do not have special textures. On the contrary, they need to be identified in a larger scene to have a higher accuracy. In this task, the size of the prior box and convolution layer of Mask-RCNN was adjusted to 64, 128, 256, 512, and 1024.

[0077] In this experiment, the Mask-RCNN network prior frame and convolution layer size were adjusted as follows. Under the same sample and epoch, this convolution kernel size combination had the best effect.

[0078] Further explanation: The micro camera collects the current image and provides real-time feedback of the micro clamp's posture error to complete precise positioning, specifically:

[0079] The current image is compared with the pre-shot template image to obtain the optical fiber deviation from the pre-positioning. The optical fiber deviation from the pre-positioning includes the following: first, the optical fiber is offset by an angle of θ1 around the Z axis; second, the optical fiber is offset by ΔY and ΔZ around the Y axis and the Z axis; third, the optical fiber is offset by an angle of θ2 around the Y axis; and fourth, the optical fiber is located at an ideal position. The above four situations will have different characteristics in the micro camera. In actual situations, the three error postures will not appear separately, but will be superimposed. Further, the imaging projection diagram of the micro camera when the optical fiber deviation from the pre-positioning is superimposed is obtained;

[0080] According to the imaging projection diagram, the fiber axis slope K1, fiber profile slope K2, fiber center length L, and distance ΔL from the fiber center to the target center are calculated. m and the thickness L0 of the optical fiber at the target position;

[0081] According to the set target position and the physical size of the optical fiber, the width W of the optical fiber and the distance Z0 from the optical fiber target position to the micro camera are determined;

[0082] Determine the optical fiber offset angle θ1 around the Z axis, the optical fiber offset angle θ2 around the Y axis, and the values ​​of the optical fiber offset ΔY and ΔZ around the Y axis and Z axis, and obtain the attitude error of the optical fiber from the ideal position.

[0083]

[0084] θ1=arctan(k1)

[0085]

[0086] The four variables obtained according to the above formula can be used to calculate the posture error of the optical fiber from the ideal position, so as to drive the robot arm to move so that the posture error is compensated, thereby making the optical fiber close to the ideal posture to facilitate the subsequent clamping operation.

[0087] Further explanation: Since the fiber splitter box is fixed, the position of its limit slot can be obtained in advance, and the fiber winding action after clamping the optical fiber can be planned offline, that is, the dual-arm robot can hold a section of optical fiber through the limit slot and then release one end of the optical fiber to complete the installation. In this process, the movement trajectory of the robot arm needs to be planned to avoid collisions between the dual-arm robots.

[0088] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A dual-arm robot dexterous assembly system for optical fiber-split box, characterized in that: include: Micro clamp: used for clamping the target optical fiber, wherein the micro clamp is embedded in the micro camera; Robotic arm: the end of the robotic arm is connected to a micro clamp; Binocular vision system: used to obtain images within the detection field of view; Computer: used to control the movement of the robotic arm, camera, micro clamp and binocular vision system. The binocular vision system is used to complete the coarse positioning of the micro clamp, and then the micro camera is used to accurately locate the clamping position of the micro clamp to realize the automated assembly of optical fiber to the fiber distribution box.

2. The dual-arm robot dexterous assembly system according to claim 1, characterized in that: The robotic arm comprises two six-degree-of-freedom joint robotic arms.

3. The dual-arm robot dexterous assembly system according to claim 2, characterized in that: The micro clamp comprises a movable jaw and a fixed jaw, and both the movable jaw and the fixed jaw have an arc.

4. The dual-arm robot dexterous assembly system according to claim 3, characterized in that: A miniature camera is arranged in a slot at the end of the fixed jaw.

5. An assembly method based on the dual-arm robot dexterous assembly system according to any one of claims 1 to 4, characterized in that: The steps include: The binocular vision system collects the image of the operating table and transmits it to the computer. The computer separates the beam splitter in the image and obtains the position and posture of both ends of the beam splitter. According to the positions of the two ends of the beam splitter obtained in the previous step, the two robotic arms move to one end of the beam splitter and drive the clamp jaws to complete the clamping of the beam splitter and insert the beam splitter into the optical fiber disk; The binocular vision system collects the operating table image again, separates the fiber outline in the image at this time, preliminarily confirms the clamping point on the fiber, and drives the robot arm to move to the position; The micro camera collects the optical fiber image and obtains the clamp posture error in real time based on the template optical fiber position image; According to the clamp posture error, the robot arm adjusts and corrects in real time, so that the micro camera image is close to the template image, and the optical fiber is precisely positioned; The driving mechanical arm drives the clamp to complete the clamping of the optical fiber, and inserts the clamped light into the optical fiber disk slot to complete the clamping and insertion of the light.

6. The assembly method according to claim 5, characterized in that: The binocular vision system collects the operating table image again, separates the fiber outline in the image at this time, preliminarily confirms the clamping point on the fiber, and drives the robot arm to move to the position, specifically: The Mask_RCNN neural network is used to segment the fiber outline from the image, and the clamping points on the fiber are further confirmed based on the matching point algorithm. The spatial coordinates of the clamping points are then determined using the triangulation principle and converted to the robotic arm coordinate system.

7. The assembly method according to claim 6, characterized in that: The Mask_RCNN neural network is a two-stage network structure.

8. The assembly method according to claim 5, characterized in that: The micro camera collects the optical fiber image and obtains the clamp posture error in real time based on the template optical fiber position image, specifically: The current image is compared with the pre-shot template fiber position image to obtain the parameters required for the fiber position; The direction and speed at which the micro camera at the end needs to move are calculated based on the image Jacobian, and the clamp posture is corrected in real time to achieve precise positioning.

9. The assembly method according to claim 8, characterized in that: The current image is compared with a pre-photographed template image to obtain a situation where the optical fiber deviates from the pre-position, and further obtain an imaging projection diagram of the micro camera when the optical fiber deviates from the pre-position in a superimposed state; According to the imaging projection diagram, the fiber axis slope, fiber profile slope, fiber center length, distance from the fiber center to the target center, and fiber thickness at the target position are calculated; According to the set target position and the physical size of the optical fiber, the width of the optical fiber and the distance between the optical fiber target position and the micro camera are determined; Determine the optical fiber offset angle θ1 around the Z axis, the optical fiber offset angle θ2 around the Y axis, and the values ​​of the optical fiber offset ΔY and ΔZ around the Y axis and Z axis, and obtain the attitude error of the optical fiber from the ideal position.

10. The assembly method according to any one of claims 5 to 9, characterized in that: It also includes a fiber coiling action, which is planned offline.

Images